LiangSong
|
95973b5de1
|
update header
|
2023-05-17 22:21:46 +07:00 |
|
LiangSong
|
ceb1fd067b
|
update vocab_size
|
2023-05-11 14:15:12 +08:00 |
|
LiangSong
|
73dafa7ad6
|
add rounding vocab_size
|
2023-05-10 17:49:52 +08:00 |
|
LiangSong
|
59b79af9d7
|
add comment
|
2023-05-09 16:53:05 +08:00 |
|
LiangSong
|
32583a41a7
|
update wudao download and preprocess
|
2023-05-09 14:47:59 +08:00 |
|
LiangSong
|
6814fdb59e
|
support gradient ckpt for peft
|
2023-05-08 23:40:03 +08:00 |
|
LiangSong
|
92caa94490
|
support peft
|
2023-05-08 22:26:39 +08:00 |
|
LiangSong
|
4a1e7bb44b
|
Optimized the structure of configs, added support for deepspeed stage3, reduced memory usage by using Auto class to load models, and added support for training 65B models.
|
2023-05-06 23:37:17 +08:00 |
|
LiangSong
|
51686b5fb8
|
add split dataset by shard option to accelerate data loading
|
2023-05-04 09:20:23 +08:00 |
|
LiangSong
|
f05e929aad
|
update config
|
2023-05-02 21:42:55 +08:00 |
|
LiangSong
|
0466673f76
|
support load model from accelerate ckpt
|
2023-04-29 20:40:42 +08:00 |
|
LiangSong
|
fc21a75d1e
|
add continue training
|
2023-04-29 20:28:39 +08:00 |
|
LiangSong
|
0fdca8b949
|
update readme
|
2023-04-28 15:01:01 +08:00 |
|
LiangSong
|
49118aad42
|
update header config and add padding to concat_multiple_sequence
|
2023-04-27 23:42:11 +08:00 |
|
LiangSong
|
db6cdb51d0
|
unified pre-training and instrcution-tuning both use train_lm and dataset
|
2023-04-27 19:42:06 +08:00 |
|