LiangSong
|
bf2cac0a45
|
update config
|
2023-05-14 01:00:50 +08:00 |
|
LiangSong
|
f6ac834ef9
|
update default config
|
2023-05-09 15:16:50 +08:00 |
|
LiangSong
|
4a1e7bb44b
|
Optimized the structure of configs, added support for deepspeed stage3, reduced memory usage by using Auto class to load models, and added support for training 65B models.
|
2023-05-06 23:37:17 +08:00 |
|
LiangSong
|
85caa97a6a
|
add xP3 dataset and belle_2M
|
2023-05-05 17:05:41 +08:00 |
|
LiangSong
|
0fdca8b949
|
update readme
|
2023-04-28 15:01:01 +08:00 |
|
LiangSong
|
49118aad42
|
update header config and add padding to concat_multiple_sequence
|
2023-04-27 23:42:11 +08:00 |
|
LiangSong
|
db6cdb51d0
|
unified pre-training and instrcution-tuning both use train_lm and dataset
|
2023-04-27 19:42:06 +08:00 |
|
LiangSong
|
f8f4cde228
|
using huggingface datasets to accelerate training, using open-llama to pretrain
|
2023-04-24 19:13:53 +08:00 |
|
LiangSong
|
a4aa109dd3
|
add trainer and utils
|
2023-04-12 17:59:05 +08:00 |
|