gnh1201/Open-Llama
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update readme

Browse Source
This commit is contained in:
LiangSong 2023-05-09 18:47:29 +08:00
parent 7d505ea303
commit 72a6f81b61
2 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -233,7 +233,7 @@ Finally, we referenced [PALM](https://arxiv.org/abs/2204.02311) and employed Sha
We use multi-GPU parallel training based on the Accelerate library, with the following start command: We use multi-GPU parallel training based on the Accelerate library, with the following start command:
```bash ```bash
accelerate launch --config_file configs/accelerate_configs/ds_stage1.yaml train_lm.py --config configs/pretrain_config.yaml accelerate launch --config_file configs/accelerate_configs/ds_stage1.yaml train_lm.py --train_config configs/pretrain_config.yaml --model_config configs/model_configs/7B.json
``` ```
In some cases, you may need to specify the following parameters: In some cases, you may need to specify the following parameters:
@ -301,7 +301,7 @@ user: {prompt}\nsystem: {completion}</s>
The startup command is basically the same as pre-training: The startup command is basically the same as pre-training:
```bash ```bash
accelerate launch --config_file configs/accelerate_configs/ds_stage1.yaml train_lm.py --config configs/instruct_config.yaml accelerate launch --config_file configs/accelerate_configs/ds_stage1.yaml train_lm.py --train_config configs/instruct_config.yaml --model_config configs/model_configs/7B.json
``` ```
In some cases, you may need to specify the following parameters: In some cases, you may need to specify the following parameters:

4
README_zh.md
View File

@ -223,7 +223,7 @@ Self Attention的计算，这对于性能有明显的提升，提升大约30%。
### 预训练 ### 预训练
我们基于Accelerate库进行多GPU并行训练，启动命令如下 我们基于Accelerate库进行多GPU并行训练，启动命令如下
```bash ```bash
accelerate launch --config_file configs/accelerate_configs/ds_stage1.yaml train_lm.py --config configs/pretrain_config.yaml accelerate launch --config_file configs/accelerate_configs/ds_stage1.yaml train_lm.py --train_config configs/pretrain_config.yaml --model_config configs/model_configs/7B.json
``` ```
某些情况下可能需要指定下列参数 某些情况下可能需要指定下列参数
``` ```
@ -285,7 +285,7 @@ user: {prompt}\nsystem: {completion}</s>
启动命令和预训练基本一致 启动命令和预训练基本一致
```bash ```bash
accelerate launch --config_file configs/accelerate_configs/ds_stage1.yaml train_lm.py --config configs/instruct_config.yaml accelerate launch --config_file configs/accelerate_configs/ds_stage1.yaml train_lm.py --train_config configs/instruct_config.yaml --model_config configs/model_configs/7B.json
``` ```
某些情况下可能需要指定下列参数 某些情况下可能需要指定下列参数
``` ```