diff --git a/README.md b/README.md index b522fc8..9ec294e 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ * @Author: LiangSong(sl12160010@gmail.com) * @Date: 2023-03-10 21:18:35 * @LastEditors: LiangSong(sl12160010@gmail.com) - * @LastEditTime: 2023-05-14 10:52:36 + * @LastEditTime: 2023-05-15 00:21:01 * @FilePath: /Open-Llama/README.md * @Description: * @@ -57,13 +57,13 @@ Using a total of 7 parts of data to constitute the Instruction-tuning data, the Below is a display of the model's multi-turn dialogue ability regarding code: -![image4](assets/multiturn_chat_en.jpeg) +![image4](assets/multiturn_chat_en.jpg) ## **Updates** **[2023.5.8] Release v2.1** -- This update adds support for larger model training. Using DeepSpeed stage3 + offload + activation checkpoint, you can **train a 65B model on a single machine with 8 A100-80G**. +- This update adds support for larger model training. Using DeepSpeed stage3 + offload + activation checkpoint, you can **train a 65B model with A100-80G**. - The peft library is introduced to **support training such as lora**. diff --git a/README_zh.md b/README_zh.md index 4a65bf2..3a7baed 100644 --- a/README_zh.md +++ b/README_zh.md @@ -2,7 +2,7 @@ * @Author: LiangSong(sl12160010@gmail.com) * @Date: 2023-03-10 21:18:35 * @LastEditors: LiangSong(sl12160010@gmail.com) - * @LastEditTime: 2023-05-14 10:52:08 + * @LastEditTime: 2023-05-15 00:02:05 * @FilePath: /Open-Llama/README_zh.md * @Description: * @@ -64,7 +64,7 @@ print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True)) **[2023.5.8] Release v2.1** -- 本次更新加入对更大模型训练的支持,使用DeepSpeed stage3 + offload + activation checkpoint可以在**单机8卡A100-80G训练65B模型**。 +- 本次更新加入对更大模型训练的支持,使用DeepSpeed stage3 + offload + activation checkpoint可以在**A100-80G训练65B模型**。 - 引入peft库**支持lora**等训练。 diff --git a/assets/multiturn_chat_en.jpg b/assets/multiturn_chat_en.jpg new file mode 100644 index 0000000..6778626 Binary files /dev/null and b/assets/multiturn_chat_en.jpg differ diff --git a/requirements.txt b/requirements.txt index c463902..dc4766d 100644 --- a/requirements.txt +++ b/requirements.txt @@ -18,4 +18,4 @@ functorch==1.13.1 xformers==0.0.16 gradio peft -git+https://github.com/huggingface/transformers.git \ No newline at end of file +transformers \ No newline at end of file