From 8c85535db32783ad34b5d1f18d454227da991252 Mon Sep 17 00:00:00 2001 From: LiangSong Date: Fri, 28 Apr 2023 19:49:51 +0800 Subject: [PATCH] update readme --- README.md | 7 ++++++- README_en.md | 8 +++++++- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c91dec5..c549624 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ * @Author: LiangSong(sl12160010@gmail.com) * @Date: 2023-03-10 21:18:35 * @LastEditors: LiangSong(sl12160010@gmail.com) - * @LastEditTime: 2023-04-16 23:49:06 + * @LastEditTime: 2023-04-28 19:49:29 * @FilePath: /Open-Llama/README.md * @Description: * @@ -37,6 +37,7 @@ pip install git+https://github.com/s-JoL/transformers.git@dev ![image4](assets/multiturn_chat.jpeg) ## **更新** +[2023.4.28] Release v2.0 本次更新主要包含以下几个方面,相对于v1版本提升有效训练速度50%,其中pad从30%减少至5%,训练速度从3200token/s提升至3600token/s。0.95 * 3600/(0.7 * 3200)=1.527 1. 使用HuggingFace的datasets库进行数据读取,具体流程如下 @@ -49,6 +50,10 @@ pip install git+https://github.com/s-JoL/transformers.git@dev 4. 提供更方便的配置,可见configs/pretrain_config.yaml 5. 提供基于其他预训练模型补充词表,继续预训练功能 +[2023.4.16] Release v1.0 + +提供基础的预训练和指令微调代码,训练速度达到Llama原文速度。预训练和指令微调后的模型已经开源在HuggingFace。 + v1版代码可见https://github.com/s-JoL/Open-Llama/tree/v1.0 ## **特性** diff --git a/README_en.md b/README_en.md index dfa2914..10da777 100644 --- a/README_en.md +++ b/README_en.md @@ -2,7 +2,7 @@ * @Author: LiangSong(sl12160010@gmail.com) * @Date: 2023-03-10 21:18:35 * @LastEditors: LiangSong(sl12160010@gmail.com) - * @LastEditTime: 2023-04-16 23:49:28 + * @LastEditTime: 2023-04-28 19:49:24 * @FilePath: /Open-Llama/README_en.md * @Description: * @@ -38,6 +38,8 @@ Below is a display of the model's multi-turn dialogue ability regarding code: ## **Updates** +[2023.4.28] Release v2.0 + This update mainly includes the following aspects, increasing the effective training speed by 50% compared to the v1 version, reducing padding from 30% to 5%, and improving training speed from 3200 tokens/s to 3600 tokens/s. 0.95 * 3600 / (0.7 * 3200) = 1.527 1. Use HuggingFace's datasets library for data reading, with the process as follows: @@ -50,6 +52,10 @@ This update mainly includes the following aspects, increasing the effective trai 4. Provide more convenient configuration, see configs/pretrain_config.yaml 5. Provide functionality to continue pre-training based on other pre-trained models and supplementing vocabulary +[2023.4.16] Release v1.0 + +Basic pre-training and instruction fine-tuning codes are provided, with a training speed comparable to that of the original Llama. The pre-trained and fine-tuned models are already open-sourced on HuggingFace. + v1 version code can be seen at https://github.com/s-JoL/Open-Llama/tree/v1.0 ## **Features**