diff --git a/README.md b/README.md index af6aae6..6b15e3d 100644 --- a/README.md +++ b/README.md @@ -2,27 +2,37 @@ * @Author: LiangSong(sl12160010@gmail.com) * @Date: 2023-03-10 21:18:35 * @LastEditors: LiangSong(sl12160010@gmail.com) - * @LastEditTime: 2023-04-29 11:41:10 + * @LastEditTime: 2023-04-29 12:06:24 * @FilePath: /Open-Llama/README.md * @Description: * * Copyright (c) 2023 by LiangSong(sl12160010@gmail.com), All Rights Reserved. --> +[**中文**](./README.md) | [**English**](./README_en.md) + # Open-Llama -[English README](https://github.com/Bayes-Song/Open-Llama/blob/main/README_en.md) +

+ GitHub + GitHub release (latest by date) + GitHub top language + GitHub last commit +

Open-Llama是一个开源项目,提供了一整套用于构建大型语言模型的训练流程,从数据集准备到分词、预训练、指令调优,以及强化学习技术 RLHF。 **可从[Demo](http://home.ustc.edu.cn/~sl9292/)直接试用本模型。** -## **效果** +## **主要内容** -**采用FastChat项目相同方法测评Open-Llama的效果和GPT3.5的效果对比,经过测试在中文问题上可以达到GPT3.5 84%的水平。** +- **支持Transformers/HuggingFace直接调用。** 经过Instruct-tuning的CheckPoint已开源在[HuggingFace: s-JoL/Open-Llama-V1](https://huggingface.co/s-JoL/Open-Llama-V1)。 -**训练速度达到3620 token/s,快于Llama原文中的3370 token/s,达到目前sota的水平。** +- **采用FastChat项目相同方法测评Open-Llama的效果和GPT3.5的效果对比,经过测试在中文问题上可以达到GPT3.5 84%的水平。** -经过Instruct-tuning的CheckPoint已开源在[HuggingFace: s-JoL/Open-Llama-V1](https://huggingface.co/s-JoL/Open-Llama-V1)。使用ckpt需要先用下面命令安装最新版本Transformers +- **训练速度达到3620 token/s,快于Llama原文中的3370 token/s,达到目前sota的水平。** + + +使用ckpt需要先用下面命令安装最新版本Transformers: ``` python pip install git+https://github.com/huggingface/transformers.git diff --git a/README_en.md b/README_en.md index e0f6960..bf32b41 100644 --- a/README_en.md +++ b/README_en.md @@ -2,27 +2,36 @@ * @Author: LiangSong(sl12160010@gmail.com) * @Date: 2023-03-10 21:18:35 * @LastEditors: LiangSong(sl12160010@gmail.com) - * @LastEditTime: 2023-04-29 11:41:20 + * @LastEditTime: 2023-04-29 12:06:05 * @FilePath: /Open-Llama/README_en.md * @Description: * * Copyright (c) 2023 by LiangSong(sl12160010@gmail.com), All Rights Reserved. --> +[**中文**](./README.md) | [**English**](./README_en.md) + # Open-Llama -[English README](https://github.com/Bayes-Song/Open-Llama/blob/main/README_en.md) +

+ GitHub + GitHub release (latest by date) + GitHub top language + GitHub last commit +

Open-Llama is an open-source project that offers a complete training pipeline for building large language models, ranging from dataset preparation to tokenization, pre-training, prompt tuning, and the reinforcement learning technique RLHF. **You can try this model directly from the [Demo](http://home.ustc.edu.cn/~sl9292/).** -## **Performance** +## **Main contents** -**By adopting the same evaluation method as the FastChat project, Open-Llama's performance is compared to GPT3.5’s. After testing, it can reach 84% of GPT3.5's performance on Chinese questions.** +- **Support Transformers/HuggingFace.** The CheckPoint after Instruct-tuning is open-source on [HuggingFace: s-JoL/Open-Llama-V1](https://huggingface.co/s-JoL/Open-Llama-V1). -**The training speed reaches 3620 tokens/s, faster than the 3370 tokens/s reported in the original Llama paper, reaching the current state-of-the-art level.** +- **By adopting the same evaluation method as the FastChat project, Open-Llama's performance is compared to GPT3.5’s. After testing, it can reach 84% of GPT3.5's performance on Chinese questions.** -The CheckPoint after Instruct-tuning is open-source on [HuggingFace: s-JoL/Open-Llama-V1](https://huggingface.co/s-JoL/Open-Llama-V1). To use the CheckPoint, first, install the latest version of Transformers with the following command: +- **The training speed reaches 3620 tokens/s, faster than the 3370 tokens/s reported in the original Llama paper, reaching the current state-of-the-art level.** + + To use the CheckPoint, first, install the latest version of Transformers with the following command: ``` python pip install git+https://github.com/huggingface/transformers.git