gnh1201/Open-Llama
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update readme

Browse Source
This commit is contained in:
LiangSong 2023-04-29 12:06:55 +08:00
parent 4c5e50e4aa
commit a2816bd23d
2 changed files with 31 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
README.md
View File

@ -2,27 +2,37 @@
* @Author: LiangSong(sl12160010@gmail.com)
* @Date: 2023-03-10 21:18:35
* @LastEditors: LiangSong(sl12160010@gmail.com)
* @LastEditTime: 2023-04-29 11:41:10
* @LastEditTime: 2023-04-29 12:06:24
* @FilePath: /Open-Llama/README.md
* @Description:
*
* Copyright (c) 2023 by LiangSong(sl12160010@gmail.com), All Rights Reserved.
-->
[**中文**](./README.md) | [**English**](./README_en.md)
# Open-Llama
[English README](https://github.com/Bayes-Song/Open-Llama/blob/main/README_en.md)
<p align="center">
<img alt="GitHub" src="https://img.shields.io/github/license/s-JoL/Open-Llama.svg?color=blue&style=flat-square">
<img alt="GitHub release (latest by date)" src="https://img.shields.io/github/v/release/s-JoL/Open-Llama">
<img alt="GitHub top language" src="https://img.shields.io/github/languages/top/s-JoL/Open-Llama">
<img alt="GitHub last commit" src="https://img.shields.io/github/last-commit/s-JoL/Open-Llama">
</p>
Open-Llama是一个开源项目，提供了一整套用于构建大型语言模型的训练流程，从数据集准备到分词、预训练、指令调优，以及强化学习技术 RLHF。
**可从[Demo](http://home.ustc.edu.cn/~sl9292/)直接试用本模型。**
## **效果**
## **主要内容**
**采用FastChat项目相同方法测评Open-Llama的效果和GPT3.5的效果对比，经过测试在中文问题上可以达到GPT3.5 84%的水平。**
- **支持Transformers/HuggingFace直接调用。** 经过Instruct-tuning的CheckPoint已开源在[HuggingFace: s-JoL/Open-Llama-V1](https://huggingface.co/s-JoL/Open-Llama-V1)。
**训练速度达到3620 token/s，快于Llama原文中的3370 token/s，达到目前sota的水平。**
- **采用FastChat项目相同方法测评Open-Llama的效果和GPT3.5的效果对比，经过测试在中文问题上可以达到GPT3.5 84%的水平。**
经过Instruct-tuning的CheckPoint已开源在[HuggingFace: s-JoL/Open-Llama-V1](https://huggingface.co/s-JoL/Open-Llama-V1)。使用ckpt需要先用下面命令安装最新版本Transformers
- **训练速度达到3620 token/s，快于Llama原文中的3370 token/s，达到目前sota的水平。**
使用ckpt需要先用下面命令安装最新版本Transformers：
``` python
pip install git+https://github.com/huggingface/transformers.git

21
README_en.md
View File

@ -2,27 +2,36 @@
* @Author: LiangSong(sl12160010@gmail.com)
* @Date: 2023-03-10 21:18:35
* @LastEditors: LiangSong(sl12160010@gmail.com)
* @LastEditTime: 2023-04-29 11:41:20
* @LastEditTime: 2023-04-29 12:06:05
* @FilePath: /Open-Llama/README_en.md
* @Description:
*
* Copyright (c) 2023 by LiangSong(sl12160010@gmail.com), All Rights Reserved.
-->
[**中文**](./README.md) | [**English**](./README_en.md)
# Open-Llama
[English README](https://github.com/Bayes-Song/Open-Llama/blob/main/README_en.md)
<p align="center">
<img alt="GitHub" src="https://img.shields.io/github/license/s-JoL/Open-Llama.svg?color=blue&style=flat-square">
<img alt="GitHub release (latest by date)" src="https://img.shields.io/github/v/release/s-JoL/Open-Llama">
<img alt="GitHub top language" src="https://img.shields.io/github/languages/top/s-JoL/Open-Llama">
<img alt="GitHub last commit" src="https://img.shields.io/github/last-commit/s-JoL/Open-Llama">
</p>
Open-Llama is an open-source project that offers a complete training pipeline for building large language models, ranging from dataset preparation to tokenization, pre-training, prompt tuning, and the reinforcement learning technique RLHF.
**You can try this model directly from the [Demo](http://home.ustc.edu.cn/~sl9292/).**
## **Performance**
## **Main contents**
**By adopting the same evaluation method as the FastChat project, Open-Llama's performance is compared to GPT3.5’s. After testing, it can reach 84% of GPT3.5's performance on Chinese questions.**
- **Support Transformers/HuggingFace.** The CheckPoint after Instruct-tuning is open-source on [HuggingFace: s-JoL/Open-Llama-V1](https://huggingface.co/s-JoL/Open-Llama-V1).
**The training speed reaches 3620 tokens/s, faster than the 3370 tokens/s reported in the original Llama paper, reaching the current state-of-the-art level.**
- **By adopting the same evaluation method as the FastChat project, Open-Llama's performance is compared to GPT3.5’s. After testing, it can reach 84% of GPT3.5's performance on Chinese questions.**
The CheckPoint after Instruct-tuning is open-source on [HuggingFace: s-JoL/Open-Llama-V1](https://huggingface.co/s-JoL/Open-Llama-V1). To use the CheckPoint, first, install the latest version of Transformers with the following command:
- **The training speed reaches 3620 tokens/s, faster than the 3370 tokens/s reported in the original Llama paper, reaching the current state-of-the-art level.**
To use the CheckPoint, first, install the latest version of Transformers with the following command:
``` python
pip install git+https://github.com/huggingface/transformers.git