LiangSong f8f4cde228 using huggingface datasets to accelerate training, using open-llama to pretrain		2023-04-24 19:13:53 +08:00
..
download_instruct.sh	update preprocess_instruction, add math/code/multiturn_chat and etc.	2023-04-05 23:51:56 +08:00
download_the_pile.sh	add high-performance Llama pre-train code	2023-03-26 23:59:53 +08:00
download_wudao.sh	add high-performance Llama pre-train code	2023-03-26 23:59:53 +08:00
preprocess_instruction.py	using huggingface datasets to accelerate training, using open-llama to pretrain	2023-04-24 19:13:53 +08:00
preprocess_the_pile.py	using huggingface datasets to accelerate training, using open-llama to pretrain	2023-04-24 19:13:53 +08:00
preprocess_wudao.py	using huggingface datasets to accelerate training, using open-llama to pretrain	2023-04-24 19:13:53 +08:00