LiangSong 32583a41a7 update wudao download and preprocess		2023-05-09 14:47:59 +08:00
..
download_instruct.sh	update ShareGPT_90K preprocess	2023-05-04 08:34:38 +08:00
download_the_pile.sh	add high-performance Llama pre-train code	2023-03-26 23:59:53 +08:00
download_wudao.sh	update wudao download and preprocess	2023-05-09 14:47:59 +08:00
preprocess_instruction.py	update science instruct-tuning datasets	2023-05-05 19:00:37 +08:00
preprocess_the_pile.py	using huggingface datasets to accelerate training, using open-llama to pretrain	2023-04-24 19:13:53 +08:00
preprocess_wudao.py	using huggingface datasets to accelerate training, using open-llama to pretrain	2023-04-24 19:13:53 +08:00