LiangSong bc16df4751 add more instruction data		2023-04-06 03:45:24 +08:00
..
collate_fn.py	update dataset, add concat sequence from multiple docs	2023-04-05 22:42:34 +08:00
data_iter.py	add more instruction data	2023-04-06 03:45:24 +08:00
instruction_dataset.py	add more instruction data	2023-04-06 03:45:24 +08:00
pretrain_dataset.py	update dataset, add concat sequence from multiple docs	2023-04-05 22:42:34 +08:00
tokenizer.py	update dataset, add concat sequence from multiple docs	2023-04-05 22:42:34 +08:00
train_tokenizer.py	update dataset, add concat sequence from multiple docs	2023-04-05 22:42:34 +08:00
validation.py	reformat code with black	2023-03-27 14:34:59 +08:00