 Howard Liberty
|
cc356b6017
Add FSDP CPU offloading option
|
1 year ago |
 Yuanhao
|
e554c1c8bf
The tokenizer will not add eos_token by default
|
1 year ago |
 tim-a-davis
|
3038020aa4
Replaced ClassVar config param with field
|
1 year ago |
 Hamid Shojanazeri
|
cfacee4302
Update LLM_finetuning.md
|
1 year ago |
|
Hamid Shojanazeri
|
62dd2b3f4b
Update docs/LLM_finetuning.md
|
1 year ago |
 varunfb
|
6f2201c655
Updated spell checker to resolve the issues in LLM_finetuning.md
|
1 year ago |
|
varunfb
|
2c1ecba170
Update LLM_finetuning.md
|
1 year ago |
 Geeta Chauhan
|
279f4d4a0b
Use OpenAssistent/oasst1 dataset for custom dataset example (#180)
|
1 year ago |
 Kautilya Kondragunta
|
b4c71bebee
Update chat_completion.py
|
1 year ago |
 Matthias Reso
|
eccf7cb3dc
Check additional example in resulting dataset
|
1 year ago |
|
Matthias Reso
|
c33ea3cacb
Fix pbar update
|
1 year ago |
 hongbo.mo
|
5e910e6a42
Fix typo
|
1 year ago |
|
hongbo.mo
|
771da57167
Fix Dataset.md link to ../examples/custom_dataset.py
|
1 year ago |
|
hongbo.mo
|
0bc6a07a80
bugfix: update tqdm bar with the fixed gradient_accumulation_steps
|
1 year ago |
|
Matthias Reso
|
ec00a2f722
Fix batching error
|
1 year ago |
|
Matthias Reso
|
631046e883
Fix spell check
|
1 year ago |
|
Matthias Reso
|
72d7d9dcb8
Fix link
|
1 year ago |
|
Matthias Reso
|
1cfd709e26
Fix typos
|
1 year ago |
|
Matthias Reso
|
71e779694f
Add documentation for oaast1 dataset
|
1 year ago |
|
Matthias Reso
|
dc507b4e55
Finish implementation oasst preprocessing for of custom dataset
|
1 year ago |
|
Matthias Reso
|
32cf7ad459
[WIP] Preprocess oasst dataset
|
1 year ago |
|
Geeta Chauhan
|
d172d883b0
Allow easier use of custom datasets (#178)
|
1 year ago |
|
Matthias Reso
|
26b9b7dbb2
Give an explicit error message if custom datset function is not found
|
1 year ago |
|
Matthias Reso
|
ba1c789c51
Rename custom dataset function to comply with documentation
|
1 year ago |
|
Matthias Reso
|
99d97010ab
Fix typos
|
1 year ago |
|
Matthias Reso
|
9cb3fddfcf
Move datasets tests in their own subfolder
|
1 year ago |
|
Matthias Reso
|
64bbe5bfe1
Added documentation for custom dataset
|
1 year ago |
|
Matthias Reso
|
33c7738dcc
Added mechanism to supply a custom dataset without changing source files
|
1 year ago |
|
Geeta Chauhan
|
df77625e48
Add missing license header in test files (#177)
|
1 year ago |
|
Matthias Reso
|
2f7b4dbd3b
Add missing license header in test files
|
1 year ago |
