 Matthias Reso
|
27e56bdfd3
Add llama_finetuning.py script to provide support for torchrun
|
il y a 1 an |
|
Matthias Reso
|
4c9cc7d223
Move modules into separate src folder
|
il y a 1 an |
 lchu
|
feaa344af3
resolve conflicts
|
il y a 1 an |
 Hamid Shojanazeri
|
51269b816f
moving Bt to the try block
|
il y a 1 an |
|
lchu
|
3d1e9cd58c
minor code optimization
|
il y a 1 an |
|
lchu
|
41ffbcab52
code cleanup to remove all unused imports
|
il y a 1 an |
|
lchu
|
1cc9df19e6
remove unused import
|
il y a 1 an |
|
Hamid Shojanazeri
|
b2a55022cb
clean up
|
il y a 1 an |
|
Hamid Shojanazeri
|
44ef280d31
adding flash attention and xformer memory efficient through PT SDPA
|
il y a 1 an |
|
lchu
|
0c51b47262
fix #90
|
il y a 1 an |
|
lchu
|
c19c5c69aa
fix fsdp construction on low_cpu_fsdp
|
il y a 1 an |
|
lchu
|
895dfcea30
add nightly check for using low_cpu_fsdp mode
|
il y a 1 an |
|
lchu
|
1e64fc98d9
switch to simpler param_init_fn and meta device init
|
il y a 1 an |
|
lchu
|
101391f46a
Revert "replace init_empty_weights with torch.device(meta)"
|
il y a 1 an |
|
lchu
|
c8d4f38d23
replace init_empty_weights with torch.device(meta)
|
il y a 1 an |
|
lchu
|
d8a81bb531
save cpu mem by leveraging FSDP rank0 broadcasting
|
il y a 1 an |
 Andrew Gu
|
71fdc4920a
Save memory and fix typos
|
il y a 1 an |
|
Hamid Shojanazeri
|
7ec390bfc8
aliging special tokens in toeknizer with HF latest
|
il y a 1 an |
 Rohan Varma
|
d3d7a1656e
Update llama_finetuning.py
|
il y a 1 an |
 chauhang
|
4767f09ecd
Initial commit
|
il y a 1 an |
