Merge pull request #4 from facebookresearch/doc-requirements-update

Doc requirements update

README.md

@@ -81,7 +81,7 @@ Here we make use of Parameter Efficient Methods (PEFT) as described in the next
 
 ### Multiple GPUs One Node:
 
-**NOTE** please make sure to use PyTorch Nightlies for using PEFT+FSDP .
+**NOTE** please make sure to use PyTorch Nightlies for using PEFT+FSDP. Also, note that int8 quantization from bit&bytes currently is not supported in FSDP.
 
 ```bash
 

docs/mutli_gpu.md

@@ -26,6 +26,8 @@ This runs with the `samsum_dataset` for summarization application by default.
 
 **Multiple GPUs one node**:
 
+**NOTE** please make sure to use PyTorch Nightlies for using PEFT+FSDP. Also, note that int8 quantization from bit&bytes currently is not supported in FSDP.
+
 ```bash
 
 torchrun --nnodes 1 --nproc_per_node 4  ../llama_finetuning.py --enable_fsdp --model_name /patht_of_model_folder/7B --use_peft --peft_method lora --output_dir Path/to/save/PEFT/model

requirements.txt

@@ -9,7 +9,7 @@ black[jupyter]
 datasets
 fire
 git+https://github.com/huggingface/peft.git
-transformers
+transformers>=4.31.0
 sentencepiece
 py7zr
 scipy