|
|
@@ -76,7 +76,7 @@ Optional dependencies can also be combines with [option1,option2].
|
|
|
|
|
|
# Where to find the models?
|
|
|
|
|
|
-You can find llama v2 models on HuggingFace hub [here](https://huggingface.co/meta-llama), where models with `hf` in the name are already converted to HuggingFace checkpoints so no further conversion is needed. The conversion step below is only for original model weights from Meta that are hosted on HuggingFace model hub as well.
|
|
|
+You can find llama v2 models on Hugging Face hub [here](https://huggingface.co/meta-llama), where models with `hf` in the name are already converted to Hugging Face checkpoints so no further conversion is needed. The conversion step below is only for original model weights from Meta that are hosted on Hugging Face model hub as well.
|
|
|
|
|
|
# Model conversion to Hugging Face
|
|
|
The recipes and notebooks in this folder are using the Llama 2 model definition provided by Hugging Face's transformers library.
|
|
|
@@ -84,7 +84,7 @@ The recipes and notebooks in this folder are using the Llama 2 model definition
|
|
|
Given that the original checkpoint resides under models/7B you can install all requirements and convert the checkpoint with:
|
|
|
|
|
|
```bash
|
|
|
-## Install HuggingFace Transformers from source
|
|
|
+## Install Hugging Face Transformers from source
|
|
|
pip freeze | grep transformers ## verify it is version 4.31.0 or higher
|
|
|
|
|
|
git clone git@github.com:huggingface/transformers.git
|
|
|
@@ -141,7 +141,7 @@ Here we use FSDP as discussed in the next section which can be used along with P
|
|
|
|
|
|
## Flash Attention and Xformer Memory Efficient Kernels
|
|
|
|
|
|
-Setting `use_fast_kernels` will enable using of Flash Attention or Xformer memory-efficient kernels based on the hardware being used. This would speed up the fine-tuning job. This has been enabled in `optimum` library from HuggingFace as a one-liner API, please read more [here](https://pytorch.org/blog/out-of-the-box-acceleration/).
|
|
|
+Setting `use_fast_kernels` will enable using of Flash Attention or Xformer memory-efficient kernels based on the hardware being used. This would speed up the fine-tuning job. This has been enabled in `optimum` library from Hugging Face as a one-liner API, please read more [here](https://pytorch.org/blog/out-of-the-box-acceleration/).
|
|
|
|
|
|
```bash
|
|
|
torchrun --nnodes 1 --nproc_per_node 4 examples/finetuning.py --enable_fsdp --use_peft --peft_method lora --model_name /patht_of_model_folder/7B --fsdp_config.pure_bf16 --output_dir Path/to/save/PEFT/model --use_fast_kernels
|
Ikko Eltociear Ashimine