JieShen 99730471fb Fix examnples.py filename error преди 1 година
..
chat_completion b4c71bebee Update chat_completion.py преди 1 година
code_llama 2374b73aad Remove __init__.py files from examples преди 1 година
hf_text_generation_inference 2374b73aad Remove __init__.py files from examples преди 1 година
vllm 3ddf755539 Move examples into subfolders преди 1 година
Getting_to_know_Llama.ipynb 0190444849 minor updates преди 1 година
README.md 99730471fb Fix examnples.py filename error преди 1 година
custom_dataset.py 8620ab8ac2 Fix invalid labels for context in custom dataset/oasst1 преди 1 година
finetuning.py 7702d702cc Add missing file extension преди 1 година
inference.py 8ac44ef3be Fix vocab size mismatch in inference due to added pad token преди 1 година
multi_node.slurm 360a658262 Adjusted docs to reflect move of qs nb + finetuning script into examples преди 1 година
quickstart.ipynb a0cd3c7c77 Added dependency to qsnb преди 1 година
samsum_prompt.txt ccda6fb8ca Move inference scripts into example folder преди 1 година

README.md

Examples

This folder contains finetuning and inference examples for Llama 2. For the full documentation on these examples please refer to docs/inference.md

Finetuning

Please refer to the main README.md for information on how to use the finetuning.py script. After installing the llama-recipes package through pip you can also invoke the finetuning in two ways:

python -m llama_recipes.finetuning <parameters>

python examples/finetuning.py <parameters>

Please see README.md for details.

Inference

So far, we have provide the following inference examples:

  1. inference script script provides support for Hugging Face accelerate, PEFT and FSDP fine tuned models. It also demonstrates safety features to protect the user from toxic or harmful content.

  2. vllm/inference.py script takes advantage of vLLM's paged attention concept for low latency.

  3. The hf_text_generation_inference folder contains information on Hugging Face Text Generation Inference (TGI).

  4. A chat completion example highlighting the handling of chat dialogs.

  5. Code Llama folder which provides examples for code completion and code infilling.

For more in depth information on inference including inference safety checks and examples, see the inference documentation here.

Note The sensitive topics safety checker utilizes AuditNLG which is an optional dependency. Please refer to installation section of the main README.md for details.

Note The vLLM example requires additional dependencies. Please refer to installation section of the main README.md for details.

Train on custom dataset

To show how to train a model on a custom dataset we provide an example to generate a custom dataset in custom_dataset.py. The usage of the custom dataset is further described in the datasets README.