Inicio Explorar Axuda
Rexistro Iniciar sesión
marx
/
llama-recipes
1
0
Fork 0
Ficheiros Incidencias 0 Pull Requests 0 Wiki
Árbore: 135052a194
Ramas Etiquetas
llama-recipes
/
recipes
/
inference
/
model_servers
Jeff Tang 43a28956d1 update llama-on-prem.md to Llama 3 - format fix hai 7 meses
..
hf_text_generation_inference 6d449a859b New folder structure (#1) hai 8 meses
vllm 6d449a859b New folder structure (#1) hai 8 meses
README.md 6d449a859b New folder structure (#1) hai 8 meses
llama-on-prem.md 43a28956d1 update llama-on-prem.md to Llama 3 - format fix hai 7 meses

README.md

Running Llama2 On-Prem with vLLM and TGI

This tutorial shows how to use Llama 2 with vLLM and Hugging Face TGI to build Llama 2 on-prem apps.

* To run a quantized Llama2 model on iOS and Android, you can use the open source MLC LLM or llama.cpp. You can even make a Linux OS that boots to Llama2 (repo).