How To Get Started Running Small Language Models at the Edge
原文英文,约1100词,阅读约需4分钟。发表于: 。In my previous article, I introduced the idea of federated language models that take advantage of large language models (LLM) The post How To Get Started Running Small Language Models at the Edge...
本文介绍了联邦语言模型的概念,使用云端的大型语言模型(LLM)和边缘设备上的小型语言模型(SLM)。作者使用Nvidia的Jetson Orin开发套件作为边缘设备,配置了轻量级模型服务器Ollama,利用GPU加速来加快Phi-3的推理速度。
![How To Get Started Running Small Language Models at the Edge](https://cdn.thenewstack.io/media/2024/07/6ecf3e41-getty-images-zyyrzizlmao-unsplash.jpg)