Overview We have been seeing amazing progress in generative AI and LLM recently. Thanks to the open-source efforts like LLaMA, Alpaca, Vicuna and Dolly, we start to see an exciting future of building our own open source language models and personal AI assistant. These models are usually big and compute-heavy. To build a chat service, we will need a large cluster to run an inference server, while c