GitHub - mlc-ai/web-llm: High-performance In-browser LLM Inference Engine

In-Browser Inference: WebLLM is a high-performance, in-browser language model inference engine that leverages WebGPU for hardware acceleration, enabling powerful LLM operations directly within web browsers without server-side processing. Full OpenAI API Compatibility: Seamlessly integrate your app with WebLLM using OpenAI API with functionalities such as streaming, JSON-mode, logit-level control,

[gfx]

OpenAI APIコンパチなLLMがブラウザに乗ると。

[efcl]

WebGPUを利用しているLLM