A lightning-fast local AI chat app using FastAPI, Python, and Ollama running Google Gemma3—streaming responses, ready for your portfolio, side projects, or prototypes!
Features
- Local LLM: Runs Google Gemma3 (
gemma3:1b,4b, etc.) on your own hardware using Ollama. - FastAPI backend: Optimized API with streaming responses for instant token display.
- Zero-JS frontend: Minimal HTML/JS for chat and live model replies—no frameworks needed.
- Easy deployment: Lockfile-based dependency management using uv.
- Docker-ready: Provided Dockerfile for reproducible builds, local development, or cloud/server hosting.
- Editable prompts: Full chat history, adjustable system prompt, model tag, and sampling controls.
- OpenAI-compatible option: Use with OpenAI client SDKs for future cloud portability.