ollama on ubuntu 24.04
On Ubuntu 24.04, Ollama runs as a background systemd service by default, which means setting environment variables in your regular ~/.bashrc will not apply to Ollama unless you configure them inside its service override file.
Step 1: Configure Ollama on Ubuntu 24.04
Run this command in your terminal to open the systemd override editor for Ollama:
bash
sudo systemctl edit ollama.service
Wees voorzichtig met code.
Add the following lines inside the [Service] section (at the top of the file, above the existing comments):
ini
[Service]
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="OLLAMA_CONTEXT_LENGTH=16384"
Wees voorzichtig met code.
Save and exit the editor (Ctrl+O, Enter, Ctrl+X if using nano), then reload and restart the Ollama service:
bash
sudo systemctl daemon-reload
sudo systemctl restart ollama
Wees voorzichtig met code.
Step 2: Verify It Fits 100% in Your RTX 3060 VRAM
1. Trigger a prompt from your IDE extension (like Cline, Roo Code, or Continue) so the model loads into memory.
2. In your Ubuntu terminal, run:bash
ollama ps
Wees voorzichtig met code.
3. Check the PROCESSOR and CONTEXT columns:
• PROCESSOR = 100% GPU: Your model plus the 16k context window fits entirely inside your 12GB RTX 3060 VRAM.
• PROCESSOR shows a CPU split (e.g., 85%/15% CPU/GPU): A 16k context pushed your model slightly over 12GB. Lower OLLAMA_CONTEXT_LENGTH to 12288 or 8192 in sudo systemctl edit ollama.service and restart until it reads 100% GPU.
Step 3: Monitor VRAM Live While Vibe Coding
Keep a terminal tab open with nvidia-smi watching your VRAM usage while your coding agent runs:
bash
watch -n 1 nvidia-smi
Wees voorzichtig met code.
If you see free VRAM left over (for example, 9500MiB / 12288MiB), you have room to push OLLAMA_CONTEXT_LENGTH even higher—up to 24576 or 32768—before needing to buy your second GPU.
(Bonus for Ubuntu 24.04: When you add a second RTX 3060 or RTX 3090 later, Ubuntu's NVIDIA driver and Ollama will detect both cards automatically with zero extra driver configuration needed.)
Which coding model (e.g., Qwen3-Coder, DeepSeek, Devstral) and IDE plugin are you running on Ubuntu right now? Let me know if ollama ps shows 100% GPU after applying this override.
Reacties
Een reactie posten