GTX 1080 + Hermes Desktop + Qwythos — Temperature Tuning That Saved My GPU

The Incident That Changed Everything

A 52-minute repetition loop. The GTX 1080 running at 182W, 89–90°C, continuously generating the same git diffs over and over. Gemini (running on the user's PC) had to physically intervene and warn: "Your GPU has been working at its absolute breaking points for nearly an hour."

⚠️ Root cause llama-server started with no generation safeguards. Greedy decoding (temp=0) locked the model into repeating tokens indefinitely. The fix was three flags added to the server startup command.

The Fix — Critical Server Flags

llama-server \
  -m /models/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf \
  -ngl 99 -c 64000 --reasoning off -t 4 \
  --temp 0.7 \
  --repeat-penalty 1.1 \
  -n 4096

Flag	Value	Why it matters
`--temp`	0.7	Temperature 0 = greedy decoding = repetition loop risk. 0.7 allows natural token diversity.
`--repeat-penalty`	1.1	Penalizes repeated token sequences. Prevents the model getting stuck generating the same lines over and over.
`-n`	4096	Max new tokens per response. Hard cap prevents runaway generations from consuming GPU indefinitely.
`--reasoning off`	—	Model is Q4_K_M quantization. Reasoning tokens break the output flow and add latency.
`-c 64000`	64K	VRAM-limited context. 8GB VRAM can't hold a full 1M token KV cache.

Without these flags, the same model that can generate a Flask API in 22 seconds will destroy your GPU for an hour.

The Full Setup

GPU: GTX 1080 8GB (Pascal, compute 6.1)
LXC host: 192.168.2.50 — Docker container with nvidia/cuda:12.6.0-runtime-ubuntu24.04
Backend: llama-server on port 8080
Desktop: Hermes Desktop (Windows) → custom provider pointing to http://192.168.2.50:8080/v1
Model: Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf (Q4_K_M, 5.5GB)
VRAM: 7.7 / 8GB steady — fully resident, no spill
Generation speed: ~30 tok/s
Effective context: 64K (VRAM-imposed)

Hermes Desktop Context Compression — The Hidden Wall

Even with a working model, Hermes Desktop has aggressive default settings that truncate sessions at ~5K tokens, causing "Context length exceeded" errors during iterative coding:

# Default (before fix) — way too aggressive
compression.enabled: true
compression.threshold: 0.5      # compresses at 50% context
hygiene_hard_message_limit: 400
agent.max_turns: 60            # dies after 60 turns

# Fixed settings
compression.threshold: 0.8
compression.target_ratio: 0.35
agent.max_turns: 200
compression.protect_last_n: 50

Test Results

Test	Method	Outcome
Snake HTML game	Desktop single prompt	✅ Clean, runs
Minimal HTTP Server	Desktop iterative	⚠️ Hit context wall, self-corrected but ran out of room
HTTP Server	Direct API (single shot)	✅ 22 sec, 629 tokens, no errors
Flask REST API (multi-file)	Desktop OpenCode	🔴 52-min repetition loop — no temp/repeat flags
Flask REST API (multi-file)	Desktop OpenCode (+ flags)	⚠️ 11 min, 9-task decomposition, real bugs found and fixed

The Model Did Real Engineering Work

The 11-minute Flask session was remarkable — Qwythos broke the project into a 9-task checklist:

Create project structure
Implement SQLite + User model
POST /register
POST /login with JWT
GET /protected
404/500 error handlers
requirements.txt
pytest tests
Run tests + fix failures

Then it worked through real engineering problems:

Discovered the app's password hashing used PBKDF2 (werkzeug) but tests expected SHA-256
Found /protected/userinfo vs /protected endpoint path mismatch
Fixed missing hash_password function
Corrected fixture password literals vs stored hashes

This wasn't slow — it was doing actual iterative software development. The same work that takes humans hours.

The Core Lesson

The model is only as safe as the server flags you set. Temperature, repetition penalty, and max tokens aren't optional tuning — they're the guardrails that separate a useful coding assistant from a GPU destroyer.

Always set --temp ≥ 0.7 (never 0 for coding tasks)

Always set --repeat-penalty ≥ 1.1

Always set -n (max tokens) as a hard cap

Raise Hermes Desktop agent.max_turns before coding sessions

Raise compression.threshold to 0.8 for less aggressive compression

Context Window — The Label vs Reality

The model is branded "1M context" but 8GB VRAM can only hold ~64K tokens for this Q4_K_M 9B model. The 1M label is the model's trained context window, not what fits in memory. This is normal — context is always VRAM-limited on consumer GPUs.