QRF Local AI Engineering Journal August 02 2026

Date: August 2, 2026 (Rev. 3.1 — audited against /Volumes/AI_DATA on disk) Hardware: Mac Studio M3 Ultra — 96 GB unified memory · AI_DATA volume 2 TB (773 GB free — plenty for all downloads below) Decision: Qwen3.5-122B-A10B mxfp4 is KEPT as the escalation tier, gated behind a warning + confirmation flow.

This section is a standalone reference — read this first for day-to-day operation. Everything from §0 onward is the deeper build log (what was done, in what order, and why) for when you need the full story behind a decision.

You have three separate, independently-controllable systems, each with its own start/stop/restart buttons. They are not layers of the same thing — each covers different services, and starting one does not start another.

What / Why / How / When for each:

ai-stack — what: one script that brings up everything a normal chat session needs (Ollama + Open WebUI + gateway + the public Funnel bridge). Why: so you don't have to remember three separate commands just to start your day. How:

bash /Volumes/AI_DATA/start-ai-stack.command

or double-click it (or its Desktop shortcut, if you made one — see local-server/README.md). When: after a reboot, or any time you want the full system up. Verify with curl localhost:8080 (Open WebUI) and ollama list (models present).

agent-server — what: the newer, actively-developed harness with knowledge-base retrieval, MCP tool calls, memory, and feedback logging — a completely different codebase from gateway.py, not a replacement for it (see AGENT_SERVER_GUIDE_v2.md QS.1 for the two-systems-side-by-side rationale). Why: apps that need retrieval-grounded answers or tool-calling need this; apps that just need a plain chat completion can stay on the simpler legacy gateway. How:

bash "/Volumes/AI_DATA/Restart-Agent-Server.command"

When: after editing anything under apps/agent-server/ (agent_loop.py, mcp_servers.json, hooks, etc.), or if :8788 isn't responding. Verify with curl localhost:8788/health (once you have a valid agent-server key).

AI Local Server — what: the narrowest of the three — only touches gateway.py. Why: editing gateway.py (like the heavy-warning-gate code added in Step 4) has zero effect until the process restarts; using the full ai-stack restart for that would also unnecessarily bounce Ollama and Open WebUI (interrupting anyone mid-chat). How:

bash "/Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/Restart AI Local Server.command"

When: any time you change gateway.py, api_keys.json, or gateway.env. Verify with a real curl test (see §4/Quick Reference below).

What / Why / How / When, grouped by role:

Text — default vs fast vs heavy. What: three tiers by size/cost. Why: one model can't be both cheap-and-fast and deeply capable — routing saves compute. How: the gateway/agent-server picks the model per the model field in the request; you choose which tier by which model name you send. When: fast tier for routing/short answers, default for everyday work, heavy only for genuinely hard problems (and only with X-Confirm-Heavy: yes, since it's exclusive-mode — see Step 4).

Vision. What: qwen3-vl:30b, plus vision built into qwen3.6:35b-a3b. Why kept separate for now: not yet proven equivalent — run the parity test (§5 #5) before retiring one. How: send an image in the request the same way you would to any vision-capable Ollama model. When: any image-input task.

Coding. What: qwen3.6:35b-a3b for most coding, qwen3-coder-next for long-horizon agentic coding jobs. Why two tiers: Coder-Next is purpose-built for long tool-use chains but has a known bug on very long prompts (§5 #3) — until tested, default to the 35B. How: swap the model field; Coder-Next needs the 35B unloaded first (memory). When: Coder-Next only after its bug test passes.

Speech (TTS/STT). What: Qwen3-TTS for text→speech (clone a voice from 3 seconds of audio), Whisper for speech→text. Why: two directions need two different model families — no single model does both well. How: invoked via mlx-audio scripts (TTS) or whisper.cpp (STT), not a persistent server port. When: any voice-input or voice-output feature.

RAG (embedding + reranker). What: nomic-embed-text turns your knowledge-bank documents into searchable vectors; the reranker re-scores the top hits for relevance before they reach the model. Why: embedding alone gets you "roughly related" results — the reranker sharpens that to "actually answers the question." How: Open WebUI calls these automatically during a knowledge-bank search; no manual invocation needed. When: automatic, every RAG-grounded query.

Image/Video. What: FLUX.2 Klein (images), LTX-2.3 (video). Why on-demand, not resident: both are large and bursty — no reason to keep them loaded when idle. How: invoked via mflux-generate (image) or your LTX inference tooling (video) as one-off jobs, not always-on servers. When: whenever an image/video generation request comes in.

# --- Status checks ---
pgrep -x ollama && echo "Ollama running"
pgrep -f open-webui && echo "Open WebUI running"
pgrep -f "gateway\.py" && echo "Gateway (AI Local Server) running"
lsof -ti tcp:8788 && echo "agent-server running"
lsof -ti tcp:8082 && echo "MLX fast tier (9B) running"
ollama list                              # models Ollama has on disk
du -sh /Volumes/AI_DATA/models/hf-cache  # total MLX model storage used
df -h /Volumes/AI_DATA                   # free disk space

# --- Start / stop / restart ---
bash /Volumes/AI_DATA/start-ai-stack.command
bash /Volumes/AI_DATA/stop-ai-stack.command
bash /Volumes/AI_DATA/restart-ai-stack.command
bash "/Volumes/AI_DATA/Restart-Agent-Server.command"
bash "/Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/Start AI Local Server.command"
bash "/Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/Stop AI Local Server.command"
bash "/Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/Restart AI Local Server.command"

# --- Health checks ---
curl localhost:8080                      # Open WebUI
curl localhost:11434/api/tags            # Ollama, list loaded/available models
curl -s -o /dev/null -w "%{http_code}\n" -X POST localhost:8787/v1/chat/completions -H 'Authorization: Bearer sk-YOUR-KEY' -H 'Content-Type: application/json' -d '{"model":"qwen3.6:35b-a3b","messages":[{"role":"user","content":"hi"}]}'

# --- Logs ---
tail -f /Volumes/AI_DATA/logs/ollama.log
tail -f /Volumes/AI_DATA/logs/open-webui.log
tail -f /Volumes/AI_DATA/logs/gateway.log
tail -f /Volumes/AI_DATA/logs/agent-server.log

# --- MLX venv ---
mlxenv                                   # activate (alias — new Terminal windows only)
deactivate                               # leave the venv

# --- Memory / system ---
sysctl iogpu.wired_limit_mb              # should read 90112
sudo memory_pressure                     # check headroom while a heavy model is loaded

# --- Keys ---
python3 /Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/manage_keys.py list
python3 /Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/manage_keys.py add <app-name>

Bottom line (re-checked Aug 2 evening): every download, every file edit, AND the gateway itself are done and confirmed live — the gate returned a real 428. Nothing left is blocking. What remains is entirely §5's validation tests (confirm header override, Coder-Next bug check, vision parity, memory headroom, 122B bake-off) — all optional-pace, none urgent.

                    ┌────────────────────────┐
                    │  gateway.py  :8787  ✅ │   LIVE — heavy gate confirmed working
                    │ OpenAI-compatible API  │   (real 428 returned, Aug 2 evening)
                    └───────────┬────────────┘
                                │
                         Task Router
                                │
       ┌────────────┬───────────┼───────────┬──────────────┐
       ↓            ↓           ↓           ↓              ↓

      FAST        DEFAULT    CODE (also    AUDIO          MEDIA
      ✅ HAVE     ✅ HAVE     = DEFAULT)   STT ✅/TTS ✅   ✅ HAVE
 Qwen3.5-9B    qwen3.6:     qwen3.6:     Whisper-turbo✅  FLUX.2
    ~6GB       35b-a3b      35b-a3b      Qwen3-TTS ✅     Klein 4B
   downloaded, (Ollama ✅,  (73.4%        downloaded       downloaded
   served once  MLX ✅)     SWE-bench)                    on demand
   on :8082 —
   not yet a
   persistent
   launch entry

                   ↓                          ↓
       HEAVY CODE JOB ✅ downloaded      VIDEO JOB ✅ downloaded
       bug test pending (§5 #3)          on demand, no test needed
                                        LTX-2.3 22B
          Qwen3-Coder-Next               distilled (v1.1)
          via Ollama/GGUF                43GB, single file
          ~45GB — unload 35B first       confirmed on disk

                   ↓
             HARD REQUEST  ⚠ GATED — warning + confirm required (once gateway is running)

           TODAY: gpt-oss:120b ✅ (~64GB, exclusive, live via Ollama)
           DOWNLOADED: Qwen3.5-122B-A10B mxfp4 ✅ (~62GB, in hf-cache)
           ~66GB — EXCLUSIVE MODE — context capped 32K
           bake-off pending (§5 #4) — keep BOTH until 122B wins your
           test set, then retire gpt-oss:120b

Qwen3.6-35B-A3B                ~20 GB   ✅ live (Ollama GGUF; MLX copy also on disk)
Qwen3.5-9B-4bit                 ~6 GB   🟨 downloaded + tested once on :8082 —
                                          not yet in a persistent launch script
                                          (add the mlx_lm.server line from §STEP 3
                                          to start-ai-stack.command to make it survive reboots)
Qwen3-TTS-12Hz-1.7B-4bit        ~2 GB   ✅ downloaded — not yet wired into a server process
Whisper-large-v3-turbo        ~1.5 GB   ✅ live
nomic-embed-text                ~0.3 GB ✅ live (keep — see §6)
Qwen3-Reranker-0.6B             ~1 GB   ✅ downloaded — not yet wired into a server process
--------------------------------------
Models                         ~31 GB
Headroom (KV/RAG/users)        ~57 GB

Note: "downloaded, not yet wired in" means the weights are on disk and ready, but no long-running process is currently serving them (TTS/Reranker don't need a persistent server the way the LLM tiers do — they're invoked per-job via mlx-audio/sentence-transformers scripts, not always-on ports). Nothing further is required here unless you want them behind a persistent API.

Total unified memory              96 GB
GPU wired limit (raised)         ~88 GB   ✅ set to 90112 MB, Aug 2
OS + runtime reserve              ~8 GB

SWAP-IN (exclusive, one at a time) — ALL WEIGHTS NOW ON DISK:
  FLUX.2 Klein       ~8 GB   ✅ downloaded — coexists with residents, no unload
  LTX-2.3 distilled ~43 GB   ✅ downloaded (v1.1, single file) — unload 35B first
  Coder-Next GGUF   ~51 GB   ✅ downloaded — unload 35B first; bug test pending (§5 #3)
  gpt-oss:120b       ~64GB   ✅ live via Ollama — ⚠ EXCLUSIVE, unload EVERYTHING
  Qwen3.5-122B mxfp4 ~62GB   ✅ downloaded, not yet served — bake-off vs gpt-oss:120b pending (§5 #4)
                              context hard-capped at 32K when either is active
                              warning + explicit confirm required (gate code
                              written, awaiting gateway restart — §4)

Note: your OLLAMA_MAX_LOADED_MODELS=2 + KEEP_ALIVE=30m already implements soft swap-in behavior for Ollama-served models. The gate in Step 4 adds the explicit warning layer for the heavy tier. Your OLLAMA_CONTEXT_LENGTH=65536 is fine for residents but the heavy model should be called with ≤32K.

Completed: plist created at /Library/LaunchDaemons/com.llmpowerhouse.wiredlimit.plist, validated (plutil → OK), loaded via launchctl bootstrap system, and verified: sysctl iogpu.wired_limit_mb = 90112 (~88 GB). Persists across reboots.

After your next reboot, sanity-check once:

sysctl iogpu.wired_limit_mb

If it ever reads 0 again, re-run the one-paste block in Appendix A.

Terminal tip learned during this step: interactive zsh does NOT treat # as a comment, so pasting lines with trailing # comments feeds them to the command as arguments (harmless noise like sysctl: unknown oid '#'). Paste blocks in this doc are now comment-free.

Done ✅: 2a venv (apps/mlx/venv — why the venv: isolates MLX from system Python, matches your open-webui pattern, survives macOS updates); 2b partial (Qwen3.5-9B ✅); 2c Qwen3.6-35B MLX ✅; 2d qwen3-coder-next ✅ pulled; 2e Qwen3.5-122B mxfp4 ✅ downloaded. hf-cache now 86 GB — all on AI_DATA as designed. Impact: fast tier + escalation tier weights are on disk; routing tiers can now actually be served. Remaining ⬜: 2b's other four — TTS, reranker, FLUX.2 Klein, LTX-2.3 (run mlxenv first, then the four hf download lines below). Where: Terminal. Why HF_HOME: keeps everything on AI_DATA (portable/backupable, your design rule). ⚠ Aug 2 lesson: interactive zsh chokes on # comments — the first attempt failed at line 1 and NOTHING ran. All blocks below are now comment-free, and block 2a's first line permanently enables comment-tolerant pasting.

2a. One-time environment setup — ✅ DONE (Aug 2 pm)

venv created at /Volumes/AI_DATA/apps/mlx/venv; mlx-lm, mlx-audio, mflux, huggingface_hub, psutil all installed successfully (large dependency tree — torch, transformers, etc. — that's expected, mflux needs them). mlxenv alias appended to ~/.zshrc.

What venv is, and the 3 rules for using it:

1. What: an isolated Python install at apps/mlx/venv holding mlx-lm / mflux / mlx-audio / the hf CLI. Bare macOS pip/hf don't have these — that's why the very first attempt failed with command not found: pip.
2. When: activate it once per new Terminal window, before any hf download, mlx_lm.server, or mflux-generate command.
3. How: type mlxenv and press enter. Your prompt gains a (venv) prefix — that's confirmation it's active. deactivate to leave it (rarely necessary). Scripts/LaunchDaemons should skip the alias and use the absolute path instead, e.g. /Volumes/AI_DATA/apps/mlx/venv/bin/mlx_lm.server.
4. What it does NOT affect: plain macOS commands — rm, ls, cd, du, mv, ollama, curl, sysctl, df — work identically whether the venv is active or not. Only hf, python, pip, mlx_lm.server, and mflux-* commands actually need it. Leaving (venv) active while running an unrelated command (like rm -rf for cleanup) is harmless.

⚠ Gotcha hit Aug 2: the alias only works in Terminal windows opened, or sourced, after it was appended to ~/.zshrc. If mlxenv says "command not found," run this once in that window:

source ~/.zshrc

then mlxenv works immediately and in every new window from then on.

Reference (already run — for rebuilds only):

python3 -m venv /Volumes/AI_DATA/apps/mlx/venv
source /Volumes/AI_DATA/apps/mlx/venv/bin/activate
python -m pip install -U pip
python -m pip install -U mlx-lm mlx-audio mflux huggingface_hub psutil
echo 'alias mlxenv="source /Volumes/AI_DATA/apps/mlx/venv/bin/activate"' >> ~/.zshrc

2b. Core downloads — ✅ 4 of 5 DONE (Aug 2 pm): Qwen3.5-9B, Qwen3-TTS CustomVoice, Qwen3-Reranker, FLUX.2 Klein are all on disk and verified (du -sh confirmed sizes match expectations). Reference commands, already run:

hf download mlx-community/Qwen3.5-9B-4bit
hf download Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
hf download mlx-community/Qwen3-Reranker-0.6B-4bit
hf download Runpod/FLUX.2-klein-4B-mflux-4bit

2f. LTX-2.3 video — ⚠ first attempt aborted, fix below.

What happened: --include "*distilled*" is a glob, not an exact match. The LTX-2.3 repo contains TWO full 46GB checkpoints whose filenames both contain "distilled" (ltx-2.3-22b-distilled.safetensors and a newer ltx-2.3-22b-distilled-1.1.safetensors) plus two 7.6GB LoRA files that also match — total 108GB queued, i.e. two copies of the same model. Aborting at ~40GB was the right call.

What mlxenv actually is: it's a one-word shortcut (a zsh alias) you created in §2a for the command source /Volumes/AI_DATA/apps/mlx/venv/bin/activate. That command switches your Terminal session into the MLX virtual environment — an isolated Python install living at apps/mlx/venv that contains mlx-lm, mflux, mlx-audio, and the hf CLI. Why it exists: macOS's system Python doesn't have pip wired up for general use (you hit this directly — the very first pip install attempt in §2a failed with command not found: pip), and even if it did, installing AI packages into system Python risks breaking other tools or getting wiped by a macOS update. A dedicated venv on AI_DATA sidesteps both problems and matches the pattern your other apps (open-webui, llm-wiki) already use — fully isolated, fully portable, fully backed up with the rest of AI_DATA. What activating it does, technically: it temporarily prepends the venv's bin/ folder to your shell's PATH, so typing hf, python, or mlx_lm.server finds the venv's copies instead of (missing) system ones. That's the ENTIRE effect — it does not touch the filesystem, install anything system-wide, or change how any non-Python command behaves.

So, do you need mlxenv for the cleanup command below? No. Since activating the venv only affects which python/pip/hf binary your shell finds, it has zero effect on plain macOS commands like rm, ls, cd, du, mv, ollama, curl, sysctl. Those work identically active or not. Rule of thumb: venv matters only for the Python-based AI tools — hf download, mlx_lm.server, mflux-generate, python. Everything else, skip it. Running rm -rf with (venv) showing in the prompt (as you did) is completely harmless — it just wasn't necessary.

Cleanup — reclaim the ~38GB of partial files (venv not required):

rm -rf /Volumes/AI_DATA/models/hf-cache/hub/models--Lightricks--LTX-2.3

Corrected download — one exact file, the newer v1.1 checkpoint (~46GB):

mlxenv
hf download Lightricks/LTX-2.3 --include "ltx-2.3-22b-distilled-1.1.safetensors"

(Verified against the live HF repo API Aug 2: this file alone is 46,149,345,334 bytes ≈ 46GB — no LoRA needed, it's a complete merged checkpoint.)

2c. Optional — ✅ DONE. MLX copy of the default model (you already have the GGUF). Confirmed complete via hf's cache check: re-running the command shows 17/17 files 100% with 0.00B downloaded — that means every file was already present and verified, so nothing new was fetched. That's the expected, correct output when a model is already fully on disk; not an error.

hf download mlx-community/Qwen3.6-35B-A3B-4bit

2d. Conditional — ✅ ALREADY DONE, no action needed. Heavy code model, confirmed pulled earlier this session (pulling manifest... success in your log, and the manifest is on disk at models/ollama/manifests/registry.ollama.ai/library/qwen3-coder-next). The download step is finished — what's still outstanding is the §5 bug test (not a download): paste a 20K+ token prompt and confirm output doesn't degrade past ~1K generated tokens before treating this model as production- ready for coding. Reference command, already run:

ollama pull qwen3-coder-next

2e. Gated — ✅ ALREADY DOWNLOADED, no action needed. Confirmed on disk: 62GB in hf-cache under models--mlx-community--Qwen3.5-122B-A10B-mxfp4.

Purpose — why this model exists in the plan: it's the "escalation" / "heavy" tier — the local frontier model reserved for your hardest reasoning requests, run occasionally in EXCLUSIVE mode (everything else unloads while it's active — that's what the Step 4 warning gate protects against triggering by accident). You already run gpt-oss:120b in that exact same role via Ollama. The 122B is a candidate replacement, not an addition — per §5.4, the plan is to run your own knowledge-bank test questions against both and keep whichever answers better, then free the disk space on the loser. It's "gated" for two reasons: (1) it only fits in memory now that the wired-limit raise is done (✅ Step 1), and (2) it shouldn't become your default heavy model until it's actually proven better than what you're already running — that bake-off is the next real action item for this model, not a download.

Reference command, already run:

hf download mlx-community/Qwen3.5-122B-A10B-mxfp4

Disk status Aug 2 (verified): hf-cache = 132 GB (9B 5.6GB + 35B-MLX 20GB

• 122B 62GB + TTS 4.3GB + Reranker 335MB + FLUX.2 4.4GB), ollama = 102 GB (unchanged) → 591 GB free of 1.9 TB. Adding LTX-2.3's 46GB leaves ~545 GB free — comfortable headroom.

Summary: Ollama hub already running via start-ai-server.sh; fast tier launched with mlx_lm.server on :8082 (it auto-downloaded Qwen3.5-9B on first start — that was the 5.98 GB download you watched). Impact: the router now has a cheap always-on tier, so simple requests stop burning the 35B's compute. Why MLX for this tier: the 20–87% MLX speed advantage is largest on models under ~14B — exactly this slot.

The (venv) in your prompt = the mlx venv is active in that Terminal (from source .../activate). It only affects that window; type deactivate to leave, mlxenv to re-enter. Launch commands in scripts should use the absolute path so they don't depend on it:

/Volumes/AI_DATA/apps/mlx/venv/bin/mlx_lm.server --model mlx-community/Qwen3.5-9B-4bit --port 8082 &

To survive reboots, add that line to start-ai-stack.command. Health check: curl localhost:8082/v1/models.

Real proof, not just a file edit: gateway started successfully, and a live curl test with a valid sk-... key returned:

curl ... -d '{"model":"gpt-oss:120b", ...}'   (no X-Confirm-Heavy header)
→ 428

That's the gate genuinely intercepting a heavy-model request and blocking it, exactly as designed. And the override was confirmed too (§5 #2, same evening) — the identical request plus -H 'X-Confirm-Heavy: yes' returned a real gpt-oss:120b completion (118 tokens, clean finish_reason: stop), not a 428. Both directions of the gate are proven live. Nothing left to do for this step.

Note on the reference code below: it was pasted into a terminal prompt a second time (same harmless mistake as the first time — zsh: parse error near '}', no damage). It doesn't need to be touched again — it's already live inside gateway.py on disk, proven by the 428 above. Keep it below only as documentation for future rebuilds; never type/paste it at a shell prompt.

Everything below this line is background on how the code got here — already done, kept for reference.

What happened: the real 200 test proved the gate code was genuinely absent (correcting the earlier mistaken "complete" mark, which was based on an inconclusive 401 auth-only result). With permission, the code was added directly to gateway.py on disk — two edits: the HEAVY_MODELS set + HEAVY_WARNING message near the other module-level constants, and the actual if model in HEAVY_MODELS and ... != "yes": return 428 check right after the existing model allow-list check in do_POST. python3 -m py_compile gateway.py confirmed the file is syntactically valid.

⚠ Restart uncovered a second, unrelated problem — also now fixed. Restarting killed the running gateway process (pid confirmed stopped), but it then refused to come back up: Missing gateway.env. Turned out gateway.env had never existed on this machine — only gateway.env.example. Whatever gateway.py process answered our earlier curl tests must have been started by hand at some point (env vars exported manually, bypassing this launcher), and the restart script's safety check correctly blocks starting without the file. Fixed: gateway.env created from the example with a freshly generated random GATEWAY_SHARED_SECRET (your real auth already runs off api_keys.json per-app keys regardless, so this satisfies the launcher's file-exists check without weakening anything).

Three separate restart scripts exist on this machine — use the right one:

Does this script need to be built? No — it already existed. Verified on disk: "Restart AI Server.command" and "Start AI Server.command" were dated Jul 25, 2026 — built well before this Round 3 session, by whoever set up the original gateway/site infrastructure. Nothing was created for this guide; the files just aren't at the AI_DATA root (where restart-ai-stack.command and Restart-Agent-Server.command live) — they're one level down, in the same folder as gateway.py itself: /Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/. That's a deliberate, sensible layout: scripts that only touch one component live next to that component, rather than every script for every app cluttering the top-level folder.

✅ Renamed Aug 2 (per request): all three gateway-only buttons were renamed to include "Local" — Start AI Server.command → Start AI Local Server.command, same pattern for Restart and Stop. Purpose: distinguish these gateway-only buttons from the full-stack Desktop icons (Start-AI-Server, etc., covering Ollama + Open WebUI + gateway together — see local-server/README.md §"Desktop integration"). Two cross-references were updated to match: README.md's own mention of the filenames, and _gateway_lib.sh's "already running" message. The scripts' own code needed no changes — none of them reference their own filename internally.

Desktop shortcuts — added the same way this repo already does it. local-server/README.md documents the exact pattern used for the full-stack buttons (ln -sf <target> ~/Desktop/<Name>, no .command extension on the Desktop icon name). I can't create files in your actual Mac's ~/Desktop myself — that folder isn't part of the AI_DATA mount I have access to — so run this once yourself, in Terminal:

ln -sf "/Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/Start AI Local Server.command"   ~/Desktop/"Start AI Local Server"
ln -sf "/Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/Stop AI Local Server.command"    ~/Desktop/"Stop AI Local Server"
ln -sf "/Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/Restart AI Local Server.command" ~/Desktop/"Restart AI Local Server"

Three new icons appear on your Desktop, each a live symlink back to the real script on AI_DATA (nothing is duplicated). Per the same README note: the first double-click of each new icon needs right-click → Open once, since they're unsigned scripts — after that, double-click works normally.

Why closing this venv Terminal window warns you, but the other two scripts' windows don't:

• What you're seeing: macOS Terminal's "Closing this window will terminate the running processes: bash, Python" dialog appears when you try to close a window whose shell still has a live child process attached — in your case, mlx_lm.server (Python), which you started directly at an interactive prompt with a trailing &.
• Why it happens here: typing a command with & backgrounds it, but it remains a child of that specific shell session. As long as the shell (bash/zsh) in that window is still running, Terminal knows closing the window would kill the shell — and killing the shell would take the attached child process down with it — so it warns you first.
• Why restart-ai-stack.command and Restart-Agent-Server.command don't warn: both are launched as double-clicked .command scripts that run nohup ... & disown-style background starts and then the launcher script itself finishes and its shell process exits (that's what "You may close this window — the server keeps running in the background" means in Restart-Agent-Server.command's own output). Once the parent shell that spawned the background process has already exited on its own, the child process gets reparented to launchd (macOS's process 1) and is no longer attached to any Terminal window's shell at all — so there's nothing left for Terminal to warn you about, even though gateway.py or agent-server is still very much alive in the background.
• What to do about your current window: the warning is accurate, not a bug — closing that window right now genuinely would kill mlx_lm.server on :8082. Either leave the window open/minimized, or relaunch it the way the other scripts do (nohup + disown, or just use the absolute-path launch line from Step 3 inside a proper .command script) so it survives the window closing. If you close it and accept the warning, just remember the fast tier (:8082) will need restarting afterward.

What "gate changes" means here, and why this specific script:

• What: any edit to the heavy-warning-gate code inside gateway.py itself — the HEAVY_MODELS set, the HEAVY_WARNING message text, or the if model in HEAVY_MODELS... check added in this step. Also covers any future edit to that same file (allow-list, rate limit, auth logic, etc.).
• How: edit gateway.py with nano (never paste Python at the shell prompt — see the warning earlier in this step), save, then run this script to pick up the change.
• Where: /Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/"Restart AI Local Server.command" — same folder as gateway.py, gateway.env, and manage_keys.py.
• Why this script and not the other two: Python only reads a .py file's code once, at process startup — editing the file on disk has zero effect on a process already running with the old code loaded in memory. The process has to be killed and relaunched to pick up the new code. This script does exactly that, and only that: stop gateway.py, start gateway.py. restart-ai-stack.command would also bounce Ollama and Open WebUI unnecessarily; Restart-Agent-Server.command wouldn't touch gateway.py at all, so a gate edit would silently never take effect.

Environment: regular shell, NOT the mlx venv. Verified on disk — _gateway_lib.sh line 48 launches it with plain nohup python3 gateway.py, and the file's own header says it's dependency-free (stdlib only, no pip packages). mlxenv is irrelevant here; don't activate it for this step. To edit the file, use nano (or your preferred editor) directly, then restart with "Restart AI Local Server.command" (double-click in Finder) or by re-running _gateway_lib.sh's restart function from Terminal.

Summary: heavy-tier requests now require explicit consent — a request for gpt-oss:120b / Qwen3.5-122B without the X-Confirm-Heavy: yes header gets an HTTP 428 warning instead of silently evicting every resident model. Impact: no accidental 66 GB loads; other users/jobs are never interrupted without someone deliberately opting in. Why: on 96 GB the heavy tier is exclusive-mode — the warning is the safety interlock for the whole stack.

⚠ Verify it's live — the test below needs a REAL app key, not the literal placeholder text <your-app-key>. A 401 means the gateway's own auth layer rejected the placeholder before ever reaching the heavy-model logic — it is NOT a verdict on the gate either way. You already have two keys issued (llmpowerhouse-site, project2-dashboard); either mint a fresh one just for testing, or reuse an existing one:

python3 /Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/manage_keys.py add gate-test

This prints a sk-... key once — copy it, then substitute it for <your-app-key> below (real command, run in a regular shell, no venv):

curl -s -o /dev/null -w "%{http_code}\n" -X POST localhost:8787/v1/chat/completions -H 'Authorization: Bearer sk-PASTE-YOUR-REAL-KEY-HERE' -H 'Content-Type: application/json' -d '{"model":"gpt-oss:120b","messages":[{"role":"user","content":"hi"}]}'

Now the result means something: 428 = gate is live and working. Anything else (a normal chat answer, or another error) = the gate code below still needs to be added to gateway.py.

⚠⚠⚠ THE PYTHON BLOCK BELOW IS FILE CONTENT — NEVER TYPE OR PASTE IT AT THE TERMINAL PROMPT. It goes INSIDE gateway.py, edited with a text editor. Pasting Python into zsh directly (as happened once already) causes a harmless parse error near '}' — nothing breaks, but nothing gets installed either. Correct procedure:

1. nano /Volumes/AI_DATA/apps/llmpowerhouse-site/local-server/gateway.py
2. Find the spot right after the request handler extracts model from the request body.
3. Type/paste the block below AT THAT LOCATION INSIDE THE FILE, matching the surrounding code's indentation.
4. Save (Ctrl+O, Enter) and exit (Ctrl+X) — that's nano, not a shell.
5. Restart the gateway: double-click "Restart AI Local Server.command" in Finder, or run _gateway_lib.sh's restart function from Terminal.
6. Only THEN re-run the curl test above.

Reference implementation (as designed — keep for rebuilds). It goes right after model extraction in gateway.py:

HEAVY_MODELS = {"gpt-oss:120b", "qwen3.5-122b", "heavy"}
HEAVY_WARNING = (
    "HEAVY MODEL WARNING: this request loads a ~64-66GB model in EXCLUSIVE "
    "mode on a 96GB machine. All other models unload; other users/jobs are "
    "interrupted; context is capped at 32K; load takes 1-3 min. "
    "Re-send with header 'X-Confirm-Heavy: yes' to proceed."
)

# inside the request handler, after `model` is known:
if model in HEAVY_MODELS and headers.get("X-Confirm-Heavy", "").lower() != "yes":
    return json_response(428, {"error": {
        "code": "heavy_model_confirmation_required",
        "message": HEAVY_WARNING}})

With Ollama upstream, OLLAMA_MAX_LOADED_MODELS=2 + a 64GB model already forces the unload for you — the gate's job is purely the warning/consent. If the heavy model moves to mlx-lm later, add the unload/reload subprocess calls from Appendix B.

Flow you'll see:

you → POST :8787/v1/chat/completions  {"model":"gpt-oss:120b", ...}
gateway → HTTP 428  ⚠ HEAVY MODEL WARNING ... X-Confirm-Heavy: yes
you → same request + X-Confirm-Heavy: yes
gateway → forwards to Ollama → Ollama evicts residents → answers

Every download and every server-side piece (gate, wired limit, MLX runtime) is done. #1 and #2 are confirmed complete (real 428 block, then real gpt-oss:120b answer with the confirm header — see both items below). What's left is #3, #4, #5, #6 — do these next, at your own pace, none blocking. None of these six need mlxenv except #4's MLX half — see each item.

#1 — Gate fires (expect HTTP 428). Environment: regular shell, no venv. ✅ DONE (Aug 2 evening) — confirmed with a real key, returned 428.

curl -s -o /dev/null -w "%{http_code}\n" -X POST localhost:8787/v1/chat/completions -H 'Authorization: Bearer <your-app-key>' -H 'Content-Type: application/json' -d '{"model":"gpt-oss:120b","messages":[{"role":"user","content":"hi"}]}'

#2 — Gate passes with the confirm header (expect a slow real answer, not 428). Regular shell, no venv. ✅ DONE (Aug 2 evening) — confirmed with a real key + X-Confirm-Heavy: yes, got a genuine gpt-oss:120b completion back ("Hello! How's your day going?...", 118 total tokens, finish_reason: stop) — not a 428, not an error. Same command as #1 plus -H 'X-Confirm-Heavy: yes'. Both halves of the gate — block and override — are now proven with live tests, not just code review. Step 4 is fully closed out.

#3 — Coder-Next long-prompt bug check. Not a terminal command at all — do this in whatever chat client talks to Ollama (Open WebUI, or an API call): paste a 20K+ token source file and ask for a refactor. If output degrades into garbage after ~1K generated tokens, keep using qwen3.6:35b-a3b for coding and re-test Coder-Next after the next mlx-lm/Ollama update.

#4 — 122B vs gpt-oss:120b bake-off. Run your knowledge-bank test-questions.md against both and compare. gpt-oss:120b is queried through Ollama/Open WebUI (no venv). Qwen3.5-122B needs mlxenv first if you serve it via mlx_lm.server (see Appendix B) — otherwise same chat-based test as #3. Winner keeps the escalation slot; run ollama rm gpt-oss:120b or delete the 122B's hf-cache folder for the loser, once you're sure.

#5 — Vision parity test. Chat-based, no venv. Give qwen3.6:35b-a3b the same images you currently send qwen3-vl:30b. If answers are equivalent:

ollama rm qwen3-vl:30b

(frees ~19GB — allowed under your rule since it's a same-slot, proven-equal replacement.)

#6 — Memory headroom check while a heavy model is loaded. Regular shell, no venv.

sudo memory_pressure

Run once with the 122B (or gpt-oss:120b) active; confirm it stays out of the red/critical zone.

Recommended order: #1 → #2 (prove the gate works, 2 minutes) → #6 while #2's heavy model is still loaded (free 2-for-1) → #3 (coding bug check) → #5 (vision parity, frees disk) → #4 last (the bake-off takes the longest since it's a real side-by-side eval).

• gpt-oss:120b — KEEP until Qwen3.5-122B wins the §4-Step-5 bake-off on your own test questions. Same slot (heavy/exclusive), so replacement is allowed by your rule — but it's your call after seeing results. Disk allows keeping both.
• qwen3-vl:30b — retirement candidate: Qwen3.6-35B-A3B has native vision. Test first (Step 5.5). Same-slot upgrade → delete allowed if parity.
• nomic-embed-text — KEEP. Swapping the embedding model forces a FULL re-index of every knowledge bank in Open WebUI (DOD-FM + K-12, 300+ docs). Qwen3-Embedding is better, but do it as a planned migration, not a casual swap.
• qwen3.6:35b-a3b (GGUF) — KEEP even if you add the MLX copy; it's the hub your 19 projects and Open WebUI point at.

1. Audited every step against /Volumes/AI_DATA — 9 items already done, marked ✅.
2. Heavy gate redesigned to bolt into your EXISTING gateway.py (:8787, with your API-key auth) instead of a new standalone gateway — one front door, not two.
3. Discovered gpt-oss:120b already fills the heavy slot → 122B is now a bake-off replacement, not a net-new install.
4. Whisper + embedding steps removed from TODO (already on disk).
5. HF_HOME pinned to /Volumes/AI_DATA/models/hf-cache (dir you already created) so MLX models follow your everything-on-AI_DATA portability rule.
6. Added §6 keep/retire decisions honoring your no-deletion rule.
7. Fast tier gains an Ollama-only fallback option (skip MLX until benchmarked).

One-paste block: creates the plist as root, validates, loads, verifies.

sudo tee /Library/LaunchDaemons/com.llmpowerhouse.wiredlimit.plist >/dev/null <<'EOF'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
 "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
  <key>Label</key><string>com.llmpowerhouse.wiredlimit</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/sbin/sysctl</string>
    <string>iogpu.wired_limit_mb=90112</string>
  </array>
  <key>RunAtLoad</key><true/>
</dict></plist>
EOF
sudo chmod 644 /Library/LaunchDaemons/com.llmpowerhouse.wiredlimit.plist
plutil -lint /Library/LaunchDaemons/com.llmpowerhouse.wiredlimit.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/com.llmpowerhouse.wiredlimit.plist
sysctl iogpu.wired_limit_mb

Expected: OK from plutil, 90112 from the final line. "Bootstrap failed: 5" while the sysctl still prints 90112 = job already registered, ignore.

import subprocess
def unload_all():
    subprocess.run(["pkill", "-f", "mlx_lm.server"])
def load_heavy():
    subprocess.Popen(["mlx_lm.server", "--model",
        "mlx-community/Qwen3.5-122B-A10B-mxfp4", "--port", "8083",
        "--max-tokens", "8192"])
# restore: relaunch resident mlx_lm.server processes / rely on Ollama keep-alive

• Heavy exclusive mode = service outage for other models during load + job + restore.
• mlx-lm bug #844/#856 (Coder-Next long-prompt corruption) still open — GGUF workaround.
• HF repo IDs verified from published sources Aug 2, 2026 — confirm each resolves.
• NEW: SYSTEM_OVERVIEW notes your public-demo gateway "has never run" (Jul 25 note; later docs may supersede) — bring it up per your guide Phase 4½.4 BEFORE adding the heavy gate, so you're changing a known-working file.