itzune/morpheus-gguf

Morpheus (Mamba-2) — Basque FIM Autocomplete (GGUF)

GGUF quantized version of Morpheus v3, a 91M-parameter Mamba-2 language model for Basque (Euskara) text autocompletion with Fill-in-the-Middle (FIM) support — the model can complete text at the cursor, not just at the end.

This is a continued-pretraining of the AR-only v2 model (step 74K) on FIM-structured data.

What's new in v3 (FIM)

Capability	v2 (AR-only)	v3 (FIM)
Next-word / end-of-line completion	✅	✅
Fill-in-the-Middle (cursor mid-text)	❌	✅
Special tokens	4000	4004 (<PRE> <SUF> <MID> <EOT>)
Vocab (padded)	4016	4016

FIM training recipe (Phase 6 v3)

• Base: v2 best.pt (AR checkpoint, step 3500 of Phase 6 v2)
• Data: 500M tokens, 70/30 FIM/AR ratio (BigCode-style token-level splitting)
• EOT loss weighting: 5× on the <EOT> stop token — forces the model to learn when to stop generating infill, not just what to generate
• Packed dataset: Greedy packing of whole examples into 1025-token windows (no cross-example contamination)
• LR: 1.0e-3, cosine decay, ~3,815 steps
• Tokenizer: basque_unigram_fim.model (original 4000 pieces + 4 FIM tokens)

The 70/30 ratio + 5× EOT weight was selected after a senior ML engineer review of the v2 failure mode (unreliable <EOT> emission → over-generation). See Option D in the analysis below.

Available Files

File	Quant	Size	Use
v3_fim.Q5_K_M.gguf	Q5_K_M	64 MB	Recommended — FIM + AR
morpheus-v2-mamba.Q4_K_M.gguf	Q4_K_M	53 MB	AR-only (legacy v2)
morpheus-v2-mamba.Q5_K_M.gguf	Q5_K_M	64 MB	AR-only (legacy v2)

FIM Evaluation Results

Trained-for-behavior eval on 147 held-out FIM examples (token-level splits, 20% at linguistic boundaries):

Metric	v2 (50/50, no EOT weight)	v3 (70/30, 5× EOT weight)	Change
EOT emission rate	76.9%	88.4%	+11.5pp
Keystrokes saved	-25.2%	-5.9%	4× better
Exact match	2.7%	6.8%	2.5×
Char accuracy	31.2%	32.3%	+1.1pp
Gen length (ref=45.0)	56.8 (over-gen)	40.3	closer to target
Prefix truncation	0.0%	1.4%	minimal
└ long-bucket truncation	0.0%	2.3%	well under 15% threshold

Key findings:

• The 5× EOT loss weighting worked as designed: EOT reliability up 77% → 88%, and the model stopped over-generating (56.8 → 40.3 chars vs 45.0 target).
• The feared failure mode — premature truncation from over-weighting EOT — did not materialize (1.4% overall, 2.3% in the long bucket; failure threshold was 15%).
• AR perplexity is stable (7.4 → 7.5) — "FIM-for-free" property holds even with 70% FIM data.

Model Details

• Architecture: Mamba-2 (State Space Model), 24 layers, d_model 768
• Parameters: 91M
• Checkpoint: Phase 6 v3 best.pt (step 3500)
• AR perplexity: 7.5 | FIM perplexity: 7.9
• Tokenizer: basque_unigram_fim.model — 4004 pieces (original 4000 + <PRE> <SUF> <MID> <EOT>)
• Trained without BOS token (add_bos_token=false)

FIM Token Format

The model uses Code Llama-style FIM tokens. To do a fill-in-the-middle completion, structure the prompt as:

<PRE>{prefix}<SUF>{suffix}<MID>

The model generates the infill and emits <EOT> when done.

Token	ID	Purpose
<PRE>	4000	Marks start of prefix
<SUF>	4001	Marks start of suffix
<MID>	4002	Marks start of generation (infill)
<EOT>	4003	End-of-infill (stop token)
4004–4015	—	Padding (unused)

Usage

With the Morpheus demo server (recommended)

The Morpheus demo includes a FastAPI proxy that handles FIM templating, token-ID encoding, and an OpenAI-compatible API:

cd demo
docker compose -f docker-compose.yml -f docker-compose.local.yml up -d --build
# Open http://localhost:9090/editor.html

Set MORPHEUS_MODEL=v3_fim.Q5_K_M.gguf to use this model.

Direct llama-server (raw FIM)

llama-server -m v3_fim.Q5_K_M.gguf --host 127.0.0.1 --port 8080 -ngl 0 --threads 8

# FIM completion:
curl http://localhost:8080/completion \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "<PRE>Kaixo, <SUF> moduz?<MID>",
    "n_predict": 20,
    "temperature": 0.2,
    "top_k": 5,
    "stop": ["<EOT>"]
  }'

⚠️ Critical: token-ID prompts, not strings

As with v2, deploy via token-ID prompts (encode with sentencepiece using basque_unigram_fim.model, send IDs to /completion). The proxy handles this automatically. See the v2 model card for the full explanation.

Decoding parameters (recommended)

These are wired into the demo proxy and editor:

Parameter	Value	Rationale
temperature	0.2	Low-but-nonzero: recovers rank-2 correct tokens that greedy misses
top_k	5	Small nucleus; P7 finding: 5 correct answers sit at rank 2 in top-5
stop (FIM)	["<EOT>", "\n\n"]	Model emits <EOT>; fallback to paragraph boundary
repeat_penalty (FIM)	1.0	FIM legitimately reuses context words — do not penalize

Intended Use

Desktop text-editor ghost-text autocompletion for Basque prose. The Mamba-2 architecture's O(1) decode cost makes it well-suited to long editing sessions where latency per token matters more than parallelism.

Not intended for: instruction following, chat, translation, or factual QA. This is a narrow autocomplete model.

License

Apache-2.0.

Citation

Morpheus v3 — Mamba-2 FIM Autocomplete for Basque.
itzune/morpheus-gguf, 2026.