BACKLOG.md

Backlog — Composer 2.5 Replication Framework

Updated 2026-05-29 to reflect shipped waves (ingestion, diloco, packaging, datagen+RL, ADR-011/012/013, cross-family review).

Active items / Honest Gaps

Framework/Docker substrate E2E (Hardware-blocked)

• We lack the local multi-node GPU environment to run the true 8-node DiLoCo + Docker/TorchForge orchestrator E2E tests. Currently isolated to unit-level and single-node pseudo-gradient checks.

Real 8B LMA run (User-budget-gated)

• The framework is proven on Qwen-0.5B and 1.5B (GSM8K/SDPO math traces).
• The ultimate goal (Llama-3-8B full LMA run with α/β ablation over 10k SWE-bench traces) requires a multi-GPU Modal drop + significant compute budget.

Modal-gated (if budget allows after gap-closers)

Spike 002a-mini — Real GPU smoke (Phase 10)

Closes: the "did we ever run gradients on GPU" ambiguity — currently everything is CPU-only.

• Goal: dispatch a 30-min A10G smoke on Modal that runs Qwen2.5-0.5B-Instruct natively on GPU.

Shipped (Past-Skeleton)

Spike 006 — Real HF model smoke (Wave 7)

Closes: V8 ("any HF model") — currently we run only mock 4-layer toy LM through composer_total_loss.

Goal: prove the 3-channel loss (grpo + α·sdpo_kl + β·trace_replay_dpo) survives a real transformers model + tokenizer with finite gradients and a decreasing loss across N steps.

Acceptance:

1. AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct") loads on CPU.
2. Real tokenizer apply_chat_template produces input_ids shape that flows through composer_total_loss(model, batch) without mock shapes.
3. 5 backward steps run on CPU without nan / inf / shape mismatch.
4. Loss is monotone non-increasing across 5 steps (trend; allow noise).
5. New tests added under spikes/006-real-hf-model-smoke/tests/ pass alongside existing 38.

Estimate: half a day, CPU only.

Spike 007 — Real trace ingestion (Wave 8)

Closes: V5 ("real LLM-application traces") — Spike 001 used 50 hand-crafted states. Brief said "real traces."

Goal: pick ONE real agent-session log format with stable, public schema, write a TraceIngester that converts it to our TraceExample dataclass, run end-to-end through the data collator + a trimmed cost-floor measurement on 5 real states.

Acceptance:

1. ADR-002 picks the trace source (Claude Code JSONL / Cline / OpenHands / Aider / SWE-Bench-Lite trajectories).
2. TraceIngester.ingest(path: Path) -> Iterator[TraceExample] is implemented + has unit tests with a fixture log file.
3. End-to-end smoke: real trace → ingester → collator → 1-step composer_total_loss runs without error.
4. Cost-floor measurement: 5 real states × 3 teachers, p95 latency + cost report appended to spikes/007-*/verdict.md.

Estimate: 1 day + ~$2 OpenRouter.

Spike 008 — Streaming DiLoCo smoke (Wave 9)

Closes: V2 (DiLoCo "deferred to v0.2" — drift from original brief).

Goal: bolt outer-loop pseudo-gradient sync onto the loss composition test using two nn.Module replicas on the same node. No real distributed training (CPU multiprocessing or single-process).

Acceptance:

1. ADR-003 picks the DiLoCo variant (vanilla DiLoCo from arXiv:2311.08105 / Streaming DiLoCo from PrimeIntellect / Async-DiLoCo).
2. outer_optimizer.py implements pseudo-gradient = (θ_local − θ_initial), Nesterov-momentum outer step.
3. Smoke test: 2 replicas × 4 inner steps × 2 outer rounds on the toy model from Spike 005, both replicas converge toward the same solution within tolerance.
4. 38 existing tests still pass (no regression).

Estimate: 2 days, CPU.

Wave 10 — Packaging

Closes: V4 ("skeleton not framework").

Goal: turn the assemblage of spike directories into an installable Python package with a clear quickstart.

Acceptance:

1. pyproject.toml at repo root, package name composer_replication.
2. composer_replication/ dir with __init__.py re-exporting composer_total_loss, OPSDLoss, TeacherReplayBuffer, compose_loss, TraceIngester, etc.
3. examples/qwen_05b_quickstart/ with end-to-end script that loads model, runs 10 training steps, prints loss curve.
4. README quickstart updated to pip install -e . + python examples/qwen_05b_quickstart/run.py.
5. pip install -e . succeeds and quickstart runs end-to-end on CPU.

Post-Skeleton Waves (Datagen, Alignment, Quality)

• Trace Ingestion: Shipped (composer_replication/ingestion/).
• DiLoCo: Shipped (composer_replication/diloco/ outer-loop pseudo optimizer).
• Packaging: Shipped (pip install -e . works perfectly).
• ADR-008/009/010 (Datagen, Layered Hints, Dr.GRPO+SDPO): Shipped, examples documented.
• Cross-Family Architectural Review: Shipped (docs/reviews/cross-family-adr-008-009-010-2026-05-29/).
• Alignment / V&V Closure: ADR-011 (SDPO alignment indices), ADR-012 (close review findings), ADR-013 (LMA integration channel-ladder) shipped.
• Test Suites: 266 passed / 62 skipped (measured 2026-06-09; canonical count + env-variance note in docs/V1_V8_COVERAGE.md).
• Real Examples: examples/gsm8k_grpo/, examples/sdpo_with_real_traces_production/.

Deferred (post-loop, GPU-gated)

• Spike 002a/002b — full trace collection on A100 ($30–50)
• Spike 003 — DPO-pair signal density study
• Spike 004 — A/B SWE-bench-lite with α=0/β=0 vs α>0/β>0
• Publication wave — author identity, thumbnail, X tags, post sequence

Process notes

• Acceptance criteria are explicit and binary. Don't claim "done" unless every box ticks.
• Each spike has its own spikes/00N-name/ dir + verdict.md recording acceptance + delta from estimate.
• Re-audit BACKLOG.md at end of each wave; archive completed items with their final SHAs.