MediaStreamAI/MOTHER_EXO_T2V

⚠️ EARLY PREVIEW / PLACEHOLDER. This is an early DiT checkpoint (step 13154) — outputs are low quality. It exists to test MOTHER Inference and hold this repo; the full-quality release follows shortly. Do not benchmark this build.

MOTHER T2V — Sovereign Text-to-Video World Model

MOTHER T2V is a from-scratch, sovereign latent text-to-video generator: MOTHER's own 3D video VAE and flow-matching Diffusion Transformer, conditioned on the frozen MOTHER CORE-7B text backbone. No external generator weights (Sora / Wan / LTX / SVD) are used — the entire generative stack is MOTHER-trained. It is the generative half of the MOTHER EXO world model: where the worldmodel_latent head predicts future latents (dynamics), MOTHER T2V synthesises video from text.

Status of this card. It describes the release target of MOTHER T2V. Architecture and training configuration are final and factual. Benchmark results are populated only from the official VBench package at release — no score is ever hand-entered or estimated. Cells marked "pending" are filled in when measured.

---

Model Overview

Task	Text → short video (latent diffusion, flow-matching)
Total parameters	~0.85 B (≈118 M VAE + ≈734 M DiT)
Resolution / length	256×256, 16 frames (temporally compressed to 4 latent frames)
Conditioning	Frozen MOTHER CORE-7B text embeddings (3072-d), classifier-free guidance
Sovereignty	100% MOTHER-trained; external models are data-only teachers, never embedded
Posture	Observe-and-advise · human-in-the-loop · Strike = 0
Files	mother_t2v_vae256.pt (VAE) · mother_t2v_dit.pt (DiT)

---

Architecture

1. 3D Video VAE — VideoVAE (≈118 M)

• Convolutional KL-regularised 3D VAE; 8× spatial / 4× temporal compression to a 16-channel spatiotemporal latent, so diffusion runs in a tractable space.
• GroupNorm + SiLU residual Conv3d; gradient-checkpointing friendly; resolution-agnostic.
• Reconstruction training: L1 + light edge/gradient term + small KL. An optional perceptual (VGG, LPIPS-style) + PatchGAN adversarial fine-tune ("VAE-GAN") sharpens high-frequency detail for the release VAE, resumed from the L1 VAE (the L1 VAE is preserved, never overwritten).

2. Latent Diffusion Transformer — LatentDiT (≈734 M)

• Diffusion transformer operating in the frozen VAE latent space — patchified Conv3d, 24 blocks, hidden 1024, 16 heads, patch 2.
• Objective: flow-matching velocity prediction (t=0 data → t=1 noise), MSE on the velocity.
• Guidance: classifier-free guidance via text-dropout during training.
• Conditioning: cross-attention to frozen MOTHER CORE-7B text hidden states (3072-d).
• adaLN-zero time conditioning; axial-sinusoidal 3D positions → resolution-agnostic (trainable at 256p, samplable higher).
• latent_scale is calibrated to the frozen VAE's measured latent variance before DiT training, so the DiT trains on ~unit-variance latents.

3. Text encoder — frozen MOTHER CORE-7B

• The sovereign CORE-7B (48 layers, 3072-d) provides text conditioning. Frozen; not fine-tuned by T2V.

---

Training

• Data: real captioned video clips at 256 px, 16 frames (~13 k clips). Real data only for the generative fit; external models are teachers for captions, not embedded weights.
• Stages: (1) train the 3D VAE → freeze; (2) optional VAE-GAN sharpening → freeze; (3) train the latent DiT (flow-matching, CFG) on the frozen VAE latents.
• Checkpointing: save-best on the loss EMA, atomic writes, resume-safe.
• Hardware: single NVIDIA GB10 (Blackwell), bf16 where it helps (transformer), fp32 for the 3D-conv VAE.

Inference

python training/mother_t2v/latent/sample_latent.py \
  --prompt "a drone shot over a snowy mountain range at sunset" --steps 50 --guidance 5
# → montage PNG (+ mp4 if imageio present)

Sampling: CORE text → latent DiT (Euler ODE, t: 1→0, classifier-free guidance) → frozen VAE decode.

---

Evaluation — VBench (official text-to-video benchmark)

Clips are generated per official VBench prompt with training/leaderboards/vbench_eval.py, then scored by the official vbench package. Numbers below are measured, never fabricated.

Dimension	Score
Subject consistency	pending
Background consistency	pending
Motion smoothness	pending
Aesthetic quality	pending
Imaging quality	pending
Object class	pending
Spatial relationship	pending
Temporal flickering	pending
Overall	pending

Scale context (reported, not hidden): MOTHER T2V is a ~0.85 B generator trained from scratch on ~13 k clips — orders of magnitude smaller than the frontier systems that top VBench. Expect a credible, sovereign-eligible score, not a frontier one. The value is full-stack ownership and a transparently-measured entry, not beating Sora/Wan on raw fidelity.

---

Intended Use & Scope

• Intended: text-conditioned short-video synthesis for world-modelling, scenario/ISR preview, simulation, and research within the MOTHER EXO stack.
• Out of scope / prohibited: any autonomous-action, targeting, fire-control, or kill-chain use. MOTHER T2V is observation and decision-support only, under meaningful human control (Strike = 0).

Limitations

• ~0.85 B params, ~13 k training clips, 256 px, 16-frame clips → limited scene complexity, motion range, and fine detail vs. frontier T2V. Expect artifacts on out-of-distribution prompts.
• Text alignment is bounded by the frozen CORE-7B text representation (MOTHER-domain-specialised).
• Not a photorealistic or long-form video model; short clips at moderate resolution.

Safety & Sovereignty

• Sovereign: only MOTHER's own trained VAE + DiT run; CORE-7B is MOTHER's frozen backbone. No Sora/Wan/LTX/SVD weights are embedded; external models are data-only teachers, codecs are renderers.
• Posture: observe-and-advise, human-in-the-loop, Strike = 0. Consistent with the MOTHER EXO non-weapon posture.

Provenance

Built and evaluated on-node; benchmark numbers recorded only as measured by the official VBench package. Part of the MOTHER EXO sovereign world model (MediaStreamAI/MOTHER_EXO).