irotem98/code-pruner-turbo

Code Pruner Turbo

Task-aware code pruning with a layer-18 Qwen3 reranker backbone and a native vLLM serving path.

GitHub source and training code · Original SWE-Pruner

Quickstart

pip install \
  "code-pruner-turbo[vllm] @ https://huggingface.co/irotem98/code-pruner-turbo/resolve/main/package/code_pruner_turbo-0.2.0-py3-none-any.whl"
code-pruner-turbo serve

curl http://localhost:8000/prune \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Where is authentication checked?",
    "code": "def authenticate(user):\n    return user.is_active",
    "threshold": 0.5
  }'

Results

Serving performance

Model	Backend	Concurrency 1 ↑	Concurrency 16 ↑
code-pruner-turbo	vLLM	70.3 req/s	133.0 req/s
code-pruner-turbo	Hugging Face	17.4 req/s	18.1 req/s
code-pruner	Hugging Face	9.83 req/s	10.03 req/s

Validation quality

Model	Loss ↓	Compression loss ↓	Accuracy ↑	Precision ↑	Recall ↑	F1 ↑
code-pruner-turbo	0.2302	0.2418	84.75%	81.41%	80.85%	81.13%
code-pruner	0.3088	0.3250	84.07%	80.02%	80.91%	80.46%

Serving was measured end-to-end on one NVIDIA RTX PRO 6000 Blackwell Server Edition. Validation used the same 6,119-example split for both models.

Architecture

Original Code-Pruner                         Code-Pruner Turbo

Qwen3 layers 1–28 (frozen)                  Qwen3 layers 1–18 (frozen)
          │                                           │
 hidden layers 7 + 14 + 28                   normalized hidden layer 18
 3 × 1024 → concatenate 3072                 1024 → gated SiLU → 2432
          │                                           │
 1× bidirectional full attention             1× bidirectional full attention
 8 heads, width 3072                         16 heads, width 2432
          │                                           │
 CRF emissions: 3072 → 256 → 2               CRF emissions: 2432 → 128 → 2

The original model executes all 28 decoder blocks and retains three intermediate representations. Turbo exits after block 18 and needs only its final representation, making native vLLM execution possible while reducing the trainable head from 38.6M to 29.0M parameters.

The serving path uses the two trained CRF emissions exactly as the evaluation code does. It does not apply Viterbi decoding.

Requirements

• Linux and an NVIDIA CUDA GPU
• Python 3.12
• vLLM 0.13.0

Prefix caching is disabled because the bidirectional fusion head requires the complete prompt representation. Inputs longer than 8,192 tokens are split with overlap and merged automatically.

Attribution

This model builds on SWE-Pruner and Qwen3-Reranker-0.6B.

@misc{wang2026sweprunerselfadaptivecontextpruning,
  title={SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents},
  author={Yuhang Wang and Yuling Shi and Mo Yang and Rongrui Zhang and
          Shilin He and Heng Lian and Yuting Chen and Siyu Ye and Kai Cai and
          Xiaodong Gu},
  year={2026},
  eprint={2601.16746},
  archivePrefix={arXiv},
  primaryClass={cs.SE}
}