YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Rubrics-RL: Rubric-Guided Reinforcement Learning for Medical Report Generation
This repository contains the code, training configurations, evaluation outputs, and processed data for the Rubrics-RL project (ICLR 2027 submission). The project investigates rubric-guided reinforcement learning for medical report generation, focusing on the "scissors gap" between LLM-judge rewards and verifiable faithfulness metrics.
Repository Structure
βββ code/ # All training and evaluation scripts
β βββ audit/ # M1 judge vulnerability audit
β βββ m2/ # M2-M4 closed-form VQA experiments
β βββ m4/ # M4 counterfactual faithfulness
β βββ m5/ # M5-M19 open-ended report generation
β βββ report_reward.py # M5 reward (5 conditions)
β βββ report_reward_m6.py # M6 routed gate reward
β βββ report_reward_m9.py # M9 RadGraph dual-gate reward
β βββ chexbert_server.py # CheXbert lie-detector service
β βββ radgraph_server.py # RadGraph lie-detector service
β βββ rubric_gen.py # RubricHub rubric generation
β βββ atom_router.py # Atom routing for gated reward
β βββ train_m5.sh # M5 training script
β βββ train_m6.sh # M6 training script
β βββ train_m9.sh # M9 training script
β βββ plot_m5.py # M5 scissors gap plot
β βββ plot_m6.py # M6 routed gate plot
β βββ plot_m9.py # M9 RadGraph gate plot
β βββ m11_calibration.py # M11 clinical calibration
β βββ m11_adversarial_probe.py # M11 adversarial CheXbert probe
β βββ m13a_overcredit.py # M13 over-credit analysis
β βββ ...
βββ data/ # Project-specific processed data
β βββ m2_vqarad/ # Processed VQA-RAD for M2 training
β βββ m5_iuxray/ # Processed IU-Xray for M5 training
β βββ new_datasets/
β βββ CRIMSON/ # RadJudge/RadPref calibration data
β βββ TAIX-VQA/ # ICU CXR faithfulness eval
βββ outputs/ # Evaluation results, plots, caches
β βββ *.json # Evaluation result JSONs
β βββ *.png # Visualization plots
β βββ *.csv # Per-sample results
β βββ *_cache/ # Judge inference caches
β βββ ...
βββ logs/ # Training logs
βββ env_backup/ # Conda environment specifications
βββ README.md # This file
Publicly Available Datasets (not included)
The following datasets used in this project are publicly available. Download them from their respective sources:
| Dataset | Access | Source | Size | Usage |
|---|---|---|---|---|
| VQA-RAD | OPEN | HuggingFace: flaviagiammarino/vqa-rad |
34 MB | M1-M4 VQA warm-up |
| SLAKE | OPEN | HuggingFace: mdwiratathya/SLAKE-vqa-english (or BoKelvin/SLAKE) |
<1 GB | VQA warm-up + spatial |
| IU-Xray | OPEN | Open-i / HuggingFace: dz-osamu/IU-Xray |
~14 GB | M5-M19 main dataset |
| TAIX-VQA | OPEN | HuggingFace: jomoll/TAIX-VQA (images: TLAIM/TAIX-Ray) |
small | ICU CXR faithfulness |
| CRIMSON | OPEN | GitHub: rajpurkarlab/CRIMSON |
small | RadJudge/RadPref calibration |
| ReXGradient-160K | HF-GATED | HuggingFace: rajpurkarlab/ReXGradient-160K (accept license first) |
~160 GB | Scale-up training |
| MIMIC-CXR | PHYSIONET | PhysioNet: mimic-cxr / mimic-cxr-jpg |
~570 GB | (Future scale-up) |
Download example:
# Open datasets
pip install huggingface_hub
huggingface-cli download flaviagiammarino/vqa-rad --local-dir data/vqa-rad
huggingface-cli download mdwiratathya/SLAKE-vqa-english --local-dir data/SLAKE
# IU-Xray (or use the HF mirror)
huggingface-cli download dz-osamu/IU-Xray --local-dir data/IU-Xray-RRG
# Gated dataset (accept license on HF page first)
huggingface-cli download rajpurkarlab/ReXGradient-160K --local-dir data/ReXGradient-160K --token YOUR_HF_TOKEN
Base Models (not included)
All base models are publicly available on HuggingFace:
| Model | HuggingFace ID | Size |
|---|---|---|
| Qwen2.5-VL-3B-Instruct | Qwen/Qwen2.5-VL-3B-Instruct |
7.1 GB |
| Qwen2.5-VL-7B-Instruct | Qwen/Qwen2.5-VL-7B-Instruct |
16 GB |
| Qwen3-VL-4B-Instruct | Qwen/Qwen3-VL-4B-Instruct |
8.3 GB |
| HuatuoGPT-Vision-7B | FreedomIntelligence/HuatuoGPT-Vision-7B |
17 GB |
Training Framework
This project uses EasyR1 (commit dd71bbd) with one compatibility patch:
# verl/workers/fsdp_workers.py β transformers>=5 compatibility
-from transformers.modeling_utils import no_init_weights
+try:
+ from transformers.modeling_utils import no_init_weights
+except ImportError:
+ from transformers.initialization import no_init_weights
External Tools
- GREEN (Grounded Radiology Report Evaluation): https://github.com/Stanford-AIMI/GREEN
- CheXbert: Used as a verifiable lie-detector (14-label chest finding classifier)
- RadGraph: Entity-relation extraction for fine-grained faithfulness verification
Experiment Overview
| Milestone | Description | Key Finding |
|---|---|---|
| M1 | Judge vulnerability audit | LLM judges exploitable via master-key attacks |
| M2-M4 | Closed-form VQA RL | Strong judge resists short-horizon hacking; gate prototype works |
| M5 | Open-ended report RL (5 conditions) | Scissors gap confirmed (judge ~0.5 vs CheXbert-F1 ~0.12) |
| M6 | Routed per-atom gate | Gate preserves faithfulness; high verif_rate makes gate invisible |
| M7 | Master-key attack (report-level) | Does not reproduce (honest negative result) |
| M8 | 2nd-seed replication | M6 conclusions seed-stable |
| M9 | RadGraph dual-gate | Gate becomes visible β finer signal exposes cross-stream gate effect |
| M11 | Clinical calibration | CheXbert-F1 vs radiologist Ο=0.63; dualgate fixes single-stream blindspots |
| M12-M19 | Extended experiments | Confound controls, long training, emergent downstream harm |
Environment Setup
Three conda environments were used:
easyr1: Training (GRPO + EasyR1)r1: Evaluation, lie-detector services (requires nvJitLink fix)chx: CheXbert CPU inference
Environment specs are in env_backup/.
Gated Access
This repository requires manual approval for access. Please provide your affiliation and intended use when requesting access.
License
Research use only. Please contact the authors for commercial licensing.
Inference Providers NEW
This model isn't deployed by any Inference Provider. π Ask for provider support