Instructions to use craterlabs/Struct-SQL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use craterlabs/Struct-SQL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="craterlabs/Struct-SQL")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("craterlabs/Struct-SQL")
model = AutoModelForMultimodalLM.from_pretrained("craterlabs/Struct-SQL")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use craterlabs/Struct-SQL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "craterlabs/Struct-SQL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "craterlabs/Struct-SQL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/craterlabs/Struct-SQL

SGLang

How to use craterlabs/Struct-SQL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "craterlabs/Struct-SQL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "craterlabs/Struct-SQL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "craterlabs/Struct-SQL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "craterlabs/Struct-SQL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use craterlabs/Struct-SQL with Docker Model Runner:
```
docker model run hf.co/craterlabs/Struct-SQL
```

KhushbooThaker commited on Jan 15

Commit

436300e

verified ·

1 Parent(s): dec4134

Update README.md

Browse files

Files changed (1) hide show

README.md +6 -4

README.md CHANGED Viewed

@@ -25,6 +25,8 @@ Unlike standard distillation methods that rely on unstructured Chain-of-Thought
 📄 **Paper:** [Knowledge Distillation with Structured Chain-of-Thought for Text-to-SQL](https://arxiv.org/abs/2512.17053)
 ## Performance
 On the **BIRD mini-dev** benchmark, Struct-SQL achieves an **Execution Accuracy (EX) of 45.0%**, outperforming standard unstructured CoT distillation baselines by **8.1 points**.
@@ -36,6 +38,7 @@ On the **BIRD mini-dev** benchmark, Struct-SQL achieves an **Execution Accuracy
 | FN-Gold Baseline | No Reasoning (SQL Only) | 34.3% |
 | Base Student (Zero-shot) | None | 17.0% |
 ## Methodology
 The model was trained on a curated dataset of **1,000 samples** generated by GPT-4o. The training data consists of:
@@ -44,10 +47,12 @@ The model was trained on a curated dataset of **1,000 samples** generated by GPT
 By forcing the model to explicitly plan the query execution (e.g., "Scan Table", "Filter by...", "Join with..."), the model learns the logical structure of SQL generation rather than just memorizing patterns.
 ## Usage
 You can use this model with the `transformers` library. It expects the input to be formatted with a specific system prompt or structure if you want to elicit the query plan.
 ```python
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -65,8 +70,7 @@ inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
 outputs = model.generate(**inputs, max_new_tokens=1200)
 print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
 ## Intended Use
 Struct-SQL-4B is intended for **research and academic use** in tasks involving **Text-to-SQL generation** and **semantic parsing over relational databases**. The model is particularly suited for studying:
@@ -79,14 +83,12 @@ Struct-SQL-4B is intended for **research and academic use** in tasks involving *
 The model is not optimized for direct deployment in production database systems without additional validation and safety constraints.
 ---
 ## Limitations
 - Evaluation is confined to the SQLite-based BIRD benchmark
 - The model may generate logically plausible but incorrect SQL for highly complex multi-hop queries
 ---
 ## Citation
 ```bibtex

 📄 **Paper:** [Knowledge Distillation with Structured Chain-of-Thought for Text-to-SQL](https://arxiv.org/abs/2512.17053)
+---
 ## Performance
 On the **BIRD mini-dev** benchmark, Struct-SQL achieves an **Execution Accuracy (EX) of 45.0%**, outperforming standard unstructured CoT distillation baselines by **8.1 points**.
 | FN-Gold Baseline | No Reasoning (SQL Only) | 34.3% |
 | Base Student (Zero-shot) | None | 17.0% |
+---
 ## Methodology
 The model was trained on a curated dataset of **1,000 samples** generated by GPT-4o. The training data consists of:
 By forcing the model to explicitly plan the query execution (e.g., "Scan Table", "Filter by...", "Join with..."), the model learns the logical structure of SQL generation rather than just memorizing patterns.
+---
 ## Usage
 You can use this model with the `transformers` library. It expects the input to be formatted with a specific system prompt or structure if you want to elicit the query plan.
+---
 ```python
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
 outputs = model.generate(**inputs, max_new_tokens=1200)
 print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
+---
 ## Intended Use
 Struct-SQL-4B is intended for **research and academic use** in tasks involving **Text-to-SQL generation** and **semantic parsing over relational databases**. The model is particularly suited for studying:
 The model is not optimized for direct deployment in production database systems without additional validation and safety constraints.
 ---
 ## Limitations
 - Evaluation is confined to the SQLite-based BIRD benchmark
 - The model may generate logically plausible but incorrect SQL for highly complex multi-hop queries
 ---
 ## Citation
 ```bibtex