pankajpandey-dev (Pankaj Pandey)

reacted to their post with ❤️👀🚀🔥 about 7 hours ago

Post

34

🇮🇳 Gemma-3-1B Hindi Instruct — a Hindi LLM that runs fully offline, anywhere.
Last week I shipped Qwen3-4B Hindi. This week I went the other direction: how tiny can a useful Hindi model get? So I fine-tuned Gemma-3-1B on quality-filtered Hindi instruction data and shipped the full GGUF ladder.
✅ Fine-tune (16-bit): pankajpandey-dev/gemma-3-1b-hindi-instruct
✅ GGUF (Q4/Q5/Q8): pankajpandey-dev/gemma-3-1b-hindi-instruct-GGUF
Runs in Ollama, llama.cpp, and LM Studio. The Q4_K_M is just 806 MB — runs on CPU, a cheap laptop, even a Raspberry Pi.
What I tried this round: chrF-filtered the training data to drop weak translations, and used response-only loss so the model learns how to answer, not how to repeat prompts.
Honest note: at 1B, Hindi fluency is strong but coherence is bounded by size — it's a lightweight/edge experiment, not a 4B replacement. Gemma-3-4B Hindi is next.
Part of my Hindi LLM Series — openly-licensed Indic models for local & edge use. Feedback welcome 🙏
#Hindi #IndicNLP #GGUF #LocalLLM #Gemma #EdgeAI

posted an update about 7 hours ago

Post

34

🇮🇳 Gemma-3-1B Hindi Instruct — a Hindi LLM that runs fully offline, anywhere.
Last week I shipped Qwen3-4B Hindi. This week I went the other direction: how tiny can a useful Hindi model get? So I fine-tuned Gemma-3-1B on quality-filtered Hindi instruction data and shipped the full GGUF ladder.
✅ Fine-tune (16-bit): pankajpandey-dev/gemma-3-1b-hindi-instruct
✅ GGUF (Q4/Q5/Q8): pankajpandey-dev/gemma-3-1b-hindi-instruct-GGUF
Runs in Ollama, llama.cpp, and LM Studio. The Q4_K_M is just 806 MB — runs on CPU, a cheap laptop, even a Raspberry Pi.
What I tried this round: chrF-filtered the training data to drop weak translations, and used response-only loss so the model learns how to answer, not how to repeat prompts.
Honest note: at 1B, Hindi fluency is strong but coherence is bounded by size — it's a lightweight/edge experiment, not a 4B replacement. Gemma-3-4B Hindi is next.
Part of my Hindi LLM Series — openly-licensed Indic models for local & edge use. Feedback welcome 🙏
#Hindi #IndicNLP #GGUF #LocalLLM #Gemma #EdgeAI

replied to their post 5 days ago

@lulavc It's great to hear you've connected with others working on Hindi AI as well. The community around multilingual AI is growing, and collaboration is what will help us build better models and tools for everyone.

replied to their post 6 days ago

Good question! v2 is fine-tuned on Hindi instruction pairs, so there's no tool-calling data in the training set — the focus is Hindi instruction-following. That said, it's built on Qwen3-4B, which has native function-calling support, and since this is a LoRA fine-tune (base weights frozen) that capability should largely carry through. I haven't benchmarked tool calling specifically yet though, so I won't make hard claims. Tool calling is something I plan to evaluate in future iterations, and I'd be happy to hear feedback if anyone tests it.

reacted to their post with ❤️👀 7 days ago

Post

14927

🇮🇳 Qwen3-4B Hindi Instruct v2 — a Hindi LLM that runs on your own machine
Most strong Hindi-capable models are either huge or cloud-only. I wanted one that's small enough to run locally but actually follows instructions in Hindi — so I fine-tuned Qwen3-4B on 10K Hindi instruction pairs and shipped it with a full GGUF quant ladder.
✅ Fine-tune (16-bit): huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2
✅ GGUF (Q4/Q5/Q8): huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2-GGUF
Runs in Ollama, llama.cpp, and LM Studio. The Q4_K_M is just 2.5 GB — fits comfortably on a laptop, CPU or GPU.
Part of my Hindi LLM Series — building openly-licensed Indic models for local and edge use. More coming (Gemma next). Feedback welcome 🙏
#Hindi #IndicNLP #GGUF #LocalLLM #Qwen

4 replies

·

reacted to their post with 👀🔥 7 days ago

Post

272

Just released Qwen3-0.6B fine-tuned on Hindi instruction data 🇮🇳

✅ Full model: pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1
✅ GGUF versions (Q2/Q4/Q5/Q8): pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF

Smallest Hindi-capable GGUF — runs on any laptop at 0.37GB.
Next: v2 with more data, better responses.

#Hindi #LLM #GGUF #OpenSource

reacted to their post with 🔥 7 days ago

Post

2690

🧬 Just uploaded K-quants of Carbon-3B for llama.cpp users!
@HuggingFaceBio released the original GGUF in bf16 only — so I added the full quant ladder for CPU/edge inference:
• Q2_K → 1.4 GB
• Q3_K_M → 1.8 GB
• Q4_K_M → 2.1 GB ⭐
• Q5_K_M → 2.4 GB
• Q6_K → 2.7 GB
• Q8_0 → 3.5 GB
🔗 pankajpandey-dev/Carbon-3B-GGUF
Now you can generate DNA sequences on your laptop. Needs a llama.cpp build with PR #23410 (HybridDNATokenizer support).
Huge thanks to the HuggingFaceBio team for the original model 🙏
#GGUF #llamacpp #genomics #DNA

reacted to their post with 🔥 7 days ago

Post

684

🇮🇳 Just shipped: MiniCPM5-1B-Hindi-Instruct (+ GGUF quants)

First Hindi instruction-tuned fine-tune of OpenBMB's brand-new MiniCPM5-1B (released this week).

Trained with Unsloth + LoRA (r=32) on AI4Bharat's anudesh + dolly Hindi splits — ~4k high-quality examples, 2 epochs on a single T4 in 60 minutes.

🔗 Model (16-bit + LoRA adapter):
pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct

📦 GGUF quants for llama.cpp / Ollama / LM Studio:
pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct-v1-GGUF

5 quant levels — from Q3_K_M (~560 MB, runs on a Raspberry Pi) to Q8_0 (~1.2 GB, near-lossless). Q4_K_M is the recommended default.

Part of my ongoing 🇮🇳 Hindi LLM Series — bringing strong open-source LLMs to Indian languages.

#Hindi #IndicNLP #MiniCPM5 #LoRA #Unsloth #GGUF #llamacpp #Ollama #LocalLLM

reacted to their post with 🔥 7 days ago

Post

14927

🇮🇳 Qwen3-4B Hindi Instruct v2 — a Hindi LLM that runs on your own machine
Most strong Hindi-capable models are either huge or cloud-only. I wanted one that's small enough to run locally but actually follows instructions in Hindi — so I fine-tuned Qwen3-4B on 10K Hindi instruction pairs and shipped it with a full GGUF quant ladder.
✅ Fine-tune (16-bit): huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2
✅ GGUF (Q4/Q5/Q8): huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2-GGUF
Runs in Ollama, llama.cpp, and LM Studio. The Q4_K_M is just 2.5 GB — fits comfortably on a laptop, CPU or GPU.
Part of my Hindi LLM Series — building openly-licensed Indic models for local and edge use. More coming (Gemma next). Feedback welcome 🙏
#Hindi #IndicNLP #GGUF #LocalLLM #Qwen

4 replies

·

posted an update 7 days ago

Post

14927

🇮🇳 Qwen3-4B Hindi Instruct v2 — a Hindi LLM that runs on your own machine
Most strong Hindi-capable models are either huge or cloud-only. I wanted one that's small enough to run locally but actually follows instructions in Hindi — so I fine-tuned Qwen3-4B on 10K Hindi instruction pairs and shipped it with a full GGUF quant ladder.
✅ Fine-tune (16-bit): huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2
✅ GGUF (Q4/Q5/Q8): huggingface.co/pankajpandey-dev/Qwen3-4B-Hindi-Instruct-v2-GGUF
Runs in Ollama, llama.cpp, and LM Studio. The Q4_K_M is just 2.5 GB — fits comfortably on a laptop, CPU or GPU.
Part of my Hindi LLM Series — building openly-licensed Indic models for local and edge use. More coming (Gemma next). Feedback welcome 🙏
#Hindi #IndicNLP #GGUF #LocalLLM #Qwen

4 replies

·

posted an update 11 days ago

Post

684

🇮🇳 Just shipped: MiniCPM5-1B-Hindi-Instruct (+ GGUF quants)

First Hindi instruction-tuned fine-tune of OpenBMB's brand-new MiniCPM5-1B (released this week).

Trained with Unsloth + LoRA (r=32) on AI4Bharat's anudesh + dolly Hindi splits — ~4k high-quality examples, 2 epochs on a single T4 in 60 minutes.

🔗 Model (16-bit + LoRA adapter):
pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct

📦 GGUF quants for llama.cpp / Ollama / LM Studio:
pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct-v1-GGUF

5 quant levels — from Q3_K_M (~560 MB, runs on a Raspberry Pi) to Q8_0 (~1.2 GB, near-lossless). Q4_K_M is the recommended default.

Part of my ongoing 🇮🇳 Hindi LLM Series — bringing strong open-source LLMs to Indian languages.

#Hindi #IndicNLP #MiniCPM5 #LoRA #Unsloth #GGUF #llamacpp #Ollama #LocalLLM

posted an update 12 days ago

Post

2690

🧬 Just uploaded K-quants of Carbon-3B for llama.cpp users!
@HuggingFaceBio released the original GGUF in bf16 only — so I added the full quant ladder for CPU/edge inference:
• Q2_K → 1.4 GB
• Q3_K_M → 1.8 GB
• Q4_K_M → 2.1 GB ⭐
• Q5_K_M → 2.4 GB
• Q6_K → 2.7 GB
• Q8_0 → 3.5 GB
🔗 pankajpandey-dev/Carbon-3B-GGUF
Now you can generate DNA sequences on your laptop. Needs a llama.cpp build with PR #23410 (HybridDNATokenizer support).
Huge thanks to the HuggingFaceBio team for the original model 🙏
#GGUF #llamacpp #genomics #DNA

posted an update 14 days ago

Post

272

Just released Qwen3-0.6B fine-tuned on Hindi instruction data 🇮🇳

✅ Full model: pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1
✅ GGUF versions (Q2/Q4/Q5/Q8): pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF

Smallest Hindi-capable GGUF — runs on any laptop at 0.37GB.
Next: v2 with more data, better responses.

#Hindi #LLM #GGUF #OpenSource

Pankaj Pandey

AI & ML interests

Recent Activity

Organizations

Pankaj Pandey

AI & ML interests

Recent Activity

Organizations

pankajpandey-dev's activity