FINAL-Bench/POCKET-KR-GGUF
Text Generation • 23B • Updated • 878 • 24
Al Sabri Bhuiyan
alsunny
·
AI & ML interests
I'm a Data Analyst and Web Developer with a passion for turning complex data into actionable insights and building user-friendly, efficient web applications. With [X years] of experience in both fields, I bring a unique combination of analytical thinking and technical expertise. My goal is to create value by leveraging data and technology to solve real-world problems.
Recent Activity
liked a model 2 days ago
FINAL-Bench/POCKET-KR-GGUF liked a model 2 days ago
FINAL-Bench/POCKET-35B-GGUF reacted to SeaWolf-AI's post with 👍 2 days ago
📱 POCKET — a 35-billion-parameter model that runs on your iPhone, and on your PC with no GPU
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud — it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
• CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s → 2.69× faster
• GPU generate (H100): 197 vs 89 tok/s → 2.22× faster
• GPU prompt processing (H100): 753 vs 1816 → 0.41× (Bonsai wins this one — MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
• Quality (HellaSwag, 400 q): 61.0% vs 60.0% → a tie (confidence intervals overlap)
On a real consumer laptop — MacBook M3 Pro (18 GB) — POCKET wins every axis, prompt processing included:
• Metal generate: 25.4 vs 12.8 → 1.99×
• CPU generate: 13.8 vs 4.4 → 3.13×
• Metal prompt: 240.7 vs 73.4 → 3.28×
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all — it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
📖 Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
📦 POCKET-35B-GGUF (PC / server, no GPU): https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF
🇰🇷 POCKET-KR-GGUF (Android): https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF
🍎 POCKET-KR-MLX (iPhone / Mac): https://huggingface.co/FINAL-Bench/POCKET-KR-MLX
🌍 POCKET-EN-GGUF (English phone / PC): https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF
🖥️ Live demo (answering on a CPU, no GPU): https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU
📚 Collection: https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
Organizations
liked 2
models 2 days ago
reacted to SeaWolf-AI's post with 👍 2 days ago
Post
3923
📱 POCKET — a 35-billion-parameter model that runs on your iPhone, and on your PC with no GPU
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud — it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
• CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s → 2.69× faster
• GPU generate (H100): 197 vs 89 tok/s → 2.22× faster
• GPU prompt processing (H100): 753 vs 1816 → 0.41× (Bonsai wins this one — MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
• Quality (HellaSwag, 400 q): 61.0% vs 60.0% → a tie (confidence intervals overlap)
On a real consumer laptop — MacBook M3 Pro (18 GB) — POCKET wins every axis, prompt processing included:
• Metal generate: 25.4 vs 12.8 → 1.99×
• CPU generate: 13.8 vs 4.4 → 3.13×
• Metal prompt: 240.7 vs 73.4 → 3.28×
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all — it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
📖 Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
📦 POCKET-35B-GGUF (PC / server, no GPU): FINAL-Bench/POCKET-35B-GGUF
🇰🇷 POCKET-KR-GGUF (Android): FINAL-Bench/POCKET-KR-GGUF
🍎 POCKET-KR-MLX (iPhone / Mac): FINAL-Bench/POCKET-KR-MLX
🌍 POCKET-EN-GGUF (English phone / PC): FINAL-Bench/POCKET-EN-GGUF
🖥️ Live demo (answering on a CPU, no GPU): FINAL-Bench/POCKET-35B-CPU
📚 Collection: FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud — it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
• CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s → 2.69× faster
• GPU generate (H100): 197 vs 89 tok/s → 2.22× faster
• GPU prompt processing (H100): 753 vs 1816 → 0.41× (Bonsai wins this one — MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
• Quality (HellaSwag, 400 q): 61.0% vs 60.0% → a tie (confidence intervals overlap)
On a real consumer laptop — MacBook M3 Pro (18 GB) — POCKET wins every axis, prompt processing included:
• Metal generate: 25.4 vs 12.8 → 1.99×
• CPU generate: 13.8 vs 4.4 → 3.13×
• Metal prompt: 240.7 vs 73.4 → 3.28×
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all — it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
📖 Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
📦 POCKET-35B-GGUF (PC / server, no GPU): FINAL-Bench/POCKET-35B-GGUF
🇰🇷 POCKET-KR-GGUF (Android): FINAL-Bench/POCKET-KR-GGUF
🍎 POCKET-KR-MLX (iPhone / Mac): FINAL-Bench/POCKET-KR-MLX
🌍 POCKET-EN-GGUF (English phone / PC): FINAL-Bench/POCKET-EN-GGUF
🖥️ Live demo (answering on a CPU, no GPU): FINAL-Bench/POCKET-35B-CPU
📚 Collection: FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
- 5 replies
liked a
dataset 2 days ago
reacted to salma-remyx's post with 🔥 2 days ago
Post
3524
Nobody can be the originator of every great idea. And the rate of innovation in AI is difficult to track in detail.
But the next great idea for your codebase may have been described in open-science publications like arXiv.
Who among us can keep up with reading, smoke testing, adapting every viral method in the feed? (not me)
And what about the long-tail of results that you won't discover for another 6 months? That's what we built Outrider for!
Now I get alerts when candidate methods look like a good fit for the engineering challenges I'm addressing in my application.
I don't waste time running down dead end leads. Instead, I choose to invest my attention into ones which have earned it with evidence.
Try it yourself: https://studio.remyx.ai
Read the docs: https://docs.remyx.ai
But the next great idea for your codebase may have been described in open-science publications like arXiv.
Who among us can keep up with reading, smoke testing, adapting every viral method in the feed? (not me)
And what about the long-tail of results that you won't discover for another 6 months? That's what we built Outrider for!
Now I get alerts when candidate methods look like a good fit for the engineering challenges I'm addressing in my application.
I don't waste time running down dead end leads. Instead, I choose to invest my attention into ones which have earned it with evidence.
Try it yourself: https://studio.remyx.ai
Read the docs: https://docs.remyx.ai
liked 2
models 2 days ago
liked 2
Spaces 2 days ago
upvoted a paper 2 days ago
liked a
model 2 days ago
liked a
model 11 months ago
liked a
Space over 2 years ago