openfree/novel-themes-library
Viewer โข Updated โข 329 โข 24 โข 4
openfree
openfree
AI & ML interests
Contact: arxivgpt@gmail.com
Recent Activity
liked a Space about 12 hours ago
VIDraft/physical-ai-spot upvoted a paper about 12 hours ago
Quantum Cryptanalysis on IBM Quantum Hardware: Extending Even--Mansour Period Recovery from N=4 to N=10 reacted to SeaWolf-AI's post with ๐ค about 14 hours ago
๐ฑ POCKET โ a 35-billion-parameter model that runs on your iPhone, and on your PC with no GPU
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud โ it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
โข CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s โ 2.69ร faster
โข GPU generate (H100): 197 vs 89 tok/s โ 2.22ร faster
โข GPU prompt processing (H100): 753 vs 1816 โ 0.41ร (Bonsai wins this one โ MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
โข Quality (HellaSwag, 400 q): 61.0% vs 60.0% โ a tie (confidence intervals overlap)
On a real consumer laptop โ MacBook M3 Pro (18 GB) โ POCKET wins every axis, prompt processing included:
โข Metal generate: 25.4 vs 12.8 โ 1.99ร
โข CPU generate: 13.8 vs 4.4 โ 3.13ร
โข Metal prompt: 240.7 vs 73.4 โ 3.28ร
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all โ it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
๐ Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
๐ฆ POCKET-35B-GGUF (PC / server, no GPU): https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF
๐ฐ๐ท POCKET-KR-GGUF (Android): https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF
๐ POCKET-KR-MLX (iPhone / Mac): https://huggingface.co/FINAL-Bench/POCKET-KR-MLX
๐ POCKET-EN-GGUF (English phone / PC): https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF
๐ฅ๏ธ Live demo (answering on a CPU, no GPU): https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU
๐ Collection: https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
Organizations
