RDTvlokip/POGAT
Updated
vloplok PRO
RDTvlokip
AI & ML interests
None yet
Recent Activity
commentedon their article 11 days ago
📊 Cross-Entropy — La fonction de perte qui SAIT punir ! 🎯🔥 posted an update about 1 month ago
🧠 I trained a French LLM from scratch. Alone. On a 1080 Ti. And honestly… it was a lot.
4 months building the dataset before even touching the model. Custom crawler, custom extractor, custom BPE tokenizer, everything from zero. Then the architecture — RoPE, RMSNorm, SwiGLU, Flash Attention. Then a 3-phase trainer. Then debugging a causal mask bug that made the model generate "ïsïsïs" for hours.
Then the power went out at epoch 10/18.
The checkpoint survived. The model learned form perfectly — grammar, markdown, structure. Substance? Still working on it. Honest conclusion.
Full write-up here 👇
🔗 https://huggingface.co/blog/RDTvlokip/i-trained-my-own-french-llm-from-scratchOrganizations
