inventions of dront
Collection
2 items • Updated
DrontGPTQ is a simple quantizer for GPT-like models.
Supports:
2 bit ✅
3 bit ✅
4 bit ✅
5 bit ✅
6 bit ✅
7 bit ❌
8 bit ✅
12 bit ✅
16 bit ✅
32 bit ✅
Consists of just two files.
DrontQ.py – a very simple quantizer. Converts HF F32 to .dq format (Dront quant).
DQuse.py – a simple way to run .dq models.
To use this, scroll to the end of the files; you will find the main variables there, which you should adjust to your needs. Quantizes local models—specifically GPT-like ones. Most Dront gpt-like models will also be released in DQ quantization! Especially the new VdrontV
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support