EXL3 models
Collection
48 items • Updated
• 39
EXL3 quants of Step-3.5-Flash
⚠️ Requires ExLlamaV3 v0.0.23 (or v0.0.22 dev branch)
Base bitrates:
2.00 bits per weight
3.00 bits per weight
4.00 bits per weight
Optimized:
2.08 bits per weight
3.05 bits per weight
(more coming soon)
| . | Ppl¹ | KL-div |
|---|---|---|
| 2.00 bpw | 2.629 | 0.653 |
| 2.08 bpw | 2.154 | 0.466 |
| 3.00 bpw | 1.521 | 0.142 |
| 3.05 bpw | 1.478 | 0.118 |
| 4.00 bpw | 1.379 | 0.053 |
| Original | 1.336 |
¹ (10 rows of wikitext2)
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for turboderp/Step-3.5-Flash-exl3
Base model
stepfun-ai/Step-3.5-Flash