Generated via:

git clone https://github.com/eauchs/mlx-lm --single-branch --depth=1 --branch add-cohere2-moe
uv venv venv
source venv/bin/activate.fish
uv pip install mlx[cpu] ./mlx-lm
python -m mlx_lm.convert \
 --hf-path "./BLS-Mini-Code-1.0" \
 -q \
 --q-bits 4 --q-group-size 64 \
 --mlx-path ./cohere-mlx-q4

Requires this fork of mlx_lm to run (main doesn't include support for cohere2_moe yet): https://github.com/eauchs/mlx-lm/tree/add-cohere2-moe

Downloads last month: 162

Safetensors

Model size

30B params

Tensor type

BF16

U32

MLX

Hardware compatibility

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for theroadnotbacon/BLS-Mini-Code-1.0-Q4-MLX

Base model

CohereLabs/BLS-Mini-Code-1.0

Quantized

(5)

this model