Unsloth releases a 1-bit GGUF quantized version of Mistral Le Chonk, enabling local use of the 1T-parameter model
Mistral Le Chonk can now run locally!
AI BriefingUnsloth released a Dynamic 1-bit GGUF of Mistral Le Chonk (Mistral Large 4, 1T parameters, MoE, 52B active parameters) that can run locally on a Mac Studio, retains 91% top-1 accuracy, and takes up about 233GB (80% smaller than 1.1TB FP8). The image shows the hardware requirements for each quantization level: 1-bit requires 256-270GB of memory, while the smallest UD-IQ1_S uses about 233GB of disk space. It also supports text and image input, a 1M context window, and 160+ languages. The GGUF and guide are available at huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF and unsloth.ai/docs/models/mistral-large-4.
Source: Unsloth · x.com
Event follow-up3
View all event coverage- vLLM releases Day 0 support for Mistral Large 4vLLM Official Blog · 18 days ago
- Unsloth releases a 1-bit GGUF quantized version of Mistral Le Chonk, enabling local use of the 1T-parameter modelUnsloth · 18 days ago
- Mistral Large 4 (Le Chonk) arrives on Hugging Face: an open-weight, multimodal MoE model with 1T parametersHugging Face Official Team Blog · 18 days ago