Mistral launches edge-optimised generative AI models

Mistral says there is a growing demand for local, privacy-focused AI solutions

Image:
Mistral launches edge-optimised generative AI models. Source: Mistral AI, Wikipedia

French AI startup Mistral has unveiled its first set of generative AI models developed to run on edge devices including laptops and smartphones.

The new range of models, branded as "Les Ministraux," can handle a variety of applications, from simple text generation to more complex tasks when used alongside larger models.

Two models from the Les Ministraux line are now available: Ministral 3B and Ministral 8B. Both offer a context window of 128,000 tokens, allowing them to process text equivalent to a 50-page book in a single instance.

Demand for privacy-focused AI solutions

Mistral highlighted the growing demand for local, privacy-focused AI solutions. In a blog post, the company stated: “Our most innovative customers and partners have increasingly been asking for local, privacy-first inference for critical applications such as on-device translation, internet-less smart assistants, local analytics, and autonomous robotics. Les Ministraux were built to provide a compute-efficient and low-latency solution for these scenarios.”

While the Ministral 8B model is now available for download, it is restricted to research purposes. Companies and developers interested in deploying Ministral 8B or Ministral 3B commercially must contact Mistral for licensing.

For others, these models will soon be accessible via Mistral’s cloud platform, La Platforme, and through other partner cloud services. Pricing for Ministral 8B is set at US 10 cents/0.1€ per million tokens (~750,000 words), while Ministral 3B will cost US 4 cents/0.04€ per million tokens.

The release follows a broader trend in the AI industry towards smaller, more efficient models that can be trained, fine-tuned, and deployed more rapidly.

Companies like Google, Microsoft and IBM have introduced similar small-scale models, with Google’s Gemma family, Microsoft’s Phi models and some of IBM’s Granite options targeting edge hardware.

Meta has also contributed to this movement, with its latest Llama models designed for edge devices.

Mistral claims that both Ministral 3B and 8B outperform comparable Llama and Gemma models, as well as its own previous offering, the Mistral 7B, across several AI benchmarks that test instruction-following and problem-solving capabilities.

Based in Paris, Mistral has been growing its AI product portfolio, supported by $640 million in venture capital funding.

Recent launches include a free developer testing service, a software development kit (SDK) for fine-tuning, and Codestral, a new generative model for coding.

Co-founded by former Meta and DeepMind engineers, the company’s ambition is to produce leading AI models that rival the industry’s top performers, including OpenAI's GPT-4 and Anthropic’s Claude.