New coding model aims to run fast on your own computer

The company Recursive has released its first AI model, Flint. It is built to help write code and can run locally, meaning on the user's own computer instead of with an online provider. That's according to TestingCatalog.

The model is aimed at developers who want coding agents (AI that carries out steps in a coding task on its own) that work offline, keep data on the device and avoid fees for using an API.

Big model, small bill

Flint is a so-called mixture-of-experts model. It has 40 billion parameters in total, but uses only 2.5 billion of them per token (a small piece of text). That reduces the computing power needed when the model responds.

According to Recursive, Flint tops MoE (mixture-of-experts) models under 50 billion parameters on the three tests SWE-Bench Pro v2, Terminal-Bench 2.1 and SWE-Bench Verified. The company also says the model performs on par with dense coding models of 25-40 billion parameters while running more than five times faster.

The model comes with an inference engine called rec, which is tuned for Flint on Apple M4 Pro and M5 Pro or Max chips. With it, Flint is said to reach more than 125 tokens per second on an M4 Pro MacBook Pro and more than 150 tokens per second on an NVIDIA RTX Spark with 64 GB. The figures apply to short contexts, and Recursive claims up to double the speed compared with similar models.

Built with AI help

According to TestingCatalog, the model was built from scratch in under four months by a small team of humans and agents. They were assisted by Recursive's automated research system, Eureka Machine, which proposed ideas, implemented them and tested the results.

Open, with caveats

The weights, meaning the trained model itself, are released under the Apache 2.0 license. They can be used in llama.cpp, Ollama, Unsloth, LM Studio, vLLM and SGLang, among others. Flint supports up to 512k tokens of context, but Recursive recommends staying within 256k.

Recursive itself points out that Flint may require more guidance than larger closed models. The company recommends breaking complex tasks into smaller subtasks and giving explicit instructions.

Benchmark results and speed figures come from Recursive itself and have not been independently verified. It is also unclear exactly which models the speed comparisons are based on.