Uncensored version of coding model Flint can run on a single graphics card with 16 or 32 GB of memory

The company OrcaRouter has published a modified version of the AI model Flint on Hugging Face, a website where people share AI models. The original Flint was built by Recursive Superintelligence, Inc. and is aimed at coding agents that need to explore codebases, fix bugs, implement features and write tests.

OrcaRouter's version is called Flint-abliterated-GGUF. Abliteration means that one specific refusal direction has been removed from the model's weights. The weights are the numbers the model has learned, which determine how it responds. The result is a model that says no to requests less often.

Far fewer refusals

According to OrcaRouter's own measurements, the share of harmful requests that are refused falls from 86.3 percent in the original Flint to 10.4 percent in the modified version. On the AdvBench test the figure is 13.0 percent, on JailbreakBench 6.0 percent and on StrongREJECT 10.7 percent. Harmless requests are refused in 1.6 percent of cases.

OrcaRouter also says the model's capabilities have been preserved. Measured on the MMLU test it scores 83.3 percent, on GSM8K 90.0 percent, on MMLU-Pro CoT 73.2 percent and on CMMLU 67.0 percent, with a margin of error of roughly two percentage points.

Two sizes for home use

The model has 39.8 billion parameters in total, but uses only about 2.5 billion per token at a time. That is because it is a so-called mixture-of-experts model, in which only a few of the model's 256 experts are used for each word or word fragment. The context window, meaning how much text the model can hold in memory, is up to 262,144 tokens. The license is Apache 2.0, inherited from the base model.

There are two compressed GGUF versions, designed to run on your own computer:

- Mix-12G is 11.99 GB and, according to OrcaRouter, is intended for a graphics card with 16 GB of memory. - Mix-24G is 23.99 GB and requires a graphics card with 32 GB.

Running them requires llama.cpp master (PR #30928 or newer).

Caveats

Compression costs quality. Mix-24G has roughly 1.1 percent higher perplexity than the uncompressed model, while Mix-12G has roughly 20.0 percent higher. Perplexity is a measure of how well a model predicts text, and higher numbers are worse. The refusal and capability figures were also measured on the uncompressed BF16 version and not on the two GGUF versions, so the smallest one may in practice perform worse than the figures suggest.

The versions have only been tested on CUDA, and fidelity on long texts has not been verified. All figures for refusals, capabilities and compression come from OrcaRouter itself. For the original Flint, it is the developer who reports 74.2 percent on SWE-bench Verified and 54.9 percent on Terminal-Bench 2.1. No independent verification of the figures has been found.

The developers of Flint warn that the model may produce incorrect or harmful content, faulty code and inappropriate tool calls, and that generated code should be tested before use.