AH-Nanbeige4.3-33B IQ3_XXS
An IQ3_XXS quantization of Nanbeige4.3-33B by AH, made with bartowski's imatrix: a strong model for coding work, which can do even better on a slightly faster GPU. A 33B model in one 13.1 GB file, for tool calls, terminal work and code. Kindi 0.6.1 runs it on one card with 16 GB or more, on two RTX 3080 or two Tesla P100 cards, and on a Quadro K6000; llama.cpp runs the same file.

- 13.1 GB one GGUF file, 52 layers
- 45.8 tok/s Kindi writes on two RTX 3080 cards (llama.cpp: 36.7)
- 2.0× a copy with changes, with prompt lookup
- 98 of 115 agent and coding tasks in our initial test (Bonsai 27B: 96)
Where it runs
Kindi 0.6.1. One card with 16 GB or more (for example an RTX 3090 with 24 GB), or several cards with about 17 GB in total. We measured two RTX 3080 cards (10 GB each) and two Tesla P100 cards (16 GB each). The engine puts the layers in order on the cards; only the hidden state goes between the cards, so a PCIe x1 slot is sufficient. The installer sets a context of 16,384 tokens. On a Quadro K6000 (12 GB), Kindi keeps 13 layers in host memory and computes a part of them on the CPU.
The output agrees with llama.cpp: kindi verify passes 11 of 11 prompts, 8 token for token and 3 that end at a tie in the
scores of llama.cpp.
curl -fsSL https://amirhusain.com/kindi/install.sh | bash -s -- --model nanbeigeOn a machine that already runs Kindi: kindi use nanbeige-33b. The installer downloads the file
and checks its sha256. llama.cpp: the GGUF works with llama.cpp and its tools; use --jinja for the chat template and the
tool calls. The model card has the command.
Speed against llama.cpp
The same cards, the same model file and the same context for both engines. Two GeForce RTX 3080 cards at 190 W, on PCIe x1, with the layers split 26 and 26 and an f16 K/V cache. Kindi 0.6.0 writes about 23% to 25% faster than llama.cpp, and reads a 512-token prompt at 1,036 tok/s (llama.cpp: 785).
Table and test conditions
| Context (tokens) | Kindi read | llama.cpp read | Kindi write | llama.cpp write |
|---|---|---|---|---|
| Short | – | – | 45.8 | 36.7 |
| 2,048 | – | – | 44.6 | 35.6 |
| 8,192 | – | – | 40.9 | 33.3 |
| 512 (read) | 1,036 | 785 | – | – |
Tokens per second, greedy, Kindi 0.6.0. Read: a prompt of 512 tokens from position 0. Write: Kindi kindi bench, 64 tokens at
each depth; llama.cpp llama-bench -ngl 99 -fa 1 -sm layer at each depth, with the same file and f16 K/V. docs/BENCHMARKS.md
Other cards
Two Tesla P100 16 GB cards (PCIe, 200 W): Kindi writes the 33B about 2.2 times as fast as llama.cpp. One Quadro K6000 12 GB, with 13 layers in host memory and a context of 8,448 tokens: 2.3 times as fast at 8K. The 33B also runs on a single RTX 3090 (24 GB).
| Cards | Context (tokens) | Kindi | llama.cpp | Times as fast |
|---|---|---|---|---|
| 2× Tesla P100 | 512 | 21.0 | 9.6 | 2.2× |
| 2× Tesla P100 | 8,192 | 19.6 | 8.9 | 2.2× |
| Quadro K6000 | 480 | 3.85 | 2.19 | 1.8× |
| Quadro K6000 | 2,016 | 3.55 | 1.96 | 1.8× |
| Quadro K6000 | 8,160 | 3.56 | 1.52 | 2.3× |
Write speed in tokens per second, the same file and the same context for both engines. P100: f16 K/V, 21.0 and 19.6 tok/s at 512 and 8,192 tokens; Kindi writes 15.7
tok/s at 32K and 12.3 at 64K. K6000: llama.cpp with -ngl 42, the other 10 layers on the CPU;
Prompt lookup
Agent work often copies text: file contents, tool output, code to change. With prompt lookup, Kindi drafts the next tokens from text that is already in the prompt, and the model checks them in one pass. A copy of a file with changes is written at 91.6 tok/s. Prose keeps its speed. The greedy replies were the same with and without lookup.
| Request | Speed-up with prompt lookup |
|---|---|
| Copy a file with changes | 2.03× |
| A tool call | 1.17× |
| Thinking, then code | 1.04× |
| Prose | 1.00× |
Two RTX 3080 cards, 400 tokens per reply, greedy, through kindi serve. Lookup is on by default
(lookup 5: up to 5 drafted tokens per pass; 0 turns it off).
For agent work
Our initial test: 115 tasks, one seed per task, the same settings for both models. Both models are almost at the limit of kbench, so the recent LiveCodeBench problems show more of the difference.
| Test | AH-Nanbeige4.3-33B IQ3_XXS | Ternary-Bonsai-2 27B |
|---|---|---|
| kbench, 65 agent tasks | 64 (98.5%) | 64 (98.5%) |
| LiveCodeBench, 50 recent problems | 34 (68.0%) | 32 (64.0%) |
| All 115 tasks | 98 (85.2%) | 96 (83.5%) |
In this test, AH-Nanbeige 33B ran in llama.cpp and Bonsai 27B ran in Kindi. Kindi gives the same output as llama.cpp for this file.
Both models run on Kindi's OpenAI-compatible server, so an agent tool such as opencode can use both: the 27B for fast steps and the 33B for harder coding steps.

kbench: 65 terminal and coding tasks for agents; the model works in the pi agent harness, and hidden tests check each result. LiveCodeBench: 50 recent problems.
License and credits

The base model is Nanbeige4.3-33B by the Nanbeige team, under the Apache License 2.0. This file has the same license; the change is the quantization to IQ3_XXS. We did not change the weights by training. The importance matrix is from bartowski. The quantization format is from llama.cpp. Kindi is free software under the AGPL-3.0, with commercial licenses available.