Benchmarks | halogen™

Flash-Next on the GPU: decode

Decode by kind of text, 0.18.0
Kind of text	v2 (default)	ht43
Writing code	72.9 tok/s	69.0 tok/s
Editing ~8,000 tokens of code	81.0 tok/s	78.6 tok/s
An agent's tool call after reading a ~3,000-token file	70.6 tok/s	72.9 tok/s
English prose	54.3 tok/s	52.3 tok/s
Six other languages	52.2 tok/s	49.7 tok/s
Explaining an ~8,000-token document	54.8 tok/s	54.2 tok/s
Repetitive output, five prompts	83 to 94 tok/s; 61 on one word 300 times	80 to 91 tok/s; 63 on one word 300 times

Source: Measured, 0.18.0 both checkpoints · measured at 0.18.0

The current release

0.18.17 against 0.18.16
	v2	ht43
`bench` over ten prompt shapes, greedy, mean	66.64 / 66.64 tok/s	67.54 / 67.53 tok/s

Each row is an A/B pair on one machine in one session. Figures are 0.18.16 / 0.18.17. Generation with drafting is unchanged.

Source: Measured · measured at 0.18.16 and 0.18.17

Flash-Next on the GPU: prefill by prompt length

Bars under the figures: full width is 2,000 tok/s.

Prefill, sweep, cache off, 0.18.0
Prompt	v2 (default)	ht43
8,192 tokens	1,821 tok/s	1,630 tok/s
32,768 tokens	1,740 tok/s	1,604 tok/s
131,072 tokens	1,675 tok/s	1,551 tok/s

Prefill, sweep, cache on (a cold request), 0.18.0
Prompt	v2 (default)	ht43
8,192 tokens	1,646 tok/s	1,502 tok/s
32,768 tokens	1,806 tok/s	1,647 tok/s

0.18.17 against 0.18.16, as the source prints each pair (0.18.16 is the left figure of a pair; each pair is two images on one machine in one session), cache off, 8,192 / 32,768 tokens: v2 1,877 / 1,823 and 1,877 / 1,823 tok/s; ht43 1,821 / 1,851 and 1,821 / 1,848 tok/s. Prefill is unchanged.

Source: Measured · measured at 0.18.0, 0.18.16 and 0.18.17

Flash-Next on the GPU: agent turns

Agent turn, 30,000 to 96,000 tokens of context, 0.18.0
	v2 (default)	ht43
Median turn	4.2 s	5.3 s
Median first token (90th percentile)	1.6 s (2.3 s)	1.6 s (2.4 s)
Median answer, and its decode	196 tokens at 83 tok/s	263 tokens at 77 tok/s
First turn of a 34,174-token agent prompt, cold	21.2 s	22.1 s

Source: Measured, 0.18.0 both checkpoints · measured at 0.18.0

Flash-Next on the GPU: many requests at once

Requests at once, total, 0.18.11
	v2	ht43
Three requests at once, greedy code	74.6 / 91.6 tok/s	72.7 / 88.8 tok/s
Four requests at once, greedy code	85.3 / 103.0 tok/s	83.7 / 99.4 tok/s

The docs also note that every request stays bitwise what it would be alone, and that the shipped context holds four conversations: what this release is not.

Source: Measured · measured at 0.18.11

The NPU models

Decisions: d3-lite and d3-nano, request wall
	d3-lite (Qwen3.5-0.8B)	d3-nano (Qwen3.5-2B)
Files on disk, picture reader included	2.5 GB	3.0 GB
A question, short state	~80 ms	~110 ms
At 600 tokens	~190 ms	~320 ms
At 3,500 tokens	~650 ms	~1.1 s
A question with a picture, 256 x 256 / 768 x 768	~0.2 s / ~0.5 s	~0.35 s / ~0.9 s

Source: NPU host details, decisions · measured at 0.18.11

Embeddings, reranking, moderation

Batched short inputs: embeddings, reranking, moderation
Case	Time
A batch of 2 inputs, or of 8	about 0.38 s
A single input up to 128 tokens	about 0.07 s
A single input above 128 tokens	0.09 s

When one request carries several short inputs they run together in one pass, each result exactly what it would be alone. A batch holds up to 8 inputs of up to 512 tokens each, padded to the bucket.

Source: NPU host details, batching · measured at 0.16.3

Images

Image generation and edits: flux2-klein-4b
Job	Time
A 512x512 image	about 3 seconds
A 1024x1024 image	about 10 seconds
An edit, one / two / three / four pictures	about 5 / 7.5 / 10 / 12.5 seconds
An edit at 1024x1024	about 12.5 seconds

Source: NPU host details, images and edits · measured at 0.18.0 and 0.18.12

Text to speech

Text to speech: qwen3-tts
	qwen3-tts-0.6b-customvoice	qwen3-tts-1.7b-customvoice
Speed	about 1.5 times faster than real time	a little faster than real time
First audio	about 0.3 seconds	about 0.4 seconds

Source: NPU host details, text to speech · measured at 0.18.14

Test setup

One Strix Halo. Figures are as the NPU page states them, with the approximation marks it uses (~, about). Host: NPU driver, XRT, firmware, IOMMU on, the GPU's fabric clock held at its top speed.

Decisions: one question a request, served alone, the request's wall (HTTP included), with one of the two loaded. Loaded together they share the NPU's memory, and d3-nano's questions about pictures take 15 to 25% longer. The pictures are scaled to between 256 x 256 and 768 x 768 worth of pixels.

The image model holds about 8 GB of the host's memory while it is loaded. One image request runs at a time on the NPU, queued with the other NPU models' work.

Speech: the talker runs on the NPU and the audio is decoded on the CPU. The NPU speaks one request at a time.

The docs publish no timing figures for decider-0.8b alone, qwen3.5-2b, or speech to text, so none appear here.