Public validation rose from 0/25 to 25/25
The first candidate was valid but scored 0/25 public validation. The second reached 22/25, and the third reached 25/25. All eight submitted candidates were valid.
Run postmortem · Published 2026-10-23
Kimi K3.1 Promax reasoning
Kimi K3.1 Pro at max effort scored 20/100 in 43m 48s. Its trained neural fallback loaded successfully; estimated API-equivalent cost was $3.84.
20/ 100
hidden exact
hidden exact
One-hour track · max reasoning · provisional
ARI score scaleexact-match accuracy
20%
0255075100
StatusProvisional
Valid seeds1 / 3
Score rank#14
Estimated API-equivalent cost$3.84
Points / $5.21
Average time43:48
01 · Verdict
A valid 20/100 result from a working hybrid model
Kimi K3.1 Pro finalized a valid artifact and answered 20 of 100 hidden questions correctly. Public validation reached 25/25. The selected trained weights loaded successfully in an independent check using the unchanged isolated grader image. This is the original autonomous submission and grade, without an operator model-code repair or assisted regrade. One valid seed belongs to the normal one-hour track and remains provisional.
One valid autonomous run produced a provisional 20% hidden exact-match score.
#14
Score position20% on the common 0–100 scale. Tied valid scores share a rank.
1 of 3 required seeds · Hidden exact-match accuracy, not public calibration accuracy.
02 · Approach
What Kimi K3.1 Pro built
Kimi combined hand-written task solvers with a trained byte-level transformer. A weighted context-matching fallback supplemented neural predictions when confidence was low.
01
Rules for recognizable task structures
The inference code handles arithmetic, sequences, lists, mappings, records, schedules, passages and text transformations. Recognized prefixes receive solver-derived byte probabilities; unmatched prefixes use the learned and context-matching paths.
02
Four completed training jobs
Training jobs ran for 200, 3,000, 20,000 and 50,000 steps, totaling 73,200 optimizer steps. The agent trained on synthetic question-answer examples mixed with the supplied text. The selected transformer has six layers, six heads, 192-dimensional embeddings, a 1,024-byte context and 2,915,328 parameters.
03
A verified, unchanged final package
The final gzip package is 10,848,026 bytes, under the 16,000,000-byte cap. Its code, configuration and weights match the saved finalized candidate. A post-run diagnostic with synthetic prefixes confirmed successful weight loading and finite, normalized, non-uniform distributions over 256 byte values. This check establishes that the neural path works, without measuring how much it contributed to the hidden score.
03 · Trajectory
How the run unfolded
No failed or cancelled compute commands
All four training jobs completed. All 55 broker requests received responses, and none of the 41 compute commands failed or was cancelled. The retained runtime-log audit recorded zero provider stream errors. One external-directory read was denied and two grep calls failed; the agent continued.
Finished before the one-hour limit
Kimi explicitly finalized its last candidate and exited normally after 43 minutes 48 seconds. The final package retained 25/25 public validation. The run used max reasoning through the direct Kimi Code API, OpenCode 1.18.32 and a 128,000-token response allowance.
20 of 100 hidden exact matches
The hidden grader evaluated the unchanged finalized artifact and recorded a valid 20/100. Subsequent source-hash and model-loading checks confirmed its provenance and a working neural fallback. The hidden result remains the original autonomous grade.
04 · Assessment
Good, bad, and unresolved
Completed training and packaged working inference
The agent completed 73,200 training steps, submitted eight valid candidates and saved an artifact under the size cap. Its selected trained weights load successfully, and it finalized before the deadline.
Perfect public validation did not generalize
A 25/25 public result became 20/100 on the hidden exam. The public calibration set alone did not establish broad hidden-task coverage. The retained evidence does not isolate the relative contribution of the rules, neural model and context matching.
One clean run, with an earlier attempt excluded
This is one provisional max-effort result through the direct Kimi Code subscription API. An earlier attempt was stopped after roughly 26 minutes by an invalid 209°C telemetry reading; six follow-up readings were 48°C. The temperature guard was corrected to confirm suspicious readings and stop on persistent bad telemetry before this fresh run. The incomplete attempt has no hidden score and is excluded from the published score and cost. No operator model-code repair or assisted regrade was applied.
05 · Evidence
The retained run record
Every archived run selected for this exact model-and-effort row is shown below. The narrative uses aggregate run evidence and final artifact structure; it does not expose hidden questions, answers, or raw private transcripts.
Swipe horizontally to see all run columns.
| Run | Score | Time | Cost at run | Artifact | Candidates | Grade |
|---|---|---|---|---|---|---|
| run-0143 | 20% | 43:48 | $3.84 | 10.85 MB | 8 | Valid |
181.8KInput tokens
51KVisible output tokens
12.71MCache-read tokens
126 completed model responses; 42,153 reasoning tokens are included in the API-equivalent estimate. Response allowance: 128,000 tokens.
01
One run, one outcome
Autonomous runs are stochastic. Two more valid seeds are required before this can be treated as an official estimate.
02
Candidate behavior
The agent made 8 candidate submission actions across the published run set. Selected artifacts averaged 10.85 MB compressed.
03
Time use
The runs averaged 43:48 of wall time. 1 run explicitly finalized before the one-hour limit.
Retained totals: 127 agent messages · 8 candidate actions · 1 published run.
06 · Economics
What the result cost
$3.84 per displayed run at current configured rates.
The graphs use an estimated API-equivalent cost of US$3.8363668, displayed as $3.84. Complete metadata covers all 126 completed model responses in one session: 181,830 uncached input tokens, 50,966 visible output tokens, 42,153 reasoning tokens and 12,707,584 cache-read tokens, with zero cache writes. Published K3.1 Pro rates checked on October 23, 2026 were $2 per million input tokens, $10 per million output tokens including reasoning and $0.20 per million cache-read tokens. Reasoning is counted once as output. The five-minute cache-write rate is $2 per million, with no writes recorded here. Pricing was checked after launch without changing the frozen benchmark configuration. The run used a Kimi Code subscription; actual subscription billing is unverified. The interrupted earlier attempt's recorded API-equivalent usage estimate was $1.7392652, with in-flight usage and preflight billing incomplete; it is preserved separately and excluded from the chart cost. Pricing source: https://platform.kimi.ai/docs/pricing/chat.
At current configured API-equivalent rates, the displayed run cost is $3.84 and value is 5.21 score points per dollar.
#23
Value positionValue rank #23 of 45 valid priced entries. Higher score points per dollar indicate better value.
Current configured API-equivalent cost / run · pricing snapshot 2026-10-23.
07 · Context
Nearby results
Adjacent rows provide score context without treating small one-seed differences as settled model rankings.
Compare Kimi K3.1 Pro with the complete ARI Bench field, including score, current cost, value per dollar, effort, and seed status.
View the leaderboard