hd-lang

Project

Daily goal metrics

hd is built for programs written by coding agents, so we judge it with the Agentic Programming Language Arena: three pillars that ask what agent-driven development actually costs and produces. Each pillar is kept on its own and compared on a Pareto frontier rather than folded into one score, because a faster compiler that makes agents retry more is not a win. Compiler correctness is a gate in front of all three. Every metric is a script with a fixed target; no AI or agent run is involved, and the same scripts run against any hd binary. They run twice a day, at 00:00 and 12:00 UTC.

First: correctness

Correctness gate2 of 3 met
Is the compiler right? Conformance and incremental soundness. A faster wrong result is not better, so this is a prerequisite rather than a pillar.

The three pillars

Single-agent development cost43 of 56 met
How cheaply does one agent get a correct program? Compile and test latency is agent wait time; clear, short, well-placed diagnostics and fix-its that resolve mistakes are fewer retries and fewer tokens.
Agent scalability30 of 39 met
How much useful work do N agents do on one machine? CPU, memory and disk per check, startup, behaviour under concurrency, and caches that are shared, bounded and safe to write at once.
Artifact quality24 of 35 met
How good is the program the agents produced? The user's program, not the compiler: test speed, runtime against Node, allocations, binary size, host-call and serde cost, memory over time.

Trend

The share of each pillar's targets that the run met. A line going up is progress; a drop after a commit is a regression worth reading. Hover a point for its run.

0%50%100%10-25 12:0510-27 02:55Single-agent development cost, 10-25 12:05: 33% of targets metSingle-agent development cost, 10-26 03:35: 76% of targets metSingle-agent development cost, 10-26 18:48: 80% of targets metSingle-agent development cost, 10-27 02:55: 80% of targets metAgent scalability, 10-25 12:05: 18% of targets metAgent scalability, 10-26 03:35: 77% of targets metAgent scalability, 10-26 18:48: 77% of targets metAgent scalability, 10-27 02:55: 77% of targets metArtifact quality, 10-25 12:05: 22% of targets metArtifact quality, 10-26 03:35: 63% of targets metArtifact quality, 10-26 18:48: 66% of targets metArtifact quality, 10-27 02:55: 69% of targets metCorrectness gate, 10-25 12:05: 67% of targets metCorrectness gate, 10-26 03:35: 100% of targets metCorrectness gate, 10-26 18:48: 67% of targets metCorrectness gate, 10-27 02:55: 67% of targets metSingle-agent development cost 80%Agent scalability 77%Artifact quality 69%Correctness gate 67%

Single-agent development cost

How cheaply does one agent get a correct program? Each bar is one metric, drawn at its worst target's distance from the goal: left of the dashed line meets it, right of it misses by that multiple. Metrics without a numeric goal show only met or missed. The full list of targets is in the table below.

goal3× or worserecheck-precision: 2 target(s); worst modules rechecked, public signature edit: 9 vs ≤ 3recheck-precision✗ 9 vs ≤ 3mistakes: 4 target(s); worst fix-it makes hd check pass: 34.3% vs ≥ 80.0%mistakes✗ 34.3% vs ≥ 80.0%check-cost: 1 target(s); worst checked run time / unchecked run time: 2.84x vs ≤ 1.30xcheck-cost✗ 2.84x vs ≤ 1.30xpathological: 30 target(s); worst large-enum-match: growth exponent: 2.31 vs ≤ 1.30pathological✗ 2.31 vs ≤ 1.30fixit-safety: 1 target(s); worst fix-its that add no error: 100.0% vs ≥ 100.0%fixit-safety✓ 100.0% vs ≥ 100.0%diag-location: 1 target(s); worst diagnostics on the mistake's line: 98.6% vs ≥ 95.0%diag-location✓ 98.6% vs ≥ 95.0%edit-latency: 2 target(s); worst p50 check after an edit: 35 ms vs ≤ 50 msedit-latency✓ 35 ms vs ≤ 50 mstest-latency: 1 target(s); worst median edit-then-test: 165 ms vs ≤ 300 mstest-latency✓ 165 ms vs ≤ 300 msanswer-size: 4 target(s); worst one failing hd test, text: 318 B vs ≤ 800 Banswer-size✓ 318 B vs ≤ 800 Bdev-speed: 2 target(s); worst geomean dev / optimized: 0.99x vs ≤ 4.00xdev-speed✓ 0.99x vs ≤ 4.00xcold-check: 1 target(s); worst median cold check, 10k lines: 138 ms vs ≤ 1000 mscold-check✓ 138 ms vs ≤ 1000 msfmt: 2 target(s); worst hd fmt, 10k lines: 18 ms vs ≤ 200 msfmt✓ 18 ms vs ≤ 200 msdeterminism: 1 target(s); worst inputs with byte-identical output: metdeterminism✓ meterrors-per-run: 2 target(s); worst mixed: 10 mistakes reported once: missederrors-per-run✗ missedlookup-latency: 2 target(s); worst p95 hd callers: n/alookup-latency– n/a

Agent scalability

How much useful work do N agents do on one machine? Each bar is one metric, drawn at its worst target's distance from the goal: left of the dashed line meets it, right of it misses by that multiple. Metrics without a numeric goal show only met or missed. The full list of targets is in the table below.

goal3× or worseio-per-check: 1 target(s); worst source files read, warm check after one edit: 79 vs ≤ 1 (79.0× goal)io-per-check✗ 79 vs ≤ 1 (79.0× goal)cache-contention: 4 target(s); worst dependency cache, 8 concurrent fetches: fetched once: 16 vs ≤ 2, one check's connections (8.0× goal)cache-contention✗ 16 vs ≤ 2, one check's connections (8.0× goal)long-session: 1 target(s); worst RSS growth, REPL inputs 100 to 1000: 88.6 MB vs ≤ 22.0 MB (4.0× goal)long-session✗ 88.6 MB vs ≤ 22.0 MB (4.0× goal)disk: 1 target(s); worst artifacts plus toolchain per worktree: 15.3 MB vs ≤ 10.0 MBdisk✗ 15.3 MB vs ≤ 10.0 MBconcurrency: 3 target(s); worst p95 latency at N = 16 vs N = 1: 8.63x vs ≤ 6.00xconcurrency✗ 8.63x vs ≤ 6.00xstartup: 6 target(s); worst hd check empty.hd: peak RSS: 13.2 MB vs ≤ 10.0 MBstartup✗ 13.2 MB vs ≤ 10.0 MBparallel-speedup: 1 target(s); worst speedup, 50k lines, 4 threads vs 1: 1.91x vs ≥ 2.40xparallel-speedup✗ 1.91x vs ≥ 2.40xcache-growth: 1 target(s); worst cache size after 100 edits, cap 5 MB: 4.4 MB vs ≤ 5.0 MBcache-growth✓ 4.4 MB vs ≤ 5.0 MBresources: 20 target(s); worst warm check, 10k lines: peak RSS: 24.0 MB vs ≤ 50.0 MBresources✓ 24.0 MB vs ≤ 50.0 MBfetch-dedup: 1 target(s); worst connections to the remote, 4 worktrees: metfetch-dedup✓ met

Artifact quality

How good is the program the agents produced? Each bar is one metric, drawn at its worst target's distance from the goal: left of the dashed line meets it, right of it misses by that multiple. Metrics without a numeric goal show only met or missed. The full list of targets is in the table below.

goal3× or worseserde-throughput: 2 target(s); worst JSON encode, hd / Node MB/s: 0.06x vs ≥ 0.50x (8.3× goal)serde-throughput✗ 0.06x vs ≥ 0.50x (8.3× goal)size-startup-heap: 4 target(s); worst tiny program, start to first output: 40 ms vs ≤ 5.0 ms (8.0× goal)size-startup-heap✗ 40 ms vs ≤ 5.0 ms (8.0× goal)host-call-overhead: 3 target(s); worst Console println, per call: 7.5 µs vs ≤ 1.0 µs (7.5× goal)host-call-overhead✗ 7.5 µs vs ≤ 1.0 µs (7.5× goal)integration-test-perf: 3 target(s); worst setup per integration test program: 104 ms vs ≤ 20 ms (5.2× goal)integration-test-perf✗ 104 ms vs ≤ 20 ms (5.2× goal)text-throughput: 3 target(s); worst regex find_all, hd / Node MB/s: 0.11x vs ≥ 0.50x (4.5× goal)text-throughput✗ 0.11x vs ≥ 0.50x (4.5× goal)proptest-perf: 2 target(s); worst property cases per second: 34k/s vs ≥ 100k/sproptest-perf✗ 34k/s vs ≥ 100k/sunit-test-perf: 2 target(s); worst 1,000 unit tests, warm: 854 ms vs ≤ 1000 msunit-test-perf✓ 854 ms vs ≤ 1000 msruntime: 6 target(s); worst geomean hd / Node: 1.18x vs ≤ 1.50xruntime✓ 1.18x vs ≤ 1.50xdead-code: 2 target(s); worst Wasm bytes per 1,000 lines: 6412 B vs ≤ 10240 Bdead-code✓ 6412 B vs ≤ 10240 Bsuspension-overhead: 3 target(s); worst await of a ready value, per await: 14 ns vs ≤ 100 nssuspension-overhead✓ 14 ns vs ≤ 100 nslong-run-memory: 1 target(s); worst RSS growth after warm-up, 1 min: 0.0 MB vs ≤ 10.0 MBlong-run-memory✓ 0.0 MB vs ≤ 10.0 MBallocations: 4 target(s); worst counted loop, bytes per iteration: metallocations✓ met

Correctness gate

Is the compiler right? Each bar is one metric, drawn at its worst target's distance from the goal: left of the dashed line meets it, right of it misses by that multiple. Metrics without a numeric goal show only met or missed. The full list of targets is in the table below.

goal3× or worseconformance: 2 target(s); worst pass rate, known failures excluded: 100.0% vs ≥ 100.0%conformance✗ 100.0% vs ≥ 100.0%incremental-soundness: 1 target(s); worst incremental equals clean, after each edit: 100.0% vs ≥ 100.0%incremental-soundness✓ 100.0% vs ≥ 100.0%

All targets

Latest run: 2026-10-27 02:55 UTC. History and raw data: the metrics-data branch.

Every target, latest run
PillarMetricTargetValueGoalStatus
Single-agent development costedit-latencyp50 check after an edit35 ms≤ 50 ms✓ met
Single-agent development costedit-latencyp95 check after an edit37 ms≤ 200 ms✓ met
Single-agent development costcold-checkmedian cold check, 10k lines138 ms≤ 1000 ms✓ met
Single-agent development costtest-latencymedian edit-then-test165 ms≤ 300 ms✓ met
Single-agent development costmistakesprograms with exactly one diagnostic95.7%≥ 95.0%✓ met
Single-agent development costmistakesone diagnostic with the expected code95.7%≥ 95.0%✓ met
Single-agent development costmistakesfix-it makes hd check pass34.3%≥ 80.0%✗ missed
Single-agent development costmistakeslargest diagnostic text (bytes / 4)93 tokens≤ 60 tokens✗ missed
Single-agent development costanswer-sizehd doc of a data type153 B≤ 1000 B✓ met
Single-agent development costanswer-sizeone failing hd test, text318 B≤ 800 B✓ met
Single-agent development costanswer-sizeone failing test, JSON object139 B≤ 400 B✓ met
Single-agent development costanswer-sizeone diagnostic, JSON object170 B≤ 600 B✓ met
Single-agent development costdeterminisminputs with byte-identical output8 of 8all identical✓ met
Single-agent development costpathologicaloverlapping-from: time at 4x42 ms≤ 2000 ms✓ met
Single-agent development costpathologicaloverlapping-from: peak RSS at 4x21.3 MB≤ 200.0 MB✓ met
Single-agent development costpathologicaloverlapping-from: growth exponentnoise≤ 1.3✓ met
Single-agent development costpathologicaldeep-blocks: time at 4x16 ms≤ 2000 ms✓ met
Single-agent development costpathologicaldeep-blocks: peak RSS at 4x16.3 MB≤ 200.0 MB✓ met
Single-agent development costpathologicaldeep-blocks: growth exponentnoise≤ 1.3✓ met
Single-agent development costpathologicaldeep-parens: time at 4x15 ms≤ 2000 ms✓ met
Single-agent development costpathologicaldeep-parens: peak RSS at 4x15.9 MB≤ 200.0 MB✓ met
Single-agent development costpathologicaldeep-parens: growth exponentnoise≤ 1.3✓ met
Single-agent development costpathologicalmethod-chain: time at 4x-≤ 2000 ms✗ missed
Single-agent development costpathologicalmethod-chain: peak RSS at 4x-≤ 200.0 MB✗ missed
Single-agent development costpathologicalmethod-chain: growth exponentnoise≤ 1.3✓ met
Single-agent development costpathologicaliterator-chain: time at 4x-≤ 2000 ms✗ missed
Single-agent development costpathologicaliterator-chain: peak RSS at 4x-≤ 200.0 MB✗ missed
Single-agent development costpathologicaliterator-chain: growth exponent-≤ 1.3✗ missed
Single-agent development costpathologicalwide-list-literal: time at 4x39 ms≤ 2000 ms✓ met
Single-agent development costpathologicalwide-list-literal: peak RSS at 4x19.6 MB≤ 200.0 MB✓ met
Single-agent development costpathologicalwide-list-literal: growth exponentnoise≤ 1.3✓ met
Single-agent development costpathologicallarge-enum-match: time at 4x366 ms≤ 2000 ms✓ met
Single-agent development costpathologicallarge-enum-match: peak RSS at 4x19.2 MB≤ 200.0 MB✓ met
Single-agent development costpathologicallarge-enum-match: growth exponent2.31≤ 1.30✗ missed
Single-agent development costpathologicaldeep-generic: time at 4x18 ms≤ 2000 ms✓ met
Single-agent development costpathologicaldeep-generic: peak RSS at 4x16.3 MB≤ 200.0 MB✓ met
Single-agent development costpathologicaldeep-generic: growth exponentnoise≤ 1.3✓ met
Single-agent development costpathologicalquestion-chain: time at 4x22 ms≤ 2000 ms✓ met
Single-agent development costpathologicalquestion-chain: peak RSS at 4x18.9 MB≤ 200.0 MB✓ met
Single-agent development costpathologicalquestion-chain: growth exponentnoise≤ 1.3✓ met
Single-agent development costpathologicalbig-file: time at 4x135 ms≤ 2000 ms✓ met
Single-agent development costpathologicalbig-file: peak RSS at 4x51.7 MB≤ 200.0 MB✓ met
Single-agent development costpathologicalbig-file: growth exponent1.05≤ 1.30✓ met
Single-agent development costrecheck-precisionmodules rechecked, private body edit1≤ 1✓ met
Single-agent development costrecheck-precisionmodules rechecked, public signature edit9≤ 3✗ missed
Single-agent development costerrors-per-runsemantic: 10 mistakes reported once10 of 10all, once✓ met
Single-agent development costerrors-per-runmixed: 10 mistakes reported once2 of 10all, once✗ missed
Single-agent development costdiag-locationdiagnostics on the mistake's line98.6%≥ 95.0%✓ met
Single-agent development costfixit-safetyfix-its that add no error100.0%≥ 100.0%✓ met
Single-agent development costlookup-latencyp95 hd callers-≤ 100 ms– n/a
Single-agent development costlookup-latencyp95 hd needs-≤ 100 ms– n/a
Single-agent development costfmthd fmt, 10k lines18 ms≤ 200 ms✓ met
Single-agent development costfmtsecond hd fmt changes nothingno changeidempotent✓ met
Single-agent development costcheck-costchecked run time / unchecked run time2.84x≤ 1.30x✗ missed
Single-agent development costdev-speedgeomean dev / optimized0.99x≤ 4.00x✓ met
Single-agent development costdev-speedworst case dev / optimized1.01x≤ 10.00x✓ met
Agent scalabilityresourceswarm check, 10k lines: CPU time30 ms≤ 200 ms✓ met
Agent scalabilityresourceswarm check, 10k lines: peak RSS24.0 MB≤ 50.0 MB✓ met
Agent scalabilityresourcescheck, small, cold0.10 CPU-s, 27.0 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcescheck, small, warm0.00 CPU-s, 14.7 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcestest, small, cold1.03 CPU-s, 82.3 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcestest, small, warm0.58 CPU-s, 74.2 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcesbuild, small, cold0.42 CPU-s, 81.7 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcesbuild, small, warm0.02 CPU-s, 24.7 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcescheck, 10k, cold0.31 CPU-s, 49.1 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcescheck, 10k, warm0.03 CPU-s, 24.0 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcestest, 10k, cold1.81 CPU-s, 101.8 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcestest, 10k, warm1.08 CPU-s, 82.9 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcesbuild, 10k, cold0.64 CPU-s, 100.0 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcesbuild, 10k, warm0.04 CPU-s, 31.5 MBcompletes ≤ 30 s✓ met
Agent scalabilityresourcescheck, 50k, cold1.28 CPU-s, 146.4 MBcompletes ≤ 150 s✓ met
Agent scalabilityresourcescheck, 50k, warm0.17 CPU-s, 62.8 MBcompletes ≤ 150 s✓ met
Agent scalabilityresourcestest, 50k, cold11.43 CPU-s, 263.3 MBcompletes ≤ 150 s✓ met
Agent scalabilityresourcestest, 50k, warm9.27 CPU-s, 117.3 MBcompletes ≤ 150 s✓ met
Agent scalabilityresourcesbuild, 50k, cold1.64 CPU-s, 243.9 MBcompletes ≤ 150 s✓ met
Agent scalabilityresourcesbuild, 50k, warm0.16 CPU-s, 69.2 MBcompletes ≤ 150 s✓ met
Agent scalabilitylong-sessionRSS growth, REPL inputs 100 to 100088.6 MB≤ 22.0 MB✗ missed
Agent scalabilitystartuphd help (no version command): wall time2.9 ms≤ 20 ms✓ met
Agent scalabilitystartuphd help (no version command): CPU time0.0 ms≤ 20 ms✓ met
Agent scalabilitystartuphd help (no version command): peak RSS3.5 MB≤ 10.0 MB✓ met
Agent scalabilitystartuphd check empty.hd: wall time11 ms≤ 20 ms✓ met
Agent scalabilitystartuphd check empty.hd: CPU time0.0 ms≤ 20 ms✓ met
Agent scalabilitystartuphd check empty.hd: peak RSS13.2 MB≤ 10.0 MB✗ missed
Agent scalabilityconcurrencyp95 latency at N = 4 vs N = 12.14x≤ 1.50x✗ missed
Agent scalabilityconcurrencyp95 latency at N = 16 vs N = 18.63x≤ 6.00x✗ missed
Agent scalabilityconcurrencytotal CPU growth exponent, N = 1 to 161.11< 1.00 (sublinear)✗ missed
Agent scalabilitydiskartifacts plus toolchain per worktree15.3 MB≤ 10.0 MB✗ missed
Agent scalabilityparallel-speedupspeedup, 50k lines, 4 threads vs 11.91x≥ 2.40x✗ missed
Agent scalabilitycache-contentiondependency cache, 8 concurrent fetches: no corruptionintactno corruption✓ met
Agent scalabilitycache-contentiondependency cache, 8 concurrent fetches: fetched once16≤ 2, one check's connections✗ missed
Agent scalabilitycache-contentioncompilation cache, 8 concurrent checks: no corruptionintactno corruption✓ met
Agent scalabilitycache-contentioncompilation cache, 8 concurrent checks: written once3 filesthe files of one check's cache✓ met
Agent scalabilitycache-growthcache size after 100 edits, cap 5 MB4.4 MB≤ 5.0 MB✓ met
Agent scalabilityio-per-checksource files read, warm check after one edit79≤ 1✗ missed
Agent scalabilityfetch-dedupconnections to the remote, 4 worktrees2= the first worktree's✓ met
Artifact qualityproptest-perfproperty cases per second34k/s≥ 100k/s✗ missed
Artifact qualityproptest-perftime to shrink a known failure107 ms≤ 1000 ms✓ met
Artifact qualityunit-test-perf1,000 unit tests, warm854 ms≤ 1000 ms✓ met
Artifact qualityunit-test-perfoverhead per unit test0.7 ms≤ 1.0 ms✓ met
Artifact qualityintegration-test-perfsetup per integration test program104 ms≤ 20 ms✗ missed
Artifact qualityintegration-test-perftotal time growth in programs0.994exponent < 1 (sublinear)✓ met
Artifact qualityintegration-test-perfcost per doc test90 ms≤ 20 ms✗ missed
Artifact qualityruntimegeomean hd / Node1.18x≤ 1.50x✓ met
Artifact qualityruntimefib: hd / Node0.59x≤ 3.00x✓ met
Artifact qualityruntimesum: hd / Node1.47x≤ 3.00x✓ met
Artifact qualityruntimestring-build: hd / Node1.27x≤ 3.00x✓ met
Artifact qualityruntimemap: hd / Node1.88x≤ 3.00x✓ met
Artifact qualityruntimesort: hd / Node1.11x≤ 3.00x✓ met
Artifact qualityallocationscounted loop, bytes per iteration0.00 B/iter0 allocations (< 1 B/iter)✓ met
Artifact qualityallocationslist loop, bytes per iteration0.00 B/iter0 allocations (< 1 B/iter)✓ met
Artifact qualityallocationsmap loop, bytes per iteration0.00 B/iter0 allocations (< 1 B/iter)✓ met
Artifact qualityallocationsiterator chain, bytes per iteration0.00 B/iter≤ 1 allocation (≤ 32 B/iter)✓ met
Artifact qualitysize-startup-heaptiny program, release Wasm size4452 B≤ 2048 B✗ missed
Artifact qualityhost-call-overheadConsole println, per call7.5 µs≤ 1.0 µs✗ missed
Artifact qualityhost-call-overheadFs read_text!, per call19 µs≤ 10 µs✗ missed
Artifact qualityhost-call-overheadserde boundary (list_dir!), per call34 µs≤ 5.0 µs✗ missed
Artifact qualitysuspension-overheadawait of a ready value, per await14 ns≤ 100 ns✓ met
Artifact qualitysuspension-overheadall! of two ready tasks, throughput9.6M/s≥ 1.0M/s✓ met
Artifact qualitysuspension-overheadrace! of two ready tasks, throughput8.4M/s≥ 1.0M/s✓ met
Artifact qualityserde-throughputJSON encode, hd / Node MB/s0.06x≥ 0.50x✗ missed
Artifact qualityserde-throughputJSON decode, hd / Node MB/s0.10x≥ 0.50x✗ missed
Artifact qualitytext-throughputstring build, hd / Node MB/s1.45x≥ 0.50x✓ met
Artifact qualitytext-throughputsplit, hd / Node MB/s2.25x≥ 0.50x✓ met
Artifact qualitytext-throughputregex find_all, hd / Node MB/s0.11x≥ 0.50x✗ missed
Artifact qualitydead-codeWasm bytes per 1,000 lines6412 B≤ 10240 B✓ met
Artifact qualitydead-codeno unused std in the programnonenone✓ met
Artifact qualitylong-run-memoryRSS growth after warm-up, 1 min0.0 MB≤ 10.0 MB✓ met
Correctness gateconformancepass rate, known failures excluded100.0%≥ 100.0%✓ met
Correctness gateconformancelisted known failures5≤ 0✗ missed
Correctness gateincremental-soundnessincremental equals clean, after each edit100.0%≥ 100.0%✓ met
Artifact qualitysize-startup-heaptiny program, start to first output40 ms≤ 5.0 ms✗ missed
Artifact qualitysize-startup-heappeak heap, map: hd / Node0.96x≤ 2.00x✓ met
Artifact qualitysize-startup-heappeak heap, string-build: hd / Node0.37x≤ 2.00x✓ met