Skip to main content
Run native measurements in release mode:
The runner generates 200 TypeScript functions and renames one function. It builds the release CLI, runs one warmup and 20 measured processes, and writes target/benchmarks.json. Timing includes startup, file reads, parsing, matching, narration, and full standalone JSON serialization. Output must stay deterministic. Other workloads need their own measurements.

Fair comparisons

Use the same input bytes, parser grammars, semantic budgets, worker count, output mode, and output sink. Record fallback counts and output quality alongside latency. A faster run that reports fewer changes or falls back more often has not demonstrated an equivalent improvement. Separate cold parser/cache startup from warm repeated parsing. Measure Git discovery/blob reads, parsing, CST conversion, matching, correlation, narration, serialization, and stdout when diagnosing end-to-end latency. Include all processes in legacy memory measurements. Use repeated samples on a quiet machine, reporting median, p95, and spread. Cover tiny invocations, every dialect, many-file changesets, wide/deep trees, repetitive syntax, rewrites, large prose/raw inputs, binary bytes, and correlation-heavy moves. Hard performance gates need a stable runner.

Baseline and claims

Frozen legacy fixture provenance lives under fixtures/parity; benchmark metadata/results live under benchmarks. The older single-run library timings are not comparable to the repeated process measurements below. Native speed and memory claims require reproducible release-build measurements at the same output quality. No fixed multiplier is promised. The core has no default size cap. Tier analysis and sequence work have budgets; reports expose fallbacks. Peak memory includes parser caches, in-flight windows, outputs, and retained correlation candidates, not only the node arena.

Migration measurement

benchmarks/results/native-comparison.json records one local macOS arm64 comparison against legacy revision 66f292a15bd494d30ff2311ecc02868262dae870. Both files total 29,163 bytes. Each runtime had one warmup and 20 measured processes, using rotating runtime order and warm filesystem caches. All runs produced exactly equal decoded JSON, including full action trees. This case was about 7.2 times faster than Bun and 10.4 times faster than Node. It measures one small standalone comparison, including the removed legacy Git probes for absolute file paths. It does not establish those gains for repository scans, every language, peak memory, or cold disk reads. Grammar initialization occurs in every new process; the process-local parser cache is not reused. To reproduce the baseline, use a separate checkout of the recorded legacy revision with its locked dependencies. Generate inputs using the native benchmark runner’s input generator in xtask/src/benchmark.rs, preserve the same absolute paths, and compare each legacy invocation against the release binary with --format=json. The Node baseline bundles the CLI for Node while retaining its native Tree-sitter modules. Never compare the earlier single-shot library timers with process measurements.

Real repository audit

cargo xtask audit-repos --repos-dir target/audit-repos --fetch --samples 7 checks pinned Express, Flask, ripgrep, chi, and Prettier histories. The manifest is benchmarks/real-repos.json. The audit verifies all 99 changed paths, expected named changes, selected LLM facts, JSON/NDJSON agreement, and equal results with automatic, one, and four workers. It also saves all three human formats for review. The paired results in benchmarks/results/real-repos.json compare the initial native migration at 7b7301c with the audited native implementation. These are Rust-before/Rust-after measurements, separate from the JavaScript comparison above. Each binary gets one warmup and seven measured processes with rotating order and warm filesystem caches; JSON stdout is captured. Repository code and dependency scripts are never run. All five comparisons had zero fallbacks. These timings were measured on macOS arm64; raw samples and machine metadata are retained with the results. Corrected line alignment reduces raw action counts for the large history/raw text inputs in Express and Prettier. The path and format checks still pass; smaller action counts are not themselves evidence of equivalent output quality. The chi audit requires all four changed receivers and rejects a phantom update inside its newly added helper. The ripgrep audit rejects false function moves caused by repeated comments and attributes. Full old/new directory snapshots and a Git working tree patched to the exact new tree were also compared for Express and chi. Their decoded JSON, including order, moves, and fallback metadata, matched the commit-range reports exactly.

Isolated stress cases

benchmarks/results/rust-audit-stress.json records component measurements collected during the audit, with source revisions, sample counts, and scope. Repeated subtrees and deep anchored chains exposed quadratic matching; indexed name lookup removed repeated candidate scans. Buffered spill I/O fixed excessive small writes and reads. Compact detached Unicode maps retain only relevant source boundaries. These synthetic cases target specific defects and do not predict ordinary repository speedups. The first two exclude tree construction and parsing. Detached metadata for a tiny node after 100,000 non-ASCII characters fell from 1,600,064 to 80 bytes, while preserving the legacy JSON projection. Those figures describe that retained metadata, not peak process memory.

Run and inspect native measurements

For example, a local macOS arm64 run with --samples 7 produced the following selected fields (times rounded to two decimals):
These numbers describe that run; use your own samples to assess your machine or compare implementations. The runner prints a JSON result and saves the same metadata, all measured samples, input sizes, platform, and measurement scope. Median and p95 are elapsed process times; the warmup is excluded. Use --samples with at least two samples. For a real-repository audit, including named-change and output checks:
The manifest pins source revisions and expected reports. It fetches Git objects; it does not install or execute the audited projects. See the contributor guide for the workspace checks.

What tends to dominate

A line diff answers a different review question and avoids semantic parsing. Use both when you need exact changed text alongside a structural summary. The recorded native/legacy measurements above replace the old Bun-only single-run figures; they do not establish a universal speed or memory ratio.