Thursday, October 29, 2026 · Windows 11 10.0.26200

Dogfood on win-x64: Pluggable-backend author

5 repos · 29 commands · 3 broken · 5 slow · 4 filed

As a pluggable-backend author on win-x64 I wrote a scratch grit-lib program against the public API only: a from-scratch HashMap ObjectStore and a per-ref-lock RefStore (both pass the shipped conformance suites), then ran commit/switch/merge/conflict/stash/log on them, differentially against a files repo and MemoryStore+MemoryRefStore, exported to system git (fsck --strict clean, SHA-1 and SHA-256), plus scaling, concurrency, commit-graph, reftable and Windows checks. Correctness and parity are good; the findings are performance cliffs in the ref-transaction layer (quadratic RefTransaction build, O(all refs) per check_transaction, reftable stack never compacted), a leaky git_dir-based refs API, and two fixed issues (#2257, #2181) that still reproduce. Filed #2279, #2281, #2282, #2285; reopened #2257 and #2181; confirmed #2184, #2264, #2269, #2263 fixed.

os: Windows 11 10.0.26200 · x86_64
machine: win-x64 (self-hosted worker): Intel Core i5-12400, 16 GB RAM, NTFS
versions: grit 0.5.24 (workspace version; built from source) · main b0617c6 · 2.55.0.windows.5

repositories5

repo	files	commits	.git	notes
octocat/Hello-World	1	3	0.1 MB	smoke tests, JSON/markdown output, concurrent branch creation
rust-lang/cargo	3,078	23,596	80.7 MB	log/status/diff/blame/rebase timings; 74 MB pack ingested into a custom store
scratch/hist-120k	0	120,000	25.7 MB	bare repo from git fast-import, merge every 10th commit, commit-graph written; commit-graph benchmark
scratch/wide-200k	200,000	1	8.7 MB	bare repo with one 200,000-entry directory for #2269 (tracked in tree, no worktree)
scratch/reftable-and-loose-ref repos	300	300	1 MB	reftable repo written by grit (300 commits), 100k loose-ref bare repo, case/reserved-name repos

commandsgrit vs. system git, wall time

command	repo	grit	git	grit/git	result	notes
grit-lib: custom MapStore + LockedRefStore everyday scenario (stage, commit, branch, switch, merge, stash, rev_list)	scratch	—	—	—	ok	Identical trees/status/stash to files backend; git fsck --strict clean; SHA-1 and SHA-256.
grit-lib: refstore_conformance::run_suite / odb_conformance suites on custom stores	scratch	—	—	—	ok
grit-lib: create_branch x100 per thread, 16 threads, MemoryRefStore	scratch	922 ms	—	—	wrong	34/1600 spurious LockHeld; 32 threads 352/3200 (#2181).
grit-lib: create_branch x16000 single thread, MemoryRefStore	scratch	67.14 s	—	—	slow	quadratic (4.2 ms avg); 1000: 441 ms, 4000: 4.5 s; files backend linear (#2282).
grit-lib: RefTransaction::push_update x100k then prepare+commit (MemoryRefStore)	scratch	21.16 s	—	—	slow	build of the batch is quadratic; prepare+commit of 30k is 68 ms (#2281).
grit-lib: backend::check_transaction one create, 100k refs in state	scratch	13 ms	—	—	slow	138 us at 1k refs, 38 ms at 300k (#2282).
grit-lib: 1000 single-ref CAS updates on reftable store	scratch/reftable	37.20 s	2.53 s	15×	slow	1002 tables vs git 3 (#2279).
grit-lib: 500 CAS updates of existing loose refs (FilesRefStore)	scratch	2.34 s	384 ms	6.1×	slow	median 1.2-4.7 ms per update (noisy) vs git update-ref --stdin 0.77 ms and raw lock+rename 0.26 ms; creates are fine (0.6 ms). Not filed: bimodal and noisy on this machine.
grit-lib: rev_list(HEAD) 120k commits, custom store with attached commit-graph	scratch/hist-120k	206 ms	62 ms	3.3×	ok	Equal to files backend (208 ms); git rev-list --count with graph 62 ms (process inclusive) so ~3x git for the library walk, not custom-store specific.
grit-lib: merge_bases_all(HEAD, side) 120k commits, custom store with graph	scratch/hist-120k	27 ms	50 ms	0.5×	ok
grit-lib: build_commit_graph (changed paths) 120k commits	scratch/hist-120k	481 ms	—	—	ok
grit merge-base main side	scratch/hist-120k	71 ms	50 ms	1.4×	ok
grit revs side..main -n 1	scratch/hist-120k	96 ms	48 ms	2.0×	ok	2x git (no early exit on -n).
grit log --oneline -n 30000	rust-lang/cargo	222 ms	224 ms	1.0×	ok
grit log --oneline -n 30000 -- Cargo.toml	rust-lang/cargo	149 ms	133 ms	1.1×	ok
grit status	rust-lang/cargo	102 ms	44 ms	2.3×	ok	2.3x git; under the 3x bar.
grit diff HEAD~50	rust-lang/cargo	14 ms	90 ms	0.2×	ok
grit blame Cargo.toml	rust-lang/cargo	117 ms	151 ms	0.8×	ok
grit branch	rust-lang/cargo	9.2 ms	25 ms	0.4×	ok
grit show HEAD	rust-lang/cargo	10 ms	26 ms	0.4×	ok
grit merge-base HEAD~500 HEAD~700	rust-lang/cargo	28 ms	43 ms	0.7×	ok
grit revs HEAD~2000..HEAD --count	rust-lang/cargo	106 ms	102 ms	1.0×	ok
grit rebase master (60 commits onto +3641)	rust-lang/cargo	3.41 s	2.73 s	1.2×	ok	single run; #2263 fixed.
grit cat HEAD:big (200k entries)	scratch/wide-200k	68 ms	345 ms	0.2×	ok	#2269 fixed.
grit-lib: ingest_pack_into 74 MB / 195k-object cargo pack into MapStore	rust-lang/cargo	9.70 s	2.71 s	3.6×	ok	git index-pack comparison is not like-for-like (no copies in memory); RSS ~1.15x the 1.48 GB uncompressed data, expected for an in-memory store.
grit init --ref-format reftable + 300 grit commit	scratch/reftable	—	—	—	wrong	314 table files remain (git: 4 after 30 commits); readable by git 2.55 (#2279).
grit branch aux\|con\|com1\|lpt1\|Aux.x ; grit tag con	scratch/windows-refs	—	—	—	wrong	creates refs git cannot read (#2257 not fixed); grit branch nul -> raw 'I/O error: Incorrect function (os error 1)'; grit fsck reports clean while git fsck errors.
6 parallel jobs x 60 grit branch + concurrent git for-each-ref loop	octocat/Hello-World	—	—	—	ok	0 failures in 360 creations / 406 git reads; no stale locks; git fsck clean.
grit status\|log\|branch\|merge-base\|reflog --json / --markdown	octocat/Hello-World	—	—	—	ok	All --json outputs parse; markdown renders.

worked7

›The pluggable ObjectStore and RefStore traits plus the backend:: helpers and shipped conformance suites made a from-scratch backend small and correct on the first try
›Commit, branch, switch, merge (clean and conflicting), abort, stash push/pop and rev-list on injected stores produce byte-identical trees and identical status to a files repo, and git fsck --strict is clean on the exported objects (SHA-1 and SHA-256)
›OdbBuilder::commit_graph with build_commit_graph gives a custom store walk speed equal to the files backend on a 120k-commit history
›RepositoryBuilder docs correctly describe the minimal on-disk footprint (config, index, COMMIT_EDITMSG)
›Reftable repos written by grit are readable by git 2.55; loose-ref concurrency between grit processes and git readers on NTFS showed no lock or Access-denied failures
›CLI: log/diff/show/branch/blame/merge-base on cargo are at or better than git; grit cat on a 200k-entry tree 5x faster than git ls-tree; rebase on cargo within 1.25x git
›Case-only branch and tag collisions on NTFS are rejected like git

problems11

›#2279 filed: reftable stack never compacts (1000 updates 37 s vs git 2.5 s, 1002 tables)
›#2281 filed: RefTransaction::push_update is O(n), 100k-ref batch build takes 21 s
›#2282 filed: backend::check_transaction / MemoryRefStore commit are O(all refs) per transaction; create_branch x16000 takes 67 s
›#2285 filed: public refs::*(git_dir) functions bypass the injected RefStore and split-brain (write_symbolic_ref creates a stray HEAD file; examples use that form)
›#2257 reopened via verify_fix: reserved-name refs still created on Windows; additionally grit fsck says clean and grit branch nul gives raw os error 1
›#2181 reopened via verify_fix: 2% (16 threads) to 11% (32 threads) LockHeld on MemoryRefStore, core.filesRefLockTimeout ignored for injected stores
›Not filed (noisy): FilesRefStore CAS update of an existing loose ref on NTFS median 1.2-4.7 ms vs git 0.77 ms and raw lock+rename 0.26 ms (creates 0.6 ms); bimodal, likely AV or replace path, needs a profiler
›Not filed (minor): grit-lib docs say flat paths such as grit_lib::reflog and grit_lib::rev_parse remain valid but they are pub(crate); reachable only as refs::reflog, revwalk::rev_parse, objects::hash
›Not filed (minor): Windows build of grit-lib emits unused-import and unreachable-code warnings (atomic_replace.rs, split_index.rs, history/apply.rs 'symlink restore requires unix')
›Not filed (design note): rev_list(HEAD) over 120k commits is ~3x git rev-list --count even with a graph on both backends (206 ms vs 62 ms), grit revs side..main -n 1 does not exit early (97 ms vs git 48 ms)
›Not filed (duplicate): ref-name rules differ between MemoryRefStore (accepts Feature and feature, aux) and the files backend on NTFS; #2198/#2185-class semantics gaps remain for backends that are not case-folding

bugs filed4

#2279 #2281 #2282 #2285already reported: #2189 #2128 #2127 #2131 #2249 #2185 #2187 #1739 #1812

next time

›Run the Windows ref-update latency investigation with a profiler (ETW/WPA) to explain FilesRefStore replace-existing 1.2-4.7 ms vs 0.26 ms baseline, with Defender exclusions on and off
›Re-run the 16-32 thread battery after #2181/#2282 are fixed and with a per-ref-lock injected store, adding reflog expire running concurrently with commits (replace_reflog is still not compare-and-swap)
›Fetch/clone/push into custom stores through ingest_pack_into glue against a real git daemon and smart HTTP (thin packs, shallow), and the reverse direction: pushing from a custom store via build_pack to git receive-pack
›Linked worktrees, sparse checkout, hooks (GIT_DIR semantics) and submodules on injected stores (#2127 open)
›Promisor/partial clone with a custom primary and a failing read source; read-through cache layering with latency injection
›Re-check #2257 after the fix: names in packed-refs, fetch/clone of refs named aux/con from a Linux remote, and the Windows 8.3 and trailing-dot spellings
›Reftable: compaction fix, then geometric-stack behaviour with concurrent grit and git writers on NTFS and reftable refs with a custom ODB
›A side store for changed-path Bloom filters on custom stores and log -- path on 100k+ commits (graph attached with changed_paths=true was built, but query speed was not measured)