tools/system_link_bots.py joins a native-build host with many lightweight
machines, each with one player, speaking the system link protocol directly:
join, settings (reassembling the fragmented record), player, map precached,
loaded, then a UDP input message each tick that acknowledges the host's
updates, as a real client keeps a lockstep game running. Each machine binds
its own loopback address (127.0.0.2 and up), since the host tells machines
apart by address.
python tools/system_link_bots.py --machines 127 --start
fills a 128-machine session on the host at 127.0.0.1 and starts it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Integrate verified source and data progress; exclude unproven COMMON ownership, held UB/asm forms, and address-named data credit. Stable sweep: 23 exact functions gained, zero lost.
The old ranking was wrong in a way that buried the cheapest work. alndiff's
region count systematically OVERSTATES the closest functions, because it counts
pure addressing noise as divergence: relocation-target spelling, the relocation
SITE address (which shifts whenever anything upstream changes length), branch
immediates, and trailing alignment NOPs. relocdiff scores all of those `=`.
Stripped, the picture changes completely:
_actor_path_refresh 12 reported -> 0 REAL
_actor_look_idle_find_prop 16 reported -> 1 REAL
_actor_perception_friend_prop_is_attacking 3 reported -> 1 REAL
_actor_situation_update 23 reported -> 2 REAL
residual_triage.py puts frame, reference multiset, real code end and REAL
regions in one row per residual and ranks cheapest-first. It takes a directory,
so bare it covers all of source/ai (51 residuals, 96,032 bytes); the committed
table is Lane A's eleven units (30 residuals, 75,392 bytes).
The top of the table is a cluster of eight functions at three or fewer real
regions, about 8,200 bytes. Two had never been probed and are decoded in the
write-up:
_actor_path_refresh (1,440 B, 0 real regions) - the whole residual is THREE
branches at 4 bytes each. January encodes them short to the NEAR copy of the
return epilogue; we encode near to the FAR copy 0x267 bytes back. Both copies
exist in both builds, only the binding differs. Source site at
actor_moving.c:2064 and a suggested lever recorded in scratch.
_actor_look_idle_find_prop (608 B, 1 real region) - January emits
`cmp ebx, dword ptr [_ai_debug+0x38]`, we emit a load then a compare. The
source is already the natural form at both of its sites and the split appears
where the scheduler fills an fcos latency gap, so it is instruction selection
under x87 scheduling. Also 3 bytes short overall, so a branch-width difference
is hiding behind the normalisation.
Also decoded and recorded: _actor_situation_update (2 real regions, one
five-instruction block January places out of line and we place inline) and
_actor_perception_friend_prop_is_attacking (1 real region, an AL/CL
two-colouring; three guard spellings measured, all dead).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found the tool's own blind spot by using it. It compared relocation SITE
addresses (`rel+0x...`) and branch immediates as if they were content, so any
upstream length change made every downstream row look like a divergence.
_actor_path_refresh read 9 REAL regions when its instruction text is in fact
identical throughout.
Now stripped: the row address, the relocation-target spelling, the relocation
site address, and branch/call immediates. Trailing alignment filler is counted
separately instead of appearing as a twelve-NOP "deletion". Output splits into
REAL / addressing-only / padding-only.
_actor_path_refresh 12 reported -> 0 REAL
_actor_perception_friend_prop_is_attacking 3 reported -> 1 REAL
And the caveat is now in the docstring rather than left to be rediscovered:
normalising a branch immediate also hides SHORT vs NEAR encoding, which is a
real byte difference. `0 REAL` means every instruction's TEXT matches, NOT that
the function is exact. _actor_path_refresh is the worked example - content
identical, our encodings about 12 bytes longer, January's 12 trailing NOPs
absorbing it into the same section size. `0 REAL` plus a padding-only region
means the gap is encoding length: go compare where real code ends.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
alndiff prints a region whenever its two normalized instruction texts differ,
and that includes a pure PRESENTATION difference in how the two objects spell a
relocation target - `target=symbol:__real@0` against
`target=defined-noncode:.rdata:__real@0`. Same address, same type, same target;
relocdiff --allow-structural scores that row `=`. It is not a divergence.
Over the Lane A AI portfolio, 86 of 860 reported regions (10%) are that noise -
but it is not spread evenly. It is concentrated on exactly the functions that
are CLOSEST, where the ratio is worst:
_actor_perception_friend_prop_is_attacking 3 raw -> 1 real
_code_00039990 11 raw -> 2 real
_ai_communication_get_player_rating 12 raw -> 2 real
_actor_emotion_update 12 raw -> 3 real
_ai_communication_actor_talk_weight 9 raw -> 3 real
_actor_perception_update 30 raw -> 14 real
A function reported at "12 regions" that is really at 2 is one a triage pass
will wrongly deprioritise, so this systematically hides the cheapest work.
Read-only, recompiles nothing, runs in no build step. --show prints the
surviving regions so you can go straight to the real divergence.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fourth read-only diagnostic, generalizing a technique a wave agent invented for
one function. Once you have decoded a source-level lever you usually know the
byte pattern it emits; count that pattern per function in both trees and every
disagreement is another instance of the same lever - found without recompiling
anything.
The built-in `square` set is the worked example. VC7 has two shapes for
multiplying a value by itself, decided by the source's expression form:
d9 c0 d8 c9 fld st(0) ; fmul st(1), st square the top of stack
d9 c1 d8 ca fld st(1) ; fmul st(2), st square the deeper operand
Counting them gives 224 functions carrying the idiom in both trees, 206
agreeing and 18 DIFFERING - a precise worklist spanning AI, physics, bitmaps,
interface, structures and units, including _physics_update_old (6,2) vs our
(9,0) and _actor_perception_update (2,2) vs our (1,0).
Two caveats are built into the output rather than left to be rediscovered.
"Only in ours" is 385 rows and is NOT a signal: it is dominated by the known
extra-COMDAT copies of header __inline helpers that January's split objects do
not carry. And a byte pattern can occur inside an embedded jump table, so a
single-count disagreement on a function with one deserves care.
Takes --set (square, fdiv, fild, fistp), --bytes for an arbitrary hex pattern,
and --unit to restrict the scan. Reads build/split and build/base, writes
nothing, recompiles nothing, runs in no build step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fourth trap in the same family, and the largest. A relocation's target NAME is
not its target: the same address can be spelled against different symbols with
different addends. January writes `_sense_ray_divergences+32` where we write
`_sense_ray_angles-4`. The comparator resolves that; comparing names does not.
_actor_move_initialize - which gate.py reports EXACT and relocdiff
--allow-structural reports as `sha equal, 0 differing rows` - leaked into the
structural tier purely on name differences. Making the exactness test "equal
normalized bytes AND equal relocation (address, type) list", with names
deliberately excluded because the comparator resolves them, removed 123 more
already-exact functions:
non-exact functions 423 -> 300
S1 184 -> 73 (151,253 -> 108,533 bytes)
T 107 -> 95 ( 90,128 -> 86,896 bytes)
T parked 67 -> 55 - the twelve csplit-relocation-alias parks
correctly vanish, since those functions
ARE exact after alias resolution, which
is exactly what that park class records
S2 is unchanged and so is the headline: 114 parked functions, 112,133 of
151,525 parked bytes, are not tie-shaped. That number has now survived four
independent corrections.
VALIDATED AGAINST THE GATE. On Lane A's eleven translation units the census now
reports exactly 33 non-exact functions and gate.py reports exactly 33 residuals,
with no disagreement in either direction. That cross-check is what caught both
this error and the previous one, and it is recorded as the first thing anyone
extending the tool should run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Third cheap board-wide diagnostic. The data gap is much larger than the code
gap - halobetacache matches 60.2% of data bytes, about 1.56 MB outstanding,
against roughly 326 KB of code - and nobody had asked whether that is a pool of
cheap near-misses or work not yet done.
Over 4,010,867 data-COMDAT bytes in 524 objects:
byte-identical after normalization 1,975,566 49.3%
symbol ABSENT from our build 2,015,060 50.2% 1,681 symbols
present but not identical 20,241 0.5% 13 symbols
And of those thirteen, ALL 13 are SIZE mismatches and ZERO differ only in
contents. Every data symbol we emit at the right size is byte-exact. There is
no pool of nearly-right tables to polish: the data gap is 1,681 unwritten
symbols plus 13 arrays or structures with the wrong element or member count.
That is a negative result for anyone hoping data was cheap, and a precise
thirteen-row work-list for the owners of those objects. Several read as one
missing row: _rasterizer_vertex_type_sizes is 26 vs 24 bytes of .rdata, i.e. 13
shorts against our 12 - one missing enumerator; _observer_maximum_accelerations
is 7 reals against our 6; _global_effect_marker_list is one byte-sized member.
Four run the other way - our .bss global is 1 to 3 bytes LARGER - all four in
the Xbox rasterizer, which is the padding signature of a wrong member type.
The part worth chasing: a .bss or .rdata size mismatch is a TYPE DEFINITION
defect in a shared header, and the same definition feeds the code. Three of the
twelve distinct objects also carry a frame-mismatched code residual, all three
parked `unclassified`: observer (+12), effects (-4), leaf_map (+20). Stated as
a correlation rather than a proof - three of twelve, and the frame table
truncates long unit names so the true count may be higher - but the mechanism
is plain, and anyone reopening one of those parks should fix the type first.
None of the thirteen is in Lane A's scope; none was touched. Measurement only.
tools/campaign/data_census.py added read-only beside the other two censuses.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both research documents committed earlier reference scripts in scratch/, which
the repository excludes - the write-ups would have pointed at files that do not
survive. Copied them to tools/campaign/ with headers explaining what they are,
what they measure and where the companion write-up is.
Scope: these ADD two read-only diagnostics beside the analysis scripts that
already live there (board.py, flag_scan.py, alias_scan.py and friends). They
read build/split and build/base, write nothing, recompile nothing, are imported
by nothing and run in no build step. No existing tool, comparator, split target,
object or build file is modified, so the lane's "no build-tool modification"
constraint is untouched. If the owner would rather they did not live here,
deleting two new files breaks nothing.
frame_census.py 56 functions with a `sub esp,N` mismatch, 101,056 B,
35 of them parked
structural_split.py 423 non-exact functions split S1/S2/T; 114 parked
functions (112,133 of 151,525 parked bytes) are not
tie-shaped
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MrBruh