Cesarus85--mjolnir-vr/docs/object_matching_logs/data_encoding_obj.md

data_encoding.obj exact-match strategy log

Current disposition (2026-08-11): strict-exact and admitted. The historical fixed-point analysis below is retained because it documents the 17 rejected source families that preceded the provenance breakthrough.

Scope and provenance

• Translation unit: source/memory/data_encoding.c
• Target: build/split/source/memory/data_encoding.obj, split from the January 2002 cachebeta.exe
• Candidate: build/base/source/memory/data_encoding.obj
• Compiler: Xbox XDK 3911 VC7 (xbox/bin/vc7/CL.Exe, version 13.00.9254.1)
• Required flags: /nologo /c /O2 /Oy- /DDEBUG /Dxbox, with the repository include paths generated by configure.py
• Starting revision: codex/campaign-integration at 8cf2d5444
• Comparator: tools/coff_compare.py at 8cf2d5444, including the hardened pairwise relocation resolver for code and non-code sections
• Target topology evidence: the January public symbols and addresses recorded at the head of data_encoding.c; older source/PDB material is a hypothesis source only and does not override the January object.

The isolated lane was configured from the repository generator. It uses the same checked-in source/configuration, shared read-only XDK 3911 junction, local cachebeta.exe, csplit.exe, and objdiff-cli.exe; no compiler flag or target object was changed.

Validated baseline

Command:

ninja build\base\source\memory\data_encoding.obj build\split\source\memory\data_encoding.obj
python tools\coff_compare.py build\split\source\memory\data_encoding.obj build\base\source\memory\data_encoding.obj <all 16 function symbols>

The hardened baseline is 15/16 strict exact, with 3,296/3,296 padded function bytes. All 14 target-owned non-code sections are strict exact: 459/459 .rdata bytes, zero relocations. The candidate additionally emits the buffer, buffer_size>=0, and count>=0 assertion literals in separate COMDATs; csplit does not assign those deduplicated literals to this target object, so they are compiler metadata/ownership differences rather than owned target data.

Ordinary objdiff reports _data_decode_memory at 66.56% in the local objdiff 3.3.1 build (earlier campaign handoff: approximately 66.64%). Ordinary objdiff is diagnostic only; strict status comes from the hardened comparator.

Function inventory

Function	Padded size T/B	Relocs T/B	Normalized SHA-256	Status
_data_encode_new	112/112	9/9	8768c68e1dd89e978d0ae821e3599c950e6bc795d64bc23baef5d32486755250	exact
_data_encode_memory	304/304	17/17	aeaf7eb5fddeb848adc29cfd6c46546c90c32c0981b02f0f430cd511286b7aaf	exact
_data_encode_integer	256/256	10/10	a960315a739a4230040a899eee18700f8685df0ca0d4111043e68e8e9b2cdf79	exact
_data_encode_structures	256/256	14/14	d0e427473335d5a915239ee2e3230b578142375541b1afda95fa477ad302ce45	exact
_data_encode_array	576/576	43/43	7e3569e19e98be057aeb330fe1973dfc2afe7e13e8e49f66c61f8ed1d4971b6b	exact
_data_encode_string	160/160	6/6	c18aa3a96c57cd1d10274595fc7573898f2e7f77c8e58f574b567351a692405b	exact
_data_decode_new	112/112	9/9	f0dea9eac3ff40d6f0da3dc2ead78ee450e3b920a592869c288c27cf1bb7a78d	exact
_data_decode_structures	240/240	13/13	0db6222030d5655dfaada7a01714785744db752226b2d6bd63a3c1b9703e1033	exact
_data_decode_memory	304/304	19/19	T eb39377415e6c3e07f763e8f0f42f6ecafaf61acc0fab30fbc06062b9dc465d8; B 9c154cab347995fa1eb87ee6b088615f4347de7947058d8336212577bfbc3cc7	residual
_data_decode_byte	112/112	4/4	91f5ba7bc99766592386506f438136daff739b16a34d0151810d759d89122d96	exact
_data_decode_short	128/128	5/5	6cc7704a74a9b48b8af5b860054bbe2ee2bee0b06fb8dc7a4a30e4611ef34118	exact
_data_decode_long	128/128	5/5	276c5b1dd2ccedc8e0807a589a4d11a699b213b4468a60d5a2ffca02bd540320	exact
_data_decode_int64	48/48	1/1	c805dccfc68c675dcfa56b15147f6009a559a3a433d23df1ac21097e37960e4d	exact
_data_decode_integer	112/112	7/7	2f9457b0cc9adc7f20a16b38881f7d52612e1b24ce15a4ddd754e85b3de0766e	exact
_data_decode_array	368/368	31/31	93eb0427d2cb6c8818cdccb5031169e42c9a2461c00022f6dc29343ead57d74c	exact
_data_decode_string	80/80	0/0	a19f040eff69a02e0703f67d49921b998501ed6e80d63327f349cbee99a1515c	exact

Owned non-code inventory

All rows are .rdata, have equal target/base size, zero relocations, and strict-equal normalized hashes.

Owner literal (abbreviated where mangled)	Size	SHA-256
source path c:\halo\SOURCE\memory\data_encoding.c	38	ac45863b58782998f06cb76da1351037cf0159cf84cfec4bd847608fee673c7c
encoder state assertion	79	4c493fd06ea83ac5ad038620f75e473ad22e9ce29b855d6ee1ca403b382f0f0c
maximum_value>0	16	882e4a931d94220788dcec61f1eba5637173959aa0f731442bb8c3d272175a7c
bs_definition	14	eba14beb7acb02e162f68248bed2808506799bb99b058f968196e698270f0a6c
source_structures	18	184b611e452e730136a62c03caae0f6a95c0a06e3fe4a44fc102205d73bcee76
element_count<=UNSIGNED_SHORT_MAX	34	8cf0d82bca902627da4f2f1a6698e6dc974e96718e775c291e07b0dc49565196
element_count<=UNSIGNED_CHAR_MAX	33	16178348f414a11c23324da5743a39b6b3ed634afa7c8a7ed25c54f35ebde403
element_count>=0	17	74e3fcb64e369c70c9d31a2c84f4c9b16504e31f50efc7aa1e560d68b9f734bb
source_array	13	39cdb9d7009172e6efa42cf694dedd287c41eac4467a37afb9ce41af15cb22f9
string-buffer bounds assertion	50	c93b160e3929d1b5c6af9360f6462f200045d8a87cab4e344bb063486cfec125
structure_count>=0	19	4503f200e32141fa94a71eab30a9e2007d290f3962461e83e9f9a2e33271ea9b
decoder state assertion	80	fb5b4e55dd29f1c0c7ca225e0a1bf51b1cfadc7b58b764967d378665f186c2e3
maximum_element_count>0	24	2825bf6555d95f6799775dbb605de1404e3f1470c603665d7bff2144bb5b5e77
element_count_reference	24	8be5ee77e570517b19ff1edfaa15cbea510c24a9f7311f44982721b8e31b4f60

Residual detail

• Meaningful target code size: 295 bytes; padded section: 304 bytes.
• First normalized byte divergence: +0x4.
  • Target begins push ebp; mov ebp,esp; push ebx; push esi; mov esi,[ebp+8]; push edi; xor edi,edi.
  • Baseline begins push ebp; mov ebp,esp; push ebx; mov ebx,[ebp+8]; push esi; xor esi,esi; push edi.
• The target owns state in ESI, the returned memory in EDI, and memory_size/the partial-width count path in EBX/BX.
• The baseline owns state in EBX, returned memory in ESI, and memory_size in EDI, using EAX plus LEA for the scaled-count cases.
• The first strict relocation divergence is relocation 10: the default-case source-path relocation is at target +0xA6 versus base +0xAA. Relocations 11-12 inherit the four-byte drift; switch-table destinations also differ at relocations 15-18. Counts and relocation kinds/semantic callees agree.
• Normalized differing bytes: 114 across the 304-byte padded section.

Accepted controls

• Keep the integrated source at 8cf2d5444 as the baseline. It is readable C and preserves all 15 strict-exact siblings plus all owned non-code data.
• The recovered switch cases, assertions, overflow gate, byte-swap call, and offset update agree semantically and structurally with the January target.
• No source change is accepted unless _data_decode_memory improves and the full 16-function plus 14-data-section sweep remains regression-free.

Experiment matrix

Historical objects and dirty worktrees were inventoried before new work. A scan of every available build/base/source/memory/data_encoding.obj found only the two normalized candidate hashes below; more than one hundred unrelated later worktrees all reproduce the canonical baseline hash.

In the table, 9c154cab... means the full canonical baseline hash 9c154cab347995fa1eb87ee6b088615f4347de7947058d8336212577bfbc3cc7.

ID	Source shape	Size T/B	Relocs T/B	Base hash / first divergence	Siblings	Decision
H00	Integrated natural decoder: memory = NULL, memory_size, direct switch and overflow gate	304/304	19/19	9c154cab...; +0x4; 114 differing bytes	15/15 exact	retained baseline
H01	register long memory_size in the earlier encoder (data-encoding-closeout-current dirty tree), intended as a TU-wide allocation hint	304/304	19/19	unchanged 9c154cab...; +0x4	15/15 exact	reverted; code-neutral and cannot affect the residual
H02	Invalid-size path reload memory_size = *(long volatile *)&count (data-decode-memory-closeout-2 dirty tree)	304/304	19/19	a85be189ba9066e2865595bdf359c1776697dc51e53db75b02b0175d5782d8c2; +0x5; 171 differing bytes; first relocation mismatch index 4	not better	reverted; worsened and volatile forcing is prohibited
H03	data-encoding-closeout and data-encoding-final dirty worktrees	304/304	19/19	9c154cab...; no content diff (line-ending/timestamp only)	15/15 exact	no experiment to preserve
E01	Declare memory_size before memory = NULL	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; declaration order is code-neutral
E02	Split void *memory; from memory = NULL; after both declarations	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; initializer spelling is code-neutral
E03	register void *memory = NULL	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; register hint is ignored for this already-register-resident local
E04	register long memory_size in the decoder	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; register hint is ignored
E05	register storage on the state parameter	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; register hint is ignored
E06	register storage on the count parameter, attempting to anchor BX before state	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; register hint is ignored
E07	Initialize memory_size = count before the switch and let case 1/default inherit it	304/304	19/19	7aec197e32f8b4791f940226f01426957e7a46f10c7309e29abe1a1c63a9ee25; +0x8; 243 differing bytes; first relocation mismatch 0	14/15 siblings exact (_data_decode_int64 also moved); data 14/14	reverted; substantially worse and violates sibling gate
E08	Add a local short element_count = count and use it after preserving the original count>=0 assertion	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; alias is fully coalesced
E09	Spell scaled counts as count*2, count*4, count*8	304/304	19/19	decoder unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	only 14/15 siblings exact: _data_decode_int64 moved despite unchanged decoder bytes; data 14/14	reverted; whole-TU compiler-state regression
E10	Swap structures/memory_size declarations in exact predecessor _data_decode_structures	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; predecessor order is code-neutral and does not move decoder allocation
E11	Split the exact predecessor’s structures = NULL initializer into a statement	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; no downstream allocation effect
E12	Initialize memory_size = count, use memory_size <<= 1/2/3, and let case 1 inherit the initializer	304/288	19/19	4760017a8585b2dc55a99aa83c2869c064be8c9c91bcd19719784a94e6820d16; +0x8; 229 differing bytes; first relocation mismatch 0	14/15 siblings exact (_data_decode_int64 moved); data 14/14	reverted; 16 bytes short and violates sibling gate
E13	Top-level const on the count definition parameter	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; ignored by codegen and VC7 warns C4028 against the header prototype
E14	Declare short element_count = count before both existing locals and use the alias after retaining the original count>=0 assertion	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; the alias is fully coalesced even when it leads the declaration list
E15	Combined donor-inspired shape: declare memory_size before memory, then split memory = NULL into a statement	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; the combined declaration/initializer spelling is also code-neutral
E16	Redundant memory_size = count initializer while retaining every switch assignment	304/304	19/19	unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 10	15/15 exact; data 14/14	reverted; dead initializer is eliminated before allocation
E17	Introduce decode_state = state before the existing locals and consistently use the alias	304/304	19/19	decoder code hash unchanged 9c154cab...; +0x4; 114 differing bytes; first relocation mismatch 1	only 12/15 siblings exact: _data_decode_byte, _data_decode_short, and _data_decode_long also moved; data 14/14	reverted; alias coalesces in the residual and causes a whole-TU compiler-state regression
E18	Remove the invented memory_size = count repair from the fatal default arm, matching the HCEA PPC invalid-path load; its exact-unique PDB record independently shows one long local	304/304	19/19	target/base eb39377415e6...; no divergence	15/15 siblings exact; data 14/14	accepted; closes the final function

The prior data-encoding-foundation, data-encoding-leaves, data-encoding-encode, data-encoding-closeout, data-encoding-closeout-current, data-encoding-final, and data-decode-memory-closeout-2 branches/worktrees were inspected. The untracked final-lane research scripts pointed to the analogous October function by its count>=0 assertion and expected address 0x11A430. A read-only Ghidra pass over the preserved October cachebeta.xbe confirmed the address and recovered the complete instruction topology. The October function uses the same ESI state, EDI zero/result, and EBX/BX size/count allocation as the January target, with the same relative switch blocks: case 1 at +0x82, the <<1, <<2, and <<3 cases at +0x87, +0x8E, and +0x96, default at +0x9E, and the post-call count reload at +0xB8. This rules out a January-only compiler accident and is strong evidence for a stable original source/lifetime shape.

A raw target-object census for the distinctive prologue 55 8B EC 53 56 8B 75 08 57 33 FF found five January functions. Four are unimplemented in this integration; the only reconstructed legal-C donor is _rasterizer_triangle_buffer_new on codex/hardware-geometry-leaves-14. Its current XDK 3911 candidate emits that exact ESI/EDI prologue from ordinary local declarations and an explicit zero assignment, proving the allocator topology is expressible in legal C. However, the donor is not strict exact as a whole (352/352 bytes, 24/24 relocations, target hash e90e066bbde0d04c87a43f665fcd7a131ff0e0d1c7a5354b82062ae5b822396c, candidate hash b0fdebbf1336f8336a37bb436ad10ea3ebbf9f105715dec74c87d16c283b457e), and it does not contain the decoder's partial-BX switch. It therefore does not meet reopen criterion 2 or justify copying its unrelated lifetime controls.

Do-not-repeat list

• Do not retry H01/H02 or spelling-only versions that compile to their hashes.
• Do not perturb any of the 15 exact siblings to influence this function.
• Do not use inline assembly, volatile forcing, undefined behavior, aliasing violations, object-byte patches, compiler pragmas, or flag changes.
• Do not accept equal size, equal relocation count, ordinary objdiff percentage, or semantic plausibility as exactness.
• Do not chase non-owned candidate-only assertion COMDATs as target data.

Historical residual classification (superseded)

Measured facts show matching padded size, relocation count, operations, assertions, calls, jump-table cardinality, and behavior, but different callee-saved register ownership, partial-width count handling, scaled-count instruction selection, and a four-byte default-block drift. Direct declaration orders, split/redundant initializers, register hints, local aliases, expression spellings, and exact-predecessor perturbations either collapse to the same bytes or regress code/strict siblings. The independent October binary confirms that the target allocation was stable, while the only legal-C prologue donor is neither strict exact nor a partial-BX analogue. This residual is therefore parked as a private register-allocation and source-lifetime fixed point (class C), not accepted as matching.

Historical reopen criteria (satisfied)

Reopen only with one of:

1. January/October source or PDB local-variable provenance showing declaration order, explicit register hints, or an intermediate count/memory-size local;
2. a same-XDK strict-exact donor with the target ESI/EDI/EBX allocation and partial-BX switch shape;
3. a legal-C, defined-behavior source control demonstrated to move the first divergence without regressing siblings.

Historical disposition (superseded)

Parked at 15/16 strict-exact functions and 14/14 strict-exact owned non-code sections. The natural integrated C remains the least invasive, regression-free candidate. The TU is not eligible for Matching while _data_decode_memory remains non-exact.

2026-08-11 provenance breakthrough and closure

The HCEA RELEASE_CACHE PDB provides an exact-unique cross-build mapping for data_decode_memory. It records the cross-build parameter shape (data_encoding_state *state, short count, long code) and exactly one local, long size; January disassembly independently proves the argument widths. The actual HCEA PPC body loads that uninitialized stack local on the invalid selector path, and a later HCEA reconstruction documents the same defect. All known January callers use only -8, -4, -2, or 1. January remains the byte oracle, but the combined provenance made the omission evidence-backed rather than speculative.

The reconstruction reference is surreptitiousresearch/halocea@0dadebb5671751b8730110f90bf6c10df5f6312c, src/data_decode_memory.c; it is corroboration only, not the byte oracle.

Removing only the fallback memory_size = count assignment after the fatal assertion and system_exit(-1) reproduces January exactly:

• _data_decode_memory: 304/304 padded bytes, 19/19 relocations, normalized SHA-256 eb39377415e6c3e07f763e8f0f42f6ecafaf61acc0fab30fbc06062b9dc465d8;
• all 15 sibling functions remain strict-exact, giving 16/16 functions and 3,296/3,296 padded function bytes;
• all 14 January-owned .rdata sections remain strict-exact, totaling 459 bytes with zero relocations.

This is admitted under the original-bug policy, not as a general undefined- behavior codegen technique. January's otherwise anomalous post-fatal reload, the instruction-identical October topology, the exact-unique HCEA PDB local record, and the HCEA PPC invalid-path load corroborate the omission. The production comment identifies the defect. A corrected build should initialize memory_size before leaving the invalid-code case; the former defined fallback is measured above as H00 and differs in 114 normalized byte positions despite matching size and relocation count.

Independent bounded safety variants confirm that exactness is specific to the recovered omission: memory_size = count restores candidate hash 9c154cab... with 114 differing bytes; a zero initializer produces a 288-byte function; an explicit default zero produces 304 bytes with 94 differing positions; and an immediate return NULL produces 304 bytes with 166 differences. These are documented with the complete source-shape matrix in research/data_decode_memory_codegen/REPORT.md.

Final admission gates

• XDK/Ninja translation-unit rebuild: exit 0.
• Consolidated halobetacache_build: exit 0.
• Hardened semantic audit: all 16 functions are in the accepted ledger; _data_decode_memory contributes 295 meaningful bytes from its strict 304-byte section and all 19 relocations.
• Admission audit: no candidate, contradiction, or revocation for source/memory/data_encoding after the Matching label.
• Post-admission ninja progress: Halo 263/468 objects, 3,133/7,574 functions, 366,428/1,770,166 code bytes, and 45.65% data.
• Tooling regression suite: 149/149 unittest tests pass.

2026-08-13 independent admission revalidation

The completed object was independently transplanted after the separately verified data_packets.obj admission, onto campaign base 5b2fbf0ff8782dc0899038e330125f1accfa1508. The current configured XDK 3911 compiler reports VC7 13.00.9254.1 and uses unchanged /O2 /Oy- /DDEBUG /Dxbox flags.

• All sixteen functions are strict-exact: 3,296/3,296 padded code bytes. The final _data_decode_memory section is 304/304 bytes with 19/19 exact relocation records and normalized SHA-256 eb39377415e6c3e07f763e8f0f42f6ecafaf61acc0fab30fbc06062b9dc465d8.
• All fourteen target-owned .rdata COMDATs are semantically and ownership-exact: 459/459 bytes and zero relocations. The three candidate-only assertion COMDATs are unchanged from the clean campaign baseline and remain known select-any ownership artifacts.
• The clean pre-change regression manifest covered data_encoding, data_packets, and data_packet_groups. The final comparison reports zero changed_nonexact functions in every unit. Its only non-green records are the reviewed Matching/semantic-ledger changes, the four newly exact functions across the two admitted objects, and compiler-local $L... movement inside those newly exact sections; external symbols and owned data are unchanged.
• The consolidated build, semantic report, and progress gate pass with zero unit errors. The current tooling suite passes 173/173 tests.
• The closing edit introduces no assembly, force-inlining, alignment directive, magic offset, raw tag/object access, compiler flag, or comparison exception. The uninitialized memory_size path is the original shipped defect proven above; the production BUG (original) comment names the safe repair explicitly.

2026-08-30 source-credibility revalidation

The later credibility audit removed the encoder's volatile parameter type-pun, volatile array-pointer qualifier, register hint, redundant count alias, and invented fatal-edge fallback assignment. Ordinary direct C remains strict exact across all 16 functions and all owned data. The evidence, bounded experiment matrix, hashes, and validation are recorded in data_encoding_obj_jonas_credibility_cleanup_20260830.md.