Files

13 KiB

Notes for agents

This repository ports the Halo: Combat Evolved decompilation (Xbox build 01.01.14.2342, cachebeta.exe) to Linux, Windows, Android and the web. The game's C sources are the decompilation; the port adds a platform layer that implements the Xbox APIs the game calls, and features the Xbox game did not have (a new netcode, internet play, the PC version's menus, Custom Edition maps, touch controls). The project is published as OpenCE.

Start with README.md. The other documents are listed at the end.

Layout

Path Contents
source/ The game: 466 C files of the decompilation, compiled as they are with few changes (each marked, below)
port/linux/src/ The platform layer the three ports share: Direct3D 8 on OpenGL (d3d8_gl.c), DirectSound and XInput on SDL3, files, threads, memory, sockets, settings (port_config.c), the mods folder (port_mods.c), internet play (p2p*.c)
port/linux/game/ The port's game-side code, compiled as the game's sources are: the distributed netcode (network_*.c), co-op, map sharing (map_share*.c), the menus (menu_tags.c, menu_functions.c), Custom Edition maps (custom_edition_*.c, cache_file_formats.c), the tag validator (tag_validate.c, tag_schema_*.c), mods (mods.c, mod_*.c), touch, FOV and more
port/linux/include/ The port's headers: the limits (halo_port_limits.h, with HALO_PORT_NETWORK_VERSION), capacities (halo_port_capacity.h), the prefix header, halo_linux_source_fixups.h (port functions the game's sources call)
port/include/xdk/ Stand-ins for the Xbox SDK's headers (declarations only)
port/windows/ The Windows build's own files (win32_*.c, headers)
port/android/ The Android app: the guest runtime, the host library (host/), the Java app (app/), host imports (host_imports.list)
port/web/ The web build: its runtime (src/: entry, sockets inside the page, memory watch), the page and its service worker (site/), the SDL port, tests
port/assets/ What the builds embed or ship: high-res HUD (hud/), fonts, titles, menus (menus/), icons, network brokers
port/third_party/ Vendored libraries, each with a README naming its upstream, version and checksum
port/tools/ Stand-alone tools (cache_file_report.c)
tools/ Build scripts (linux_build.py, windows_build.py, android_build.py, web_build.py, ci_build.py, web_serve.py), generators (ce_menus.py, port_settings.py, hud_assets.py, title_assets.py, embed_assets.py, xdk_headers.py) and tests
docs/ Longer design notes (custom_edition_caches.md, mods.md)
pgo/ Profile-guided optimisation profiles
assets/ Game data for local runs (gitignored: maps/, custom_maps/)
.github/workflows/ CI: builds every platform on each push, publishes releases from main

Building

python configure.py            # debug build; add --release, --portable, --profile, --pgo=off, --lto=off
ninja linux                    # build/linux/halo
ninja android                  # the Android guest image and native libraries
ninja android_apk              # the APK (Gradle)
ninja web                      # build/web/site (Emscripten: port/web/README.md)
ninja windows                  # on Windows only
python tools/ci_build.py linux release   # what CI builds
  • configure.py writes build.ninja; configure_args at its top records the options it was run with. Restore them after trying other options.
  • The Linux and Windows builds are 32-bit x86 with clang, in MSVC's ABI (-fms-extensions -fshort-wchar -malign-double -fcommon), C89 (gnu89): declarations come before statements.
  • Every native build compiles without fused multiply-add and with the vendored musl maths (halo_math.h), so that every machine of a network game computes the same results. Do not call the system's maths functions.
  • Android: the game is an ILP32 AArch64 guest (arm64_32) in a 64-bit app. Guest code can call only the host imports declared in port/android/host_imports.list and port/android/guest/runtime/guest_host.h, and only the SDL functions guest_sdl.c bridges. Run ninja android after changing anything it compiles: port/linux/src, port/linux/game, port/android, source/, the build scripts.
  • Web: the game is wasm32 (also ILP32), run on a Web Worker; calls must match their callee's type exactly (the link fails on a mismatch), and WebGL 2 is ES 3.0. Run ninja web after changing what it compiles (as for Android, and port/web). Refer to port/web/README.md.

Defines

Define Set by Meaning
HALO_ANDROID Android The app: its display, input, files and lifecycle
HALO_GLES Android, web The OpenGL ES renderer
HALO_ARM64_GUEST Android The guest's ABI (ILP32 AArch64)
HALO_WEB Web The browser: its frames, storage, sockets and WebGL 2's limits (with HALO_GLES, not HALO_ANDROID)
HALO_WINDOWS Windows (halo_windows_prefix.h) The Windows build
HALO_RELEASE --release No assertions checked
HALO_PROFILE --profile The profiling build; a normal build must be unchanged by it

HALO_LINUX and HALO_CUSTOM_EDITION are not defined by any build: code guarded by them compiles to nothing. Custom Edition support is a setting (game.custom_edition), checked at run time with custom_edition_cache_tags_loaded().

Testing

python -m pytest -q tools/harness tools/test_touch_menu.py   # the engine's functions in a fake world
python -m pytest -q tools/test_linux_port.py                # the build, the maps' checks (needs assets/maps)
python -m pytest -q tools/test_mods.py                      # mods played over a map (needs assets/maps)
python -m pytest -q tools/test_cache_file_formats.py tools/test_bmp_files.py tools/test_profile.py
python tools/test_touch_layout.py                           # the Android touch layout (JDK 17+)
python tools/test_light_storage.py
python tools/test_death_timing.py
python -m pytest -q tools/test_web_build.py                 # the web build's graph, sockets, menus
node --test "port/web/tests/*.test.js"                      # the web page's importer and service worker
node port/web/tests/webrtc_native.mjs chromium firefox      # the native builds' WebRTC with browsers (Playwright)

tools/harness/README.md explains how to add an asset-free test: the code under test is taken from the sources, not copied.

Running the game without a person at it:

  • debug.network_test (HALO_NETWORK_TEST=host:<map> or join) hosts or joins a game without the menus and logs each player's state every second; several copies play on one machine on different loopback addresses (HALO_NET_ADDRESS, HALO_NET_BROADCAST). NETCODE.md describes the test settings.
  • debug.exit_after, debug.hidden_window, debug.null_renderer, debug.menu_open (start on one menu screen), debug.telnet_console.
  • Use a virtual display (Xvfb, with SDL_VIDEO_DRIVER=x11) for windowed runs, never the desktop of the person you work for, and stop only the processes you started.

Rules of the project

The game's sources

  • Change source/ as little as possible, and mark each change from the original with a comment that starts port:. Write like the surrounding code: its naming, its comment density, and the project's plain, descriptive comment style.
  • A port function the game's sources call is declared in halo_linux_source_fixups.h or included with a note naming its file (#include "view_fov.h" /* port: port/linux/game/view_fov.c */).
  • Notes such as BUG (original, preserved for exact matching) mark the original's bugs. Fixing one is allowed; say so in a port: comment.

The Xbox SDK

port/include/xdk was written without the SDK's headers, from the January build's debug information and public documentation. Never read the SDK's headers or anything derived from them, never copy from them, and do not read xbox/include. To add a name, follow port/include/xdk/README.md.

Networking

  • Raise HALO_PORT_NETWORK_VERSION (port/linux/include/halo_port_limits.h) with any change to what the machines send each other, or to anything they must agree on, and say why in the commit. Machines of different versions do not play together.
  • Only the host decides; clients predict their own players. Read port/linux/NETCODE.md before changing the netcode.
  • Internet play is the same code in every build, the web's too: a browser's tunnel is a WebRTC data channel (port/web/src/web_p2p.c), which the native builds take on their tunnel's socket (p2p_webrtc.c, posix_dtls.c). Check a change with both (port/web/tests/webrtc_native.mjs, and with the maps port/web/tests/internet.mjs).
  • Anything another machine sends is untrusted: check sizes, indices and rates, as the existing handlers do. debug.network_corrupt tests this.

Menus

  • The menus must match Halo Custom Edition's. Their XML (port/assets/menus/ce/) is generated by tools/ce_menus.py from the PC version's tags; the settings screens come from tools/port_settings.py. Do not edit the XML by hand: change the generator and run it (the command is in port/assets/menus/README.md, "Writing them again"), then check that git status shows only the screens meant to change.
  • Functions the menus call but the port has not wired are listed in port/assets/menus/UNWIRED.md.

Art and credit

  • Replacement HUD, title and menu art is made of hand-drawn redraws.
  • Prefer an existing redraw (the ui-svg-handmade set), then a new redraw. A Custom Edition picture that cannot reasonably be redrawn (a 3D render, a screenshot, a logo) may be used only if it is listed in port/assets/menus/NON_HANDDRAWN.md.
  • AUTHORS.md credits art only.
  • Fonts: Overpass and OpenCE (Newtown, respaced) only; see port/assets/fonts/README.md. Never ship the maps' commercial typefaces.

Custom Edition maps

Custom Edition maps live in the data root's custom_maps folder with bitmaps.map, sounds.map and loc.map; their level names are custom_maps\<name>. OpenSauce features are not supported and must not be added: a map that needs them is refused, and one that only carries OpenSauce's header runs as stock Custom Edition runs it. Map files are untrusted input: read them through bounds-checked readers. See docs/custom_edition_caches.md.

Settings

A setting is an entry in port/linux/src/port_config.c, with a default, an environment variable and a comment that config.toml gets. Document it in the settings table of port/linux/README.md (and the Android README for an Android-only one). A departure from the stock game's look or behaviour should be a setting, off or stock by default, unless the project decides otherwise.

Third-party code and licences

The repository is CC0. Vendored code goes in port/third_party/<name>, copied unchanged, with a README naming the upstream, the version or commit, the checksum and what uses it. GPL code (OpenSauce, Reclaimer) may be read as documentation of file formats, never copied.

Commits and pull requests

  • Commit subjects are plain sentences saying what changed; the body says why and what was checked.
  • A pull request is merged with a merge commit (Merge pull request #N from <owner>/<branch>), and follow-up changes go in separate commits whose subject ends with (#N).
  • A commit taken from a fork keeps its author and is made with git cherry-pick -x.

Documents

Document About
README.md Downloads, game data, building, build options, profiling builds
port/linux/README.md The Linux build, controls, menus, every setting, updates, frame rate, field of view, system link, internet play, voice chat, the server browser, map sharing, security, map checks, how the port operates, the game source changes
port/linux/NETCODE.md The distributed netcode, network versions, joining a game in progress, map sharing, transport, testing
port/windows/README.md The Windows build, its headers, inline functions, crash reports
port/android/README.md The Android build, the guest and host, touch controls, Android settings, finding problems
port/web/README.md The web build, the page, the game data in the browser, limits, how it operates, hosting on GitHub Pages
port/include/xdk/README.md How the SDK declarations were written and are checked
docs/custom_edition_caches.md Loading and converting Custom Edition maps, and the evidence for each layout
docs/mods.md Mods: folders of Halo PC tag files whose sounds, bitmaps, text and models play over a map's on one machine
port/assets/menus/README.md The menu files' format and how to write them again
port/assets/menus/NON_HANDDRAWN.md Menu pictures that are not redraws
port/assets/menus/UNWIRED.md Menu functions that do nothing yet
port/assets/fonts/README.md The fonts and their licences
tools/harness/README.md Asset-free regression tests
port/third_party/*/README.md Each vendored library
AUTHORS.md Art credits