13 KiB
Notes for agents
This repository ports the Halo: Combat Evolved decompilation (Xbox build
01.01.14.2342, cachebeta.exe) to Linux, Windows, Android and the web. The game's C
sources are the decompilation; the port adds a platform layer that
implements the Xbox APIs the game calls, and features the Xbox game did not
have (a new netcode, internet play, the PC version's menus, Custom Edition
maps, touch controls). The project is published as OpenCE.
Start with README.md. The other documents are listed at the end.
Layout
| Path | Contents |
|---|---|
source/ |
The game: 466 C files of the decompilation, compiled as they are with few changes (each marked, below) |
port/linux/src/ |
The platform layer the three ports share: Direct3D 8 on OpenGL (d3d8_gl.c), DirectSound and XInput on SDL3, files, threads, memory, sockets, settings (port_config.c), the mods folder (port_mods.c), internet play (p2p*.c) |
port/linux/game/ |
The port's game-side code, compiled as the game's sources are: the distributed netcode (network_*.c), co-op, map sharing (map_share*.c), the menus (menu_tags.c, menu_functions.c), Custom Edition maps (custom_edition_*.c, cache_file_formats.c), the tag validator (tag_validate.c, tag_schema_*.c), mods (mods.c, mod_*.c), touch, FOV and more |
port/linux/include/ |
The port's headers: the limits (halo_port_limits.h, with HALO_PORT_NETWORK_VERSION), capacities (halo_port_capacity.h), the prefix header, halo_linux_source_fixups.h (port functions the game's sources call) |
port/include/xdk/ |
Stand-ins for the Xbox SDK's headers (declarations only) |
port/windows/ |
The Windows build's own files (win32_*.c, headers) |
port/android/ |
The Android app: the guest runtime, the host library (host/), the Java app (app/), host imports (host_imports.list) |
port/web/ |
The web build: its runtime (src/: entry, sockets inside the page, memory watch), the page and its service worker (site/), the SDL port, tests |
port/assets/ |
What the builds embed or ship: high-res HUD (hud/), fonts, titles, menus (menus/), icons, network brokers |
port/third_party/ |
Vendored libraries, each with a README naming its upstream, version and checksum |
port/tools/ |
Stand-alone tools (cache_file_report.c) |
tools/ |
Build scripts (linux_build.py, windows_build.py, android_build.py, web_build.py, ci_build.py, web_serve.py), generators (ce_menus.py, port_settings.py, hud_assets.py, title_assets.py, embed_assets.py, xdk_headers.py) and tests |
docs/ |
Longer design notes (custom_edition_caches.md, mods.md) |
pgo/ |
Profile-guided optimisation profiles |
assets/ |
Game data for local runs (gitignored: maps/, custom_maps/) |
.github/workflows/ |
CI: builds every platform on each push, publishes releases from main |
Building
python configure.py # debug build; add --release, --portable, --profile, --pgo=off, --lto=off
ninja linux # build/linux/halo
ninja android # the Android guest image and native libraries
ninja android_apk # the APK (Gradle)
ninja web # build/web/site (Emscripten: port/web/README.md)
ninja windows # on Windows only
python tools/ci_build.py linux release # what CI builds
configure.pywritesbuild.ninja;configure_argsat its top records the options it was run with. Restore them after trying other options.- The Linux and Windows builds are 32-bit x86 with clang, in MSVC's ABI
(
-fms-extensions -fshort-wchar -malign-double -fcommon), C89 (gnu89): declarations come before statements. - Every native build compiles without fused multiply-add and with the
vendored musl maths (
halo_math.h), so that every machine of a network game computes the same results. Do not call the system's maths functions. - Android: the game is an ILP32 AArch64 guest (
arm64_32) in a 64-bit app. Guest code can call only the host imports declared inport/android/host_imports.listandport/android/guest/runtime/guest_host.h, and only the SDL functionsguest_sdl.cbridges. Runninja androidafter changing anything it compiles:port/linux/src,port/linux/game,port/android,source/, the build scripts. - Web: the game is
wasm32(also ILP32), run on a Web Worker; calls must match their callee's type exactly (the link fails on a mismatch), and WebGL 2 is ES 3.0. Runninja webafter changing what it compiles (as for Android, andport/web). Refer to port/web/README.md.
Defines
| Define | Set by | Meaning |
|---|---|---|
HALO_ANDROID |
Android | The app: its display, input, files and lifecycle |
HALO_GLES |
Android, web | The OpenGL ES renderer |
HALO_ARM64_GUEST |
Android | The guest's ABI (ILP32 AArch64) |
HALO_WEB |
Web | The browser: its frames, storage, sockets and WebGL 2's limits (with HALO_GLES, not HALO_ANDROID) |
HALO_WINDOWS |
Windows (halo_windows_prefix.h) |
The Windows build |
HALO_RELEASE |
--release |
No assertions checked |
HALO_PROFILE |
--profile |
The profiling build; a normal build must be unchanged by it |
HALO_LINUX and HALO_CUSTOM_EDITION are not defined by any build:
code guarded by them compiles to nothing. Custom Edition support is a
setting (game.custom_edition), checked at run time with
custom_edition_cache_tags_loaded().
Testing
python -m pytest -q tools/harness tools/test_touch_menu.py # the engine's functions in a fake world
python -m pytest -q tools/test_linux_port.py # the build, the maps' checks (needs assets/maps)
python -m pytest -q tools/test_mods.py # mods played over a map (needs assets/maps)
python -m pytest -q tools/test_cache_file_formats.py tools/test_bmp_files.py tools/test_profile.py
python tools/test_touch_layout.py # the Android touch layout (JDK 17+)
python tools/test_light_storage.py
python tools/test_death_timing.py
python -m pytest -q tools/test_web_build.py # the web build's graph, sockets, menus
node --test "port/web/tests/*.test.js" # the web page's importer and service worker
node port/web/tests/webrtc_native.mjs chromium firefox # the native builds' WebRTC with browsers (Playwright)
tools/harness/README.md explains how to add an asset-free test: the code
under test is taken from the sources, not copied.
Running the game without a person at it:
debug.network_test(HALO_NETWORK_TEST=host:<map>orjoin) hosts or joins a game without the menus and logs each player's state every second; several copies play on one machine on different loopback addresses (HALO_NET_ADDRESS,HALO_NET_BROADCAST).NETCODE.mddescribes the test settings.debug.exit_after,debug.hidden_window,debug.null_renderer,debug.menu_open(start on one menu screen),debug.telnet_console.- Use a virtual display (Xvfb, with
SDL_VIDEO_DRIVER=x11) for windowed runs, never the desktop of the person you work for, and stop only the processes you started.
Rules of the project
The game's sources
- Change
source/as little as possible, and mark each change from the original with a comment that startsport:. Write like the surrounding code: its naming, its comment density, and the project's plain, descriptive comment style. - A port function the game's sources call is declared in
halo_linux_source_fixups.hor included with a note naming its file (#include "view_fov.h" /* port: port/linux/game/view_fov.c */). - Notes such as
BUG (original, preserved for exact matching)mark the original's bugs. Fixing one is allowed; say so in aport:comment.
The Xbox SDK
port/include/xdk was written without the SDK's headers, from the January
build's debug information and public documentation. Never read the SDK's
headers or anything derived from them, never copy from them, and do not
read xbox/include. To add a name, follow port/include/xdk/README.md.
Networking
- Raise
HALO_PORT_NETWORK_VERSION(port/linux/include/halo_port_limits.h) with any change to what the machines send each other, or to anything they must agree on, and say why in the commit. Machines of different versions do not play together. - Only the host decides; clients predict their own players. Read
port/linux/NETCODE.mdbefore changing the netcode. - Internet play is the same code in every build, the web's too: a browser's
tunnel is a WebRTC data channel (
port/web/src/web_p2p.c), which the native builds take on their tunnel's socket (p2p_webrtc.c,posix_dtls.c). Check a change with both (port/web/tests/webrtc_native.mjs, and with the mapsport/web/tests/internet.mjs). - Anything another machine sends is untrusted: check sizes, indices and
rates, as the existing handlers do.
debug.network_corrupttests this.
Menus
- The menus must match Halo Custom Edition's. Their XML (
port/assets/menus/ce/) is generated bytools/ce_menus.pyfrom the PC version's tags; the settings screens come fromtools/port_settings.py. Do not edit the XML by hand: change the generator and run it (the command is inport/assets/menus/README.md, "Writing them again"), then check thatgit statusshows only the screens meant to change. - Functions the menus call but the port has not wired are listed in
port/assets/menus/UNWIRED.md.
Art and credit
- Replacement HUD, title and menu art is made of hand-drawn redraws.
- Prefer an existing redraw (the ui-svg-handmade set), then a new redraw. A
Custom Edition picture that cannot reasonably be redrawn (a 3D render, a
screenshot, a logo) may be used only if it is listed in
port/assets/menus/NON_HANDDRAWN.md. AUTHORS.mdcredits art only.- Fonts: Overpass and OpenCE (Newtown, respaced) only; see
port/assets/fonts/README.md. Never ship the maps' commercial typefaces.
Custom Edition maps
Custom Edition maps live in the data root's custom_maps folder with
bitmaps.map, sounds.map and loc.map; their level names are
custom_maps\<name>. OpenSauce features are not supported and must not be
added: a map that needs them is refused, and one that only carries
OpenSauce's header runs as stock Custom Edition runs it. Map files are
untrusted input: read them through bounds-checked readers. See
docs/custom_edition_caches.md.
Settings
A setting is an entry in port/linux/src/port_config.c, with a default, an
environment variable and a comment that config.toml gets. Document it in
the settings table of port/linux/README.md (and the Android README for an
Android-only one). A departure from the stock game's look or behaviour
should be a setting, off or stock by default, unless the project decides
otherwise.
Third-party code and licences
The repository is CC0. Vendored code goes in port/third_party/<name>,
copied unchanged, with a README naming the upstream, the version or commit,
the checksum and what uses it. GPL code (OpenSauce, Reclaimer) may be read
as documentation of file formats, never copied.
Commits and pull requests
- Commit subjects are plain sentences saying what changed; the body says why and what was checked.
- A pull request is merged with a merge commit (
Merge pull request #N from <owner>/<branch>), and follow-up changes go in separate commits whose subject ends with(#N). - A commit taken from a fork keeps its author and is made with
git cherry-pick -x.
Documents
| Document | About |
|---|---|
| README.md | Downloads, game data, building, build options, profiling builds |
| port/linux/README.md | The Linux build, controls, menus, every setting, updates, frame rate, field of view, system link, internet play, voice chat, the server browser, map sharing, security, map checks, how the port operates, the game source changes |
| port/linux/NETCODE.md | The distributed netcode, network versions, joining a game in progress, map sharing, transport, testing |
| port/windows/README.md | The Windows build, its headers, inline functions, crash reports |
| port/android/README.md | The Android build, the guest and host, touch controls, Android settings, finding problems |
| port/web/README.md | The web build, the page, the game data in the browser, limits, how it operates, hosting on GitHub Pages |
| port/include/xdk/README.md | How the SDK declarations were written and are checked |
| docs/custom_edition_caches.md | Loading and converting Custom Edition maps, and the evidence for each layout |
| docs/mods.md | Mods: folders of Halo PC tag files whose sounds, bitmaps, text and models play over a map's on one machine |
| port/assets/menus/README.md | The menu files' format and how to write them again |
| port/assets/menus/NON_HANDDRAWN.md | Menu pictures that are not redraws |
| port/assets/menus/UNWIRED.md | Menu functions that do nothing yet |
| port/assets/fonts/README.md | The fonts and their licences |
| tools/harness/README.md | Asset-free regression tests |
port/third_party/*/README.md |
Each vendored library |
| AUTHORS.md | Art credits |