Cesarus85--mjolnir-vr/port/linux

Linux

ninja linux compiles the game with clang for 32-bit x86 Linux. The result is a native executable, build/linux/halo. The game shows its graphics with OpenGL 4.5. It plays sound through SDL3. It accepts keyboard, mouse and gamepad input.

The game is 32-bit code because its data (tags, cache files, saved games) contains 32-bit pointers, as on the Xbox.

Requirements

You do not need the Xbox SDK. The declarations that the game uses are in port/include/xdk.

To build:

• Python and ninja.
• clang. The option --linux-cc of configure.py selects a different compiler.
• The 32-bit glibc development files: lib32-glibc on Arch Linux, gcc-multilib and libc6-dev-i386 on Debian and Ubuntu.
• The 32-bit SDL3: lib32-sdl3 on Arch Linux, libsdl3-dev:i386 on Debian and Ubuntu.

To start the game:

• The 32-bit OpenGL libraries (lib32-mesa).
• The 32-bit PipeWire or PulseAudio client libraries (lib32-pipewire or lib32-libpulse).

Build the game

1. Go to the root folder of the repository.
2. Enter python configure.py.
3. Enter ninja linux.

Start the game

Enter build/linux/halo.

The game data is the folder that contains maps/, from an Xbox disc image of any version of the game. The game looks for this folder in this sequence:

1. paths.data in config.toml.
2. The current folder.
3. The folder of the executable.
4. assets/ in the current folder, and assets/ in the repository that contains the executable.

If the game finds no data, it asks for an Xbox disc image (.xiso or .iso). This occurs at the first start:

• Select "No" to stop the game.
• Select "Yes" to open a file picker. Select the disc image. The game copies maps/ next to the executable and shows the progress.

The game writes the copy to maps.partial. When the copy is complete, the game changes the name to maps. If the copy stops before it is complete, the game asks for the disc image again at the next start.

Files and folders

Xbox drive	Folder
d:\	The data root: the folder that contains maps/.
z:\	z/ in the save root. This folder contains the cache (approximately 800 MB of map data) and the saved games.
u:\	u/ in the save root. This folder contains the user data.

The save root is paths.saves in config.toml. If that setting is empty, the save root is $XDG_DATA_HOME/halo-linux (usually ~/.local/share/halo-linux).

The game makes the folders when it needs them. Names of files and folders are not case-sensitive, as on the Xbox.

These files are in the data root:

File	Contents
debug.txt	The log of the game. At start-up, the game shows the data root in the terminal.
init.txt	Console commands that the game does at start-up. For example, map_name levels\a10\a10 starts the first campaign level.

The settings are in config.toml next to the executable. Refer to "Settings".

If the game stops because of a fatal signal, it writes the address and a backtrace to the standard error. To find the function at the address, enter addr2line -e build/linux/halo <address>.

Controls

The keyboard and the mouse operate controller 1. The game adds the input of the first gamepad to controller 1. The other gamepads operate controllers 2 to 4.

Key	Controller	Function in the game
W, A, S, D	left stick	move
mouse	(direct aim)	aim
left mouse button	right trigger	fire
right mouse button, G	left trigger	throw a grenade
space, enter	A	jump, accept
F, backspace, mouse button 4	B	melee, back
E, R	X	action, reload
tab, mouse wheel	Y	change the weapon
Q	white	flashlight
X	black	change the grenade
left ctrl, C	left stick click	crouch
Z, middle mouse button	right stick click	zoom
arrow keys	D-pad
escape	start	pause menu
F1	back
`		open the developer console
F12		release or capture the mouse
F11		change between fullscreen and window

One movement of the mouse wheel changes the weapon one time. A second movement after a short pause changes it again.

In the menus, the mouse moves a pointer:

• The item below the pointer gets the focus.
• A left click selects the item. On a setting with values, a click on the left or right half changes the value. On a button in the key of a screen (for example "B = Back"), a click pushes that button.
• A right click goes back.
• The mouse wheel moves through the items.

The keyboard also operates the menus. When the game continues, the mouse aims again. A mouse button that you hold from the menu does not fire until you push it again.

Settings

The settings are in config.toml next to the executable (build/linux/config.toml). At the first start, the game writes the file with the default values and a comment for each setting. To get the default values again, delete the file.

The game reads the file one time, at start-up. If a key is not correct, or a value has the wrong type, the game writes the line to the log and uses the default value.

Each setting has an environment variable. The environment variable changes the setting for one start of the game. It has priority over the file.

Setting	Default	Environment variable	Function
display.fullscreen	true	HALO_FULLSCREEN	true: fullscreen at the resolution of the display. The picture has 480 lines of the game and the width of the display. false: a window with the 640x480 picture of the Xbox. F11 changes between the two.
display.window_scale	2	HALO_WINDOW_SCALE	The size of the window, as a multiple of 640x480. You can change the size of the window.
display.vsync	true	HALO_NO_VSYNC=1 sets false	true: each frame waits for the display.
display.interpolation	true	HALO_INTERPOLATION	true: one frame for each refresh of the display. false: 30 frames each second, as on the Xbox. Refer to "Frame rate".
display.direct_camera	true	HALO_DIRECT_CAMERA	true: in first person, on foot, the view points where the player aims in each frame, not where the last tick left it. Refer to "Frame rate".
audio.enabled	true	HALO_NO_AUDIO=1 sets false	false: no audio device. The sound continues without output.
audio.volume	1.0	HALO_VOLUME	The master volume.
input.mouse_sensitivity	1.0	HALO_MOUSE_SENSITIVITY	The multiplier for the mouse aim.
input.invert_mouse	false	HALO_MOUSE_INVERT=1 sets true	true: the vertical mouse aim is inverted.
input.mouse_aim_assist	false	HALO_MOUSE_AIM_ASSIST	true: the magnetism of the controller also operates for the mouse. false: when the mouse moved after the right stick, the view is not slowed or dragged by a target. The autoaim of the bullets operates in both cases.
game.language	""	HALO_LANGUAGE	The language of the menus: ja, de, fr, es or it. Empty: English.
paths.data	""	HALO_DATA_ROOT	The data root. Refer to "Start the game".
paths.saves	""	HALO_SAVE_ROOT	The save root. Refer to "Files and folders".
network.netcode	"distributed"	HALO_NETCODE	"distributed": each machine moves its own player at once, and the host makes the decisions (refer to NETCODE.md). "lockstep": as on the Xbox. The host's setting applies: a machine that joins a game uses the netcode of the host.
network.address	""	HALO_NET_ADDRESS	The IPv4 address of this machine for system link. Refer to "Play on one computer".
network.broadcast	""	HALO_NET_BROADCAST	IPv4 addresses, with commas between them, that get the broadcasts of the game. Empty: 255.255.255.255.
network.online	true	HALO_NET_ONLINE	true: internet play. false: system link on the local network only.
network.join_from_clipboard	true	HALO_NET_JOIN_FROM_CLIPBOARD	true: when the game comes to the front, it joins the game of an invite link on the clipboard.
network.tunnel_port	0	HALO_NET_TUNNEL_PORT	The UDP port for internet play. 0: the game selects a port. Refer to "Internet play".
network.allow_upnp	true	HALO_NET_ALLOW_UPNP	true: internet play can ask the router to forward its port (UPnP). false: the game does not ask. Refer to "Internet play".
network.signalling_brokers	three public brokers	HALO_NET_BROKERS	The public MQTT brokers (host:port, with commas between them) that let the machines of an invite find each other.
network.stun_servers	Google and Cloudflare	HALO_NET_STUN	The public STUN servers (host:port, with commas between them) that give the internet address of a machine.
discord.application_id	the application of the project	HALO_DISCORD_APPLICATION	The Discord application for invites. Empty: no Discord.
update.auto	true	HALO_UPDATE_AUTO	true: at start-up, the game looks for a new version. Refer to "Updates". false: the game does not look.
debug.update_answer	""	HALO_UPDATE_ANSWER	The answer to the update question, for automatic tests: yes, no or never. Empty: the game asks.
debug.exit_after	0.0	HALO_EXIT_AFTER	The game stops after this number of seconds. 0: never.
debug.screenshot_directory, debug.screenshot_every	"", 0	HALO_SCREENSHOT_DIR, HALO_SCREENSHOT_EVERY	The game writes each Nth frame to this folder as a BMP file.
debug.hidden_window, debug.null_renderer	false	HALO_HIDDEN_WINDOW, HALO_NULL_RENDERER	true: no visible window, or no graphics.
debug.gpu_stats, debug.gpu_trace_frame, debug.gpu_trace_constants, debug.gpu_dump_shaders, debug.texture_dump_directory, debug.texture_log, debug.gl_debug, debug.texture_no_cache	off	HALO_GPU_STATS, HALO_GPU_TRACE, HALO_GPU_TRACE_CONSTANTS, HALO_GPU_DUMP_SHADERS, HALO_TEXTURE_DUMP, HALO_TEXTURE_LOG, HALO_GL_DEBUG, HALO_TEXTURE_NO_CACHE	Tools to find problems in the graphics: counts for each frame, all the GL state of one frame, the GLSL code, the textures.
debug.gpu_skip_vertex_shaders, debug.gpu_debug_expression, debug.gpu_debug_flat, debug.gpu_debug_texture0	off	HALO_GPU_SKIP_VS, HALO_GPU_DEBUG_EXPR, HALO_GPU_DEBUG_FLAT, HALO_GPU_DEBUG_T0	Tools to find problems in the graphics: skip the draws of a vertex shader, or replace the output of all pixel shaders with a GLSL expression (for example t0.rgb).
debug.network_test, debug.network_test_start, debug.network_test_kill, debug.network_test_shoot, debug.network_test_vehicle, debug.network_test_pickup, debug.test_input	off	HALO_NETWORK_TEST, HALO_NETWORK_TEST_START, HALO_NETWORK_TEST_KILL, HALO_NETWORK_TEST_SHOOT, HALO_NETWORK_TEST_VEHICLE, HALO_NETWORK_TEST_PICKUP, HALO_TEST_INPUT	Automatic tests of system link (game/network_test.c). Refer to NETCODE.md.
debug.network_latency, debug.network_loss	0	HALO_NETWORK_LATENCY, HALO_NETWORK_LOSS	The game holds all the data that it receives for this number of milliseconds, and ignores this percentage of the datagrams. Use these settings to test the netcode as on the internet.

With Mesa drivers, the game sends its GL calls through the GL thread of Mesa. To stop this, set the environment variable mesa_glthread=false.

Updates

The builds from GitHub Actions (refer to the main README) can update themselves. At start-up, the game asks GitHub for the latest release. The game does not wait for the answer. If the latest release is not newer, the game does nothing.

If the latest release is newer, the game asks: "Do you want to update?"

• Select "Yes" to update. The game downloads the release for this platform, replaces its files and starts the new version. The old files get the extension .old. The new version deletes them.
• Select "No" to continue. The game asks again at the next start.
• Select "Do not ask again", then "Yes", to stop the questions. The game writes auto = false in the [update] section of config.toml. To get the questions again, set auto = true.

The game downloads through HTTPS. It examines the certificate of the server against the certificate authorities of the system: on Linux, the bundle of the distribution (src/posix_update.c, with Mbed TLS); on Windows, the certificate store of Windows (WinHTTP). The folder of the executable must let the game write to it.

Builds that you make yourself have no build number. They do not look for updates.

Frame rate

The game calculates its world at 30 Hz, as on the Xbox. On the Xbox, the game showed one frame for each calculation (tick). This port shows one frame for each refresh of the display, for example at 60, 120 or 240 Hz.

Each frame shows the world between the last two ticks (game/render_interpolation.c):

• After each tick, the game keeps the camera, the position of each part of each object, and the first-person weapon.
• Each frame mixes the last two ticks. The mix agrees with the time since the last tick.
• Rotations use quaternions. Positions and scales are linear.
• A teleport, a respawn or a cut of the camera does not mix. It jumps.

Thus the frames are one tick (33 ms) after the calculation. The calculation does not change.

The direction of the view is an exception. The game reads the mouse and the sticks in each frame. In first person, on foot, each frame points the view where the player aims at that time (display.direct_camera). Thus the view turns in the frame that the mouse moves. In a vehicle and in cinematics, the view mixes as the other things do. On Android, the view mixes as before.

To get 30 frames each second, set display.interpolation = false.

To see the frame rate:

1. Push ` to open the developer console.
2. Enter display_framerate true.

The frame rate shows at the bottom right of the screen. It is the mean over half a second.

System link

The Xbox game lets 16 players on 4 machines play a system link game. This port lets up to 128 players on up to 128 machines play. Each machine can have up to 4 players (split screen).

• include/halo_port_limits.h sets the limits.
• include/halo_port_capacity.h sets the memory for the limits. The game state is 16 MB at 0x81A00000 (3.3 MB on the Xbox). The pools of objects, effects, particles, contrails, lights and sounds are also larger.
• The byte-matching build keeps the limits of the Xbox. All the changes are in #ifdef HALO_LINUX.

Obey these rules:

• All the machines in a game must use a build with the same limits.
• The port uses protocol version 2. It does not see the Xbox game or older builds of the port. They do not see the port.

These are the differences from the Xbox:

• The host waits up to 60 seconds (15 seconds on the Xbox) for the other machines to load the map.
• If a machine does not read the messages of the host for two seconds, the host removes it from the game.
• The saved games contain all the game state. Thus a saved game is 16 MB. Saved games from older builds of the port do not operate.
• In campaign and in games of up to 16 players, the game removes garbage (bodies, dropped weapons) as on the Xbox. In larger games, it keeps more garbage, in proportion to the players.
• The lobby shows the local machine and the first three remote machines. The other machines are also in the game.
• In free-for-all games, each player is a team.

Linux, Windows and Android machines can play in the same game. With the netcode "lockstep", each machine must calculate the same floating-point results. Thus all the ports:

• Compile without fused multiply-add (-ffp-contract=off).
• Use the math functions of musl (port/include/halo_math.h, port/third_party/musl-math), not the math functions of the system.

Play on one computer

More than one copy of the game can play on one computer. Each copy must have a different loopback address. A copy with an address gets no broadcasts. Thus each copy must send its broadcasts to the other copies.

For a host and two clients, enter these commands in three terminals:

HALO_NET_ADDRESS=127.0.0.200 HALO_NET_BROADCAST=127.0.0.201,127.0.0.202 build/linux/halo
HALO_NET_ADDRESS=127.0.0.201 HALO_NET_BROADCAST=127.0.0.200 build/linux/halo
HALO_NET_ADDRESS=127.0.0.202 HALO_NET_BROADCAST=127.0.0.200 build/linux/halo

Do not give 127.0.0.1 to a copy. Each copy gets to its own address through 127.0.0.1. Linux and Windows send all of 127.0.0.0/8 to the loopback interface.

Test with many machines

tools/system_link_bots.py adds simple machines to a game. Each machine has one player. The machines obey the system link protocol and send input, but they do not calculate the game.

1. Start a game on the host.
2. Enter python tools/system_link_bots.py --host 127.0.0.200 --machines 127 --start.

Each machine uses its own loopback address, from 127.0.0.2. The option --start starts the game when all the machines are in the lobby. If the host has no network.address, do not give --host.

Internet play

Machines with an invite link can play system link on the internet. This project has no server.

When a copy of the game starts to host a system link game, it makes an invite link: halo://join/<44 hexadecimal digits>. The game writes the link to the standard error and puts it on the clipboard.

To join a game, do one of these steps:

• Open the link. The game is the handler of halo:// links. If the game already operates, the new copy gives the link to it and stops.
• Copy the link (or the 44 digits) and go to the game.
• Enter halo <link>.
• Accept a Discord invite. Refer to "Discord".

When the machines connect, the game of the host shows in Multiplayer, System Link. Join the game as on a local network. System link on a local network does not need an invite.

Security

Only machines with the invite can find the game:

• The link contains the identifier of the host and a random 16-byte token.
• The machines exchange their addresses through public MQTT brokers (network.signalling_brokers). The topics are HMACs of the token. A key from the token encrypts and authenticates the messages (src/p2p_signal.c, src/p2p_crypto.c).
• A key from the host encrypts and authenticates each packet between two machines.
• An invite operates while the copy of the game that made it operates.

Connection

Each machine gets its public address from public STUN servers. Then the two machines send packets to each other until the packets get through (UDP hole punching). There is no relay.

Some networks give a different port for each destination (for example some mobile and company networks). Two machines behind such networks cannot connect. To connect, forward network.tunnel_port on the router of one of the machines.

The game can ask the router to forward the port (UPnP, src/posix_upnp.c, with port/third_party/miniupnpc):

• The host asks its router when a player uses its invite.
• A player that joins asks its router when it does not reach the host in 5 seconds.
• The forwarded port is one more address that the machine gives to the other machine.
• The forward has a duration of one hour. The game makes it longer while it operates, and removes it when the game stops.
• UPnP does not help behind a second NAT, for example the NAT of a mobile network provider. Then the router has a private address, and the game does not ask.

To stop all UPnP requests, set network.allow_upnp to false.

In the game, each machine has an address in 100.64.0.0/10:

• src/xnet.c sends the traffic of the game to such an address through local sockets on 127.0.0.1 (or network.address).
• src/p2p.c sends that traffic through one UDP socket. UDP datagrams go as they are. TCP connections go as KCP streams (port/third_party/kcp).
• The broadcasts of the game go to all the machines. Thus the game of the host shows on the other machines.

Discord

If the Discord desktop client operates, the game of the host shows in Discord (through the application of discord.application_id). The activity has a private party with the invite as its join secret. The host can send the invite with the invite button of Discord. When a person accepts it, that person joins the game. If the game does not operate, Discord starts it.

What operates

Area	Status
Game code	All 466 C files of the game. The changes are in "Game source changes".
Graphics	Direct3D 8 on OpenGL 4.5 core through SDL3 (src/d3d8_gl.c). The port translates the NV2A vertex shaders and register combiners to GLSL. It decodes all the Xbox texture formats. The vertex and index buffers come from a GL copy of the Xbox memory.
Sound	Xbox DirectSound on SDL3 audio (src/dsound_sdl.c): PCM and Xbox ADPCM, mixed at 48 kHz, with volume, pitch, mix bins, distance, stereo pan, occlusion and obstruction. There is no Doppler effect, no cones and no reverb.
Input	XInput on SDL3 (src/xinput_sdl.c): keyboard, mouse, gamepads with rumble, and the debug keyboard for the console.
Files	The Win32 file functions and the MSVC file functions on POSIX, with the translation of Xbox paths.
Threads	Threads, events, mutexes, critical sections, interlocked operations and alertable waits.
Memory	The port reserves the Xbox memory at 0x80000000. Thus the game gets the fixed addresses that it expects.
Saved games	The Xbox UDATA layout, with SHA-1 signatures.
Networking	Winsock on BSD sockets. System link on a local network and on the internet.
Bink video	Not available. The game skips the movies.

How the port operates

The compiler

tools/linux_build.py compiles the game with clang and these options, which give the ABI of the MSVC compiler:

• --target=i686-linux-gnu: 32-bit x86.
• -fms-extensions: the MSVC extensions.
• -fshort-wchar: 16-bit wchar_t.
• -malign-double: 8-byte alignment of 64-bit members.
• -fcommon: tentative definitions, as in C89.

glibc gives only ISO C (__STRICT_ANSI__). Thus POSIX names, for example random, do not conflict with the names of the game.

These files supply the MSVC functions that clang does not have:

File	Contents
include/halo_linux_prefix.h	The first header of each file: the architecture macros of the SDK, MSVC __inline, SEH keywords, __declspec(selectany).
include/	Headers that add MSVC names to the C runtime headers.
port/include/xdk	The Xbox SDK declarations. The compiler reads this folder after all the other folders.
tools/linux_msvc_semantics.py	Makes a header that declares each struct tag at file scope, as MSVC does. It also makes the header inline functions weak, as the COMDAT functions of MSVC. game/msvc_comdat.c gives one external copy of each.
include/halo_linux_winsock_names.h	Gives new names to the Winsock functions of the SDK. Thus they do not link to the glibc functions with the same names.
include/halo_linux_source_fixups.h	Repairs one declaration conflict (rasterizer_debug_drawing_begin).

tools/linux_link_check.py stops the link if a weak reference has no definition. Without this check, the linker gives the reference the address 0.

The platform layer (src/)

• The files posix_*.c use glibc. The compiler uses the ABI of the host for these files, because some glibc structures have a different layout with -malign-double.
• The other files include the SDK declarations through platform.h. Thus the compiler examines each definition against the SDK prototype.
• src/halo_linker_common.c gives weak storage for some globals of the January link, and for fast_ftol_C and main_crash.
• main/d3d_intimacy.cpp reads a private structure of the Xbox Direct3D. The Linux build does not use this file. src/d3d8_gl.c gives d3d_find_flipcount.
• The build returns small structures and unions in registers (-freg-struct-return), as on Win32.

Game source changes

Five files of the game have changes for clang. These changes do not change the MSVC objects: a comparison of all 612 C objects showed no difference in code or data.

File	Change
cseries/cseries.c	The naked function stristr uses [ebp+8] and [ebp+12] for its parameters.
bitmaps/bitmap_drawing.c	*((word *)p)++ is now *(*(word **)&p)++.
rasterizer/xbox/rasterizer_xbox_hardware_bitmaps.c	&(T *)x is now (T **)&x.
hs/hs.c	Local prototypes that did not agree with ai_script.h are removed.
units/vehicles.c	The local prototype of unit_update_animation uses the type of units.h.

math/real_math.h had a copy of plane2d_from_points that did not agree with the function in effects/decals.c. clang used the copy, and parts of levels were not visible. The copy now agrees with the function.

Other changes are in #ifdef HALO_LINUX. All the native ports define HALO_LINUX. The byte-matching build does not define it.

File	Change
scenario/scenario.c	The BSP connection tables have names, not MSVC offsets.
rasterizer/xbox/rasterizer_xbox_environment_fog.c	A local pointer gets its value from the file-scope array with the same name.
game/player_control.c	The mouse aims the player on controller 1 directly.
sound/game_sound.c	The game calculates the obstruction of each sound one time for each tick, not for each frame.
cseries/errors.c	debug.txt stays open between lines.
networking/, game/, interface/, bungie_net/network/ and the pools of objects, effects and sounds	The system link limits and the memory for them.
game/, objects/, units/, networking/	The distributed netcode. Refer to NETCODE.md.

The x86 inline assembly of the game has C replacements in #ifdef HALO_LINUX. Thus the compiler can optimize that code for each processor:

File	Assembly	Replacement
cseries/cseries.h	x87 fistp (fast_ftol)	__builtin_rint
bitmaps/bitmaps_inlines.h	x87 conversions	C conversions
math/matrix_math.c	SSE matrix4x3_multiply	a C loop
effects/decals.c	an x87 conversion	a C conversion
cseries/profile.c	rdtsc	QueryPerformanceCounter
cseries/cseries.c	naked stristr	a C stristr
cseries/stack_walk_windows.c	a read of EBP	__builtin_frame_address
interface/hud_draw.c	a read of [ebp+4]	__builtin_return_address(1)
bink/bink_playback.c	int 3	__builtin_trap

The x87 control and status words (_control87, _statusfp, _clearfp in src/msvc_crt.c) use fenv.h. On Android, they use the FPCR and FPSR.