Architecture

Why a schema database instead of hardcoded layouts

A UE3 tagged property carries its name, type name, size and array index — and nothing else. The element type of a TArray, the field layout of an immutable struct, the labels of an enum, and the entire native serialization after the property stream are all absent from the tag. That information lives in the engine's reflection objects, which are themselves just exports in Core.u, Engine.u and friends.

So rather than shipping a type table per game, UE3Tools reads those exports. Type recovery follows the import graph on demand and caches as it goes; nothing is game-specific.

Extraction pipeline

bytes → .uo + sidecars

1. Open and flatten — upk_header_cursor() Reads the header. If compressed, every chunk is LZO-inflated and a flat in-memory image is rebuilt with a rewritten header, preserving inter-chunk gaps and the trailing bytes. Everything downstream sees one contiguous buffer, so compressed and plain packages take identical paths.

2. Read the tables — UPKPak::parse_upk() Names, exports, imports. UPKPak then supplies UE's naming rules on top — . between levels, : before a subobject.

3. Index the game root — SchemaDb::new() One directory walk builds stem → path maps for packages and for .tfc files. No package is opened yet. The package being extracted is injected into the cache so it's never re-read.

4. Walk the exports — extract_by_name() Per export: compute the full name and mirrored path, slice serial_offset..+serial_size, resolve the export's class to a ResolvedRef (positive index → same package, negative → import resolution through the database).

5. Decode — write_extracted_file() Three branches. Meta-objects render as UnrealScript declarations. Regular objects read a net index, then the tagged property stream up to its None terminator — everything after that is the native tail. The tail goes to the class's NativeSerializer if one is registered, else to a positional walk of the class's CPF_Native properties, else it's preserved verbatim as hex.

6. Emit — pseudo::write_uo_file() Header comments, property lines, the @native(...) block, one @sidecar line per file written.

Packing pipeline

pack-mod is an overlay, not a regeneration. It starts from the decoded original and applies only what the .uo changed:

1. Collect and parse every .uo; skip declaration-only files; group by the pkg= token.
2. Per package: re-open the original, build a schema database, copy the name table.
3. Per object: locate the original export (full path first, then export=#), re-decode its original properties, and merge — matched properties are overlaid value-by-value with the original's type as authority, None drops a property, and a property that exists only in the .uo is created from the schema.
4. Re-inject sidecars via NativeSerializer::inject_external.
5. Serialize net index + property tags (sizes back-patched) + native tail → <Dotted.Path>.bin. Write the grown name table as .namemap, and any invented export slots as .newexports.

Module breakdown

src/ — 17 files, ~370 KB.

Execution model

Single-threaded and synchronous throughout. SchemaDb is interior-mutable (RefCell caches, Rc<SchemaEntry> handles) and passed by reference down the whole call tree. Packages are held whole in memory — there is no streaming or mmap.

Failure model

Three non-fatal strategies, used deliberately:

Type-graph cycles are broken by a visiting set. Every linked-list walk is bounded — 8192 children, 64 superclasses, 256 outer hops.