Network Architecture

Authoritative project content

[RISK] This is the highest-cost, highest-risk area of the project. Persistent, server-authoritative, raidable PvPvE for ~40–60 players is genuinely hard. Treat the networking + persistence backbone as a Phase-1 first-class deliverable, not something to retrofit.

Model — [CONFIRMED]

• Dedicated-server authoritative. The server owns all gameplay truth: damage, inventory, stats, building, loot, AI. Clients are never trusted.
• No listen-server / P2P for live play (a listen-server mode may be used purely for local testing).
• ~40–60 players per server instance, plus AI hordes.
• Linux dedicated server build target (MetaHuman plugins are already configured for Win64 + Linux).

Core principles

1. Server authority everywhere. Clients send requests (RPCs/input); the server validates and applies. This is the foundation of anti-cheat for a full-loot/raid game.
2. Client prediction for feel. Movement and firing feedback are predicted client-side and reconciled with the server.
3. Replication discipline. With ~40–60 players + AI + bases, naïve replication will not scale. Use relevancy, dormancy, update-frequency tuning, and consider the Replication Graph for relevancy management. [RECOMMENDATION], validate need by profiling.
4. Lag compensation for hitscan combat (server rewind) so shooting feels fair. [RISK] non-trivial; budget for it.

Replication hotspots (plan for these)

Persistence backbone — [RISK], design [CONFIRMED]

Persistent world means the server must save and restore world/player state across restarts:

• Player data: inventory, equipment, survival stats, unlocked recipes, permanent boss buffs, position/respawn point, clan/squad membership.
• World/base data: placed building pieces (transform, tier, HP, ownership), container contents, deployables, vehicles, territory anchors.
• Loot/world state: container respawn timers, dropped death-bags (with despawn timers).

Confirmed storage model

• Dedicated-server durable store: one SQLite database per server world stores world state, player records, inventories, buildings, vehicles, and containers.
• Live state: authoritative gameplay state remains in dedicated-server RAM while the server is running.
• Write path: dirty records are saved asynchronously and periodically so gameplay does not block on disk I/O. Critical transitions such as logout and graceful shutdown request a final flush.
• Client storage: UE SaveGame is restricted to local settings such as graphics options and keybinds; it never stores authoritative gameplay state.
• Evolution: SQLite schema and records are versioned and migrated. Gameplay code accesses storage through the save subsystem rather than issuing database operations directly.

P07 player/world contract — [CONFIRMED 2026-09-14]

Owner resolved the P07 load, recovery and first durable-record rules below. Runtime now uses one SQLite file per world through USaveSubsystem; the in-memory backend remains only while no world is open. Building, vehicle and craft-queue records stay out of this increment and are added when those systems land. Owner-reported pass 2026-09-14 on Dedicated Server + two Preview clients closed ROAD-P07.

Single Player topology

[CONFIRMED 2026-09-14] TBD-HUD-UI-003: shipping Single Player uses embedded authority as NM_Standalone in the same process. This preserves the confirmed SP world-pause rule. Multiplayer live play remains dedicated-server authoritative. Listen-server stays local-test only. A later local dedicated process is optional hardening, not the P07 topology. Frontend save-list UI stays session-agnostic.

Write cadence and crash safety

• Dirty records flush asynchronously every 30 seconds of world-up time. This is the approved P07 interval, not a later tuning pass.
• Single Player Pause exposes a manual required-player Save. Exit to Main Menu measures real elapsed time from the latest durable snapshot (the loaded snapshot is the initial baseline); when that age is more than 30 seconds, a successful required-player flush is mandatory before travel. A failed required flush cancels the exit.
• Logout, graceful shutdown, death-bag spawn, death itself, authored pickup collection, and runtime world-drop spawn request a final flush.
• Use SQLite. Persist only committed transactions. A crash mid-write must not yield a half-applied player or world record. The P07 contract asked for WAL; UE 5.8 SQLiteCore is compiled with SQLITE_OMIT_WAL, so runtime uses BEGIN IMMEDIATE / COMMIT plus the engine rollback journal instead of write-ahead logging.
• Full backup/restore operations remain later ops work; WAL plus explicit failure states are the P07 recovery surface.

[PAUSE SAVE/EXIT IMPLEMENTED 2026-09-15; OWNER PIE PENDING] The 30-second exit threshold uses monotonic real elapsed time, so time spent in a paused Standalone world is still counted. Manual Save and a required pre-exit Save both use the existing required-player capture, transaction and read-back verification path. The loaded durable state establishes the initial time baseline; every successful snapshot refreshes it. Threshold boundary automation, the closed-Editor native build, Persistence 8/8, HUD 24/24, Frontend 4/4 and WBP_PauseMenu compile/save/readback pass. The existing GameplayCueNotifyPaths fixture warning remains unrelated. Owner PIE must validate the visible buttons, save feedback, under/over-30-second exit behavior and multiplayer absence.

Load validation and failure recovery

[P08 PROBE FIX 2026-09-15; OWNER-REPORTED PASS 2026-09-16] World-list probing must preserve its input database path independently from the output summary. A caller previously supplied Summary.DatabasePath as the input while the same Summary was reset as output, producing a false Missing result for an intact database. ProbeFile now copies the path before resetting output and the caller passes a separate local value. The reported owner database remained intact and passed schema 1, metadata and SQLite quick_check; post-fix native build and Lambeer.Persistence 6/6 pass with direct alias regression coverage. Owner later reported Save World Select/Continue probing passed.

[P08 REQUIRED-PLAYER FLUSH HARDENING 2026-09-15; OWNER-REPORTED PASS 2026-09-16] Owner PIE showed that a player could commit initial appearance, enter gameplay, move, run Lambeer.SaveFlush, receive ok, and still have no durable player row on the next entry. A player-scoped flush must now capture the supplied Controller's ALambeerPlayerState explicitly, merge that record into the snapshot, commit it, and read it back successfully before reporting success. Initial appearance commit, first starter-town selection, logout and Lambeer.SaveFlush use this required-player path. World-only periodic flushes retain the aggregate path. Pre-commit RequiresCreation/Error states remain deliberately non-durable and cause a required-player flush to fail rather than return a misleading ok. Owner later reported remaining persistence/restart checks passed.

P07 persisted player record

Lifecycle correction validation: Closed-Editor LambeerEditor Win64 Development build passed. Frontend 4/4, HUD 24/24 and Persistence 7/7 (including the new early-read/Controller-destruction regression) passed. The fixture's first GAS initialization emitted the existing missing GameplayCueNotifyPaths configuration warning. Owner later reported Continue skips Character Creation and remaining persistence/restart checks passed.

[SPAWN RECOVERY IMPLEMENTED 2026-09-15; OWNER-REPORTED PASS 2026-09-16] The owner reports an underground-looking camera and no character after Continue. Live MCP logs at 15:43:49 and 15:43:58 UTC show successful lookup of pie-0 in the existing Apple world, restoration at (0,0,0), then BP_PlayerCharacter_C spawn failure due to collision. The owner requested implementation after the initial diagnostic-only handoff. RestartPlayer now checks whether saved-position spawning produced a pawn. On failure, the map's normal PlayerStart path supplies a holding pawn, the saved appearance/loadout/survival state is restored, and a living character is locked in starter-town selection without reopening Character Creation. Dead records retain the normal death/bed/town flow and are not resurrected. If fallback also fails, entry becomes Error and returns to Main Menu with an explicit failure; capture cannot overwrite the existing row. No hard-coded recovery coordinates, forced collision bypass, schema bump, or direct owner-database repair is used. Owner later reported remaining spawn-recovery gameplay checks passed.

Recovery persistence and validation: Pending living recovery uses the existing bInitialSpawnChosen=false capture/restore route, so saves and interrupted recovery preserve the town-selection requirement and restored data. The chosen town returns entry to Ready and uses the required-player flush. Valid saved-position spawning remains unchanged. Closed-Editor LambeerEditor Win64 Development build passed; MCP Frontend 4/4, HUD 24/24 and Persistence 8/8 passed (36/36). The existing early-entry fixture emitted the known GameplayCueNotifyPaths warning; the new Lambeer.Persistence.SavedPositionSpawnRecovery passed without warnings/errors. It exercises opposing collision geometry, actual GameMode restart/possession, restored inventory/attributes, interrupted recovery reentry, town selection/write/readback, subsequent normal Continue, dead-record recovery and total spawn-failure preservation in an isolated temporary database without PIE, BeginPlay or world ticking. Owner later reported Continue, town selection, visible controllable character, move/Lambeer.SaveFlush and restart/Continue without another creation/town prompt passed. Previously overwritten data remains unrecoverable from the current row.

[P08 LOAD-ORDER ROOT CAUSE CORRECTED 2026-09-15; OWNER-REPORTED PASS 2026-09-16] The owner reported that required-player flush hardening did not fix Continue. Live logs prove appearance commit, starter-town selection, SaveFlush and readback all succeeded for pie-0, then the next RestartPlayer reported Missing before OnWorldBeginPlay activated persistence. The old TryLoadPlayerRecord returned Missing whenever bPersistenceActive was false without querying the already-open database. Reads now ensure the authoritative store is open independently of periodic capture activation; BeginPlay reuses that store. Storage unavailability returns Unavailable and refuses entry instead of creating a replacement character. This supersedes the earlier missing-player-in-snapshot diagnosis; required-player write/readback checks remain useful but were insufficient. Owner later reported remaining Continue/restart persistence passed.

Exit capture: UE calls APlayerController::Destroyed before AController::Destroyed invokes GameMode Logout; the former destroys or unpossesses the pawn. Capture now runs before the PlayerController superclass teardown. A pawnless PlayerState cannot overwrite a complete record with default zero transform and empty inventory. Existing records already overwritten before this fix are not reconstructed automatically. The non-PIE Lambeer.Persistence.EntryBeforeBeginPlay regression uses an isolated GameInstance/world and temporary SQLite store to verify early reads, genuine missing identity, unavailable service and teardown preservation; no PIE/BeginPlay is started.

• Identity and FAppearanceState (gender, preset, hair, skin, eye).
• bInitialSpawnChosen distinguishes a committed appearance awaiting starter-town selection (initial entry or living saved-position spawn recovery) from a returning playable character. It is backward-compatible as true for records written before P08. Appearance is retained during recovery; town choice marks the field and flushes the record.
• World transform/rotation and last bed (the bed that last started Sleep).
• bDead, death location, death-bag id, killer name and cause. A dead record restores the spawn-select screen, not a playable locked corpse. Lambeer.Kill on an already-dead character reopens that screen.
• Inventory, equipment, held hands, nested contents, durability, magazine contents, chamber, attachments, item BloodAmount/WetAmount, and character body/hair blood and wetness.
• Survival attributes that exist in the session (including Energy/Sleep/Immunity).
• Q1–Q0 bindings as references to existing item instances, never cloned stacks.
• World rain intensity so a wet character does not reload under a clear sky.

Player rows are keyed by a durable login LambeerPlayerKey, not by OnlineSubsystemNull’s per-session RoboPC-* UniqueNetId. Dedicated Server + two PIE Preview clients use pie-{PIEInstance} so the same two windows rejoin the same two characters. A real online UniqueNetId is used when it is durable. Packaged offline identities fall back to a local settings UUID. Server Browser join now appends that same login option to the connect URL. Dedicated and listen persistable worlds advertise a Null/LAN session for development discovery. One-process PIE lists those hosts by listen port because OnlineSubsystemNull’s LAN beacon cannot FindSessions against itself; a PIE client returning to Main Menu must not destroy the hosted GameSession. Steam discovery remains blocked on TBD-HUD-UI-004. Owner reported Null/LAN Server Browser Refresh/Join 2026-09-16.

Do not persist observed chamber/magazine HUD knowledge. Unlocked recipes, boss buffs, clan/squad membership, craft queues, buildings and vehicles wait for their owning phases. P07 does not persist AI blood or AI wetness; those remain open in their feature documents.

Death bags and runtime world drops

• Death bags and runtime world drops (player drop, throw-to-pickup, drop-trade) persist across restart. Restart itself never deletes them.
• Each uses a 24-hour real-time remaining despawn that counts only while that world is running. Remaining time is saved. Time spent while the world is shut down does not consume the timer; after reopen the leftover duration continues.
• Authored map-placed pickups remain level actors. They do not receive this 24-hour runtime-drop timer unless they become a runtime drop. After a player collects one, its stable actor id is stored in world state and that instance is not restored on later loads. Container loot-table respawn timers stay later world/loot work.
• Place collect-once sandbox items as AWorldItemPickup / BP_WorldItemPickup (or a child such as BP_Pickup_Sledgehammer_Test) in the map with a unique actor label and a non-empty Stack. Do not spawn them from BeginPlay every session, and do not use Lambeer.SpawnPickup for fixtures that must stay collected — that cheat creates a runtime drop with the 24-hour timer.

Local world index

• Each local world is one SQLite database, same persistence semantics as a dedicated server world.
• Players may create, rename, list and delete worlds. Delete requires one confirmation and removes that world’s database.
• P07 has no world-count cap.
• The remembered-world pointer for Continue lives in client settings, not inside a world database. Start Single Player always opens the save list and never overwrites automatically.

Full vehicle records — [CONFIRMED]

Vehicles use full gameplay-state persistence, not a transform-only respawn record. The stable vehicle record and explicitly linked child records preserve identity/class, world placement and original spawn point, cabin lock state, ignition/key requirement, permanent Hotwire state, inserted-key link, overall and component condition, fuel and implemented operational resources, integrated inventory with complete item-instance state, installed parts/upgrades/cosmetics, and lifecycle timestamps/flags. The authoritative field-level contracts live in Vehicles — Full vehicle persistence and FEAT-LOCK-ACCESS.

Lock and access records — [CONFIRMED DESIGN 2026-09-17]

Later P16/P17/P23 records persist each one-time initial lock result, current built-in state, independent installed Lock-item link/owner, exact Vehicle-ID key binding and current item location, original vehicle spawn point, inserted key and permanent Hotwire. Restoring or streaming an existing record never rerolls the 30% world door/window or 90% vehicle initial chance. Squad access is authorized from current server membership rather than copied as a stale durable allow-list. This design is outside the implemented P07 player/pickup record scope and must extend USaveSubsystem / ULambeerWorldPersistenceSubsystem with matching Lambeer.Persistence automation when implemented.

[RISK]: P07 cadence, committed transactions and explicit corrupt/incompatible entry states are implemented. Schema-migration automation shipped with the SQLite backend (Lambeer.Persistence 6/6). Engine SQLite omits WAL, so rollback-journal atomicity is the crash-safety surface until a later ops/SQLite change. Full backup/restore remains later ops work before persistent-world release.

Connectivity & ops (later phases)

• Matchmaking/server-browser, sessions, and a backend for official servers are Phase-2/3. See 02-Servers-and-Hosting.md.
• Anti-cheat: Easy Anti-Cheat (EAC) — [CONFIRMED]. Server authority remains the first line of defense; EAC integration and operational hardening are required before launch.

Implementation guidance

Measure server CPU and each client connection with TECH-PERFORMANCE-PROFILING — Dedicated Server + Client1 + Client2. The runbook separates shared-process PIE, isolated captures, replication traffic and persistence stalls; no capacity pass is implied.

• Build gameplay systems server-authoritative from day one — retrofitting authority is far more expensive than designing for it.
• Use GameState/PlayerState for replicated shared/per-player state; keep large/private data in owner-only-replicated components.
• Co-locate persistence with a UGameInstanceSubsystem/UWorldSubsystem save service. See ../05-Technical/04-Source-Code-Structure.md.