Into the world: zones, entities and movement

Architecture walkthrough, part 3 of 5

Walkthrough contents Part 3 of 5
  1. Overview
  2. Logging in: from the launcher to character select
  3. The wire: from bytes to packets to actions
  4. Into the world: zones, entities and movement
  5. Fighting: spells, creatures, loot and quests
  6. Behind the scenes: data, database, events and the website

The player has picked a character and the client is loading the map. This page is about the world server: which processes it runs, how a player gets into a zone, what a zone does, and how movement travels from one player to the others.

The world server's processes

When the world server starts (BezgelorWorld.Application, apps/bezgelor_world/lib/bezgelor_world/application.ex), it registers its packet handlers and then starts a supervisor with about two dozen children. These are the ones this page is about:

The world server's processes BezgelorWorld.Supervisor supervises, among others: WorldManager (player sessions and the GUID counter); ZoneRegistry (finds a zone process by world and instance); InstanceSupervisor (one supervisor per world); Progress.Supervisor (one tracker per character); and the TcpListener on port 24000, through which Ranch starts one Connection process per player. InstanceSupervisor starts a WorldSupervisor for each running world, and each WorldSupervisor supervises that world's zone instances: world A has instance 1, world B has instances 1 and 2. A Connection and its Zone.Instance talk by messages. Grey arrows mean supervises: the parent restarts the child if it fails. BezgelorWorld.Supervisor strategy: one_for_one WorldManager player sessions, GUID counter ZoneRegistry finds a zone by {world, instance} InstanceSupervisor one supervisor per world Progress.Supervisor one tracker per character TcpListener port 24000, via Ranch World A WorldSupervisor World B WorldSupervisor Zone.Instance instance 1 Zone.Instance instance 1 Zone.Instance instance 2 Connection player A Connection player B messages Grey arrows: supervises (restarts the child if it fails). Pink dashed: messages. Not shown: eighteen more children (groups, guilds, mail, trade, chat, telemetry...).
  • WorldManager keeps the list of players in the world and hands out GUIDs.
  • ZoneRegistry is a Registry: give it {world id, instance id} and it returns the zone process for it.
  • Zone.InstanceSupervisor starts a Zone.WorldSupervisor for each running world, and each world supervisor starts that world's zone instances.
  • Progress.Supervisor runs one tracker process per character for achievements, titles and reputation, apart from the connection and the zone, so a zone change doesn't interrupt it.
  • The TcpListener on port 24000, through which Ranch starts one BezgelorProtocol.Connection per player (The wire).

The others run groups, the group finder, guilds, mail, chat and friends (Social.Supervisor), trades, the marketplace, login notices, bug reports and the server's metrics. Each is its own process, so a fault in mail can't stop combat, and the supervisor restarts whatever fails.

Entering a zone

When the client has loaded the map it sends ClientEnteredWorld. BezgelorProtocol.Handler.WorldEntryHandler (apps/bezgelor_protocol/lib/bezgelor_protocol/handler/world_entry_handler.ex) handles it, in the player's connection process:

  1. It builds the player's entity from the character record (BezgelorCore.Entity.from_character/2), with a GUID made from the character id (BezgelorCore.EntityId.player/1).
  2. It works out the zone key, {world id, instance id}. Open-world zones have one instance, number 1; dungeons, expeditions and raids get a separate instance per group or player (instances).
  3. It finds the zone process, starting it if needed: Zone.InstanceSupervisor.get_or_start_instance for the open world, BezgelorWorld.Instances.ensure_running/1 for instanced maps.
  4. It asks the zone to take the player in: Zone.Instance.enter/5, a call that waits for the zone's answer (at most ten seconds).
  5. It tells the client about the creatures around it (BezgelorProtocol.CreatureView.enter_world/3) and sends ServerPlayerEnteredWorld. The loading screen goes away.

The handler can run twice without harm, and it takes the player out of their old zone first when the key changes, which is what happens on a teleport.

A zone instance

BezgelorWorld.Zone.Instance (apps/bezgelor_world/lib/bezgelor_world/zone/instance.ex) is the heart of the server. There is one zone instance process, a GenServer, for each running {world, instance}, and it is the authority for everything in it.

Inside a zone instance Three connection processes, for players A, B and C, send messages such as move and cast to one Zone.Instance process. The zone instance owns the players (entity, spells, quests), the creatures (Zone.Creatures), a spatial grid of who is near whom, loot and duels, and runs a tick every 250 milliseconds for creature AI, auras and regeneration. It handles one message at a time. It reads the game data in ETS. When something happens it encodes the packet once and sends the same bytes to each player's connection. Connection player A Connection player B Connection player C move, cast Zone.Instance one process per {world, instance} players entity, quests creatures Zone.Creatures spatial grid who is near whom loot, duels and more tick, every 250 ms creature AI, auras, regeneration handles one message at a time Packet encoded once, sent to each connection Game data ETS tables reads the same bytes go to every player's connection

What it owns:

  • players: each player's entity (position, health and the rest), their connection process, their spell state (Spell.Caster), their quests (Zone.PlayerQuests), their path missions, mount and more;
  • creatures: their state, AI and combat (Zone.Creatures, Zone.CreatureCombat);
  • a spatial grid (BezgelorCore.SpatialGrid), cells of 50 units, so "who is near this point?" is a quick lookup rather than a check against every entity;
  • loot on the ground, duels, pets and the zone's scripts.

Because one process owns all of this and handles one message at a time, nothing in a zone needs a lock: a spell and a movement for the same creature can't happen at the same moment. Other processes never reach into the zone's state; they send it messages, almost always through the functions at the top of instance.ex (enter/5, leave/2, player_moved/4, cast_spell/3 and so on).

The tick. Every 250 milliseconds the zone sends itself a message and runs its tick: creature combat, pet and creature casts, scripts, periodic auras and their expiry, regeneration and shields. Every fourth tick (once a second) it also runs idle creature movement and aggro checks, resource regeneration and loot clean-up. On a continent with tens of thousands of creatures, those far from every player sleep until someone comes within 200 units (Zone.CreatureActivity); a published benchmark has the before and after.

Telling players. When something happens, the zone encodes the packet once and sends the same bytes to each player's connection. Players in a zone see each other everywhere in it; creatures are introduced to a player within 200 units. BezgelorWorld.CreatureViewers, an ETS table, remembers which connections know about which creature, so updates go only to those.

Watching connections. The zone monitors each player's connection process. If the connection dies, for example because the client crashed, the zone gets a message and removes the player, even though the connection never said goodbye.

Starting and stopping zones

Open-world zones with spawn data, and the two arkship tutorials, start when the server boots (BezgelorWorld.Zone.Manager), so their world bosses and public events run before anyone arrives. Other zones start when the first player enters. Instanced maps start for the group that enters them and stop after ten minutes empty (BezgelorWorld.Instances.Rules.empty_unload_ms/0); BezgelorWorld.Instances.Manager keeps track of which group owns which instance, lockouts and resets.

Each world's Zone.WorldSupervisor is the failure domain for that world: a zone instance that crashes is restarted, and one that keeps crashing takes down only its own world's supervisor, never another world.

Entities and GUIDs

Everything a player can see or target is an entity (BezgelorCore.Entity): players, creatures, objects, vehicles, corpses. Each has a 32-bit unit id, its GUID. BezgelorCore.EntityId (apps/bezgelor_core/lib/bezgelor_core/entity_id.ex) splits the number range by type, so the id alone says what kind of thing it is:

Kind GUID
Player 0x2000_0000 + character id
Creature 0x4000_0000 + a counter
Object 0x6000_0000 + a counter
Vehicle 0x7000_0000 + a counter
Corpse 0x8000_0000 + a counter

The counter (WorldManager.generate_guid/1) is an atomic counter that any process can bump without asking WorldManager, and it survives a restart of WorldManager, so ids are never handed out twice.

Movement

The client sends where the player is going as ClientEntityCommand messages. EntityCommandHandler, in the player's connection, does three things with each:

  1. relays the movement to the other players as ServerEntityCommand, through the zone;
  2. tells the zone the new position (Zone.Instance.player_moved/4, a cast), which updates the entity and the grid, interrupts a cast if the player really moved, and lets quests, duels and path missions react;
  3. keeps its own copy, saved to the database at most every five seconds (MovementHandler).

Creatures move on the server: BezgelorCore.Movement builds their paths and BezgelorCore.Spline follows the client's own spline data for creatures that patrol along one.

Leaving

When the player logs out or the connection drops, the connection process ends and, on its way out, calls Connection.leave_world/2. That asks the zone to remove the player (Zone.Instance.leave/2), which returns the zone's final copy of the entity, and saves the position (BezgelorDb.Characters.update_position). Quests, experience and currencies don't wait for this: they are written to the database as they change.

Where to look

  • apps/bezgelor_world/lib/bezgelor_world/application.ex: the world server's supervision tree.
  • apps/bezgelor_world/lib/bezgelor_world/zone/instance.ex: the zone instance (the moduledoc at the top is a good start).
  • apps/bezgelor_world/lib/bezgelor_world/zone/: the zone's helpers (creatures.ex, creature_combat.ex, player_quests.ex, creature_activity.ex, ...).
  • apps/bezgelor_protocol/lib/bezgelor_protocol/handler/world_entry_handler.ex, entity_command_handler.ex, movement_handler.ex.
  • apps/bezgelor_core/lib/bezgelor_core/entity.ex, entity_id.ex, spatial_grid.ex.

Next: Fighting.