Bezgelor

How we work

Bezgelor is a WildStar server written in Elixir, and a lot of what you can see of it from the outside is a long list of small fixes: a quest that now completes, a boss that now uses the right ability, a packet the client stops rejecting. This page explains the habits behind that list, for players who wonder why a report gets the answer it gets, and for anyone thinking of contributing who would like to know what a change goes through before it reaches the game.

None of this would exist without NexusForever. Its message models, names and captured world data are the ground we build on, and we try to work in the same spirit: write down what we found, and offer it back. Where our process is stricter than we would need to be, that is mostly because their work set a standard worth keeping up.

What it runs on

Besides the usual Elixir and PostgreSQL, two tools of our own do the heavy lifting:

  • spoond spoond on GitHub hands out short-lived, isolated virtual machines (Firecracker microVMs) on request. A machine starts from a snapshot in moments, gets a fresh copy of the repository, does one job and is thrown away.
  • Honey, to be released, runs work as flights: it takes a ticket from the tracker, leases a microVM from spoond, takes the work through implement, check and review rounds, asks a person when it must, and pushes the result as a branch. It also builds the machine images, so every machine starts with the same toolchain, the compiled dependencies and the client oracle.

Both run on a spare server on our own network, set aside for testing, so a batch of machines never slows down the development machine or a game anyone is playing.

The loop

The loop: from play to play Nine steps in a circle. Play and reports from joint sessions lead to triage, then a written spec, then building on an isolated branch, then testing (the test suites and the client oracle), then independent review, then merge, then automated checks (audit and sweep), then published reports. The reports lead back to play, where players check what changed. Play and report joint sessions Triage issues and scope Spec written first Build own branch Test suites and client oracle Review by someone else Merge blockers fixed Checks audit and sweep Reports published in full every pass makes the next round of play more useful Tests, review and checks can send a change back; it is not merged until they agree.

Everything starts with somebody playing. Maintainers and players test together in the real client, and anything that looks wrong is reported from inside the game, with /c bug or the Report a Bug window (see reporting a bug). Those reports land in a tracker.

Triage turns reports into issues and decides what each one is really about. The scope today is a solid experience from level 1 to 15, for both factions, every class and every path. Higher levels come after that release, and nothing is sped up to get there sooner. A bug at level 40 is written down, not forgotten, but it waits.

An issue then goes through a written spec and an isolated branch. Before anyone reviews it, the change has to pass its tests: the test suites, including the new tests the spec named, and the client oracle, which runs every packet the change touches through the real client's own reader (see testing in layers). Only then does it go to an independent review (the next sections). After the merge, automated checks run over the whole game and their results are published. Then we play again, and the loop closes: the reports that come out of the next session tell us whether the last round worked.

Specs before code

Each change starts as a short written spec. It has four parts:

  • The problem, with evidence. A file and line, or a way to reproduce it. "Quests feel off" is not a spec; "using this quest item does nothing, and the message it sends has no handler" is.
  • The general mechanism to build. We would rather build one thing that explains many cases than fix one quest.
  • The tests that prove it. Named before the change is written.
  • What counts as done.

The preference for one mechanism over many special cases is the one we hold hardest. A real example: quest credit for using something in the world now comes when an activation spell actually succeeds, and it is counted once per unit for each objective. Before, each quest that went wrong would have asked for its own patch. The research note behind it, using a unit, has the evidence, and item use covers the sibling problem of items used from the bag.

Isolated branches

Every change is built in its own git worktree and branch, with its own test database (the repository's testing notes call this one test database per checkout). Parallel work therefore never collides: two changes can run their whole test suites at the same moment without blocking on each other's transactions.

Bulk, data-driven jobs go to a pool of build workers: Honey flights on spoond machines. Placing missing quest NPCs from the client's location tables is a good example: over a hundred similar records, each with a clear rule. Each worker runs in its own virtual machine, clones the repository, pushes a branch and runs its own verification before it reports back. The coordinator writes the spec and checks the result; the workers do the repetitive middle.

Independent review before merge

One change's path A change starts as a written spec, is built on its own branch and worktree, is tested, and goes to an independent reviewer. If the reviewer finds blockers the change goes back to be fixed and tested again. Once approved it is merged, then the coverage audit and sweep run, and the results are published as a report. Spec problem, tests, done Branch own worktree, own DB Tests new test fails without fix Review reads diff against spec Merge only when approved Audit and sweep rerun on merged code Report published in full blockers: fix, then review again

A second reviewer, someone who did not write the change, reads the diff against the spec. They check each finding against the code rather than taking it on trust, run the tests themselves, and look for tests that were quietly weakened to make a change pass. We also check that a new test fails without the fix. A test that passes either way proves nothing.

A change goes back until its blockers are fixed. Review earns its keep here, with a real case. A rule of "an activation counts once per unit" looked right, and it would have made some quests impossible, because the game data had fewer spawns than the quest needed. The reviewer caught it by checking the spec against the data, not just the code. The fix tied the rule to each unit's life, so a respawned unit counts again.

Testing in layers

Seven layers of testing A ladder of seven kinds of test, from cheap and fast at the top to slow and close to the real game at the bottom: unit tests of game rules, zone tests with fake connections, handler tests with real packet bytes, the client oracle that reads our packets with the real client's own parser, headless clients over the real network listeners, the static coverage audit of every level 1 to 15 quest, path mission and quest item, and playing in the real client together. cheap and fast 1 Unit tests game rules as plain functions, no processes 2 Zone tests a real zone process with fake connections 3 Handler tests real packet bytes in, the packets out are checked 4 Client oracle our bytes read by the real client's own parser 5 Headless clients real listeners: end-to-end, levelling bot, sweep 6 Coverage audit every level 1-15 quest, mission and item vs the code 7 Playing together the real client, with players slow, and close to the real game

Each layer catches things the one above it cannot, and costs more to run.

  1. Unit tests of game rules. Damage, experience, loot and quest logic as plain functions. Fast, and they run all the time.
  2. Zone tests. A real zone process with fake connections, checking what a zone does when players move, cast and leave.
  3. Handler tests. Real packet bytes go in, and the packets that come out are checked. See the protocol notes.
  4. The client oracle. Our packet bytes are read by the real client's own parser, run under emulation. If the client would reject a packet, the test fails. See the oracle.
  5. Headless clients over the real network listeners. End-to-end tests, the levelling bot and the coverage sweep connect the way a player's client does, through the auth, realm and world servers.
  6. The static coverage audit. Every level 1 to 15 quest, path mission and quest item is checked against the server's code paths and data, without playing anything. The latest is the coverage audit of 2026-10-16.
  7. Playing in the real client, together. Nothing replaces it, and it is where the loop starts again.

scripts/test_all.sh runs every app in order and prints a table, then says whether the two optional parts actually ran. The database tests need PostgreSQL; the client-oracle tests need the real client's files, which are not part of the repository. When either is missing those tests are skipped, and the script says so rather than reporting a green run that quietly tested less. scripts/test_all.sh --no-oracle leaves the oracle out on purpose.

Evidence, and honesty about it

The thing we most want to avoid is a guess that looks like a fact. Every data record carries a label: reference (taken straight from a retail source), derived (computed from retail data by a rule we wrote) or guess (designed or extrapolated, with the reasoning written down). The convention is in provenance, and the values we chose ourselves are listed in designed-values.md.

What we know and what we have only inferred is spread over several pages. The fidelity map rates every system and zone on one scale, wildstar-delta.md lists where we differ from retail and how sure we are, and ground-truth.md says what survives of how the live game behaved. Research notes, one per investigation, are in docs/research: the evidence and the decision, written down at the time.

Publishing the results

Test reports go up in full under test-reports/: levelling runs, audits, sweeps and encounter simulations, with the files that let you rerun them. They use test characters on a test server, never real players. Bad news is published too; a run that stalls is more useful to the next person than a run that was quietly left out. If a report does not match how you remember the game, we would like to hear it.

How to help

  • Play and report. /c bug in game is the fastest route, and a report that says what you were doing helps most.
  • Share a memory or a recording. Footage, notes and your recollection of how a fight or a vendor behaved can settle questions that the client files cannot. Ground truth says what is most useful.
  • Pick up an issue. Write a short spec first, as described above, before you change anything. It is quicker than it sounds, and it makes the review easy.