Self-Play Calibration of Heuristic Agents for Mexican Train Under Competing Objectives

Digital Defiance
https://github.com/digitaldefiance/warp12

2026-07-18

Abstract

Mexican Train dominoes is commonly played under two incompatible victory conditions: points (lowest pip total when the round ends) and go-out (first player to empty their hand). This paper describes the development, calibration, and empirical validation of AI agents for both objectives in Warp 12, an open-source Mexican Train implementation. We introduce the Tactical Effectiveness Index (TEI), a dual-track OpenSkill-based rating system with gamified grade display, anchored to fixed AI reference tiers.

Empirical results from thousands of self-play games show that skill ordering is stable under the points objective (consistent rating gaps between tiers), while go-out exhibits higher variance and compressed skill gaps. We demonstrate that search algorithms provide different advantages depending on context: expectimax wins in 2-player points (64% vs heuristics), while ISMCTS excels in 4+ player go-out (31% vs 23% baseline).

A 274,500-game points self-play campaign across 549 configuration cells (4 Warp factors ×\times every legal fleet size ×\times module configs, with Zeta only on eligible even fleets) is the skill/luck instrument for board choice and points-module taxonomy. From the baseline cells alone we find that skill expression increases with both tile set size and player count, contradicting the intuition that larger games devolve into luck, and that Warp factor 12 is the empirically optimal choice for competitive rating. Across the full points matrix, most optional modules preserve the baseline skill ceiling; Module Iota (Double Down) slightly raises it. Module Epsilon (Drafting) collapses it (1.08/4 indicators) — counter-intuitively, choosing your hand makes outcomes more luck-dominated mid-game — and is therefore designated Warped / party. Module Zeta (Squadrons) also preserves skill (2.94/4); it never writes free-for-all (FFA) TEI — rated Warp 12 crew sectors update a dedicated Squad TEI track (squadRating) after team-track calibration (SQUADRONS_RATING_CALIBRATED), not because Zeta is luck-dominant.

A separate 255,500-game go-out campaign (511 cells; Epsilon omitted) evaluates go-out module forks (Salamander Surge, Trail Momentum, Desperation Dig, Hand Exchange, Hot Potato pass). Those mechanics are not apples-to-apples with points; we report them in their own figures and do not merge SS averages across objectives except in one deliberate shared-ID contrast plot.

Finally, we report on Class Ω\Omega, a neural self-play agent that achieves parity with heuristic baselines in small games and outperforms in large fleets. All code, data, and reproduction scripts are available as open source.

1 Introduction

1.1 Mexican Train Dominoes

Mexican Train is a popular multi-player domino game typically played with a double-twelve set (91 tiles with pips ranging from 0 to 12). Players build linear sequences called trains from a central engine (starting double), attempting to minimize their hand’s pip count. The game features both private trains (one per player) and a shared public train (the “Mexican Train” or Neutral Zone), creating strategic tension between building one’s own position and blocking opponents.

Unlike classic domino games with a single win condition, Mexican Train is commonly played under two incompatible objectives:

These objectives create fundamentally different strategic games on the same rules engine: points rewards consistent pip minimization and flexible play across many turns, while go-out rewards tempo, connectivity, and race dynamics.

1.2 Warp 12: Terminology and Presentation

This paper describes work conducted in the context of Warp 12, an open-source Mexican Train implementation with a Star Trek-themed presentation layer. Throughout this paper, we use standard Mexican Train terminology in formal discussion, but introduce Warp 12’s thematic terminology where relevant to implementation details. The table below provides a complete mapping.

@lll@ Mexican Train & Warp 12 & Technical

Domino / tile & Navigational Coordinate & Coordinate
Engine (starting double) & Spacedock & engine
Player’s private train & Warp Trail & ownTrail
Mexican Train (public) & Neutral Zone & neutralZone
Train marker / forced play & Distress Beacon & beacon / shieldsDown
Unsatisfied double & Red Alert & redAlert
Draw pile & Uncharted Sectors & pile

Player & Captain & PlayerId
Skill rating & TEI grade & TeiDisplay (E/V/C/I/P + 0–99)
Underlying skill & OpenSkill (μ,σ)(\mu,\sigma) & PlayerRating
Game table & Sector / Fleet & game

Easy AI & Class IV (Ensign) & ensign
Medium AI & Class III (Lieutenant) & lieutenant
Hard AI & Class II (Commander) & commander
Expert (human only) & Class I & High TEI score / low σ\sigma

Warp factors refer to the maximum pip value in the domino set: Warp 9 (W9) uses double-nine tiles (55 tiles total), Warp 12 (W12) uses double-twelve (91 tiles), etc. Throughout this paper, we use compact notation WX/Yp to denote configurations: W12/4p means Warp factor 12 (double-twelve) with 4 players. This paper focuses on W12 as the primary rated configuration, with analysis of W9, W15, and W18 in Section 8.

1.3 Motivation

Despite Mexican Train’s popularity, it remains under-studied in the game AI literature compared to chess, Go, or poker. Several challenges make it interesting for AI research:

For online play, these challenges create practical design questions: How do we measure player skill fairly? How do we provide appropriately challenging AI opponents? How do we ensure coaching tools don’t contaminate competitive ratings?

1.4 Contributions

This paper makes the following contributions:

  1. Dual-track TEI rating system — We describe an OpenSkill-based rating with independent tracks for points and go-out objectives, anchored to fixed AI reference tiers rather than floating human populations. The system uses Bayesian inference with μ±σ\mu \pm \sigma skill estimates and presents ratings as gamified grades (E/V/C/I/P + 0-99 score).

  2. Self-play calibration methodology — We present a systematic approach to validating AI skill levels through tier-vs-tier matrices, symmetric seating tests, and multi-player focus matchups.

  3. Empirical objective comparison — Through thousands of self-play games, we demonstrate that points calibrates cleanly with consistent rating gaps between tiers, while go-out exhibits compression and higher variance due to its racing mechanics.

  4. Search algorithm analysis — We show that different search methods (expectimax, ISMCTS) provide advantages in different contexts: expectimax excels in 2-player points (64% win rate), while ISMCTS works better in 4+ player go-out (31% vs 23% baseline).

  5. Luck vs skill and module integrity — A 274,500-game points matrix (549 cells) measures skill expression across Warp factors, fleet sizes, and module configs for TEI taxonomy. A separate go-out matrix (255,500 games / 511 cells) evaluates forked go-out modules without conflating SS averages. Baseline cells justify rating W12; Iota slightly raises the points ceiling; Epsilon collapses it (Warped/party); Zeta preserves skill and rates on a dedicated Squad TEI track (never free-for-all / FFA).

  6. Neural self-play agent (Class Ω\Omega) — We demonstrate that a pure self-play neural policy can achieve competitive performance (parity in small games, advantages in large fleets) without hand-crafted heuristics, and discuss the challenges of mapping training performance to appropriate rating anchors.

  7. Open-source implementation — All code, data, and reproduction scripts are available through the warp12-engine package and associated repositories.

2 Related Work

This section briefly surveys prior work in domino game AI, game AI paradigms relevant to Mexican Train, and skill rating systems. We identify the specific gap this work fills.

2.1 Domino and Tile Games

Domino games have received limited attention in the game AI literature compared to board games like chess and Go or card games like poker. Most prior work focuses on simpler variants like straight dominoes or block dominoes, which lack Mexican Train’s multi-player train mechanics and dual objectives. The DoubleEighteen rendering library  provides visualization for domino games but does not include AI agents. To our knowledge, this is the first published calibration study of AI agents for Mexican Train specifically.

2.2 Game AI Paradigms

Several AI approaches are relevant to imperfect-information multi-player games like Mexican Train:

Heuristic policies: Hand-crafted evaluation functions and decision rules remain competitive in many domains, especially where interpretability matters for player trust and coaching features. Our Class IV–II agents follow this tradition with weighted heuristics for pip dumping, trail pressure, and blocking.

Monte Carlo Tree Search (MCTS): MCTS and its variants  have achieved strong results in perfect-information games (Go, Hex) and some imperfect-information settings. Information Set MCTS (ISMCTS)  handles hidden information by sampling determinizations and maintaining search trees over information sets rather than individual states. We use ISMCTS for our Fleet Admiral and Class Ω\Omega+ search backends.

Deep reinforcement learning and self-play: The AlphaZero family  demonstrated that pure self-play with neural networks can surpass human expertise in chess, Go, and shogi. However, these successes required massive computational resources and perfect information. Our Class Ω\Omega agent represents a more modest self-play approach suitable for imperfect-information games with limited training compute.

Counterfactual regret minimization: CFR and its variants have achieved superhuman performance in poker , another imperfect-information game. However, CFR typically requires extensive offline precomputation and works best in 2-player zero-sum settings, making it less applicable to multi-player cooperative-competitive games like Mexican Train.

2.3 Skill Rating Systems

Several rating systems are widely used in competitive games:

Elo rating : Originally developed for chess, Elo computes expected win probabilities based on rating differences and updates ratings based on actual outcomes. It assumes transitive skill relationships and works best for 2-player zero-sum games.

Glicko : Extends Elo by modeling rating uncertainty (RD), allowing ratings to drift during inactivity periods and providing confidence intervals.

TrueSkill : Microsoft’s rating system extends to multi-player and team games using Bayesian inference. It models each player’s skill as a Gaussian distribution and updates beliefs after each match.

OpenSkill: An open-source implementation of rating systems inspired by TrueSkill and Weng-Lin ranking , designed for multi-player and team-based games. Like TrueSkill, it uses Gaussian skill models with mean (μ\mu) and standard deviation (σ\sigma) parameters, updated through Bayesian inference. We adopt OpenSkill as the foundation for TEI due to its multi-player support, open implementation, and flexible team rating capabilities.

Most online game platforms (Chess.com, Lichess, etc.) use Elo or Glicko variants. Some platforms that offer AI opponents (e.g., Lichess’s Stockfish bots) provide fixed rating estimates for bot difficulty levels, similar to our TEI anchor approach. However, we are unaware of prior work that systematically calibrates and validates AI skill tiers through self-play specifically for the purpose of providing stable rating anchors.

2.4 Contribution Relative to Prior Work

This work makes several novel contributions relative to the existing literature:

  1. Dual-objective calibration: We systematically compare points and go-out objectives on the same rules engine, showing empirically that they require different strategic optimization and exhibit different skill/variance tradeoffs.

  2. Fixed AI reference anchors: Rather than deriving ratings entirely from human populations (which can drift over time), we anchor rating bands to validated AI tiers with measured performance characteristics.

  3. Multi-player self-play methodology: Most game AI calibration focuses on 2-player matchups. We include 3–8 player configurations and focus matchups (one strong player vs multiple weaker opponents) to validate skill expression in realistic play scenarios.

  4. Luck vs skill empirics: The points module matrix (Section 9) measures how tile set size, player count, and optional modules affect skill expression for TEI — including Warped/party designation for Epsilon. The go-out module matrix (Section 10) is a separate instrument for forked go-out modules.

  5. Open-source implementation: All code (rules engine, AI agents, calibration scripts) and data are publicly available for reproduction and extension.

3 Game Model and Rules Engine

3.1 Core Mexican Train Rules

Mexican Train follows standard multi-trail domino game mechanics:

  1. Setup: All players draw a fixed hand size (typically 15 tiles for 2–4 players, fewer for larger games). The engine double (e.g., 12-12 for Warp 12) is placed in the center. Remaining tiles form the draw pile.

  2. Turn structure: On each turn, a player must play one tile that matches the open end of any available train (their own, another player’s with a marker, or the public Mexican Train). If unable to play, they draw one tile from the pile and may play it immediately if legal.

  3. Doubles: When a double is played, it must be “satisfied” (another tile played on it) before anyone can play elsewhere. This creates a Red Alert state that forces immediate resolution.

  4. Train markers: When a player cannot play on their turn, they place a marker on their train, making it public for one round. In Warp 12 terminology, this is called a Distress Beacon and the state is Shields Down.

  5. Mexican Train: The public train is always available to all players, providing a release valve when private trains are blocked.

  6. Round end: A round ends when a player empties their hand (go-out objective) or when the pile is exhausted and no legal plays remain (points objective).

3.2 Implementation as State Machine

Our implementation models the game as an immutable state machine with pure functions:

This design ensures that AI agents, human players, and replay validation all use identical game logic. No special-case code paths exist for AI vs human play.

3.3 House Rules and Modules

Beyond core rules, Mexican Train has many common variants. Warp 12 implements these as runtime configuration rather than hard-coded variants:

House rules (boolean toggles):

Modules (optional mechanics; Warped = exhibition only, never free-for-all rated):

All modules affect legal move generation and scoring through the same applyAction code path, ensuring AI agents see exactly the same game mechanics as human players. Product classification (Promote vs Warped) is grounded in the Section 9 study.

3.4 Information Structure and Game Complexity

Mexican Train is an imperfect information game:

This differs fundamentally from perfect-information games like chess or Go, where optimal play can theoretically be computed through exhaustive search. In Mexican Train:

  1. Belief states: Agents must reason about probability distributions over possible opponent hands and pile orderings, not a single deterministic state.

  2. Determinization: One common approach samples possible hidden states consistent with observations, then searches forward in each determinized world. However, this can lead to strategy fusion  where different opponent hand distributions should lead to different plays.

  3. Information set search: Methods like Information Set Monte Carlo Tree Search (ISMCTS)  explicitly handle information sets, avoiding some determinization pitfalls.

The combination of hidden information, multi-player dynamics, and stochastic draws makes Mexican Train significantly more complex than its simple rules suggest, and prevents the existence of a "solved" optimal strategy comparable to checkers or Connect Four.

3.5 Two Objectives as Two Different Games

The choice of victory condition fundamentally changes optimal strategy:

Strategic differences between points and go-out objectives
Dimension Points Go-out
Win condition Lowest pips at round end First empty hand
Horizon Multi-round campaign (1–13) Often single-round race
Core skill Pip shedding, blocking, flexibility Tempo, connectivity, mayhem
Variance Lower Higher
Search benefit Modest at 2p; expectimax depth 4 \sim64% Helpful at 2p; ISMCTS \sim31% /4p
AI μ\mu gaps (anchors) 8.5 / 8.5 (IV\toIII\toII) 10.5 / 13.5 (wider to fight compression)

In the points objective, players accumulate penalty points over 13 rounds, rewarding consistent pip minimization and defensive blocking. The multi-round horizon allows recovery from bad draws and rewards strategic flexibility.

In the go-out objective, the first player to empty their hand wins immediately. This creates race dynamics where tempo and hand connectivity matter more than pip values. A player holding three 12-12 doubles might still win if they can chain them quickly.

Figure 1 shows empirically that these objectives lead to measurably different strategic landscapes: go-out compresses implied |Δμ||\Delta\mu| relative to the calibrated anchors, while skill ordering survives.

Points vs go-out strategic divergence under OpenSkill. Left: calibrated |Δμ||\Delta\mu| anchors versus |Δμ||\Delta\mu| implied by 2,000-game win rates — go-out compression is stark. Right: weaker-side win rates show points keeps clearer tier separation than the go-out race.

4 Agent Architecture

Agent architecture overview. Top: Heuristic policy stack (candidate generation through action selection). Middle-left: Class I* residual MLP adds learned correction to heuristics. Middle-right: Class Ω\Omega standalone policy/value heads (no heuristics). Bottom: Fleet Admiral multi-engine routing by player count.

4.1 Policy Stack

Candidate generation from legal moves + special actions. Weighted heuristic scoring + temperature + blunder rate. Optional determinized lookahead: sample hidden hands consistent with counts \to forward simulate in engine.

4.2 Skill Presets (Class IV–II)

Points presets: pip dump, trail pressure, Red Alert safety, Q timing. Go-out presets: sprint heuristics (goOutWin, goOutFeasibility, block leader, avoid mayhem). Separate goOutTuning thresholds per tier.

4.3 Lookahead Policy (Product Decision)

Lookahead baked into tier, not user-toggle — keeps TEI comparable across clients. Class II go-out: depth 2 at 2 players only; greedy at 3+. Class II points: greedy at all sizes — Commander is a local maximum in 2p points (ISMCTS \sim51% at 1,000 games; see §4.6). Monte Carlo Tree Search  with determinization  provides the search foundation for our lookahead implementations.

4.4 Tactical Advisor

Class II profile, blunder rate 0, lookahead on. Explainability: explainWarpAiAction, turn-resolution hints. Unassisted-only TEI — advisor use tracked separately. Move ratings and reasons are generated by the explainable heuristic Commander scorer; the suggested alternative (“advisor would play”) uses the Commander neural policy (Ω\Omega) when available, but falls back to the module-aware heuristic coach on rounds where an active module distorts scoring (e.g. Module Kappa’s inverted rounds, where the module-blind net would wrongly recommend going out).

5 TEI (Tactical Effectiveness Index)

OpenSkill TEI ladder. Left: calibrated AI anchors as μ±σ\mu \pm \sigma by objective, with diamond markers for the conservative display rating μ3σ\mu-3\sigma. Right: presentation layer — letter grade from σ\sigma (confidence) plus a 0–99 score from the display rating.

5.1 Bayesian skill model

TEI is a presentation layer over OpenSkill : each captain stores a Gaussian skill estimate (μ,σ)(\mu,\sigma) per track (points / go-out), updated after ranked match outcomes. Defaults for new humans: μ=25\mu=25, σ=8.33\sigma=8.\!33 (OpenSkill’s μ/3\mu/3 prior). Matchmaking uses an ordinal μσ\mu-\sigma; the UI never exposes raw Elo integers.

FFA means free-for-all: every captain is an individual competitor in the OpenSkill update (solo vs AI or online human-pool tables). Those outcomes write the per-captain FFA ladder stored as humanRating. Squad TEI is the opposite track — Module Zeta crews rate as teams via updateTeamRatings into squadRating and never update FFA humanRating.

5.2 Reference anchors

Rated local and online play pits humans against fixed AI opponents. Each AI class holds a frozen (μ,σ)(\mu,\sigma) that never updates with match outcomes:

Calibrated OpenSkill AI anchors (Phase 1.3; 2,000 games/matchup). Gaps: points 8.5/8.58.5/8.5; go-out 10.5/13.510.5/13.5 (widened against racing compression).
Track Ensign (IV) Lieutenant (III) Commander (II)
2-3(lr)4-5(lr)6-7 μ\mu σ\sigma μ\mu σ\sigma μ\mu σ\sigma
Points 18.0 4.0 26.5 3.5 35.0 3.0
Go-out 17.5 4.5 28.0 4.0 41.5 3.5

v1 vs v2 Class II: product history distinguished heuristic Commander (v1) from neural Class Ω\Omega Commander (v2). Shipping Class II is the Ω\Omega policy at the Commander commission; stored human (μ,σ)(\mu,\sigma) tuples are not re-banded when the Commander implementation changes — only the fixed AI anchor row is.

Calibration rule of thumb: place Commander μ\mu near where a human of comparable TEI score wins \sim50% in the table sizes you actually rate (Warp 12 solo play is mostly 2–4). Fleet-mean fair-share against legacy Commander is a promotion metric, not a direct Δμ\Delta\mu.

5.3 Update rule

After a sector, ranks feed updateFFARatings (free-for-all individual tables), updateTeamRatings (Zeta squads), or updateVsAI (openskill.js Bayesian update). Humans move; AI anchors stay fixed. Separate tracks: objective ×\times (human profile / FFA vs squad). No Elo KK-factor schedule.

5.4 TEI Grade display

Players see a gamified string such as V67:

Federation Academy commission ranks (Cadet\toFleet Admiral) are a flavor map over the same grade+score; they are not a second rating.

5.5 Percentile boards

Go-out |Δμ||\Delta\mu| compresses; percentile (“Top X%”) preserves rank meaning alongside the grade badge.

5.6 Federation Academy

One-time starting grade pick per track within a provisional / improving band.

6 Calibration Methodology

6.1 Self-Play Loop

playSelfPlayGame drives full games through applyAction. Blocked-round stall guard for pile-empty lockups (drafting phases are exempt — mid-draft force-ends previously corrupted Spacedock recycle).

6.2 Evaluation Suites

  1. Head-to-head matrix — all Class IV–II pairs, both objectives (OpenSkill calibration: 2,000 games/matchup).

  2. Symmetric seating — same-skill first-seat win rate \approx 50%.

  3. Focus matchups — one strong captain vs N1N-1 weaker; rotate seat; table sizes 3–8 (go-out).

  4. House-rule sanity — e.g. Drop to Impulse penalty pass (calibrate:ai-tei-dti).

  5. Module integrity — 500 games ×\times Warp ×\times fleet ×\times module (§9).

6.3 Metrics

Completion rate (\geq 85% games decisive). Higher-skill win rate vs ordering thresholds. Implied |Δμ||\Delta\mu| from weaker-side win rate under the OpenSkill logistic. Expected win rate from calibrated anchor gaps (target \sim76% / \sim91% for successive / skip tiers). Skill-indicator count (0–4) for module balance.

7 Results: AI Calibration

See docs/openskill-calibration-log.md for dated OpenSkill self-play runs and calibration-log.md for legacy heuristic benches.

OpenSkill calibration heatmaps (2,000 games per matchup). Left: Points — Ensign win rates 15.7% / 9.2% vs Lieutenant / Commander; Lieutenant vs Commander still the softest rung (36.3%). Right: Go-out — extreme compression (38–44% weaker-side wins) even after widening μ\mu gaps.

7.1 Points (Default Rules, 2,000 games/matchup)

7.2 Go-out

7.3 Class I*, Fleet Admiral, and Class Ω\Omega Benches

AI bench results across experimental tiers. Top-left: Class I* neural residual iterations (imitation ceiling at 48–51%). Top-right: Fleet Admiral search backends (expectimax wins 2p, ISMCTS wins 4p). Bottom-left: Class Ω\Omega fair-share rises with fleet size (points objective). Bottom-right: Fair-share hazard — naive Elo-style translation overestimates shipping anchors vs the tempered OpenSkill Commander μ\mu.

Neural residual (MLP - Class I*):

Fleet Admiral (Search):

Class Ω\Omega (Self-Play \to Class II):

8 Luck vs Skill Across Warp Factors

8.1 Motivation and Configuration Matrix

While Sections 6–7 calibrate AI tiers at a fixed Warp factor (double-12), Warp 12 ships with four playable Warp factors (9 / 12 / 15 / 18), each supporting different fleet sizes:

Warp factor configurations in Warp 12
Warp Factor Max Pip Fleet Limit Tiles Rated?
W9 9 2–4 players 55 Exhibition
W12 12 2–8 players 91 Rated (TEI)
W15 15 2–12 players 136 Exhibition
W18 18 2–18 players 190 Exhibition

Players and designers routinely ask:

  1. Does Warp factor affect skill expression? Higher pip sets increase hand entropy — does this create meaningful strategic depth or just pip-dumping chaos?

  2. Does fleet size (player count) affect skill? Conventional wisdom suggests more players = more noise, but is that true?

  3. Should TEI be calibrated differently for W9 vs W18? If skill expression differs, should we adjust reference bands or restrict rating to W12?

  4. Why is W12 the rated factor? Is it merely tradition (Mexican Train’s historical default), or does the data justify it?

To answer these questions empirically we use the baseline (no-module) cells of the Section 9 campaign — the same Commander points self-play instrument over 38 configurations (Warp factor ×\times fleet size), 500 games per cell. Earlier standalone or single-module pilot collections (including broken Drafting runs) are not treated as evidence; all claims in this section come from that integrated matrix:

Baseline cells within the 274,500-game points matrix (no modules) used for Warp-factor / fleet analysis
Warp Factor Fleet Sizes Configs Total Games
W9 2–4 players 3 1,500
W12 2–8 players 7 3,500
W15 2–12 players 11 5,500
W18 2–18 players 17 8,500
Total 38 19,000 (baseline subset)

All games used the points objective (lowest cumulative pip total) with Class II (heuristic Commander) self-play as part of the July 2026 module campaign (parallel workers on M4 Max). Compact notation: e.g., W12/4p denotes Warp 12 with 4 players.

8.2 Metrics and Composite Indices

Per-turn telemetries are collected by the engine sampler (LuckSkillMetricsSampler) and averaged per game. All §8 tables and figures are computed from those game-level averages after applying the composites below — matching tools/nn/process-luck-skill-data.py, which produced luck-skill-processed.csv.

Raw metrics (game averages):

Instrument design (constructs \to composites). We wanted continuous summaries that rise when a configuration offers more ways for better play to matter, and fall when turns are forced or undifferentiated. The three raw ingredients of skillIndex map to that idea: more legal moves (room to choose), larger value spread (choices are not interchangeable), and higher near-optimal fraction (the policy can still identify better lines). The luckIndex ingredients are the dual: forced attachments, low decision quality (near-optimal rarely available), and sparse legal moves. Weights are fixed heuristics chosen once for interpretability — not fit to TEI outcomes — and were not re-tuned after looking at module rankings:

skillIndex=0.4𝚊𝚟𝚐𝚅𝚊𝚕𝚞𝚎𝚂𝚙𝚛𝚎𝚊𝚍+0.3𝚊𝚟𝚐𝙽𝚎𝚊𝚛𝙾𝚙𝚝𝚒𝚖𝚊𝚕𝙵𝚛𝚊𝚌𝚝𝚒𝚘𝚗+0.3𝚊𝚟𝚐𝙻𝚎𝚐𝚊𝚕𝙼𝚘𝚟𝚎𝚜luckIndex=0.5(1𝚊𝚟𝚐𝙽𝚎𝚊𝚛𝙾𝚙𝚝𝚒𝚖𝚊𝚕𝙵𝚛𝚊𝚌𝚝𝚒𝚘𝚗)+0.3𝚊𝚟𝚐𝙲𝚘𝚗𝚜𝚝𝚛𝚊𝚒𝚗𝚎𝚍𝚃𝚒𝚕𝚎𝙵𝚛𝚊𝚌𝚝𝚒𝚘𝚗+0.2(1/(𝚊𝚟𝚐𝙻𝚎𝚐𝚊𝚕𝙼𝚘𝚟𝚎𝚜+1))decisionQuality=𝚊𝚟𝚐𝙻𝚎𝚐𝚊𝚕𝙼𝚘𝚟𝚎𝚜×𝚊𝚟𝚐𝚅𝚊𝚕𝚞𝚎𝚂𝚙𝚛𝚎𝚊𝚍\begin{align} \text{skillIndex} &= 0.4\,\texttt{avgValueSpread} + 0.3\,\texttt{avgNearOptimalFraction} + 0.3\,\texttt{avgLegalMoves} \\ \text{luckIndex} &= 0.5\,(1 - \texttt{avgNearOptimalFraction}) + 0.3\,\texttt{avgConstrainedTileFraction} + 0.2\,\bigl(1 / (\texttt{avgLegalMoves} + 1)\bigr) \\ \text{decisionQuality} &= \texttt{avgLegalMoves} \times \texttt{avgValueSpread} \end{align}

Reading the composites:

Relation to Section 9. Section 9 keeps the same raw sampler but ranks modules with a different secondary instrument: four absolute pass/fail indicators on cell-mean telemetry (0–4 count). §8 hypotheses use the continuous composites above; Promote / Warped product calls use the §9 indicator count. Both are reported so board-choice and module-taxonomy questions stay separable.

8.3 Hypotheses

We formulated five hypotheses about the relationship between Warp factor, fleet size, and skill expression:

H1 (Warp Factor Effect):

Higher Warp factors exhibit higher skill indices due to increased tile diversity and hand complexity.

H2 (Fleet Size Effect):

Larger fleets correlate with higher skill expression within each Warp factor.

H3 (Interaction):

The relationship between fleet size and skill differs across Warp factors (non-additive effects).

H4 (Complexity-Coherence):

Decision complexity (avgLegalMoves) and hand coherence (avgHandEntropy) exhibit meaningful correlation.

H5 (Monotonic Trends):

Skill index increases monotonically with fleet size within each Warp factor.

8.4 Statistical Results

Cross-factor skill index comparison for 2–4 player configurations. W12 exhibits the highest skill expression in the balanced comparison range.

8.4.1 H1: Warp Factor Effect (ANOVA)

We restricted analysis to the 2–4 player range (balanced design across all factors) to isolate the Warp factor effect. One-way ANOVA on skillIndex by Warp factor:

Mean skill index by Warp factor (2–4 players)
Warp Factor Mean skillIndex SD nn
W9 1.533 0.087 1,500
W12 1.950 0.330 1,500
W15 1.829 0.272 1,500
W18 1.802 0.192 1,500

Result: F(3,5996)=817.14F(3, 5996) = 817.14, p<0.001p < 0.001, η2=0.290\eta^2 = 0.290 (large effect size). H1 is strongly supported.

Post-hoc pairwise comparisons (Tukey HSD) show W12 exhibits the highest skill expression, with W15 and W18 slightly lower but still significantly above W9. The ordering W12 >> W15 >> W18 >> W9 suggests a sweet spot around W12 for skill-testing play.

8.4.2 H2: Fleet Size Effect (Correlation)

Fleet size effect on skill expression by Warp factor. W12 shows the steepest gradient (r=0.875r = 0.875), indicating highest sensitivity to player count.

Within each Warp factor, we computed Pearson correlation between playerCount and skillIndex:

Fleet size effect on skill expression by Warp factor
Warp nn Fleet Sizes Pearson rr pp-value R2R^2
W9 1,500 2–4 0.051 0.049 0.003
W12 3,500 2–8 0.875 <0.001<0.001 0.766
W15 5,500 2–12 0.893 <0.001<0.001 0.798
W18 8,500 2–18 0.840 <0.001<0.001 0.706

Result: H2 is strongly supported for W12/15/18 (all r>0.84r > 0.84, p<0.001p < 0.001). W9 shows a weak but significant positive effect (r=0.051r = 0.051, p=0.049p = 0.049). W12 exhibits the steepest fleet size effect (r=0.875r = 0.875), making it the most sensitive Warp factor to player count.

Unexpected finding: Conventional wisdom predicts more players = more noise = less skill. Our data show the opposite: skill index increases with fleet size. This suggests that larger fleets create more strategic depth (blocking opportunities, trail diversity, timing decisions) rather than diluting skill signal with variance.

8.4.3 H3: Interaction Effect

Decision complexity heatmap (avgLegalMoves) by Warp factor and fleet size. Note the relatively constant branching factor (\sim2 moves/turn) despite large variations in configuration.

To test whether the fleet size effect differs across Warp factors, we compared regression slopes for the 2–4 player range (balanced design):

Interaction effect: slope variance across Warp factors
Warp Factor Slope (skillIndex \sim playerCount)
W9 0.0054
W12 0.3889
W15 0.3172
W18 0.2085

Slope variance = 0.0209. H3 is supported: fleet size matters much more for W12 than for W9. This interaction justifies W12 as the rated factor — it exhibits both high baseline skill and high sensitivity to player count, making it the best discriminator of player ability.

8.4.4 H4: Decision Complexity vs Hand Coherence

Hand entropy (coherence) vs near-optimal play fraction. The weak correlation (r=0.009r = 0.009) suggests that tactical richness and strategic coherence are orthogonal dimensions.

Correlation between avgLegalMoves (decision complexity) and avgHandEntropy (hand coherence):

Result: H4 is not supported. Decision complexity and hand entropy are largely independent. This suggests that tactical richness (number of legal moves) and strategic coherence (hand composition) are orthogonal dimensions of game state.

Kendall’s τ\tau (rank correlation) between playerCount and skillIndex:

Monotonic trend test (Kendall’s τ\tau)
Warp Factor Kendall’s τ\tau pp-value
W9 0.044 0.025
W12 0.727 <0.001<0.001
W15 0.767 <0.001<0.001
W18 0.672 <0.001<0.001

Result: H5 is supported for W12/15/18 (strong positive monotonic trends). However, examining individual fleet sizes reveals non-monotonic patterns at the highest player counts: W18 skill peaks at 16p (skillIndex = 3.002), then drops at 17p (2.757) and 18p (2.834). This suggests ergodic limits: at extreme fleet sizes, some configurations may hit resource constraints (e.g., near-empty draw pile, forced blocking).

8.5 Key Findings

Skill vs luck balance across all configurations. Points above the diagonal (more skill than luck) include all W12/15/18 configurations. W18/18p exhibits the highest skill/luck ratio (8.10).

8.5.1 W18/18p is Not a Crapshoot

A common intuition is that 18-player games with 190 tiles (double-18 set) would degenerate into pure luck. Our data contradicts this:

W18: 2-player vs 18-player comparison
Config skillIndex luckIndex Skill Ratio Luck Ratio
W18/2p 1.631 0.289 1.00×\times 1.00×\times
W18/18p 2.834 0.350 1.74×\times 1.21×\times

Interpretation: At 18 players, skill expression is 1.74×\times higher than heads-up, while luck increases only 1.21×\times. The skill/luck ratio actually improves at extreme fleet sizes. This suggests that large multi-player games create rich tactical opportunities (blocking, trail pressure, timing) that outweigh variance from hidden information.

8.5.2 Cross-Factor Comparison at Maximum Fleet Size

Skill and luck at maximum fleet size per Warp factor
Configuration skillIndex luckIndex Skill/Luck Ratio
W9/4p 1.528 0.334 4.57
W12/8p 2.629 0.405 6.49
W15/12p 2.803 0.379 7.40
W18/18p 2.834 0.350 8.10

W18/18p has the highest skill/luck ratio of any configuration tested. This challenges the intuition that larger games are necessarily more luck-dependent.

8.5.3 W12 Justification for TEI Rating

Why is W12 the sole rated factor? The data provide three empirical justifications:

  1. Highest baseline skill (2–4p): W12 exhibits mean skillIndex = 1.950 in the 2–4p range, higher than W9 (1.533), W15 (1.829), and W18 (1.802).

  2. Steepest fleet size gradient: W12 shows r=0.875r = 0.875 for fleet size effect — the most sensitive to player count of any factor. This means W12 best discriminates between skill levels across table sizes.

  3. Practical rating context: Most solo rated play occurs at 2–4 players (W12 supports 2–8). W12 provides the widest skill range within the most common fleet sizes.

Conclusion: W12 is not merely historical tradition — it is the empirically optimal choice for a skill-testing rating system.

8.5.4 Hand Entropy Increases with Warp Factor

Hand coherence (avgHandEntropy) increases monotonically with Warp factor:

Mean hand entropy by Warp factor (all fleet sizes)
Warp Factor Mean Hand Entropy (bits)
W9 2.09
W12 2.34
W15 2.52
W18 2.66

Higher entropy indicates more diverse hands (less clustering around specific pips). This confirms that larger tile sets create more strategic flexibility but also more decision complexity.

8.5.5 Decision Complexity Stays Constant

Despite large variations in Warp factor and fleet size, avgLegalMoves remains remarkably constant at \sim1.8–2.2 moves per turn across all configurations. This suggests that the branching factor of Mexican Train is inherently bounded by game mechanics (trail constraints, hand composition), not by tile set size or player count.

8.6 Practical Implications

8.6.1 TEI Calibration Strategy

  1. Anchor TEI on W12 only. Exhibition factors (W9/15/18) should not have separate AI (μ,σ)(\mu,\sigma) rows — they share the same Class IV–II opponents with W12 anchors.

  2. Do not soften OpenSkill updates by fleet size. Larger fleets increase skill expression, not variance; Bayesian σ\sigma shrinkage already captures confidence.

  3. Prefer percentile boards alongside grades for go-out. While W12 skill gradients are smooth, the monotonic-but-noisy pattern at W18 (drops at 17p) suggests percentiles stabilize feedback when |Δμ||\Delta\mu| compresses.

8.6.2 Future Exhibition Modes

8.6.3 Go-Out Objective

This study used points only. Go-out exhibits higher variance (Section 7.2), suggesting:

9 Module Balance (Points): A 274,500-Game Competitive-Integrity Study

Section 8 used the baseline slice of this points campaign to show that W12 is the right board for rating. The full points matrix answers the product question: which optional modules preserve that skill signal, and which should be barred from TEI? Go-out module forks are evaluated separately in Section 10; we do not pool SS averages across objectives.

9.1 Design questions

9.2 Method

We collected 500 Commander self-play games on each of 549 configuration cells — Warp factor {9,12,15,18}\{9,12,15,18\} ×\times every legal fleet size ×\times module configs (baseline, Alpha–Mu, Official Warp 12, “all”, and Zeta on eligible fleets only) — for 274,500 games under the points objective, using the same LuckSkillMetricsSampler as Section 8.2. (Earlier drafts that counted Zeta on ineligible fleets as 570 cells are superseded.)

Module ranking instrument (distinct from §8 composites). For each cell we average the raw telemetries, then score four absolute pass/fail indicators (implemented in collect-luck-skill-modules.ts):

  1. Legal moves / turn 3.0\geq 3.0 — enough branching that play is not almost forced.

  2. Constrained-tile fraction >0.5> 0.5 — a majority of hand tiles have a narrow legal attachment (pressure / planning).

  3. Move-value spread 2.0\geq 2.0 — heuristic scores still separate best from worst candidates.

  4. Unique pips in hand 5.0\geq 5.0 — enough pip diversity that connectivity choices remain non-trivial.

The integer skill-indicator count S{0,1,2,3,4}S \in \{0,1,2,3,4\} is the primary ranking key. Cell taxonomy: skill-dominant if S3S \geq 3, mixed if S=2S = 2, luck-dominant if S1S \leq 1. Module averages of SS drive product recommendations (Promote vs Warped), not the continuous skillIndex of Section 8. Thresholds were chosen as coarse absolute guards on the same constructs as §8 (branching, force, discrimination, hand diversity); they were frozen before the Promote/Warped labeling pass and were not optimized to separate Epsilon after the fact.

Zeta requires equal squads: even fleets of at least 4 captains when squadronSize=2. Ineligible Zeta cells from earlier runs are excluded from Zeta-specific claims (17 eligible cells remain). The collector auto-omits Zeta from “all” on odd/small fleets.

9.3 Results: most modules preserve the ceiling

Module skill ranking — points instrument (274,500 Commander self-play games, 500/cell; Zeta on even fleets \geq4 only). Go-out forks are not in this table. Rec: Promote = rated-eligible; Warped/party = Epsilon; Warped = Kappa (score inversion).
Module Rec Skill Legal Constr. Spread Pips Skill/Luck
Iota (Double Down) Promote 3.00 2.24 59% 3.64 8.1 38/0/0
All modules Promote 2.95 2.06 57% 3.37 8.4 36/2/0
Zeta (Squadrons) Promote 2.94 1.86 60% 3.14 7.5 16/1/0
Alpha (Continuum) Promote 2.89 2.09 57% 3.19 7.8 34/4/0
Beta (Salamander) Promote 2.89 2.10 57% 3.19 7.8 34/4/0
Delta (Spool) Promote 2.92 2.09 57% 3.19 7.7 35/3/0
Eta (Temporal Debt) Promote 2.89 2.09 57% 3.19 7.7 34/4/0
Gamma (Sensor Grid) Promote 2.89 2.10 57% 3.21 7.8 34/4/0
Kappa (Inversion) Warped 2.89 2.09 57% 3.19 7.8 34/4/0
Lambda (Wormholes) Promote 2.89 2.09 57% 3.17 7.8 34/4/0
Baseline Promote 2.89 2.09 57% 3.20 7.7 34/4/0
Theta (Longest Trail) Promote 2.89 2.09 57% 3.19 7.7 34/4/0
Mu (Fracture) Promote 2.87 2.12 55% 3.25 7.9 33/5/0
Official Warp 12 Promote 2.82 2.12 54% 3.06 7.5 32/5/1
Epsilon (Drafting) Warped/party 1.08 1.30 36% 2.08 5.0 0/11/27
Points instrument. Module skill ceiling across Warp factors and fleet sizes. Nearly every module sits at or above the baseline Promote line (2.5). Epsilon is the sole collapse (Warped/party). Zeta remains high-skill (Promote; Squad TEI, never FFA).
Points instrument. Mean skill indicators by module and Warp factor. Epsilon is red across the board; the Official preset and single-module variants stay skill-stable, especially on W12+.
Decision-quality profiles for rating-relevant modules. Epsilon trails on every axis; Iota leads on legal moves, constrained tiles, and move-value spread.

Key observations:

9.4 The drafting paradox: Epsilon as a party module

Epsilon collapse versus baseline, Iota, and Official across all Warp factors. Drafting never recovers the skill ceiling that pack-and-pass seems to promise.
Warp 12 fleet curves for rating-relevant modules. Epsilon is the unique trough at every fleet size; Iota and Official track near the skill ceiling.

Intuition says drafting should increase skill: captains choose their hands. Empirically, Epsilon averages 1.08/4 indicators and is luck-dominant in 27/38 cells (0/4 on every W9 fleet). Relative to baseline it shows:

Interpretation: Drafting front-loads luck into the deal and produces more coherent personal hands. Mid-game branching then shrinks: fewer forced tiles, fewer discriminating choices, thinner pip diversity. Outcomes are decided earlier by who drafted the lucky connectors, not by who navigates the Neutral Zone. That makes Epsilon an excellent party / social module — the requisition ritual is the entertainment — and a poor competitive one.

Figure 16 maps the deficit cell-by-cell; Epsilon never recovers baseline skill indicators on any Warp/fleet cell we collected. Figure 17 shows the same squeeze on constrained-tile fraction and unique pips — the two mid-game pressure gauges most diagnostic of Warped play.

Epsilon skill-indicator deficit versus baseline across Warp factors and fleet sizes (negative == worse than pack-and-pass). The collapse is wide, not a single bad fleet.
Hand-pressure bars for rating-relevant modules. Left: constrained-tile fraction. Right: unique pips in hand. Epsilon sits alone at the bottom of both axes.

9.5 Iota and outcome mix

Iota (Double Down) is the dual product win: Figure 18 shows its W12 move-value-spread lift versus baseline (+0.26 to +0.86 depending on fleet). Figure 19 stacks skill-/mixed-/luck-dominant cell counts; Iota is all skill-dominant, while Epsilon is mostly luck. Figure 20 places each cell in (legal moves, move-value spread) space — Epsilon clusters low; Iota/baseline/Zeta occupy the skillful cloud.

Iota move-value-spread lift on Warp 12 (fleet sizes 2–8). Double Down widens discriminating choices without collapsing branching.
Skill / mixed / luck outcome mix by module (fraction of configuration cells). Promote modules are skill-dominated bars; Epsilon is mostly luck.
Per-cell scatter of mean legal moves vs move-value spread for baseline, Iota, Official, Zeta, and Epsilon. Epsilon occupies the lower-left Warped pocket.

9.6 Zeta: skillful team play, Squadron TEI (never FFA)

Zeta is the dual of Epsilon on the skill instrument: 2.94/4 indicators on eligible fleets (16/17 skill-dominant). Shared trails and bridge seating preserve decision pressure. It is not Warped. The product separation is infrastructural, not skill-based: free-for-all (FFA) TEI stays on individual tables; eligible rated Warp 12 Zeta sectors write OpenSkill updates to a dedicated team track (squadRating), gated by SQUADRONS_RATING_CALIBRATED (now true after 2v2 ordering calibration). Crew nights never pollute the FFA human ladder.

9.7 Product recommendations

10 Go-out Module Balance: A Separate 255,500-Game Instrument

Go-out rewrites several modules (Salamander Surge, Hot Potato pass draws, Trail Momentum, Desperation Dig, Hand Exchange). Those forks change mid-game pressure relative to the points campaign, so this section is a separate instrument: same sampler and S{0..4}S\in\{0..4\} indicators, but not an apples-to-apples comparison of module means to Section 9. We never plot forked modules on a shared points/go-out taxonomy bar except where noted below.

10.1 Method (go-out)

511 cells ×\times 500 Commander games = 255,500 games under go-out. Epsilon is omitted (unavailable under go-out). Zeta remains eligible-fleet-only (17 cells). Figure generation uses go-out display names and writes distinct assets (figure21figure28, Table 4).

Module skill ranking — go-out instrument (255,500 Commander self-play games, 500/cell; Epsilon omitted). Labels use go-out forks (Surge, Trail Momentum, Dig, Hand Exchange). Do not compare SS averages to the points table — mechanics differ. Rec: Promote = skill-preserving under go-out; Warped = Kappa (Hand Exchange) by product policy.
Module Rec Skill Legal Constr. Spread Pips Skill/Luck
Iota (Double Down) Promote 3.00 2.31 58% 3.83 8.3 38/0/0
Alpha (Continuum) Promote 2.97 2.19 56% 3.46 8.0 37/1/0
Beta (Salamander Surge) Promote 2.97 2.20 56% 3.47 8.0 37/1/0
Delta (Spool / Hot Potato) Promote 2.97 2.22 57% 3.57 8.0 37/1/0
Eta (Desperation Dig) Promote 2.97 2.20 56% 3.47 8.0 37/1/0
Gamma (Sensor Grid) Promote 2.97 2.20 56% 3.47 8.0 37/1/0
Lambda (Wormholes) Promote 2.97 2.19 57% 3.46 8.0 37/1/0
Baseline Promote 2.97 2.19 56% 3.45 8.0 37/1/0
Theta (Trail Momentum) Promote 2.97 2.20 56% 3.48 8.0 37/1/0
Zeta (Squadrons) Promote 2.94 1.88 60% 3.26 7.6 16/1/0
Kappa (Hand Exchange) Warped 2.89 2.41 54% 3.73 8.8 35/2/1
Mu (Fracture) Promote 2.84 2.22 54% 3.49 8.0 32/6/0
Official Warp 12 Promote 2.84 2.21 54% 3.32 7.7 32/6/0
All modules Promote 2.82 2.41 55% 4.05 9.4 29/9/0
Go-out instrument only. Module skill ceiling under go-out forks. Do not compare bar heights to Figure 11 as a single ranking.
Go-out instrument. Mean skill indicators by module and Warp factor under go-out.

10.2 Results (go-out)

10.3 Deliberate contrast: shared module IDs only

Figure 23 is the only dual-objective module plot. It includes modules whose rules are shared across objectives (baseline, Iota, Official, Zeta, Gamma, Lambda, Mu) and excludes forks (Beta/Delta/Eta/Theta/Kappa). The point is compare/contrast of instruments on comparable rules — not a merged taxonomy.

Compare/contrast only: average skill indicators for shared module IDs under the points vs go-out instruments. Forked modules omitted.

11 Discussion

11.1 What Calibration Teaches

11.2 Design Recommendations

11.3 Limitations

12 Conclusion

TEI and self-play calibration provide a practical, honest skill ladder for a game too messy for classical solving. Points and go-out should be treated as two calibration targets on one engine. Class I* and Fleet Admiral benches show that no single algorithm wins every mode: Commander heuristics are near-optimal in 2p points (ISMCTS  \sim51%), expectimax extracts a \sim64% edge there via explicit tree search, and ISMCTS outperforms greedy seats in 4p go-out (\sim31% vs \sim23%). Class Ω\Omega shows that a pure self-play net can replace heuristic Class II without a fourth lobby tier — but promotion benches and TEI anchors are different jobs: fleet fair-share can look large while heads-up play stays near parity; REF_TEI must follow the tables you rate.

The 274,500-game points module study (549 cells; Section 9) is the authoritative skill/luck instrument for TEI taxonomy. From its baseline Warp×\timesfleet slice:

  1. Skill expression increases with both Warp factor and fleet size. Contrary to conventional wisdom (“more players = more luck”), skill index rises strongly with player count (r=0.84r = 0.840.890.89 for W12/15/18). W18/18p exhibits 1.74×\times the skill of W18/2p, with only 1.21×\times the luck — the highest skill/luck ratio of any configuration tested.

  2. W12 is the empirically optimal rated factor. W12 exhibits the highest baseline skill in the 2–4 player range (where most rated play occurs) and the steepest fleet size gradient (r=0.875r = 0.875), making it the best discriminator of player ability. This justifies restricting TEI rating to W12 while offering W9/15/18 as exhibition modes.

  3. Hand entropy increases with Warp factor, but decision complexity stays constant. Larger tile sets create more diverse hands (W9: 2.09 bits \to W18: 2.66 bits entropy) but do not increase branching factor (mean legal moves \sim1.8–2.2 across all configs). This suggests Mexican Train’s tactical depth scales with tile diversity, not combinatorial explosion.

Across the full module matrix the product constraints are:

  1. Most modules are rating-safe. Baseline, Official Warp 12, and the Alpha–Mu singles (except Epsilon) preserve \sim2.8–3.0 skill indicators.

  2. Epsilon is Warped/party. Drafting collapses skill to 1.08/4 — keep it social, never on TEI.

  3. Zeta is skill-promote with live Squad TEI. Squadrons preserve skill (2.94/4); rated Warp 12 crew sectors update squadRating only.

The right product move is one neural Class II, tempered OpenSkill anchors, unrated search as hard mode, not more named tiers — plus a clear Promote vs Warped module taxonomy grounded in the skill instrument. Superhuman Mexican Train remains a research program (belief-state search + Path A value + human validation), not required for an excellent commercial experience.

Acknowledgments

To Deborah, Hannah, and Don for taking me in to their family and for introducing me to Mexican Train Dominoes.

This research was conducted as part of the open-source Warp 12 project. All code, data, and reproduction scripts are available at https://github.com/digitaldefiance/warp12.

13 Figure List

All figures are generated from empirical data and located in tools/nn/figures/.

  1. Cross-factor skill index comparison (Section 8) — W12 exhibits highest skill expression in 2–4p balanced range.

  2. Fleet size effects by Warp factor (Section 8) — W12 shows steepest gradient (r=0.875r = 0.875), most sensitive to player count.

  3. Decision complexity heatmap (Section 8) — Constant branching factor (\sim2 moves/turn) across all configurations.

  4. Hand entropy vs near-optimal play (Section 8) — Weak correlation (r=0.009r = 0.009) suggests orthogonal dimensions.

  5. Skill vs luck balance (Section 8) — W18/18p has highest skill/luck ratio (8.10) of any configuration.

  6. TEI ladder visualization (Section 5) — OpenSkill μ±σ\mu\pm\sigma AI anchors and TEI grade presentation (E/V/C/I/P).

  7. Calibration matrix heatmap (Section 7) — 2,000-game win rates; points clarity vs go-out compression.

  8. AI bench results comparison (Section 7) — Class I* parity, Fleet Admiral wins (expectimax 64%), Ω\Omega fair-share by fleet size, fair-share hazard.

  9. Architecture diagrams (Section 4) — Policy stack, Class I* MLP, Class Ω\Omega policy/value heads, Fleet Admiral routing.

  10. Points vs go-out divergence (Section 3)|Δμ||\Delta\mu| anchor vs implied gaps; weaker-side win rates.

  11. Module skill ranking (Section 9) — 274.5K-game points Promote vs Warped taxonomy; Epsilon collapse; Zeta skill-promote. Go-out forks in Section 10.

  12. Module ×\times Warp heatmap (Section 9) — Skill indicators by module and Warp factor.

  13. Epsilon collapse panels (Section 9) — Drafting vs baseline/Iota/Official across W9–W18.

  14. Module metric profiles (Section 9) — Legal moves, constrained tiles, spread, unique pips.

  15. W12 module fleet curves (Section 9) — Rating-relevant modules across 2–8 captains.

  16. Epsilon deficit heatmap (Section 9) — Cell-level skill-indicator deficit vs baseline.

  17. Iota spread lift on W12 (Section 9) — Move-value-spread gain from Double Down.

  18. Module outcome mix (Section 9) — Skill / mixed / luck cell fractions by module.

  19. Legal moves vs spread scatter (Section 9) — Per-cell depth cloud; Epsilon Warped pocket.

  20. Hand-pressure bars (Section 9) — Constrained tiles and unique pips for key modules.

14 Tables

Summary Statistics by Configuration (500 games per cell)
Warp Fleet N Skill Index Luck Index Decision Quality
Factor Size μ±σ\mu \pm \sigma μ±σ\mu \pm \sigma μ±σ\mu \pm \sigma
W9 2 500 1.517±\pm0.082 0.332±\pm0.011 3.309±\pm0.428
W9 3 500 1.553±\pm0.090 0.339±\pm0.012 3.494±\pm0.483
W9 4 500 1.528±\pm0.085 0.334±\pm0.011 3.364±\pm0.443
W12 2 500 1.564±\pm0.068 0.314±\pm0.006 3.392±\pm0.337
W12 3 500 1.944±\pm0.094 0.352±\pm0.010 5.525±\pm0.580
W12 4 500 2.341±\pm0.099 0.389±\pm0.009 8.264±\pm0.729
W12 5 500 2.299±\pm0.099 0.384±\pm0.009 7.914±\pm0.717
W12 6 500 2.550±\pm0.094 0.406±\pm0.008 9.873±\pm0.777
W12 7 500 2.455±\pm0.099 0.393±\pm0.007 9.128±\pm0.782
W12 8 500 2.629±\pm0.107 0.405±\pm0.007 10.562±\pm0.912
W15 2 500 1.532±\pm0.055 0.293±\pm0.003 3.114±\pm0.250
W15 3 500 1.788±\pm0.078 0.318±\pm0.007 4.359±\pm0.405
W15 4 500 2.167±\pm0.098 0.350±\pm0.009 6.556±\pm0.609
W15 5 500 2.239±\pm0.105 0.356±\pm0.009 7.008±\pm0.672
W15 6 500 2.518±\pm0.107 0.375±\pm0.009 8.954±\pm0.775
W15 7 500 2.460±\pm0.103 0.369±\pm0.009 8.554±\pm0.718
W15 8 500 2.649±\pm0.102 0.380±\pm0.008 9.996±\pm0.796
W15 9 500 2.639±\pm0.104 0.376±\pm0.007 9.952±\pm0.796
W15 10 500 2.774±\pm0.107 0.383±\pm0.007 11.056±\pm0.885
W15 11 500 2.684±\pm0.115 0.373±\pm0.007 10.394±\pm0.922
W15 12 500 2.803±\pm0.112 0.379±\pm0.007 11.404±\pm0.965
W18 2 500 1.631±\pm0.051 0.289±\pm0.002 3.458±\pm0.233
W18 3 500 1.725±\pm0.061 0.299±\pm0.003 3.873±\pm0.288
W18 4 500 2.048±\pm0.090 0.323±\pm0.007 5.533±\pm0.489
W18 5 500 2.165±\pm0.093 0.332±\pm0.008 6.197±\pm0.530
W18 6 500 2.475±\pm0.105 0.352±\pm0.008 8.136±\pm0.677
W18 7 500 2.457±\pm0.111 0.350±\pm0.008 8.054±\pm0.712
W18 8 500 2.670±\pm0.111 0.362±\pm0.008 9.541±\pm0.780
W18 9 500 2.648±\pm0.116 0.358±\pm0.007 9.450±\pm0.816
W18 10 500 2.809±\pm0.113 0.366±\pm0.007 10.654±\pm0.848
W18 11 500 2.731±\pm0.111 0.358±\pm0.007 10.143±\pm0.818
W18 12 500 2.844±\pm0.109 0.364±\pm0.006 11.018±\pm0.855
W18 13 500 2.975±\pm0.112 0.369±\pm0.006 12.095±\pm0.909
W18 14 500 2.806±\pm0.110 0.356±\pm0.006 10.846±\pm0.855
W18 15 500 2.905±\pm0.112 0.360±\pm0.006 11.653±\pm0.915
W18 16 500 3.002±\pm0.121 0.364±\pm0.006 12.456±\pm1.025
W18 17 500 2.757±\pm0.117 0.347±\pm0.006 10.635±\pm0.913
W18 18 500 2.834±\pm0.118 0.350±\pm0.006 11.257±\pm0.955
Statistical Test Results for Primary Hypotheses
Hypothesis Test Statistic Value pp-value Result
H1: Warp factor effect One-way ANOVA FF 0.00 <0.001<0.001 Supported
η2\eta^2 0.290 (large effect)
H2: Fleet size effect Pearson rr (W12) rr 0.000 <0.001<0.001 Supported
Pearson rr (W15) rr 0.000 <0.001<0.001 (strong positive)
Pearson rr (W18) rr 0.000 <0.001<0.001
H3: Interaction Slope variance s2s^2 0.0209 Present
H4: Complexity-coherence Pearson rr rr 0.009 >0.05>0.05 Weak
H5: Monotonic trends Kendall’s τ\tau τ\tau 0.8–0.9 <0.001<0.001 Supported
Correlation Matrix for Key Game Metrics (N=19,000N=19{,}000)
Skill Index 0.70 0.99 0.38 0.87 -0.76 0.52 0.84
Luck Index 0.70 0.73 -0.08 0.88 -0.97 -0.16 0.40
Decision Quality 0.99 0.73 0.32 0.92 -0.81 0.43 0.84
Hand Entropy 0.38 -0.08 0.32 0.01 0.06 0.82 0.21
Legal Moves 0.87 0.88 0.92 0.01 -0.95 0.05 0.71
Near-Optimal % -0.76 -0.97 -0.81 0.06 -0.95 0.12 -0.51
Warp Factor 0.52 -0.16 0.43 0.82 0.05 0.12 0.53
Player Count 0.84 0.40 0.84 0.21 0.71 -0.51 0.53

Note: Bold values indicate strong correlation (|r|>0.5|r| > 0.5); gray values indicate negligible correlation (|r|<0.1|r| < 0.1).

15 Code Map

Concern Location
Skill presets libs/engine/src/lib/ai/skill.ts
Heuristics libs/engine/src/lib/ai/heuristics.ts
Self-play libs/engine/src/lib/ai/self-play.ts
Calibration libs/engine/src/lib/ai/ai-elo-calibration.ts
Optimizer libs/engine/src/lib/ai/ai-weight-optimizer.ts
Fleet Admiral / ISMCTS libs/engine/src/lib/ai/fleet-admiral.ts, ismcts.ts
Expectimax preset resolveFleetAdmiralExpectimaxLookahead() in fleet-admiral.ts
Parallel bench libs/engine/src/lib/ai/bench-fleet-admiral-parallel.ts
Class I* policy libs/engine/src/lib/ai/class1-star-policy.ts
Class I* features libs/engine/src/lib/ai/feature-encoder.ts
Class I* training tools/nn/ (collect, train.py, bench)
Class Ω\Omega agent / search libs/engine/src/lib/ai/omega-agent.ts, omega-search-agent.ts
Ω\Omega collect / bench collect-omega-trajectories.ts, bench-omega.ts
Human TEI update apps/Warp12/src/firebase/stats-service.ts, libs/engine/.../rating/
OpenSkill anchors / grades anchors.ts, tei-grade.ts, update-{ffa,team,vs-ai}.ts
Rules profile / AI tiers warp12-official-v1 / v2 in rules-profile.ts
Luck/skill metrics libs/engine/src/lib/ai/luck-skill-metrics.ts
Luck/skill collection tools/nn/collect-luck-skill-single-config.ts
Module analysis tools/nn/collect-luck-skill-modules.ts, analyze-module-results.ts
Statistical analysis tools/nn/process-luck-skill-data.py, test-hypotheses.py
Figure generation create-figures.py, create-paper-figures.py, create-module-figures.py
Rules spec RULES.md
Module report docs/MODULE-ANALYSIS.md
OpenSkill calib log docs/openskill-calibration-log.md

16 Reproducibility

16.1 Self-Play Calibration

yarn calibrate:ai-tei
AI_CALIBRATION_GAMES=500 yarn calibrate:ai-tei
yarn calibrate:ai-tei-dti
yarn optimize:ai-weights
# OpenSkill AI-anchor calibration (engine unit suite)
yarn test:engine -- openskill-calibration

16.2 Fleet Admiral Benchmarks

yarn fleet-admiral:bench:500
FLEET_BENCH_SEAT=b yarn fleet-admiral:bench:500
yarn fleet-admiral:bench:go-out-4p:500
yarn jiti tools/nn/compare-fleet-search.ts

16.3 Class I* Training

yarn class1-star:pipeline:deep
yarn class1-star:pipeline:go-out
yarn class1-star:pipeline:deepblue

16.4 Skill/Luck and Module Matrices (points + go-out, separate)

# Points instrument (549 cells × 500) — TEI taxonomy
WARP12_ANALYSIS_DATA_DIR=tools/nn/data/points-modules-rerun \
  MODULE_OBJECTIVE=points MODULE_GAMES=500 MODULE_WORKERS=14 \
  bash tools/nn/run-module-analysis-parallel.sh
WARP12_ANALYSIS_DATA_DIR=tools/nn/data/points-modules-rerun \
  MODULE_FIGURE_OBJECTIVE=points python3 tools/nn/create-module-figures.py

# Go-out instrument (511 cells × 500) — forked modules; separate figures
WARP12_ANALYSIS_DATA_DIR=tools/nn/data/go-out-modules \
  MODULE_OBJECTIVE=go-out MODULE_GAMES=500 MODULE_WORKERS=14 \
  bash tools/nn/run-module-analysis-parallel.sh
WARP12_ANALYSIS_DATA_DIR=tools/nn/data/go-out-modules \
  MODULE_FIGURE_OBJECTIVE=go-out python3 tools/nn/create-module-figures.py

# Optional shared-ID contrast only (excludes forks)
MODULE_FIGURE_OBJECTIVE=contrast python3 tools/nn/create-module-figures.py

MPLBACKEND=Agg python3 tools/nn/create-paper-figures.py
MPLBACKEND=Agg python3 tools/nn/create-figures.py
MPLBACKEND=Agg python3 tools/nn/create-tables.py

All data and scripts are available in the warp12-engine package and tools/nn/ directory.

17 Target Venues

Venue Fit
AIIDE Best fit — game AI + evaluation
IEEE CoG Strong — agents + competition
FDG Game design + dual objective angle
CHI PLAY Advisor / TEI integrity angle
arXiv cs.AI White paper / preprint

99

Browne, C.B., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., and Colton, S. (2012). A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games, 4(1):1–43.

Silver, D., Schrittwieser, J., Simonyan, K., et al. (2017). Mastering the game of Go without human knowledge. Nature, 550(7676):354–359.

Elo, A.E. (1978). The Rating of Chessplayers, Past and Present. Arco Publishing, New York.

Cowling, P.I., Powley, E.J., and Whitehouse, D. (2012). Information Set Monte Carlo Tree Search. IEEE Transactions on Computational Intelligence and AI in Games, 4(2):120–143.

Frank, I., and Basin, D. (1996). Search in Games with Incomplete Information: A Case Study Using Bridge Card Play. Artificial Intelligence, 100(1-2):87–123.

Glickman, M.E. (1999). Parameter Estimation in Large Dynamic Paired Comparison Experiments. Applied Statistics, 48(3):377–394.

Herbrich, R., Minka, T., and Graepel, T. (2006). TrueSkill™: A Bayesian Skill Rating System. Advances in Neural Information Processing Systems, 19:569–576.

Weng, R.C., and Lin, C.-J. (2011). A Bayesian Approximation Method for Online Ranking. Journal of Machine Learning Research, 12:267–300.

Silver, D., Hubert, T., Schrittwieser, J., et al. (2018). A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play. Science, 362(6419):1140–1144.

Bowling, M., Burch, N., Johanson, M., and Tammelin, O. (2015). Heads-up Limit Hold’em Poker is Solved. Science, 347(6218):145–149.

Digital Defiance (2024). DoubleEighteen: Open-Source Domino Tile Rendering Library. https://github.com/digitaldefiance/double-eighteen

Tesauro, G. (1995). Temporal Difference Learning and TD-Gammon. Communications of the ACM, 38(3):58–68.


  1. This is a deliberately coarse instrument: it uses pip totals rather than the full Commander scorer, and counts candidates near the best pip value rather than judging the seated policy’s chosen move. It is used only as a fixed, interpretable ingredient in the §8 composites and is not tuned against TEI. A higher value therefore reflects a construct (“good lines remain distinguishable”) that the proxy only approximates.↩︎