Environments & CI/CD

The environments

Three environments carry the code, left to right:

  • Dev workstation — a single machine
  • SIT — the company network
  • Production — the open internet

Code moves left to right between them — built on a workstation, tested on the company network, released to the open internet.

Underneath all three sits one more piece of infrastructure that isn't a stage in that pipeline: the Spark, the single AI model server in the whole system. There's only one of it, not a copy per environment — each environment writes work into its own queue, and that same single Spark reaches out to check each one and do the work, rather than each running its own model server.

Here's the part that reads as confusing until you've seen it: SIT is the Spark. For dev and production, the Spark reaches their queue over a network hop; for SIT, there's no hop at all — SIT's own deploy checkout lives directly on the Spark's disk, at the deploy path, and its containers run on that same box. So "tested on the company network" and "the machine that serves the model to every environment" are, today, one and the same physical machine wearing two hats.

Each environment has a different exposure and a different worst case. The design question isn't "is each one secure" — it's what does compromising one get you access to.

The Spark is dedicated GPU hardware — not a rented cloud instance or a simulated backend. It runs a real, locally-hosted model (gpt-oss:20b, served by Ollama) side by side with the hosted Claude API on the exact same task. See AI: Foundation vs. Open-Weight for that comparison in full.

Dev

a single workstation

Holds
Real credentials and keys
Worst case
The keys to everything else

SIT

company network only

Holds
Realistic test data
Worst case
Contained — the cheap place to be wrong

Production

public cloud · open internet

Holds
Real user data
Worst case
The most likely thing to be attacked

The Spark

private network · shared machine

Holds
The AI models, and work shared with other teams
Worst case
Other teams' data, not just ours

Shared by all three · the Spark always dials out to each one's queue, never the reverse

Dev

flowchart LR
    subgraph DEVBOX["Laptop"]
        direction LR
        APP["Tourney App"]
        QUEUE["Job queue
(a table in this same database)"]
    end
    CLAUDE["Claude
(vendor API)"]
    SPARK["The Spark"]

    APP -->|"vendor API"| CLAUDE
    APP --- QUEUE
    SPARK -.->|"polls, claims & returns results
same private network"| QUEUE

    style DEVBOX fill:transparent,stroke:#64748b,stroke-width:1px
    style QUEUE fill:transparent,stroke:#64748b,stroke-width:2px
    style SPARK fill:transparent,stroke:#b8953a,stroke-width:2px
    linkStyle 1 stroke:#64748b,stroke-width:1px

SIT

flowchart LR
    subgraph SITBOX["SIT server"]
        direction LR
        APP["Tourney App"]
        QUEUE["Job queue
(a table in this same database)"]
        SPARK["The Spark
same physical machine"]
    end
    CLAUDE["Claude
(vendor API)"]

    APP -->|"vendor API"| CLAUDE
    APP --- QUEUE
    SPARK -.->|"polls & claims work"| QUEUE

    style SITBOX fill:transparent,stroke:#64748b,stroke-width:1px
    style QUEUE fill:transparent,stroke:#64748b,stroke-width:2px
    style SPARK fill:transparent,stroke:#b8953a,stroke-width:2px
    linkStyle 1 stroke:#64748b,stroke-width:1px

Production

flowchart LR
    subgraph PRODBOX["AWS"]
        direction LR
        APP["Tourney App"]
        QUEUE["Job queue
(a table in this same database)"]
    end
    CLAUDE["Claude
(vendor API)"]
    SPARK["The Spark"]

    APP -->|"vendor API"| CLAUDE
    APP --- QUEUE
    SPARK -.->|"polls, claims & returns results
outbound only, mTLS"| QUEUE
    APP -.-x|"must not exist"| SPARK

    style PRODBOX fill:transparent,stroke:#64748b,stroke-width:1px
    style QUEUE fill:transparent,stroke:#64748b,stroke-width:2px
    style SPARK fill:transparent,stroke:#b8953a,stroke-width:2px
    linkStyle 1 stroke:#64748b,stroke-width:1px
    linkStyle 3 stroke:#9b4b3f,stroke-width:2px,color:#9b4b3f

The mechanism is identical in all three: the app writes a job to its own queue, and the Spark polls, claims it, and returns results. Nothing about that changes environment to environment — only the wire underneath does, and the app's code never touches it.

The invariant

Nothing on the internet ever opens a connection toward the Spark. The Spark reaches out, asks whether there's work, does it, and reports back. It never listens for anyone.

This is why compromising production doesn't get you the Spark — there's no door to walk through, only a phone that dials out. Reversing that arrow for any reason, however sensible it sounds, turns an internet break-in into access to a machine shared with other teams' work.

Two more rules that hold this up

The Spark runs nothing on its own behalf. The AI can ask for an action; the action is carried out elsewhere and only the answer comes back. The model on the private machine has no way to run a command on it.

The AI is never shown certain personal data. A participant's full_name, alongside their bracket_name, may go to a model — someone who enters a bracket pool expects to appear on a scoreboard. Their email and cell may never go to any model, in any environment. See AI: Foundation vs. Open-Weight for exactly what a handler can and cannot select.

One more gate worth naming here: agentic tool-calling itself defaults to off in every environment, so a routine release can't turn it on by accident — AI: Foundation vs. Open-Weight owns the detailed treatment.

What moves the code between them

Three refs, three environments: main, sit, prod. main moves on every push — whoever pushes decides. sit moves automatically, the instant a push to main passes all five automated gates; nobody chooses that one, the gates do. prod only moves when the Principal clicks Approve on the required-reviewer gate — the one ref that needs a human signature. See Deploy for the full path from a reviewed commit to a released one.