Hacking the Big Apple: Running the Teleport NYC CTF with Nemesis

11 minutes ago

6 min read

We built the thing the attendees had to break

On August 26-27, Teleport ran a two-day game-day in New York, and Persistent Security Industries was there as a partner to run its capture-the-flag. Three of us made the trip: our founder Markus, our colleague Babette, and me. Our job was not to play, it was to build the board. We built the environment the attendees spent two days attacking, the chain of "proofs" they had to recover, and the automation that kept it all repeatable.

We built and ran it with Nemesis, our own adversary-emulation platform. This post describes that game-day from the side of the room most people never see. That is the side that seeds the clues, resets the maze, and tunes the difficulty while eight teams race each other through it.

A detective story with real stakes

The whole event ran on a detective theme. Flags came back as SLEUTH{...}. The eight competing teams took their names from the trade: Dragnet, Stakeout, Wiretap, Alibi, Caper, Gumshoe. Our red team ran as the Syndicate, and the facilitators kept order as the Precinct. Even the challenges took their names from Holmes stories: The Game Is Afoot to warm up, The Six Napoleons for a command-line-only forensic hunt, The Empty House for a "who broke prod?" timeline. The theme did real work. It framed every scenario as an investigation, which is the skill a defender needs: reconstruct what happened from the evidence, not only notice that something happened.

The two days got harder on purpose. Day 1 laid the foundations: the early footholds, the hardening work, a first forensic wrap-up. Day 2 went advanced: insider misuse, an SSO pivot that correlated one identity across Okta, GitHub, AWS, and the database, and a timed capstone to close.

Why a game-day is the right way to show a security product

Most security tooling is sold on slides. Teleport sells infrastructure access, the layer that decides who gets to touch which server, cluster, and database, and that records what they did when they got there. A game-day is a more honest demo than any slide. You put real operators in front of a real cluster and let them try to get away with something. Either the product sees it or it doesn't.

That is the setup Teleport chose in New York. Eight teams, each in its own sandbox, worked through a ladder of scenarios: a stolen credential, a role quietly escalated, SSH access that tried to slip around the audit log, a multi-party session used to move data out, just-in-time access abused across team boundaries, a Kubernetes namespace breakout, an insider walking off with data, and a full kill-chain capstone to finish. For each one we watched two questions: can the attendees pull it off, and does the identity-aware access layer leave a trail that a defender can follow back. Teams kept finding that their cleverest moves were sitting right there in the audit record, which makes the case for the product better than anything we could have written for them.

The board we built

The CTF was a ladder of scenarios that climbed from a single stolen credential to a full kill chain. Each one dropped the attendees somewhere plausible and asked them to recover a proof: a secret, a token, or a file. They could only reach it by chaining realistic tradecraft. Every scenario copied a real attacker technique instead of a puzzle invented for the room, so the detections meant something. The table below shows a sample, not the full roster.

Scenario	What the attacker attempts	What it put on display
Stolen credential	Log in as a real user and hunt for keys left on a host	How quickly a leaked credential becomes lateral movement
Role-hopping	Request and escalate a role, then reach into Kubernetes	Privilege escalation across access boundaries
Shadow SSH access	Reach a host by paths that try to dodge the audit log	That the "quiet" path still gets recorded
Multi-party session exfil	Abuse a moderated session to move data out	Controls on shared and approved sessions
Just-in-time access	Turn a cross-team JIT grant into access it shouldn't give	Where approval workflows can be gamed
Kubernetes breakout	Escape a namespace and pivot through the cluster	Container and workload isolation under pressure
Insider threat	Walk off with data as a trusted, authenticated user	Detection when nothing is technically "broken"
SSO token pivot	Steal an SSO token and pivot across Okta, GitHub, AWS, and the database	How one identity ties every connected system together
Capstone kill chain	Stitch the earlier footholds into one end-to-end attack	A full campaign reconstructed from the audit trail

Alongside these we ran hardening tracks and pure-forensic challenges, for example a command-line-only investigation (The Six Napoleons) and a "who broke prod?" timeline (The Empty House). There the teams played defender instead of attacker: they fixed the very weaknesses the other scenarios exploited, or reconstructed an incident from nothing but the evidence left behind. So the two days moved from breaking in to shutting the door, and from "it happened" to "here is how."

Behind the scenes with Nemesis

What an event slide never shows is the maintenance. A CTF environment is not built once, it is built, broken, and rebuilt constantly. Attendees find a proof, so you reset it. A scenario turns out too easy, so you bury the clue deeper; too hard, so you leave a thread hanging. Eight teams run in parallel, each needing its own clean copy of the same world. Doing that by hand over two days is how game-days fall apart.

Nemesis held it together. We used it in three ways:

Seeding the proofs. The secrets and artifacts the attendees were meant to find didn't get placed by hand. We described each one as an adversary-emulation step, mapped to the real technique it imitated, and let Nemesis plant it. Every team's environment then carried the same proof in the same realistic spot.
Repeatable, automatic resets. Because the whole setup was expressed as Nemesis scenarios, we could tear an environment down and stand it back up on demand, identically, as many times as we wanted. Reset between rounds stopped being a scramble and became one command.
Tuning difficulty live. The attendees were good, better than we had sized for, and very quick to spot traces. Because the difficulty lived in the automation, we could adjust it between rounds: make a scenario harder by cleaning up the breadcrumbs the sharper teams were following, or easier by leaving a little more signal, without rebuilding anything by hand.

There was a fourth use we kept to ourselves. Before and during the event, we ran Nemesis against our own board, undercover, and pointed its AI operator at the challenges to solve them the way the attendees would. It worked the scenarios end to end, which gave us an honest read on where each one really sat on the difficulty curve and let us re-tune before a human ever hit it. The best way to know whether a maze is solvable is to watch something solve it.

I want to stress that last point. The attendees were so good at finding traces that a static environment would have been solved and stale by lunch. Nemesis kept the board alive, so we could re-tune it while the event ran. That is what keeps a CTF a genuine contest instead of a demo.

What we took away

Three things stuck with us after the floor cleared:

Good operators go straight for the traces. The fastest teams were not the ones who knew the most exploits. They were the ones who read the environment like a crime scene. That reshaped how we seed and clean up proofs, and we have fed that back into Nemesis.
Repeatability is the whole game. Expressing an environment as code is not about elegance. It means you can run it again at 2pm exactly as it ran at 10am, for the eighth team as for the first. Every manual step is a step that drifts.
Emulation and detection belong in the same room. Building the attack and watching Teleport catch it, side by side, taught more in two days than either half would alone. So it is worth running offense and defense against each other on a shared, instrumented platform.

None of this works without a cluster worth attacking. Teleport gave the attendees real infrastructure access to subvert and a real audit trail to outrun, and the trail won more often than they'd have liked.

Thanks, and an invitation

Thank you to Teleport for having us in New York and for running the kind of event that puts its own product on the line in front of a room full of people trying to break it. That takes confidence, and it earned ours. Markus, Babette, and I came home with a notebook full of ideas and a lot of respect for the teams who played.

If you run security exercises such as game-days, CTFs, purple-team drills, or detection validation, Nemesis is built for the problem we hit in New York: it makes a realistic adversary repeatable, resettable, and tunable, so your environment stays a live contest instead of a static demo. We would be glad to show you what it can do. Get in touch with Persistent Security Industries if you want a board worth breaking.