Code Challenge
Try the demo

Take-homes and live coding interviews

Hire on real engineering signal, not interview performance.

Your own repositories become challenges that run inside the candidate's VS Code — their machine, a real toolchain, and an AI agent whose every turn you read afterwards. You find out how someone builds instead of how well they perform in a stranger's editor.

Take the demo challenge See how it works

Fifteen minutes in your own editor, from the candidate's side. No call to sit through.

checkout-service — discounts.py

1from decimal import Decimal
2
3from checkout.errors import UnknownDiscount
4
5
6def apply(cart, code):
7 """The cart total with `code` applied."""
8 rule = RULES.get(code.upper())
9 if rule is None:
10 raise UnknownDiscount(code)
11
12 if cart.subtotal < rule.minimum:
13 return cart.subtotal
14
15 return cart.subtotal - rule.amount
Challenge started pytest — 12 passed, 2 failed 41:52 left
Their editor, not a cloud IDE
Their keybindings, their extensions, their machine.
Ready before the session starts
Dependencies installed and the toolchain checked in advance.
Questions that don't leak
The task stays on the server until the challenge starts.
No bill per candidate
Nothing to provision. It runs on the laptop they already own.

A browser sandbox measures how well someone uses a browser sandbox.

It is the one skill nobody needs the day after they are hired. The usual alternative — email them a repository the night before — fails differently and just as often. Four things go wrong, and every one of them is the format rather than the question you asked.

You test an unfamiliar toolbox

No keybindings, no shell they know, no extensions, none of the muscle memory that makes them fast. Strong engineers look slow, and the ones who look fast are the ones who took the test last week.

A repository in an email is a coin toss

They turn up with the wrong runtime, a half-built virtual environment, or a lockfile that will not resolve, and the first fifteen minutes go on debugging their laptop. You find out on the day, in front of them.

Setup eats the session

The clock starts, then the install runs. Four minutes of a forty-five minute slot spent watching a progress bar, and everyone pretending not to notice.

AI is banned, or invisible

Ban it and you are policing a rule you cannot check, for a job that no longer works that way. Allow it and you learn nothing about how they used it.

How it works

Two sides, and they only meet at the green light. Your team prepares a challenge once and reuses it; the candidate never sees any of that.

Your team

  1. 1

    Import a repository

    One you already have. The one your team works in every day is the best challenge you own.

  2. 2

    Confirm what was detected

    The toolchains, the install commands, the checks that prove a machine is ready. Detection proposes; a person confirms. No Dockerfile to write.

  3. 3

    Choose how it starts

    Start a live interview when everyone is ready, or let the candidate start a take-home after setup. Only then do the task files reach their disk.

  4. 4

    Read the work and the transcript

    What they wrote, and every turn they took with the agent to get there.

The candidate

  1. 1

    Enter a code in VS Code

    The extension, a twelve-character code, and a folder to work in. No account to create.

  2. 2

    Set up before anything is timed

    Hours or days ahead, on their own time. Dependencies install and the toolchain is checked.

  3. 3

    Green light, and the work arrives

    The challenge appears in a project that already runs, and the agent panel unlocks with it.

  4. 4

    Submit

    In an hour or in three days. A take-home is the same challenge with a longer clock.

What a real development environment tells you

Their editor, their machine

The candidate opens a folder on their own machine and works the way they work. Real language servers, real debugger, real tests they can run as often as they like. You are watching an engineer, not a person learning your test harness.

The same environment for every candidate

You pick the toolchains a challenge needs and we build the container from them, so every candidate works in the same pinned environment and "it runs on my machine" stops being a variable in your decision. It builds and installs when they redeem their code, days before you need it, so nobody is ever timed while a package manager thinks.

Questions that don't leak

The task files stay on the server until the challenge starts, so nobody can read ahead and nobody can pass them on. One challenge survives a whole hiring round instead of being retired after the third candidate posts it somewhere.

Live sessions and take-homes, one product

A session ends when the candidate submits or when you end it, so a take-home can span days and still be there when they come back to it. The same challenge runs both ways; only the clock changes.

The agent

AI is not the problem. Not seeing it is.

Your candidates use an agent at work and they will use one here. So the agent is part of the challenge: it sits in the panel beside the code and unlocks with the task.

It reaches our proxy rather than a key of their own, which is what makes the transcript yours. Not a screen recording somebody has to sit through — the actual turns, in order, with the tools it ran. You see the question they asked, what they did with the answer, and the moment they worked out it was wrong.

Every turn kept

Recorded on the server, not on the candidate's machine, so it is a record rather than a claim.

Revocable per session

Access is granted at the green light and dies with the session. No provider key ever lands on their laptop.

Bounded cost

A budget per challenge. An exhausted one degrades the agent and leaves the challenge running.

Nothing to configure

They install the extension, and that is the whole of it — no account to create, no API key to paste, no model to choose.

You are not renting a computer per candidate

A cloud IDE bills you twice: once for the machine the candidate types on, and again for the tokens their assistant burns. Neither is a line item here.

Nothing to provision per candidate

The challenge runs on the laptop the candidate already owns, so there is no sandbox to spin up, no container billed by the hour and no idle session quietly costing you money across a three-day take-home. A take-home that runs for three days costs no more than a forty-five minute call, because nothing is running but their laptop.

You pick the model, and the ceiling

We run the inference, and you choose what it runs on: a frontier model where the judgement matters, an open-weight one like DeepSeek V4 where it does not — which is the difference between dollars and cents a session. Each challenge carries a spend limit, so what a candidate costs is a number you set rather than one you find out afterwards.

No trust required

Everything below is enforced by the server, or checked on the candidate's machine before a single file lands. None of it depends on anyone remembering a rule.

  • The start gate is server-side. Task content is unreachable before the challenge starts. Not hidden by the client — absent from it.

  • Git history never ships. Nor environment files. A solution branch and the commit that deleted the answer are a challenge's worst leak.

  • Nothing you upload reconfigures their machine. Editor, container and agent configuration are stripped on the way in, whoever authored the challenge.

  • Every file is verified before it is unpacked. Bundles carry a digest the client recomputes, and it refuses absolute paths, symlinks and anything reaching outside the folder.

  • Scoped, expiring, revocable tokens. Anything on a candidate's laptop is readable by that candidate, so nothing durable is ever put there.

  • Nothing sensitive reaches a log. Not a session token, not an inference credential, not a line of candidate code. Opaque identifiers only, in the logs and in the error reports alike.

Questions we get asked

The ones that come up on every first call.

What does the candidate have to install?

VS Code, which most of them already have, our extension, and Docker with the Dev Containers extension — the challenge only ever runs in a dev container, so everyone gets the same environment and setup cannot touch their own machine. Then they paste the code you sent and pick a folder. There is no account to create and nothing to sign up for.

Can we use our own repository as the challenge?

That is the intended way to use it. Import the repo, confirm the toolchain we detected, and mark which files are the task. Candidates work in the codebase they would actually be joining, which tells you more than any puzzle does.

Won't candidates just let the agent do it?

Some will, and that is a finding rather than a failure of the format. You read the transcript beside the diff, so the difference between someone who directed the agent and someone who pasted whatever it said is visible in a way it is not in a call.

Live coding or take-home?

Both, from the same challenge. A session ends when the candidate submits or when you end it, so a take-home simply runs on a longer clock and survives the candidate closing their laptop.

What does a session actually cost to run?

There is no compute to pay for: the challenge runs on the candidate's own machine, so a three-day take-home costs the same as a forty-five minute session. The only variable is inference, and you choose the model — an open-weight one is a few cents a session.

What happens to the candidate's code and transcript?

Both are personal data belonging to a job applicant, and they are handled that way. The transcript and the code they submitted are kept as the record of the challenge, reachable by your team and nobody else's, and nothing sensitive is ever written to a log.

Who can create accounts?

An administrator on your team adds colleagues. There is no self-signup, and candidates never get an account at all — a code is the whole of their access.

The fastest way to judge this is to take one yourself.

Leave an email address and we will send you a code for our demo challenge. Fifteen minutes in your own editor, with the agent beside you, exactly as a candidate would see it.

Take the demo challenge