/scoring

# how scoring works

// here is exactly how we measure it, so nothing is a surprise on competition day.

# the slop score

scoring is deduction-only. you start at zero (clean) and every probe that finds a problem adds slop. passing a probe, or not having the feature it targets, adds nothing. lower is better, and zero is perfect. it is golf, not basketball.

you do not get credit for not having SQL injection. that is the baseline we expect. you get penalized for having it.

defending 7 of 8 SQL endpoints is not 87% safe. it is a breach through the 8th. the seven clean ones add nothing, the one failure adds its full penalty. that is how real security works, so that is how slop works.

# what gets tested

one catalog, three bundles, run identically against every submission:

security     sql injection · xss · auth bypass · access control
             csrf · file upload · sensitive-path exposure
qa           crash resistance · error hygiene · http semantics · encoding
performance  speed gates (ttfb/fcp/inp) · load · dos resistance

each penalty scales by how common the flaw is and how bad its worst case is, discounted by how hard the fix is in 24 minutes. a competent engineer under a real deadline triages, and the scoring credits that judgment instead of demanding perfection.

# the process, step by step

submit→deploy→discover→match→run probes→tally→results
0

submit

at the 24-minute freeze you upload your code with its Dockerfile through the portal and it is queued for testing. you get a 3-minute grace window after the buzzer, so a slow upload does not cost you work you finished in time. the buzzer still ends the build.

1

deploy

we build your Dockerfile and run your app in an isolated container. the only thing it can reach on the network is hackletleague.com, which is where we hand out keys if your app needs to call the AI at runtime. everything else is blocked. we check it answers on $PORT within ~60 seconds. if it never responds, that is a DNF.

2

discover

we explore your running app over HTTP (pages, forms, APIs, even JS-rendered routes) to map its surface. we only ever see what your app exposes, never your source or your stack.

3

match

every probe in the catalog that applies to your surface gets queued. probes for features you do not have are simply skipped, with no penalty.

4

run the probes

each probe checks for one specific problem and either finds slop (+penalty) or does not (0). timing and concurrency checks run several times and take the median, so one unlucky moment will not sink you.

5

tally

we sum the penalties into your slop score, weighted by breadth and severity. one SQL injection outweighs a pile of minor issues, and repeating the same small mistake everywhere does not stack up linearly.

6

results

your slop score joins your communication score (pitch + cross-examination), and Best Overall is decided by rank across both. your container is then destroyed; nothing you ran sticks around.

# your part: a dockerfile

your application is graded as deployed. whatever answers on your port at the buzzer is what is scored.

your submission includes a Dockerfile (we ship starter templates for every common stack, so you barely touch it). your app listens on the port we hand it via $PORT. that is the whole contract. if you need to store data, ship a SQLite file in your repo, or keep it in memory, or use the browser. we do not hand you a database.

if your app does not deploy and answer HTTP, it is a DNF, the worst possible outcome, ranked below everyone who shipped something that actually runs. a thing that runs beats a clever thing that does not.

# you can study the catalog

the catalog is public. study it, and self-test against it while you build. beating it payload by payload is still the wrong way to use it, though. genuine defense (parameterized queries, output encoding, real access control) fixes the actual flaw instead of the specific payload, and that is what keeps holding up as the catalog grows.

# tested the same way, whatever your stack

once your container answers HTTP, the runner is stack-blind. Flask, Express, Go, Rust, a hand-rolled server, it does not matter. the catalog is identical for everyone. your stack choice affects how easily you defend (some frameworks handle a lot by default), not what gets tested. same ruler for all.

← the two formats · leaderboard →