straight_segments at
47.65%
of 2,000 trials, difficulty medium.
Validiti publishes the worst number — not the best. The drift signature
feed at signatures.validiti.com/captcha/feed.jsonl closes
classes as they're identified; deployments running the receiver poll it
every 60 s by default.
| Attacker class | Trials | Passed | Pass rate | 95% CI | Mean score | Elapsed |
|---|---|---|---|---|---|---|
naive_linear | 2,000 | 0 | 0.00% | 0.00% – 0.19% | 0.538 | 0.78s |
straight_segments | 2,000 | 953 | 47.65% | 45.47% – 49.84% | 0.825 | 0.98s |
bezier_no_noise | 2,000 | 0 | 0.00% | 0.00% – 0.19% | 0.774 | 1.26s |
bartimaeus_basic | 2,000 | 46 | 2.30% | 1.73% – 3.05% | 0.834 | 1.16s |
bartimaeus_humanized | 2,000 | 406 | 20.30% | 18.59% – 22.12% | 0.826 | 1.23s |
wrong_order | 2,000 | 0 | 0.00% | 0.00% – 0.19% | 0.635 | 1.15s |
replay_attack | 2,000 | 0 | 0.00% | 0.00% – 0.19% | 0.635 | 0.52s |
Every row is a synthetic attacker class generating CAPTCHA submissions
in a different way and submitting them to the same sealed
Validator that ships in every distribution shape. The validator
returns a passed boolean plus a confidence score and a list of
flags; this table aggregates pass rates across thousands of trials with
Wilson-score 95% confidence intervals.
naive_linear / straight_segments /
bezier_no_noise are strawman bots. They pass rarely
because the validator's motor-pattern layer rejects machine-shaped
paths.bartimaeus_basic is the best-effort path a typical bot
library produces — bezier with Gaussian noise, randomized delays.
This is roughly the sophistication of off-the-shelf
undetected-chromedriver attacks.bartimaeus_humanized is Validiti's own visual-attack
engine running its most sophisticated path generator. If this
can't beat the system, no purely-visual attack can.wrong_order + replay_attack exercise
structural defenses (connection ordering, cross-path consistency).
The validator weights timing realism more heavily than path-shape regularity.
A bot that produces a perfectly-straight path with realistic
inter-connection delays — the straight_segments class above —
slips past at a meaningfully higher rate than the more-sophisticated
bartimaeus_humanized. This is a known gap and is exactly
why ShiftCAPTCHA ships a drift signature feed: classes identified
in adversarial testing become signed signatures (matched on
motor_entropy_max, solve_ms_max,
idle_streak_min) and propagate to every customer
instance within one poll interval.
Customers running their own threat-intel work can mint signatures against attack classes they observe in production and submit them to the public feed (paid drift-publishing tier, post-launch). Internal-only signatures (a customer's fleet only, never public) are also supported via a private feed URL.
git clone https://github.com/validiti/shiftcaptcha
cd shiftcaptcha
bash deploy/shiftcaptcha/dist/server/pack_runtime.sh
python3 deploy/shiftcaptcha/dist/benchmark/adversarial_harness.py \
--trials 2000 --difficulty medium