[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6,"_7":8},"actionData","errors","root",{"_28":-5,"_38":-5,"_39":40,"_41":42,"_43":44,"_45":46,"_47":48,"_49":50,"_51":52,"_53":54,"_55":56,"_57":58},"routes/_store+/_readers+/email_author.$slug/route",{"_9":10,"_11":-5},"book",{"_12":13,"_14":15,"_16":17,"_18":19,"_20":21,"_22":23},"currentUser","id","Z2lkOi8vbGVhbnB1Yi9TdG9yZTo6Qm9va1Byb2R1Y3QvODE1Mjk","title","Measured. Not believed","state","published","contactAuthorMethod","email","aboutText","

\"Our model wins 68%.\" \"The strategy returns 8% a day.\" \"We're #1 on the leaderboard.\" Every one is a number computed from a model of reality and handed to you as if it *were* reality — and from the outside, an inflated number looks exactly like an honest one. You have to go and check.

This is a field guide to checking. It recomputes the AI benchmarks that rank today's models — a celebrated judge that rewards length over quality, a #1 ranking no better than a coin flip — and post-mortems a trading strategy carried to real money and lost, for reasons you could measure first: the fee bigger than the edge, the queue no backtest can simulate.

Both meet in one discipline: four questions that catch a lying number, in a leaderboard or a bot. Then the author turns them on himself — a discovery that survived cross-validation, held-out testing, and every robustness check, and was still an artifact — and on twenty-three recent papers making the same quiet mistake.

No hype. No secret edge. Just the reflex that saves a launch, a fundraise, or three months and $53 — and the nerve to run it on your own work.

","authors",[24],{"_12":25,"_26":27,"_28":29},"Z2lkOi8vbGVhbnB1Yi9TdG9yZTo6UHJvZHVjdEF1dGhvci84NzE0MA","name","Ilpo Vaatainen","user",{"_30":31},"avatar",{"_32":33,"_34":35,"_36":37},"url","https://d39qdlcrvnra4b.cloudfront.net/avatars/3752021/medium/avatar.jpg?1784478895","styleName","medium","exists",true,"shoppingCartId","shoppingCart",{"_12":71,"_72":73,"_74":37,"_75":76,"_77":78,"_79":80,"_81":37},"locale","en","readerTheme","light","constants",{"_69":70},"publishers",{"_65":66,"_67":68},"learnerOrganizations",[],"needsLocaleCookieSync",false,"announcements",{"_59":60,"_61":-5},"sentryEnvironment","production","sentryRelease","e91c4ce1b656a7c9ee875bc5061e04bc48fe0c81","announcement_store",{"_62":52,"_63":64},"announcement_authors","flag","text","

Kick off your book project, get started with GhostAI, get better at marketing, or spend the day doing all three! Free live workshops on Zoom. Saturday, June 27, 2026.

\n","companies",[],"universities",[],"paypalClientId","AX53Cwz8jR6rVVzzZiZNWWXl0dGagi-GZrjG-S3Ua14E5nrKMn7DxDQ730O4R8LusLPp49ALn6aRVaEy","no-cart","count",0,"free","updatedAt","","paymentGateway","STRIPE","cartItems",[],"loading"]

Email the Author