Prediction markets

The short version

Prediction markets have a famous documented flaw: longshots are overpriced. Unlikely events trade above their true odds, so selling them should pay. It's been in the academic literature since the 1940s. We wrote down a pass/fail test before running it, pulled 2,000 real resolved markets spanning January 2024 to July 2026, and measured it.

It lost 1.9¢ per contract after real fees and spreads.

Worse for the idea: the mispricing showed up clearly in the older half of the data and then nearly vanished in the newer half it was tested against - +0.0198 in training, +0.0043 out of sample. That's the signature of a pattern that isn't really there. On a second time horizon it flipped negative outright.

So we parked the sector. The test cost a few days and saved us the biggest engine build in the project's history.

What a prediction market is

Exchanges like Kalshi let you trade yes/no contracts on real-world events - "will X happen by this date?" - priced between 1¢ and 99¢. The price is the market's probability: a contract trading at 30¢ means the market says 30%. If the event happens, YES pays $1; if not, NO does. It's a market whose entire job is putting honest numbers on uncertainty - which makes it a fascinating place to test whether bots can find edges.

Why we tested before building

Three reasons, in order of size:

1. It's a new kind of instrument. Every bot in the fleet trades one symbol with a continuous price stream. Event contracts are binary, priced as probabilities, and - the hard part - they resolve and disappear. The engine needs a new layer for discovering markets, trading their short lives, and booking their resolutions. That's a build on the scale of the covered-calls sector.

2. The obvious bot is the least testable one. The exciting idea - an AI that reads the news and estimates probabilities better than the market - cannot be backtested honestly at all: any model you'd use already knows how past events turned out, so "would it have predicted this?" is look-ahead bias in its purest form. That bot could only ever prove itself in live forward testing, from zero, over months. We'd publish it as exactly that - an experiment - never as a proven edge.

3. The boring bots come first. There are documented structural patterns worth testing that don't require understanding the news at all - like the famous longshot bias, where unlikely events are systematically overpriced. Those can be partially backtested with real fee models, which is where our rules say we have to start. Fees matter enormously here: they're highest exactly where markets are most uncertain.

What we found, in more detail

The test had four conditions, all written down before we looked at any result: the mispricing had to appear in both halves of the data, the strategy had to make money after costs, the sample had to be big enough to mean anything, and it couldn't depend on one category of market. We checked three different points in time before each market resolved - a day, a week, and a month - and took the weakest of the three as the answer, so that testing three horizons couldn't become three chances to pass.

Only the one-day horizon collected enough markets to judge, and it failed on money. The other two didn't reach the minimum sample, and we report them as "no verdict" rather than quoting their nicer-looking numbers, which rested on 24 and 14 tradable markets. Small samples are how people fool themselves with this kind of strategy: it wins often and loses rarely but enormously, so a short run of wins looks like an edge right up until it doesn't.

There's a structural reason we think this is a real dead end and not just a thin dataset. We deliberately tested liquid markets, because those are the only ones where trading costs can be measured honestly. But liquid markets are exactly where a famous public pattern gets competed away. The mispricing most likely still exists in tiny, barely-traded markets - which are precisely the ones where you can't get a real price, can't trade any size, and where the spread would eat the edge anyway. Where the edge plausibly lives is where you can neither verify nor use it.

One mistake worth admitting

Partway through, our stress test - which doubles the assumed trading cost to see if an edge survives - reported that the strategy got better when costs went up. That's impossible, and it was a bug: doubling the cost made the most extreme longshots untradable, and those were exactly the ones carrying the rare catastrophic losses. The stress test was quietly throwing away the losers.

The first attempt to fix it flipped the verdict from fail to pass. That's the moment to be suspicious, not pleased - and the fix was wrong. Corrected properly, the result is a fail. We're writing that down because a testing process you only check when you dislike the answer isn't a testing process.

Where this sits

Parked, not deleted. The venue adapters, the dataset builder and the study all still work, and the whole thing is reproducible if we ever want to revisit it - with more data, a different strategy, or a different venue.

There's also a regulatory wrinkle we won't pretend away: a Washington court blocked Kalshi from offering event contracts in this state in July 2026, which is where this project is based, and Washington's regulator considers prediction markets generally unauthorised here. That didn't decide the outcome - the numbers did - but it's part of the honest picture.

Effort goes back to the four sectors that already have live bots and forward results. A negative result you paid for is still a result.