The judgment score
Everyone has opinions. You have a percentile.
In the age of AI that knows everything, one thing keeps gaining value: judgment. One real case a day to measure yours, on method and calibration, never on luck.
The only training to become a contrarian
The ritual
One case a day. Thirty cases for a verdict.
You decide
One real case, the same for everyone. You set your probability and you lock it.
Your method counts
Base rate, weak signal, one-sentence thesis. Method weighs in the score, because you can reason well and land badly.
The verdict drops
The real outcome and the community distribution, visible only after locking.
Your percentile moves
One score per domain, a calibration curve. Nothing public before thirty resolved cases.
No one is good at everything: everyone reigns somewhere.
The score
What the score measures.
Being right once is often luck. Being right at the right level of confidence, week after week, is judgment. The score measures it across three dimensions.
Everyone plays the same case. If your estimate beats the crowd's, your score goes up; otherwise it goes down, by 25 points at most. It is an average per case: playing more does not inflate the score, it makes it reliable.
When you say “70%”, does it happen 70% of the time? Almost no one knows their answer. Your curve shows it to you, in black and white.
Writing down a thesis before deciding is a verifiable reflex, and it counts in the score. Because you can decide well and land badly, method pays even when the outcome betrays you.
In forecasting tournaments, a single hour of calibration training cuts error by 6 to 11%, durably.2 What was missing was a place to train every day.
The diagonal is perfect calibration: saying 70% and being right 70% of the time. The curve below is most of us: too sure of ourselves. At thirty resolved cases, you discover yours.
Your score is an average: it does not depend on how many cases you played, and each case weighs at most 25 points. The case count is shown next to the score, and nothing goes public before thirty resolved cases: no one gets judged on noise.
The reference
A score that means something.
What the TOEFL is to English, Percentyl wants to be to judgment: a standardized measure of your ability to decide under uncertainty. Here is what makes a score trustworthy.
“Top 3% Tech & AI” is not an opinion. It is a measured fact, verifiable from a link.
Join the distribution.
Free. One case a day, the contrarian thinking course included, a percentile you earn.
Create my accountAlready have an account? Sign in.
The course · included with your account
Reason has become free. Judgment has not.
AIs reason fast, without fatigue or ego, and they carry the consensus of everything ever written. What is becoming rare, and therefore precious, is the spark: knowing when the consensus is right, and when it is wrong.
That is what the contrarian thinking path trains: six levels built notably on the work of Kahneman, Tetlock, Galef, Thiel, Duke and Flyvbjerg, one lesson a day, spaced reviews, a final exam and a certificate you earn. Theory is learned there. The edge is proven afterwards, one case a day.
Contrarian thinking can be learned, and the full course is already here. Built to fit a busy life: the first lesson takes ten minutes, and every validated lesson stays yours for good.
The leaderboard
The top of the distribution.
One leaderboard per domain: no one is good at everything. Updated as soon as a case resolves, reserved for players with at least thirty resolved cases across all domains.
References
They said it before us.
A century of research and practice converges on the same idea: judgment is not a gift. It is a discipline, and it is maintained through repetition.
“The most contrarian thing of all is not to oppose the crowd but to think for yourself.”
“Truth, or more precisely, an accurate understanding of reality, is the essential foundation for any good outcome.”
“The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt.”
“The first principle is that you must not fool yourself. And you are the easiest person to fool.”
“It is remarkable how much long-term advantage we have gotten by trying to be consistently not stupid, instead of trying to be very intelligent.”
“What makes a decision great is not that it has a great outcome. A great decision is the result of a good process.”
“The strongest predictor of rising into the ranks of superforecasters is perpetual beta, the degree to which one is committed to belief updating and self-improvement.”
In the same research, this trait predicts three times better than intelligence. Judgment can be trained. Percentyl is where you do it, one case a day.
Original quotes. Full sources in the footer.³
The questions we already get
What people ask before they start.
Q.01How do you measure the quality of your judgment?
By comparing what you announce to what happens, across a large number of decisions. Percentyl applies the method of forecasting tournaments: you put a number on a real case, you lock it before the outcome is known, and reality settles it. After thirty resolved cases, two things become measurable: your accuracy against the crowd, and your calibration, meaning the gap between the confidence you state and how often you turn out to be right.
Q.02What exactly is calibration?
A decision maker is well calibrated when the things they announce at 70 % happen about 70 % of the time. It has nothing to do with being right often: you can be very knowledgeable and very badly calibrated, saying 95 % for things that happen two times out of three. Calibration is the rarest and the most transferable judgment skill, because it bears on how you dose your confidence, not on a field.
Q.03What is contrarian thinking?
It is not being against the crowd on principle, which still amounts to letting the crowd decide. It is knowing when the consensus sees right, and when it is wrong. Peter Thiel puts it in a single question: what important truth do very few people agree with you on? A useful contrarian follows the majority most of the time, and departs from it at the precise moment it is wrong, with reasons that can be checked before the outcome, not told afterwards.
Q.04Doesn't an AI forecast better than a human?
On knowledge and speed, by far. But a language model gives back the consensus of what has been written: massively reasonable, rarely contrarian at the right moment. And the value of a decision plays out mostly when common thinking is wrong. That is exactly the ground Percentyl measures, and that is why the ability to depart from the consensus with good reasons becomes more valuable as reasoning becomes free.
Q.05How is the score calculated?
On each case, your error is compared to the median player's: you gain or lose 25 points at most, never more. Your displayed score is the average of those gaps, so it does not depend on how many cases you have played: playing more does not inflate the score, it makes it reliable. The method adds a bounded bonus, and nothing is public before thirty resolved cases, all fields combined.
Q.06Can judgment really be trained?
Yes, and it has been measured. In the geopolitical forecasting tournaments of the 2010s, one hour of calibration training durably reduced participants' error, and the performance of the same forecasters persisted from one year to the next: it was not luck. Philip Tetlock identifies the best predictor not as intelligence, but as the habit of revising your beliefs, which weighs about three times more.
Q.07Is Percentyl gambling?
No, and it is structural: no stakes, no purchasable currency, no cash winnings, no investment advice. Points and percentiles are fictional units that cannot be transferred. What you earn here is a score that can be checked, and the right to bring it up at lunch.
Q.08Am I going to be bad at this?
Probably at first, like almost everyone: overconfidence is the best documented bias in the literature, and discovering yours as a number is precisely the information worth gold. Your score stays private before thirty resolved cases, time enough to learn without an audience. After that, you decide what you show.
Q.09Is the service free, and what is the business model?
The app is free and will stay free: one case a day, the contrarian thinking course included, the leaderboard and the public profile. Premium services will come later, around the community. No ads, no data selling, no financial advice.
Percentyl · the judgment score
The distribution is here. Only your spot is missing.
Free. One real case a day, the contrarian thinking course included, a percentile you earn. The first case is waiting.
Already have an account? Sign in.