What each number on a listing means, and how we work it out
Plain English, real numbers
Every score comes from something we measured
Each listing has a quality chart with six parts and one FolioScore out of 100. This page explains where each number comes from and the exact sums we use, then walks through the automated review every tool goes through.
For each safety check and the points it is worth, see How we check tools.
The quality chart
Six parts, each scored from 0 to 100. The further a point sits from the centre, the better. Each score is coloured from red (low) through amber to green (high). A part we haven't measured yet shows as a hollow point with no number, never as a 0.
Example onlyA made-up tool, not a real listing. Its numbers are worked through in each part below.
Performance
Google PageSpeed, on a phone
Google's PageSpeed test loads the site as a phone would and gives it a speed score out of 100. We use that score as it is. The tests run each night, 50 tools a night, starting with the ones tested longest ago.
No score if the site blocks Google's tester or can't be loaded. The listing says which.
Example: PageSpeed gave 82.
Accessibility
Google PageSpeed
The same test also scores how usable the site is for everyone, including people who use a screen reader or a keyboard. We use that score as it is.
No score when the speed test couldn't run.
Example: PageSpeed gave 94.
Reliability
Our uptime checks
The share of our uptime checks over the last 30 days in which the site answered. A slow answer, a sign-in page or a bot wall still counts as answering. An error such as 404 or 500, or no answer at all, counts as down. The checks run once a day at midnight UK time, and when a listing is published.
No score until a week of daily checks has built up, so that a few lucky checks can't read as 100%.
Example: It answered 99.6% of the checks.
Ratings
Reviews from signed-in people
The average star rating, pulled towards 3.8 stars as if 8 extra average reviews had been added, then multiplied by 20 to give a score out of 100. The more reviews a tool has, the less the pull matters: a single 5-star review gives 79, not 100.
How many times people have opened the tool, on a curve that rises quickly at first and then levels off: 50,000 visits or more scores 100. For example, 100 visits score 43 and 10,000 visits score 85.
Popularity is on the chart but isn't part of the FolioScore. It says how many people use a tool, not how good it is, and every new tool starts at 0.
No score when we don't know the number of visits.
Example: 1,200 visits give 66.
Safety
Our safety checks and the AI review
Two measurements, averaged when there are both: our rule-based safety checks, which give a score out of 100, and the risk level from the AI review, turned into a score as shown here. With only one of them, that one is the score. A Google Safe Browsing flag sets Safety to 0, whatever the rest says.
Risk level as a score
Minimal
100
Limited
80
Elevated
55
High
25
Unacceptable
0
No score until one of the two has been measured.
Example: Safety checks 86 and risk level Limited (80) average to 83.
The FolioScore: one number
The circle on every card and listing page. It's the average of the parts measured so far out of Performance, Accessibility, Reliability, Ratings and Safety, rounded to a whole number. Popularity is left out.
Example only
The example tool's FolioScore
89
(82 + 94 + 100 + 86 + 83) ÷ 5 = 89
Each part is shown rounded. The average is worked out before rounding.
A score needs at least 3 of those five parts measured. Until then the circle shows a dash, and hovering over it says why.
A part that hasn't been measured is left out of the average, not counted as 0.
For a tool's first 7 days the circle has a dashed ring, because the score is provisional.
Hover over any circle to see each part.
The automated review
Every tool is reviewed before it can be listed. The AI in these steps is Claude, made by Anthropic. Everything a listing says is treated strictly as content to judge, never as instructions to follow.
Tools that makers submit
1
Embed check
We check whether the site lets itself be shown inside another page. If it doesn't, it opens in a new tab instead.
2
Read the live page
Google PageSpeed loads the page, taking a screenshot and noting every website it contacts, and we read the page ourselves through a guarded connection. The scripts, frames and form destinations it uses become the approved list that later checks compare against.
3
Google Safe Browsing
The tool's address, where it ends up, and every website the page contacts are checked against Google's list of unsafe sites.
4
AI review
Claude looks at the listing together with the live page, its screenshot and the websites it contacts. It checks the tool is legitimate and that what it asks for matches what it claims, and looks for phishing or misleading claims. It gives a recommendation, a risk level and any findings.
5
Risk level
A separate check grades the tool Minimal, Limited, Elevated, High or Unacceptable. Asking for the camera, microphone, location or payments raises the starting level, and the AI can only raise it, never lower it.
6
Decision
The results decide what happens next, as set out here.
Published automatically only when all of these are true
Google Safe Browsing came back clean.
The AI review recommends approval.
The AI review's risk level is Minimal or Limited.
The separate risk level is Minimal.
There is no high or critical finding.
We could read the live page.
The maker has proved they own the domain.
The domain hasn't asked not to be listed.
Rejected, with the reasons sent to the maker
A Google Safe Browsing flag.
The AI review recommends rejection.
A risk level of High or Unacceptable.
Everything else goes to a person
Anything that is neither published nor rejected waits for someone on our team. So does a listing edited while it was being reviewed, one a moderator has put on hold, and one taken down for safety before.
If the AI can't answer
If the AI review or the risk check fails for any reason (an error, a refusal, an answer we can't read, or the day's AI budget running out), or the whole review fails after its retries, the tool goes to a person. An AI failure never publishes a tool and never rejects one.
Tools our team adds
1
The same review
The same AI review, risk level and Google Safe Browsing lookup run on the tool's live page.
2
Three outcomes
A High or Unacceptable risk, a recommendation to reject, a high or critical finding or a Safe Browsing flag blocks publishing, and takes the tool down at once if it is already live. An Elevated risk, any AI failure, or a Safe Browsing lookup that didn't finish needs an editor to sign it off by hand. Anything else passes.
3
Editor checks
Before publishing, an editor checks the pricing, and whether the tool trains on what you give it, against the tool's own pages. The description must be written for the listing, not copied from the site.
4
Published by a person
Tools our team adds are always published by an editor, never automatically.
Checks while a tool is listed
The review isn't a one-off. These checks keep running for as long as a tool is listed. Times are UK time.
UptimeDaily, at midnight
We check the site answers, trying once more as a web browser would if the first try fails. Each night the last 30 days of checks become its Reliability score.
Down for too long6 checks in a row
A tool that fails 6 uptime checks in a row is suspended. A newly published tool gets 24 hours' grace first.
Back up again2 healthy checks
A tool suspended for being down comes back by itself after 2 healthy checks in a row, as long as its page still matches what was reviewed and Google Safe Browsing finds nothing. After an outage of more than 72 hours, or if its address has changed, it goes back for review instead.
A fault on our sideOver 20% down
If more than 20% of the tools just checked look down at once, with at least 10 tools checked, we assume the fault is ours and suspend nothing.
Page changesEvery 6 hours
We read the page for scripts, frames and form destinations from websites no review approved. The same new website seen on 2 checks in a row takes the tool off the site until it is reviewed again. Ordinary changes to the page's text never do.
Security scanNightly
HTTPS, key security headers and any exposed .git or .env file. A high-severity finding on 2 nights in a row suspends the tool until a person restores it.
Safety checksNightly, and on publishing
The rule-based checks behind the Safety score. A score needs at least 40 points' worth of checks that could run.
Speed and accessibilityNightly
Google PageSpeed, 50 tools a night, starting with the ones tested longest ago, for the Performance and Accessibility scores.
Google Safe BrowsingNightly, or every 6 hours
Makers' tools are checked against Google's list of unsafe sites each night, and tools our team adds every 6 hours. A flag suspends the tool.
Domain ownershipMonthly
A maker's proof that they own the domain is checked again each month. If it has gone, we check daily for 3 days, then remove the verified tick.
Every safety check, and its points
How we check tools lists each safety check, how many points it is worth, what fails a tool outright and what our checks can't tell you.