Our verification standard was written for code: read the repository, check the install command, confirm the maintainer, look at the last commit. It works because a repository is durable and inspectable.
Media tools break that model. A video or image generator is a hosted service whose behaviour changes without a commit, and whose quality is a matter of observation rather than inspection. Pretending otherwise would produce entries that look verified and are not.
The short answer
For media tools, verification means: dated observation of what the tool produced, stated limitations we actually hit, and an explicit note that quality claims are observations rather than measurements. We do not reproduce vendor benchmarks and we do not claim to.
What can be checked
- Access and pricing shape. Is there a free tier, what does a paid plan cost per unit, does it need a billing-enabled cloud project.
- Input and output surface. Text, image, or reference frame in; what formats and lengths out.
- Stated limitations. Watermarks on free tiers, content filters, regional availability.
- Behaviour on a known prompt. We can generate with a fixed prompt and describe the result — artefacts, text rendering, motion coherence.
That last item is the closest thing to a real test, and it is inherently a sample. Two runs of the same prompt do not necessarily produce the same output, which is a property of the tool, not a flaw in the test.
What cannot be checked
| Claim | Why we cannot verify it |
|---|---|
| "Best quality" | Not measurable without a defined task and human raters |
| "Trained on licensed data" | Takes the provider's word; we can only quote the claim |
| "No copyrighted output" | Unprovable in general; specific cases are anecdotal |
| Benchmark scores | Vendor-run, task-specific, rarely reproducible |
This is why media entries carry shorter descriptions and more careful language than code entries. An entry that overstates what we know is worse than a short one.
The verification date matters more here
A code entry's date tells you when the repository was last checked. A media entry's date tells you when the tool last worked as described — which can be invalidated by a model update overnight.
That makes the date load-bearing rather than decorative. It is also why we refuse to remove it for visual cleanliness: the date is the entry's honesty.
What we do instead of benchmarks
We describe the failure modes we encountered, in the same terms a reader would use: hands render wrong, on-screen text mangles, clips beyond ten seconds drift, free tier watermarks. These are observations, and we label them as such.
When an observation is genuinely stable across tools — like text rendering being unreliable in every video model — we state it as a general limitation rather than a per-tool verdict.
Why this is useful anyway
A reader deciding between Sora, Veo, and Kling is not asking which model wins a benchmark. They are asking which one can hold the shot they need, and what it will cost in retries.
Answering that honestly means describing capabilities and limits, not ranking. It also means admitting the category moves monthly, which is what the date is for.
What an honest entry says
Compare two hypothetical entries. One says "the best video model for realistic motion". The other says: generated six clips from identical briefs; hands failed in four; on-screen text failed in all six; motion beyond eight seconds drifted in every take; free tier includes a watermark.
The second entry is less quotable and more useful. It also dates honestly: a reader knows exactly what was observed and when, and can discount accordingly when the model updates.
The standard, written down
Our rule for media entries is three sentences of description and a checked date. No ranking language, no superlatives, no benchmark numbers, and a stated limitation for every capability we mention.
Where we cannot observe something — training data provenance, infringement risk — the entry says "claims" or omits it. The absence is information: a reader can see we did not verify it.
Why this is a policy, not a shortcut
It would be faster to copy a vendor's feature list and add a date. It would also be a different product. The directory's value is that a listing tells you what someone observed, not what a press release asserts.
That standard is harder to apply to media than to code, and it is the reason media entries are shorter. Short and true beats long and unverifiable.
Frequently asked questions
Are media entries reviewed like code entries? No, and the entry says so. Code entries can be inspected; media entries are observed and dated.
Why include a tool you cannot fully verify? Because the reader needs a starting point and an honest description of what is known. Excluding everything unverifiable would leave an empty category.
Do you update media entries when a model changes? Yes, and the verified date changes with the update. An old date is a signal, not a defect.
Last checked: 9 November 2026.
Related: what verified means here, how we label official servers, and our methodology.