How the Software AI-Integration Index was built — and why you can trust the ratings.

Every software vendor says they have an API. It’s on the integrations page, usually next to a wall of partner logos. “Connect anything.” “Fully open.” “Built for automation.”

A lot of the time, it isn’t true — or it’s true in a way that’s useless to you. The “API” is the login your customers use, not one you can build against. The “integration” is a single Zapier zap a partner maintains. The endpoint reads data but won’t write it back, so it can’t actually run your workflow. None of that shows up on the marketing page. You find out three weeks into a build, when the thing you were promised turns out not to exist.

So when we set out to rate how AI-integratable the software in 19 Australian industries actually is, we made one rule for ourselves: believe nothing a vendor says about its own API until we’ve tried to prove it false.

The result is the Software AI-Integration Index — 525 tools across 19 sectors, each rated on the real path to a working AI integration. This is how we built it, and what the doubt caught.

The job: not “does it have an API,” but “can you actually build on it”

It’s easy to put a green tick next to “has an API.” It’s nearly meaningless. The questions that decide whether an AI agent can actually do useful work in a tool are narrower and more annoying:

• Is it a provider API you can build against, or just the consumer login your staff use?
• Is it real and shipping, or a “coming soon” on a developer page that’s been there for two years?
• Does it write, or only read? An agent that can see your data but can’t act on it is a dashboard, not an automation.
• What does auth actually take — an API key, or a six-week enterprise partner application?

For every one of the 525 tools we rated each of those, from the vendor’s actual developer documentation, and rolled it into a single ease band — from Easy through to Closed — that says how hard the path to a real integration is. Not whether an API exists. Whether you can get work done through it.

The method: rate it, then attack it

The part that makes the ratings trustworthy isn’t the rating. It’s what happens next.

For every tool, the process ran in two halves. First, a research pass: find the genuine incumbents in a sector — the software people in that industry actually run — and rate each one from its developer docs. That gets you a confident, tidy table. Confident and tidy is exactly the problem. AI is a machine for producing plausible answers, and plausible is not the same as true.

So the second half attacks the first. Every claim that mattered — every “yes, it has an open API” — was handed to independent AI skeptics whose only job was to refute it, each coming at it from a different angle: Is this a provider API or a consumer one? Is it real or vaporware? Does it actually write, or only read? Three skeptics per shaky claim, majority rules. And the tie-breaker that does the real work: if a verifier can’t find the provider documentation, the claim is assumed false. The burden of proof sits on the vendor, not on us wanting to believe them.

This matters because the failure it guards against is specific and embarrassing. Ask a leading question and you get your own answer back, dressed up as a finding. One AI agreeing with another isn’t two opinions — it’s one assumption, echoed twice. The skeptic pass exists to break that echo.

What the doubt caught: 47 claims that didn’t survive

Across two verification sweeps, 47 API claims that looked solid in the first pass were overturned and downgraded to Closed. Not edge cases — tools we’d initially rated as integrable, that turned out not to be.

The patterns repeated:

• “Integrations” that aren’t an API. A whole tier of automotive workshop and dealer software markets an integrations marketplace — logos, partners, “connect your tools” — while exposing no provider API you can build against at all. It was the single heaviest correction in the whole index.
• Consumer login mistaken for a developer platform. Several tools, legal software especially, have a slick portal your staff sign into and no way for anything else to sign in. That’s not an integration surface. It’s a front door with no loading dock.
• Read-only dressed up as automation. Endpoints that hand back data but won’t accept it, sold as “automation-ready.” An agent can watch, but it can’t do.

The honest part — the part that’s the whole point — is that we downgraded these even when it made our own recommendation list shorter. The index isn’t there to sell a happy story about how integrable everything is. Of the 525 tools, 80 (15%) are flatly Closed, and we say so. Only 312 (59%) expose a provider API at all. The number we’re proudest of isn’t the green ticks. It’s the 47 we took away.

What the ratings don’t promise

Credibility is mostly in the caveats, so here are ours, plainly.

The index measures the documented integration surface — what a vendor’s developer docs say is possible — at the time of research. That is not the same as a working integration in your tenancy. APIs ship, get deprecated, and disappear behind enterprise plans. An Easy rating means “safe to plan around,” not “already built.” Before anyone commits real engineering effort, the integration gets tested against a live account. The index is the map, not the territory — it’s there to stop you walking into a wall, not to promise the road is paved.

That’s also why a handful of ratings still carry a “to confirm” marker rather than a false certainty. We’d rather show you the one cell we haven’t nailed down than pretend we have.

Why this is how we work

This is the same discipline we bring to client work, pointed at a dataset instead of a website: get AI to do the synthesis — the research, the first draft, the tedious cross-referencing of 525 tools — and then attack what it produced before trusting a word of it.

AI is brilliant at the spaces between things and completely indifferent to whether it’s right. The skill was never getting an answer out of the model; you can get an answer out of it all day. The skill is the doubt you wrap around it. The AI is the cheap part.

If you’re making integration decisions off a vendor’s marketing page — or letting an AI mark its own homework — that’s the gap. Happy to show you how it runs.

The Software AI-Integration Index covers 525 tools across 19 Australian sectors: 312 (59%) expose a provider API, 80 (15%) are flatly Closed, 21 ship native AI integration (MCP), and 190 sit in the easiest-to-integrate tier. Built and maintained by Real Minds AI, Melbourne.