How your test automation cost is worked out

What you will need

Most of it you already have. Five numbers do most of the work, and the calculator starts with editable industry estimates for everything else.

• How many test cases are in the regression suite you would automate.
• How many times a year you run a full regression. Monthly releases are about 12; fortnightly, 26; a packaged application on its vendor’s upgrade cadence, 4–6.
• How many defects a regression cycle typically finds. Count the bugs raised from the last few cycles in your tracker and average them.
• What an engineer costs you — loaded annual cost and productive hours. The default is $120,000 over 1,800 hours, about $67 an hour.
• The quotes, when they arrive. Licence and infrastructure lines are blank or estimated until you fill them in.

If you can also get each vendor to run a proof of concept on your own application, time three things: how long it takes to build a test case, how many cases break across one release, and how long a failure takes to triage. Those replace the estimates that matter most.

Cost lives in three phases

A suite is paid for three times: once to build, every release to run, and continuously to keep working as the product changes. The licence is usually the smallest of the four lines, which is why a comparison on licence price alone tends to pick the wrong tool. Every name below links to its glossary entry.

Build — paid once

Authoring dominates. The gap between a codeless tool at fifteen minutes a case and a hand-written framework at four hours is, across a few hundred cases, measured in person-months — and it all lands in year one. Set-up, training and scoping are counted too, because they are real hours your team will spend before the first case is written.

Run — paid every release

People are charged for what they do, never for watching a machine. For a tool, execution is machine time — it decides how long a run takes, not what it costs. What costs money is what someone does afterwards: look at each failure and decide whether it is a real defect, a broken test, an environment problem or a flake; write up the real ones; sign the run off. That deciding and writing up is defect validation, and the failures it rejects are the false positives that cost triage time whichever tool produced them.

You are never asked how many failures a run produces — no one evaluating tools can know that per tool. It is worked out from three things you do know: how many defects a cycle finds (your product), how many tests the product broke (the breakage rate, spread over the year’s runs), and the tool’s flake rate. A run finds broken tests; Maintain fixes them, so nothing is counted twice. For manual testing the tester is the run, so execution is counted in full and there is no separate triage step.

Maintain — paid continuously

This is where automation programmes stall, and it is the line a proof of concept never shows because nothing has changed yet. When a vendor claims self-healing, or fewer false positives, they are claiming a lower breakage rate — and this is where you put a value on that claim: change the rate from 30% to 10% and watch the five-year total. What healing does not remove is the person: every automatic repair still gets a short review before it is trusted, so a heal is priced as a review rather than as nothing.

Turning hours into money

Hours times your rate; nothing cleverer than that, so anyone in finance can reproduce it. Where a division would hit a legitimate zero — no engineers budgeted, no runs a year — you see a dash, never a zero pretending to be an answer.

Hours you buy from the vendor

A quote often includes the vendor’s own people: an onboarding package, two hundred hours of implementation, a managed service that runs the suite for you. Enter the services rate, the share of each phase they take on, and any fixed package, per vendor. The calculator charges only the remaining hours to your team at your rate, shows the rest as an invoice, and takes the invoiced hours out of build duration and FTE demand — the services team is assumed to keep pace. Nothing about the amount of work changes; only who does it and what it costs.

Adding it up

The build lands entirely in year one; the other lines recur for every year of the horizon (five by default — set it in Settings). That is why the cumulative chart matters more than any single year: it is where a tool with the higher licence and the lower maintenance overtakes the cheaper licence, and the month it happens is the number to remember.

Licences are counted once, not once per application

If you are comparing across several applications, licence and infrastructure belong to the vendor, not to each application. Adding them up per application would bill the same enterprise licence four times on a four-app estate, which quietly favours whichever tool has the smallest licence.

The All Apps view counts them once. Each application’s own view shows an apportioned share — by that application’s share of test cases. The shares always add back to the total.

Can your team actually build it in time?

A cost model can produce a plan nobody can execute. Your applications compete for the same people, so the build duration is worked out across the whole workspace, not per application. Where the build would outrun the gap between your releases, or need more engineers than you have budgeted, the calculator flags it — before the contract is signed rather than after.

Why we do not price named products

Cost estimates describe a category of tool — AI-native, script-based, open-source framework, manual — never a named product. Comparison pages name vendors and state sourced facts about them; they do not attach a price to a name. When a comparison opens in the calculator, every licensed product starts from the same editable estimates, so the only numbers that separate two named tools are the ones you entered.

A site that priced a named company’s software could not be neutral ground: any vendor that lost such an evaluation could argue the numbers were weighted against them — and so could your procurement team.

The default scenario

Figures on comparison pages describe this suite unless a page says otherwise:

• One application with 300 test cases.
• 12 full regression runs a year, each finding about 5 defects.
• 40 new cases added a year as the product grows.
• A loaded cost of $120k per engineer per year across 1800 working hours.
• A 5-year horizon.

It is a generic mid-size estate, chosen to be recognisable rather than favourable to anything. Open the Calculator to replace it with yours.

What is deliberately left out

The calculator counts labour, licensing, infrastructure and AI token spend. It does not price the quality outcomes that are usually the real reason to automate: defects that reach production, incidents avoided, and the release velocity a faster feedback loop buys.

Those are real and often larger than the labour saving. Leaving them out understates the case for automation, which is the right direction of error for a number you have to defend in front of a finance function — but say so when you present it, because someone will ask.

Also left out on purpose: discounted cash flow, confidence ranges and sensitivity analysis. Discounting is not how organisations choose a test tool, and the other two double the input burden for a precision the underlying estimates do not support.