Test analytics · Release gates · API monitoring

Every failure
has a story. Discover it.

inverseStory turns the test reports you already produce into a history you can question — what broke, why, and whether it is new — and gives every build a verdict your pipeline can act on. Your APIs are watched around the clock in the same place.

  • Free plan, no card needed
  • One step in any CI
  • Isolated data per organisation
6
test frameworks
9
ways to compare
7
alert channels
inversestory.com/app
Sample data
Health
90.7%
Execution trendPass rate · %
70%80%90%100%11 days ago5 days agoToday
82.0%

Passed ÷ tests each day — the dip is the regression.

Day comparison
Today95.0%
Yesterday93.7%
1 newly failing4 fixed5 still failing
Recent executions
live
  • #1843
    checkout-web
    feat/promo-codes
    Failed
  • #611
    payments-api
    release/2.4
    Passed
  • #1842
    checkout-web
    main
    Passed

Reads the reports you already produce · from any CI

CucumberJSONPlaywrightJSON · JUnit XMLTestNGXMLPytestJSONJUnitXMLCypressMochawesome JSONGitHub ActionsGitLab CIJenkinsAzure DevOpsBitbucketCircleCITeamCity
How it works

From raw report to a decision, in one pass.

No agents to install and nothing to run by hand. Every upload updates the whole history.

STEP 01

Send

Upload a report in the browser, or add one step to CI. Playwright, Cypress, Cucumber, JUnit, TestNG and Pytest.

STEP 02

Understand

Failures are grouped by cause and tagged New, Always failing, Flaky or Slow — from each test's own history.

STEP 03

Decide

Your quality gate gives the build a pass or fail your pipeline can act on, with every rule's real number.

STEP 04

Explore

Dashboards, daily reports and comparisons — and every number opens the runs and tests behind it.

Test analytics

Ask your test history anything.

The questions every team asks every morning, answered from your own runs — and every number opens the tests behind it.

Day comparison
checkout-web · all suites
Sample data
Today
97.6%
1,284 tests · 31 failed · 12 skipped
Yesterday
97.4%
1,284 tests · 33 failed · 15 skipped
3
Newly failing
5
Fixed
26
Still failing
12
New tests
Newly failing — passed yesterday
  • applies promo code after refreshcheckout-web › cart
  • refund shows on statementpayments-api › refunds
  • guest checkout keeps addresscheckout-web › guest

See your own history in minutes.

Upload one report in the browser — or add one step to CI — and every page above fills in.

Flaky tests & regressions

A real regression, or just flaky?

The question behind every red build — and the one that eats a tester's morning. inverseStory answers it from each test's own history, before anyone opens a stack trace.

Run 1843 · feat/promo-codes
48 failed of 1,284 tests
Sample data

48 tests failed. Which of them did this change break?

Regressions–
Passed last run, failing now — 2 causes
Flaky–
Flip between runs, or passed only on retry
Always failing–
Failed each of the last 5 runs — not new
Regressions
Passed in Run 1842 · failing in Run 1843
Sample data
  1. Cause 1New5 tests · checkout-web
    TimeoutError: waiting for locator('#apply-promo')
    • applies promo code at checkoutpassedfailed
    • promo code survives cart editpassedfailed
    • removes an expired promopassedfailed
    • +2 more
  2. Cause 2New2 tests · billing
    AssertionError: expected 1180.00, received 1298.00
    • order total includes GSTpassedfailed
    • invoice total matches cartpassedfailed
4 tests fixed since Run 1842 — the recoveries are listed too, so a fix gets noticed.
Flaky tests
each test's last 30 runs, oldest first
Sample data
  • applies promo code after refresh
    checkout-web
    Chaotic
    22 flips ÷ 29 = 0.76
  • search suggests after 2 chars
    search
    Intermittent
    8 flips ÷ 29 = 0.28
  • exports invoice as PDF
    billing
    Broken, not flaky
    9 failures in a row
  • restores a saved cart
    checkout-web
    Fixed
    17 passes in a row
  • applies loyalty points
    checkout-web
    Not enough runs
    4 of 6 runs

Score = flips ÷ (runs − 1). The third test flickered early, then failed nine runs straight — a flip-rate alone would call it flaky; this calls it broken.

Regressions, by cause

Tests that passed last run and fail now, grouped by the error behind them — with the ones that got fixed alongside.

A flake score, not a hunch

Flips ÷ runs across each test's last 30 runs — then why: order-dependent, environment-brittle, slow swings, a new error every time. Nothing is called flaky before 6 runs and 5 flips.

Broken is never “just flaky”

Five failures in a row is a broken test, not a flaky one, and it stays on your list. Once a test passes ten in a row it is marked Fixed and leaves it.

Gate on regressions, not noise

Quality gates count new failures and flaky tests separately: block on one, set a ceiling for the other.

Release confidence

Ship on a verdict, not a hunch.

Decide once what a shippable build looks like. Every run is checked against it, and your pipeline gets a clear pass or fail — with the reasons.

  • Quality gates

    Write down what “good enough” means — pass rate, nothing newly broken, no never-seen errors — and every run gets a verdict. Presets for main and pull requests.

  • Smart tags

    Each result is tagged New, Always failing, Flaky or Slow, worked out from that test's own history. Look at New first.

  • Test muting

    A known-broken test keeps running and reporting without failing the build. Every mute has an end date, so none is forgotten.

  • Top failure clusters

    Every failed run opens with where to start: the biggest cause, its share of the failures, and the tests behind it.

  • Parallel shards, one run

    Uploads from many machines join into one run with one pass rate and one verdict. A retried upload is never counted twice.

ci / test-and-shipmain
API monitoring

Your API journeys, watched and explained.

Run real API journeys on a schedule. When one breaks, you learn which step failed first, what changed since the previous run — in Slack, Teams, PagerDuty or email.

Live console
Sample data
Flow: Checkout journeyevery 5 min · production
Scheduled
STEP 01POST
Get token
/auth/token
{{token}}—
STEP 02GET
Load profile
/users/me
{{userId}}—
STEP 03POST
Create order
/orders
{{orderId}}—
STEP 04GET
Verify order
/orders/{{orderId}}
status = CONFIRMED—

Values captured in one step — a token, an id — feed the next. Run only if, repeat until and for each keep the journey real.

Conditions & loops

Journeys

Chain requests into a real journey — log in, create, pay, verify — with run-if, repeat-until and for-each.

Values find their source

Dependencies

A request says where its token or id comes from. That is fetched first, so any request can run on its own.

OpenAPI · Postman · HAR · cURL

Import

Bring what you already have. Folders, auth and bodies come across, and the import says plainly what could not.

Required vs optional

Response contracts

Mark the fields that must match and the ones allowed to drift. A drifting optional field marks the run Degraded, not failed.

7 channels

Alerts

One alert per outage, naming the step that failed, with a link to the run and a note when it recovers.

Run-to-run

What changed

Every run is compared with the one before: which check flipped, which step failed first, real latency moves.

Monitor your first API free.

Requests, schedules, checks and alerts are on every plan, Free included. Multi-step journeys start on Team.

Trust

Your data stays yours.

Isolated per organisation

Each organisation's data lives in its own database schema.

Secrets stay secret

Vault values and credentials are encrypted with AES-256-GCM and never shown again.

Roles that fit your team

Organisation and project roles decide who can view, analyse, mute or administer.

Nothing lost by accident

A deleted project or organisation can be recovered for 10 days.

Never sold

Your test data is never sold, and never used to train models for anyone else.

Pricing

Start free. Grow when your suite does.

Team and Business start with a free trial — no card required. Starter is ready the moment you buy it.

What's next

AI failure analysis · Slack & webhooks for test runs · Single sign-on (SAML) · Self-hosted deployment|In progress, not part of any plan until it ships.

FAQ

Questions, answered.

Which test frameworks do you support?

Cucumber, Playwright, TestNG, Pytest, JUnit and Cypress: Cucumber JSON, Playwright JSON or JUnit XML, TestNG XML, Pytest JSON, JUnit XML and Cypress Mochawesome JSON. Anything that writes JUnit XML can be uploaded with our command-line uploader. If your framework is missing, tell us.

Do I need to change my CI?

One step. Add a ready-made uploader (Java, Python, Node.js or bash) or a single curl call with your project's API key. It works with GitHub Actions, GitLab CI, Jenkins, Azure DevOps, Bitbucket, CircleCI and TeamCity, reads branch, commit and build for you, and never fails your build unless you ask it to.

What is a quality gate?

Your team's rule for “good enough to ship”, checked on every run — for example, pass rate at least 95% and nothing newly broken. The verdict shows on the run, and one curl call lets your pipeline stop when the gate fails. It never changes the run's own result.

How do you tell a real failure from a flaky one?

From each test's own history. Every result is tagged New (passed last time), Always failing, Flaky (passed on retry or keeps flipping) or Slow, and failures that share a cause are grouped together — so fifty red tests read as the one problem they are.

What does API monitoring include?

Build REST and GraphQL requests, or import them from OpenAPI, Postman, HAR or cURL; chain them into journeys with conditions and loops; run them on a schedule with checks on every response. When something breaks you get one alert per outage in Slack, Microsoft Teams, Discord, Google Chat, PagerDuty, email or a webhook — with the failing step named, and a note when it recovers.

Does inverseStory use AI today?

No. Everything you see is computed from your own runs — failure grouping, smart tags, flake detection and trends — so it is the same every time and can't make anything up. AI failure analysis is on our roadmap, and no plan is sold with it.

Is my data private?

Each organisation's data lives in its own database schema, secrets are encrypted at rest with AES-256-GCM, and your test data is never sold or used to train models for anyone else.

What happens when my trial ends?

Nothing is deleted. Your organisation moves to the Free plan's limits and keeps every project and run. A deleted project or organisation can still be recovered for 10 days.

Your reports already have the answers.
inverseStory finds them.

Start free. Upload your first report and see the story behind your test suite in minutes.