A Page Is Not an Endpoint

"How fast is the users page?" sounds like a question with an answer.

Opening one screen in our admin dashboard fires a fan-out. The list, plus counts, plus filter options, plus permission lookups, plus whatever widgets sit above the table. Across a handful of pages I counted 57 distinct endpoints. The p95 of any one of them tells you very little about what a person waits for, because the person waits for the slowest thing in the fan-out, and sometimes for several slow things stacked behind each other.

Which is how endpoint dashboards look healthy while users are unhappy. Every individual number is defensible. The experience is still bad.

Measure the page

So the harness works page-first. Sign in, load a target page, record the actual fan-out with every request and payload. Then replay that recorded set under concurrency, reporting p50, p95 and p99 for each request against a budget, with the option to isolate a single endpoint for a closer look. The output is a report you can hand to someone.

That last part matters more than it sounds. A finding that lives in my terminal convinces nobody. Per-endpoint percentiles against a stated budget is something a product manager can act on and QA can attach to a ticket.

Concurrency is not a detail

Replaying sequentially would have missed the most interesting bug I found with it.

One page looked acceptable with requests going out one at a time. Under a ten-way concurrent cold burst, which is just ten people opening the page after a deploy, its p95 was around 7.5 seconds. A stampede: nothing cached, so every concurrent request started computing the same expensive result instead of one computing while the rest waited. After the fix, the same burst measured 70 to 96 milliseconds.

Roughly eighty to a hundredfold, and it exists only under concurrency. A sequential test would have called the page fine, twice, and moved on.

Empty databases lie

The other route to a comfortable useless result is testing against a dev database with a few hundred rows.

Nearly every performance problem worth finding is a scale problem. Deep pagination is fine at page three. An unindexed sort is fine when the collection fits in memory. A missing projection is fine when documents are small. They all pass on a laptop and fail in production, which is the worst combination, because your test suite is actively reassuring you.

So the companion to the harness is a seeding tool generating production-shaped volume. Tens of thousands of companies, tens of thousands of jobs, with realistic relationships rather than isolated rows.

Trust the instrument first

Briefly, since I've written about it elsewhere: early on my own harness reported a p95 computed from five samples. A 95th percentile over five runs is the maximum of five runs with a better name. The dashboard looked fine.

Changed it to nearest-rank over a hundred samples. A measurement tool is code. It can be wrong in ways that are invisible precisely because the output looks plausible.

Giving it away

This is the part I'd want a hiring manager to take, more than any number above.

For a while I was the performance testing capability. Someone reports slowness, I investigate, I produce numbers, I fix it. That works and it doesn't scale, because every performance question in the company queues behind one person's attention. The tool being good made the bottleneck worse, not better. Now there was a good tool only one person knew how to point at anything.

So I packaged it into parameterized workflows QA can run directly, exposing the two variables that change the answer: which client the run targets, and which kind of user is signed in.

The second is easy to underestimate. Performance differs wildly by who's looking. A system administrator seeing everything runs unbounded queries across the whole dataset. An administrator scoped to one company runs the same page against a tiny slice. Test only as a superuser and you learn about the worst case and nothing about the common one. Test only as a scoped user and you get the opposite. Making the actor a parameter turned "run the perf test" from a judgment call into a checklist.

Performance checks now happen because QA runs them as normal work, not because someone escalated and I went digging.

Reporting the misses

One staging run produced 171 samples across 57 endpoints, one error, six endpoints over a ten-second p95, and one page still hitting a fifty-second ceiling.

Elsewhere a page with an explicit acceptance criterion of under 1.5 seconds measured 30.0 seconds before a round of fixes and 28.1 after. Real improvement, clear failure against the requirement, and the only useful way to report it is exactly like that. Improved, still failing, here's the gap.

Performance work pulls hard toward reporting the flattering half. Resisting that is most of what makes the numbers worth anything later, because the moment a team learns your green reports are selective, all your reports become decorative.

Test the fan-out, not the endpoint. Test under concurrency. Test against production-shaped data. Audit the instrument as carefully as the thing being measured. And hand the tool to whoever will use it routinely, because a capability only you can operate is a bottleneck wearing a cape.