If you take the right metrics, you can plot a good-enough approximation from your production system. In general, the keys are “measure percent busy, not queue depth” and “measure close”.

For instance: Don’t (just) measure how long it takes for the database to respond to an average query. That includes network, disk, and DB server time, so it’s not “close”. And it will take a lot longer with one request ahead of you than with none, so you’re measuring queue depth, too.

Instead, measure the percent of time your DB server is busy (i.e. the inverse of its idle time). Do the same with its disks, and CPU, and network card.

You’ll pretty quickly see patterns. Think back to shared Ethernet: It doesn’t get 100% utilized, ever. But if you look at what percent of time the network’s available, you’ll see a pretty sharp curve as the utilization goes up. Once you’ve found the knee of the curve, you know what won’t scale.

If you don’t know for certain that it’s linear or better, it’s very likely to be curved upward. Any single points of failure from a high availability standpoint (one DB server, one firewall, etc.) will probably be performance bottlenecks also.

But the graph goes to +infinity on both axes, so it’s not really practical to try and map it all out with the results from your current architecture. Fortunately, as I mentioned near the end of the article, there’s stuff that you can do later to improve scalability, so the whole curve is subject to change shape over time. You’re usually not stuck with the same architecture forever.

For relatively small changes in load (say, 2x-5x), you can approach it like an engineer would. Set a goal for a certain peak load that you’d like to be able to handle, and do synthetic tests (with load generating tools) to make sure that you can get there (plus some margin of safety above that goal) safely. Then put throttling and/or a “high traffic” mode in place if you get vastly more traffic, i.e. if you get Slashdotted/Dugg.

Then you can use the published experiences of other folks to put together a roadmap for getting your own architecture to scale up to the next few big milestones you want to hit (10x, 100x, etc.). That plan will give you estimates on what your curve will look like: buy this and do this work, and we can support 10x as much traffic; buy these and do this additional work and we can grow by another 10x, etc.

The sooner you have such a plan, the sooner you can see what parts of your architecture will need to be replaced in the short term, and so you can hold back on your investments in the things you know you won’t be using for long.