When the load on any application climbs to a certain point the slowest part of the transaction path will limit application throughput. Response time will increase. Users will be unhappy.
Long before computers, two chemists named Sprengel and Liebig were working in the area of agricultural chemistry. They developed the Law of the Minimum, which states: Growth is limited by the least available resource. If a plant is starving for nitrogen, then only additional nitrogen will help. Everything else you give it has no effect. Oddly enough the Law of the Minimum applies to computers, too.
When the load is high, all transactions race through the well provisioned parts of the system only to find an ever-increasing delay as they wait for slowest part of the transaction path. There are lots of potential bottleneck points in any transaction path, but only one at a time is your current bottleneck.
The double-necked hourglass illustration below shows a bottleneck at point A. Beyond that point life is easy for the rest of the system as, no matter how many transactions arrive; the workload is throttled by the upstream bottleneck.
When you find and fix bottleneck A, then performance will be really good for only about 45 milliseconds until that great load of transactions hits bottleneck B with a sickening “WHUMP!” The throughput of this system will hardly change at all, and you will have some explaining to do.
When capacity planning, it is important to explain this drawing to the decision makers so they comprehend how one bottleneck can hide a downstream bottleneck. It is also important for you to meter all the resources, not just the one that is the obvious bottleneck.
If you’ve ever watched little kids play soccer, you know they tend to clump around the moving ball. People new to performance tend to focus on the obvious bottleneck and ignore everything else. If the hourglass had meters at A and B they would likely show bottleneck A to be 99% busy and bottleneck B to be 60% busy. If you needed the hourglass to process three time the transactions as it is doing now then take those meters, do a little simple multiplication (60% * 3 = 180% busy) and notice that both A and B need work if you are going to hit your throughput goals.
The hourglass could easily be drawn with many more bottlenecks, but I’ve never seen a performance problem where there were more than two bottlenecks that had to be cleared up to get the needed throughput. If you are working on the fourth bottleneck, then you are most likely deep in the weeds and I’d advise you to get some help.
Bob Wescott is the author of “The Every Computer Performance Book”.
Related Links:
More information on this, and many other useful ideas, can be found in Wescott's book:
The Every Computer Performance Book
Bob Wescott, Author of "The Every Computer Performance Book" Joins the BSM Blog
