When a website is placed under heavy load, it can fail in a wide variety of ways. The site may become unresponsive or return an HTTP status code indicating an error (500). Or it might return a web page that is mostly complete (the headers, footers and navigation are all present) but is missing a critical part that was related to the failure. If the load test scenarios (testcases) do not validate page content well enough to catch these problems, the results could be misleading. I.e. the system could be failing but the results indicate success. In the process of helping … Continue reading »

In previous posts, I’ve talked a lot about the dangers of not accurately simulating the real-world usage. But there is another side to this story: obsessing too much over an accurate simulation can be very costly. There is one very important thing to understand about load testing: No matter how hard we strive to make tests accurate, they are still only an estimate. Your load test will never match even one single hour of production usage, not even once, no matter how hard you try. Bold statement? Perhaps. Consider this: even if you could capture the exact usage of real-world customers … Continue reading »

So your license is obsolete, what exactly does that mean?  By default a Load Tester license is not set to expire unless it is a temporary license.  An obsolete license essentially means that the license version does not match the Load Tester version number. When a license is mismatched, the following message appears: As you can see on the License Key Summary, the version code is listed as 4.2, however the current version of Load Tester I have installed is 4.3. When a license shows up as obsolete, you have two options: Match up the license version with a downgraded version … Continue reading »

As I described in my previous post on choosing test scenarios, the early tests should focus on the high-traffic scenarios. After testing and optimizing those, the next step is to identify and test the rarely-used but high-risk operations. These are operations that may have an impact on performance that is not proportional to their frequency. Even though these are not as common, they can have large impacts due to their system-wide nature and can cause sporadic performance drops during production hours that defy explanation. Examples include: Complex searches Updates that touch many joined tables at once Mass changes to data that affect many … Continue reading »

When the first few testcases are ready, the next step is to put them together into a load test. How they are combined and executed will have a big impact on the accuracy of the test. Testcase ratios – Make sure that each testcase is exercised in the correct ratio relative to the other testcases. For example, it is likely that many more people will be searching for products than buying them – the load configuration should reflect this ratio. Page ratios – Re-visit the hit rate that will be exerted on each page based on the ratio of testcases and the … Continue reading »

When you have identified the first group of scenarios to test, it is time to turn them into testcases, which is the term we use for the representation of the scenario in software. In it’s simplest form, a testcase is a list of pages to visit. In the simplest tools, it may simply be a list of URLs. With most tools, the testcase is a program generated by the tool and customized by the tester. More sophisticated tools, such as Load Tester, present the testcase visually. The testcase represents not only the page … Continue reading »

One of the key capabilities of Web Performance Load Tester is the page grouper.  This is one of those systems for which, the fewer people notice it, the better it is performing.  Essentially the page grouper takes a long list of HTTP transactions in a test case and organizes them, using timing, and/or content and referrer analysis, into a series of logical pages. If we do this well, we assume that the list of pages we see in a test script after a recording are the same as the logical pages an end user would witness in a browser — each … Continue reading »

Draft Load Tester is designed to recognize common use patterns and transform them into working test cases with a minimum of user intervention.  Our clients appreciate that Load Tester does a tremendous amount of work automatically.  But, any load testing tool must be prepared to exercise the complexity of its target, and for some web applications even a simple test case can involve hundreds of variables. This post will try to elucidate the stages of the recording process, which are, roughly: Recording Initial Inspection Automatic Configuration Manual Configuration Replay Validation During the recording stage, we manually perform a use case in a web browser.  Load Tester monitors our work … Continue reading »

Society for Human Resources Management website able to service its 250,000 members while introducing potential savings of $200,000. Durham, NC – February 19, 2009 – The new Society for Human Resources Management (SHRM) website was recently given a huge capacity boost in time for its unveiling, enabling the site to handle large amounts of traffic from its 250,000 members. Web Performance, Inc., (WPI) developers of industry-leading Web Performance Load TesterTM software, increased the site’s capacity from 100 to 2000 users, and reduced server requirements to less than 25% CPU utilization, eliminating two servers altogether and reducing potential software license expenses by … Continue reading »

Functional test suites do NOT make good load tests! I know it can be tempting to take your suite of functional tests and use that as the basis for your load test. This is especially true if your testing tool supports both functional and load testing. But functional test suites have an inherently different purpose than performance test suites – and as a result, make poor load tests. The purpose of functional testing is to determine if the site works correctly. As every good tester knows, this means testing the edge cases. What happens when the user does something unexpected? What … Continue reading »

Functional test suites do NOT make good load tests! Functional test suites have an inherently different purpose than performance test suites. It is always tempting to take your suite of functional tests and use that as the basis for your load test. This is especially true if your testing tool supports both functional and load testing. The purpose of functional testing is to determine if the site works correctly. As every good tester knows, this means testing the edge cases. What happens when the user does something unexpected? What if I complete the “How many children do you have?” field by … Continue reading »

In Load Tester 4.2 we added a new fields view with vastly expanded options to configure each field.  In Load Tester 4.3 we added support for JSON as well as ad-hoc custom regular expression and name-value delimited parsers.  In Load Tester 5, we are adding support for XML automation.  Hierarchical XML data structures that appear in form fields or as HTTP POST content will appear in the fields view, and Load Tester’s application state management (ASM) tool will automatically assign any XML value or attribute for which it can identify an appropriate data source. Each XML field will be named after … Continue reading »