We recently had a case of debugging a testcase where Load Tester reported an error with a page that looked and appeared just as it should, except it was missing a link the user needed to move forward. Debugging this case was easy: examine the replay two pages prior to the error, and the server had added a red error message to the page indicating the selected criteria was not valid. So why didn’t Load Tester flag this error from the server? Beneath the error message, the page was exactly identical to the expect page: form fields were present and filled … Continue reading »

When I first started writing test cases with Load Tester, I found it easy to fall into the psychological trap of writing functional test cases.  But load testing requires a different approach, and inadequate tests can cost you time and money. Functional test cases (such as the unit tests popularized by JUnit) confirm the correctness of a system.  These tests should be highly specific and have excellent code coverage.  A good engineer approaches functional testing as though she were designing a jet engine: if any screw, flange, or circuit fails then the entire system is completely … Continue reading »

There are a few errors that are frequently reported by customers. One of them being the missing user variables error. A missing user variables error basically means that a variable that was suppose to be extracted on an earlier page was not extracted, and the user is now trying to use that variable. The error can be caused by a number of things, the most common are server connectivity issues and improper configuration of a dynamic value. If the missing user variable error was caused by server connectivity issues, there tends to be a server error before … Continue reading »

We’ve seen that modern browsers have made great strides to improve efficiency in rendering and javascript performance. But, the differences seem to be less noticeable when the network is stubborn. On a recent trip, I ran into just this problem. During the evening, the hotel’s local Wi-Fi seemed at peak capacity, the DNS servers were sluggish, and just getting connections opened often added a couple seconds. Getting onto common sites like Slashdot seemed to take 30 seconds or more from the overburdened local connection. This seemed like a golden opportunity to try out one of Opera’s new features: Opera … Continue reading »

In most cases Load Tester handles a file download during a testcase automatically. Fundamentally, there is no difference between downloading a spreadsheet or an image on the page. But in some cases, Load Tester does not automatically handle the dynamic URLs used in certain situations. Most commonly, these are cases where the file is being generated on-demand and is assigned a unique identifier that is part of the URL path, rather than a query parameter. In these cases, because there is no unique identifier that can be used to locate the value (i.e. a form field name or query parameter … Continue reading »

We are well into the development schedule for the next release of Load Tester – version 4.2. There are a number of great new features on the way as well as some enhancements to our underlying architecture that will enable Load Tester to handle far more complicated testcases in future versions. One of the features coming soon is a version of Load Tester for 64-bit Windows. Load Tester can run on 64-bit Windows, of course, but it currently runs as a 32-bit process and is subject to the same limitations as if it were running on 32-bit Windows. Testers working … Continue reading »

With Load Tester 4, we’ve made it easy to harness Amazon’s power for Load Engines quickly and inexpensively. Once you’ve familiarized yourself with the basics, here are a few rules of thumb: 1. Get to know the Amazon EC2 Console . The console will quickly tell you what instances you have running – and subsequently what you are getting billed for. Note that the console shows you only data for a selected region, so you should check each region separately. Load Tester will automatically prompt you after a test and before shutting down Load Tester if you have running engines … Continue reading »

NACON Consulting, provider of cyber security training solutions for the defense community, was aware of the concurrent user limitations of their online learning management system. In preparation for a major version release for an important client, COO Kent Leonard told his team that he wanted to see an improvement in performance, with documented proof that the system could not only handle the client’s required user load, but exceed it. “In my eighteen years of experience in IT development, I’ve found that testing is typically not as robust as it should be. I wanted to change that and do proper testing,” … Continue reading »

When a testcase has a file upload in it, Load Tester automatically handles the file upload during replays and load tests. However, in many tests each user should upload different file contents (and possibly use a different file name). Load Tester has streamlined this process to make it very easy to handle. Just follow these easy steps: 1) Gather the set of files to be uploaded 2) Import them into Load Tester (Window > Preferences > Web Performance > File Upload) and generate a dataset 3) Find the file field in the testcase using the Fields View 4) Configure the field to use the … Continue reading »

One of the common questions our customers ask is: “should we buy software and do the load testing ourselves or hire someone to do it?”. The question is one with few easy answers, but in this blog entry I’ll lead you through the thought process so you can decide which option is the best for your situation. The first question to ask is how often do you need to do load testing? If you’re planning on testing more than twice a year its usually cost effective to purchase software and learn how to use it. This is true of … Continue reading »

Once you’ve been using Load Tester for a while, it’s easy to find the minimum, maximum, and average page durations. But how does that compare against the median page duration, or percentiles? Starting with Load Tester 3.6, these measurements can now be made, but it’s been made easier to find in Load Tester 4.1. First, you will need to enable “Detailed Page Durations”. In Load Tester 4.1, simply make sure the “Detailed Page Durations” option is checked on the “Data Collection” section of the Load Test Configuration editor. With that setting enabled, just run your Load Test as normal. When examining the … Continue reading »

The Charlotte Motor Speedway is perhaps better known for NASCAR, but recently it was host to the Web Performance car in the Spec E30 NASA Firecracker Run on July 2, 2010 where our where our rookie driver Jason Tower finished third for his first podium finish! The two heat series started off on the right foot with Jason qualifying for second place, putting him on the front row for pacing before the green flag is thrown. The whole point of a spec class in the cars are as identical as possible, so this course is particularly interesting because … Continue reading »

Maybe you’ve just finished your first crack at testing, handed some results up to the development team, and are just got a note that the dev team has revamped the site in preparation for the next test. Or maybe you completed testing last month and are ready to retest the site against any regressions. Are all the testcases setup from the last round of testing still going to work? Are you going to have to create all new testcases? Here are a few quick ways to find out: 1. Run a replay with your existing testcases. If the replay fails … Continue reading »

First, this blog post is just my opinion based on my hiring experiences. Other hiring managers are sure to have other priorities, so take this advice with a grain of salt and use what you can. As a bit of background, I graduated in 1987 with an BSEE from The Ohio State University, and over the years have interviewed many job candidates, either directly hiring or giving my opinions on dozens of entry-level and senior candidates for programming, systems administration, and other positions. What this did is give me the opportunity to see correlations between how candidates presented themselves … Continue reading »

Static vs. Dynamic Web Performance Last week I attended Oreilly’s Velocity Conference in San Jose, the only conference solely dedicated to the subject of website performance and testing. It was sold out for the first time; many of the presentations were standing room only and I was unable to get a room at the hotel. One distinction that I was disappointed that was overlooked is the difference between static and dynamic web performance. All of the performance issues in the presentations I heard were all related to static web page performance, which is how long a web page takes to … Continue reading »

The website is completed or mostly completed and you have been advised to do a load test, but you are not quite sure were to begin.  First things first, a basic understanding of load testing procedure is needed.  The load testing procedure mostly involves understanding the web-application that will be tested.  Understanding the applications refers to many different factors such as: Understanding the functionality of the website. Understanding the general network layout. Understanding the software and hardware used by the system. These factors are useful for a number of reasons, not only do they make the load testing process easier, but they can help … Continue reading »

Velocity Conference Day 1 Last week I was in San Jose at Oreilly’s Velocity Conference, the only conference solely dedicated to the subject of website performance and testing. The first talk of the day was Metrics 101: What to Measure on Your Website, given by Sean Power, one of the authors of Complete Web Monitoring. He is an excellent speaker, and the talk was full of good information. These are my thoughts on the subjects he brought up, but includes quite a bit of my own extrapolations from his thoughts, and will try to make it clear … Continue reading »

The 4.1 release expands Load Tester’s cloud support by adding a third source for load generation – the Amazon EC2 USA West region. When added to the USA East and European Union regions, the combination supports generating massive amounts of load against websites in North American and Europe. If you already have one or more cloud accounts configured in Load Tester, you simply add another account, provide the account credentials and select USA West from the region selection. Chris Merrill, Chief Engineer Related Posts:Custom Report Logo in Load Tester™ 4.1Using Elastic IPs with Amazon EC2 cloud-based … Continue reading »

Load Tester is a web site load testing tool, and as such we deal primarily with the most popular Internet communications protocol: the Hypertext Transfer Protocol, or HTTP, which controls the request and transmission of web pages between browser clients and web servers.  HTTP is based on a lower-level protocol known as the Transmission Control Protocol, or TCP.    For the most part, TCP works in the background, but its proper function is critical to your website, and problems at the TCP level can show up in many different ways during a load test.  … Continue reading »

An experiment by a vendor of website accelerator appliances showed that optimization of website performance had a measurable improvement on visitor retention. Visitors to the optimized site: had a 1% lower bounce rate visited 4.5 more pages spent 27% more time on the site For the retail site they tested, the impact was felt on the bottom line: conversion rate increased 16% order value increased 5.5% For details, read the full article: Proof that speeding up websites improves online business. Chris, Chief Engineer