For most who read a software testing blog, web access is a given — it’s always on, always up, usually fast, and even available on-the-go (as long as you remember to bring your Nexus One, Curve, iPhone, etc).

But not too long ago, the web was still in early adopter mode.  It was available (maybe) after you fired up that block you called a desktop computer; and after you endured the sound of your dial-up connection; and only if you exhibited zen-like patience with pop-ups and page load times.

Why am I taking this trip down memory lane?  Well, it could be because I saw the extended trailer for Hot Tub Time Machine (destined to be a classic, but NSFW).  More likely, however, is the fact that yesterday I read a couple of interesting pieces from Mashable & the BBC — about the global adoption of the Internet in the past decade, and the provocative question of whether or not web access is an inalienable human right in this day and age.  Both are worth checking out, if for no other reason than to make us appreciate what we have.

And since we have a global community of software testers, I’m extremely interested to hear what the software-savvy readers from every corner of the globe have to say about this very cool interactive map from the BBC.  Does this fit with your experience in your home countries?  What do you think this chart looks like in 2012?  2020?  Share your thoughts in the comments.

Now if you’ll excuse me, I have to go complain to the barista that the wi-fi in this Starbucks is taking way too long to download songs from iTunes and ripped files from BitTorrent, while I watch 30 Rock on Hulu.

Well, today he’s sticking close to home in Boston. Tomorrow he’ll land in London… and before the week is out, he’ll hit Tel Aviv.

Doron starts Wednesday morning off (after his usual 10-mile run, of course!) in London with some tea and networking with friend and colleague, James Whittaker and UK partner, TCL.

Then he’s off to QCon London, an excellent conference for the enterprise software community. On Friday, 3/12 @ 2pm, he’ll be presenting at QCon re: The Mobile App Quality Challenge & How Crowdsourcing Can Help.

Doron is one of five software testing leaders chosen to present in the “How Do You Test That?” track. This track explores unique solutions created to address situations in which automated testing does not suffice.

And on the last leg of his marathon journey, Doron will present at Garage Geeks in Israel on Monday, 3/15 @ 8pm. There, Doron will be taking a deep dive into the topic of Crowdsourcing, and how smart recruiting, training and incentives can turn an unstructured, loosely assembled mob into a unified, professional community.

So, where in the world is Doron this week?  Catch him if you can!

Developers have long awaited the death of Internet Explorer 6; web heavyweight like Google, Facebook, Reddit, Justin.tv and Digg have all announced the expiration date for their support of IE6; Microsoft has been steering users away from IE6 for more than a year.  And last week, a funeral was held for the outdated browser which was two parts wake and one part wish.  Even Microsoft joined in the fun, sending a card to the festivities services.

So what will it take to kill the undead browser once and for all?  Well, it’s worth noting — and shocking — that IE6 still drives nearly 20 percent of all web access from beyond the grave.

How is this possible?  What outdated luddite segment of web users is still stuck in 2001?  Well, the prime culprit is large enterprises like Intel who bemoan the cost and complexity of upgrading thousands of employees and legacy apps that were built specifically for IE6.  So while the web citizenry has moved on and is ready to pull the plug, developers (and testers), IE6 will continue to be part of the web app testing matrix for much longer than any of us would like to believe.

Just to further illustrate the insanity of IE6’s continued survival, here are a few other things that were going on in 2001:

• Popular movies from that year were Shrek, Ocean’s Eleven, and A Beautiful Mind
• Top songs included “Survivor” (Destiny’s Child), “Yellow” (Coldplay), “I’m Like A Bird” (Nellie Furtado)
• Facebook co-founder Mark Zuckerburg was a 17-year old high school student
• And for you basketball fans, Latrell Sprewell, Allan Houston and Vlade Divac were NBA all-stars

Does anyone have any IE6 horror stories (as a user, a developer or a tester) that they’d like to share?  Anyone still willingly use IE6?  Anyone’s company still force them to use it?  Admitting that we have a problem is half the battle.

When it comes to software development and programming, few people have been read, linked to, tweeted, quoted or plagiarized more than Joel Spolsky (@spolsky). But despite his adoring fans, the widely known blogger and entrepreneur has decided to give up the former (his wildly popular blog) to focus on the latter (his growing business).

Joel’s final farewell – Let’s Take This Offline – appeared on Inc.com a few days ago, where he discussed the fallacy of blogging as business strategy, time commitment and the common mistakes of most company blogs. Of course, he also addressed his reasons for “retiring”:

So, having become an Internet celebrity in the narrow, niche world of programming, I’ve decided that it’s time to retire from blogging. March 17, the 10th anniversary of Joel on Software, will mark my last major post. This also will be my last column for Inc. For the most part, I will also quit podcasting and public speaking. Twitter? “Awful, evil, must die, CB radio, sorry with only 140 chars I can’t tell you why.”

The truth is, as much as I’ve enjoyed it, blogging has become increasingly impossible to do the way I want to as Fog Creek has become a larger company. We now have 32 employees and at least six substantial product lines. We have so many customers that I can’t always write freely without inadvertently insulting one of them. And my daily duties now take so much time that it has become a major effort to post something thoughtful even once or twice a month.

Hopefully, this retirement will be of the Brett Favre variety. If not, here are a few of his memorable posts on software testing, in tribute.

• Why Testers?: Joel explains why testers should be a part of any software organization, and offer tips as to what qualities make some testers better than others. (Hint: a background in programming is not one of them).
• Top Twelve Tips for Running a Beta Test: Open betas don’t work and they can’t be completed in under ten weeks. These are just a few of his tips for running beta tests.
• Top Five (Wrong) Reasons You Don’t Have Testers: There’s a lot you don’t know about why you don’t have testers. Confused? You should be. You better read this article.
• Painless Bug Tracking: “One of the biggest incorrect facts that programmers consistently seem to believe is that they can remember all their bugs or keep them on post-it notes.”

Good luck Joel. Your views and insights will be missed.

Self improvement is a lousy business model. Mobile app developers understand this better than most. For every app to help you lose weight or improve your IQ, there are basically 10x as many to help you drink more, find your nearest trans-fat vendor or change the channel without standing up to get the remote. What a world we live in!

But if sloth and gluttony aren’t your thing, you can rest easy knowing that your vices have also been covered. And so to illustrate, I’ve posted an app for each of the seven sins.

Gluttony: “Happy Hours, is a free application for the  iPhone,  Android, and the mobile web. With it, you get access to some 15,000 happy hours in 30 different cities around the country. You simply load the app up, tell it where you are (which it can know automatically on the iPhone and Android phones), and let it show you happy hours close by.” (from the washingtonpost.com)

Lust: Girl Zoomer – “This application turns your iPhone camera into a pair of binoculars with 4x zoom, so you can see “the details that other people can only furtively glance at.” (from reuters.com)

Anger: “Boston now lets angry citizens file complaints about potholes, graffiti and missed trash pick-ups via iPhone. Boston’s Citizens Connect, which city officials say has been downloaded 5,000 times since it’s October 2009 debut, won’t be the only way people can let city government know what’s awry in their fair city. The Cradle of Liberty aims to be the city of smartphone apps thanks to a new one called Boston Urban Mechanic Profiler, or BUMP.” (from cultofmac.com)

Pride: Birth Buddy – “You’re about to give birth. So pull out your iPhone, fire up this paid-for app, and track your contractions with Birth Buddy. No. You’re giving birth. Put the damn phone down. I pity the kid already.” (from itpro.uk)

Envy: iEnvy – This application lets you upload pictures of yourself for others to rate based on your “envy-ability.” Of course, you can also rate the envy-ability of others and share the results on Twitter, where else? (from teameros.com)

Greed: E*Trade Mobile Pro – “Receiving huge praise from the Apple community, E*Trade Mobile Pro gives members and non-member alike access to streaming quotes, charts, and watch list. Obviously, E*Trade members can execute live trades using this application.” (from speakstocks.com)

Sloth: “Comcast Corp has added a new feature to its Apple iPhone and iTouch application that allows users to program their set-top box digital video recorder remotely.” (from homemediamagazine.com)

With thousands of sinful apps now available, surely I have missed some good ones (pride is not a problem for me). So if you have any to add – and I hope you do – the comment section is all yours.

We usually like to keep things pretty light around here.  But this post is a public service announcement of the most urgent nature.  I don’t want to alarm anyone, and I’m not prone to exaggeration, but clearly software apps are rising up for the coming war against the humans.

First it was our cars (and then more cars); then it was unmanned aircraft.  But now, it’s gotten serious – because now the software uprising of 2010 is messing with our games.

Nick Saint (@ncsaint) over at Business Insider describes just how bad things have gotten in this latest battle between man and machine:

Owners of older models of Sony’s PS3 have been afflicted by a bug in the system’s internal clock. Unless you have a PS3 Slim, leave your machine off until word comes down that the bug has been fixed, or risk permanently losing data.

What’s next — our Foreman grills?  Our laser pointers?  Our lava lamps?  So consider this a call-to-arms for all who develop and test software.  The war is on.  And lately, the software (and its well-hidden bugs) are winning.  Izzy Mandelbaum was right:  It’s go time here, people!

These days, it’s hard not to know about Lady Gaga (and if you don’t know her, here’s her latest music video to get you started).  She’s become one of the hottest pop acts in the world, all by combining music, fashion, and a little bit of “Andy” (either Warhol or Kaufman – take your pick).  So what does this have to do with software?  Well get this: an astonishing 89% of the people who create an account on LadyGaga.com choose to do so using third party authentication from Facebook, Twitter, or Myspace.

Just think about that for a second.  That means that nearly 9 out of 10 people creating special purpose web accounts are doing so using their social networking platforms (skipping all those annoying new account questions like password, age, location, and favorite pet in the process).

This is huge, and it represents a big shift in the way people are going to interact with your website in the future.  If 89% of users are doing this on a mass market website like LadyGaga.com, then I guarantee they’re going to be doing it on other sites as well.  So what does this mean?

This presentation (PDF) from Brian Ellin of JanRain, the company that developed the authentication for Ms. Gaga’s site, provides some great answers and ideas.  For example, good services like Facebook or Twitter will have already verified the user’s email address, meaning you won’t need to do that again. They also handle the user’s passwords, take care of tracking all 15 of their email addresses, and know all that extra information about their age and location.

Third party authentication can also simplify joining a site.  By using simple buttons and small popup windows to complete the login process, you can avoid taking the user off the site and away from the action.  In many cases, once a new user has logged in for the first time, their future logins can be completed with the click of a single button.

With all that in mind, these fancy new authentication systems deserve careful scrutiny by security experts and software testers.  Most of these third party authentication systems have a good reputation, and there’s no reason to believe otherwise.  However, good testing is always essential.  Here are a few things for testers to consider:

• Does the site log you in correctly?
• Does it let you logout or disconnect?
• Does the site adhere to the social network’s privacy guidelines?  In other words, will the site spam your friends, fill your wall with garbage, and take your personal information without your consent?
• What happens if you login with multiple systems (eg. Facebook and Twitter at the same time)?
• Can you manage your settings for things like email notifications and reminders the same way as if you had created an account the old fashioned way?
• Does the site offer a way to merge accounts?  Does it work?
• How often does the site post things to your social network, if at all?

Have you used third party authentication?  I would be interested to hear what you think about it, either from a development or testing point of view.

SCMagazine reported this week that researchers in Malta have discovered a decade-old vulnerability, present in all versions of Windows since 2000.  This bug can cause PCs to crash instantaneously and without warning, as well as reeling the compromised machine into a distributed denial-of-service (DDoS) attack.  This exploit is only dangerous if the user is duped into running an app with the malicious code (according to Paul Gafa, CTO of 2X Software).

The bug was discovered while Gafa was writing a software testing app:

“You can be the least privileged user on the system and still crash it,” Gafa said. “I believe it is very easy for Microsoft to sort it out. They just need to validate arguments passed to Windows APIs.” (source: SC Magazine)

Microsoft is currently aware of the defect and responded with this insight:

“Our initial assessment of the report is that malicious code would have to already be running or a user would have to be able to run a specially crafted application to cause the system to crash. In either case, the system has already been compromised or the user has rights to logon to the system.”

I’m curious to hear if anyone has other stories of old bugs causing new problems or vulnerabilities?

Retired NASA Astronaut Mike Mullane* (pictured left) said it best when he asked: “Why is there never time to do it right, but always time to do it over?” He could have easily been talking about the recent problems Toyota has been dealing with, but he wasn’t. He was talking about today’s software companies.

Conversely, this recent article from The Economist could just as well be about today’s software companies, but it isn’t. It is about Toyota’s recent problems.

Like everyone else, the author wants to know how the auto giant could so quickly lose its reputation for safety and quality (things that can happen to ANY company if they are not careful). The culprit? You guessed it: software bugs.

Instead (of trying to keep pace with competitors), two recent trends, both software related, hint at the reason behind Toyota’s unexpected decline. One is the shortening of product-development cycles generally in the car industry. These are down from a typical four or five years to little more than 15 months, thanks to computer-aided design and manufacturing, and the virtual simulation of the resulting products. To save money and time, Toyota has even dispensed on occasion with building test “mules” and other engineering prototypes.

The other trend is the wholesale replacement of mechanical components with electronic controls. It started with ignition systems, then spread to air-conditioning, cruise-control, engine-management, throttle linkages, transmissions, and now the steering and braking systems. Drive-by-wire is not cheap, but it reduces the number of components needed to do the job. It also allows them to do extra things as well as to compensate for wear and changes in driving style and road conditions.

But software is not hardware, and software “engineers”, despite their appropriation of the name, are a different breed from the sort that bash metal. Programming digital controllers is not one of Toyota’s core competences. Even with the most diligent of testing, bugs will always find their way into software. Right now, it seems Toyota is learning that lesson the hard way.

But Toyota isn’t the only one learning lessons the hard way. As it turns out, the NHTSA does not have one single software engineer on the payroll to analyze the Toyota situation properly. CarConnection.com puts this into context:

If it cannot properly analyze those systems, or even understand at a deep-code level how they work, then the agency is useless at overseeing the entire “Safety” part of its mandate.

The agency has an annual budget of more than $800 million, and it employs 635 thousands of people. That not a single one of them is an EE or software engineer borders on the criminally insane.

So what have we learned from all this? Basically, that it’s okay to launch on extremely short development cycles if – and only if – the product has been tested extensively beforehand. Also, since there is no safety net, you are on your own when it comes to ensuring a quality. No one else can do it for you.

*By the way, Mike Mullane – a terrific writer and lecturer -  was the keynote speaker at STPCon this past Fall.  For a primer on his career and general philosophy, you should watch his appearance on The Daily Show.

In the 18 months since our August 2008 launch, the name uTest has become synonymous with functional testing.  We help companies hunt down and kill the bugs in every corner of their web, desktop or mobile apps.  But a funny thing happened along the way:  as our customers have grown (in number, in size, in technical sophistication), we’ve found ourselves getting pulled into QA-related conversations outside of just functional testing.

Among the most popular topics has been load & performance testing.  Companies of all shapes and sizes have been asking for our advice; asking our opinions about various synthetic load tools; asking us what other companies are using; and ultimately, asking us to help them ensure their web apps are ready to perform under peak loads.

So after extensive research and a great deal of planning, uTest is ready to announce a new and better way to perform load testing on your web app.  We’re offering three different flavors of load testing services:

• Live Load: A team of live testers from around the globe can test an application simultaneously, enabling customers to see how their web app performs under truly real-world usage conditions
• Simulated Load: Requiring no live testers, uTest’s simulated load testing provides customers with a complete analysis of a web app’s performance under peak synthetic load
• Hybrid Load: Combining live testers with best-of-breed simulated load tools, uTest’s hybrid load testing enables customers to perform functional testing while their web application is under heavy synthetic load

We think our approach to load testing is altogether unique and will be extremely valuable to companies of all types, but we’re also exceedingly biased.  But our early load testing customers and the software testing pundits seem to agree:

• SearchSoftwareQuality from TechTarget:  Crowdsource specialist uTest launching new performance, load test offerings by Yvette Francino (@yvettef)
• StickyMinds:  uTest Launches Load & Performance Testing Services

We’ll update this post with more links as the news rolls in.  Questions about how load testing works via a community of professional testers?  Check out our load testing section for details.  Or drop us a note and ask us anything.

In the third and final installment of our Testing the Limits interview with Google’s Patrick Copeland, we get him to share his advice for youngsters who want to reach tech executive status; his programming language of choice; mobile testing challenges and his New Year’s resolutions (for testing of course). In case you missed our previous interviews, here’s Part I and Part II.

uTest: What advice would you give to young software turks who aspire to become a senior tech exec at a global powerhouse?

PC: Here’s my advice:

1. Be technical: At Google all of our engineering execs have very strong technical backgrounds. They were programmers and many of them can – and do – still program today. Advice: remember your computer science and stay sharp.
2. Be a connector and innovator: One exec commented that his job could be described as being “a very expensive email router.” Get connecting within your company and establish a network of strong peers.
3. Drive your career: Don’t wait for someone to recognize you. Inform people about your plans and successes. You are your own best billboard. People will create a myth about you, like it or not. If you are smart you’ll help them create the myth you want them to have.
4. Play to your strengths: Pick projects that you are passionate about and where you can rock worlds.
5. Bite the hand that feeds you (a little): Don’t be a jerk, but make sure you don’t go passively along with bad ideas. A weak willed tester is a bad thing. Make your concerns known…and know when to fold.
6. Go where there’s opportunity for growth: Not all projects or companies can offer you an ability to grow. Move when you think you are topping out. The biggest mistake I’ve seen is staying too long in a place that hinders growth.

uTest: Google is booming in the mobile apps space; what QA testing challenges do mobile apps present that web or desktop apps do not?

PC: Where do I begin?

A good starting place is the huge diversity of devices. For instance, significant variations in HW and SW: OS, memory, processors, screen size, and input methods. Just having access to a fraction of the devices on the market for testing purposes is a challenge.

On top of that, if you want any automation, you have the problem of simulating input and capturing the actual output on the device. We needed to develop tools that allow up to test and build against multiple platforms. Normalizing the testing is tough. We debated record and playback vs writing abstraction layers vs other methods. All can be brittle in the extreme. We also needed to think about the long term cost of maintaining tests suites on fast moving SUTs.

A couple other issues are the carrier networks and the numerous languages. As I said, I could go on for a long time, but I believe I hit some of the major pain points. Even with all of the automation in the world, a community based test approach can be complimentary.

uTest: Is there a particular programming language to which you are loyal (PHP, Java, Ruby, etc)?

PC: Nope. Not really. At Google the most common ones are Java, JavaScript, Python, Perl, and C++. Perhaps a personal failing, but I still think in C and my code ends up looking kind of C-ish even if it’s in another language. My preference also depends on the thing I’m trying to do, obviously each has strengths and weaknesses.

On a similar topic, I recently met famed computer language innovator, Niklaus Wirth at the last GTAC conference in Zurich. He had in interesting comment on testing and quality, “The experience, judgment, and intuition of programmers who have survived the rigors of testing are what make programs of the present day useful, efficient, and correct.” In other words, you can’t test quality in and quality comes from experience.

uTest: Any new year’s resolutions for Google’s testing & QA efforts?

PC: Yes. Every year I write up a “one pager” describing what I think we need to do. I try to stay focused on general productivity and not specifically testing. Testing is one tool among many we use to make software high quality and delivery faster. We are doing well on many levels, but we aspire to do even better. Here are a few of the high level bullets:

• Greater release velocity. We apply appropriate use of automation, tools, process, and information that solve Google challenges. We introduce and use metrics that fit naturally into the product cycle. We are introducing key metrics that are actionable and representative of the health of the project that allow team members to understand issues and take action quickly.
• Cohesive tools. We’ve built an impressive set of tools that have made significant impact. In our next stage, we need to better integrate some of the data and tools and focus on accelerating the end to end developer workflow. This includes identifying bottle necks that slow people down and keeping an eye on innovative ideas coming from product teams. And of course, continual improvement of build and test speed.
• Driving improvement upstream. We will take advantage of our ability to make good ideas useful across the company. We use our broad view of projects to push adoption of ideas and methodologies that work. We look for ways to compliment existing product strategy by identifying changes where the adoption is in a project’s self interest. We constantly evaluate our own practices by looking at their effectiveness in relation to customer and business needs.

uTest: You went to school at University of Arizona, and USC; you’ve worked at Microsoft and Google; are you a lifetime west coaster or has it just worked out that way? Also, Dodgers or Giants (or D-backs)?

PC: Are there any good East Coast teams? I heard it was so foggy the other day that the Washington Nationals couldn’t even see who was beating them.

They may not be sports teams, but I’m a fan of the testing teams we have on the east coast: NYC “Bubble Sorts”, Boston “Red SOX”, and Pittsburgh “Open Sourcers”. BTW, we just hired a Test Director on the East Coast named John Turek who’s looking for a few good players.

BTW, I actually have lived in other places. My family lived in Maryland and Tallahassee for a bit. I also lived in Madagascar, which is definably on the East Coast (of Africa).

uTest: What sites, blogs or magazines do you read to stay ahead of the curve in the areas of business, technology, etc?

PC: Track and Field, Cosmo and, occasionally, Monster Truck Weekly . Seriously, I have a bunch of custom news feeds on certain topics, like Google, our competitors, and testing. I also read wsj on line and at least one interesting industry technical paper per quarter.

Editor’s note: We hoped you’ve enjoyed our interview with Patrick Copeland. If you have suggestions for our future Testing the Limits guests, we’d love to hear them.

In Part II of our interview with Google’s Patrick Copeland, we discuss the challenges of managing a global engineering team; rewarding developers with food pellets; the difference between a good tester and a great tester; and why some companies will never launch a high-quality app. By the way, did you miss Part I of our interview?

uTest: What are some of the challenges that come with managing teams in dozen (or more) countries, as you’re currently doing? How difficult is it to maintain control over the people, processes and products? And when do you sleep?

PC: “Maintaining control over people” <smiling and laughing like Dr. Evil>.

But that’s not how it works at Google. The truth is…our team structure is atypical in the industry. For one, we are a flat company with many Nooglers being a few steps below senior executives. The expectation is that people and teams are semi-autonomous. In this model it’s impractical for managers to be controllers. And regardless, I’d rather set up teams that are made of great people who can run their areas themselves. My focus is on helping teams to be effective. Managers at Google are generally judged on their ability to enable smart people to get things done. Many have 15 or more direct reports, introducing some chaos and reducing the time available to micromanage.

One way we get everyone moving in a similar direction is to use OKRs, it came to Google thanks to board member John Doerr back in 2000. John stressed the importance of setting overall company Objectives and Key Results that would help develop departmental objectives; in turn, individual OKRs for every employee would support achievement of team and company wide goals. In Q1 of 2000, we rolled out our first company-wide OKRs, which included “8 million searches/day” and “Select CEO.” We’ve come a long way since then.

uTest: A lot’s been made of the unique and friendly work environment Google offers its employees. Does this also apply to your engineers? Or are they handcuffed to their desks and given food pellets for every line of code written (like we do at uTest)? Seriously though, how does an open atmosphere lend itself to better software?

PC: We have a pretty open culture where engineers have a good degree of freedom to invest in areas that interest them. We have an idea called “20% time” where engineers are encourage to invest a portion of their time in projects outside of their primary area.

Does culture help to make better software? I think so, but it’s a tough thing to quantify. On occasion the idea of collecting and measuring “productivity metrics” has come up (ex. lines of code, # of check-ins). We uniformly resist this because similar metrics can result in undesirable outcomes as people try to “succeed” in the system. Our biggest measure of performance, besides the velocity of innovation being delivered to customers, is quarterly peer reviews. This type of peer system reinforces ideas like teamwork, making sure you are having an impact, and building respect from peers. It’s more subjective, but harder to “game” and, ultimately, much more valuable in reflecting the real value and contribution of an individual.

BTW, “Food-pellets” wouldn’t work because our engineers already have free access to gourmet food.

uTest: What’s the difference between a good tester and a great tester?

PC: Good testers can be trained. Among many others, the basics you need to be good are: depth in computer fundamentals, an understanding of the application domain, an strong understanding of the use case, empathy of the user, mastery of metrics, and a focus on the development process.

Great testers, meaning the 90th percentile, are rare and special. Not everyone can be truly great. In my experience great developers do not always make great testers, but great testers (who also have strong design skills) can make great developers. It’s a mindset and a passion. From the 100s of interviews I’ve done, “great” boils down to: 1) a special predisposition to finding problems and 2) a passion for testing to go along with that predisposition. In other words, they love testing and they are good at it. They also appreciate that the challenges of testing are, more often than not, equal or greater than the challenges of programming. A great “career” tester with the testing gene and the right attitude will always be able to find a job. They are gold.

uTest: What’s the greatest barrier to designing, developing and launching high-quality products on a consistent basis (deadlines, prioritizing, competing visions, etc)? Said different, why can’t every company launch world-class apps every time?

PC: Every product I’ve worked on required a unique approach. There are so many variables in play and most of them are situational. Some of them you can control almost completely (e.g. technology choices). Many others you can control somewhat. Others still are completely outside of your control (ex. what the competition decides to do).

Some companies try to normalize their development process. One thread common to formal development models are that they focus on a few of the many variables: improving efficiency, predictable process, estimation of quality, or others. Process-heavy development models may work well for manufacturing airplanes and have been successfully applied by some companies, they have been viewed by many developers as burdensome and contrary to the creative nature of writing innovative software. Conversely, “process-less process”, can lead to a heroic culture that’s unable to repeatedly deliver. There needs to be balance.

Consider the physics of flight as an analogy to software process. In addition to reasonable flying conditions and an experienced pilot, the key to getting airborne is having an appropriate balance of factors that match the situation: too much weight or too little thrust can be disastrous depending on the situation. Similarly, teams, products and process all have virtual physics. For instance, adding engineers late in a product cycle doesn’t necessarily provide more lift. Adopting a new process may give a team more thrust momentarily, but may also incur a longer term drag that makes them incapable of innovation.

The popularity of Agile, while not a wholesale rejection of more rigid processes, indicates that developers desire more balance and creativity. Whatever we do to make software higher quality and more predictable to build, we must maintain a balance that encourages the innovative aspects of the art form. We need to motivate smart minds to solve hard problems and deliver rich features to customers. In other words, we need to be adaptable to stay airborne for the long term.

Editor’s note: Part III of our interview with Patrick will be posted tomorrow, so be sure to check back in.

In this month’s Testing the Limits interview, we’ll put Patrick Copeland on the hot seat. Patrick is the Senior Engineering Director for a promising young upstart named Google (we’re not familiar with them ourselves, but we’ve heard good things) where he oversees a global team of about 800 engineers. But this isn’t his first rodeo –  prior to Google, Patrick spent a decade at Microsoft, where he specialized in all things related to software engineering.

So what do you ask someone who’s probably forgotten more about software than we’ll ever know? Well, in this installment, we’re going to get his views on catering to a global base of users; his criteria for evaluating testers based on their “tester DNA”; the recent addition of our good friend James Whittaker; the challenges of launching new products like the Nexus One, as well as other tidbits from inside the GooglePlex. Stay tuned for Parts II and III in the days ahead.

uTest: What are some of the challenges that come with having a global base of customers and users? Are certain products noticeably more popular in some areas rather than others? And how does this affect your future planning?

PC: Yes, of course some products and features do better than others. Our approach is to do lots of experimentation and to release and iterate. We push bits to customers early and often, and then we listen and watch usage. Customers help us by “voting with their feet.” Popular features and products are improved, and poorly performing products are deprecated. With a big focus on innovation, we also need to “fail fast” and customer feedback helps us make those decisions.

Not surprising, our global customers have different demands of our products. We want products to “feel local” and we need to support features that may be unique to specific markets. For instance, in Indic based languages using a standard keyboard is difficult, so we develop strategies like virtual keyboards or category browsing for search. As we specialize our products for certain markets, it introduces more challenges for testing (eg. requiring special cultural knowledge). When we can’t find internal talent, community-based testing is an interesting solution to this challenge.

We base staffing and planning decisions on several criteria:

• Strategic: Maybe a new feature, but in a market with existing competition (like Android).
• Financial: Obviously Ads and Search, but we have several emerging businesses that are also getting important.
• Customer usage: For example, popular high-traffic applications like GMail.
• Legal or Compliance: Certain areas need to be prioritized high for legal reasons. For example, SOX compliance for CheckOut.
• Ability to Impact: We look at our capability and decide if investing testers in an area would have a significant impact.

uTest: A few years back, you were the keynote speaker at GTAC, where you said something to the effect that “the longer I’ve been in the business, the less I know about it.” How important is it for testers and developers (and those who manage them) to maintain this student-for-life mindset?

PC: Very. When I hire people I look for folks with a “testing DNA.” These are people who are great computer scientists at their core, but also are very curious, love software, and are passionate about test engineering. People who have those characteristics tend to pursue challenges and continue to learn.

By the way, what I meant is that the surface area of the challenges we face in software has grown exponentially over time. Just in the past few years we’ve witnessed a paradigm shift to cloud computing that stretches our imagination and challenges the limits of software. Our process for developing that software is going through an equally dramatic revolution. We are reconsidering the appropriateness of the lessons we’ve taken from traditional software development companies. Testing teams need to continue to evolve quickly and innovate just keep up with what’s happening in the industry.

uTest: Not too long ago, you added testing guru and author, James Whittaker to the Google team, who is well-known to the uTest community. Why was he such an important addition, and has there been anything that’s surprised you since he came on board?

PC: James has been a great addition to the team. His passion for testing is incredible. The biggest surprise is that he’s having fun being a manager. James accepted our offer without knowing exactly what he was going to do. This is a pretty common situation for new senior people at Google. I remember giving him our offer and he said repeatedly, “I’m not interested in being a manager.” I kept saying, “ok, ok, no manager position for you.” But he’s decided to run the Kirkland and Seattle Test Teams, in addition to several groups in Mountain View. He loves it. I think that might even be a surprise even to him.

uTest: Google seems to break into another market every week, with the new Nexus One phone being the most recent example. How much of your time is spent looking forward to new apps and categories, as opposed to concentrating on existing products?

PC: At Google we have a 70-20-10 rule for investing in technology. While the numbers are not carved in stone, the basic concept is that we invest ~70% on our existing core products, ~20% in related but new categories and ~10% on riskier bets or areas outside our existing core competencies.

Innovation is a high order bit of course. We’re having to do a lot of innovation on supportive infrastructure to keep up with the pace. For instance, we’ve created an infrastructure that allows us to innovate very rapidly while maintaining high quality despite a huge and complex code base. On any given day, there are thousands of projects under development with over 5000 CPU cores producing 65K compilations of build targets, and over 100M executed test suites. The build and test system accommodates the load, and an annual growth rate of 75%, using distributed execution, caching, and work avoidance techniques. The average build and test cycle take just 4 minutes and we are quickly moving toward a model where developers can receive results before they formally check-in code changes (‘pre-submit checking’).

In our model, product teams maintain a build that’s always green (compiling and testing without error). We built the supporting system based on three priorities:

• Speed: All test and analysis systems need to return results very fast. If it takes too long, engineers will either ignore or not bother looking for that data.
• Feedback: The focus of test systems must be on high quality feedback. We want engineers to keep code at production quality at all times, not fixing code that is broken.
• Simplicity: Engineers should not have to understand how the underlying systems work. All data and feedback must be easy to understand, integrated into commonly-used productivity tools, and presented in a workflow that allows them to take appropriate action.

Thanks for reading Part I of our sit-down with Google’s Patrick Copeland. Check back Tuesday for Part II of our interview, where we’ll cover the challenges of managing a global engineering team; rewarding developers with food pellets; the difference between a good tester and a great tester, and more.

So far, our Guest Blogger series has demonstrated the incredible domain expertise of our community – and this post will be no exception. With us this month is uTester David Vydra. A resident of San Mateo, California, David is an Agile Tester for Guidewire Software (he’ll explain what they do in a bit). If his name sounds familiar, that’s because he is the co-author of testdriven.com – a very popular testing site covering developer testing, automated exploratory testing, model-based testing and more.

In this post, David offers us his “notes from the field”, where he’ll be addressing the role of  software testers; criteria for hiring testers; the experts he follows and more. We’re confident that you’ll like it. Enjoy!

I am thrilled to be invited to share my testing experiences with the uTest community. I hope my account will encourage more organizations to adopt the agile way and more testers to find fun and fulfilling jobs.

I joined Guidewire Software about seven months ago as an Agile Tester. We make core software for the Property and Casualty insurance carriers. If you have car or homeowners insurance, you may be serviced using our software. Right from the start, Guidewire has used agile development methods and credits a good part of its success to this philosophy.

Our applications are complex. It typically takes several months for a tester to become fully productive because there is so much domain knowledge and in-house tooling to learn. Our applications are enviably flexible, and each installation is customized to fit the specific needs of the insurance company. In order to empower the customization process, we provide a number of development tools including a custom java-compatible scripting language, an IDE, a GUI framework, and a screen painter.

For testing purposes, we ship a testing framework and have developed many internal tools such as a distributed continuous integration server consisting of several hundred hosts running various permutations of the operating systems, databases, and web servers that we support. We have an advanced UI testing framework that integrates with Selenium and a set of domain ‘builder’ classes that help with rapid test data creation.

Perhaps the biggest difference from other places where I have worked is the way testers are treated here. We sit together with developers grouped by functional areas. There is no dedicated QA space with the proverbial wall to throw releases over. We attend all product requirement meetings and participate in the product demos. Testers are truly first-class citizens of the development team.

Everyone does testing at Guidewire. Developer interviews include a large testing component, and the test-driven nature of our development is emphasized to make sure that new developers are on board with testing practices from the beginning. We have started a weekly Agile Testers brown bag lunch, which is well attended by testers and developers, as well as development and program managers. During this lunch, everyone is encouraged to demonstrate techniques that worked well or share war stories of challenging bugs. We discuss the latest books on testing and upcoming conferences, but no business decisions are made.

At this point you are probably wondering who writes the automated tests, developers or testers? On my team we have agreed to the following: both developers and testers write automated tests, but developers own and maintain the tests that run on the continuous integration server. Testers are focused on exploratory testing and use automation when appropriate to speed up the process. The advantage of this approach is that we recognize testers’ testing skills over their programming skills. It takes programming skills to maintain a large body of tests over a long period of time in an object-oriented language, and developers are typically better and quicker at this, whereas testers excel at domain knowledge, analytical skills, and the testing heuristics necessary for effective testing on tight schedules.

We work in two-week sprints and test continuously. Because developers practice test-driven development and the continuous integration server constantly reports test failures, the system is almost always runnable from the main branch in the source control system. We never have to wait for an official release to start testing. Both developers and testers work from story cards posted on the project board. If a requirement is unclear, the product managers sit next to us and provide instant clarification.

When we write test scripts that drive exploratory testing, we meet with developers from time to time to discuss which tests are good candidates for automated regression functional tests and turn them over to developers. Both developer-owned and tester-owned tests can coexist in the same source file by using the @ManualTest annotation. Because our application runs inside a web server, it’s important that our exploratory tests run inside a live application, not just in a testing harness. To accomplish this, one of our engineers wrote a tool called QuickTest that can load and run tests inside a live server. We recognize that tight engineer-tester collaboration is one of the most important aspects of our process.

But what if you are a tester with significant programming experience—is there a place for you at Guidewire? Absolutely! What I have described thus far is the testing approach we use on my application team. We also have testers on the platform team that possess significantly higher programming proficiency because they are testing our scripting language, IDE, and related frameworks. And last, but not least, we have our ‘testing labs’ projects such as the distributed, pseudo-random, model-based testing system I work on during my slack time.

If by now you are suspicious that all this sounds too good to be true, let me confess some of the challenges that remain with us. Due to the inherent complexity of our platform, unit tests incur a significant start-up time. We are fighting this by finding ways to tune our core code and by buying faster workstations with solid state drives. Over the years, we have accumulated a very large number of tests, and test maintenance is taking a noticeable chunk of engineering time. We are looking for ways to reduce test fragility and triage the regression test suites based on their relative value. Lastly, we have our share of non-deterministic tests—these are always ‘fun’ to diagnose and fix—they are often a symptom of concurrency or data locking problems and thus may be some of the more valuable tests in our arsenal. Unlike some smaller agile projects that boast their systems are always in a shippable state, we acknowledge the existence of “dark matter” – the stuff that is invariably found and must be fixed prior to a release – even as we are working hard to reduce its volume.

I would like to conclude with a short treatise on hiring testers. Too many development organizations are enamored with building out test automation at the expense of useful, efficient testing and have favored hiring programmers into testing jobs regardless of their true interest and talent for testing or their actual testing skills. This is a mistake. Some of the best testers I have worked with studied math, physics, philosophy, and psychology. Testing, like programming, is a performance art. When hiring testers, it’s important to have them test something during the interviews. It can also be very helpful to look at the testing they have done previously. This is typically impossible when someone has worked for a private company, but it is possible on the public projects on uTest.com. “Wow! How did you find that bug?” is a great question to get from the hiring manager during an interview. I value uTest as a fantastic resource for learning testing skills from peers all around the world, and I visit the site often.

I owe a debt of gratitude to the following individuals who have nurtured and supported me in my studies of testing. I encourage you to seek out their writing on the topic:

• Cem Kaner
• Elisabeth Hendrickson
• James Bach
• Paul Carvalho
• Michael Bolton
• Brian Marick
• Bret Pettichord

Special thanks to my peer and mentor at Guidewire, Dave Liebreich.

Sometimes new technologies can inflame old problems.  For example, consider location based social networks.  Many sites like Twitter and Foursquare make it easy to post both what you’re doing and your current location.  This is a great concept, and as technologies go there are huge possibilities for combining location information with social networking.  But there’s just one catch: if you’re out and Tweeting about it, then you’re probably not at home.  And that makes your home a perfect target for robbery.

To help people become more aware about the ramifications of announcing that their plasma TV is unguarded, a new site has appeared called Please Rob Me.  Using the magic of social search, they track various networks and then list the posts from people who are clearly not at home.  Of course, this has caused quite a stir online as many have wondered whether or not something like this is legal, ethical, or even right?

What does Please Rob Me have to say?  Have a look at their page simply titled “why“:

The goal of this website is to raise some awareness on this issue and have people think about how they use services like Foursquare, Brightkite, Google Buzz etc. Because all this site is, is a dressed up Twitter search page. Everybody can get this information.

On the Internet, your information is never really safe and sometimes privacy is just an illusion.  What do you think?  Do social location technologies have a place, despite the fact they advertise that your Playstation is unguarded?  Or will people start being more careful announcing they’re at Joe’s Bar downtown?

As for me, I’m currently at the uTest Mother Ship.  My house is guarded by two dogs.  My driveway is kind of long, and good luck driving up it in the winter.  I don’t own a Playstation or a plasma TV.

Darkreading.com published an article yesterday about a new proposal that could hold software developers accountable for security bugs. Not the “my bad” type of accountable – the legal kind. With support from some high-profile public and private entities, the proposal would likely require developers to make their software free of the CWE/SANS Top 25 Most Dangerous Programming Errors before it’s shipped. Needless to say, such a measure would drastically affect the day-to-day responsibilities of testers.

Stanton blogged about the Top 25 list around this time last year, noting that although it was comprehensive, it lacked meaningful context for testers. It appears that his feedback was incorporated into the 2010 version. Writes Kelly Jackson Higgins:

SANS’ annual list had been criticized by security experts as more of a laundry list rather than offering a solution, but this year the list came with so-called “focus profiles” that broke the programming errors into groups based on categories of weaknesses and also provided mitigation information. The list is in order of priority this year, with failure to preserve Web page structure (think cross-site scripting) as No. 1, and race condition mistakes as No. 25.

Not surprisingly, the proposal has sparked a lively debate among industry participants – testers, developers and consumers. Here’s how the pros and cons boil down:

• Pro: This measure would help protect software consumers from disastrous security bugs, and would give developers and testers a better idea of the standards their product should meet before it hits the market.
• Con: You can’t legislate quality software. Any attempts to put all developers under a one-size-fits-all policy will be futile, disorderly and will increase the costs of software production.

Anyway, I’m interested to hear what those on the front lines think of such a proposal. Yay or nay?

Over in Barcelona at the Mobile World Congress, 24 of the world’s leading wireless carriers and mobile OEMs announced their plans to create the Wholesale Applications Community (WAC) — a unified platform which developers can use to build a mobile app once and have it run seamlessly on any handset, OS or carrier.  Among the impressive roster of backers are mobile heavyweights like AT&T, Verizon, Orange, LG and Sony.  Sounds like a utopia for mobile developers, right?  It could be… if it works.

There are more than a few skeptics, including Jason Kincaid (@jasonkincaid) over at TechCrunch.  As Kincaid states (with a bit of help from Google’s Andy Rubin):

If it sounds too good to be true, that’s because it probably is. Andy Rubin, Google VP of Engineering (and the man in charge of Android) has already shared his skepticism, saying, “There is always a dream that you could write [a program] once and [have it] run anywhere and history has proven that that dream has not been fully realised and I am sceptical that it ever will be“. To put it another way, this is a pipe dream from carriers looking to loosen Apple’s stranglehold over mobile applications and there’s very little chance that it’s going to work.

The reasons Kincaid thinks the WAC won’t work out include:

1. Fragmentation:  Every device maker and carrier rolls out the latest system upgrades at their own pace.  Will they really coordinate their schedules so closely that developers don’t have to tweak their apps to work with each configuration?
2. App Trade-Offs:  Will app makers really trade horsepower for compatibility?  It sounds good in principal, but that’s always a tough pill to swallow for developers who want to create the next killer app.
3. App Store Arms Race:  Beyond PR and marketing bragging rights, does it really matter if your app store of choice has 10,000 apps or 100,000?  In short, no.

This is an interesting concept (who doesn’t love open, unified standards), but there’s an enormous gap between theory and practice.  And that gaping chasm is filled with failed industry standard initiatives that looked great in the press releases that announced them.  What do you think — will WAC work?  If not, why?

QUEST, one of the top software testing conferences, will be held in Dallas this year (April 19-23).  And uTest is getting geared up and is thrilled to be a part of this conference.

In addition to inviting Doron to be a keynote presenter, QUEST features a week-long agenda packed with more than 100 opportunities for attendees to build new skills and prepare for the testing professions of the future.

From exploratory testing to test automation to security audits to crowdsourced testing,  QUEST will cover a wide range of testing topics that give attendees insight into the latest best practices and innovative approaches to testing today. To learn more, here’s a sneak peek at the QUEST Magazine.

Special Note: Members of the uTest community interested in registering for QUEST are eligible for

a 10% discount, so please shoot us a note if you’re interested in attending.

Doron will be speaking on Crowdsourced Testing for Mobile Apps. In his keynote, he’ll be speaking about adoption of mobile apps in enterprises and the implications for QA leaders.  This includes addressing how testing methods fall short of meeting mobile’s ‘in-the-wild’ testing requirements and how crowdsourcing is a fresh approach to help meet the mobile app testing challenge.

Every day, software testers must put themselves in the shoes of their users.  Testers should always think about how their users – their customers, actually – are going to use their product.  Is the application workflow clear?  Does a particular bug interfere with the overall experience of the product?  Could the UI be more standards compliant or intuitive?

Good usability testing is both an art and a challenge.  ReadWriteWeb, a popular web startup blog, learned that fact the hard way this week after writing a post titled Facebook Wants to Be Your One True Login.  The post was about the partnership between Facebook and AOL to integrate AIM and Facebook chat. Pretty standard Internet news stuff.

But take a look at that headline – it uses both the words “Facebook” and “Login”. For Google that was enough to make this ReadWriteWeb article the number two link for those search terms, and after that all hell broke loose.

It turns out a lot of people don’t use the URL bar in the web browsers.  Instead, they set their homepage to Google and search for whatever they want to find.  For example, they type “facebook login” to visit Facebook and click on the first thing that appears – or the second by accident.

What happened next is even more astonishing.  A large number of people completely ignored the fact that ReadWriteWeb is a blog, uses a red color scheme, contains a logo that is both spelled and pronounced differently than “Facebook,” and does not actually feature a Facebook login.  Instead, they scrolled to the bottom of the page, found the Facebook connect icon, and tried to login there.  When they couldn’t, they posted comments and complained – loudly.

This is a software testing corner case, but an informative one.  What are the chances that a random blog post could appear in the number two Google spot for “facebook login?”  (Note: if you are visiting from Google, the Facebook login is here.)

After the dust settled, ReadWriteWeb posted some awesome thoughts about this experience.  All of their points are excellent, but I think #4 is particularly important for software testers:

4. Users rule the Internet.

Finally, this is the reason we’ve stopped mocking the poor folks who left those comments long enough to write this post.

400 million people now use Facebook, and they don’t all have CS Master’s degrees from Stanford. But if you work in the IT/tech/Internet/online media industries, they do manage to pay your bills. They’re the ones who open emails, click ads, make purchases, sign up for subscriptions and generally take the majority of actions that make our whole ecosystem work.

And most of them have no idea what a web browser is or how it differs from a search engine or a social network. They’ve chosen to be smart about other things, like building cars or making art or raising families. I’ll bet some of them are terrific dancers. We have to build the Web for them, too.

As testers, you play a critical role in building the web for the average person.  What kind of usability testing experiences have you faced?

VentureBeat has an interesting article about eBay’s announcement that they’re going to tap into their user base to test new features — a kind of opt-in, ongoing beta program for new features.  The title for this article:

eBay to Use Crowdsourcing to Test New Features, Starting with Streamlined Search

Those who know me well know that defending the purity of the term”crowdsourcing”  against misuse is a pet cause of mine (e.g. – Meet-ups are not crowdsourcing; online polls are not crowdsourcing; asking your Twitter followers a question is not crowdsourcing). But don’t worry… this won’t be another rant about the importance of definitions and how critical labels are.  Well, at least not about the word “crowdsourcing”.

My problem with this piece is the flagrant misuse of the word “test”.  Don’t get me wrong — I think what eBay is doing is great.  Beta programs like this can be highly useful, as proven by Google Labs.  But it is not testing.  As the author states about eBay’s program:

It allows the company to see how users respond to new ideas, and gives intrepid users new toys to play with.

Does this sound like the kind of testing that you do?  “Playing with” an app… doesn’t sound like testing to me.  Users attempt to get an app to work and then maybe provide like/dislike feedback.   Testers structure their use to isolate variables and systemically probe for weaknesses and defects.   I guarantee that the author or the eBay spokesperson didn’t mean any offense or slight to professional testers, but this illustrates how far we, as an industry, have to go in clearly defining and upholding the discipline of testing.

At the end of the day, users use; and testers test.  What do you think — how do we  as an industry differentiate between beta user programs and professional software testing?