You may have noticed that not many people know how the websites, software and apps they use everyday actually came into existence. They know that a developer built the app, but, as Lorinda Brandon mentioned in a recent column, the concept of software testing seems to leave them stumped. I’m not even a tester but I’ve noticed this lack of understanding whenever I tell people I work for a software testings company – they have no idea what that is or what we do. Once I explain it seems to make sense to them, but the overall concept of software testing just doesn’t seem to be in the main stream consciousness. Until now. As Lorinda pointed out in her column, the national discussion around the launch of healthcare.gov propelled testing and QA into the spotlight.
While several “software glitches” have been featured on the evening news, I can’t recall any that have caused a national conversation about the process of building and testing software until the Healthcare.gov debacle. Suddenly, Americans are sitting at their kitchen tables – in suburbs, in cities, on farms – and talking about quality issues with a website and who might be at fault.
The average American was given nightly tutorials on load testing and performance bottlenecks when the site first launched, then crumbled moments later. We talked about whether the requirements were well-defined and the project schedule reasonably laid out; we talked about who owns the decision to launch and whether they were keeping appropriate track of milestones and iterations. After that came the public discussions about security holes, which is not an unfamiliar concept to most people. But with those discussions came a healthy dose of encrypted passwords, third-party information sharing, and authentication protocols. School children and grandparents alike are worried about whether their passwords are being passed in the clear now. Imagine. There was even a major congressional hearing about the site, much of which focused on whether it was tested well enough.
It got really interesting when the media went from talking about the issues in the website to the process used to build the website. This is when software testers stepped out of the cube farm behind the coffee station and into the public limelight. Who were these people – and were they incompetent or mistreated? Did the project leaders not allocate enough time for testing? Did they allocate time for testing but not time to react to the testing outcome? Did the testers run inadequate tests? Were there not enough testers? Did they not speak up about the issues? If they did, were they not forceful enough?
So rejoice you software testers, QA specialists and quality evangelists – the world (or at least the US) now knows what you do!
Today, it’s easier said than done. The agile method has long been discussed in the software development space, but how many companies have actually migrated to it?
Gradually enterprise development teams have not only recognized the benefits of agile, but have successfully shifted from their waterfall methodologies. Still, others have lagged behind sticking to their longer waterfall dev methods. Now the pace of technology is beginning to leave these companies with no choice. As Matt Asay in ReadWrite says, agile is no longer an alternative:
“Agile development is no longer an alternative way to develop software. With the pace of technology adoption accelerating at a frenetic pace, agile is increasingly the only way to develop software. That is, if you want to stay in business.“
The number of mobile, tablet and connected devices is growing rapidly. But beyond that, usage rates are higher than any other. Rita McGrath, in the Harvard Business Review, covered the adoption rates from today compared to adoption of older forms of technology:
“ It took 30 years for electricity and 25 years for telephones to reach 10% adoption but less than five years for tablet devices to achieve the 10% rate. It took an additional 39 years for telephones to reach 40% penetration and another 15 before they became ubiquitous. Smart phones, on the other hand, accomplished a 40% penetration rate in just 10 years, if we time the first smart phone’s introduction from the 2002 shipment of the first BlackBerry that could make phone calls and the first Palm-OS-powered Treo model.
It’s clear that in many arenas things are indeed speeding up, with more players and fewer barriers to entry.”
This increased adoption requires an efficient, more flexible model of development. Agile helps teams reach release sooner, forcing them to be more efficient with their time and efforts. It helps companies of all sizes keep pace with the adoption of technology, with the competition and with their user’ expectations. That is… if it’s done correctly.
Even Asay admits agile isn’t the picture perfect solution, “Agile development isn’t some holy grail that will solve all a developer’s problems, but it is a savvy way to keep pace with technology adoption and to tackle large-scale development projects.” There are many things that can go wrong with an agile method, such as overlooked design and planning, condensed testing and a lessened focus on fragmentation. That’s why it’s critical that brands move beyond agile, maintaining efficiency and quality. That way brands can keep up with the pace of technology – and maintain their brand reputation.
For more resources on Beyond Agile, download this free whitepaper>>
As we mentioned in the announcement about our pending name change to Applause – the app world is much bigger than ever before. Not only are there more apps and more companies making apps, but the business of app quality is no longer solely in the hands of developers, testers or the IT department. It’s now “an engineering problem, a product problem, a marketing problem and a sales problem. In truth, it’s a CEO problem.” A company’s digital presence is now intimately tied to the brand’s reputation and success.
This fact is highlighted by studies and stats, but its also evident in the rise of the Chief Digital Officer. At the moment, CDOs are typically found in more “traditional” companies that recognize they need to change and adapt to keep up with the new digital world. According to Gartner, 6% of companies currently have chief digital officers, but that number is growing quickly. So what, exactly, does this digital head do? According to an article on CIO.com, it depends on the company but generally they have a hand in steering the company’s overall digital strategy and collaborate with many departments.
“As mobile technology and data analytics completely reshape the business landscape, building a truly digital business DNA is an imperative for survival in today’s competitive app economy. Many companies are seeking CDOs to lead this enterprise-wide transformation,” said Bryan Kirschner, director of the Apigee Institute. …
The CDO is not there to make technology decisions or run the company infrastructure. It’s a transformative role that is being used to break up the siloed functions within organizations. The CDOs are more about analyzing the data and how it relates to the business and customer experience. “The way to best describe the CDO is that you need to be a silo-buster connecting different disciplines and departments,” says [Andy Gilman, president and CEO of CommCore Consulting Group].
According to Tricia Blair, Chief Digital Officer at Lincoln Financial Group, a CDO can strengthen the company and ensure that the company understands digital’s impact on everything it does, from product planning to customer relations. Essentially, the chief digital officer is the one that understands that a company’s digital presence is everyone’s problem and it’s the CDO’s job to make sure all the key players recognize and embrace this new way of thinking.
Blair notes that her CDO peers in different organizations have varied hierarchy, but with Lincoln she considers the CIO and CMO peers, as opposed to reports. At Lincoln the three core components of her position include the Web, social media and mobile technology. As CDO she sees the role as the third leg in a good foundation. “There’s a focus that the CMOs take, a focus that the CIOs take and the focus of the chief digital officer that really makes for an extraordinarily strong partnership in a world that continues to get much more complex,” says Blair.
There is some debate as to whether this position will be a lasting one or if it’s just “transitional” as companies play catch-up with the digital age. But what the role of chief digital officer represents – that companies need to approach digital strategies and app quality with an all-in, comprehensive plan that has far-reaching effects – won’t be going away any time soon.
The Thanksgiving and Black Friday online shopping frenzy have just come to an end and, again, the trending confirmed another record-breaking year in online sales.
This past weekend, traffic and sales via online shopping crushed last year’s numbers with total sales marked at $3 billion, a 19.7% growth compared to the same period last year. This data came from IBM’s Benchmark Reports and covers over 800 merchant websites.
This report also shows a consistent growth of online shopping traffic and sales on Black Friday from mobile devices which accounted for 39% of all online traffic and 21.8% of all online sales. The chart below shows how smartphones performed against tablets in traffic, sales, conversion rate, and average order value:
While smartphones drove more online traffic than tablets did, tablets generated twice the sales for online merchants with a 3x conversion rate and +15% average order value. It proves that customers still prefer a larger view when doing online shopping since they cannot physically touch or try the products.
Adobe, which analyzed 400 million visits over 2,000 shopping sites, compared the two most popular mobile operating systems (as reported by Yahoo!):
Of the $3 billion online sales, $417 million was done on iPads and $126 million on iPhones, while users spent $106 million in purchasing with Android phones and $42 million with Android tablets.
The Mobile Commerce Index of Branding Brand, a leading mobile commerce platform for online retailers, also indicates that retailers who have optimized their sites and apps for mobile devices have shown a more significant growth in traffic (+75%), sales (+186%), and average order value (+22%) on Black Friday, compared to the same time last year.
Given the fact that online shopping will continue to grow while mobile becomes more popular, online retailers should thoroughly assessment their e-commerce sites and apps to make sure users can shop in a mobile-friendly environment and check out successfully.
Lyndon Cerejo, a user experience and usability strategist at Capgemini, has more than 375 apps in his phone. The vast majority of those apps sit untouched in what he calls his “app graveyard” – the back pages of his phone.
My app graveyard is the final resting place for apps that I have downloaded, tried or used briefly but have since left neglected. I leave them on my phone and tablet as a constant reminder of what killed these apps.
Lyndon isn’t alone when it comes to downloading then abandoning an app. He highlights some scary stats about just how often users ditch apps.
Guesstimates by analysts put the number of mobile app downloads this year at somewhere between 56 and 82 billion, with the average user downloading somewhere between 26 and 41 apps, with a smaller subset of those apps being used on a regular basis. Other numbers indicate that 95% of downloaded apps are abandoned within a month and 26% of apps are only used once.
But, as an expert, Lyndon also offers some important takeaways that can help teams keep their apps in active use. In an article for Smashing Magazine, Lyndon outlines 10 lessons to be gleaned from the app graveyard. Here’s a peek at those important takeaways.
Validate the Need for an App
Lyndon points out that sometimes, there doesn’t need to be an app for that. Native apps cost time and money to create and if users can get the same (or more) benefit out of a responsive website, companies should heavily consider that option.
The decision should be driven by business goals, user needs and the user experience. The short version is, if you plan to primarily offer content and basic functionality that users will access infrequently from different platforms and devices, then those users might be best served with a responsive website.
Make Sure the App Works as Expected
If you haven’t figured out that users will abandon your app if it doesn’t work, you might want to consider a career change. Sadly, many companies suffer from apps that just don’t work well in the wild.
This one might sound like common sense, but you would be surprised by how many apps do not work as expected or end up crashing, often after an update — and that’s not just from personal experience. One- and two-star reviews in the App Store often complain about just that. …
One way to avoid this is by thoroughly testing your app when releasing a new version or after an OS release. Test on actual devices, with bonus points for testing in mobile contexts (for example, testing an alarm app during deep sleep).
Applause, a mobile app analytics tool that crawls ratings and reviews from the Apple App Store, Google Play and Windows Phone Store, makes it easy to keep an eye on app store reviews and ratings, pinpoint problem areas and see how your new app version holds up in the real world. And testing on real devices in “mobile contexts” is what in-the-wild testing is all about! It’s the only way to make sure your app will work as intended under real conditions where your users live, work and play.
Don’t Drain the User’s Device
Users are wising up to the fact that some apps are a bigger battery drain than others. If they suspect your app is the offender, it’ll be axed. Lyndon notes that location based functionality is often an issue.
Apps should judiciously use a device’s resources, including memory, bandwidth and power. Apps that do not minimize their use of a device’s location capabilities or that do not disable location updates as soon as possible are common offenders.
Follow Design Standards and Guidelines
Though mobile usability standards are constantly changing, it’s important to pay attention and keep up with the trends. If users find your app difficult to use, confusing or feel it lacks attention to detail, they’ll go elsewhere.
Lyndon suggests offering a tutorial option when users first launch the app if it doesn’t follow normal usability conventions. And as always, know the design and usability standards of the platform you’re developing for.
Earn Your Users’ Trust by Addressing Privacy and Security Concerns
If news stories over the past two years have taught us anything, it’s that for all the social media over-sharing, users really do care about their privacy. They expect your app to collect only the data it needs and to keep that information secure. Keep users in-the-know and they’ll trust you and your app. Hide information from them or go behind their backs to collect data and you can expect an angry backlash.
Ask only for what you need — don’t collect or access information that is not required for the app to function. Many apps need access to such device information as contacts, calendar and photos to be useful, but an app should be transparent about why it needs that data and how it will be used.
That’s just a peek at the first five lessons Lyndon covers. He goes more in depth on each lesson and has five more for you to learn at Smashing Magazine.
None of these lessons should be new to you, they’re topics we cover often on this blog and our sister blog, the Applause App Analytics Blog. They really all boil down to two major themes, pay attention to your users and pay attention to how your app performs and is perceived in the real world.
While our uTester friends in the US are gearing up for a big holiday weekend filled with turkey and all the trimmings, our uTesters over in Italy just had their own celebration – of the uTest variety.
This past Saturday, our silver-rated functional tester and Test Team Lead Giovanni Pagani hosted a group of seven Milan-area testers, some of whom are aspiring TTLs and prospective uTesters. While some of the testers were familiar with each other from virtual chats over months and even years in cycles at uTest, it was the first time all had actually met each other in person.
uMeetups are local networking events organized by top testers and sponsored by uTest. Invites are extended to uTesters in the local vicinity to meet in person, listen to presentations on software testing topics, network, and learn more about participating in paid projects with uTest.
This weekend’s meetup started mid-afternoon Saturday at a central meeting point for the testers at the Mondadori Multicenter Duomo, then moved to a cozier café once everyone had found each other. Discussions during the meetup ranged from best practices for providing the best results for customers while maximizing their personal growth and income, to what the potential is for uTester saturation in the Italy market, to what brought testers to uTest!
We’re glad the turnout was good, folks got to put a face with the names of testers they’ve seen so often in testing cycles, the discussions were great, everyone got to take home some uTest swag, and most importantly, that a good time was had by all!
Our next uMeetup is this Saturday, November 30, in Sydney, Australia, and other meetups are in the works for Tokyo and Vancouver in the early months of 2014. If you’re a uTest Community member, check out all of the local discussions going on in your area and plan your own uMeetup today. These meetups are a shining example of just how far our 100,000+ strong community’s global reach is.
It’s that time of year again. A time when all of us here in the U.S.A. give thanks for what we have and show that appreciation by watching football, consuming copious amounts of L -Tryptophan and subsequently napping. Showing thanks and appreciation is year-round Standard Operating Procedure for tech startups, however. Be it the perks companies use to show appreciation for their staff or the design and UX considerations companies put into apps, it’s the little things that matter. So let’s reflect on some of those little things, shall we?
As we noted a few weeks back, it’s no longer enough for your app to just work. Users expect that. What makes your app successful is how it delights; how elegant it is, how pleasant it is to use. As our own Rich Weiss so, ahem, elegantly put it, “Memorable or intuitive apps and websites that connect with and understand the user have proven to be far more successful in the market than those that don’t.” When it comes to the difference between “fine” or “serviceable” and “great!” it’s the little things that matter.
The little things – or “perks” as they are known in workplace vernacular – also matter for companies looking to delight current and prospective employees. Workers in today’s tech startup world (and, increasingly, beyond) are often looking for something more than the standard, minimum viable job environment. From Emily Micucci in the Worcester Business Journal:
Tech companies seem to lead the way in designing workplace perk programs designed to lure candidates and keep talented employees, with Google an obvious example. It counts free on-site haircuts, laundry service and table games among its famous perks.
But you don’t need to be a Google employee to break up your workday with a game of ping pong. Many … companies offer similar benefits to their employees in an effort to create a pleasant work environment conducive to creativity — and hard work.
Citing the Society for Human Resource Management (SHRM) annual Workplace Forecast, Micucci notes that “attracting and retaining talented employees looms large in the minds of human resource managers.” She points out that survey respondents clearly selected it as the biggest challenge facing them over the next decade. Micucci also speaks with an entrepreneur who is currently focusing on issues facing employees:
Fil Firmani has a wealth of insight. He has spent a career working for technology startups and is trying to launch his own, called workforceM, a mobile productivity app for salespeople. Firmani has enjoyed his share of perks over the years.
But he said the most important one is intangible: flexibility…”You’re balancing out when you’re working and taking the breaks when you need it,” Firmani said. “It’s the biggest perk you can offer.”
True that, Fil. And, as one of the people quoted in the article, I’d add it’s important not to placate workers with perks but rather to use a holistic approach to show appreciation. It’s about showing gratitude and creating a culture and environment of collaboration and trust, not just free stuff.
So here’s to the little things. Be it for the usability of your favorite app that keeps you coming back for more or the perks you may enjoy that make your job awesome, what better time of year to give thanks? Now if you’ll excuse me, I have some football to watch.
This may not come as a surprise, but large companies are still lagging when it comes to adopting and integrating mobile into their company cultures and workflows. Large companies are not exactly known for moving quickly, then again, it seems like mobile is everywhere and has been here forever. Still, a survey done by IBM/Oxford Economics found that “‘The mobile challenges that organizations are wrestling with are much the like the challenges they saw when dealing with the emerging Internet 15 years ago,’ said Eric Lesser, an author of the study and a research director at IBM’s Institute for Business Value.”
The survey covered 600 companies with more than $500 million in annual revenues (a decent number of those companies hit the $5 billion a year mark) in 29 countries. It also spoke to 30 “mobile leaders.” Only half the companies report having a structured, integrated, formalized mobile plan. From CIO:
Only 50% of the organizations surveyed agreed that their mobile strategy is aligned with the overall business strategy. Even fewer said their organizations had set up a clear funding mechanism for mobile, or had an established governance structures for mobile initiatives.
Those companies that haven’t figures out the mobile world yet are really only hurting themselves. Consumers and customers expect companies to have a mobile presence and if they don’t those customers might just go elsewhere. That’s not just lip service, embracing mobile has had strong, quantifiable results.
Of the mobile strategy leaders (about 14% of the total participants), 73% reported measurable returns from mobile investments. Some said that a key benefit of using mobile technology to improve employee productivity is faster customer response time — answering complaints and requests, or fulfilling orders.
Of the industry leaders, 51% percent of banks in the study reported measurable returns on investment from their mobile maneuvers, compared to 34% for other organizations.
Hopefully those numbers will help large companies realize that embracing mobile strategies is good for customer engagement, worker morale (another part of the study found the mobile-centric social networking helps coworker connect and solve problems), brand reputation and the bottom line.
Want to be successful in your mobile approach? The survey found that those companies who are moving ahead in mobile are also discovering that a mobile strategy isn’t just an IT or development thing – it’s a whole company thing. Getting more departments involved in the effort helps companies have a more holistic, successful mobile experience.
Mobile leaders said they have found success in bringing the Chief Marketing Officer into the mobile planning process earlier, Lesser noted. A senior advisor to an electronics company quoted as saying that it’s important in mobile strategy planning to “make sure the voice of the customer is heard by the engineer.”
From ensuring your software is bug-free – to estimating time to complete tasks – to simply explaining your job to a non-developer. Developers have to deal with all sorts of road blocks, and most of them aren’t even related to writing code.
Phil Johnson, of ITWorld, recently came upon a Quora discussion where developers shared their hardest tasks. Here’s a look at a few of them:
#1 Writing Documentation
The Task: Create documentation explaining what your code does or how an application works. It can include stand alone documents and code comments. The intended audience can range from end users to other developers.
The Challenge: It can be a time consuming task, that can feel like a waste of time if nobody reads it. Programmers usually prefer writing code to documenting it.
#2 Writing Tests
The Task: Write unit tests, i.e., programmatic tests of small units of code to ensure they function properly.
The Challenge: It can be a tedious process to choose tests to write and to code them, which can feel like significant additional work on top of building the application.
#3 Working with Someone Else’s Code
The Task: Having to maintain, debug or enhance an application or piece of code that was written by another developer(s).
The Challenge: Trying to understand how a piece of legacy code works and divine the intentions of the original developer. This is even harder when that developer isn’t around and the code is poorly written, commented or documented.
#4 Explaining What I Do (or don’t do)
The Task: Convey to non-programmers (family members, friends, non-tech coworkers) what your job entails – and also what it doesn’t.
The Challenge: Having your loved ones not understand what you do for a living. Being constantly asked to solve any and all computer-related problems.
#5 Estimating Time to Complete Tasks
The Task: At the outset of a project, come up with time estimates for the work to be done.
The Challenge: Guessing how long something that you possibly haven’t done before will take, making estimates based on vague requirements and trying to allot time for dealing with unforeseen problems.
Challenges such as writing unit tests and working with someone elses’ code are extremely challenging – not to mention they leave room for error. The headache of not being able to test properly or free up your team to complete tasks can be easily mitigated by extending your in-house QA efforts out through crowdsourced testing methods. This will allow for more thorough testing, and will free up your test team so they can meet deadlines and complete tasks on time.
What do you think is the biggest challenge you encounter as a developer? Let us know in the comments section.
When it comes to building high quality apps, ensuring that they can handle even the toughest loads is one of the most important steps in software testing. But despite its importance, companies often see load testing as too difficult, too expensive, and too distracting from the goal of simply shipping the app. For many companies, their first load test comes in the “real world,” in the hands of their devoted users. And for all too many of them, the results are disastrous.
Our customers have been asking us for more and more help with load testing, and earlier this year we started seeking a partner who could help us execute high volume load tests. When we met the team at BlazeMeter, we immediately knew we had made the right connection. When it comes to modern load testing, these guys get it. They’ve built an outstanding cloud platform to run load tests using JMeter, a hugely popular load testing tool, with the ability to quickly scale from a few thousand to hundreds of thousands of virtual users.
Today we’re pleased to announce a new partnership with BlazeMeter, and we think that by working together we will make it substantially easier for companies to load test their web and mobile apps quickly and affordably.
BlazeMeter has built their load testing technology on JMeter - an open source platform with thousands of man hours of active development – for companies like Nike, Adobe, MIT and others. By creating a cloud platform for JMeter scripts, BlazeMeter has made it easy to deliver load tests on any scale without their customers having to manage expensive infrastructure.
uTest brings to the table our team of performance experts to help create JMeter scripts and interpret the results. With our community of load testing professionals, we have some of the smartest JMeter experts in the world who know how to build testing scripts, execute them, and then interpret the results. Companies can work with us to have their web and mobile apps load tested using industry-leading tools, even if they have have no prior background in running a load test.
And because JMeter is an open platform, companies are not locked into an expensive tool with recurring licensing fees. Customers can reuse their JMeter scripts however they like, including on their own systems.
Are you launching a new app soon? Getting ready for the upcoming big holidays? Planning for an ad blitz around some upcoming sporting events? You have absolutely no excuse to think load testing is too complicated or expensive. Using uTest, we can help you plan, execute, and interpret a full suite of load tests very affordably for your web or mobile app. Learn more on our load testing overview page or contact us for pricing.
Ben Kelly has literally tested around the world. His career has taken him to Australia, Japan and the UK and Ben is currently a Software Engineer in Test for eBay. A regular presenter at conferences in the US and Europe, Ben also blogs at TestJutsu (when he has a spare moment).
In this month’s Testing the Limits interview, Ben discusses his testing experiences, his passion for exploratory testing, advice for new testers and his ultimate dream for the testing profession.
uTest: So, how did you become a software tester and what drew you to this field?
Ben Kelly: It should have been obvious to me in my youth. I had a habit of taking stuff apart to see how it worked. That coupled with my tendency toward dark humour and pessimism should have been a clue. I wish I’d known earlier that testing was a possible vocation.
Like many others, I fell into the field accidentally. In my case, I wanted to be a programmer. I was an okay coder, but no company seems to want ‘okay’ out of university. They want propeller hats, pocket protectors, coke-bottle glasses and social ineptitude. A friend suggested testing as a way of bridging into a programming role. I gave it a shot and then discovered that I was much more successful at finding out ways that stuff does what it shouldn’t than making it do what it should. I also enjoyed it a lot more.
You’re a big proponent of exploratory testing. What draws you to that style and why do you think it’s a good approach?
BK: All testing is exploratory to some degree. Even in the case where you have heavily prescribed test steps, there is still room for interpretation. There are often multiple ways of performing a certain action. There are cases where you will notice potentially interesting things that are not completely relevant to what you’re testing. It’s up to you as a skilled tester how you act in those situations. You could choose simply to ignore it and follow your script. You could also be a robot made entirely of meat.
I’m not against test scripts or documentation. I’m against following rules over applying skill. When you’re given the freedom to do your job as a skilled knowledge worker, it’s much easier to avoid regularly abused and frequently meaningless metrics like bug counts and test case completion percentages and instead to focus on finding information that is important to the people you serve.
Since you’re a manual, exploratory tester, what are your feelings on test automation?
BK: Test automation and I are not strangers. I don’t see automation as a dichotomy of ‘automation vs. manual.’ Automation in its many forms serves to augment testing and/or programming to some degree whether it be a throwaway script, a heavy-duty test framework or something else.
Automation should solve a problem. Much like any other software development effort, you want to know what problem you’re solving and which tool is right for the job. Testing is testing. It’s the thought behind the effort that makes it good or bad. Throwing GUI automation at every problem because that’s all you know is ignorant at best. Mandating things like percentage of manual test cases to be automated is stupid. If you’re automating without understanding why, then the tail is wagging the dog. If you’re automating in the hope of doing away with sapient manual testing then you’re doing it wrong.
You’ve worked for companies literally around the world. Do you see different approaches or attitudes toward testing in different countries?
BK: Not really. I see stereotypes more in terms of industry, company size and individual company culture rather than a country-specific attitude. Behavior is driven by what is rewarded. Companies that value commoditization of testing and protecting the bottom line will get very different testers to those that value skilled knowledge work. That seems to be true no matter what country you’re in.
eBay is one of the top names on the online retail world – not to mention one of the first in the space. How does eBay approach software testing?
BK: eBay is a big company, so I can’t say how every testing group approaches testing, but as for Europe, the testers are some of the best I have ever worked with; seriously world class. With eleven markets and seven languages, there’s a lot of ground to cover.
Software quality is everyone’s responsibility. The programmers in Europe take testing seriously. They’re serious about doing good testing and getting better at it. They write both programming-supporting automation at the unit/integration level as well as acceptance checks. That means as a tester I can focus on finding issues that might be a real danger to a project as opposed to wasting time on the trivial and the obvious.
I’m embedded with a team of coders, so I’m involved from story creation and estimation, pairing with programmers while they work and putting the end product through its paces. It’s a great place for a tester who is serious about their craft.
You’re a founding member of the International Society for Software Testing. Why did you join?
BK: I joined because I see the society as having the potential to make a real difference to the profession of software testing. There is a prodigious amount of talent and experience in the founding members. I want to tap into that and use it to reach out to people who have the ability to influence testing, but aren’t necessarily testers themselves. My goals for the society are quite ambitious, but talk is cheap. I will let my actions speak for me.
Your blog – Testjutsu – includes a section about your favorite testing tools. How often do you try out new tools? Are there any cool new tools you’ve found recently that our readers should know about?
BK: Truth be told, I’m probably a bit of a dinosaur when it comes to tools. I have what works well for me and I haven’t had too much cause to go hunting new ones. To torture a weapon analogy, a sword is merely a bit of metal, wood and leather. It’s the mind of the person wielding it that makes it a weapon. Having a good quality sword certainly helps, but knowing when to use it and when not to is key.
I do a fair amount of web testing, so my mainstays tend to be Firebug/Chrome developer tools, Snagit for screenshots and recording test sessions, Hexawise for putting together various combinatorial test scenarios, Bash and Ruby for quick scripts, Dropbox for keeping information synchronized, Freeplane for mind mapping, Skype for chatting with my testing peers, Rikaichan for helping me read Japanese, LiveHTTP Headers, Webscarab for mucking around with http requests.
I’ve been looking at doing more accessibility testing of late, but I’m not far enough along there to recommend any tools at this point.
You wrote a guest blog for us in 2010 dreaming of a day when more people would buy into and support testing and not think of it as “boring.” In your opinion, have we made any progress toward this better testing world?
BK: There’s a long way to go yet, but the future looks promising. For testing to flourish and be recognized as a skilled profession, we need to change the way that non-testers perceive testing. We need our non-testing peers to expect more from the tester role. In order to do that, we need to make sure that skilled testing is visible to non-testers in a way that is meaningful and useful to them.
If there’s one thing you could go back and tell yourself at the beginning of your testing career, what would it be?
BK: Explore your confusion, surprise, doubt and emotion – they’ll often lead you somewhere interesting.
What’s been your proudest moment as a tester?
BK: My proudest moments as a tester come from watching the people from teams I have built as they go on to do new, cool things. I can’t take credit for their hard work, but I’m super proud of what they have achieved and I have a sort of selfish pride in having played a small part in that.
You don’t have to give us company/product names, but what’s the most important bug you’ve ever found?
BK: I’ve found bugs that would have cost the company substantial amounts of money, or possibly public embarrassment, but I tend to remember the ones I didn’t find and should have rather than the big ones that I did. I recall bugs that I thought were showstoppers that the business side shrugged off as inconsequential. I’ve seen what should have been an innocuous single character change take out an entire website. It’s a strong reminder to stay alert. Any time you catch yourself thinking ‘yeah it’ll be fine’ – there’s a guy called Murphy who’s itching to prove otherwise.
If you had to sum up your perfect idea of a tester and their goals in one or two sentences, what would you say?
BK: A tester is someone who embraces the inevitability that expectation and reality are different and explores those differences for potential problems. They explain these problems to people that matter in a way those people can understand and act on.
What is Ben Kelly doing when he’s not testing?
BK: I practice kendo, I have a couple of screenplays on the go, I appreciate fine scotch, I read, I’m trying desperately to improve my terrible grasp of the Japanese language and pick up a bit of German along the way. I also try occasionally to convince my incredibly patient wife that I still remember who she is.
The Federal Communications Commission is the latest organization to leverage the power of in-the-wild testing to obtain user feedback under real-world conditions. Last week the FCC announced that it will be using ordinary folks like you and I to assess mobile broadband speeds. The free “FCC Speed Test” app is designed to capture device performance metrics such as download speeds and latency. The app will run in the background and is designed not to exceed 100MB a month so that it won’t overuse a customer’s data plan. Individual users can view their own device’s performance within the app. Currently, the app is only available on Android but an iPhone version is expected to be released in January 2014.
The purpose of the initiative is to develop cumulative, nationwide data on mobile broadband speeds. Right now, there is a lack of unbiased data regarding the subject and carriers can claim their download speeds based on their own controlled lab tests. By utilizing testers in-the-wild, the FCC can accurately gather metrics from consumers operating their devices under actual conditions. This will provide consumers with valuable statistics when choosing a service provider.
What does this mean for you as a developer or tester? You already know that connection speeds vary by location and they certainly aren’t as reliable as lab connections. Once the full FCC data becomes publicly available (reportedly sometime next year) you’ll have a road map of trouble spots.
Use this information to guide your own in-the-wild testing. Perform extra testing in areas that have poor connectivity levels or focus extra testing on carriers with spotty networks. This information will empower you to make your app better by making sure it works in the hands of all your users, no matter where they are or what type of connection they have.
Retailers are preparing for the flock of early morning shoppers who will descend on brick and mortar stores on Black Friday – the morning after Thanksgiving. And online retailers are gearing up for Cyber Monday, when their apps will be put to the ultimate eCommerce test. But increasingly these two worlds are merging. A new study shows that more shoppers than ever plan to do their Black Friday bargain hunting online.
From shopping to research, your eCommerce site better be ready for a wave of traffic in the coming weeks. From Internet Retailer:
Black Friday this year promises to be more digital than last year, suggest survey results from Accenture. The management and information technology consulting firm says that 30% of consumers will do most of their day-after-Thanksgiving shopping online this year, up from 25% who said the same in 2012. …
A separate survey from the National Retail Federation finds that nearly half of consumers expect to go online to research gift ideas.
Shopping around for gift ideas won’t be contained to retail sites, though. According to the Nation Retail Federation survey, shoppers will go to different types of online media to find the perfect gift idea.
- 47.9% of consumers will seek out holiday gift ideas online
- 21.5% will use e-mail marketing messages
- 14.0% will use Facebook
- 10.1% will use retailers’ apps
- 7.2% will use Pinterest
And those numbers don’t account for shoppers who will use apps while in physical stores. This year, 55% of Accenture’s survey respondents expect to shop in some form on Black Friday, and 38% plan to shop on Thanksgiving Day itself.
Make sure your retail or eTail app is functional, usable and ready to stand up to the traffic load. (If you’re concerned your retail app isn’t ready for prime time, check out our free resources: Retail App Testing and Optimized eCommerce.)
As new technologies (hello, smartwatches and Google Glass) emerge onto the app scene and the pre-existing mobile app market further explodes in growth, it’s hardly a surprise that testing teams are scrambling to keep up with the rising costs and complexities of software testing.
In fact, a recent Sogeti survey of software testing/QA professionals confirms this fact – 92% of testing pros surveyed find that the cost and complexity of software testing is on the rise. What’s even worse is that only 22% of teams are ready for the challenges.
One of those biggest challenges, according to Sogeti UK’s CEO Brian Shea, is maintaining quality.
“Quality has never been a higher priority for businesses as they work hard to attract customers in highly competitive markets,” said Shea. “Continual re-evaluation of test process and capability is necessary to ensure the smooth roll out of transformational projects that will ultimately ensure they stay at the top of their game.”
Of course, uTest’s testing model is a textbook opportunity for organizations to re-evaluate test processes and supplement their in-house QA structure with the added benefits of an in-the-wild approach. The quality may be there inside the comfy confines of the organization, but it doesn’t accurately reflect the quality where the company’s users work, live and play.
As to addressing the rising costs, uTest’s all-you-can-test model ensures organizations get maximum value from each testing cycle, keeping costs in check for organizations from start-up to SMB to enterprise.
As organizations struggle to cope with costs and complexities of software testing spiraling out of control, in-the-wild testing may just be worth a look as part of that “continual re-evaluation of test process.”
For those of you that don’t already know, today is World Usability Day! Yes, indeed, an entire day dedicated to celebrating the joy of simple, easy to use interfaces that make our day-to-day lives easier and more manageable. This year’s focus is on the healthcare industry, which is even more exciting!
What, you aren’t jumping with joy? Well, let’s talk about usability and its role in healthcare and see if it puts you in a more celebratory mood.
Doctors, as we all know, don’t have a lot of time. They’re always busy, always on the move, and they see countless patients each and every day. The high demand and lowered supply of healthcare professionals coupled with the push to digital record keeping has given rise to the healthcare application as an industry all of its own. Which is great! Easy access to patient records, test results, chart data, and more right at a doctor’s fingertips means higher efficiency, lower costs, and happier doctors and patients alike, right? Right! But only if that access actually is easy.
Doctors, nurses, and other practitioners don’t have the time nor energy to be painstakingly sorting through oceans of buttons, links, and settings just to find the one thing they need right away. There’s no room for error, human or otherwise, when we’re talking about someone’s health – which in some cases is a life or death situation.
Healthcare is one of the few industries where poor usability affects not just the direct user of the app and its information, but also the receiver of that information. In essence, healthcare patients are secondary, or indirect, users of healthcare applications and their experience also matters.
You wouldn’t want to wait an extra hour for your doctor because they weren’t able to locate your e-chart, would you? And you certainly wouldn’t want to be given the incorrect diagnosis because your test results were buried in dust somewhere deep inside an application. Or to be given the wrong care, prescription, or prognosis just because the doctor’s tools were insufficiently laid out to meet his or her needs in any respect.
Healthcare applications need to work the first time, every time, in a way that is easy to use and understand at a glance. They need to be fast, accurate, stable, secure and intuitive. Not just for your doctor, but also for you.
So join with me and uTest in celebration of World Usability Day, not because usability is an amazing thing all around, but because good user experience really does affect our lives each and every day in a number of ways, and they should all be as great an experience as possible.
There’s an old saying in the military that “generals always fight the last war, especially if they have won it.” Quite simply it means that when preparing for the next kind of threat, one should resist looking too closely at past wars fought at different times with different technologies and circumstances. The French were overrun despite the Maginot Line – the perfect defense against further aggression by the Germany of World War I, but practically useless against the German Blitzkrieg of World War II.
So it is with app security. Early in the era of web security, there was still a strong fear of classic desktop vulnerabilities like stack smashing and buffer overflows. Now that we’re in the era of mobile apps, the security world is still stuck in the realm of web security. Despite the fact that it’s been over two years since the OWASP group released the top 10 vulnerabilities for mobile, few mobile developers think to security test their apps.
Last week, researchers revealed yet another vulnerability type for iOS applications. By using a man-in-the-middle approach, an attacker can trick an iOS app into communicating with a web API on a malicious URL, not just once but forever after the fact. Ars Technica has the details of the attack, discovered by the security group Skycure, but the summary is simple.
As it turns out, many iOS apps do not have strong protections against malicious 301 redirects for API calls. A 301 redirect is basically a web command that says, “This thing isn’t here anymore. It’s over there instead.” Web masters use it when moving content from one place to another so that links work smoothly even when the content address has changed. But if I can interfere with your app’s communications and say “your API is really over here on my malicious server,” then I can theoretically intercept or even modify content used by your app. Because the 301 code signals a permanent redirect, your app will continue to use my malicious server long after I have stopped directly interfering.
The fix is simple. Make sure your app uses SSL instead of communicating in plaintext. It’s odd to think that in this day and age anyone would use an unencrypted API, but here’s the perfect reason to do so. By going the extra step to encrypt your API’s traffic, you’re also making sure that your users only see the right content and not something malicious or bogus. SSL certificates are cheap compared to the cost of your app losing its reputation because of security problems.
It has been a little over a month since the newest members of the Apple iPhone family, the 5s and 5c, were released. A new report suggests that apps running on the 5s are experiencing a higher than normal crash rate. While apps crashing on new devices can be fairly common, iPhone 5s owners have experienced the errors at a rate double that of 5c users. Consumers report being plagued by a “blue screen of death” followed by the device rebooting. While the cause of the problem is uncertain, the iPhone 5s features an upgraded 64-bit, A7 processor. The 5c contains the 32-bit A6 processor found in the iPhone 5. From TechCrunch:
This is likely due to the fact that, though they prepared relentlessly for the new iOS 7 operating system, developers had no way of preparing for the new hardware architecture, especially that 64-bit processor.
This issue brings up an important point when testing new apps. Obviously, it was impossible for developers to test their apps before the new iPhone hardware was released, but they now have to account for a variety of devices moving forward. Apps must now be tested across the range of iPhones – in addition to Android and manufacturers if applicable – to ensure quality on each. If you don’t have an iPhone 5S in your testing matrix, add one now. Either go out and buy one for in-house testing to employ in-the-wild testing to access to testers with this new tech.
Are your users having issues with your app on their new 5s? If so, it could have a real impact to user feedback and your credibility moving forward.
Companies have been trying to find the consumer trust line when it comes to security and privacy for a while now. Users want more features and better convenience, and companies want to supply those features and conveniences – but there’s a fine line they must not cross for fear of making user angry. Where exactly is that line? It seems to shift all the time, but a recent survey by Boston Consulting Group will give you a better idea of what users hold sacred. TIME boils down some of the stats:
Among younger millennials (18 to 24), 60% consider Internet surfing history to be moderately or extremely private, while 61% of older millennials (25 to 34) and 60% of Gen-Xers (35 to 48) hold the same view. Only 51% of Baby Boomers (49 to 67) and 54% of Silents (68 and older) consider surfing history private. With respect to information about your exact location—the geolocating data that helps guide you to your destination and guide advertisers to you—the young appear even more privacy-conscious than the old. Only 9% of younger millennials consider data on your exact location to be slightly or not very private, compared to 20% of Silents.
That last set of stats is particularly important – 91% of consumers aged 18-24 consider location information to be at least somewhat private. This is a big deal for a lot of apps as geolocating features play an increasingly larger role in a range of apps.
Does this mean your app can’t have a geolocating feature? Not at all. Instead, what you need to focus on is being transparent about that feature. Explain to your users what information is being collected, why you’re collecting it, how it’s being used (including if other companies have access to that information) and how it’s being stored. Give them a way to opt out of geolocation if it’s not an integral feature. If it’s a vital part of your app, do your best to assure your users that you value their privacy and take appropriate measures to keep private information safe. (If you want to know what users are saying about the privacy of your native app, check your Applause Score and dive into the Privacy attribute.)
Another important take away is paying attention to the opinions of your target market. While transparency is important across the board, you’ll want to place special emphasis on spreading your message if your target users are part of the Millennial generation. Contrary to the popular stereotype, Millennials do care about privacy and security. It may seem like they over-share and divulge personal information, thoughts and feelings on a whim, but that sharing is intentional and on their own terms. If you share their personal information, they’ll certainly share their anger at you with the world.
Testers are masters at taking note of imperfections and overall being pretty perceptive folks. So it’s only natural that some of these qualities have rubbed off in the real-world, too, for better or for worse.
In a recent uTest Community forums post, we put this hypothesis to the test by asking testers how their day jobs affect their behavior off the clock.
Here are the top 10 responses to ‘You know you’re a tester when….’:
- You are happy when an app crashes
- When you find at least one bug in literally every app you use or website you visit (whether you’re looking for them or not)
- You go to a restaurant and find a spelling error in the menu
- When you can tell developers where the bugs are…without seeing the app
- When you realize that what you see isn’t always the truth
- When the phrase ‘It works fine on my machine’ turns you into the Incredible Hulk!
- You file a bug at 1am. Then you file another bug at 5am.
- When you cannot wait to check whether your filled bugs/issues/defects have been dealt with
- When you start analyzing everything around you in terms of quality…starting with a hair clip
- When your thinking power tries to find an error in everything you see or observe
We’re curious as to the ‘habits’ you’ve formed as a result of being a software tester as well! Sound off in the comments below.
Last week I wrote a bit about the latest Android version – 4.4, aka Kit Kat. Mostly, I highlighted the fact that Google is addressing the issue of Android fragmentation by designing Kit Kat to work on a wider range of devices – including lower spec phones.
But this release is a major change for the operating system, and as such, it’ll have some other major implications for developers (just like when iOS 7 shook things up a few months ago). To give you a better idea of what to expect, here’s a look at some of the biggest, most exciting and potentially most challenging changes Kit Kat will introduce. From ReadWrite:
Full Screen Immersive Mode
Developers can choose to hide menu and navigation bars, buttons and other chrome to give users a true full screen app. … To reveal the system user interface, KitKat has a new gesture where a user swipes from the top or the bottom to reveal navigation, menus and buttons.
Developers can now animate between “scenes”—different pages or events within [an] app—through a new transitions framework in KitKat 4.4. … If developers don’t want to individually manage animations and transitions by setting specific scenes, a function called TransitionManager can do it automatically within an apps view hierarchy.
Storage Access Framework
The new Storage Access Framework in KitKat allows users to browse and open documents in an app from a variety of cloud storage providers. … Cloud providers or local storage services (such as those offered by individual Android manufacturers) can use the new system by implementing a new document provider class within Android for their service.
New WebView & Chromium Features
New Near Field Communications Platform Support
Host Card Emulation (HCE) in Android 4.4 is a new platform to support Near Field Communications (NFC) transactions. Google says that with HCE, “any app on an Android device can emulate an NFC smart card, letting users tap to initiate transactions with an app of their choice—no provisioned secure element (SE) in the device is needed.” Apps can also act in a new “Reader Mode” to receive NFC functionality (like payment processing, building access, tickets etc.).
Android 4.4 has platform support for hardware sensor batching to optimize power consumed by various sensors. Google is working with hardware manufacturers to allow for collection and delivery of sensor data on an Android device while allowing device’s processor (CPU) to stay in low-power mode. … This should improve battery life and performance for sensor-driven applications.
RenderScript Takes Advantage of Device Hardware
A new C++ API in the Android Native Developer Kit in KitKat 4.4 lets you access RenderScript through the Android platform framework. Big tasks that put a lot of pressure on the device’s hardware can now be integrated into an apps native code and allow for support from multiple smartphone CPU and GPU cores.
That’s just a condensed peek at the features and changes ReadWrite highlights and the article goes into quite a bit more detail about these changes, so be sure to read the whole thing.
Like any major OS version release, Kit Kat seems to be bringing a lot of cool new features to both users and developers. And with Google’s plan to make Kit Kat the Android version that finally brings the ecosystem together, it’s important that your apps going forward take advantage of all the cool new things Kit Kat allows you to do. Once users get used to these features, they’ll be disappointed if your app doesn’t support them.