Skip to content

Feed aggregator

New Website

Watir - Web Application Testing in Ruby - Thu, 09/22/2016 - 16:43


Recently, we updated the Watir website to be hosted on GitHub Pages. The existing and sites will no longer be maintained, but most if not all of the information currently on them has been transferred over to the new site. Check it out at now redirects to this new site, and soon will become

If you think something is missing or have any comments or feedback, make an issue on the site’s GitHub page or join us on our Slack channel.

Categories: Open Source

Team Building and Quality Assurance

Sauce Labs - Thu, 09/22/2016 - 15:00

Ashley Hunsberger, Greg Sypolt and Chris Riley contributed to this post.

How do you build an effective team of quality assurance (QA) engineers? Where do you look to recruit the best QA professionals? How should you integrate your QA team within the rest of your organization? These and other questions related to the topic of team building came up during a recent webinar hosted by Chris Riley, Ashley Hunsberger, and Greg Sypolt. Here are links to the first and second post in this series.

Team building is an essential part of an effective QA operation, of course. Your testing strategy and execution are only as strong as the people who design and conduct them. And in today’s IT world, where DevOps is changing the way different types of engineers interact, and the supply of good QA professionals outstrips demand, having a well-planned and executed approach to team building is crucial.

Below are some of the most important team-building questions that came up during the webinar, along with answers from participants.

One thing I have noticed lately is that QA engineers are getting more respect. With automation, their skillset looks more like those of a developer. Do you agree?

Ashley: That’s a great observation. I think that as we shift QA ‘left’, thereby helping define the acceptance criteria and the definition of done, we are finding there is much more visibility into the mindset of testing and what is required to meet our expectations. This visibility into our world has helped us be seen as vital members of the team, and we have earned the respect of our engineers.

Greg: It’s odd how developers respect changes when shifting to Quality Engineers (QE). It shouldn’t be this way, but I have observed similar reactions to non-technical and technical QA team members over the years. It shouldn’t matter, but when QEs start working closer with developers by sharing automated testing, operation tasks, and vocalizing the necessary quality gates on user stories, they start to respect QA value. It starts by changing the culture—by educating the entire team, letting them know that everyone owns quality. It’s more than a QA task.

What strategy can you suggest for retaining engineers with strong technical and testing backgrounds?

Ashley: I think it’s a matter of perception and buy-in from the team. One thing I look for when building teams is a passion behind testing, not viewing it as a means to an end. For existing team members, I look at how I can foster and grow that passion. What makes testing interesting? How can you apply your technical aptitude to implement what interests you in testing? I don’t see it as a lack of respect. They are very different roles and viewpoints—It’s unlocking that potential and showing the teams what you’ve been up to, and why you love what you do.

Greg: Is it such a bad idea for QA engineers to transition to full-stack developers? I have seen many of our young software developers in testing make this change over the last year. The expectation when they make this transition is that they have learned how to test the application, and they will test their code better. It is another way to share their testing knowledge with the rest of the developers and promote that everyone owns quality.

If you were going to gather a functional QA team for an Agile project, would you select only QA analysts with automation experience? Or do you think manual testers are still needed?

Ashley: I’ve said it before, and I’ll say it again. Manual testing is still needed. There are things that automation just can’t tell you. But, I do think it’s important to find people who can think, and have the ability to learn a new skillset. In an Agile world, the TEAM owns quality—So it may be engineers writing the automated tests (as it is where I am), but our functional QA team members need to understand what needs to be tested (driving the overall test strategy), they need to know how to write tests so they are able to be automated, need to be able to review what the engineers wrote, be able to debug any failures, etc.

Greg: I agree and disagree that you need a full-time manual tester on your project teams. The team owns quality and must understand what needs to be tested. The software developer in test should be driving the overall testing strategy, but it is a team effort. I do see the value of having manual testers when attempting to automate complex scenarios (i.e. upgrades or push notifications).

Modern QA to me means that QA is involved in dev designs, participates in code reviews with devs, does upfront testing in dev boxes, and so on. Do you agree with this tight integration between the QA and dev teams?

Greg and Ashley: Yes, we agree. Modern QA is part of that ‘shift left’ mentality where QA is very involved from the get-go, and not just when a dev finishes code and says ‘test this now.’ Even if we aren’t writing the low level tests, we should be involved in determining what tests we need, at whatever level.

There are manual QA testers who are hesitant to transition to modern QA. How do you help your team embrace the transition to modern QA?

Greg and Ashley: You might have seen this article about getting your team onboard with automated testing.

Can you define a list of essential skills every modern QA team member should have?

Greg: We don’t have dedicated DevOps resources in a Platform-as-a-Service (PaaS), so we are experimenting with what the modern QA team member would need in terms of both QA and DevOps responsibilities. QA is the gatekeeper of quality, and it makes sense for the QA team to maintain and champion continuous integration tooling.

Ashley: I still maintain that QA engineers, whether they practice automation or not, are incredibly valuable in determining a holistic test strategy. You always need to be able to consider the end user, how the system works at a broader level than your team’s feature du jour, what other types of testing to consider (not just automated tests, but also accessibility, localization, performance, usability, security, exploratory). The key is figuring out how to incorporate that into your sprint and really push to the definition of done.

Since QA team members generally do not have a developer background, what route do you recommend to help them begin automated testing?

Greg and Ashley: In general, start with best practices. It doesn’t matter what your tool of choice is. Understand locators, the DOM, etc. Get familiar with Git. Start with something simple—Do a very simple update, commit changes, learn the process. Start fixing a broken test. Write a new test.

This blog post about test automation for newbies may be helpful.


A modern QA team looks very different from one a decade ago. The rise of test automation has helped blur the lines between developers and QA engineers. So has the DevOps revolution, and shift-left testing. These changes mean that acquiring and keeping QA talent requires organizations to integrate their QA teams closely with the rest of the software engineering staff. Encouraging automation is also important, but organizations should let QA professionals choose their own toolsets, and keep in mind that manual testing is still sometimes necessary.

Chris Riley (@HoardingInfo) is a technologist who has spent 12 years helping organizations transition from traditional development practices to a modern set of culture, processes and tooling. In addition to being a research analyst, he is an O’Reilly author, regular speaker, and subject matter expert in the areas of DevOps strategy and culture. Chris believes the biggest challenges faced in the tech market are not tools, but rather people and planning.

Ashley Hunsberger is a Quality Architect at Blackboard, Inc. and co-founder of Quality Element. She’s passionate about making an impact in education and loves coaching team members in product and client-focused quality practices. Most recently, she has focused on test strategy implementation and training, development process efficiencies, and preaching Test Driven Development to anyone that will listen. In her downtime, she loves to travel, read, quilt, hike, and spend time with her family.

Greg Sypolt (@gregsypolt) is a senior engineer at Gannett and co-founder of Quality Element. The last 5 years focused on creation and deployment of automated test strategies, frameworks, tools, and platforms.

Categories: Companies

How Big is Big

I caught a fish this big
“I caught a fish this big” by Mark Hanna

When we talk to Jenkins users, about half of the people we talk to will say something like:

“We have a really big Jenkins instance”

This is a statement that always intrigues me… quite often when we probe a little into exactly how big their Jenkins instance is, we find answers like “more than 500 jobs” and “five or ten build agents”. Now some people are probably laughing at the thought of such an instance as “really big”, but that is never my reaction.

Growing a Jenkins instance is, in some ways, very much like growing a company. When I started in CloudBees, there were very few employees… in fact we all were listed on the Company Team page… everyone knew everyone else.

When your Jenkins instance starts off, you only have a few jobs, maybe one or two build agents. Everyone knows what every job does. Everyone knows what every build agent is for.

After a while, the company grows… there are more people… the Company Team page no longer lists everyone… not because people have stopped being important to the company, but because those kinds of pages should not list all 60 or 70 people. It’s still a village though. Everyone knows everyone else to see, even if not personally.

In the same way, as the Jenkins instance grows, we see people start to use things like Views to customise what they see. Everyone knows who owns each of the jobs, even if they don’t know exactly what they are for. Everyone knows who owns the build agents, etc.

At some point in the growth of a company, it will transition from the “village” phase to the “town” phase. This is the hypothesised Dunbar’s number… which is rumoured to lie somewhere between 100 and 250. This can be a breaking point for some companies. Thankfully, the CloudBees executive team have all gone through this before and as soon as they put me back on the Company Team page any potential problems will have been resolved (joke!).

As you grow your Jenkins instance to the 500+ jobs range (which will typically require 10+ build agents) you start to hit the same issues that a growing company hits. No longer will Views solve organising all your jobs. You need to start putting jobs in folders. You probably will need naming conventions. You certainly will need more complex permissions systems. In short, you feel pain.

To grow your Jenkins instance from 500 to 1,000+ jobs will require you to start to solve some of these problems, and until you realise that there are various solutions to these problems, it can feel almost as if you have hit some fundamental limit of scalability for Jenkins.

So while 500 jobs or so and 10 build agents or so is not a big instance, to the person who has grown their instance to this point, it can feel really really big.

But what is a really big instance then?

Well obviously I have the answer. I researched the subject as part of the preparation for my Jenkins World 2016 talk: “So you want to build the worlds largest Jenkins cluster”.

Jenkins instances have a built in “phone home” facility that reports usage statistics to the Jenkins community home page. The reporting of statistics is anonymised, only includes some basic information and can also be turned off by the Jenkins administrator. But there are enough instances reporting that we can at least put a measure on actual really really big instances.

For the latest reported statistics (covering the month of August 2016):

  • There were 61 instances with more than 8,000 jobs. The instance with the largest number of jobs had 67,563 jobs.
  • There were 19 instances with more than 450 agents. The instance with the largest number of agents had 6,794 agents.

Those are considerably bigger numbers than 500 jobs or 10 build agents and getting to those sizes will have required a different management strategy from that used when initially growing.

  • The instances with many build agents probably have some sort of automation to create their build agents. The agents are probably all nearly identical and created from some sort of template image or recipe.
  • The instances with many jobs are probably using some sort of automation to create the jobs. This may be as simple as using something like the GitHub Organisation Folders plugin so that all the organisation’s GitHub repositories are scanned, with jobs being auto-created for each branch of each repository with a Jenkinsfile (as well as Pull Requests). Or it may be more complex automation using the CloudBees Templates plugin or the Job DSL plugin. But in any case, there are very few “special snowflake” jobs.

So, in summary, today in 2016 a “really big Jenkins instance” is probably one with either more than 10,000 jobs or more than 1,000 build agents.

Blog Categories: JenkinsCloud Platform
Categories: Companies

Get up-to-date on performance testing by attending these on-demand performance webinars

HP LoadRunner and Performance Center Blog - Thu, 09/22/2016 - 12:54

Performance Testing teaser.png

If you have missed the recent online webinars for Performance Center, LoadRunner, StormRunner Load and TruClient--don't panic.  You can now easily find them!

Categories: Companies

User Conference on Advanced Automated Testing (UCAAT), Budapest, Hungary, October 26-28 2016

Software Testing Magazine - Thu, 09/22/2016 - 09:00
The international ETSI User Conference on Advanced Automated Testing (UCAAT) is a software testing conference dedicated to advanced test automation. Jointly organized by ETSI Technical Committee “Methods for Testing and Specification” (TC MTS) and QualityMinds, a software testing company, the conference introduces the latest innovations made in test automation. In the agenda of the User Conference on Advanced Automated Testing (UCAAT) you can find topics like “Human Factors for Test Automation and Industrialization”, “Testing and Domain-Specific Modelling with TDL”, “Using Mobile Analytics to Improve Testing and Development Practice”, “Big Data Interpretation and Challenges in Mobile Network Testing”, “Continuous Testing Starts with the Smart Architectural Decisions”, “Streamlining Performance Verification Through Automation and Design”, “Automated Testing for Autonomous Cars? Challenges, Solutions, Opportunities”, “Modelling of Complex Distributed Test Scenarios”, “Model-Based Test Design at Unity”, “Model-Based Testing of Large-Scale Enterprise IT Systems”, “Test systems, software systems. Is there a difference?”, “Requirements and Challenges with Advanced Test Automation, The Industry Perspective”, “Model Based Testing: introduction of the methodology at”, “What’s wrong with Test Automation?”. Web site: Location for User Conference on Advanced Automated Testing (UCAAT): Larus Étterem és Rendezvényközpont, 1124 Budapest, Csörsz u. 18/b.
Categories: Communities

Giving 'Back

Hiccupps - James Thomas - Thu, 09/22/2016 - 06:51

The Test team book club at Linguamatics is currently reading What Did You Say? The Art of Giving and Receiving Feedback. Here's a couple of quotes that I picked out for our last session:
  • If you’re really interested in helping people, you’ll do well to start your feedback by opening your own motives to inspection.
  • Even when it’s given at the receiver’s request, feedback describes the giver more than the receiver.
  • When the data and their model don’t match, most people discard the data.

I recall an instance when, engaged in discussion with a colleague I'll call Russell, about the data analysis he was presenting, I spotted an opportunity to offer feedback. It was about something that I knew Russell wanted to change. It was about something that I knew was open to me to give feedback on, because we had talked about it. It was about something that I thought would be beneficial for Russell in multiple ways and, I hoped, would provide some insight into a particular behaviour pattern that he had.

However, it was also the first time that I had seen this particular thing. A data set of size one. I had no evidence, yet, that it would lead to the end point that Russell desired to alter. A data set of size zero.

Against this: my instinct, my gut, and my experience. And a sense of goodwill, built up over time, over repeated interactions, over sometimes difficult sessions where I had tried to demonstrate that I do care to assist and support and advise because I want to help Russell to be the best he can be, in the respects that matter to him and for his work.

But I was still cautious. I have unwittingly burned and been burned enough times over the years to know that each of these conversations carries with it risks. Risks of misreading the context, risks of misreading the agreements, risks of misreading the mood, risks, risks, risks, ...

But I went ahead anyway. The potential benefit and the goodwill in the bank outweighed the risks, I calculated, on this occasion. And I gave my feedback. And Russell agreed with me. And I breathed a deep internal sigh of relief.

Comparing this anecdote to the quotes I pulled from the book:
  • My motives, I think, were good: I wanted to help Russell achieve a personal goal.
  • But the feedback does reflect something about me: an interest in reducing unnecessary complexity, an interest in making presentation clear, the ego that is required to believe that my colleagues will want to listen to any advice from me, ...
  • In this case, it turned out my suggestion didn't contradict Russell's model but exposed it, and in any case I had little concrete data to present.

I use this episode as an example not because it ended well, particularly, but because it's an illustration for me of how much I have been influenced by What Did You Say? in the couple of years since I first read it. I consciously I go about my day-to-day business, doing my best to be careful about when I choose to offer feedback, about when I deliberately choose not to, and about picking up and picking up on any feedback that's coming my way in return.

I try to treat this as a testing task where I can, in the sense that I try hard to observe my own actions and the responses they generate, and I think about ways in which they might be related and how I might approach things differently in the next exchange, or at another time, with this person, or someone else.

Easier said than done, of course, so I'll finish with another quote from the book, another quote that I've taken to heart and act on, that regularly helps guide me with pretty much everything that I've said above:
Don’t concentrate on giving feedback; concentrate on being congruent–responding to the other person, to yourself, and to the here-and-now situation. Don’t go around hunting for opportunities to give feedback, because feedback is effective only when the need arises naturally out of congruent interactions.Some details have been changed.
Image: Leanpub

Categories: Blogs

Back to Blogging! - Thu, 09/22/2016 - 02:50

My blog has been offline for a long time, as you can see. The last prior post was in 2009!

Recently, I found a backup copy of the old blog and was able to re-establish it. Watch for some new posts in the near future.

Categories: Open Source

Testing on the Toilet: What Makes a Good End-to-End Test?

Google Testing Blog - Thu, 09/22/2016 - 00:59
by Adam Bender

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.

An end-to-end test tests your entire system from one end to the other, treating everything in between as a black box. End-to-end tests can catch bugs that manifest across your entire system. In addition to unit and integration tests, they are a critical part of a balanced testing diet, providing confidence about the health of your system in a near production state. Unfortunately, end-to-end tests are slower, more flaky, and more expensive to maintain than unit or integration tests. Consider carefully whether an end-to-end test is warranted, and if so, how best to write one.

Let's consider how an end-to-end test might work for the following "login flow":

In order to be cost effective, an end-to-end test should focus on aspects of your system that cannot be reliably evaluated with smaller tests, such as resource allocation, concurrency issues and API compatibility. More specifically:
  • For each important use case, there should be one corresponding end-to-end test. This should include one test for each important class of error. The goal is the keep your total end-to-end count low.
  • Be prepared to allocate at least one week a quarter per test to keep your end-to-end tests stable in the face of issues like slow and flaky dependencies or minor UI changes.
  • Focus your efforts on verifying overall system behavior instead of specific implementation details; for example, when testing login behavior, verify that the process succeeds independent of the exact messages or visual layouts, which may change frequently.
  • Make your end-to-end test easy to debug by providing an overview-level log file, documenting common test failure modes, and preserving all relevant system state information (e.g.: screenshots, database snapshots, etc.).
End-to-end tests also come with some important caveats:
  • System components that are owned by other teams may change unexpectedly, and break your tests. This increases overall maintenance cost, but can highlight incompatible changes
  • It may be more difficult to make an end-to-end test fully hermetic; leftover test data may alter future tests and/or production systems. Where possible keep your test data ephemeral.
  • An end-to-end test often necessitates multiple test doubles (fakes or stubs) for underlying dependencies; they can, however, have a high maintenance burden as they drift from the real implementations over time.
Categories: Blogs

Join Seapine at STARWEST 2016

The Seapine View - Wed, 09/21/2016 - 17:45

starwestCalling all STARWEST attendees! Join us in Anaheim this October for STARWEST 2016, one of the longest-running, and most respected conferences on software testing and quality assurance.

The event week features over 100 learning and networking opportunities and covers a wide variety of some of the most in-demand topics.

Come visit us at booth 10 during the Expo on Wednesday, October 5 and Thursday, October 6. Our team of experts will be on hand to answer your questions and show you how Seapine can help you manage your development and testing processes. We’re also raffling off an Amazon Fire TV Stick with Voice Remote on Thursday, October 6—don’t miss out on your chance to win!

Categories: Companies

From jUnit to Mutation-Testing

Testing TV - Wed, 09/21/2016 - 17:36
JUnit is a well-known tool for java developers in the area of TD where it is accepted that code coverage can be measured. In this case we distinguish between coverage on the level of classes, methods and rows. The goal is to get the code coverage as high as possible on the row level, but […]
Categories: Blogs

Oblique Testing

Software Testing Magazine - Wed, 09/21/2016 - 16:24
In theory, we can consider software testing as a very rationale approach. You start from unit of code or requirements and then you create the tests that will prove that your software does what it is expected to do… and doesn’t create problems with edge cases. In his book Oblique Testing, Mike Talks propose to add an additional perspective to software testing using the oblique strategies approach. The oblique strategies is a set of card originally used by music producer Brian Eno to make recording artists try something new. This book adapts this approach to the software testing world by providing a set of cards. Based on fictional bad reviews for apps., they are mainly focused on the testing of a mobile app, but they are generic enough to be applied to other contexts where you have many customers using your software. The book provides also some suggestions on how to apply this approach in Agile and traditional software development projects. Oblique Testing is an original approach to perform some exploratory testing where the full project team is involved and not only the software testers. I will suggest to read this very short book to every software tester or project manager that wants to introduce a different perspective in its software testing activities. Reference: Oblique Testing, Mike Talks, Quotes As I mentioned before, when we are testing, we’re using our imagination to work out “ways the software could go wrong”. The most annoying bugs are the ones which get [...]
Categories: Communities

BlazeMeter Acquired by CA Technologies

Software Testing Magazine - Wed, 09/21/2016 - 12:43
CA Technologies has announced it has signed a definitive agreement to acquire BlazeMeter, a provider of load testing services. Founded in 2011, BlazeMeter has offices in Tel Aviv, Israel and Palo Alto. The acquisition of privately-held BlazeMeter will enable CA Technologies to extend its DevOps portfolio. BlazeMeter will seamlessly integrate with CA’s continuous delivery solutions to further improve testing efficiency and accelerate the deployment of applications. BlazeMeter’s commercial, self-service continuous application performance testing solution is fully compatible with Apache JMeter™ as well as other open source software tools like Selenium, Gatling and Locust. If we examine the development of all companies acquired by CA Technologies, formerly Computer Associates, this mean that all the open source development of the JMeter ecosystem (plugins, integration with other testing tools, etc.) currently funded by Blazemeter will likely decrease if not just be abandoned.
Categories: Communities

Jenkins World 2016 Wrap-up - Introduction

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. That’s a Wrap! Any way you look at it, last week’s Jenkins World Conference 2016 was a huge success. In 2011, a few hundred users gathered in San Francisco for the first "Jenkins User Conference". Over successive years, this grew into several yearly regional Jenkins user conferences. This year, over 1,300 people came from around the world to "Jenkins World 2016", the first global event for the Jenkins community. This year’s Jenkins World conference included: Keynote presentation by Jenkins creator, Kohsuke Kawaguchi, announcing a number of great new Jenkins project features, such as "Blue Ocean". More than 50...
Categories: Open Source

CloudBees Jenkins Platform 2.7.19

We are happy to announce the immediate availability of CloudBees Jenkins Platform 2.7.19. This is the first version of the CloudBees Jenkins Platform based on Jenkins 2. The User eXperience of Jenkins has been dramatically revisited and you will benefit from the following improvements.

Improved User Experience Installation wizard

An installation wizard helps you to select the plugins that are relevant for your continuous delivery platform. CloudBees has selected a set of cohesive plugins to propose a default installation.

(Click on thumbnails to display larger image)

Revisited wizard to create new items

The screen to create new jobs and new items is much more intuitive. It helps Jenkins users to instantly find the type of job they need.

Revised job configuration screen

The Jenkins configuration pages for jobs and items have been revisited with tabs to help clarify the information.

Secured by default

The CloudBees Jenkins Platform is now secured by default. New instances are secured by an “Unlock Jenkins” screen and Jenkins admins are invited to enable security and create a first user on the system.

Better Installation and Upgrades Offline installation

The CloudBees Jenkins Platform can now be installed offline with a large and cohesive set of plugins available through the plugin selection wizard.

Better upgrades

Upgrades of Jenkins could be hectic in the past, due to issues with some plugins not getting upgraded. The root cause of the issue was a concept called “pinned” plugins. The concept behind pinned plugins has been retrofitted and plugins now get updated through the installation process.

Beekeeper Upgrade Assistant and the CloudBees Assurance Program

The CloudBees Jenkins Platform is now integrated with the CloudBees Assurance Program so that you can keep your servers up-to-date with plugin versions verified by CloudBees. We will provide details of the new Beekeeper Upgrade Assistant and the CloudBees Assurance Program in a follow-on blog post.

New Release Model to Get the Best Out of Your Platform

In addition to the CloudBees Assurance Program, CloudBees has adopted a Rolling Release Model and a continuous delivery approach to more efficiently deliver new features with smaller increments. Rolling releases of the CloudBees Jenkins Platform will be published regularly (multiple times per quarter) so that you will be able to more frequently apply smaller upgrades to your platform. Customers who prefer to use a very stable version, rather than benefiting from ongoing, smaller feature improvements, will be able to choose our fixed release that will be published yearly. The fixed release will limit changes during the year, and before the next fixed release, to security fixes and the correction of critical bugs.

Smooth upgrade path

To upgrade your CloudBees Jenkins Platform, you just perform the standard upgrade procedure that you already use for CJP 1.x (replacing the war file, installing the latest .rpm or .deb…) with one additional requirement: Start upgrading CloudBees Jenkins Operations Center and then upgrade the client masters. CloudBees Jenkins Platform 16.06 will remain supported until April 2017. More details are available on our Support Lifecycle and Update Policies page.

Getting Started

Visit our Getting Started page and try CloudBees Jenkins Platform V2!

Blog Categories: Jenkins
Categories: Companies

Accenture and Applause Form Crowdtesting Alliance

Software Testing Magazine - Tue, 09/20/2016 - 17:33
Accenture has formed an alliance with Applause, a provider of software testing tools and services, strengthening the capabilities and geographic scope of its own testing services. As part of the alliance relationship, Accenture Ventures also made a minority investment in Applause. The testing market is exploding as the number of devices, operating systems and applications keeps growing. In addition, countries have varied security requirements, privacy laws, and other regulations, making testing requirements more complex—creating an ideal scenario to tap into the testing capabilities provided by Applause’s global, diverse crowd of testers. The Applause crowd community features more than 250,000 experienced quality assurance testers around the world. They complement Accenture’s 35,000 testing professionals to provide an unmatched scope of testing services and geographic coverage. Accenture is already working with Applause to test, secure and monitor digital applications for a seamless customer experience across mobile devices, desktops, kiosks, smart TVs, and wearables. Working with Accenture and Applause, companies can also access specialized crowds of testers that match the profile of their users and customers. For example, the 60,000 women testers in the Applause crowd can be tapped to effectively test applications for products and services specifically targeted for women.
Categories: Communities

Selenium Load Testing Launched by BlazeMeter

Software Testing Magazine - Tue, 09/20/2016 - 17:15
BlazeMeter has launched new technology which enables users to execute Selenium scripts as load tests. Selenium users can run thousands of different nodes in the cloud for large scale load tests across multiple geographical locations. BlazeMeter has also launched converter transforms all Selenium scripts into JMeter. This innovative functionality is made possible by using BlazeMeter in conjunction with its new open source performance testing automation framework, Taurus. Taurus automates the execution of native Selenium tests locally and seamlessly switches into the BlazeMeter cloud to run the tests at a massive scale. All tests can be added as a step in the Jenkins pipeline, enabling Selenium users to include performance testing in the Continuous Integration cycle while also enjoying key features like enhanced reporting and collaboration. BlazeMeter has also released an additional new feature for the JMeter community – the Selenium Converter. The converter transforms all Selenium scripts into JMeter in less than ten minutes, eliminating the JMeter scripting step and dramatically reducing costs and test creation time from hours or days down to just minutes. It fully automates the process, once more facilitating the implementation of testing into the Jenkins CI pipeline.
Categories: Communities

Introducing UrbanCode MBeans!

IBM UrbanCode - Release And Deploy - Tue, 09/20/2016 - 15:59

The latest release of IBM UrbanCode Deploy version 6.2.2 comes with the ability to collect program-specific metrics. These metrics include application deployment duration and component version size. This information is useful in determining how your usage of UrbanCode Deploy has changed over time or in making informed decisions about using the product.

These metrics can be gathered by any monitoring tool that supports JMX MBean data collection such as IBM® Performance Management on Cloud or New Relic.

For usage details please the the linked documentation below. This documentation covers:
I. UCD Server Configuration
II. The Available MBean Metrics
III. How to View Mbeans using the JConsole
UrbanCode Deploy JMX MBean Documentation

Get more information about monitoring tools in these posts:

Categories: Companies

Announcing TestRail 5.3 – Agile Test Management, Resource Planning & Project Favorites

Gurock Software Blog - Tue, 09/20/2016 - 15:47


We are constantly working on new versions and improvements for TestRail, our modern test management tool, and we are happy to announce the release of TestRail 5.3 today! The new version adds various new agile test management features, better resource planning, improved forecasts and project favorites. Especially if you are managing larger and more complex projects with many releases, testing sprints and iterations, the new version will make it easier for your team to track and organize your tests and to make sure that everyone is focused on the next upcoming release.

TestRail has always been the favorite tool of many agile software testing teams for its fast and productive interface, great integration features with issue and project management tools as well the support for agile testing methods such as exploratory testing and session-based test management. With TestRail 5.3 we are adding more unique features for agile and traditional software teams to make it easier to plan their tests and track the project progress.

Besides many other improvements, the new version introduces sub milestones in TestRail to make it easier to track iterations, sprints and testing phases in addition to releases. We are also adding the capability to track the start and end dates for milestones and releases, added progress warnings if your team is not on track to meet project deadlines, we improved the forecast & progress reports and we updated the to-do lists to enable easier workload distribution. TestRail 5.3 also comes with new features to manage project favorites so users can better focus on their main projects from the dashboard pages. See below for detailed descriptions of the new features and learn how to start using the new version!

Try TestRail 5.3 Now

Get started with TestRail in minutes
and try TestRail free for 30 days!

Scheduling Milestones & Runs

With TestRail 5.3 we are introducing new features to make it easier to plan and track active, upcoming and completed milestones and related resources. You have always been able to set due dates for your milestones in TestRail so you can plan and control your testing progress easily. With the new version we are also adding options to plan the future start date of milestones (and thus related test runs), provide options to mark milestones as started and redesigned the Milestone tab to visualize your milestone roadmap.

Not only does this make it easier for your team to keep an eye on upcoming milestones and planned start dates, it also helps your team plan future test assignments, resources and upcoming test activities as you can already plan future test runs and assign tests separately from active releases. With the recorded start and due dates of the project milestones, TestRail also includes new warnings on Progress pages if your team is likely not able to meet the goal based on the current progress. Additionally, to make it easier to report on your test runs, new filter options for various built-in reports allow you to select test runs based on the completion date now.


Schedule upcoming releases & milestones and plan resources

Sub Milestones for Iterations, Sprints & Builds

TestRail’s rich project structure already makes it easy to manage projects, releases and iterations or sprints via milestones, test plans and test runs. Especially test plans are a great way to structure, group and start test runs and to test your projects against different platforms such as operating systems, web browsers, mobile devices or other hardware platforms and configurations. For now, most teams used test plans to manage their sprints, iterations and other testing phases that are part of a release in TestRail. With TestRail 5.3 we are introducing sub milestones to add another level to group and manage your test runs.

The new sub milestones allow you to add another layer of milestones from the Milestones tab. We specifically added this to make it easier to manage sub releases, iterations, agile sprints and other milestones of your projects. When you assign test runs to a sub milestone, the statistics of the parent milestone include and show the combined result of all your releases, iterations and sprints tracked as part of the milestone. The new sub milestone feature makes it much easier to plan and keep track of more complex roadmaps and milestone hierarchies in TestRail, and we also updated various filters and pages in TestRail to take advantage of the new milestone hierarchy.


Introducing sub milestones to manage sprints, iterations and sub releases

Updated To-Do Page for Resource Planning

Assigning tests to users is a great way to distribute tasks across the team so that everyone knows exactly what they should be working on. TestRail makes it easy to assign entire test runs, specific configurations or individual sections and tests to different team members. TestRail’s advanced reports with our workload summary report also makes it very easy to track the workload across the team so it’s easy to re-balance assigned tests depending on the testers’ progress.

A central piece of TestRail’s workflow and starting point for most testers is TestRail’s Todo page. TestRail’s Todo page combines multiple use cases and features in a central place: it makes it easy to track and filter progress and assigned tests for team leads, it makes it easy to change assignments between users to adjust the workload and it’s the best way for individual testers to see all their assigned tests for a project. For TestRail 5.3 we’ve improved the Todo page to color mark the assigned test chart for active, upcoming and completion-pending test runs so it’s easier to see the different types of tests grouped by release planning. We also improved the Todo page’s filters by adding new milestone filter options.


Updated Todo page to highlight active, upcoming and completion-pending tests

Project User Favorites

TestRail’s strong project management features already make it more efficient to work with a large number of projects and with huge teams with 1,000s of testers. You can already easily grant access and deny access to projects, make entire projects read-only, hide projects or archive them. This allows you already to configure which team members should be able to see and access specific projects, and you can easily assign access and permissions on a per user and project basis. Users also have the option to view the project list in different view modes to make it easier to work with a larger number of projects.

With TestRail 5.3 we are introducing a new useful way for users to keep track of projects and to focus on the main projects a user is interested in: project favorites. With project favorites users can easily mark (star) their main projects so they are highlighted in a separate list at the top of the project overview dashboard. This not only makes it easier and faster for users to access their favorite projects, it also gives more options to individual users to organize their dashboard page based on their current tasks.


Helping testers keep track of important projects with project favorites

Additional Improvements

TestRail 5.3 also comes with various additional new features, improvements and productivity enhancements. We’ve listed some of the additional enhancements below and please see our full changelog for a complete list of changes.

Improved Forecast Calculation

With TestRail’s new resource planning features, we’ve also updated the forecast calculations for the Progress reports to take the new milestone start dates into account. TestRail also falls back to the first test results for more accurate forecasts. Updated API Methods

We also constantly review and update TestRail’s API for additional API capabilities and useful attributes in the result sets. For TestRail 5.3 we’ve updated the API to return more attributes for the test plan methods among other improvements.

Project Progress Warnings

TestRail’s unique Progress reports and forecasts always helped teams track and compare their testing efforts. TestRail 5.3 now also compares the forecast with the milestone targets and issues a warning if they don’t match. Easier Integration Troubleshooting

Most teams integrate and use TestRail together with an issue and bug tracking tool such as JIRA, Redmine, FogBugz etc. With TestRail 5.3 we’ve added new warning messages to make it easier to troubleshoot project-level integration settings.

Using & Upgrading to TestRail 5.3

Upgrading to TestRail 5.3 is easy and we recommend upgrading to benefit from the new resource planning and agile management features such as sub milestones. We’ve included all the required details below to get TestRail 5.3 up and running, depending on the edition you use:

  • TestRail Cloud: your account has already been updated!
  • TestRail Server (licensed): you can download the latest version or renew your support plan from your customer portal account.
  • TestRail Server (trial): please contact us to upgrade your download trial.
  • New user: want to try TestRail? Get a free trial.

You can also review the full change log to learn more about all new features, improvements and bug fixes included in TestRail 5.3. If you have any questions or feedback about the new version, please let us know!

Try TestRail 5.3 Now

Get started with TestRail in minutes
and try TestRail free for 30 days!

Categories: Companies

The Importance of Eliminating Network Hops

Sauce Labs - Tue, 09/20/2016 - 15:00

Are you experiencing slower execution times while running Selenium scripts in the Selenium cloud network? Too many network hops will add latency and slow down your test execution. Plus, every additional network hop adds cost to your execution. One way to optimize Selenium execution performance is to eliminate as many network hops as possible.

“A hop is one portion of the path between source and destination. Data packets pass through bridges, routers and gateways on the way. Each time packets are passed to the next device, a hop occurs.”1)“Hop (networking) – Wikipedia, the free encyclopedia.” 2011. 26 Jan. 2016 jQuery("#footnote_plugin_tooltip_6740_1").tooltip({ tip: "#footnote_plugin_tooltip_text_6740_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The Sauce Labs team is working on a new tool that may be interesting to you. The solution will eliminate a network hop, making running tests in parallel on multiple browsers automatic and simple.

Why Hops Matter

In my first time using a cloud-based solution for Selenium, I was both thrilled and impressed by the possibilities. A cloud-based provider such as Sauce Labs maintains and uses configuration management tools to spin up and tear down virtual machines with various OS platforms and browsers. I understand the value in this type of service. The amount of time and resources needed to build and maintain infrastructure is expensive. Been there, done that!

The downside of testing in the cloud is that you are sending commands outside your network through the cloud and back. It adds another device between source and destination, which adds latency and an additional network hop. The cloud-based approach will slow down test execution when comparing it to local execution from a developer machine.

The Solution

One of the benefits of being a writer for and client of Sauce Labs is the ability to leverage new tools in beta. One of the newest tools in beta is called Sauce Runner. This new tool is designed to make running tests in parallel on multiple platforms and browsers fast and easy. Sauce Runner facilitates server-side execution of your Selenium scripts, reducing latency, eliminating network hops, and it allows Sauce Labs to automatically handle things like test parallelization and platform selection.



What are the prerequisites to use Sauce Runner?

  • GitHub or Bitbucket repository using the supported frameworks
  • Add a deployment key to your private repository (only for private repos)
  • YAML configuration file, placed in your repository — Your repo is ready to run with Sauce Runner

If your existing repo is using one of the supported frameworks, the configuration will take less than 5 minutes to set up to use the Sauce Runner.

The screenshot below demonstrates a fast and easy way to create a test suite by using Sauce Runner:


Sauce Runner integrates with a few Continuous Integration platforms such as Jenkins, Bamboo, and TeamCity. Once you’ve created your test suite, you’re able to copy the REST Curl command from the test suite details page, then add it to your desired Continuous Integration platform.

Yes, the tool may be geared toward newbies, but I am more excited about the server-side execution since it will reduce latency issues and eliminate network hops. By leveraging the server-side execution of your Selenium script, it will reduce build time by as much as 10%.

Takeaways: Why you should try Sauce Runner
  • Server-side execution
  • Improve build speed and reliability
  • Eliminate network hops and reduce latency for remote execution
  • Simplifies the onboarding process for new Sauce Labs clients
  • Supports Continuous Integration platforms
  • No configuration needed to use parallel testing
  • As a beta user, you can help test and submit feature requests


(1)“Hop (networking) – Wikipedia, the free encyclopedia.” 2011. 26 Jan. 2016

Greg Sypolt (@gregsypolt) is a senior engineer at Gannett and co-founder of Quality Element. He is a passionate automation engineer seeking to optimize software development quality, while coaching team members on how to write great automation scripts and helping the testing community become better testers. Greg has spent most of his career working on software quality – concentrating on web browsers, APIs, and mobile. For the past five years, he has focused on the creation and deployment of automated test strategies, frameworks, tools and platforms.

References   [ + ]

1. ↑ “Hop (networking) – Wikipedia, the free encyclopedia.” 2011. 26 Jan. 2016 function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }
Categories: Companies

Even Messi sometimes needs to start from scratch

PractiTest - Tue, 09/20/2016 - 14:01

I am not a big soccer fan.  Even though I grew up in Costa Rica, where futbol (as they call it there) is something everyone needs to know and play, I was never a good player and so I’ve never really enjoyed seeing the game.

This does not mean that I don’t get to follow and even learn from some of the players.

For example, I had the pleasure of sitting with my oldest son next to Keylor Navas (goalkeeper for Real Madrid & Costa Rica’s national team) on a plane trip last year.  And there I was able to see how someone who is a renowned figure can still be kind and patient with every single person in the plane who came to take a picture with him, and then (once we finally got to be airborne) he was placid enough to play with the 1-year old toddler of the lady sitting next to him – whom he did not know.

The guy with the suicide blond hair

messi rubio goes blondBut today I want to talk about Leo Messi and his hair color.

In case you live under a rock, Leo (Lionel) Messi is one of the 2 top players in the world today (I won’t enter into the discussion if he is slightly better or worse than Cristiano Ronaldo!) and some weeks ago he surprised the world by dying his hard blond!

I mean, crazy rock stars do that all the time, but if you follow Messi you know this does not really fit him…

Well, he finally explained why he did it, and actually it made perfect sense!

Sometimes you need external changes to drive an internal change

On a satirical interview Messi provided a simple and good explanation on why he colored his hair.

In case you are not good at Spanish, he said that he wanted to start from “zero” (as we sometimes say, start from scratch) and so he choose to make a change.

For those of you who do not follow him, lately Messi has been in a small slump where among other things he ran into issues with the law in Spain for tax issues, and he also blew up a penalty shot that cost his National Team the title for the Copa America – so much so that for a while he even quit the National Team.

Apparently there were other things also bringing him down, and so he decided he needed to make a radical change, to start from scratch, and this made him go and dye his hair blond.

Changes need to start from within, but a little help from outside can also help!

I am no Messi (not even in the field of Testing) but I can understand 100% why he did this, as sometimes I do this myself…

I have not yet dyed my hair blond, I am afraid that if I do this the little hair I have left will finally decide to leave me altogether, but I do sometimes change my annotations’ notebook, or re-position my desk differently in the room, or simply change my morning habits in subtle ways that only I can notice.  And I do this in order to signal to myself that today is a new start, and I can leave all the stuff that was bringing me down literally behind me.

I know that this should not have any effect other than psychological in the way I behave.  But you know what?  Sometimes all the things that are actually bringing us down reside ONLY in our heads, and so this cheap psychological trick is exactly what I need

Categories: Companies

Knowledge Sharing

SpiraTest is the most powerful and affordable test management solution on the market today