Skip to content

Feed aggregator

Get up-to-date on performance testing by attending these on-demand performance webinars

HP LoadRunner and Performance Center Blog - Thu, 09/22/2016 - 12:54

Performance Testing teaser.png

If you have missed the recent online webinars for Performance Center, LoadRunner, StormRunner Load and TruClient--don't panic.  You can now easily find them!

Categories: Companies

User Conference on Advanced Automated Testing (UCAAT), Budapest, Hungary, October 26-28 2016

Software Testing Magazine - Thu, 09/22/2016 - 09:00
The international ETSI User Conference on Advanced Automated Testing (UCAAT) is a software testing conference dedicated to advanced test automation. Jointly organized by ETSI Technical Committee “Methods for Testing and Specification” (TC MTS) and QualityMinds, a software testing company, the conference introduces the latest innovations made in test automation. In the agenda of the User Conference on Advanced Automated Testing (UCAAT) you can find topics like “Human Factors for Test Automation and Industrialization”, “Testing and Domain-Specific Modelling with TDL”, “Using Mobile Analytics to Improve Testing and Development Practice”, “Big Data Interpretation and Challenges in Mobile Network Testing”, “Continuous Testing Starts with the Smart Architectural Decisions”, “Streamlining Performance Verification Through Automation and Design”, “Automated Testing for Autonomous Cars? Challenges, Solutions, Opportunities”, “Modelling of Complex Distributed Test Scenarios”, “Model-Based Test Design at Unity”, “Model-Based Testing of Large-Scale Enterprise IT Systems”, “Test systems, software systems. Is there a difference?”, “Requirements and Challenges with Advanced Test Automation, The Industry Perspective”, “Model Based Testing: introduction of the methodology at”, “What’s wrong with Test Automation?”. Web site: Location for User Conference on Advanced Automated Testing (UCAAT): Larus Étterem és Rendezvényközpont, 1124 Budapest, Csörsz u. 18/b.
Categories: Communities

Giving 'Back

Hiccupps - James Thomas - Thu, 09/22/2016 - 06:51

The Test team book club at Linguamatics is currently reading What Did You Say? The Art of Giving and Receiving Feedback. Here's a couple of quotes that I picked out for our last session:
  • If you’re really interested in helping people, you’ll do well to start your feedback by opening your own motives to inspection.
  • Even when it’s given at the receiver’s request, feedback describes the giver more than the receiver.
  • When the data and their model don’t match, most people discard the data.

I recall an instance when, engaged in discussion with a colleague I'll call Russell, about the data analysis he was presenting, I spotted an opportunity to offer feedback. It was about something that I knew Russell wanted to change. It was about something that I knew was open to me to give feedback on, because we had talked about it. It was about something that I thought would be beneficial for Russell in multiple ways and, I hoped, would provide some insight into a particular behaviour pattern that he had.

However, it was also the first time that I had seen this particular thing. A data set of size one. I had no evidence, yet, that it would lead to the end point that Russell desired to alter. A data set of size zero.

Against this: my instinct, my gut, and my experience. And a sense of goodwill, built up over time, over repeated interactions, over sometimes difficult sessions where I had tried to demonstrate that I do care to assist and support and advise because I want to help Russell to be the best he can be, in the respects that matter to him and for his work.

But I was still cautious. I have unwittingly burned and been burned enough times over the years to know that each of these conversations carries with it risks. Risks of misreading the context, risks of misreading the agreements, risks of misreading the mood, risks, risks, risks, ...

But I went ahead anyway. The potential benefit and the goodwill in the bank outweighed the risks, I calculated, on this occasion. And I gave my feedback. And Russell agreed with me. And I breathed a deep internal sigh of relief.

Comparing this anecdote to the quotes I pulled from the book:
  • My motives, I think, were good: I wanted to help Russell achieve a personal goal.
  • But the feedback does reflect something about me: an interest in reducing unnecessary complexity, an interest in making presentation clear, the ego that is required to believe that my colleagues will want to listen to any advice from me, ...
  • In this case, it turned out my suggestion didn't contradict Russell's model but exposed it, and in any case I had little concrete data to present.

I use this episode as an example not because it ended well, particularly, but because it's an illustration for me of how much I have been influenced by What Did You Say? in the couple of years since I first read it. I consciously I go about my day-to-day business, doing my best to be careful about when I choose to offer feedback, about when I deliberately choose not to, and about picking up and picking up on any feedback that's coming my way in return.

I try to treat this as a testing task where I can, in the sense that I try hard to observe my own actions and the responses they generate, and I think about ways in which they might be related and how I might approach things differently in the next exchange, or at another time, with this person, or someone else.

Easier said than done, of course, so I'll finish with another quote from the book, another quote that I've taken to heart and act on, that regularly helps guide me with pretty much everything that I've said above:
Don’t concentrate on giving feedback; concentrate on being congruent–responding to the other person, to yourself, and to the here-and-now situation. Don’t go around hunting for opportunities to give feedback, because feedback is effective only when the need arises naturally out of congruent interactions.Some details have been changed.
Image: Leanpub

Categories: Blogs

Back to Blogging! - Thu, 09/22/2016 - 02:50

My blog has been offline for a long time, as you can see. The last prior post was in 2009!

Recently, I found a backup copy of the old blog and was able to re-establish it. Watch for some new posts in the near future.

Categories: Open Source

Testing on the Toilet: What Makes a Good End-to-End Test?

Google Testing Blog - Thu, 09/22/2016 - 00:59
by Adam Bender

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.

An end-to-end test tests your entire system from one end to the other, treating everything in between as a black box. End-to-end tests can catch bugs that manifest across your entire system. In addition to unit and integration tests, they are a critical part of a balanced testing diet, providing confidence about the health of your system in a near production state. Unfortunately, end-to-end tests are slower, more flaky, and more expensive to maintain than unit or integration tests. Consider carefully whether an end-to-end test is warranted, and if so, how best to write one.

Let's consider how an end-to-end test might work for the following "login flow":

In order to be cost effective, an end-to-end test should focus on aspects of your system that cannot be reliably evaluated with smaller tests, such as resource allocation, concurrency issues and API compatibility. More specifically:
  • For each important use case, there should be one corresponding end-to-end test. This should include one test for each important class of error. The goal is the keep your total end-to-end count low.
  • Be prepared to allocate at least one week a quarter per test to keep your end-to-end tests stable in the face of issues like slow and flaky dependencies or minor UI changes.
  • Focus your efforts on verifying overall system behavior instead of specific implementation details; for example, when testing login behavior, verify that the process succeeds independent of the exact messages or visual layouts, which may change frequently.
  • Make your end-to-end test easy to debug by providing an overview-level log file, documenting common test failure modes, and preserving all relevant system state information (e.g.: screenshots, database snapshots, etc.).
End-to-end tests also come with some important caveats:
  • System components that are owned by other teams may change unexpectedly, and break your tests. This increases overall maintenance cost, but can highlight incompatible changes
  • It may be more difficult to make an end-to-end test fully hermetic; leftover test data may alter future tests and/or production systems. Where possible keep your test data ephemeral.
  • An end-to-end test often necessitates multiple test doubles (fakes or stubs) for underlying dependencies; they can, however, have a high maintenance burden as they drift from the real implementations over time.
Categories: Blogs

Join Seapine at STARWEST 2016

The Seapine View - Wed, 09/21/2016 - 17:45

starwestCalling all STARWEST attendees! Join us in Anaheim this October for STARWEST 2016, one of the longest-running, and most respected conferences on software testing and quality assurance.

The event week features over 100 learning and networking opportunities and covers a wide variety of some of the most in-demand topics.

Come visit us at booth 10 during the Expo on Wednesday, October 5 and Thursday, October 6. Our team of experts will be on hand to answer your questions and show you how Seapine can help you manage your development and testing processes. We’re also raffling off an Amazon Fire TV Stick with Voice Remote on Thursday, October 6—don’t miss out on your chance to win!

Categories: Companies

From jUnit to Mutation-Testing

Testing TV - Wed, 09/21/2016 - 17:36
JUnit is a well-known tool for java developers in the area of TD where it is accepted that code coverage can be measured. In this case we distinguish between coverage on the level of classes, methods and rows. The goal is to get the code coverage as high as possible on the row level, but […]
Categories: Blogs

Oblique Testing

Software Testing Magazine - Wed, 09/21/2016 - 16:24
In theory, we can consider software testing as a very rationale approach. You start from unit of code or requirements and then you create the tests that will prove that your software does what it is expected to do… and doesn’t create problems with edge cases. In his book Oblique Testing, Mike Talks propose to add an additional perspective to software testing using the oblique strategies approach. The oblique strategies is a set of card originally used by music producer Brian Eno to make recording artists try something new. This book adapts this approach to the software testing world by providing a set of cards. Based on fictional bad reviews for apps., they are mainly focused on the testing of a mobile app, but they are generic enough to be applied to other contexts where you have many customers using your software. The book provides also some suggestions on how to apply this approach in Agile and traditional software development projects. Oblique Testing is an original approach to perform some exploratory testing where the full project team is involved and not only the software testers. I will suggest to read this very short book to every software tester or project manager that wants to introduce a different perspective in its software testing activities. Reference: Oblique Testing, Mike Talks, Quotes As I mentioned before, when we are testing, we’re using our imagination to work out “ways the software could go wrong”. The most annoying bugs are the ones which get [...]
Categories: Communities

BlazeMeter Acquired by CA Technologies

Software Testing Magazine - Wed, 09/21/2016 - 12:43
CA Technologies has announced it has signed a definitive agreement to acquire BlazeMeter, a provider of load testing services. Founded in 2011, BlazeMeter has offices in Tel Aviv, Israel and Palo Alto. The acquisition of privately-held BlazeMeter will enable CA Technologies to extend its DevOps portfolio. BlazeMeter will seamlessly integrate with CA’s continuous delivery solutions to further improve testing efficiency and accelerate the deployment of applications. BlazeMeter’s commercial, self-service continuous application performance testing solution is fully compatible with Apache JMeter™ as well as other open source software tools like Selenium, Gatling and Locust. If we examine the development of all companies acquired by CA Technologies, formerly Computer Associates, this mean that all the open source development of the JMeter ecosystem (plugins, integration with other testing tools, etc.) currently funded by Blazemeter will likely decrease if not just be abandoned.
Categories: Communities

Jenkins World 2016 Wrap-up - Introduction

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. That’s a Wrap! Any way you look at it, last week’s Jenkins World Conference 2016 was a huge success. In 2011, a few hundred users gathered in San Francisco for the first "Jenkins User Conference". Over successive years, this grew into several yearly regional Jenkins user conferences. This year, over 1,300 people came from around the world to "Jenkins World 2016", the first global event for the Jenkins community. This year’s Jenkins World conference included: Keynote presentation by Jenkins creator, Kohsuke Kawaguchi, announcing a number of great new Jenkins project features, such as "Blue Ocean". More than 50...
Categories: Open Source

CloudBees Jenkins Platform 2.7.19

We are happy to announce the immediate availability of CloudBees Jenkins Platform 2.7.19. This is the first version of the CloudBees Jenkins Platform based on Jenkins 2. The User eXperience of Jenkins has been dramatically revisited and you will benefit from the following improvements.

Improved User Experience Installation wizard

An installation wizard helps you to select the plugins that are relevant for your continuous delivery platform. CloudBees has selected a set of cohesive plugins to propose a default installation.

(Click on thumbnails to display larger image)

Revisited wizard to create new items

The screen to create new jobs and new items is much more intuitive. It helps Jenkins users to instantly find the type of job they need.

Revised job configuration screen

The Jenkins configuration pages for jobs and items have been revisited with tabs to help clarify the information.

Secured by default

The CloudBees Jenkins Platform is now secured by default. New instances are secured by an “Unlock Jenkins” screen and Jenkins admins are invited to enable security and create a first user on the system.

Better Installation and Upgrades Offline installation

The CloudBees Jenkins Platform can now be installed offline with a large and cohesive set of plugins available through the plugin selection wizard.

Better upgrades

Upgrades of Jenkins could be hectic in the past, due to issues with some plugins not getting upgraded. The root cause of the issue was a concept called “pinned” plugins. The concept behind pinned plugins has been retrofitted and plugins now get updated through the installation process.

Beekeeper Upgrade Assistant and the CloudBees Assurance Program

The CloudBees Jenkins Platform is now integrated with the CloudBees Assurance Program so that you can keep your servers up-to-date with plugin versions verified by CloudBees. We will provide details of the new Beekeeper Upgrade Assistant and the CloudBees Assurance Program in a follow-on blog post.

New Release Model to Get the Best Out of Your Platform

In addition to the CloudBees Assurance Program, CloudBees has adopted a Rolling Release Model and a continuous delivery approach to more efficiently deliver new features with smaller increments. Rolling releases of the CloudBees Jenkins Platform will be published regularly (multiple times per quarter) so that you will be able to more frequently apply smaller upgrades to your platform. Customers who prefer to use a very stable version, rather than benefiting from ongoing, smaller feature improvements, will be able to choose our fixed release that will be published yearly. The fixed release will limit changes during the year, and before the next fixed release, to security fixes and the correction of critical bugs.

Smooth upgrade path

To upgrade your CloudBees Jenkins Platform, you just perform the standard upgrade procedure that you already use for CJP 1.x (replacing the war file, installing the latest .rpm or .deb…) with one additional requirement: Start upgrading CloudBees Jenkins Operations Center and then upgrade the client masters. CloudBees Jenkins Platform 16.06 will remain supported until April 2017. More details are available on our Support Lifecycle and Update Policies page.

Getting Started

Visit our Getting Started page and try CloudBees Jenkins Platform V2!

Blog Categories: Jenkins
Categories: Companies

Accenture and Applause Form Crowdtesting Alliance

Software Testing Magazine - Tue, 09/20/2016 - 17:33
Accenture has formed an alliance with Applause, a provider of software testing tools and services, strengthening the capabilities and geographic scope of its own testing services. As part of the alliance relationship, Accenture Ventures also made a minority investment in Applause. The testing market is exploding as the number of devices, operating systems and applications keeps growing. In addition, countries have varied security requirements, privacy laws, and other regulations, making testing requirements more complex—creating an ideal scenario to tap into the testing capabilities provided by Applause’s global, diverse crowd of testers. The Applause crowd community features more than 250,000 experienced quality assurance testers around the world. They complement Accenture’s 35,000 testing professionals to provide an unmatched scope of testing services and geographic coverage. Accenture is already working with Applause to test, secure and monitor digital applications for a seamless customer experience across mobile devices, desktops, kiosks, smart TVs, and wearables. Working with Accenture and Applause, companies can also access specialized crowds of testers that match the profile of their users and customers. For example, the 60,000 women testers in the Applause crowd can be tapped to effectively test applications for products and services specifically targeted for women.
Categories: Communities

Selenium Load Testing Launched by BlazeMeter

Software Testing Magazine - Tue, 09/20/2016 - 17:15
BlazeMeter has launched new technology which enables users to execute Selenium scripts as load tests. Selenium users can run thousands of different nodes in the cloud for large scale load tests across multiple geographical locations. BlazeMeter has also launched converter transforms all Selenium scripts into JMeter. This innovative functionality is made possible by using BlazeMeter in conjunction with its new open source performance testing automation framework, Taurus. Taurus automates the execution of native Selenium tests locally and seamlessly switches into the BlazeMeter cloud to run the tests at a massive scale. All tests can be added as a step in the Jenkins pipeline, enabling Selenium users to include performance testing in the Continuous Integration cycle while also enjoying key features like enhanced reporting and collaboration. BlazeMeter has also released an additional new feature for the JMeter community – the Selenium Converter. The converter transforms all Selenium scripts into JMeter in less than ten minutes, eliminating the JMeter scripting step and dramatically reducing costs and test creation time from hours or days down to just minutes. It fully automates the process, once more facilitating the implementation of testing into the Jenkins CI pipeline.
Categories: Communities

Introducing UrbanCode MBeans!

IBM UrbanCode - Release And Deploy - Tue, 09/20/2016 - 15:59

The latest release of IBM UrbanCode Deploy version 6.2.2 comes with the ability to collect program-specific metrics. These metrics include application deployment duration and component version size. This information is useful in determining how your usage of UrbanCode Deploy has changed over time or in making informed decisions about using the product.

These metrics can be gathered by any monitoring tool that supports JMX MBean data collection such as IBM® Performance Management on Cloud or New Relic.

For usage details please the the linked documentation below. This documentation covers:
I. UCD Server Configuration
II. The Available MBean Metrics
III. How to View Mbeans using the JConsole
UrbanCode Deploy JMX MBean Documentation

Get more information about monitoring tools in these posts:

Categories: Companies

Announcing TestRail 5.3 – Agile Test Management, Resource Planning & Project Favorites

Gurock Software Blog - Tue, 09/20/2016 - 15:47


We are constantly working on new versions and improvements for TestRail, our modern test management tool, and we are happy to announce the release of TestRail 5.3 today! The new version adds various new agile test management features, better resource planning, improved forecasts and project favorites. Especially if you are managing larger and more complex projects with many releases, testing sprints and iterations, the new version will make it easier for your team to track and organize your tests and to make sure that everyone is focused on the next upcoming release.

TestRail has always been the favorite tool of many agile software testing teams for its fast and productive interface, great integration features with issue and project management tools as well the support for agile testing methods such as exploratory testing and session-based test management. With TestRail 5.3 we are adding more unique features for agile and traditional software teams to make it easier to plan their tests and track the project progress.

Besides many other improvements, the new version introduces sub milestones in TestRail to make it easier to track iterations, sprints and testing phases in addition to releases. We are also adding the capability to track the start and end dates for milestones and releases, added progress warnings if your team is not on track to meet project deadlines, we improved the forecast & progress reports and we updated the to-do lists to enable easier workload distribution. TestRail 5.3 also comes with new features to manage project favorites so users can better focus on their main projects from the dashboard pages. See below for detailed descriptions of the new features and learn how to start using the new version!

Try TestRail 5.3 Now

Get started with TestRail in minutes
and try TestRail free for 30 days!

Scheduling Milestones & Runs

With TestRail 5.3 we are introducing new features to make it easier to plan and track active, upcoming and completed milestones and related resources. You have always been able to set due dates for your milestones in TestRail so you can plan and control your testing progress easily. With the new version we are also adding options to plan the future start date of milestones (and thus related test runs), provide options to mark milestones as started and redesigned the Milestone tab to visualize your milestone roadmap.

Not only does this make it easier for your team to keep an eye on upcoming milestones and planned start dates, it also helps your team plan future test assignments, resources and upcoming test activities as you can already plan future test runs and assign tests separately from active releases. With the recorded start and due dates of the project milestones, TestRail also includes new warnings on Progress pages if your team is likely not able to meet the goal based on the current progress. Additionally, to make it easier to report on your test runs, new filter options for various built-in reports allow you to select test runs based on the completion date now.


Schedule upcoming releases & milestones and plan resources

Sub Milestones for Iterations, Sprints & Builds

TestRail’s rich project structure already makes it easy to manage projects, releases and iterations or sprints via milestones, test plans and test runs. Especially test plans are a great way to structure, group and start test runs and to test your projects against different platforms such as operating systems, web browsers, mobile devices or other hardware platforms and configurations. For now, most teams used test plans to manage their sprints, iterations and other testing phases that are part of a release in TestRail. With TestRail 5.3 we are introducing sub milestones to add another level to group and manage your test runs.

The new sub milestones allow you to add another layer of milestones from the Milestones tab. We specifically added this to make it easier to manage sub releases, iterations, agile sprints and other milestones of your projects. When you assign test runs to a sub milestone, the statistics of the parent milestone include and show the combined result of all your releases, iterations and sprints tracked as part of the milestone. The new sub milestone feature makes it much easier to plan and keep track of more complex roadmaps and milestone hierarchies in TestRail, and we also updated various filters and pages in TestRail to take advantage of the new milestone hierarchy.


Introducing sub milestones to manage sprints, iterations and sub releases

Updated To-Do Page for Resource Planning

Assigning tests to users is a great way to distribute tasks across the team so that everyone knows exactly what they should be working on. TestRail makes it easy to assign entire test runs, specific configurations or individual sections and tests to different team members. TestRail’s advanced reports with our workload summary report also makes it very easy to track the workload across the team so it’s easy to re-balance assigned tests depending on the testers’ progress.

A central piece of TestRail’s workflow and starting point for most testers is TestRail’s Todo page. TestRail’s Todo page combines multiple use cases and features in a central place: it makes it easy to track and filter progress and assigned tests for team leads, it makes it easy to change assignments between users to adjust the workload and it’s the best way for individual testers to see all their assigned tests for a project. For TestRail 5.3 we’ve improved the Todo page to color mark the assigned test chart for active, upcoming and completion-pending test runs so it’s easier to see the different types of tests grouped by release planning. We also improved the Todo page’s filters by adding new milestone filter options.


Updated Todo page to highlight active, upcoming and completion-pending tests

Project User Favorites

TestRail’s strong project management features already make it more efficient to work with a large number of projects and with huge teams with 1,000s of testers. You can already easily grant access and deny access to projects, make entire projects read-only, hide projects or archive them. This allows you already to configure which team members should be able to see and access specific projects, and you can easily assign access and permissions on a per user and project basis. Users also have the option to view the project list in different view modes to make it easier to work with a larger number of projects.

With TestRail 5.3 we are introducing a new useful way for users to keep track of projects and to focus on the main projects a user is interested in: project favorites. With project favorites users can easily mark (star) their main projects so they are highlighted in a separate list at the top of the project overview dashboard. This not only makes it easier and faster for users to access their favorite projects, it also gives more options to individual users to organize their dashboard page based on their current tasks.


Helping testers keep track of important projects with project favorites

Additional Improvements

TestRail 5.3 also comes with various additional new features, improvements and productivity enhancements. We’ve listed some of the additional enhancements below and please see our full changelog for a complete list of changes.

Improved Forecast Calculation

With TestRail’s new resource planning features, we’ve also updated the forecast calculations for the Progress reports to take the new milestone start dates into account. TestRail also falls back to the first test results for more accurate forecasts. Updated API Methods

We also constantly review and update TestRail’s API for additional API capabilities and useful attributes in the result sets. For TestRail 5.3 we’ve updated the API to return more attributes for the test plan methods among other improvements.

Project Progress Warnings

TestRail’s unique Progress reports and forecasts always helped teams track and compare their testing efforts. TestRail 5.3 now also compares the forecast with the milestone targets and issues a warning if they don’t match. Easier Integration Troubleshooting

Most teams integrate and use TestRail together with an issue and bug tracking tool such as JIRA, Redmine, FogBugz etc. With TestRail 5.3 we’ve added new warning messages to make it easier to troubleshoot project-level integration settings.

Using & Upgrading to TestRail 5.3

Upgrading to TestRail 5.3 is easy and we recommend upgrading to benefit from the new resource planning and agile management features such as sub milestones. We’ve included all the required details below to get TestRail 5.3 up and running, depending on the edition you use:

  • TestRail Cloud: your account has already been updated!
  • TestRail Server (licensed): you can download the latest version or renew your support plan from your customer portal account.
  • TestRail Server (trial): please contact us to upgrade your download trial.
  • New user: want to try TestRail? Get a free trial.

You can also review the full change log to learn more about all new features, improvements and bug fixes included in TestRail 5.3. If you have any questions or feedback about the new version, please let us know!

Try TestRail 5.3 Now

Get started with TestRail in minutes
and try TestRail free for 30 days!

Categories: Companies

The Importance of Eliminating Network Hops

Sauce Labs - Tue, 09/20/2016 - 15:00

Are you experiencing slower execution times while running Selenium scripts in the Selenium cloud network? Too many network hops will add latency and slow down your test execution. Plus, every additional network hop adds cost to your execution. One way to optimize Selenium execution performance is to eliminate as many network hops as possible.

“A hop is one portion of the path between source and destination. Data packets pass through bridges, routers and gateways on the way. Each time packets are passed to the next device, a hop occurs.”1)“Hop (networking) – Wikipedia, the free encyclopedia.” 2011. 26 Jan. 2016 jQuery("#footnote_plugin_tooltip_7236_1").tooltip({ tip: "#footnote_plugin_tooltip_text_7236_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The Sauce Labs team is working on a new tool that may be interesting to you. The solution will eliminate a network hop, making running tests in parallel on multiple browsers automatic and simple.

Why Hops Matter

In my first time using a cloud-based solution for Selenium, I was both thrilled and impressed by the possibilities. A cloud-based provider such as Sauce Labs maintains and uses configuration management tools to spin up and tear down virtual machines with various OS platforms and browsers. I understand the value in this type of service. The amount of time and resources needed to build and maintain infrastructure is expensive. Been there, done that!

The downside of testing in the cloud is that you are sending commands outside your network through the cloud and back. It adds another device between source and destination, which adds latency and an additional network hop. The cloud-based approach will slow down test execution when comparing it to local execution from a developer machine.

The Solution

One of the benefits of being a writer for and client of Sauce Labs is the ability to leverage new tools in beta. One of the newest tools in beta is called Sauce Runner. This new tool is designed to make running tests in parallel on multiple platforms and browsers fast and easy. Sauce Runner facilitates server-side execution of your Selenium scripts, reducing latency, eliminating network hops, and it allows Sauce Labs to automatically handle things like test parallelization and platform selection.



What are the prerequisites to use Sauce Runner?

  • GitHub or Bitbucket repository using the supported frameworks
  • Add a deployment key to your private repository (only for private repos)
  • YAML configuration file, placed in your repository — Your repo is ready to run with Sauce Runner

If your existing repo is using one of the supported frameworks, the configuration will take less than 5 minutes to set up to use the Sauce Runner.

The screenshot below demonstrates a fast and easy way to create a test suite by using Sauce Runner:


Sauce Runner integrates with a few Continuous Integration platforms such as Jenkins, Bamboo, and TeamCity. Once you’ve created your test suite, you’re able to copy the REST Curl command from the test suite details page, then add it to your desired Continuous Integration platform.

Yes, the tool may be geared toward newbies, but I am more excited about the server-side execution since it will reduce latency issues and eliminate network hops. By leveraging the server-side execution of your Selenium script, it will reduce build time by as much as 10%.

Takeaways: Why you should try Sauce Runner
  • Server-side execution
  • Improve build speed and reliability
  • Eliminate network hops and reduce latency for remote execution
  • Simplifies the onboarding process for new Sauce Labs clients
  • Supports Continuous Integration platforms
  • No configuration needed to use parallel testing
  • As a beta user, you can help test and submit feature requests


(1)“Hop (networking) – Wikipedia, the free encyclopedia.” 2011. 26 Jan. 2016

Greg Sypolt (@gregsypolt) is a senior engineer at Gannett and co-founder of Quality Element. He is a passionate automation engineer seeking to optimize software development quality, while coaching team members on how to write great automation scripts and helping the testing community become better testers. Greg has spent most of his career working on software quality – concentrating on web browsers, APIs, and mobile. For the past five years, he has focused on the creation and deployment of automated test strategies, frameworks, tools and platforms.

References   [ + ]

1. ↑ “Hop (networking) – Wikipedia, the free encyclopedia.” 2011. 26 Jan. 2016 function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }
Categories: Companies

Even Messi sometimes needs to start from scratch

PractiTest - Tue, 09/20/2016 - 14:01

I am not a big soccer fan.  Even though I grew up in Costa Rica, where futbol (as they call it there) is something everyone needs to know and play, I was never a good player and so I’ve never really enjoyed seeing the game.

This does not mean that I don’t get to follow and even learn from some of the players.

For example, I had the pleasure of sitting with my oldest son next to Keylor Navas (goalkeeper for Real Madrid & Costa Rica’s national team) on a plane trip last year.  And there I was able to see how someone who is a renowned figure can still be kind and patient with every single person in the plane who came to take a picture with him, and then (once we finally got to be airborne) he was placid enough to play with the 1-year old toddler of the lady sitting next to him – whom he did not know.

The guy with the suicide blond hair

messi rubio goes blondBut today I want to talk about Leo Messi and his hair color.

In case you live under a rock, Leo (Lionel) Messi is one of the 2 top players in the world today (I won’t enter into the discussion if he is slightly better or worse than Cristiano Ronaldo!) and some weeks ago he surprised the world by dying his hard blond!

I mean, crazy rock stars do that all the time, but if you follow Messi you know this does not really fit him…

Well, he finally explained why he did it, and actually it made perfect sense!

Sometimes you need external changes to drive an internal change

On a satirical interview Messi provided a simple and good explanation on why he colored his hair.

In case you are not good at Spanish, he said that he wanted to start from “zero” (as we sometimes say, start from scratch) and so he choose to make a change.

For those of you who do not follow him, lately Messi has been in a small slump where among other things he ran into issues with the law in Spain for tax issues, and he also blew up a penalty shot that cost his National Team the title for the Copa America – so much so that for a while he even quit the National Team.

Apparently there were other things also bringing him down, and so he decided he needed to make a radical change, to start from scratch, and this made him go and dye his hair blond.

Changes need to start from within, but a little help from outside can also help!

I am no Messi (not even in the field of Testing) but I can understand 100% why he did this, as sometimes I do this myself…

I have not yet dyed my hair blond, I am afraid that if I do this the little hair I have left will finally decide to leave me altogether, but I do sometimes change my annotations’ notebook, or re-position my desk differently in the room, or simply change my morning habits in subtle ways that only I can notice.  And I do this in order to signal to myself that today is a new start, and I can leave all the stuff that was bringing me down literally behind me.

I know that this should not have any effect other than psychological in the way I behave.  But you know what?  Sometimes all the things that are actually bringing us down reside ONLY in our heads, and so this cheap psychological trick is exactly what I need

Categories: Companies

The Best of the UrbanCode Deploy 6.2.2

IBM UrbanCode - Release And Deploy - Tue, 09/20/2016 - 13:00
The Best of the UrbanCode Deploy 6.2.2 (or – Why you should look at UrbanCode Deploy v6.2.2) Fall is a great time for IBM UrbanCode!

The development team has been heads down, tying up weeks of work on version 6.2.2 of UrbanCode Deploy which has impressive new features and capabilities. By doing all they do, many of the largest global companies in the world, including IBM, are using UrbanCode Deploy to be successful in their DevOps transformation.
What follows is a countdown of the 10 most interesting capabilities the team has delivered this month (and by team, I mean development, testers, project and product managers, writers, support engineers and yes – marketing – we all contribute in our own ways):

  1. Have fun writing UrbanCode Deploy Processes with the improved usability of the Process Editor.  We know that working in the UCD Process Editor hasn’t always been pretty, but all that has changed! For starters, enhancements include making the step palette easier for you to navigate.  The Process Editor now allows you to add steps quickly without specifying properties until later, and when you drop steps into the diagram, a new Auto Layout feature will automatically lay out the diagram.  If a step has any script, you can now open that script from the property dialog in a large Script Editor with syntax coloring and search/replace. You can also apply the new auto layout capability to process layouts from earlier versions of UrbanCode Deploy and revert to the original layout if desired.
  2. UrbanCode Deploy Blueprint Designer integration with VMware VRealize Automation (vRA). If you are using vRA, you will find the combination of UrbanCode Deploy with vRA makes provisioning and deploying a lot easier. The UrbanCode Deploy Blueprint Designer now provides integration with vRA. You can create a cloud blueprint in UCD that incorporates vCenter IaaS constructs managed through vRA, including virtual machine templates and network profiles. Additionally, you can add in vRA software components and UCD software components to create a full stack blueprint that can be executed through UCD to create and deploy a vRA blueprint. Then, you can manage the created resources from vRA in UCD to view environment status and delete the environment in vRA when no longer needed. The key benefits of using UrbanCode Deploy with vRA used together is that the combination is very strong: UCD for the deployment automation and VRA to manage the infrastructure. UrbanCode Deploy also allows VMware users to seamlessly move their vRA/vCenter blueprints to other cloud providers, including OpenStack, SoftLayer, and others.
  3. New Open Development Community for Plugins – Use Git Hub to learn about and contribute plugins. See what is available in this new community:
  4. Numerous new and improved integrations:
    1. The new plugin for Jenkins Pipeline adds support for creating components, invoking UCD component imports, running steps multiple times in a single job, deploying snapshots or component versions; compatible with Jenkins Pipeline DSL
    2. New API Management plugin
    3. Improved Docker Compose plugin
    4. New ODM source plugin and support for deploying RuleApps remotely with the existing plugin
    5. Artifactory plugin has improved governance and metadata exchange and handling of Maven snapshot
    6. The Cloud Foundry Plugin has improved support for manifest deployments and managing User Provided Services
  5. Configuration management for Microsoft Internet Information Server (IIS) plugin – BETA. Now you can use UrbanCode Deploy to manage a heterogeneous application server environment. With this new plugin, you can discover the IIS site, tokenize it, create properties in UrbanCode Deploy, deploy the configurations to another environment.
  6. Monitor key deployment metrics through integrations with applications such as New Relic or IBM APM. Key metrics that cover agents, individual components and applications, as well as component summary metrics.
  7. Updated WebSphere Application Server (WAS) plugins (Deploy,Configure). UrbanCode Deploy has updated and improved repeatable deployments to WebSphere Application Server, capturing, applying and comparing WebSphere Application Server Configuration, and added the ability to migrate captured configurations from an older version of WAS to a newer version.
  8. Support for WebSphere Liberty Collectives in the UrbanCode Deploy IBM WebSphere Liberty plugin.
  9. Enhancements to full stack provisioning.  Application processes can now be executed from an UrbanCode Deploy cloud blueprint to provision Windows or Linux VMs on multiple clouds (Softlayer, AWS, Azure).
  10. For mainframe, UrbanCode Deploy can now manage the deployment of both MVS data sets and HFS files in a single component version, making it easier to deploy applications with both types of artifacts, for example CICS Web Services.

Oh… and 1 MORE!

As a service.  We eagerly announce a new UrbanCode Deploy offering: IBM Cloud UrbanCode Deploy (as a service), now available in September 2016. 
For an organization, as you offload the “utility” functions, you free up talented and usually expensive resources to do more higher value and rewarding work.  There is a talent war for DevOps folks, and offering to take care of the plumbing will help with talent recruiting and retention. Banks are competing with Google and Netflix, so we have to have a Talent Strategy that can compete with that …” – Former Global 100 Banking Exec, UrbanCode Deploy proponent

UrbanCode Deploy has been available as a product to install, configure, administer and maintain on premises for several years. Many IBM customers have done this, have become experts, and will continue to use UrbanCode Deploy on premises. However, some customers are ready to start using more applications in the cloud, or wish to simply get started with deployment automation without investing in the infrastructure or administration. IBM Cloud UrbanCode Deploy (as a service) provides a way to get started quickly for faster time to value.  The value of IBM Cloud UrbanCode Deploy at a glance:

  • Fast time-to-value
  • Use headcount for innovation
  • Easy to expand
  • Pay for the service from operational budget, not as a capital
  • Full application deployment automation & support – all from IBM
  • Upgrades and licensing management done by IBM

Learn more how IBM UrbanCode Deploy can help your organization! 

Categories: Companies

Jenkins Online Meetup report. Plugin Development - WebUI

On September 6th we had a Jenkins Online Meetup. This meetup was the second event in the series of Plugin Development meet ups. At this meetup we were talking about Jenkins Web UI development. Talks 1) Classic Jenkins UI framework - Daniel Beck In the first part of his talk, Daniel presented how Stapler, the web framework used in Jenkins, works, and how you can add to the set of URLs handled by Jenkins. In the second part he was talking about creating new views using Jelly and Groovy, and how to add new content to existing views. Keywords: Stapler, Jelly, Groovy-defined UIs 2) Developing modern Jenkins UIs with Javascript - Tom Fennelly Feel...
Categories: Open Source

How to Integrate JMeter into Jenkins

This is a guest post by Dmitri Tikhansi from BlazeMeter.

Continuous integration (CI), test automation and “shifting left” are becoming the standard for DevOps, developers and QA engineers. But despite the importance of performance, and the understanding that systems are complex and it can be challenging to identify and fix bottlenecks in a short time, load testing is still not an integral part of the CI workflow.

Jenkins easily enables users to integrate load testing into its workflow. By using Jenkins as part of the CI process and to trigger jobs by commits, users are taking advantage of automation and process speed-up capabilities.

Advantages of Integrating JMeter into Jenkins

JMeter is one of the most popular open-source load testing systems. By integrating JMeter into Jenkins, users can enjoy:

  • Unattended test executions right after software build and deploy
  • Automatic build failures in case of performance degradation
  • Easy access to test reports that show application metric trends - all tests are in one place and available to anybody with the correct permissions
  • Automated routine work of test configuration, execution and baseline results analysis. Users’ hands and minds are free for more important, complex and interesting tasks.
How to Integrate JMeter into Jenkins
  1. Store JMeter results as XML (recommended because they are easier to use) or CSV
  2. Specify the command to run your test in the Execute Shell Batch Command section
  3. Check the Console Output tab to verify the execution was successful
  4. Find your files in the project’s workspace
  5. Specify Build Parameters
How to Use the Performance Plugin

To view JMeter reports on Jenkins, you can use the Performance plugin.

  1. Install the Performance plugin
  2. Configure the plugin
    The Performance plugin can be added as a “Post-build Action”. When the JMeter test is finished, the plugin will:
    • Collect the data
    • Conditionally fail the build if the error threshold is exceeded
    • Build or update the performance trend chart for the project

Configuration options explained:

  • Performance report - For JMeter you will need to upload a file in XML format.
  • Select mode -  The choices are in the Relative Threshold and Error Threshold. The Relative Threshold compares the difference from the previous test, and if it exceeds the defined value the build will be marked as failed/unstable. The Error Threshold marks the the build as unstable or failed if the amount of errors will exceed the specified value.
  • Build result - If the JMeter test doesn’t generate the output jtl file(s) - the build will be failed.
  • Use error thresholds on single build - Define error thresholds for the current build.
  • Average response time threshold - Set the maximum acceptable value of the Average Response Time metric.
  • Use relative thresholds for build comparison - Set the percentage difference of errors. The “source” build can be either the previous build or “known good” build which is used as a baseline.
  • Performance per Test Case Mode - If you need to see separate graphs for each single test case on the Performance Trend chart you can trigger the behavior by this option.
  • Show Throughput Chart - Set whether to display “Throughput” trend chart on project dashboard or not.

Jenkins can also be integrated with BlazeMeter and Taurus, for faster and easier results and more analysis options. 

Congratulations on adding performance tests to your continuous integration process!


Blog Categories: Jenkins
Categories: Companies

Knowledge Sharing

SpiraTest is the most powerful and affordable test management solution on the market today