Skip to content

Open Source

SonarQube 5.0 in Screenshots

Sonar - Wed, 01/28/2015 - 17:56

The team is proud to announce the release of SonarQube 5.0, which includes many new features

  • Issues page redesign
  • Keyboard shortcuts added to Issues
  • Built-in SCM support

Issues page redesign

With this version of the SonarQube platform, the Issues page has had a complete overhaul.

Issues are grouped in the list by file, and from the list of issues, you can easily navigate to the issue in the context of the code, as you’re used to seeing it,

Issue search has also been overhauled. You can still choose from criteria, but now next to each facet of the criteria, you see a count of the relevant issues.

Selected facets are highlighted in blue, and selecting/deselecting a facet immediately (okay, there’s a quick trip to the server and back) updates your search results and the issue counts next to all the other facets.

Keyboard shortcuts added to Issues

The intent behind the redesign is to allow you to use the Issues page quickly and efficiently to manage issues on a daily basis. To that end, extensive effort has gone into providing a broad set of keyboard short cuts. ‘?’ brings up the list, and Esc closes it.

From the list of issues, right-arrow takes you to the issue-in-context, and left-arrow takes you back to the list. In either context, up-arrow/down-arrow takes you to the next issue – in the same file or the next one – and you can use the shortcuts to comment, assign…

Built-in SCM support

SCM “blame” information has been an important data point in the SonarQube interface for a long time, but until now a plugin was required to use it. Now SCM data comes as part of the platform, with built-in support for SVN and Git.

Speaking of Git, its rise in popularity has meant that the use of ‘/’ in branch names has become increasingly common. Until now, that was unsupported by SonarQube. That changes with 5.0, presumably making many Git-ers happy. :-)

That’s all, Folks!

Time now to download the new version and try it out. But don’t forget that you’ll need Java 7 to run this version of the platform (you can still analyse Java 6 code), and don’t forget to read the installation or upgrade guide.

Categories: Open Source

2015 Jenkins User Conferences - Call for Papers

The Jenkins User Conference 2015 is seeking submissions that reflect the latest innovations in Jenkins usage. This is your chance to educate, share and inspire the community with stories of how you've used Jenkins to continuously build that amazing project or how you developed that popular plugin that everyone is using.

If you're gamed, here are some suggestions to get your creative juice going:

  • Scaling Jenkins within the enterprise
  • Jenkins as the orchestrator for continuous delivery
  • Plug-in development
  • Jenkins techniques that solve testing/building problems in specific application areas: mobile, enterprise/web/cloud and UI testing
  • War stories that speak to a problem you faced, the Jenkins solution you implemented to solve it and the results you realized
  • Jenkins best practices, tips and tricks
  • Jenkins in the cloud - if you or your company is currently using Jenkins in the cloud we’d love to hear your story
  • Beyond Java (Jenkins with PHP, Ruby, etc.)

We are upping the ante at this year's JUCs. We are moving from a 1 day conference to a 2 days conference for SF and London - that's 18 additional cutting edge sessions to be learned.

SUBMISSION DEADLINE IS MARCH 8, 2015!

There's also a wide variety of event sponsorship opportunities available. There are offerings from Gold to Silver packages, exhibitor packages in our world-class expo hall, speaking sessions, free passes, and many branding opportunities. For inquiries, pls contact juc-sponsorship@cloudbees.com

Looking forward to receiving your amazing proposals!

Categories: Open Source

Office Hours tomorrow: workflow security model & plugin compatibility

In tomorrow's Jenkins office hours, Jesse Glick will talk about two topics in the workflow plugin that he has been asked about:

  • Security model: script security, permissions
  • Plugin compatibility: SimpleBuildStep and friends, custom steps, etc.

The session should be interesting to anyone using workflow or thinking about using workflow. Jesse is one of the top contributors in the community, so it'd be definitely worth your time!

Categories: Open Source

COBOL is… Alive!

Sonar - Wed, 01/14/2015 - 20:20

Most C, Java, C++, C#, JavaScript… developers reading this blog entry might think that COBOL is dead and that SonarSource should better focus its attention on more hyped languages like Scala, Go, Dart, and so on. But in 1997, the Gartner Group reported that 80 percent of the world’s business ran on COBOL, with more than 200 billion lines of code in existence and an estimated 5 billion lines of new code annually. COBOL is mainly used in the banking and insurance markets, and according to what we have seen in the past years, the erosion of the number of COBOL lines of code used in production is pretty low. So not only is COBOL not YET dead, but several decades will be required to see this death really happen. We released the first version of the COBOL plugin at the beginning of 2010 and this language plugin was in fact the first one to embed our own source code analysis technology, even before Java, C, C++, PL/SQL, … So at SonarSource, COBOL is a kind of leading technology :).

Multiple vendor extensions and lack of structure

The COBOL plugin embeds more than 130 rules, but before talking about those rules, let’s talk about the wide range of different COBOL dialects that are supported by the plugin. Indeed, since 1959 several specifications of the language and preprocessor behavior have been published, and most COBOL compilers have extended those specifications. So providing an accurate COBOL source code analyser means supporting most of those dialects: IBM Enterprise Cobol, HP Tandem, Bull GCos, IBM Cobol II, IBM Cobol 400, IBM ILE Cobol, Microfocus AcuCobol, OpenCobol, … which is the case for our plugin Moreover for those of you who are not familiar with COBOL source code: let’s imagine a C source file containing 20,000 lines of code, no functions, and just some labels to group statements and to make it possible to “emulate” the concept of function. Said like this, I guess everyone can understand how easy it can be to write unmaintainable and unreliable COBOL programs.

Need for tooling

Starting from this observation, managing a portfolio of thousands of COBOL programs, each one containing thousands of COBOL lines of code, without any tooling to automatically detect quality defects and potential bugs is a bit risky. The SonarSource COBOL plugin allows to continuously analyse millions lines of COBOL code to detect such issues and here are several examples of the rules provided by the plugin:

  • Detection of unused paragraphs, sections and data items.
  • Detection of incorrect PERFORM ... THRU ... control flow, where the starting procedure is located after the ending one in the source code, thus leading to unexpected behavior.
  • Tracking of GO TO statements that transfer control outside of the current module, leading to unstructured code.
  • Copy of a data item (variable) into another, smaller data item, which can lead to data loss.
  • Copy of an alphanumeric data item to a numeric one, which can also lead to data loss.
  • Tracking of EVALUATE statements not having the WHEN OTHER clause (similar to an if without an else).
  • Detection of files which are opened but never closed.

And among those 130+ rules, 30+ target the SQL code which can be embedded into COBOL programs. One such rule tracks LIKE conditions starting with *. Another tracks the use of arithmetic expressions and scalar functions in WHEREconditions. And last but not least, here are some other key features of this SonarSource COBOL plugin :

  • Copybooks are analysed in the context of each COBOL program and issues are reported directly on those copybooks.
  • Remediation cost to fix issues is computed with help of the SQALE method: www.sqale.org.
  • Even on big COBOL applications containing thousands of COBOL programs and so potentially millions of lines of code and thousands of issues, tracking only new issues on new or updated source code is easy.
  • Duplications in PROCEDURE DIVISION and among all COBOL programs can also be tracked easily.
  • To make sure that code complies with internal coding practices, a Java API allows the development of custom rules.

How hard it is to evaluate this COBOL plugin ?

So YES, Cobol is alive, and the SonarSource COBOL plugin helps make it even more maintainable and reliable.

Categories: Open Source

SonarQube 5.x series: It just keeps getting better and better!

Sonar - Fri, 01/09/2015 - 15:03

We recently wrapped up the 4.x series of the SonarQube platform by announcing its Long Term Support version: 4.5.1. At the same time, we sat down to map out the themes for the 5.x series, and we think they’re pretty exciting.

In the 5.x series, we want the SonarQube platform to become:

  • Fully operational for developers: with easy management of the daily incoming technical debt, and “real” cross-source navigation features
  • Better tailored for big companies: with great performance and more scalability for large instances, and no more DB access from an analysis

Easy management of the daily incoming technical debt A central Issues page

If you came home one day to find an ever-growing puddle of water on your floor, what’s the first thing you’d do? Grab a mop, or find and fix the source of the water? It’s the same with technical debt. The first thing you should care about is stopping the increase in debt (shutting off the leak) before fixing existing debt (grabbing a mop).

Until now, the platform has been great for finding where technical debt is located, but it hasn’t been a good place for developers to efficiently manage the incoming technical debt they add every day. Currently, you can subscribe to notifications of new issues, but that’s all. We think that’s a failing; developers should be able to rely on SonarQube to help them in this daily task.

To accomplish this, we’ll make the Issues page central. It will be redesigned to let users filter issues very efficiently thanks to “facets”. For instance, it will be almost effortless to see “all critical and blocker issues assigned to me on project Foo” with a distribution per rule. Or “all JavaScript critical issues on project Foo“.

With these new capabilities, the central Issues page will inevitably replace the old Issues drilldown page and eliminate its limitations (e.g. few filters, static counts that aren’t updated when issues are changed in the UI, …). In other words, when dealing with issues and technical debt, users will be redirected to the Issues space, and benefit from all those new features.

Issues will also get a tagging mechanism to help developers better classify pending technical debt. Each issue will inherit the tags on its associated rule, so it will be easy to find “security” issues, for instance. And users will be able to add or remove additional tags at will. This will help give a clearer vision of what the technical debt on a project is about: is it mainly bugs or just simple naming conventions? “legacy framework”-related issues or “new-stack-that-must-be-mastered” issues?

The developer at the center of technical debt management

Developing those great new features on the Issues page is almost useless if you, as a developer, always have to head to the Issues page and play with “facets” to find what you’re looking for. Instead, SonarQube must know what matters to you as a developer, i.e. it must be able to identify and report on “my” code. This is one reason the SCM Activity plugin will gently die, and come back to life as a core feature in SonarQube – with built-in support for Git and Subversion (other SCM providers will be supported as plugins). This will let SonarQube know which changes belong to which developer, and automatically assign new issues to the correct user. So you’ll no longer need to swim through all of the incoming debt each day to find your new issues.

“Real” cross source navigation features

For quite some time, SonarQube has been able to link files together in certain circumstances – through duplications (you can navigate to the files that have blocks in common with your code) or test methods (when coverage per test is activated, you can navigate to the test file that covers your code, and vice-versa). But, this navigation capability has been quite limited, and the workspace concept that goes with it is the best proof of that: it is restricted to the context of the component viewer.

With the great progress made on the language plugin side, SonarQube will be able to know that a given variable or function is defined outside of the current file, and take you to the definition. This new functionality can help developers understand issues more quickly and thoroughly, without the need to open an IDE. You no longer have to wonder where a not-supposed-to-be-null-but-is attribute is defined. You’ll be able to jump to the class definition right from the Web UI. And if you navigate far away from the initial location, SonarQube will help you remember your way, and give quick access to the files you browsed recently – wherever they were. In fact, we want SonarQube to become the central place to take a quick look at code without expending a lot of effort to do it (i.e. without the need to go to a workstation, open an IDE, pull the latest code from the SCM repository, probably build-it, …).

Focus on scalability and performance

SonarQube started as a “small” application and gradually progressed to become an enterprise-ready application. Still, its Achilles’ heel is the underlying relational database. This is the bottleneck each time we want SonarQube to be more scalable and performant. What’s more, supporting 4 different database vendors multiplies the difficulty of writing complex SQL queries efficiently. So even though the database will remain the place where we ensure data integrity, updating that data with analysis results must be done through the server, and searching must use a stack designed for performant searches across large amounts of data. We’ve implemented this new stack using Elasticsearch (ES) technology.

Throughout the 5.x series, most domains will slowly get indexed in ES, giving a performance boost when requesting the data. This move will also open new doors to implementing features that were inaccessible with a relational database – like the “facets” used on the Rules or Issues pages. And because ES is designed to scale, SonarQube will benefit from its ability to give amazing performance while adding new features on large instances with millions of issues and lines of code.

Decoupling the SonarQube analyses from the DB

The highest-voted open ticket on JIRA is also one of the main issues when setting up SonarQube in large companies: why does project analysis make so many queries to the database? And actually, why does it even need a connection to the database at all? This has big performance issues (when the analysis is run far away from the DB) and security issues (DB credentials must be known by the batch, some specific ports must be opened).

Along the way, the SonarQube 5.X releases will progressively cut dependencies to the database so that in the end, analysis simply generates a report and sends it to the server for processing. This will not only address the performance and security concerns, it will also greatly improve the design of the whole architecture, clearly carving it into different stacks with their own responsibilities. In the end, analysis will only call the analysers provided by the language plugins, making source code analysis blazing fast. Everything related to data aggregation or history computation (which once required so many database queries during analysis) will be handled by a dedicated “Compute Engine” stack on the server. Integration in the IDE will also benefit from this separation because only the language plugin analysers will be run – instead of the full process – opening up opportunities to have “on-the-fly” analyses.

Enhanced authentication and authorization system

A couple of versions ago, we started an effort to break the initial coarse-grained permissions (mainly global ones) into smaller ones. The target is to be able to have more control over the different actions available in SonarQube, and to be able to define and customize the roles available on the platform. This is particularly important on the project side, where there are currently only 4 permissions, and they don’t allow a lot of flexibility over what users can or cannot do on a project.

On the authentication side, the focus will be providing a reference Single Sign-On (SSO) solution based on HTTP headers – which is a convenient and widespread way of implementing SSO in big companies. API token authentication should also come along to remove the need to pass user credentials over the wire for analysis or IDE configuration.

All this with other features along the way

These are the main themes we want to push forward for the 5.x series, but obviously lots of other “smaller” features will come along the way. At the time I’m writing this post, we’ve already started working on most of those big features and we are excited about seeing them come out in upcoming versions. I’m sure you share our enthusiasm!

Categories: Open Source

Selenium Hangout 6 Recap

Selenium - Tue, 12/23/2014 - 03:28

01:35 – 9:45 W3C Update
Notes from most recent W3C Meeting
Highlights:
– changes to the get_attribute method call
– screenshots (changing to viewport only, eventually will support whole page)
The WebDriver W3C working group has a GitHub repo now
– WebDriver will move from a “REST-ish” to a more “RESTful” interface

11:23 – 16:00 Selenium 3 Status Update

16:05 – 17:10 Marionette (FirefoxdDiver rewrite) testing help 
Marionette Roadmap

17:20 – 19:27 ChemistryKit rewrite
Announcement blog post

17:28 – 20:24 Visual Testing Part 1
Getting Started with Visual Testing
Applitools (visual testing cloud solution built on top of WebDriver)

20:25 – 23:47 Selenium Guidebook in Java!
The Selenium Guidebook

23:52 – 29:51 Visual Testing Part 2
Web Consistency Testing
Why MogoTest won’t be open sourcing it’s code after shutting down
Michael Tamm’s GTAC talk on Fighting Layout Bugs
Getting Started with Visual Testing


Categories: Open Source

#BreakingBuilds

A lot of us has grown fond of our loyal butler Mr.Jenkins over time, which was created by Frontside and chosen as a result of a logo contest. In the true open-source style, the logo has since evolved into many different derivative works, such as a plugin, a 3D model, and a bobble head.

Our friends at CloudBees are running a #BreakingBuilds social media contest through Jan 5th to have some fun with Mr.Jenkins. Read Sacha Labourey's blog post, where he draws parallels between what a butler does and what continuous delivery can do.

I especially agree with him on this point: I always loved the idea of using a butler to represent what Jenkins is about, as it projects all of the qualities that define continuous delivery: it is built to be proactive, it will help you fix problems before they happen, it orchestrates your entire pipeline to production without you having to worry about the sophisticated underlying sequence of steps and, if things go wrong Jenkins uses his fingerprint database to trace back the source of the issue. Full service. As your right arm, Jenkins is the reliable and trustworthy guy you want on your team!

Check out the contest rules and participate. Let's raise the visibility of Jenkins and have some fun in the process!

Categories: Open Source

Announcing Test Automation Bazaar Jan 16-17 in Austin

Watir - Web Application Testing in Ruby - Tue, 12/16/2014 - 17:38

Originally posted on Testing with Vision:

I am pleased to announce that the Test Automation Bazaar will be held in Austin, Texas on  January 16-17, 2015 (Fri – Sat). I am convening this event with Zeljko Filipin and the Austin Homebrew Testers, and we are pleased that the event will be sponsored by the Open Information Foundation, a non-profit which we have recently joined and which also sponsors the Citcon conferences. This is a follow up to the 2012 Test Automation Bazaar, also held in Austin. Like all OIF events, this conference will be free and open to the public, but we also will be asking for donations and sponsors to cover the expenses of the event. We are currently confirming a location in the Domain. We invite people to come, share their experiences with test automation and learn from others. The organizers have a bias for Ruby, Webdriver (Watir/Selenium), and open-source tools, but we…

View original 134 more words

Categories: Open Source

Walking the Tightrope: Balancing Agility and Stability

Sonar - Fri, 12/12/2014 - 14:22

About a year ago we declared a Long Term Support (LTS) version for the first time ever, and recently, we declared another one (version 4.5.1). But we never talked about what LTS means or why we did it.

Here’s the story:

SonarSource is an agile company. We believe deeply in the agile principles, including this one:

Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.

So that’s why we deliver a new version about every two months. We know we need to get our changes out to the users so they can tell us what we’ve done well, and what we haven’t. (Feedback of the latter kind is more frequent, but both kinds are appreciated!) That way we can fix our goofs before they’re too deeply embedded under other layers of logic to fix easily.

That’s the agility part: release frequently and respond to customer feedback. But what about stability?

We know that many of our users are in large organizations. Since many of us came to SonarSource from large companies, we understand how well such organizations deal with frequent change: not well at all, sometimes. Instead, they need stability and reliability.

For a while, that left us with a conundrum: how could we be responsive to the customer need for stability, and still be agile?

(Drum roll, please!) Enter the LTS version.

An LTS version marks a significant milestone for us: it means that through incremental, we’ve brought a feature set to completion, and worked out the kinks. The vision that was established (a year ago, in this case) has been achieved, and it’s time to set out a new vision. (More on that soon!)

Once a version is marked LTS, we pledge to maintain it and fix any non-trivial bugs until the next LTS version is released. That way, users who need stability know that they’ll never be forced to upgrade to a non-LTS version just for a bug fix.

And if a bug fix is required for an LTS version, you know you can upgrade to it without any other change in behavior or features. I.e. it’s completely transparent to your users. Of course, we don’t mark a version LTS until we know it’s stable, and we’ve fixed all the bugs in it that we’re aware of. So the chance that you’ll need to perform this kind of transparent upgrade are small.

Of course, there are trade-offs. We release frequently, and pack a lot work into each release. By opting to stay with an LTS version, you trade benefiting from the cool new features for stability (yes, I know that’s a trade worth making for many people).

But there’s another trade-off to be aware of. When you go from one LTS version to the next, you’ll see a lot of change all at once. This time the jump from one LTS to the next includes significant UI changes, plugin compatibility differences, and changes that impact project analysis (possibly requiring analysis job reconfigurations). There’s an overview in the docs.

On the whole, a jump from one LTS to the next one will be awesome, but you need to be aware that it may not feel as trivial as SonarQube upgrades usually do. Really, it’s just a question of how you want to rip off the Band-Aid(R).

Categories: Open Source

Test Framework Feature Comparisons – What If We Cooperated?

NUnit.org - Sun, 04/07/2013 - 03:14

Software projects often publish comparisons with other projects, with which they compete. These comparisons typically have a few characteristics in common:

  • They aim at highlighting reasons why one project is superior – that is, they are marketing material.
  • While they may be accurate when initially published, competitor information is rarely updated.
  • Pure factual information is mixed with opinion, sometimes in a way that doesn’t make clear which is which.
  • Competitors don’t get much say in what is said about their projects.
  • Users can’t be sure how much to trust such comparisons.

Of course, we’re used to it. We no longer expect the pure, unvarnished truth from software companies – no more than from drug companies, insurance companies, car salesmen or government agencies. We’re cynical.

But one might at least hope that open source projects might do better. It’s in all our interests, and in our users’ interests, to have accurate, up-to-date, unbiased feature comparisons.

So, what would such a comparison look like?

  • It should have accurate, up-to-date information about each project.
  • That information should be purely factual, to the extent possible. Where necessary, opinions can be expressed only if clearly identified as opinion by it’s content and placement.
  • Developers from each project should be responsible for updating their own features.
  • Developers from each project should be accountable for any misstatements that slip in.

I think this can work because most of us in the open source world are committed to… openness. We generally value accuracy and we try to separate fact from opinion. Of course, it’s always easy to confuse one’s own strongly held beliefs with fact, but in most groups where I participate, I see such situations dealt with quite easily and with civility. Open source folks are, in fact, generally quite civil.

So, to carry this out, I’m announcing the .NET Test Framework Feature Comparison project – ideas for better names and an acronym are welcome. I’ll provide at least a temporary home for it and set up an initial format for discussion. We’ll start with MbUnit and NUnit, but I’d like to add other frameworks to the mix as soon as volunteers are available. If you are part of a .NET test framework project and want to participate, please drop me a line.

Categories: Open Source

Software Testing Latest Training Courses for 2012

The Cohen Blog — PushToTest - Mon, 02/20/2012 - 05:34
Free Workshops, Webinars, Screencasts on Open Source Testing Need to learn Selenium, soapUI or any of a dozen other Open Source Test (OST) tools? Join us for a free Webinar Workshop on OST. We just updated the calendar to include the following Workshops:
And If you are not available for the above Workshops, try watching a screencast recording.

Watch The Screencast

Categories: Companies, Open Source

Selenium Tutorial For Beginners

The Cohen Blog — PushToTest - Thu, 02/02/2012 - 08:45
Selenium Tutorial for Beginners Selenium is an open source technology for automating browser-based applications. Selenium is easy to get started with for simple functional testing of a Web application. I can usually take a beginner with some light testing experience and teach them Selenium in a 2 day course. A few years ago I wrote a fast and easy tutorial Building Selenium Tests For Web Applications tutorial for beginners.

Read the Selenium Tutorial For Beginners Tutorial

The Selenium Tutorial for Beginners has the following chapters:
  • Selenium Tutorial 1: Write Your First Functional Selenium Test
  • Selenium Tutorial 2: Write Your First Functional Selenium Test of an Ajax application
  • Selenium Tutorial 3: Choosing between Selenium 1 and Selenium 2
  • Selenium Tutorial 4: Install and Configure Selenium RC, Grid
  • Selenium Tutorial 5: Use Record/Playback Tools Instead of Writing Test Code
  • Selenium Tutorial 6: Repurpose Selenium Tests To Be Load and Performance Tests
  • Selenium Tutorial 7: Repurpose Selenium Tests To Be Production Service Monitors
  • Selenium Tutorial 8: Analyze the Selenium Test Logged Results To Identify Functional Issues and Performance Bottlenecks
  • Selenium Tutorial 9: Debugging Selenium Tests
  • Selenium Tutorial 10: Testing Flex/Flash Applications Using Selenium
  • Selenium Tutorial 11: Using Selenium In Agile Software Development Methodology
  • Selenium Tutorial 12: Run Selenium tests from HP Quality Center, HP Test Director, Hudson, Jenkins, Bamboo
  • Selenium Tutorial 13: Alternative To Selenium
A community of supporting open source projects - including my own PushToTest TestMaker - enables you to apply your Selenium tests as functional tests for smoke testing, regression testing, and integration tests, load and performance tests, and production service monitors. These techniques and tools make it easy to run Selenium tests from test management platforms, including HP Quality Center, HP Test Director, Zephyr, TestLink, QMetry, from automated Continuous Integration (CI) tests, including Hudson, Jenkins, Cruise Control, and Bamboo.

I wrote a Selenium tutorial for beginners to make it easy to get started and take advantage of the advanced topics. Download TestMaker Community to get the Selenium tutorial for beginners and immediately build and run your first Selenium tests. It is entirely open source and free!

Read the Selenium Tutorial For Beginners Tutorial

Categories: Companies, Open Source

5 Services To Improve SOA Software Development Life Cycle

The Cohen Blog — PushToTest - Fri, 01/27/2012 - 00:25
SOA Testing with Open Source Test Tools PushToTest helps organizations with large scale Service Oriented Architecture (SOA) applications achieve high performance and functional service delivery. But, it does not happen at the end of SOA application development. Success with SOA at Best Buy requires an Agile approach to software development and testing, on-site coaching, test management, and great SOA oriented test tools.

Distributing the work of performance testing through an Agile epoc, story, and sprints reduces the testing effort overall and informs the organization's business managers on the service's performance. The biggest problem I see is keeping the testing transparent so that anyone - tester, developer, IT Ops, business manager, architect - follows a requirement down to the actual test results.

With the right tools, methodology, and coaching an organization gets the following:
  • Process identification and re-engineering for Test Driven Development (TDD)
  • Installation and configuration of a best-in-class SOA Test Orchestration Platform to enable rapid test development of re-usable test assets for functional testing, load and performance testing and production monitoring
  • Integration with the organization's systems, including test management (for example, Rally and HP QC) and service asset management (for example, HP Systinet)
  • Construction of the organization's end-to-end tests with a team of PushToTest Global Professional Services, using this system and training of the existing organization's testers, Subject Matter Experts, and Developers to build and operate tests
  • On-going technical support
Download the Free SOA Performance Kit On-Site Coaching Leads To Certification
The key to high quality and reliable SOA service delivery is to practice an always-on management style. That requires on-site coaching. In a typical organization the coaches accomplish the following:
  • Test architects and test developers work with the existing Testing Team members. They bring expert knowledge of the test tools. Most important is their knowledge of how to go from concept to test coding/scripting
  • Technical coaching on test automation to ensure that team members follow defined management processes
Cumulatively this effort is referred to as "Certification". When the development team produces quality product as demonstrated by simple functional tests, then the partner QA teams take these projects and employ "best practice" test automation techniques. The resulting automated tests integrate with the requirements system (for example, Rally), the continuous integration system, and the governance systems (for example, HP Systinet.)
Agile, Test Management, and Roles in SOA
Agile software development process normally focuses first on functional testing - smoke tests, regression test, and integration tests. Agile applied to SOA service development deliverables support the overall vision and business model for the new software. At a minimum we should expect:
  1. Product Owner defines User Stories
  2. Test Developer defines Test Cases
  3. Product team translates Test Cases into soapUI, TestMaker Designer, and Java project implementations
  4. Test Developer wraps test cases into Test Scenarios and creates an easily accessible test record associated to the test management service
  5. Any team member follows a User Story down into associated tests. From there they can view past results or execute tests again.
  6. As tests execute the test management system creates "Test Execution Records" showing the test results
Learn how PushToTest improves your SOA software development life cycle. Click here to learn how.


Download the Free SOA Performance Kit

Categories: Companies, Open Source

Application Performance Management and Software Testing Trends and Analysis

The Cohen Blog — PushToTest - Tue, 01/24/2012 - 16:25
18 Best Blogs On Software Testing 2011 began with some pretty basic questions for the software testing world:
  • To what extent will large organizations dump legacy test tools for open source test tools?
  • How big would the market for private cloud software platforms be?
  • Does mankind have the tools to make a reliable success of the complicated world we built?
  • How big of a market will SOA testing and development be?
  • What are the best ways to migrate from HP to Selenium?
Let me share the answers I found. Some come from my blog, others from friends and partner blogs. Here goes:

The Scalability Argument for Service Enabling Your Applications. I make the case for building, deploying, and testing SOA services effectively. I point out the weakness of this approach comes at the tool and platform level. For example, 37% of an application's code simply to deploy your service.

How PushToTest Uses Agile Software Development Methodology To Build TestMaker. A conversation I had with Todd Bradfute, our lead sales engineer, on surfacing the results of using Agile methodology to build software applications.

"Selenium eclipsed HP’s QTP on job posting aggregation site Indeed.com to become the number one requisite job experience / skill for on-line posted automated QA jobs (2700+ vs ~2500 as of this writing,)" John Dunham, CEO at Sauce Labs, noted.

Run Private Clouds For Cost Savings and Control. Instead of running 400 Amazon EC2 machine instances, Plinga uses Eucalyptus to run its own cloud. Plinga needed the control, reliability, and cost-savings of running its own private cloud, Marten Mickos, CEO at Eucalyptus, reports in his blog.

How To Evaluate Highly Scalable SOA Component Architecture. I show how to evaluate highly scalable SOA component architecture. This is ideal for CIOs, CTOs, Development and Test Executives, and IT managers.

Planning A TestMaker Installation. TestMaker features test orchestration capabilities to run Selenium, Sahi, soapUI, and unit tests written in Java, Ruby, Python, PHP, and other langauges in a Grid and Cloud environment. I write about the issues you may encounter installing the TestMaker platform.

Repurposing ThoughtWorks Twist Scripts As Load and Performance Tests. I really like ThoughtWorks Twist for building functional tests in an Agile process. This blog and screencast shows how to rapidly find performance bottlenecks in your Web application using Thoughtworks Twist with PushToTest TestMaker Enterprise test automation framework.

4 Steps To Getting Started With The Open Source Test Engagement Model. I describe the problems you need to solve as a manager to get started with Open Source Testing in your organization.

Corellation Technology Finds The Root Cause To Performance Bottlenecks. Use aspect-oriented (AOP)  technology to surface memory leaks, thread deadlocks, and slow database queries in your Java Enterprise applications.

10 Agile Ways To Build and Test Rich Internet Applicatiions (RIA.) Shows how competing RIA technologies put the emphasis on test and deploy.

Oracle Forms Application Testing. Java Applet technology powers Oracle Forms and many Web applications. This blog shows how to install and use open source tools to test Oracle Forms applications.

Saving Your Organization From The Eventual Testing Meltdown of Using Record/Playback Solely. The Selenium project is caught between the world of proprietary test tool vendors and the software developer community. This blog talks about the tipping-point.

Choosing Java Frameworks for Performance. A round-up of opinions on which technologies are best for building applications: lightweight and responsive, RIA, with high developer productivity.

Selenium 2: Using The API To Create Tests. A DZone Refcard we sponsored to explain how to build tests of Web applications using the new Selenium 2 APIs. For the Selenium 1 I wrote another Refcard, click here.

Test Management Tools. A discussion I had with the Zephyr test management team on Agile testing.

Migrating From HP Mercury QTP To PushToTest TestMaker 6. HP QTP just can't deal with the thousands of new Web objects coming from Ajax-based applications. This blog and screencast shows how to migrate.

10 Tutorials To Learn TestMaker 6. TestMaker 6 is the easier way to surface performance bottlenecks and functional issues in Web, Rich Internet Applications (RIA, using Ajax, Flex, Flash,) Service Oriented Architecture (SOA,) and Business Process Management (BPM) applications.

5 Easy Ways To Build Data-Driven Selenium, soapUI, Sahi Tests. This is an article on using the TestMaker Data Production Library (DPL) system as a simple and easy way to data-enable tests. A DPL does not require programming or scripting.

Open Source Testing (OST) Is The Solution To Modern Complexity. Thanks to management oversite, negligence, and greed British Petroleum (BP) killed 11 people, injured 17 people, and dumped 4,900,000 barrels of oil into the Gulf of Mexico in 2010. David Brooks of the New York Times became an unlikely apologist for the disaster citing the complexity of the oil drilling system.

Choosing automated software testing tools: Open source vs. proprietary.  Colleen Fry's article from 2010  discusses why software testers decide which type of automated testing tool, or combination of open source and proprietary, to best meets their needs. We came a long way in 2011 to achieve these goals.

All of my blogs are found here.

Categories: Companies, Open Source

Free Webinar on Agile Web Performance Testing

The Cohen Blog — PushToTest - Tue, 01/10/2012 - 19:22
Free Open Source Agile Web Application Performance Testing Workshop
Your organization may have adopted Agile Software Development Methodology and forgot about load and performance testing! In my experience this is pretty common. Between Scrum meetings, burn-down sessions, sprints, test first, and user stories, many forms of testing - including load and performance testing, stress testing, and integration testing - can get lost. And, it is normally not only your fault. Consider the following:
  • The legacy proprietary test tools - HP LoadRunner, HP QTP, IBM Rational Tester, Microsoft VSTS - are hugely expensive. Organizations can't afford to equip developers and testers with their own licensed copies. These tools licenses are contrary to Agile testing, where developers and testers work side-by-side building and testing concurrently.

  • Many testers still cannot write test code. Agile developers write unit tests in high level languages (Java, C#, PHP, Ruby.) Testers need a code-less way to repurpose these tests into functional tests, load and performance tests, and production service monitors.

  • Business managers need a code-less way to define the software release requirements criteria. Agile developers see Test Management tools (like HP Quality Center QC) as a needless extra burden to their software development effort. Agile developers are hugely attracted to Continuous Integration (CI) tools like Hudson, Jenkins, Cruise Control, and Bamboo. Business managers need anintegrated CI and test platform to define requirements and see how close to 'shipping' is their application.
Lucky for you there is a way to learn how to solve these problems and deliver Agile software development methodology benefits to your organization. The Agile Web Application Performance Testing Workshop is your place to learn the Agile Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices to follow. Plus, you will see a demonstration of a dozen open source test tools all working together.

Registration is free! Click here to learn more and register now:

Register Now

Categories: Companies, Open Source

Free Help To Learn TestMaker, Selenium, Sahi, soapUI

The Cohen Blog — PushToTest - Fri, 01/06/2012 - 05:57
Help Is Here To Learn TestMaker, Selenium, Sahi, soapUI Do you sometimes feel alone? Have you been trying any of the following:
  • Writing Load Test Scripts
  • Building Functional Tests for Smoke and Regression Testing
  • Trying to use Selenium IDE and needing a good tutorial
  • Configuring test management tools working with TestMaker, Sahi, and soapUI
  • Needing To Compare Selenium Vs HP QuickTest Pro (QTP)
  • Stuck While Doing Cloud Computing Testing
  • Need Help Getting Starting with Load Testing Tools
If you feel stuck, need help, or would like to see how professional testers solve these situations, then please attend a free live weekly Webinar.

Register Now

Here Is What We Have For You Bring your best questions, issues, and bug reports on installing, configuring, and using PushToTest TestMaker to our free weekly Workshop via live Webinar. PushToTest experts will be available to answer your questions.

Frank Cohen, CEO and Founder at PushToTest, and members of the PushToTest technical team will answer your questions, show you where to find solutions, and take your feedback for feature enhancements and bug reports.

Every Thursday at 1 pm Pacific time (GMT-8)
Registration Required

At the Webinar:
  1. Register for the Webinar in-advance
  2. Log-in to the Webinar at the given day and time
  3. Each person that logs-in will have their turn, ask their question, and hear our response
  4. You may optionally share/show your desktop for the organizers to see what is going wrong and offer a solution
  5. The organizers will hear as many questions as will fit in 1 hour. No guarantee that everyone will be served.
See how these tools were made to work together. Bring your best questions for an immediate answer!

Register Now

Categories: Companies, Open Source

Free Training Selenium IDE soapUI TestMaker PushToTest

The Cohen Blog — PushToTest - Wed, 01/04/2012 - 16:43
A Look Forward To Open Source Load Testing Tools Albert EinsteinThomas EdisonMarie Skodowska CurieNokola Tesla

You and I have come after some incredibly smart people. They inspire us to do our best when testing software for functionality, performance under load, and scalability.

The problems you need to solve are testing applications and business processes that use Rich Internet Application (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) SOA, BPM, and SOAP and REST Web Service interfaces.

Thankfully you don’t have to be Einstein, Edison, Curie, or Tesla to “get” this stuff. You just need a good set of free open source test tools, a good methodology, and a good coach.
Upcoming Free Webinar Workshops On Open Source Load Testing
PushToTest will host 6 free Workshops via live Webinar in January 2012. Each Workshop features training for  performance testing using Selenium, soapUI, Sahi, JUnit, and TestMaker. Registration is free. Sign-up now while seats last.

Agile Open Source Performance Test Workshop for CIOs, CTOs, Business Managers
January 4, 2012
 
Agile Open Source Performance Test Workshop for Developers, Testers, IT Managers
January 5, 2012
  
Open Source Test Workshop for CIOs, CTOs, Business Managers
January 11, 2012
 
Selenium, soapUI, Sahi, TestMaker Workshop for Testers, Developers, IT Ops
January 12, 2012
 
Use Selenium, soapUI, Sahi, TestMaker Performance Testing In Your Organization
January 25, 2012
 
Load Testing Using Agile Open Source Tools for Developers, Testers, IT Managers
January 26, 2012
 
Agile Open Source Performance Test Workshop for CIOs, CTOs, Business Managers
February 14, 2012
 
Agile Open Source Performance Test Workshop for Developers, Testers, IT Managers
February 16, 2012
 
Free Webinar: Solve Performance Bottlenecks and Function Problems In Your Web Applications
February 22, 2012
 
Open Source Test Workshop for Developers, Testers, IT Ops
February 23, 2012

All Workshops are free, registration is limited, and this is an interactive Webinar where you ask your best questions.

Categories: Companies, Open Source

Don’t Cross the Beams: Avoiding Interference Between Horizontal and Vertical Refactorings

JUnit Max - Kent Beck - Tue, 09/20/2011 - 03:32

As many of my pair programming partners could tell you, I have the annoying habit of saying “Stop thinking” during refactoring. I’ve always known this isn’t exactly what I meant, because I can’t mean it literally, but I’ve never had a better explanation of what I meant until now. So, apologies y’all, here’s what I wished I had said.

One of the challenges of refactoring is succession–how to slice the work of a refactoring into safe steps and how to order those steps. The two factors complicating succession in refactoring are efficiency and uncertainty. Working in safe steps it’s imperative to take those steps as quickly as possible to achieve overall efficiency. At the same time, refactorings are frequently uncertain–”I think I can move this field over there, but I’m not sure”–and going down a dead-end at high speed is not actually efficient.

Inexperienced responsive designers can get in a state where they try to move quickly on refactorings that are unlikely to work out, get burned, then move slowly and cautiously on refactorings that are sure to pay off. Sometimes they will make real progress, but go try a risky refactoring before reaching a stable-but-incomplete state. Thinking of refactorings as horizontal and vertical is a heuristic for turning this situation around–eliminating risk quickly and exploiting proven opportunities efficiently.

The other day I was in the middle of a big refactoring when I recognized the difference between horizontal and vertical refactorings and realized that the code we were working on would make a good example (good examples are by far the hardest part of explaining design). The code in question selected a subset of menu items for inclusion in a user interface. The original code was ten if statements in a row. Some of the conditions were similar, but none were identical. Our first step was to extract 10 Choice objects, each of which had an isValid method and a widget method.

before:

if (...choice 1 valid...) {
  add($widget1);
}
if (...choice 2 valid...) {
  add($widget2);
}
... 

after:

$choices = array(new Choice1(), new Choice2(), ...);
foreach ($choices as $each)
  if ($each->isValid())
    add($each->widget());

After we had done this, we noticed that the isValid methods had feature envy. Each of them extracted data from an A and a B and used that data to determine whether the choice would be added.

Choice pulls data from A and B

Choice1 isValid() {
  $data1 = $this->a->data1;
  $data2 = $this->a->data2;
  $data3 = $this->a->b->data3;
  $data4 = $this->a->b->data4;
  return ...some expression of data1-4...;
}

We wanted to move the logic to the data.

Choice calls A which calls B

Choice1 isValid() {
  return $this->a->isChoice1Valid();
}
A isChoice1Valid() {
  return ...some expression of data1-2 && $this-b->isChoice1Valid();
}
Succession

Which Choice should we work on first? Should we move logic to A first and then B, or B first and then A? How much do we work on one Choice before moving to the next? What about other refactoring opportunities we see as we go along? These are the kinds of succession questions that make refactoring an art.

Since we only suspected that it would be possible to move the isValid methods to A, it didn’t matter much which Choice we started with. The first question to answer was, “Can we move logic to A?” We picked Choice. The refactoring worked, so we had code that looked like:

Choice calls A which gets data from B

A isChoice1Valid() {
  $data3 = $this->b->data3;
  $data4 = $this->b->data4;
  return ...some expression of data1-4...;
}

Again we had a succession decision. Do we move part of the logic along to B or do we go on to the next Choice? I pushed for a change of direction, to go on to the next Choice. I had a couple of reasons:

  • The code was already clearly cleaner and I wanted to realize that value if possible by refactoring all of the Choices.
  • One of the other Choices might still be a problem, and the further we went with our current line of refactoring, the more time we would waste if we hit a dead end and had to backtrack.

The first refactoring (move a method to A) is a vertical refactoring. I think of it as moving a method or field up or down the call stack, hence the “vertical” tag. The phase of refactoring where we repeat our success with a bunch of siblings is horizontal, by contrast, because there is no clear ordering between, in our case, the different Choices.

Because we knew that moving the method into A could work, while we were refactoring the other Choices we paid attention to optimization. We tried to come up with creative ways to accomplish the same refactoring safely, but with fewer steps by composing various smaller refactorings in different ways. By putting our heads down and getting through the other nine Choices, we got them done quickly and validated that none of them contained hidden complexities that would invalidate our plan.

Doing the same thing ten times in a row is boring. Half way through my partner started getting good ideas about how to move some of the functionality to B. That’s when I told him to stop thinking. I don’t actually want him to stop thinking, I just wanted him to stay focused on what we were doing. There’s no sense pounding a piton in half way then stopping because you see where you want to pound the next one in.

As it turned out, by the time we were done moving logic to A, we were tired enough that resting was our most productive activity. However, we had code in a consistent state (all the implementations of isValid simply delegated to A) and we knew exactly what we wanted to do next.

Conclusion

Not all refactorings require horizontal phases. If you have one big ugly method, you create a Method Object for it, and break the method into tidy shiny pieces, you may be working vertically the whole time. However, when you have multiple callers to refactor or multiple implementors to refactor, it’s time to begin paying attention to going back and forth between vertical and horizontal, keeping the two separate, and staying aware of how deep to push the vertical refactorings.

Keeping an index card next to my computer helps me stay focused. When I see the opportunity for a vertical refactoring in the midst of a horizontal phase (or vice versa) I jot the idea down on the card and get back to what I was doing. This allows me to efficiently finish one job before moving onto the next, while at the same time not losing any good ideas. At its best, this process feels like meditation, where you stay aware of your breath and don’t get caught in the spiral of your own thoughts.

Categories: Open Source

My Ideal Job Description

JUnit Max - Kent Beck - Mon, 08/29/2011 - 21:30

September 2014

To Whom It May Concern,

I am writing this letter of recommendation on behalf of Kent Beck. He has been here for three years in a complicated role and we have been satisfied with his performance, so I will take a moment to describe what he has done and what he has done for us.

The basic constraint we faced three years ago was that exploding business opportunities demanded more engineering capacity than we could easily provide through hiring. We brought Kent on board with the premise that he would help our existing and new engineers be more effective as a team. He has enhanced our ability to grow and prosper while hiring at a sane pace.

Kent began by working on product features. This established credibility with the engineers and gave him a solid understanding of our codebase. He wasn’t able to work independently on our most complicated code, but he found small features that contributed and worked with teams on bigger features. He has continued working on features off and on the whole time he has been here.

Over time he shifted much of his programming to tool building. The tools he started have become an integral part of how we work. We also grew comfortable moving him to “hot spot” teams that had performance, reliability, or teamwork problems. He was generally successful at helping these teams get back on track.

At first we weren’t sure about his work-from-home policy. In the end it clearly kept him from getting as much done as he would have had he been on site every day, but it wasn’t an insurmountable problem. He visited HQ frequently enough to maintain key relationships and meet new engineers.

When he asked that research & publication on software design be part of his official duties, we were frankly skeptical. His research has turned into one of the most valuable of his activities. Our engineers have had early access to revolutionary design ideas and design-savvy recruits have been attracted by our public sponsorship of Kent’s blog, video series, and recently-published book. His research also drove much of the tool building I mentioned earlier.

Kent is not always the easiest employee to manage. His short attention span means that sometimes you will need to remind him to finish tasks. If he suddenly stops communicating, he has almost certainly gone down a rat hole and would benefit from a firm reminder to stay connected with the goals of the company. His compensation didn’t really fit into our existing structure, but he was flexible about making that part of the relationship work.

The biggest impact of Kent’s presence has been his personal relationships with individual engineers. Kent has spent thousands of hours pair programming remotely. Engineers he pairs with regularly show a marked improvement in programming skill, engineering intuition, and sometimes interpersonal skills. I am a good example. I came here full of ideas and energy but frustrated that no one would listen to me. From working with Kent I learned leadership skills, patience, and empathy, culminating in my recent promotion to director of development.

I understand Kent’s desire to move on, and I wish him well. If you are building an engineering culture focused on skill, responsibility and accountability, I recommend that you consider him for a position.

 

===============================================

I used the above as an exercise to help try to understand the connection between what I would like to do and what others might see as valuable. My needs are:

  • Predictability. After 15 years as a consultant, I am willing to trade some freedom for a more predictable employer and income. I don’t mind (actually I prefer) that the work itself be varied, but the stress of variability has been amplified by having two kids in college at the same time (& for several more years).
  • Belonging. I have really appreciated feeling part of a team for the last eight months & didn’t know how much I missed it as a consultant.
  • Purpose. I’ve been working since I was 18 to improve the work of programmers, but I also crave a larger sense of purpose. I’d like to be able to answer the question, “Improved programming toward what social goal?”
Categories: Open Source

Thu, 01/01/1970 - 02:00