Having just finished Steve Jobs biography, and being of the school of gated platforms – at least for my phone, where I don’t want to deal with bugs the way I might in my work laptop (sorry Matt B and the uTest IT team) – I found this concept very interesting.

According to the BI article, “Android Hackers Plan App Store of Banned Apps,” a group of Android Developers are looking to start their own app store for all the banned and rejected apps that didn’t make the cut.  The article includes a quote from the potential founder that, “apps removed from the Market include, one-click root apps, emulators, tether apps, Visual Voicemail apps, and more.”

It sounds great but we already know about the growing number of malware on phone operating systems, the Android especially.  The other alternative for apps is to create mobile-specific landing pages (i.e. HTML5 apps), like Grooveshark (music) and Uptappd (beer reviews) have done, making the apps available via your mobile browser.  Since their launch, Untappd has launched a native app for iOS and Android but has not shared details on traffic comparisons.  [It won’t be applicable to most mobile users but we cover some security exploits and common attacks in our Security Testing whitepaper.]

Am I the only one uber-sensitive about the integrity of my phones OS and Apps?  Would you download an app that isn’t scrutinized for security?

Just encountered a collegue having difficulties ensuring if a folder exists. And if a folder does not exists then it has to be created in its full path, not only the lowest level folder. The code was bloated with if/then’s, instr(), instrrev() and left()/right() functions. Even the lowest level folder name was hardcoded.

To help him out, we could go for the fancy recursive solution to show off our 1337 programming skillz, but I decided to just do it with a simple loop. Enjoy.

Public Function EnsureFolder(fPath)
    dim fso, fols, fol, pathBuild
    set fso=CreateObject("Scripting.FileSystemObject")

    fols = split(fpath, "\")

    For each fol in fols
        If fol = "" Then exit for
        pathBuild = pathBuild & fol & "\"
        If Not fso.FolderExists(pathBuild) Then
            Call fso.CreateFolder(pathBuild)
        End If
    Next

    EnsureFolder = pathBuild
End Function

When writing a file modification script I find it annoying (read: boring) to write the logic of saving the file under a different name, moving to override original and then cleaning up. This is what I recently had to do and instead decided to look around at what other options were available, one of which was sed.

The following command will replace the word testdomain with proddomain.

$ echo 'www.testdomain.com' > test.txt
$ cat test.txt 
www.testdomain.com  
$ sed -i '' s/testdomain/proddomain/ test.txt
$ cat test.txt
www.proddomain.com

The key to how this works is the -i '' argument of the sed command. -i specifies the file extension, by passing an empty string it uses the same file.

SOA Testing with Open Source Test Tools PushToTest helps organizations with large scale Service Oriented Architecture (SOA) applications achieve high performance and functional service delivery. But, it does not happen at the end of SOA application development. Success with SOA at Best Buy requires an Agile approach to software development and testing, on-site coaching, test management, and great SOA oriented test tools.

Distributing the work of performance testing through an Agile epoc, story, and sprints reduces the testing effort overall and informs the organization's business managers on the service's performance. The biggest problem I see is keeping the testing transparent so that anyone - tester, developer, IT Ops, business manager, architect - follows a requirement down to the actual test results.

With the right tools, methodology, and coaching an organization gets the following:

• Process identification and re-engineering for Test Driven Development (TDD)
  
• Installation and configuration of a best-in-class SOA Test Orchestration Platform to enable rapid test development of re-usable test assets for functional testing, load and performance testing and production monitoring
• Integration with the organization's systems, including test management (for example, Rally and HP QC) and service asset management (for example, HP Systinet)
• Construction of the organization's end-to-end tests with a team of PushToTest Global Professional Services, using this system and training of the existing organization's testers, Subject Matter Experts, and Developers to build and operate tests
• On-going technical support

On-Site Coaching Leads To Certification
The key to high quality and reliable SOA service delivery is to practice an always-on management style. That requires on-site coaching. In a typical organization the coaches accomplish the following:

• Test architects and test developers work with the existing Testing Team members. They bring expert knowledge of the test tools. Most important is their knowledge of how to go from concept to test coding/scripting
  
• Technical coaching on test automation to ensure that team members follow defined management processes
  

Cumulatively this effort is referred to as "Certification". When the development team produces quality product as demonstrated by simple functional tests, then the partner QA teams take these projects and employ "best practice" test automation techniques. The resulting automated tests integrate with the requirements system (for example, Rally), the continuous integration system, and the governance systems (for example, HP Systinet.)
Agile, Test Management, and Roles in SOA
Agile software development process normally focuses first on functional testing - smoke tests, regression test, and integration tests. Agile applied to SOA service development deliverables support the overall vision and business model for the new software. At a minimum we should expect:

1. Product Owner defines User Stories
2. Test Developer defines Test Cases
3. Product team translates Test Cases into soapUI, TestMaker Designer, and Java project implementations
4. Test Developer wraps test cases into Test Scenarios and creates an easily accessible test record associated to the test management service
   
5. Any team member follows a User Story down into associated tests. From there they can view past results or execute tests again.
6. As tests execute the test management system creates "Test Execution Records" showing the test results
   

Learn how PushToTest improves your SOA software development life cycle. Click here to learn how.

Answer: In-the-wild testing.

This according to our very own CMO Matt Johnston, who recently sat down with Rich Hand of softwaretestpro.com for an in-depth discussion on the growing importance of in-the-wild testing. This lively Q&A was then adapted into a featured article  (no word yet on its big screen debut).

Anyway, if you’re new to the concept of in-the-wild testing or want to know more about it, then I highly recommend giving it a proper read.  Here are a few clips to get you started.  Enjoy!

Does it ever seem that no matter how much time, effort and money your QA team spends improving and refining software testing processes that some sort of defect is always found in the application, website, or mobile app after launch? Surprisingly, it has little to do with your organization’s in-the-lab testing – whether in-house or outsourced, manual or automated. In fact, it’s likely due to the fact that the lab environment you’re testing in cannot adequately replicate real-world conditions (real users, real devices, diverse locations, imperfect connectivity, not to mention a range of devices, operating systems, browsers, etc.).

And thus, companies test extensively in the lab, launch their apps into the real-world conditions of users, and we’re all surprised when these products don’t perform as expected. But what kind of testing can fill this gap effectively and affordably? It was out of this persistent question that uTest and crowdsourced testing was born. And the by-product of this was a new category of testing which has become a must-have for mobile, social and local apps known as “In-The-Wild Testing.”

“In-The-Wild Testing” (ITWT) is an effort to educate tech leaders about how to help QA teams and organizations launch higher quality software, quicker, faster, and cheaper. The idea of in-the-wild testing is about providing organizations with the real-world testing data necessary to make informed decisions about releasing products to market. According to Matt Johnston, Chief Marketing Officer for uTest, “Don’t be fooled by the word ‘wild’ when it comes to testing software. When you think of the term ‘In-the-wild testing’ think of it as ‘real-world vs. laboratory conditions.’” This is not outsourcing or beta testing, and it’s definitely not suggesting you replace the QA teams or solid processes you have in place within your test lab. Rather, this is about complementing, scaling, and aligning professional testing resources with your in-house or outsourced QA team. I predict that this concept will explode in the next five years . But the first step is to understand what ITWT is (and isn’t).

Continue reading >>>

For a while now, CodeCollaborator has had support for creating code reviews from changes located in a Git repository.  ...

This article presents a strategy for unit testing in Python, using a set of utility functions that converts to and from Roman numerals. The author discusses the benefits of a Test-Driven Development (TDD) approach and propose a detailed testing process for all the cases of the functions.

 The FDA General Principles Of Software ValidationThe FDA requires software validation for compliance with their Quality System regulation. Outlining how to achieve effective software validation, the FDA’s General Principles of Software Validation recommends a comprehensive software development lifecycle (SDLC) that integrates risk management strategies with principles for software validation.
An integrated SDLC integrates validation and verification activities such as peer code reviews, static analysis, unit testing, manual testing, and regression testing throughout the entire software development lifecycle. Parasoft supports the FDA’s vision of an integrated SDLC for C, C++, Java, and .NET with Parasoft Concerto for Medical Device Software Development.
This paper outlines what the FDA expects from software development organizations, and how Parasoft helps you meet those expectations.

Read More here ...

I have a lot of respect for Node, but sometimes certain npm packages become out of sync with the latest version and break. Today, that npm package was node-compress.

While the package looked to build, when attempting to require the module it would error.


$ node
> var c = require('compress')
Error: Cannot find module 'compress'


After a quick look around on GitHub I found a fork with the fix https://github.com/elliotttf/node-compress/

In your package.json, simply reference the tarball, clean out the node_modules directory, install and everything should work again.

Package.json

    , "compress":"https://github.com/elliotttf/node-compress/tarball/1edaa48bf33f7c836f1e275691e1d8645f0a71c3"

Wow. Chris Kemp (Advance technical solutions team at Salesforce.com) has create a great short video based on my blog post about How To Deploy a .NET App with Mono on Heroku. A huge thank you!



Link: http://www.youtube.com/watch?v=LkdiTzgqqsk&hd;=1

It’s alwys amazing how every year the month of January sparks the desire in many people to reflect on past promises, challenges and successes. Once the moment has gone, a sense of clarity and a surge of determination takes root, and a new list of promises kick-starts a change and a passion to do well [...]

Today I attended an excellent presentation by Denali Lumma at the Selenium SF meetup. Especially pleasant for me was an “Aha!, I told you so” moment when she described Okta Continuous Integration setup that relied on Git Top Branches and Selective testing. When I ran CI for the Google Checkout project, the lowest point of [...]

Today, EnterpriseDB announced the availability of Postgres Plus Cloud Database. Some of the advantages of this DBaaS (Database as a Service) are point-and-click provisioning, online backups, automatic scaling and failover.

Developers who prefer Postgres can now use this service with the CloudBees Platform as a Service (PaaS) to build database-backed applications.

Configuring web applications (on CloudBees) to talk to the database requires minimal configuration changes. Developers end up just changing their datasource configuration settings.

I'd recommend that developers change an existing application to play with this database. The delta upload feature in CloudBees (where only touched changes are uploaded during redeploys) makes this process fairly painless (dare I say enjoyable :-)).

A detailed article that builds a web application from scratch, configures Maven to pull in the right Postgres libraries and uses JPA and JDBC is available here. The source code of the application is available on github. Feel free to use the code to get started quickly.

- Harpreet Singh, CloudBees

With every new release of CodeCollaborator we have a lot to talk about, and Version 7.0 is no exception! In ...

Over the last year or so, interest in and development for Node.js has accelerated dramatically. There is no shortage of excitement and promise in this burgeoning community and for many of the same reasons cited by others, about six months ago we decided it made sense to dive in ourselves.

Coming from a committed testing background on multiple web frameworks, including Rails, we are strong adherents to BDD. Naturally the first thing we did when spinning up on the Node stack was look around for good BDD acceptance testing tools. Not finding anything that fit for us, we went ahead and rolled our own. In this post, we’ll introduce GivenWhenThen, a DSL and runner that allows anyone to easily construct acceptance tests with BDD semantics in straightforward, sentence-like statements and then run them against Sauce Labs.

Writing a Story

Although you can also directly access the Selenium RemoteWebDriver that is under the covers, GivenWhenThen is primarily intended to provide a DSL for writing executable stories in the Dan North format. To write a story, begin with a BDD description in a xxx_test.coffee file:

 

story 'Executing a Google search',
  """
  As a human
  I want to perform a search
  So that I can access the world's information
  """, ->

 

And then add one or more scenarios:

scenario "Search for info about Node.js", (browser) ->
  browser
    .given "I am on the homepage", ->
      browser.step(steps.visitHomepage)
    .when "I enter search terms", ->
      browser.typeInElement('q', 'nodejs', using:'name')
    .and "submit the search", ->
      browser.clickElement('btnG', using:'name')
    .then "I see search results", ->
      browser.assertTextPresent('results')
    .and "the results contain information about nodejs", ->
      browser
        .assertTextPresent('node.js')
        .assertTextPresent('nodejs.org')

 

Each scenario has “given“, “when“, and “then” steps.

• Given: Set up the initial conditions for the scenario.
• When: Take the action the scenario is testing.
• Then: Assert the conditions expected after taking the tested action.

Each step contains one or more chained calls to WebDriver commands in the form of browser.someCommand.

Each step (given, when, then) can have an arbitrary number of and steps following it (see above example).

Steps Often there are steps that are repeated throughout many scenarios, for example “visit homepage” or “sign in”. This kind of functionality can be defined in steps and referred to in scenarios via browser.step():

 

scenario "Search for info about Node.js", (browser) ->
  browser
    .given "I am on the homepage", ->
      browser.step(steps.visitHomepage)

 

Steps are defined in *_steps.coffee files. Multiple steps per file can be defined as follows:

 

steps.visitHomepage = (browser) -> browser.get 'http://www.google.com'

 

Multiple steps files can be created to organize your steps in a manageable way.

Configuration Configuration is simple and contains:

• Overall story and Sauce Labs configuration.
• Browser / OS definitions. Stories will be run against each browser/os configuration defined.

For example:

config.credentials =
  'username':   'sauce_labs_username'
  'access-key': 'sauce_labs_access_key'

config.settings =
  'max-duration': '180'

config.browsers = [
  {
    'platform':     'VISTA'
    'browserName':  'firefox'
    'version':      '7'
  }
  {
    'platform':     'LINUX'
    'browserName':  'firefox'
    'version':      '7'
  }
]

 

Subtitles In Test Videos One of our favorite features of Sauce Labs is test videos. It is incredibly useful to be able to view a video of a failed test and quickly understand what went wrong. However, one issue that has been raised about the videos is that they could sometimes benefit from some context about what is happening on screen.Well, it just so happens that this context is conveniently codified in the steps of BDD scenarios! So, by flipping a switch in GivenWhenThen, you can add BDD subtitles to your Sauce test videos. When the subtitle flag is passed to the GivenWhenThen runner, subtitle divs will be inserted into the test browser, which will show up in the Sauce Labs test videos and illustrate which of the BDD steps is currently being executed:

 

 

Design Background

In addition to being built on top of Sauce Labs and Selenium 2, GivenWhenThen is heavily influenced by the BDD movement.

While it is also clearly influenced by Cucumber, aficionados will surely notice differences. Most obviously, in GivenWhenThen there is no prose-to-code translation step. Cucumber uses the Gherkin language, which parses the prose into features, scenarios, and steps.

In GivenWhenThen, the DSL is embedded directly in the host language. We do realize there is real benefit to the full Cucumber vision – including Gherkin – but we took a leaner approach to getting the BDD structure without completely porting Cucumber.  To us, the core power of the BDD approach comes from the conceptual framework Dan North developed. Just stating the story and scenario in that formal way and using the “given, when, then” flow gives powerful structure and coherence to your specifications.

Conclusion GivenWhenThen fills a need we have by providing a Selenium 2 driver and Sauce Labs integration for Node.js, and adding BDD structure to our tests, while staying light and keeping us close to the host language. We hope you find it useful as well!

PIT is a fast mutation testing system for Java that makes it possible to test the effectiveness of your unit tests.

You can think of mutation testing as either as an automated test of your tests, or as a much more in depth form of code coverage.

PIT runs your tests against automatically modified versions of your application code. When the application code changes, it should cause a test to fail. If a test does not fail it may indicate an weakness in the suite.

See http://pitest.org for further details.

TestLodge recently announced their latest feature that allows Assembla users to link their accounts to TestLodge's test case management tools. This integration makes many essential software testing features available to Assembla users without having to manually enter data for their tests and bug tickets multiple times. TestLodge created this custom integration using Assembla's REST API.

This first iteration from TestLodge offers the following features:
 

• Automatically create tickets in Assembla in the background when a test fails. Each ticket includes a description and steps to replicate, along with the expected result and actual result.
• Optionally, choose who the Assembla ticket should be assigned to, and set the priority.
• Associate an Assembla project workspace with a TestLodge project.
• View a test report that provides links to all created tickets.

About TestLodge TestLodge is a relatively new hosted tool that is designed to be a lot simpler than the traditional test management software by providing only the essentials to get the job done well. The system focuses on helping you create your test plans, input your requirements, create and manage your test suites & cases, and allow you to easily perform multiple test runs & generate reports.
Special offer As an introductory offer, TestLodge is offering a 20% discount for a limited time to anyone who makes use of the new Assembla integration. Simply sign up for your TestLodge account, then email TestLodge about the discount within a week of signing up.

A number of people have asked me this, since our new book is now out.

This book took us 2 and a half years. This doesn’t include the effort put in by the case study authors and other contributors, so this book represents a lot of work! What exactly had we been doing all that time? I wondered that too, so here is where the time went.

In August 2009, I have my first note of our plan to solicit contributions for a new book on automation experience. We sent emails, put a call for contributions on my web site, and talked to people at conferences, and began gathering potential contributions.

I started keeping track of the hours we spent from December 2009. We had a big initial “push” (the first peak on the graph) and produced a “protobook” – 4 chapters with an introduction. We were sure this would be snapped up by a publisher!

We submitted to the publisher of our previous book in mid-February, but initially they weren’t very keen! This was a blow, as we were convinced this would be a great book! I tried several other publishers over the next few months, and got rejected; I continued to try and convince Pearson/Addison Wesley that they should publish our book.

They eventually relented in July and we signed a contract. We worked steadily on the book over the rest of that year, a total of around 400 hours between us. The complete draft manuscript, ready for independent review was due on the 15th April, and the final manuscript on the 15th October. “No problem”, we thought.

However, we found that we did need to work at a more intense level - we spent another 300 hours to the end of April (the double peak with some time out in February), and another 300 hours to the end of 2011. The final peak was editing the final page proofs and doing the index, more work that we had anticipated at that stage. The total for us comes to just under 1000 hours. We don't know how much time the contributors spent, but it their time was equivalent to ours, the collaboration represents around 2000 hours of time - that's more than one working year.

We enjoyed working on this book and reading all the stories as they came in. There are many useful lessons, some heartfelt pain, and many gratifying successes among the stories.

You can follow the book on Twitter on @AutExpBook. The book tweets tips and good points every few days!

Thanks to all the book contributors, and to co-author Mark Fewster.

Ranorex 3.2.1 is now available and includes the following release notes:

General changes/Features

• Added support for Firefox 9.0
• Added support for SVN 1.7 working copies (SVN version configurable via Ranorex Studio settings)
• Added elements for menu- and title bars in Java AWT/Swing applications

Bugs

• Fixed download of Flash debug player to use the default proxy when instrumenting Flash/Flex
• Fixed geometry calculation for Flash/Flex applications with custom scalemode and align settings
• Fixed object recognition issue with Flash/Flex control classes having an empty namespace
• Fixed tracking issue when tracking stacked Flex popup dialogs
• Cell size calculation for certain customized Flex datagrids has been improved
• Fixed object recognition problems with Flex data-bound controls contained in dynamically loaded modules
• Reduced memory consumption of Ranorex Studio (especially when dealing with large data sources)
• Fixed exception thrown when clicking on the very first char in the compiler output pad in Ranorex Studio
• Fixed duplicate output in Debug pad in Ranorex Studio
• Improved support for adding references to HTTPS web services in Ranorex Studio
• Fixed problem causing recordings to be invalid when converting a project to a different programming language
• Fixed exception thrown when renaming variables in recordings or repositories using the variable store editor
• Fixed code generation for modules, repository items, and variables if their names conflict with programming language keywords
• Fixed exceptions thrown when setting the CheckState and Expanded attributes on MSAA elements
• Fixed EnsureVisible for tabs if multiple Firefox windows are opened and closed again
• Fixed detection of Chrome installation directory on certain system configurations
• Improved performance of object recognition for WinForms applications with many resource assemblies
• The UseCache property of newly auto-created repository folders is now correctly set to the configured default
• Improved performance and fixed a number of cosmetic issues related to the Ranorex report stylesheet
• Fixed evaluation of variables in RanoreXPaths within repositories when accessed from user code
• Fixed image capturing problem related to "Aero Peek" feature when perfoming image-based recording

Download Ranorex version 3.2.1 here.
(You can find a direct download link for the latest Ranorex version on the Ranorex Studio start page.)

Today, we’re thrilled to announce the results of our third annual uTester of the Year Awards. Every year, we recognize uTesters who have consistently gone above and beyond their call of duty in their participation with uTest projects. This year’s winners were selected by our community and project management teams, who have had the privilege of working closely with such an extremely talented community of professional testers. From test automation to test team lead, these winners are truly experts in a variety of testing domains. The level of talent continues to impress, with each year’s accolades becoming more and more difficult to attain (and judge). So without further ado, let’s meet our 2011 winners!

Top honor for the 2011 award goes to David Honeyball from the United Kingdom!

David joined uTest in June of 2009. Since joining us, David has become a Gold rated tester in Functional, Load and Localization testing, as well as having achieved the silver rating in both Usability and Security testing. David also became our Top Test Team Lead in 2011, successfully leading nearly 200 projects alongside of uTest project managers. David had this to say about his experience with uTest during the past year:

I joined uTest back in the summer of 2009 and have to say I have never looked back. What started out as something extra in my spare time has taken up more and more of my time in a good way and has increased my confidence as a tester to levels I never thought possible. I have been a tester for nearly 15 years but can safely say that every day with uTest is a new experience and a new challenge.

I have met so many wonderful people including other testers, CMs, PMs and customers who are all committed to achieving their goals and creating a wonderful service. They have guided me and helped me in the last year and increased my communication skills as well.

TTL (Test Team Lead) Experience:

2011 has been a big success for me personally, as it has for uTest in regards to growth and development, in many ways due to my journey from tester to TTL. I started out as a TTL early in the year and felt at the time that it was a great way forward and would be of huge assistance and help to customers. Since then I have been involved in something nearing 100 cycles as TTL, but that could be more now!  I enjoy the TTL role immensely as I get to speak to the PMs and testers and help out others who are stuck. I hope if you have worked with me that you know that I take that side of things very seriously. One of the great rewards is helping someone who is stuck to complete a test case or test for example. Above all, I believe that with my experience I am a very fair TTL and have the best interests of customer and testers close at heart.

As time goes on I hope to grow more and more into this role and improve further as there is still so much to learn.

Special Projects 

Apart from testing and TTL work which does take up some time as you can imagine, I also help with test case writing for certain PMs which I enjoy and get value from. I was also heavily involved in the startup of the test case conversion to uTest which is the new system we see today. This side of things linked in with my testing and other roles forms a vital component of what I do as well and hope it adds value to the company as a whole.

So, just want to say a huge thanks to everyone involved with uTest for the opportunity and long may we continue to grow together and become invaluable to customers across the globe…

The complete list of winners is shown below:

• Most Valuable Tester: David H., United Kingdom
• Top Test Team Lead: David H., United Kingdom
• Top Android Tester: Elena H. , United States
• Top iOS Tester: Bo V., United States
• Top Forums Moderator: Amit K., India
• uTester of the Year: Anand A., Australia
• uTester of the Year: Andy M., United States
• uTester of the Year: Arsiadi S., Indonesia
• uTester of the Year: Atul A., India
• uTester of the Year: Bo V., United States
• uTester of the Year: Brian R., United States
• uTester of the Year: Carl S., United States
• uTester of the Year: Jason Y., Canada
• uTester of the Year: John K., United States
• uTester of the Year: Moritz S., Germany
• uTester of the Year: Nicola S., United Kingdom
• uTester of the Year: Paul T., United Kingdom
• uTester of the Year: Peggy F., United States
• uTester of the Year: Shruthi P., United States
• uTester of the Year: Travis H., United States

We’d like to congratulate and thank all of our 2011 winners. As mentioned in the opening paragraph, each year has seen so much growth and improvement. Therefore, the bar for 2012 will undoubtedly be set even higher! With several weeks into the new year, the competition has already begun for our 2012 accolades. Remember, it is your consistent performance throughout the year that matters, engaging in as many opportunities as you can without sacrificing quality work. With that said, best of luck to our 2012 candidates and please take some time to congratulate our 2011 winners by dropping a comment below!