Jason Huggins, one of the founders of Selenium, spoke at the Agile Developer user group meeting recently. Matt Raible of DZone gives a good summary of Jason's comments. Among them:

 

• Selenium started at Thoughtworks. They were challenged to fix an Ajax bug in their expense reporting system. JWebUnit, HtmlUnit, JsUnit, Driftwood, and FIT did not meet their needs. They invented Selenese as a notation for tests first. 
• Selenium Core, as a JavaScript embedded test playing robot, came next. Then Selenium RC and Grid.
• Selenium test playback is slow. Parallelization can solve some of the slowness problems.
• JavaScript sandbox, Flash, Applets, Silverlight, and HTML 5's Canvas all present problems in Selenium.

 

Read the article here.

 

Here are my thoughts: 

Thanks for the good write-up. Too bad about your battery. I would like to hear more.

PushToTest integrated Selenium into TestMaker a couple of years ago. Selenium works very well for testing Ajax applications. And, TestMaker runs Selenium tests as functional tests, load and performance tests, and business service monitors. TestMaker runs these tests in your QA lab, in the Cloud, or both. See http://www.pushtotest.com/products/cloudtesting
Jason's comments about FlexMonkey and GPL are an interesting take.  FlexMonkey is a GPL implementation of the new Adobe Automation API. The Automation API defines a Selenese-like testing language. So it's the API that needs to be implemented, not adoption of FlexMonkey. PushToTest is working on a contribution to Selenium IDE that implements the Automation API and test language. That will provide a standard way to test Flex applications within the Selenium (and consequently TestMaker) environments. http://www.adobe.com/devnet/flex/samples/custom_automated/    We are also building a new Ajax-based record/playback utility that will output Selenese tests. This is aimed at providing an open source alternative to QTP. The design is available at http://downloads.pushtotest.com/201001/TM_IDE_Design.pdf    Lastly, I've written up a comparison of QTP to Selenium at http://www.pushtotest.com/docs/thecohenblog/web-testing-tools-comparison

 

Thanks!

-Frank

 

CMC Media produced a set of interviews at the STARWest testing conference. They asked a dozen thought leaders in the testing space:

"Is agile affecting testing?"

Watch the video interviews at: 

http://www.youtube.com/watch?v=zhITAFaITCA

 

I am part of the interviews. Here is my opinion:

"Seems like 2008 was the year that a lot of people came to the realization that using waterfall techniques to build software incrementally - where you went from requirements into development and then into testing - is pretty much an easy way to kill your developers and testers. It is just to hard to in an Ajax environment or an SOA environment, to use waterfall. 

Agile, on the other hand, says "Let's test first! Let's build the interfaces to the application first." That's great. Agile also brings peer development and a bunch of other methodologies.

I am seeing testing move up to leverage Agile techniques as well. When you are building new applications and services using some of the new frameworks there is no reason you can't build the test at the same time you build the application. That is beyond what Agile itself originally said. Agile wants you to do test first. I say build and test at the same time using Agile techniques."

-Frank

 

PushToTest introduced Cloud Testing in TestMaker last year. The results have been outstanding. In 2009 we saved our customers $650,000 in fees that they would have otherwise spent on building out their own QA labs. With this enabling TestMaker technology, customers run their automated tests using continuous integration servers like Bamboo and Hudson.

Cloud computing environments used to be the pervue of businesses like Amazon, Collabnet, and Rackspace. An open source software stack emerges to build and manage your own cloud:

• Ubuntu Enterprise Cloud combines Ubuntu Linux with Eucalyptus and other cloud management tools. Eucalyptus joined the world of commercial open source with the launch of Eucalyptus Systems in 2009. Eucalyptus incorporates the Apache Axis2 Web services engine, Mule enterprise service bus, Rampart security, and Libvirt virtualization. Eucalyptus comes with its own implementation of the Amazon API.
  
• The Red Hat Deltacloud Portal manages all of the cloud deployments that may exist within an environment, providing a common integration platform. Deltacloud helps ease the integration between public and private clouds. Deltacloud creates a REST-based API to map to Amazon EC2 and private clouds that use VMware or Red Hat Enterprise Linux.
  
• Nimbus is an open source toolkit that turns clusters into a cloud environment. Nimbus has a suprisingly complete set of APIs and management systems to provision customized compute nodes.
  
• PHP developers have Simple Cloud API from Zend Technologies to manage cloud services from GoGrid, IBM, Microsoft, Nirvanix Storage Delivery Network, and Rackspace Files. Java developers have Typica to access Amazon's SQS, EC2, SimpleDB and DevPay LS web services. We use Typica in TestMaker with great results.
  
• PushToTest TestMaker repurposes Selenium, soapUI, and Java, Ruby, Perl, PHP unit tests as functional tests, load and performance tests, and business service monitors. TestMaker operates tests from a private cloud, public cloud, or both.

Thanks to Michael Biddick's article in Information Week to motivate me to write this blog entry.

 

Frank

Download the Press Release (Adobe Acrobat PDF format, 81 Kbytes)

 

CAMPBELL, Calif. and BRISBANE, Calif., Jan. 21, 2010 —  PushToTest® and CollabNet® announced today an expansion of their business relationship to deliver test capability in the cloud – backed by the latest releases of CollabNet TeamForge® and PushToTest TestMaker™ for enterprise clients, government organizations, and open communities worldwide.

TestMaker, the open source alternative for test automation, has been added to the CollabNet collabXchange™ marketplace.  CollabNet, the leader in distributed application lifecycle management (ALM) solutions, and PushToTest have integrated TestMaker with the CollabNet platform; organizations can reduce unplanned service interruptions, increase developer and tester productivity, and improve the ability to achieve service level agreements (SLA). 

“We designed PushToTest TestMaker to leverage Cloud Computing. CollabNet TeamForge enables us to deliver huge price and feature advantages to our customers,” said Frank Cohen, chief executive officer, PushToTest, and world-renowned Test Geek. “With TeamForge Lab Management support built into TestMaker 5.3, users click the start button, watch TestMaker create load injector machine instances in their own QA lab or a cloud provider such as Amazon Web Services (EC2), run the test, and then take the machines down. This saves customers huge amounts of budget, relieves them of running their own QA lab, enables tests with up to millions of simulated virtual users, and runs tests from multiple geographic locations – and all on-demand.”

For organizations needing to surface and solve performance bottlenecks and functional problems, TestMaker provides a single tool to verify advanced Rich Internet Applications (RIA), Service Oriented Architecture (SOA,) and Business Process Management (BPM) applications.  

PushToTest supplies the complete test automation platform to deliver functional testing, load testing, and business service monitoring by repurposing the same single test script. CollabNet’s TeamForge Lab Management and CollabNet Subversion reduce time spent on configuring build and test environments and securely managing component reuse libraries or test assets in a centralized repository.

“CollabNet has built the collabXchange to best serve our customer base with options to extend and expand their CollabNet platform,” said Jim Ensell, vice president, business development, CollabNet. “Together, TestMaker and CollabNet TeamForge Lab Management provide a compelling solution for test automation in the cloud.”

 

About PushToTest

PushToTest is the open source application performance management and test automation solutions provider. PushToTest is paving the open source test (OST) solutions migration into the mainstream by making high-quality, low-cost technology accessible to medium and large size global organizations and by providing world class professional services. PushToTest provides a software testing orchestration platform along with cloud testing capabilities and outsourced testing solutions, as well as support, training, and consulting services to customers worldwide and through top-tier partnerships. PushToTest's open source strategy offers customers a long-term plan for building testing infrastructures that are based on and leverage open source technologies with a focus on rapid adoption of the latest technologies, reuse of test asset investments, and ease of management. For more information, visit www.pushtotest.com or send email info@pushtotest.com

 

About CollabNet

CollabNet is the leader in application lifecycle management (ALM) platforms for distributed software development teams.  CollabNet TeamForge is the industry’s most open ALM platform, supporting every environment, methodology, and technology.  With an integrated suite of easy-to-use tools that share a centralized repository, it is the only ALM platform that enables a culture of collaboration, improving productivity 10-50% and reducing the cost of software development by up to 80%.  As the corporate sponsor of the open source Subversion project, the best version control and software configuration management (SCM ) solution for distributed teams, collaborative development is in CollabNet’s DNA.  Millions of users at more than 900 organizations, including Applied Biosystems, Capgemini, Deutsche Bank, Reuters, and the U.S. Department of Defense, have transformed the way they develop software with CollabNet.  For more information, visit www.collab.net or call (888) 778-9793.

 

Media Contact:

Paige Schoknecht

Prequent, Inc. (for CollabNet)

+1 (408) 275-1419 office

+1 (650) 223-4085 mobile

paige@prequent.com

 

Frank Cohen

PushToTest

+1 (408) 871-0122

info@pushtotest.com

TestMaker 5.4 is a major feature enhancement release. The new software includes major feature improvements in the following areas: 

• Easier and More Productive User Interface - The new TestMaker Editor is a fun and easy graphical tool to build and operate tests.
• Continuous Integration Features - Compatibility with Hudson, Bamboo, Collabnet TeamForge, Ant, JUnit, and other continuous integration environments
• Updated Tutorials - 30-minute tutorials on building tests of Ajax applications using Selenium, building tests of SOAP-based services using Selenium, and building tests of Flex-based services using TestMaker AMF protocols.
• Improvements to Selenium IDE - We began to make changes to Selenium IDE to improve its support for Ajax-based Web applications.
• Bug Fixes - Dozens of small fixes and improvements.

The project includes contributions from dozens of engineers in the TestMaker open-source community and PushToTest funded engineering effort by Andrew Zuercher, Ali Faiz, William Martinez Pomares, Luis Carlos Lara Lopez, and Peter Schumacher.

We are planning to release TestMaker 5.4 next week. We would appreciate your feedback now. Please try one of the download links for TestMaker Enterprise. 

For Macintosh users:

http://tm53rc4.s3.amazonaws.com/PushToTest_TestMaker_Install_Mac.zip

 

For Windows users:

http://tm53rc4.s3.amazonaws.com/PushToTest_TestMaker_Install_Windows.zip

 

For Linux/Unix users:

http://tm53rc4.s3.amazonaws.com/PushToTest_TestMaker_Install_Linux.tar.gz

 

Barring no show-stoppers, this is the final release candidate and the final 5.4 software.

Please let me know any issues you run into by posting a message to the support forums. 

Thank you.

-Frank

 

The Open Source Test Workshop is a free meet-up for people and organizations needing to test information systems for functionality, load and performance, and service delivery. PushToTest hosts the meet-ups. One recent student to the Workshop said this afterwards:

"Thank you so much for inviting and presenting a great useful webinar. The information present there was excellent, I got lots of information. Just after the webinar I google the importance and advantage of selenium over the QTP as tools available in market. I underestimated this webinar and didnot prepare myself for it. But I gained alot. Thank you so mcuh. Your webinar has sparked me alot for learning selenium. I would appreciate if you let me know some more good resources for that."

PushToTest recently announced the dates for the Workshop in January and February via live Webinar.

PushToTest hosts two forms of the Open Source Test Workshop:

Workshop for CIOs, CTOs, and Senior Managers

 

 

Workshop for QA Testers, Developers, and IT Managers

 

We hope to see you at the Workshop!

-Frank

PushToTest Global Services provides our customers with test management and business performance management solutions that include our TestMaker solution. We do so by integrating TestMaker into a test management stack that includes the following technology:

• Test case\suite management - PushToTest partners with Collabnet TeamForge and other open source packages to provide test management, bug tracking, and revision control for tests
• Test data management, (parsing , sanitisation & storage) - TestMaker Data Production Library (DPL) technology data enables tests from RDBMS solutions like MySQL. We train your team to build tests using DBUnit for data generation and replication.
• Security test case management - TestMaker supports security testing of X509, SSL, SAML, and application level security in test suites.
• Automated regression with command interpreter - TestMaker repurposes unit tests to be automated regression tests, load and performance tests, and business service monitors from one test.
• Integrated defect management - TestMaker provides integrated defect management through Collabnet TeamForge, Trac, and other ticket system.
• Test result storage and reporting - Fully automation storage and results reporting.
• Key Performance Indicator (KPI) reporting - Provided as part of PushToTest's reporting capability.

PushToTest delivers the above solution, provide training, and on-going support.

We would be glad to discuss this you in more depth. Please contact us.

Frank

Many testers, developers and IT managers ask PushToTest to compare HP QuickTest Professional (QTP) to PushToTest TestMaker with Selenium IDE. PushToTest provides open source test automation solutions for organizations needing to improve the reliability and reduce service interruptions in their Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash,) Web services, and Business Process Management services. PushToTest is the Open Source Test (OST) expert for dozens of tools. This comparison document helps organizations looking for an inexpensive open source solution that works with their latest applications.

Download the full report: TestMaker_Selenium_QTP Comparison (528 K bytes, Adobe Acrobat PDF format)

Here is the high-level executive summary of the report: 

HP QTP 10 TestMaker 5.4/Selenium 1.0 Technical experience required to be successful x  xxx
Web 1.0 Record/Playback x x Rich Internet Application (RIA, Using Ajax) Support x Drill down to root cause of functional issues/crashes xxx x Record Tests MS IE 6, 7, 8 Firefox 2, 3 Playback Tests MS IE 6, 7, 8 MS IE 5, 6, 7, 8, Firefox 2, 3, Opera, Safari, Chrome Data-driven Testing x x Reusable Test Components x Agile software development lifecycle (SDLC) support x Add-ons available xxx x Language Support VBScript Selenese, Java, Ruby, Perl, PHP, Python, C#, Groovy Functional Testing, Load Testing, Service Monitoring from one test script x Run tests in the Cloud, in your QA lab, or both x Price $6000 USD Per Seat, Plus $1500 Annual Maintenance Free (GPL 2, Apache 2)

 

PushToTest intends to update this document from time to time. Please let us know your input.