PushToTest funded a new open source project to deliver a modern Web application test record/playback system. We named the project TestMaker Object Designer (TOD.) Our hope is to move Selenium IDE and other record/playback test tools into the era of modern Web and Rich Internet Applications (RIA, using Ajax, Flex, Flash.) The work on this new tool started in March 2010.

Last week we hosted a special briefing meeting with the PushToTest community to show our progress on the project. We recorded the briefing.

Watch the TestMaker Object Designer (TOD) June 2010 Briefing (45 minutes)

We intend to release a “beta” version of the Designer in August 2010. We will have another briefing shortly before the release.

Please post your comments and feedback here.

Enjoy!

-Frank

PushToTest officially releases TestMaker 5.5 today! TestMaker 5.5 is a major feature enhancement and bug fix release.

Software engineering met Continuous Integration (CI) years ago. Tools like Hudson, Bamboo, Collabnet TeamForge, and Cruise Control automate the build and deploy cycle. Now, PushToTest drives an innovative way to achieve Continuous Testing (CT) in your organization.

New capabilities in TestMaker 5.5 include:

• Grid Testing – TestMaker runs your tests on a Grid of your test equipment, in a Cloud Computing environment, and both.
• New Script Runners to operate tests written in .NET, Visual Basic (VB) and from the command line
• Scalability Index Plus ™ – TestMaker shows graphically where the application under test should be performing with linear scalability
• Reports now show error messages and codes for easy drill-down
• Relational Database logging
• Macintosh native application packaging

Download the new software at http://www.pushtotest.com/index.php/comparison

Read the Release Notes for details.

PushToTest tests TestMaker 5.5 on Windows XP SP2, Windows Vista SP1, Windows 7, Mac OS X 10.5.8, and Ubuntu 9.10 (Karmic Koala).

TestMaker 5.5 and later requires a special Microsoft patch to Windows XP for the Editor to operate correctly.

Last year the European Commission invited me to speak at their meeting of IT managers. During the questions and answers panel the IT managers asked many questions about cloud computing technology. I heard a common theme in the questions: data security and how to handle failures were keeping the IT manager community from moving applications into the cloud.

A friend is the IT manager for a large digital advertising agency. He is tasked with preparing the agency with enough equipment, bandwidth, and test automation capability to serve the agency customers. I asked him the same questions I heard from the EC IT managers: How do you secure customer data in the cloud?

My friend pointed to SAS 70, Cloud Security Alliance, Amazon and Rackspace.

SAS 70 is the Statement on Auditing Standards from the American Institute of Certified Public Accountants (AICPA). SAS 70 is different from the ISO 9000 standard certification in that SAS 70 is not a pre-determined set of standards that a service organization must meet to “pass”. Instead an independent auditor evalutes and audits a service organization’s security controls. You sit down with an auditor to conduct a security review. The auditor asks you how you comply with standard. And it only costs you $25,000 USD to go through the audit.

The problems begin when you are on the receiving end of a SAS 70. For example, a customer of Amazon or Rackspace may see SAS 70 certification on the Web site and believe the company to be risk free for security controls. SAS 70 is a reporting tool for auditors. While the Web site may herald SAS 70 certification, the actual audit may determine that the company is only complying with a few security controls. For example, an SAS 70 certification may have found the security controls are inadequate and the staff not trained. Making sense of a SAS 70 requires the reader to be well versed in IT control and compliance. My friend says some are pretty clear, others obscure.

As an IT industry we need the following:

1) A standards body to define cloud security best practices and controls. The Cloud Security Alliance does a good job at fulfilling this need.

2) A standard way to report an organization’s compliance with the best practices and controls. SAS 70 publishes the data in an “auditor”-focused lingo. It is not entirely obscured.

3) A way to transform the certification and audit publication into a set of APIs that are easy for software developers to adopt in their cloud-hosted applications. Platform-level security management is an opportunity for the platform providers: Oracle for Java, Microsoft for .NET, Zend for PHP.

Combining security policy management with scalable platform support will help cloud computing take off. And hopefully our jobs will multiply to support the infrastructure. Every IT and development organization leader will eventually need to answer the changing requirements for data security and when to build a better hosting platform.

-Frank

We have a functional prototype to show everyone. Please join us for a briefing to see it work and for us to get your feedback, ideas, and criticisms. All are invited.

—

TestMaker Object Designer Briefing
Join us for the briefing on Tuesday, June 29, 2010 at 8 am Pacific time

Meeting details at:
https://www1.gotomeeting.com/register/584248945

PushToTest is working on a next generation Record/Playback tool for modern Web applications. We are building this tool to be a compelling open source alternative for HP QTP and VUGen Users. This live Webinar will provide an early look at the new TestMaker Object Designer (TOD) utility. You will hear from the engineers the current status, see a demonstration, and learn the next steps. The Briefing will be open to your questions, feedback, and criticisms. We look forward to your participation!

Title:    TestMaker Object Designer Briefing
Date:   Tuesday, June 29, 2010
Time:   8:00 AM – 9:00 AM PDT

After registering you will receive a confirmation email containing information about joining the Webinar.

System Requirements
PC-based attendees
Required: Windows® 7, Vista, XP, 2003 Server or 2000
Macintosh®-based attendees
Required: Mac OS® X 10.4.11 (Tiger®) or newer

Thanks!

We are very happy to make TestMaker 5.5 Release Candidate 3 available for immediate download. TestMaker 5.5 is a major feature enhancement and bug fix release. Download the new software from these links:

TestMaker 5.5 Release Candidate 3 for Windows

TestMaker 5.5 Release Candidate 3 for Macintosh

TestMaker 5.5 Release Candidate 3 for Linux

Release Candidates are functionally complete, are in-use by our customers, and may contain bugs. We had expected Release Candidate 2 was the final release; However, there turned out to be a significant SeleniumHtmlunit issue we needed to solve. We are issuing Release Candidate 3 expecting this is the final release and that we will declare this the final version on Monday, June 21, 2010. Please discuss issues here.

Release Candidate 3 has these changes:

• Added MaxRampDown timer to force the end of a test ramp down. Set in the Options tab of the Editor.
• Added Htmlunit headless Web browser version selector to Options tab of the Editor. TestMaker 5.5 ships with Htmlunit 2.7 and 2.8.1705. In our experience Htmlunit 2.7 is better for performance and 2.8 is better for Ajax compatibility.
• Added Grid support for multiple TestNodes running on one machine. The Grid Configurator utility (in the Tools drop-down menu) accepts a full URL, including port number and path.

PushToTest tests TestMaker 5.5 on Windows XP SP2, Windows Vista SP1, Windows 7, Mac OS X 10.5.8, and Ubuntu 9.10 (Karmic Koala).

-Frank

We are very happy to make TestMaker 5.5 Release Candidate 3 available for immediate download. TestMaker 5.5 is a major feature enhancement and bug fix release. Download the new software from these links:

TestMaker 5.5 Release Candidate 3 for Windows

TestMaker 5.5 Release Candidate 3 for Macintosh

TestMaker 5.5 Release Candidate 3 for Linux

Release Candidates are functionally complete, are in-use by our customers, and may contain bugs. We had expected Release Candidate 2 was the final release; However, there turned out to be a significant SeleniumHtmlunit issue we needed to solve. We are issuing Release Candidate 3 expecting this is the final release and that we will declare this the final version on Monday, June 21, 2010. Please discuss issues here.

Release Candidate 3 has these changes:

• Added MaxRampDown timer to force the end of a test ramp down. Set in the Options tab of the Editor.
• Added Htmlunit headless Web browser version selector to Options tab of the Editor. TestMaker 5.5 ships with Htmlunit 2.7 and 2.8.1705. In our experience Htmlunit 2.7 is better for performance and 2.8 is better for Ajax compatibility.
• Added Grid support for multiple TestNodes running on one machine. The Grid Configurator utility (in the Tools drop-down menu) accepts a full URL, including port number and path.

PushToTest tests TestMaker 5.5 on Windows XP SP2, Windows Vista SP1, Windows 7, Mac OS X 10.5.8, and Ubuntu 9.10 (Karmic Koala).

-Frank

PushToTest is working on integration of JMeter tests and TestMaker.

PushToTest TestMaker is an Open Source Test (OST) orchestration engine that repurposes tests written in soapUI, Selenium, HtmlUnit, Mozmill, and unit tests written in Java, Jython, JRuby, PHP scripting languagesto be functional tests, load and performance tests and production monitors. We have long admired JMeter for its ability to quickly author and stage a test. We plan to build a JMeter script runner for TestMaker.

JMeter is well known and well used to model load and performance tests.Our goal is to offer JMeter test developers the benefits of the TestMaker system:

• TestMaker repurposes a single JMeter test as a functional test, load and performance test, and business service monitor.
• Scale-up your JMeter tests to operate in your QA lab, in the PushToTest OnDemand Cloud Test Environment, or both!
• TestMaker data-enables JMeter tests for test-driven development.
• Dramatically reduce test operating costs. TestMaker efficiently operates multiple JMeter test in a grid concurrently.
• Run JMeter tests with soapUI tests, Selenium tests, and unit tests written in Java, Jython, Ruby, PHP
• Produce actionable knowledge by analyzing the results into 350 or more performance and functional charts.

The initial integration of JMeter and TestMaker seems to be working. TestMaker runs JMeter tests inside a TestNode. Choose a Thread group and run the test.  TestMaker ignores the orchestration item of the JMeter test and uses the TestMaker TestScenario to orchestrate the test. JMeter Orchestration items include: TestTime, number of threads, Listener. Easily set in TestMaker.

PushToTest seeks JMeter scenarios (both succeeding and failing) to complete the project. We encourage people to email their projects to info@pushtotest.com.

We have more work to complete. JMeter does not have a pluggable API.JMeter ties to a directory system. We are working to avoid having toadd more folders to the TestNode.

We are also testing against the most basic JMeter scenario. We need more experience testing a complex scenario.

Details on TestMaker are available at http://www.pushtotest.com

-Frank

PushToTest is working on integration of JMeter tests and TestMaker.

PushToTest TestMaker is an Open Source Test (OST) orchestration engine that repurposes tests written in soapUI, Selenium, HtmlUnit, Mozmill, and unit tests written in Java, Jython, JRuby, PHP scripting languagesto be functional tests, load and performance tests and production monitors. We have long admired JMeter for its ability to quickly author and stage a test. We plan to build a JMeter script runner for TestMaker.

JMeter is well known and well used to model load and performance tests.Our goal is to offer JMeter test developers the benefits of the TestMaker system:

• TestMaker repurposes a single JMeter test as a functional test, load and performance test, and business service monitor.
• Scale-up your JMeter tests to operate in your QA lab, in the PushToTest OnDemand Cloud Test Environment, or both!
• TestMaker data-enables JMeter tests for test-driven development.
• Dramatically reduce test operating costs. TestMaker efficiently operates multiple JMeter test in a grid concurrently.
• Run JMeter tests with soapUI tests, Selenium tests, and unit tests written in Java, Jython, Ruby, PHP
• Produce actionable knowledge by analyzing the results into 350 or more performance and functional charts.

The initial integration of JMeter and TestMaker seems to be working. TestMaker runs JMeter tests inside a TestNode. Choose a Thread group and run the test.  TestMaker ignores the orchestration item of the JMeter test and uses the TestMaker TestScenario to orchestrate the test. JMeter Orchestration items include: TestTime, number of threads, Listener. Easily set in TestMaker.

PushToTest seeks JMeter scenarios (both succeeding and failing) to complete the project. We encourage people to email their projects to info@pushtotest.com.

We have more work to complete. JMeter does not have a pluggable API.JMeter ties to a directory system. We are working to avoid having toadd more folders to the TestNode.

We are also testing against the most basic JMeter scenario. We need more experience testing a complex scenario.

Details on TestMaker are available at http://www.pushtotest.com

-Frank

The TestMaker development team is working on a refactoring of the TestNode software. Below is a list of the improvements we intend to make. We are looking for your feedback on the list: Are the items on the list important to you? Which improvements stand out as the most important to you? If the list is missing something then please let us know. Thanks, in advance.

TestNode Refactoring List (In order of priority):

1) TestNode stability. TestNode runs TestScenarios in crash-proof processes. If the test crashes the TestScenario the TestNode remains running and can run another TestScenario. When a TestScenario crashes the current set of transaction logs is still available to the console. Prevent TestNode from stopping even if a transaction invokes System.exit(0). TestNode preferences control the number of concurrently running TestScenarios; Default is 1.

2) TestNode should have logs folder, where agents can saves log files, and this folder should be sent to the console. TestNode should deny an agent trying to save or modify files outside the logs folder.

3) TestNode should have a status web page and REST interface (Current scenario, threads, time left to process scenario, log files)

4) Refactor the control runner to make it easier to add new ScriptRunners.

5) MultiLoad, a load test that has multiple levels of virtual users without a rampdown-up between

6) Re-enable Shock Test

7) Run TestNode as a Windows service (http://www.jpackages.com/execj/)

Jar files in a scenario should work like they were in the bootstrap classpath

9) TestNode should be able to start viral start up.

10) TestNode should have credential system (Currently is an open door)

Please give us your feedback. Thanks!

The TestMaker development team is working on a refactoring of the TestNode software. Below is a list of the improvements we intend to make. We are looking for your feedback on the list: Are the items on the list important to you? Which improvements stand out as the most important to you? If the list is missing something then please let us know. Thanks, in advance.

TestNode Refactoring List (In order of priority):

1) TestNode stability. TestNode runs TestScenarios in crash-proof processes. If the test crashes the TestScenario the TestNode remains running and can run another TestScenario. When a TestScenario crashes the current set of transaction logs is still available to the console. Prevent TestNode from stopping even if a transaction invokes System.exit(0). TestNode preferences control the number of concurrently running TestScenarios; Default is 1.

2) TestNode should have logs folder, where agents can saves log files, and this folder should be sent to the console. TestNode should deny an agent trying to save or modify files outside the logs folder.

3) TestNode should have a status web page and REST interface (Current scenario, threads, time left to process scenario, log files)

4) Refactor the control runner to make it easier to add new ScriptRunners.

5) MultiLoad, a load test that has multiple levels of virtual users without a rampdown-up between

6) Re-enable Shock Test

7) Run TestNode as a Windows service (http://www.jpackages.com/execj/)

Jar files in a scenario should work like they were in the bootstrap classpath

9) TestNode should be able to start viral start up.

10) TestNode should have credential system (Currently is an open door)

Please give us your feedback. Thanks!

We are very happy to make TestMaker 5.5 Release Candidate 2 available for immediate download. TestMaker 5.5 is a major feature enhancement and bug fix release. Download the new software from these links:

TestMaker 5.5 Release Candidate 2 for Windows

TestMaker 5.5 Release Candidate 2 for Macintosh

TestMaker 5.5 Release Candidate 2 for Linux

Release Candidates are functionally complete, are in-use by our customers, and may contain bugs. We expect this is the final Release Candidate and we will declare this release final next week. Please discuss issue here.

Release Candidate 2 has these changes:

• Upgraded to soapUI 3.5.1 script runner
• Removed soapUI authoring tool to reduce overall TestMaker download size
• Added chapter on Calibration Testing to User Guide
• Upgraded to Selenium RC 1.0.3 for compatibility with Firefox 3.6
• Fixed issues in the Editor
• Added Scalability Index Plus chart to contrast performance to linear scalability
• Native Mac OS X packaging
• Transformer now creates a TestScenario for Java tests
• Added examples for .NET, Visual Basic, Ruby, PHP test scenarios
• PTTMonitor for Mac now monitors network bandwidth
• Added Transformer tutorial to User Guide

We plan to upgrade to soapUI 3.5.1 in Release Candidate 2.

PushToTest tests TestMaker 5.5 on Windows XP SP2, Windows Vista SP1, Windows 7, Mac OS X 10.5.8, and Ubuntu 9.10 (Karmic Koala).

-Frank

We are very happy to make TestMaker 5.5 Release Candidate 2 available for immediate download. TestMaker 5.5 is a major feature enhancement and bug fix release. Download the new software from these links:

TestMaker 5.5 Release Candidate 2 for Windows

TestMaker 5.5 Release Candidate 2 for Macintosh

TestMaker 5.5 Release Candidate 2 for Linux

Release Candidates are functionally complete, are in-use by our customers, and may contain bugs. We expect this is the final Release Candidate and we will declare this release final next week. Please discuss issue here.

Release Candidate 2 has these changes:

• Upgraded to soapUI 3.5.1 script runner
• Removed soapUI authoring tool to reduce overall TestMaker download size
• Added chapter on Calibration Testing to User Guide
• Upgraded to Selenium RC 1.0.3 for compatibility with Firefox 3.6
• Fixed issues in the Editor
• Added Scalability Index Plus chart to contrast performance to linear scalability
• Native Mac OS X packaging
• Transformer now creates a TestScenario for Java tests
• Added examples for .NET, Visual Basic, Ruby, PHP test scenarios
• PTTMonitor for Mac now monitors network bandwidth
• Added Transformer tutorial to User Guide

We plan to upgrade to soapUI 3.5.1 in Release Candidate 2.

PushToTest tests TestMaker 5.5 on Windows XP SP2, Windows Vista SP1, Windows 7, Mac OS X 10.5.8, and Ubuntu 9.10 (Karmic Koala).

-Frank

PushToTest is building a new Ajax-based record/playback utility that will output Selenese tests. We will name this TestMaker IDE, and it may be adopted as Selenium IDE 2. PushToTest’s goal is to provide an open source alternative to HP QTP and VUGen. The design document for this project is at http://downloads.pushtotest.com/201004/TM_IDE_Design_v2.pdf

Here is an engineering status update for May 1, 2010:

The work on TestMaker IDE (TMIDE) is going well. The code leverages Sahi in a way that records Ajax applications much better than Selenium can handle. Venky and Narayan have proven the architectural choices (Titanium, ExtJS, JQuery) of deploying TMIDE as an Ajax application. There is a lot more design work needed to achieve a usable tool.

The top level TMIDE goal is to provide an open source alternative to HP’s QTP tool. We need TMIDE to provide a smooth transition from QTP to TestMaker. QTP is missing test orchestration user interface elements – because HP intends QTP only for functional testing. QTP also depends on its proprietary hooks into the real browser. TMIDE builds test scripts that are intended for playback in HtmlUnit to repurpose the test script as a functional test, load and performance test, and business service monitor.

Perhaps the most time consuming customer and pre-sales support issue at PushToTest is:

“Why does my test work in Firefox and fail in HtmlUnit?”

HtmlUnit is designed to simulate the browser experience for the purposes of testing, not to duplicate the Firefox experience. There are many things that PushToTest can do to improve the test authoring and test debugging tasks while running the application in HtmlUnit.

I am working on a new design for TMIDE that addresses this issue. Look for the design in the coming week. Here are the primary goals for the new design:

1) Test authors graphically see what HtmlUnit sees in a Web page as it operates the test. For example, TMIDE runs an individual test script command, automatically uses the savePage() method we implemented in SeleniumHtmlUnit, then views the saved page in Firefox, then repeats for the additional commands in the test.

2) Control Javascript function operations in HtmlUnit. This will use the new ProofTest code. ProofTest in TestMaker selectively skips execution of functions. For example, ProofTest dynamically removes the Google Analytics <script> tag operation from a Web page without modifying the page.

3) Use HtmlUnit to identify test objects. For example, TestMaker IDE enables test authors to identify DOM nodes (and their children) as test objects.

4) Provide debug trace logging utilities in HtmlUnit. For example, dynaTrace Ajax Edition functions running in HtmlUnit identify the call stack of Javascript functions.

With these new features, test authoring of Rich Internet Applications (RIA using Ajax, Flex, Flash) becomes a whole lot easier.

I would love your feedback on the above goals.

Thanks.

-Frank

PushToTest is building a new Ajax-based record/playback utility that will output Selenese tests. We will name this TestMaker IDE, and it may be adopted as Selenium IDE 2. PushToTest’s goal is to provide an open source alternative to HP QTP and VUGen. The design document for this project is at http://downloads.pushtotest.com/201004/TM_IDE_Design_v2.pdf

Here is an engineering status update for May 1, 2010:

The work on TestMaker IDE (TMIDE) is going well. The code leverages Sahi in a way that records Ajax applications much better than Selenium can handle. Venky and Narayan have proven the architectural choices (Titanium, ExtJS, JQuery) of deploying TMIDE as an Ajax application. There is a lot more design work needed to achieve a usable tool.

The top level TMIDE goal is to provide an open source alternative to HP’s QTP tool. We need TMIDE to provide a smooth transition from QTP to TestMaker. QTP is missing test orchestration user interface elements – because HP intends QTP only for functional testing. QTP also depends on its proprietary hooks into the real browser. TMIDE builds test scripts that are intended for playback in HtmlUnit to repurpose the test script as a functional test, load and performance test, and business service monitor.

Perhaps the most time consuming customer and pre-sales support issue at PushToTest is:

“Why does my test work in Firefox and fail in HtmlUnit?”

HtmlUnit is designed to simulate the browser experience for the purposes of testing, not to duplicate the Firefox experience. There are many things that PushToTest can do to improve the test authoring and test debugging tasks while running the application in HtmlUnit.

I am working on a new design for TMIDE that addresses this issue. Look for the design in the coming week. Here are the primary goals for the new design:

1) Test authors graphically see what HtmlUnit sees in a Web page as it operates the test. For example, TMIDE runs an individual test script command, automatically uses the savePage() method we implemented in SeleniumHtmlUnit, then views the saved page in Firefox, then repeats for the additional commands in the test.

2) Control Javascript function operations in HtmlUnit. This will use the new ProofTest code. ProofTest in TestMaker selectively skips execution of functions. For example, ProofTest dynamically removes the Google Analytics <script> tag operation from a Web page without modifying the page.

3) Use HtmlUnit to identify test objects. For example, TestMaker IDE enables test authors to identify DOM nodes (and their children) as test objects.

4) Provide debug trace logging utilities in HtmlUnit. For example, dynaTrace Ajax Edition functions running in HtmlUnit identify the call stack of Javascript functions.

With these new features, test authoring of Rich Internet Applications (RIA using Ajax, Flex, Flash) becomes a whole lot easier.

I would love your feedback on the above goals.

Thanks.

-Frank

A group of senior executives from open source companies get together once a year in Napa Valley, California for the Open Source Think Tank. The group includes WSO2, Talend, Makara, Alfresco, AlienVault, Hippo, Mulesource, and, of course, PushToTest. About 150 people attended the event in April 2010. Their motivation is to understand how to commercialize open source products and services.

The event is hosted by the Olliance Group. They are management consultants that specialize in providing expertise on building businesses. For example, is your business spending enough on marketing? Do you have the right open source license? How much should you be paying your CEO?

The event is a great place to think about big things:

Terms and Terminology

As organizations adopt a variety of software and services they consider the following options:

Community Open Source Projects Free love and the revolution Commercial Open Source Products Revenue, quality, support for crazy new stuff Proprietary Products Buy it once, pay for maintenance yearly, wait in line for new features and bug fixes Open Source VS Cloud Computing

Much of the Think Tank conversation was around a perception in the IT industry that cloud computing would have a negative impact on open source. The fear is that open source licenses do not support the concept of cloud computing. In a cloud environment the cloud host takes on the responsibility to provide an always-on service to run a set of applications and their related APIs. Why would open source code even matter when the infrastructure is inexpensive, always-on, and maintained/upgraded by the host? From that perspective cloud computing is a competitor to open source.

That perspective is very narrow. One might even have to squint to see it.

I see a big world filled with many platforms vying for our attention. Cloud computing gives us a new choice to run our application software in a hosted environment where certain services are available. For example, my sales automation and customer relationship management applications running on the new VMForce cloud gives me a hosted place to run the application and access to SalesForce data locally. My choice of cloud is based on the service level agreement offered, the price, and the services offered.

The cloud computing services available in 2010 are mostly built on open source. It is my hope that the cloud computing environments continue to be built on open source technology as a way to keep costs low and quality high. Time will tell.

Mobile Vs Open Source

The Think Tank coincided with the first week of the Apple iPad shipping. One delegate put her iPad on top of her MacBook Pro. She completed the stack by placing her iPhone on top of the iPad. I asked her why she has all three. “I have a use for the laptop and phone. I’m looking forward to figuring out a use for the iPad,” she told me.

The Think Tank program asked delegates to identify how they would deliver solutions with a mobile requirement. For example, build a company around an service that helps salespeople find customer information based on their current location. We decided on platform (Blackberry, iPhone, Android, and others,) telco network provider, app store, backend provider, and handheld device. We broke into 10 working groups. When presenting the solutions 8 out of the 10 groups said they would not open source their code. All 10 agreed they would open source the APIs to access their code.

Gated Communities Instead of Lofty Clouds

Taken together, the Think Tank is representative of a trend where the building blocks (operating environment, app server, database, service bus, adaptors) of an information system will remain open source and free. The cloud computing environments will grow specialized and proprietary. Think of gated communities of application hosts instead of clouds floating in the air. And IT organizations will access to these specialized clouds through open APIs.

-Frank

A group of senior executives from open source companies get together once a year in Napa Valley, California for the Open Source Think Tank. The group includes WSO2, Talend, Makara, Alfresco, AlienVault, Hippo, Mulesource, and, of course, PushToTest. About 150 people attended the event in April 2010. Their motivation is to understand how to commercialize open source products and services.

The event is hosted by the Olliance Group. They are management consultants that specialize in providing expertise on building businesses. For example, is your business spending enough on marketing? Do you have the right open source license? How much should you be paying your CEO?

The event is a great place to think about big things:

Terms and Terminology

As organizations adopt a variety of software and services they consider the following options:

Community Open Source Projects Free love and the revolution Commercial Open Source Products Revenue, quality, support for crazy new stuff Proprietary Products Buy it once, pay for maintenance yearly, wait in line for new features and bug fixes Open Source VS Cloud Computing

Much of the Think Tank conversation was around a perception in the IT industry that cloud computing would have a negative impact on open source. The fear is that open source licenses do not support the concept of cloud computing. In a cloud environment the cloud host takes on the responsibility to provide an always-on service to run a set of applications and their related APIs. Why would open source code even matter when the infrastructure is inexpensive, always-on, and maintained/upgraded by the host? From that perspective cloud computing is a competitor to open source.

That perspective is very narrow. One might even have to squint to see it.

I see a big world filled with many platforms vying for our attention. Cloud computing gives us a new choice to run our application software in a hosted environment where certain services are available. For example, my sales automation and customer relationship management applications running on the new VMForce cloud gives me a hosted place to run the application and access to SalesForce data locally. My choice of cloud is based on the service level agreement offered, the price, and the services offered.

The cloud computing services available in 2010 are mostly built on open source. It is my hope that the cloud computing environments continue to be built on open source technology as a way to keep costs low and quality high. Time will tell.

Mobile Vs Open Source

The Think Tank coincided with the first week of the Apple iPad shipping. One delegate put her iPad on top of her MacBook Pro. She completed the stack by placing her iPhone on top of the iPad. I asked her why she has all three. “I have a use for the laptop and phone. I’m looking forward to figuring out a use for the iPad,” she told me.

The Think Tank program asked delegates to identify how they would deliver solutions with a mobile requirement. For example, build a company around an service that helps salespeople find customer information based on their current location. We decided on platform (Blackberry, iPhone, Android, and others,) telco network provider, app store, backend provider, and handheld device. We broke into 10 working groups. When presenting the solutions 8 out of the 10 groups said they would not open source their code. All 10 agreed they would open source the APIs to access their code.

Gated Communities Instead of Lofty Clouds

Taken together, the Think Tank is representative of a trend where the building blocks (operating environment, app server, database, service bus, adaptors) of an information system will remain open source and free. The cloud computing environments will grow specialized and proprietary. Think of gated communities of application hosts instead of clouds floating in the air. And IT organizations will access to these specialized clouds through open APIs.

-Frank

We are very happy to make TestMaker 5.5 Release Candidate 1 available for immediate download. TestMaker 5.5 is a major feature enhancement and bug fix release. Download the new software from these links:

TestMaker 5.5 Release Candidate 1 for Windows

TestMaker 5.5 Release Candidate 1 for Macintosh

TestMaker 5.5 Release Candidate 1 for Linux

Release Candidates are functionally complete, are in-use by our customers, and may contain bugs. We are set-up to take issue reports here. Each Monday we will roll-up solutions to the issue reports and release Release Candidate 2, 3. We will do that for 3 weeks total and then declare TestMaker 5.5 as final.

We plan to upgrade to soapUI 3.5.1 in Release Candidate 2.

PushToTest tests TestMaker 5.5 on Windows XP SP2, Windows Vista SP1, Windows 7, Mac OS X 10.5.8, and Ubuntu 9.10 (Karmic Koala).

TestMaker 5.5 and later requires a special Microsoft patch to Windows XP for the Editor to operate correctly.

Thank you to the dozens of people who contributed improvements and patches! Here is a summary of the improvements:

1. Grid Testing – TestMaker runs your tests on a Grid of your test equipment, in a Cloud Computing environment, or both.
2. New Script Runners to operate tests written in .NET, Visual Basic (VB) and from the command line
3. Failure and Warning Report Detail – 16 new reports detailing test errors and warning messages
4. SeleniumHtmlUnit 2.8 Update – Better compatibility with Ajax applications
5. savePage – Debugging command for Selenium saves HtmlUnit’s view of the current HTML and supporting files
6. EncryptDPL – Reads encrypted data from CSV files
7. UniqueDPL – Provides unique values to virtual users in a test
8. RDBMS Logging – Logs test and monitor results data to a relational database
9. TestMaker Editor improvements and bug fixes, including file save issues, unused resource removal, and options issues, saves chart settings
10. Updated Tutorials – 30-minute tutorials on building tests of Ajax applications using Selenium, building tests of SOAP-based services using Selenium, and building tests of Flex-based services using TestMaker AMF protocols.
11. Properties file support to dynamically set TestScenario and test settings
12. Preference Settings for external soapUI installation location
13. Improvements to Selenium IDE – We began to make changes to Selenium IDE to improve its support for Ajax-based Web applications.
14. Email Notifications – TestMaker sends emails on test and production monitor status
15. J-Unit Style Results Report for integration with Continuous Integration (Hudson, Cruise Control, Bamboo) environments
16. Upgrade to Selenium IDE 1.05, XPather 1.4.3 for Firefox 3.6 compatibility
17. Updated Cloud Testing support in Amazon EC2 AMI
18. Fix Editor save bug on Windows environments
19. Fix soapUI wrong arguments bug
20. Bug Fixes – Dozens of small fixes and improvements.

Enjoy!

Frank

We are very happy to make TestMaker 5.5 Release Candidate 1 available for immediate download. TestMaker 5.5 is a major feature enhancement and bug fix release. Download the new software from these links:

TestMaker 5.5 Release Candidate 1 for Windows

TestMaker 5.5 Release Candidate 1 for Macintosh

TestMaker 5.5 Release Candidate 1 for Linux

Release Candidates are functionally complete, are in-use by our customers, and may contain bugs. We are set-up to take issue reports here. Each Monday we will roll-up solutions to the issue reports and release Release Candidate 2, 3. We will do that for 3 weeks total and then declare TestMaker 5.5 as final.

We plan to upgrade to soapUI 3.5.1 in Release Candidate 2.

PushToTest tests TestMaker 5.5 on Windows XP SP2, Windows Vista SP1, Windows 7, Mac OS X 10.5.8, and Ubuntu 9.10 (Karmic Koala).

TestMaker 5.5 and later requires a special Microsoft patch to Windows XP for the Editor to operate correctly.

Thank you to the dozens of people who contributed improvements and patches! Here is a summary of the improvements:

1. Grid Testing – TestMaker runs your tests on a Grid of your test equipment, in a Cloud Computing environment, or both.
2. New Script Runners to operate tests written in .NET, Visual Basic (VB) and from the command line
3. Failure and Warning Report Detail – 16 new reports detailing test errors and warning messages
4. SeleniumHtmlUnit 2.8 Update – Better compatibility with Ajax applications
5. savePage – Debugging command for Selenium saves HtmlUnit’s view of the current HTML and supporting files
6. EncryptDPL – Reads encrypted data from CSV files
7. UniqueDPL – Provides unique values to virtual users in a test
8. RDBMS Logging – Logs test and monitor results data to a relational database
9. TestMaker Editor improvements and bug fixes, including file save issues, unused resource removal, and options issues, saves chart settings
10. Updated Tutorials – 30-minute tutorials on building tests of Ajax applications using Selenium, building tests of SOAP-based services using Selenium, and building tests of Flex-based services using TestMaker AMF protocols.
11. Properties file support to dynamically set TestScenario and test settings
12. Preference Settings for external soapUI installation location
13. Improvements to Selenium IDE – We began to make changes to Selenium IDE to improve its support for Ajax-based Web applications.
14. Email Notifications – TestMaker sends emails on test and production monitor status
15. J-Unit Style Results Report for integration with Continuous Integration (Hudson, Cruise Control, Bamboo) environments
16. Upgrade to Selenium IDE 1.05, XPather 1.4.3 for Firefox 3.6 compatibility
17. Updated Cloud Testing support in Amazon EC2 AMI
18. Fix Editor save bug on Windows environments
19. Fix soapUI wrong arguments bug
20. Bug Fixes – Dozens of small fixes and improvements.

Enjoy!

Frank

We are very happy to make TestMaker 5.5 Release Candidate 1 available for immediate download. TestMaker 5.5 is a major feature enhancement and bug fix release. Download the new software from these links:

TestMaker 5.5 Release Candidate 1 for Windows

TestMaker 5.5 Release Candidate 1 for Macintosh

TestMaker 5.5 Release Candidate 1 for Linux

Release Candidates are functionally complete, are in-use by our customers, and may contain bugs. We are set-up to take issue reports here. Each Monday we will roll-up solutions to the issue reports and release Release Candidate 2, 3. We will do that for 3 weeks total and then declare TestMaker 5.5 as final.

We plan to upgrade to soapUI 3.5.1 in Release Candidate 2.

PushToTest tests TestMaker 5.5 on Windows XP SP2, Windows Vista SP1, Windows 7, Mac OS X 10.5.8, and Ubuntu 9.10 (Karmic Koala).

TestMaker 5.5 and later requires a special Microsoft patch to Windows XP for the Editor to operate correctly.

Thank you to the dozens of people who contributed improvements and patches! Here is a summary of the improvements:

1. Grid Testing – TestMaker runs your tests on a Grid of your test equipment, in a Cloud Computing environment, or both.
2. New Script Runners to operate tests written in .NET, Visual Basic (VB) and from the command line
3. Failure and Warning Report Detail – 16 new reports detailing test errors and warning messages
4. SeleniumHtmlUnit 2.8 Update – Better compatibility with Ajax applications
5. savePage – Debugging command for Selenium saves HtmlUnit’s view of the current HTML and supporting files
6. EncryptDPL – Reads encrypted data from CSV files
7. UniqueDPL – Provides unique values to virtual users in a test
8. RDBMS Logging – Logs test and monitor results data to a relational database
9. TestMaker Editor improvements and bug fixes, including file save issues, unused resource removal, and options issues, saves chart settings
10. Updated Tutorials – 30-minute tutorials on building tests of Ajax applications using Selenium, building tests of SOAP-based services using Selenium, and building tests of Flex-based services using TestMaker AMF protocols.
11. Properties file support to dynamically set TestScenario and test settings
12. Preference Settings for external soapUI installation location
13. Improvements to Selenium IDE – We began to make changes to Selenium IDE to improve its support for Ajax-based Web applications.
14. Email Notifications – TestMaker sends emails on test and production monitor status
15. J-Unit Style Results Report for integration with Continuous Integration (Hudson, Cruise Control, Bamboo) environments
16. Upgrade to Selenium IDE 1.05, XPather 1.4.3 for Firefox 3.6 compatibility
17. Updated Cloud Testing support in Amazon EC2 AMI
18. Fix Editor save bug on Windows environments
19. Fix soapUI wrong arguments bug
20. Bug Fixes – Dozens of small fixes and improvements.

Enjoy!

Frank

This week Apple chimed-in on its open development platform strategy. Apple released a beta of the iPhone OS 4 SDK, including a new license agreement that bans Flash, but also may ban Appcelerator Titanium and other cross-platform development tools.

A good write-up on Apple’s new terms is here. The result has been to set the software developer community on fire.

The secondary effect of Apple’s move impacts my company PushToTest. Apple’s proposed changes to the iPhone OS 4 SDK license appear to lock out Appcelerator Titanium from the iPad/iPhone/Touch marketplace. That would have a negative impact on Appcelerator’s prospects as a viable business.

Last year I asked “Maybe we don’t need objects to build client side apps?” on TheServerSide.com. I wondered what it would be like to build a client side desktop application using Web technology (Ajax, Javascript, etc.) I put my money where my mouth is by deciding to use Appcelerator Titanium to build the TestMaker Editor. I documented the design here. I recorded a screencast with the Editor developers to explain the process we followed, the techniques we used, and the generally favorable experience.

It just does not make sense to me to be building anything requiring a user interface with Java SWING. The advantage of using Web technology to deliver desktop, mobile, and dedicated (aka iPad) user interfaces is strong: Single code base, large developer community, end-user acceptance of Web interface standards. From that perspective Objective C and XCode is just as bad of a decision as Java SWING.

The closest I’ve seen Apple and Jobs response to criticism is in Greg Slepak’s (of TaoEffect) blog. Jobs apparently responded directly to Greg and pointed him to John Gruber’s blog.

I don’t have any issue with a proprietary vendor trying to deliver a great user experience for its customers. My problem is how wide Apple limited support of its platform and the secondary effect it has on PushToTest’s technology partner. I hope that Apple finds a way to enable Appcelerator to participate.

Secondarily I don’t think Apple understands how widely used Flash is for content playback (as opposed to application development.)

Lastly, it seems to me a very USA-centric thought process at work at Apple. It’s the same thinking that got us into Iraq, that we just saw in the Healthcare debate, and that used to be a defining Microsoft behavior. It is the belief in one’s own right to make decisions with an assumption that we know better than anyone else and without regard to some better good in the world. We have to get past this thinking. The world will be much better off when we aspire to creative, maintainable/sustainable, and high quality solutions to our problems.

-Frank