During a flurry of tweets and blog posts about whether or not ATDD is a good thing, I posted a blog on Stickyminds with my opinion. I’ve gotten some great comments, please add yours!

Creating a master test plan not only consists of creating a draft MTP, doing a product risks analysis and creating the test strategy. You also have to estimate the project. In this post I describe how I do this. In January I asked Pradeep Soundararajan (@testertested) how he estimated test projects. He posted an blog on [...]

This video demonstrates how easy it is to add dependencies using m2eclipse.    Because m2eclipse understands how to interact with a repository index, it can quickly locate a dependency by class name or by GAV coordinate.    Don’t know which artifact contains a particular class?   Just start writing code and use an Eclipse Quick Fix to search all Maven repositories for an artifact that contains a particular class.    Want to inspect and browse a Maven repository? Don’t use a web browser.  Use the built-in dependency search feature in m2eclipse.

 

Previous posts in this series:

• A Primer
• Aggregate Construction

One of the common themes throughout the DDD book is that much of the nuts and bolts of structural domain-driven design is just plain good use of object-oriented programming.  This is certainly true, but DDD adds some direction to OOP, along with roles, stereotypes and patterns.  Much of the direction for building entities at the class level can, and should, come from test-driven development.  TDD is a great tool for building OO systems, as we incrementally build our design with only the behavior that is needed to pass the test.  Our big challenge then is to write good tests.

To fully harness TDD, we need to be highly attuned to the design that comes out of our tests.  For example, suppose we have our traditional Customer and Order objects.  In our world, an Order has a Customer, and a Customer can have many Orders.  We have this directionality because we can navigate this relationship from both directions in our application.  In the last post, we worked to satisfy invariants to prevent an unsupported and nonsensical state for our objects.

We can start with a fairly simple test:

[Test]
public void Should_add_the_order_to_the_customers_order_lists_when_an_order_is_created()
{
    var customer = new Customer();
    var order = new Order(customer);

    customer.Orders.ShouldContain(order);
}

At first, this test does not compile, as Customer does not yet contain an Orders member.  To make this test compile (and subsequently fail), we add an Orders list to Customer:

public class Customer
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string Province { get; set; }
    public List<Order> Orders { get; set; }

    public string GetFullName()
    {
        return LastName + ", " + FirstName;
    }
}

With the Orders now exposed on Customer, we can make our test pass from the Order constructor:

public class Order
{
    public Order(Customer customer)
    {
        Customer = customer;
        customer.Orders.Add(this);
    }

And all is well in our development world, right?  Not quite.  This design exposes quite a bit of functionality that I don’t think our domain experts need, or want.  The design above allows some very interesting and very wrong scenarios:

[Test]
public void Not_supported_situations()
{
    // Removing orders?
    var customer1 = new Customer();
    var order1 = new Order(customer1);

    customer1.Orders.Remove(order1);

    // Clearing orders?
    var customer2 = new Customer();
    var order2 = new Order(customer1);

    customer2.Orders.Clear();

    // Duplicate orders?
    var customer3 = new Customer();
    var customer4 = new Customer();
    var order3 = new Order(customer3);

    customer4.Orders.Add(order3);
}

With the API I just created, I allow a number of rather bizarre scenarios, most of which make absolutely no sense to the domain experts:

• Clearing orders
• Removing orders
• Adding an order from one customer to another
• Inserting orders
• Re-arranging orders
• Adding an order without the Order’s Customer property being correct

This is where we have to be a little more judicious in the API we expose for our system.  All of these scenarios are possible in the API we created, but now we have some confusion on whether we should support these scenarios or not.  If I’m working in a similar area of the system, and I see that I can do a Customer.Orders.Remove operation, it’s not immediately clear that this is a scenario not actually coded for.  Worse, I don’t have the ability to correctly handle these situations if the collection is exposed directly.

Suppose I want to clear a Customer’s Orders.  It logically follows that each Order’s Customer property would be null at that point.  But I can’t hook in easily to the List<T> methods to handle these operations.  Instead of exposing the collection directly, I will expose only those operations which I support through my domain.

Moving towards intention-revealing interfaces

Let’s fix the Customer object first.  It exposes a List<T> directly, and allows wholesale replacement of that collection.  This is the complete antithesis of intention-revealing interfaces.  I will now only expose the sequence of Orders on Customer:

public class Customer
{
    private readonly IList<Order> _orders = new List<Order>();

    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string Province { get; set; }
    public IEnumerable<Order> Orders { get { return _orders; } }

    public string GetFullName()
    {
        return LastName + ", " + FirstName;
    }
}

This interface explicitly tells users of Customer two things:

• Orders are readonly, and cannot be modified through this aggregate
• Adding orders are done somewhere else

I now have the issue of the Order constructor needing to add itself to the Customer’s Order collection.  I want to do this:

public class Order
{
    public Order(Customer customer)
    {
        Customer = customer;
        customer.AddOrder(this);
    }

Instead of exposing the Orders collection directly, I work through a specific method to add an order.  But, I don’t want that AddOrder available everywhere, I want to only support the enforcement of the Order-Customer relationship through this explicitly defined interface.  I’ll do this by exposing an AddOrder method, but exposing it as internal:

public class Customer
{
    private readonly IList<Order> _orders = new List<Order>();

    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string Province { get; set; }
    public IEnumerable<Order> Orders { get { return _orders; } }

    internal void AddOrder(Order order)
    {
        _orders.Add(order);
    }

There are many different ways I could enforce this relationship, from exposing an AddOrder method publicly on Customer or through the approach above.  But either way, I’m moving towards an intention-revealing interface, and only exposing the operations I intend to support through my application.  Additionally, I’m ensuring that all invariants of my aggregates are satisfied at the completion of the Create Order operation.  When I create an Order, the domain model takes care of the relationship between Customer and Order without any additional manipulation.

If I publicly expose a collection class, I’m opening the doors for confusion and future bugs as I’ve now allowed my system to tinker with the implementation details of the relationship.  It’s my belief that the API of my domain model should explicitly support the operations needed to fulfill the needs of the application and interaction of the UI, but nothing more.

Visual Studio 2010 is almost here – Microsoft just released the first Release Candidate which looks pretty solid and good. Microsoft added new interfaces for performance management solutions like dynaTrace to extend the Web- and Load-Testing capabilities (check out Ed Glas’s blog on what’s in VSTS Load Testing) to go beyond .NET environments and deeper [...]

We try not to plug Klocwork on this blog too much and keep the discussions around general software development issues, topics, and trends (with an obvious bias towards static analysis and software verification since those are our areas of expertise).

So, today I’ll go ahead and break that rule and encourage you to visit our new online resource library that includes videos, webinars, whitepapers, analyst reports, etc. All the assets are organized by category and tagged with their subject/content so everything is easy to find. I encourage you to check it out… here’s an example of the kinds of demos we’ve put up there so people can quick see what our tools are all about.

Next week I will be in Santa Clara, CA speaking at the CloudConnect Conference Event which is looking like it will live up to its billing as “the defining event that brings together the entire cloud computing community, including IT leaders, industry executives and developers”. My speaking timetable is as follows:

Keynote & General Sessions
10:00 AM–11:35 AM, Wed 17th March 2010

New Pricing Models: How Will They Impact ROI?
4:00 PM–5:00 PM, Wed 17th March 2010

In the last session, which consists of a panel of speakers, I will have a few minutes at the start to introduce myself, our technology and its innovative approach to management of cloud service executions and interactions.

And if I am very quick I will also have a small window in which to present our vision of cloud computing when there is increased competition in the delivery of common (shared) software services across an entire supply chain residing in one or more clouds.

If you are in attendance and would like to discuss further aspects of what is presented then please feel free to approach me after the sessions.

Pickle adds many convenient Cucumber steps for generating models. Also learn about table diffs in this episode. Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid – all rolled into one format. Pickle gives [...]

The Maven team has gone to the ends of the earth to ensure backward compatibility, improve usability, increase performance, allow safe embedding, and pave the way for implement many highly demanded features. This talk will briefly cover the process and tooling changes that have occurred in the Maven project in order to accomplish what we [...]

Register now for the next Sogeti Ireland webinar which we are hosting on 25 March from 11am-12pm. This webinar will examine some best practices, challenges, and benefits for IT departments in both the Public and Private sector that are seeking to support timely and effective business transformation.
Agenda
This webinar will look at:

• Software development and testing where Lean/Agile approaches are being adopted to be more flexible responsive to rapidly changing business needs
• Flexible delivery models such Software as a Service
• The vendor management aspects of outsourcing, and its variations, be it an onsite Managed Services, Nearshore, Farshore or blended RightShore™

The webinar, presented by Fran O’Hara (Practice Manager, Sogeti Ireland) and Declan Kavanagh (CEO, Sogeti Ireland), will draw on Sogeti’s global experience as well as current economic trends and extensive analyst research.

• Register Now

Business Transformation Series
We have assembled a team of thought leaders to present a series of five webinars on the theme of Recovery through Business Transformation. These webinars will look at topics such IT Transformation, SaaS, Cloud Computing, Agile/Lean and Public Sector Shared Services. Read more
If you have any questions, please contact us.

Goanna version 1.2 has been released. Download it now.

The major change is More Checks, in fact 40% more than were previously available in v1.1. Over the next few months we will continue to add new checks with each release. You can expect to see up to 100 additional high quality checks within the coming 6 months, which as usual will be free for all existing customers. Additionally, should you require a 30 day Trial Extension for your version 1.2 update please complete this trial extension request form.

We are also very pleased to announce the Beta release of Goanna for Command Line. This new command line version enables more flexibility and freedom for those wishing to integrate our powerful C/C++ static analyzer into their own development process. The Beta is currently available for Linux users and a version for Windows users is scheduled to be available in May. Linux users can now access a fully gcc-compatible solution integrated with over 60 classes of flow-sensitive quality checks to detect critical bugs as early as possible in the development cycle.

Inter-procedural analysis is also well under way, so stay tuned for a public Beta release soon!

Welcome to the roundup of blog posts and pages that mentioned Sonar last month…

Measuring code quality with Sonar
By Allard Buijze, 26 February 2010

“At JTeam, we continuously strive for good quality code. The reason is very simple: bad quality code slows down the development process. The small investment pays out in even the simplest of projects.”

Sonar: a valuable tool for every Java shop
By Hans Westerbeek, 22 February 2010

“If your organization develops Java applications, chances are that Sonar could be very valuable for you. Sonar helps you calculate your technical debt by analyzing your project’s source code.”

Getting More Agile – 2009 in Review
By Thomas Ferris Nicolaisen, 21 February 2010

“With our mavenization efforts, we were recently able to make use of Sonar (which rocks, btw) instead of the old, more primitive (boring) Ant reports in Hudson.”

A Sonar Cobol Plugin
By SonarSource, 18 February 2010

Coming soon… “SonarSource has developed its own Cobol parser and packaged it in a Sonar Cobol analyzer. It allows to perform objective and automated Cobol code quality and standards reviews against pre-defined rule sets and coding best practices. It also generates standard code metrics like number of lines or density of comments”

Add code quality metrics to your GateIn Dashboard with Sonar
By Jeremi Joslin, 11 February 2010

“Sonar is an open source platform to manage code quality. It enables to collect, analyze and report metrics on source code. At eXo Platform, we use Sonar to manage and monitor the quality of our codebase.”

Sonar Integration for AnthillPro
By Eric Minick, 10 February 2010

“A Sonar plugin for AnthillPro is now available through Urbancode’s Supportal (support portal).”

Sonar SCM Activity Plugin version 0.1
By Evgeny Mandrikov, 27 January 2010

“People, who follow me on Twitter, already know about new plugin for Sonar developed by me. Meet – Sonar SCM Activity Plugin.”

Java Build Server
By Manuel Küblböck, 23 Januray 2010

“In my last Java project, I set up a build server with Continuous Integration (CI) capability. I am a big fan of Test Driven Development (TDD) and I quite enjoyed Hudson telling us right away when someone checked in code that broke the build. It just gives you so much more confidence in your code and keeps it releasable at all times. In addition, we used Sonar to measure the quality of our code.”

Even though we focus very heavily on full-stack acceptance testing for the rails world, we know other forms of automated tests are critical as well. Our rails developers here make pretty heavy use of RSpec unit tests, and it’s nice to understand how to run those in parallel as well.

If you’re looking at how to setup a rails and selenium testing environment, check out our last post.

Parallelize the specs

We’ll use the excellent parallel_specs to beat a bit of parallelism into our specs. It prepares a separate database for each test environment, groups the specs to divide amongst processes, and then starts up a rails environment with a separate database for each group of processes.

I’ll paraphrase the installation instructions for convenience.

Install the required plug-in/gem:

sudo gem install parallel
script/plugin install git://github.com/grosser/parallel_specs.git

Here’s the semi-ingenious point – yaml can interpret ERB, so we can pass in an environment variable to the database.yml specifying at launch which database we want it to connect to.

Open config/database.yml and add the following:

test:
  adapter: sqlite3
  database: db/xxx_test<% TEST_ENV_NUMBER %>.sqlite3
  pool: 5
  timeout: 5000

(You can of course replace xxx_ with your project name)
So for example, to have our tests run against the xxx_test2 database, we would use:

export TEST_ENV_NUMBER=2; rake db:test:prepare

But it doesn’t make much sense to invoke it manually. That’s what plug-ins are for! Let’s go head and create/migrate a few test databases:

export TEST_ENV_NUMBER=0; rake db:test:create; rake db:test:migrate;
export TEST_ENV_NUMBER=1; rake db:test:create; rake db:test:migrate;
export TEST_ENV_NUMBER=2; rake db:test:create; rake db:test:migrate;

Great, now you’re able to run your non-Selenium tests in parallel!

But what about Selenium tests?

Stay tuned for our article on Sauce Labs’ SpecStorm plugin, that allows you to run your Selenium tests in true parallel fashion with Selenium Grid or our very own Sauce OnDemand service.

Here’s Tuesday’s links from My Twitter Stream.

• 21:16: How to Teach Yourself Programming in 14611 days http://bit.ly/cubVFb
• 18:03: [Blog] DevOps is a good cause, but what about OpsOps? http://bit.ly/96WaM5
• 08:15: The Panic Status Board http://bit.ly/9oCdEi [Oh, that's beautiful] (via @jmlumpkin)
• 07:13: Optimise your acceptance tests http://bit.ly/apihS5 [not so sure about #2]
• 00:03: [Blog] Links for 2010-03-08 http://bit.ly/9VZWc9

Share with the group:

Related posts:

1. Links for 2010-01-15 Here’s Friday’s links from My Twitter Stream. 10:15: How Fanboys...
2. Links for 2010-02-15 Here’s Monday’s links from My Twitter Stream. 10:03: Bringing DevOps...
3. Links for 2010-03-04 Here’s Thursday’s links from My Twitter Stream. 23:30: [Blog] Platform...

Related posts brought to you by Yet Another Related Posts Plugin.

I must confess: I don't believe in regression testing. Not that I don't believe it can work, just that it doesn't.

After spending over 20 years in software testing and working with hundreds of IT shops, I have to report that effective regression testing--which is supposed to protest against unintended impact of intended changes - is about as common as development projects that are ahead of schedule and under budget. There are exceptions; some IT shops no doubt do a great job. But it's also true that some people have survived jumping off the Golden Gate bridge, which is not a strong recommendation for doing it.

So why doesn't regression testing work? I think it has to do with the way it is perceived and practiced.

For starters--and I'm only being halfway facetious here--what's with that name, anyway? Regression sounds bad--too much like repression, depression, and oppression. It sounds pejorative. And it doesn't seem to add a lot of value: Just makes sure that what used to work, still works. Most IT shops are rewarded for what they create--more features, new applications, cooler technologies. What glory is there in maintaining the status quo?

Furthermore, what's the difference between regression, system, or acceptance testing? Sure, there are subtle distinctions between the definitions of each, but at the end of the day, aren't they all about making sure the stuff works? Regression testing is about proving a negative: making sure that there is no unexpected impact from changes. How do you prove that something you don't expect to happen, didn't happen?

What's in a change?

Regression testing assumes you know what the application used to do before you make any changes, which implies a known set of requirements. But the fact is, few IT shops maintain current application requirements, relying instead on the individual expertise of testers conscripted from the ranks of business users. Thus, regression testing is only as complete as the knowledge of the person(s) performing the tests.

Aside from the inconsistency of this approach, there is a deeper problem. A "change" to an application's functionality is not necessarily the result of actual modifications to its code. In today's complex environment of interdependent and multi-tiered applications, there are myriad factors that can affect operations: a new version of the operating system, middleware, database, or change to the network topology or hardware configuration can result in downtime.

For example, I recently reviewed the production trouble tickets for a large IT shop and found that fewer than 25% of the incidents could truly be described as defects in the software itself. The rest were caused by ancillary system resources. Regression testers borrowed from the ranks of users can hardly be expected to know about, let alone understand and test for, the potential impact from changes of this nature. But if regression testing sounds bad and is poorly practiced, what's the answer? Well, how about renaming, redefining, and repositioning it?

What’s in a name?

First, lose the name “regression testing”. It not only conjures up negative images, it's also all about proving a negative, which is impossible.

Instead, start talking about Business Process Validation, which assures that critical business processes are still available and accurate after changes have been made.

Notice what we are going to assure: not what the system used to do, whatever that is, but that critical business processes are still available and accurate. The problem with regression testing is that what used to work is undefined. If you don't know everything the software used to do, how can you test it?

Consider the word critical, which means that BPV is concerned with those aspects of an application that are essential. In other words, we're not talking about every possible error condition, every bug ever detected, or each and every combination, boundary, data type, etc. This distinction is crucial because it implies risk management. If you know you don't have enough time or resources to get complete coverage, then prioritization is key.

Now think about a business process. The "business" half of this term places this type of testing squarely outside of development. The "process" half says that these are user scenarios, examples of everyday tasks that run the business, not mathematically derived test cases. Taken together, this means that the business-user community is both a participant and a beneficiary in BPV.

Finally, let's examine the phrase “after changes have been made”. Notice the word "changes" is not qualified: Instead of just modifications to application software, a change can be to the hardware or any aspect of the environment--even the business process itself. Critical applications reside in highly complex, interconnected environments, relying on multiple tiers and layers of functionality. It is unrealistic and downright naive to apply application-centric test techniques to integrated IT environments.

The implication is that the test environment should be a microcosm of your production environment--not just in its configuration, but in its processing cycles. Thus, BPV testing can't be cadged from a corner of a development system, making do with volatile data and unstable software configurations. It requires a tightly controlled and well-managed test environment.

The last detail to settle here is exactly where Business Process Validation fits in your delivery process. The most logical place is as a condition prerequisite to promotion into production. In other words, BPV should be the gateway to operations, the point at which the business confirms that it can continue to carry on uninterrupted. It happens last.

Why is that so important? Because someone in a deadline crunch might slight a process as obscure as regression testing, but who would dare fail to assure that critical business processes are available and accurate? Perish the thought!

I was testing a radar system that controlled the guns and missile launchers on a naval combat ship.  The radar system consisted of a Fore subsystem, Aft subsystem and a Weapons subsystem. After several months of testing we discovered that if we started the subsystems in a certain sequence the Fore guns and Aft guns would reverse and point at the ship. The scary thing was this bug had existed for years without being detected! Fortunately all ships have mechanical interlocks to guard against any weapon pointing at the ship itself and committing suicide.

-- John Mac Millan

 

Do you have a bug whose story you love to tell? Let me know!

A few recent blog posts have attempted to explain the Devops bloody revolution movement. I’m overdue to post one of my own. Lest we get carried away, let’s not just focus on the Developer < -> Systems Administrator axis. As Graham expounded on Friday night (and I paraphrase, for the Build Doctor had prescribed himself ale):

We need Ops-Ops! There’s so many teams where the admins hate the DBA’s, and the Networks teams hate the admins. And the DBAs aren’t fond of anyone. There’s enough trouble getting those guys to work together.

He’s got a point. Those damn siloes are everywhere.

Share with the group:

Related posts:

1. Devops lightning talk Chris argues for us all to get along …. ...
2. London DevOps Meetup There’s some fantastic DevOps talent in London.  It’s been a...

Related posts brought to you by Yet Another Related Posts Plugin.

For most who read a software testing blog, web access is a given — it’s always on, always up, usually fast, and even available on-the-go (as long as you remember to bring your Nexus One, Curve, iPhone, etc).

But not too long ago, the web was still in early adopter mode.  It was available (maybe) after you fired up that block you called a desktop computer; and after you endured the sound of your dial-up connection; and only if you exhibited zen-like patience with pop-ups and page load times.

Why am I taking this trip down memory lane?  Well, it could be because I saw the extended trailer for Hot Tub Time Machine (destined to be a classic, but NSFW).  More likely, however, is the fact that yesterday I read a couple of interesting pieces from Mashable & the BBC — about the global adoption of the Internet in the past decade, and the provocative question of whether or not web access is an inalienable human right in this day and age.  Both are worth checking out, if for no other reason than to make us appreciate what we have.

And since we have a global community of software testers, I’m extremely interested to hear what the software-savvy readers from every corner of the globe have to say about this very cool interactive map from the BBC.  Does this fit with your experience in your home countries?  What do you think this chart looks like in 2012?  2020?  Share your thoughts in the comments.

Now if you’ll excuse me, I have to go complain to the barista that the wi-fi in this Starbucks is taking way too long to download songs from iTunes and ripped files from BitTorrent, while I watch 30 Rock on Hulu.

Rating: 5.0/5 (5 votes cast)

Much has been said in the blogosphere about the differences between Maven and Ant.  Lines have been drawn between tech bloggers, and one thing has become clear; people love to argue about Maven versus Ant.  And we love the debate.  Constructive criticism is what keeps companies fresh, and products user-friendly.  Here is another chapter in the debate:

Standardization of build systems is one of the main benefits that Maven brought to the world. If you know one Maven project, you can switch to any other project built with Maven and feel comfortable immediately. This increases productivity, both personally and for your company, which is one of the reasons more and more companies switch over from Ant to Maven.

To read the full post, click here.

 

Sonatype is excited to announce that p2 is one of the finalists for the Eclipse Community Award!  P2 is a finalist in the ‘most open project’ category.  The winners will be announced at EclipseCon 2010 on March 22.  Sonatype’s Pascal Rapicault was also nominated in the ‘Top Committer’ category.  To watch a screencast of Sonatype’s p2 support, click here.