In a previous article, I said Test-Driven Development is a possible development workflow that incorporates Developer Testing, but Developer Testing does not necessarily only exist in Test-Driven Development (TDD). Later I pulled together this graphic I could equate all of the steps between TDD and Waterfall, but would gloss over the design phase. Then, one [...]

When Noah Sussman asked me to give a Code as Craft Technology Talk last week when I was consulting for Etsy I immediately said yes. However, I was a bit surprised when the talk was announced under the title "An Evening with Sebastian Bergmann". When I read that title the first time, it was just minutes after Arne, Stefan and I had talked about one of our favourite scenes from "A Knight's Tale":

Chaucer: I'm a writer.

Wat: A what?

Chaucer: A wha- a what? A writer. You know, I write, with ink, and parchment. Geoffrey Chaucer's the name, writing's the game. You've probably read my book? The Book of the Duchess? No? Well, it was allegorical.

Roland: Well, we won't hold that against you, that's for every man to decide for himself.

Just like with a talk that I gave last year, I suddenly had a chain of associations in my head that I just had to follow. And down the rabbit hole I went once more ...

Hi! I have no idea why this talk is titled "An Evening with Sebastian Bergmann". I hope that this evening will turn into an interesting discussion about PHPUnit and all things testing. To break the ice, and to not appear completely unprepared, I came up with the following slides ...

So I recently talked with my friends Arne and Stefan about the movie "A Knight's Tale". Somehow the idea stuck in my head that a variation of that title might be a good idea for a talk of mine. After dismissing "A Fool's Tale" I arrived at "A Tool's Tale". So that's what we're stuck with and what I am going to try right now ...

 

 

 

What I just did was a variation on how Geoffrey Chaucer introduces himself in the movie.

Some time in 2001 I started to work on PHPUnit because I wanted to have something like JUnit for PHP. The initial "port" was completed within one weekend. The code was ugly because I had to emulate exceptions which PHP 4 did not have. On November 27th 2001 I checked the code into cvs.php.net.

A lot has changed since 2001. Hopefully not very many developers are still stuck with PHP 4. As of PHPUnit 2.0, which was released on July 14 2004, the day after PHP 5.0.0 was released, PHP 5 is required to run PHPUnit.

PHPUnit is neither the only testing framework nor the only quality assurance tool for PHP. Over the last years a nice ecosystem of static analysis tools started to grow.

I get this question a lot: why do only Germans work on tools that tell me that my code is bad? I do not have an answer for this, sorry. But the statement is also not true: PHP_CodeSniffer is not developed by a German.

PHP has changed and so the have the tools we use to write and maintain code. In 2006 the PHPUnit code was migrated from CVS to Subversion ...

... and in December 2009 from Subversion to Git and GitHub.

PHPUnit is no longer one big monolithic package. It has been refactored to components that are being reused by other testing frameworks, for instance. Unfortunately, this refactoring was not without problems.

PHPUnit gets more and more convenience functionality. Here are a couple of examples:

 

 

 

Tests can be written, both unintentionally and intentionally, in bad ways. Bad tests can lie. They can give you a false sense of security by reporting that something is tested when it really is not.

This is why I started iplementing coutermeasures against bad tests such as the (ultimately useless but still interesting) assertion counting ...

... as well as the strict execution mode.

There is code in PHPUnit that I am not proud of. I am trying to make the world a better place by eliminating one singleton at a time, for instance. Unfortunately, cleaning up code sometimes breaks things. Contrary to what a popular opinion on Twitter is, I do not like breaking backwards compatibility in PHPUnit and try really hard to avoid it.

This is a "good" example of such a backwards compatibility breakage. It happened in PHPUnit 3.6 because I eliminated a Singleton. Had I known that many developers used this API (instead of the XML configuration file) to set up a code coverage blacklist or whitelist I would probably not have done the change. If more people would have tested the release candidates leading up to PHPUnit 3.6 they could have told me ... oh well.

At this point the "Evening with Sebastian Bergmann" turned into the interesting discussion I had hoped for.

So, now that 2012 is here, I can confidently say that I have accomplished two things…the first is proving that I can indeed completely neglect my site for a year. The second is that given a free weekend I can still finish things. I just got done rolling the 1.0.0 stable release of Phake.

I spent the better part of yesterday and today working out the last of the kinks and I am pretty happy with the results. It provides a great alternative mocking framework for PHPUnit that is compatible not just with older versions of PHPUnit but also older versions of PHP (5.2). You can look through my blog for some of my initial posts about it, or you can peruse the Phake documentation. It allows you to do quite a few nifty things.

A big thanks to all those who have contributed feedback / code thus far.

Happy new year!

Yesterday I was working on some PHPUnit test cases and couldn’t get two of them to work as I wanted them to.

The answer was actually in the documentation and taught me not only about why it wasn’t working but also a lesson about documentation.

What I was trying was to have two texts from which one depended on the other. They were similar to a map/reduce pattern. The fist test was asserting the mapping and the second asserted the reducing.

It made sense to have more than one test data set to map and reduce so I added @dataProvider to the first test.

It then made sense to use the mapped result from the first test as te input to the second so I made my second test @depend on the first.

Nothing worked anymore..

The answer is actually in the documentation at Chapter 4. Writing Tests for PHPUnit / Data Providers:

When a test depends on a test that uses data providers, the depending test will be executed when the test it depends upon is successful for at least one data set. The result of a test that uses data providers cannot be injected into a depending test.

and

All data providers are executed before the first call to the setUp function. Because of that you can’t access any variables you create there from within a data provider.

So there we go what I wanted it not supposed to work and I have to find another way. Fine.

The lesson I learned about documentation?

Documentation can be redundant. The piece of information above can be read two ways. It is documented at the @dataProvider part which I was looking at first before I decided to use @depends as well. When I looked at the @depends chapter I was already past the @dataProviders chapter and couldn’t find this information.

You see that a piece of information about two things needs to be available for both things. I’m sure that is a bugger to maintain but then documentation isn’t easy.