Moxy lets you generate mock objects on the fly from C++ header files. Mock objects allow interaction-based unit testing and can improve object decoupling.

CMock is a module/object mocking framework for C projects. It generates dummy modules that conform to the interface specified in a header file. This allows APIs to be proven out and exercised before committing to an underlying implementation.

Unity is a lightweight xUnit-style unit test framework for C. It was developed for resource constrained environments and includes a number of features helpful for embedded development. Unity has been used successfully in a range of projects from very ...

Embedded Technology 2008 - Yokohama, Japan – November 17, 2008 – Coverity, Inc., the leader in improving software quality and security automatically, today announced the availability of Coverity Architecture Analyzer. This new version of ...

Siege is an http regression testing and benchmarking utility. It was designed to let web developers measure the performance of their code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, ...

• Twitter / Jeffrey Fredrick: CruiseControl 2.8 availabl ...
  CruiseControl 2.8 available to download ... !
• Green & Red Local Builds (adding colour to the local build process)
  Why not?
• TeamCity Dev Blog: A chance to try almost TeamCity 4.0 before the official release
  It's almost here ...

Super Smack is a benchmarking, stress testing, and load generation tool for MySQL (and PostgreSQL).

If you've been following along, by now you know that WatirCraft is a major contributor to the ongoing development of Watir.  Many of the fixes and enhancements over the past six months have been a direct result of Bret's 100% dedication to the project, and more recently the involvement of Jim Matthews, all funded by WatirCraft.

Going forward, we'd like to hear from as many Watir users as possible about what features or capabilities you'd like to see in Watir in 2009 and beyond.  How can we make Watir easier to use, more productive, or better in any way?  What should our priorities be?

To that end, we've added another channel for feedback.  We hope you will go to the Watir page on Uservoice, register, and start entering your suggestions and voting.

Fair warning: we may choose to implement your suggestions in either open-source Watir or in a commercial (not free) add-on from WatirCraft.  We hope you'll agree this is fair, because WatirCraft will need some revenue as a company to fund our ability to continue actively supporting open-source Watir development.   Seem reasonable?

So - what do you need most from Watir that you're not getting right now? 

Isn't that what mock frameworks do? Yes. But.

In order to make our tests pass, we need to isolate everything our test pass through. This means ignoring void calls, and return values on methods in order for our test to keep running. And not crash in the middle.

So assuming we landed a big hunk of code we need to test, we start specifying behaviors left and right. Ignore this, return a not-null object. Just to test a very specific piece of code. But do we need to actually write this code? Can't we just ignore anything we don't care about?

Recursive fakes do just that. It starts with a root object. This object has all kind of methods and properties that return values. Any returned object from these methods also has methods and properties, and so on. In one short line of code, I can ignore any calls on children objects, and return fake objects from their methods.

For example: In this method, there's a chain of calls, which basically are method calls on children objects:

public void SignOut()
{
HttpContext current = HttpContext.Current;
HttpSessionState session = current.Session;
session.Abandon();
}
To ignore everything, along with calls to children, I'd do this:

Isolate.Fake.StaticMethods<HttpContext>(Members.ReturnRecursiveFakes);
That's it. No need to set any other behaviors. All calls in SignOut will be faked, and no NullReferenceException in sight.

Here's a twist for you. Let's say I want to return the item count in a session. This time we need to return a value on this:
HttpContext.Current.Session.CountEven without declaring a fake on HttpContext, I can do this:
Isolate.WhenCalled(() => HttpContext.Current.Session.Count).WillReturn(5);That's easy enough. But I can also do this:
Isolate.Fake.StaticMethods<HttpContext>(Members.ReturnRecursiveFakes);
HttpSessionState fakeSession = HttpContext.Current.Session;
Isolate.WhenCalled(()=>fakeSession.Count).WillReturn(5);Since HttpContext is now recursively faked, I can just grab the fake Session from the object tree and set behavior (or verify) on it. This is not intuitive at first, but in the long run makes tests more readable.

This technique is great for object models. Just grab the main object, slap a recursive fake on it, and focus on what you really need to test.

Note that you can click the link to learn more on Unit Testing . Or click the following link to learn how to unit test a Microsoft SharePoint application.

Technorati Tag:
.Net , Visual Studio , Microsoft , Agile ,Test Driven Development ,Mock

I've promised to do this a while back. So here goes.

From version 5.0 on, Isolator comes in two license flavors - commercial and open-source. The free open source license gives an opportunity to open source projects to use Isolator as their platform for unit tests.

If I'm a developer who's working on an open-source project, I'd go to the download page at the Typemock site and download the open-source installer. After installation and registration I get a free license key and I'm good to go. I can, download the source of my project, which contains Isolator tests, and work from there. I can add tests and check them in so others can do the same. Each contributor needs a license key, which he receives following the download from the site.

If I already have a commercial license (from my company or personal license), I can use the Isolator version and license I already have installed to work also on the open source projects. There's no need for license switching. I can work with what I have.

Now, there is a need to communicate the version of the Isolator you're using in the project to all the contributors and users, otherwise, tests may not compile or run properly. But that's true to all 3rd party components in projects. Once a new version of Isolator comes out, assuming the tests support it, I can download it and continue working with the former license still intact.

That's it. Very simple. Go ahead, get the free open source license now, and simplify your software development.

Technorati Tag:
Open Source , free license , Software Development

How do you perceive quality assurance? Back when I was evangelizing QA in my previous job, I often used the question "What do you think of when you hear the term QA?" to jump-start introductory presentations. When speaking to developers I would almost invariably get the following response: "They are there to test my product".
So isn't it just testing?!This perception of QA and QA people is very narrow and misguided; classic Quality Assurance is made up of three widening circles of responsibility:

• Software Testing: this is where the product testing is done. This is what most developers think of when they hear the term QA.
• Quality Control: this is where the product's actual quality is quantified and managed through test planning, test environment and execution planning, defect measurements and more. One of the QC activities is software testing.
• Quality Assurance: this is where the product's development quality is monitored and (hopefully) iteratively improved in order to assure it will product a quality product. This includes monitoring the product through testing and measuring it (in other words, QC), and also monitoring the product's development processes.

Process improvement through QA?Yes. Quality Assurance literaly means making sure we develop a quality product. It does not mean only testing that the product we made is of high quality; at this stage it may already be too late. If the development processes is in disarray, corners are being cut, assumptions being made etc., the end product will most probably be of low quality, and you don't need to wait until a host of testers take over it to get the news. Making sure the QA process is methodical, correct (i.e. answers the customer's requirements) and often reviewed, helps get a more mature product to the testing phase.
A call to arms
One way I know of (which deeply impressed me) to get development done in a highly methodical, self-reviewing and highly measurable way is TDD: everything is tested well before it's out of development, everything is requirements-driven and by utilizing an automated build you can get a very quick indication the moment a mistake is made; the process is practically self maintaining.
QA engineers, leads and managers, this is a shout out to you. Do you want to improve the product quality and get outside the "you're there to test the product" box some developers place you in? You can promote implementation of TDD and unit testing in your organization. It will be good for the development process. It would even be good for your (or your employees') personal development as it expands your horizons into the world of development practices and coding skills.
So, are you going to pick up the gauntlet? Want to get started and don't know how? Discuss it in the comments or in our forums!


Technorati Tags:

QA , Quality Assurance , mock, Software Testing,Software , Net Development , Asp.net , C# , Agile ,Test Driven Development , Mock

One of the new features in Isolator 5.1.1 is fixing an old issue. Let's say I have a collection I'd like to fake. I use the following:

MyCollection coll= Isolate.Fake.Instance<MyCollection>();What happens when I try to do the following?
foreach (item in coll)
{
Console.WriteLine(item.Price);
}The foreach statement, actually calls the collection's method GetEnumerator, which returns an IEnumerator implementer. Makes sense, but what does it really return? It's an enumerator that "knows" how to enumerate its parent. But what happens for a fake collection? Even if it does return something, the object returned is usually a fake also. It doesn't have any knowledge about the fake parent, and if it calls on some of the parent's methods we'll probably come to a screeching exception halt.

But let's get back to what I want to solve: I want a collection, that although it's a fake, still acting like a collection. That means for a for-each, I want to return my custom values. And not crash, which is also a reasonable requirement.

Isolate.WhenCalled(() => coll).WillReturnCollectionValuesOf(new List<RealLogger>{item1,item2});
The syntax requires a bit more attention than usual. Notice that the WhenCalled Lambda expression contains coll, which is different than the usual method call. With this syntax (which is equivalent to the property get syntax), we identify a collection that we plan to enumerate on. The WillReturnCollectionValuesOf specifies another collection, that will be enumerated when the foreach is invoked on our collection. So in this case, when the foreach clause above runs, it will return two items: item1 and item2.

Let's take a look at a more advanced case, where a collection is used inside another collection. Here's some SharePoint code we'd like to test:
public void DeepIterationWithForEach()
{
SPSite site = new SPSite("http://sharepoint.typemock.com");
using (SPWeb web = site.OpenWeb())
{
foreach (SPList list in web.Lists)
{
foreach (SPItem item in list.Items)
{
item.Update();
}
}
}
}Let's concentrate on the foreach-inside-foreach. SPLists contain SPItems. Here is some test code:
[TestMethod]
public void SwappingSharepointCollections_IterateOverCollectionWithForEach_SwappedIteratorsAreUsed()
{
SPSite fakeSite = Isolate.Fake.Instance<SPSite>(Members.ReturnRecursiveFakes);
Isolate.Swap.NextInstance<SPSite>().With(fakeSite);

Isolate.WhenCalled(() => fakeSite.OpenWeb().Lists[2].Items).WillReturnCollectionValuesOf(new List<SPItem>
{
Isolate.Fake.Instance<SPItem>(),
Isolate.Fake.Instance<SPItem>(),
Isolate.Fake.Instance<SPItem>()
});
SharepointUsingClass target = new SharepointUsingClass();
target.DeepIterationWithForEach();

var fakeItemList = fakeSite.OpenWeb().Lists[2].Items;
foreach (SPItem item in fakeItemList)
{
Isolate.Verify.WasCalledWithAnyArguments(() => item.Update());
}
}

After faking SPSite recursively, let's take a look at the WhenCalled clause. It focuses on fakeSite.OpenWeb().Lists[2].Items, which is the third list in the external collection. In the spirit of "fake everything that you don't care about", we don't care about the first or second list. We care about the 3rd one, and this is where we specify what items we return in the collection. We use the WillReturnCollectionValuesOf to specify the items we want, in our case, a List of fake SPItems. Apart from foreach-ing inside the target method, we also enumerate over the list in the test to Verify the calls.

So now you can fake collections easily with Isolator. If you'd go a step further we'll get to duck-typing. But that's another post.

Note that you can click the link to learn more on using SharePoint or to learn how to unit test a SharePoint application

Technorati Tags:

Microsoft , Microsoft sharepoint , Unit Testing SharePoint, mock, Software Testing, Swapping Collections

As I mentioned in my previous post on Unit Testing SharePoint , the Pattern and Practices group now use Typemock Isolator for their SharePoint Guidance tests. With the recent progress we do in our development, I took one of their tests, which use Natural Mocks, and translated it into AAA new API.

Let's look at the original test (which is bundled with the Microsoft SharePoint guidance package):

[TestMethod]
public void CanAdd_NaturalMocks()
{
SPWeb web = this.RecordAdd();

Registration registration = new Registration();
registration.Title = "UnitTest";
registration.CourseId = 1234;
registration.UserId = 100;
registration.RegistrationStatus = "Pending";

RegistrationRepository repository = new RegistrationRepository();
int id = repository.Add(registration, false, web);

Assert.AreEqual(1, id);
MockManager.Verify();
}Which calls the RecordAdd() method that contains the expectations:
private SPWeb RecordAdd()
{
SPWeb web = RecorderManager.CreateMockedObject<SPWeb>();
SPUser user = RecorderManager.CreateMockedObject<SPUser>();
SPListItem item = RecorderManager.CreateMockedObject<SPListItem>();

using (RecordExpectations recorder = RecorderManager.StartRecording())
{
recorder.ExpectAndReturn(web.SiteUsers.GetByID(1), user);
recorder.ExpectAndReturn(web.Lists[Lists.Registrations].Items.Add(), item);
item[Fields.Title] = "UnitTest";
recorder.CheckArguments();
item[Fields.CourseId] = 1234;
recorder.CheckArguments();
item[Fields.UserId] = 100;
recorder.CheckArguments();
item[Fields.User] = user;
recorder.CheckArguments();
item[Fields.Status] = "Pending";
recorder.CheckArguments();
web.AllowUnsafeUpdates = true;
item.Update();
web.AllowUnsafeUpdates = false;
recorder.ExpectAndReturn(item[Fields.Id], 1);
}

return web;
}
The expectations are based around the hierarchical model of SharePoint objects. The bulk of the expectations are meant to check the expectations. Let's look at how the test looks now with the AAA syntax:

[TestMethod]
[Isolated]
public void RepositoryAdd_FakeSPServer_PropertiesAreSetWithCorrectArguments()
{
var fakeWeb = Isolate.Fake.Instance<SPWeb>(Members.ReturnRecursiveFakes);
var fakeUser = fakeWeb.SiteUsers.GetByID(1);
var fakeItem = fakeWeb.Lists[""].Items.Add();

Isolate.WhenCalled(() => fakeItem[0]).WillReturn(1);

Registration registration = new Registration(){
CourseId=1234,
RegistrationStatus = "Pending",
Title="UnitTest",
UserId = 100
};

RegistrationRepository repository = new RegistrationRepository();
int id = repository.Add(registration, false, fakeWeb);

Assert.AreEqual(1, id);
Isolate.Verify.WasCalledWithExactArguments(()=> fakeItem[Fields.Title] = "UnitTest");
Isolate.Verify.WasCalledWithExactArguments(()=> fakeItem[Fields.UserId]= 100);
Isolate.Verify.WasCalledWithExactArguments(()=> fakeItem[Fields.User]= fakeUser);
Isolate.Verify.WasCalledWithExactArguments(()=> fakeItem[Fields.Status]= "Pending");
}

First, you can see that it's much shorter. The secret sauce is using the Members.ReturnRecursiveFakes option. This is very useful in the case of SharePoint or any hierarchical code model.

Another thing to notice, that once we use this recursive construct, we can now extract from the hierarchy the objects (fakeUser and fakeItem) we need for setting behavior and verification. Although, we could use the same chain inside the verification and behavior clauses if we want to. This is contrast to creating the mocks explicitly, then setting them as return values.

I like the power I get from using the recursive options. The tests are shorter and are more readable.

A note of warning: I'm running this with a build that will be out very shortly. We've stabilized the existing features in version 5.0, so you'll be able to use this method from the upcoming release forward with more certainty. The next version is coming very, very soon.

Note that you can click the link to learn more on using SharePoint or to learn how to unit test a SharePoint application.

Technorati Tags:

Microsoft , Microsoft sharepoint , Unit Testing SharePoint, mock, Software Testing

Looking for more information on unit testing SharePoint? Andrew Woodward from 21 Apps posted a blog post and a detailed white paper with great examples for unit testing SharePoint with Typemock Isolator. The article (and the previous article linked from there) is a good read on unit testing, SharePoint specific challanges and practical examples on using Typemock Isolator.


From the article: "Typemock Isolator has become the tool of choice for unit testing SharePoint as it supports faking everything including the sealed classes found throughout the object model ... Typemock have embraced SharePoint which is great news for us SharePoint developers and will help us do TDD on our Web Part".



Technorati Tags:

Microsoft , Microsoft sharepoint , Unit Testing SharePoint, mock, Software Testing

There's a brilliant post by Peter Gillard-Moss answering James Shore's post on The Decline and Fall of Agile and others.

It's a bit long, but worthwhile. He goes through the process of analyzing of agile, and comes to the conclusion:

Nothing's changed, agile or not. The majority of the industry still doesn't value the basic, fundamental skills it takes to write software of acceptable quality. It didn't before and it still doesn't now. Instead it's obsessed with solving the problem by bringing in the right highly paid manager with the right powerpoint presented methodology. Until the industry gets that ain't the way it will drag every shining beacon of light (agile, Ruby whatever) down into Hades with it.

It's all about expecting to find the silver bullet. And it has to be fast. But there is no quick way. And the road is tough. People are not born with it, we have to work hard and make mistakes on the way:

Developers who have never come across TDD before, who've never experienced it's value first hand, who've spent years doing things a certain way are struggling to grasp the concept and oh sorry that's surprising why? I guess you were doing TDD from that first Hello World in Pascal?

So how do we do it? By succeeding, step by step:

By being successful and then telling people about why and how we were successful. That's how this whole agile thing started, that's how it built up it's great reputation and that's how it's going to survive and get better.

I believe agile is going to succeed. It may not be that fast, but it will prevail, because success speaks for itself. Where do you stand on this big issue?

Technorati Tags:

Agile, Test Driven Development , Unit Testing, mocking, Software Testing

Continuing the theme, another misguided idea is that your CI build must never break. I think the real issue here is that this idea focuses on the wrong thing: it’s not really broken builds that matter, it’s why they break and what you do about it.

Broken Builds Are Information

If your build never breaks, then it doesn’t necessarily tell you much. Saying an always-green build implies your product works is analogous to saying that high test coverage implies your code is fully tested. Green builds are good, but how informative they are is highly dependent on their quality. I particularly like Paul Duvall’s characterisation of always-green builds as a possible case of Continuous Ignorance.

When a build breaks, however, you know something is wrong1. This is a concrete indicator that your build has some value: it has found a problem.

Broken Builds != Broken Windows

So a broken build gives you information; the key thing now is how you react to it. Teams in tune with CI react to broken builds by:

1. Fixing them fast: to reduce the impact on the team, and in the knowledge that the cheapest time to make the fix is now.
2. Learning from this failure: and finding ways to prevent it in the future.

Teams that don’t deal with broken builds suffer “Broken Windows” syndrome. This is not because the build broke, however, but because the team didn’t respond as they should have.

What About the Cost?

I’m not saying that a broken build is all good — after all productivity is easily lost while the build is red. In Pulse we have even added many features like personal builds specifically to help reduce the frequency of build failures. However, aiming for completely unbreakable builds also has a cost. All solutions must in the end rely on complete serialisation of checkins and builds. I see this as analogous to pessimistic locking: you take a producitivity hit on every commit just in case something breaks. Not to mention the fact that:

1. For many larger projects this is simply impractical due to the combination of commit frequency and build time.
2. There is still no guarantee of a green build thanks to the existence of non-deterministic bugs.

We advocate an optimistic approach: by using local smoke testing and optional personal builds on each checkin, the vast majority of problems are found. This leaves only the much less frequent chance of a failure due to a logical merge conflict2 or non-determinism. The great thing about this approach is it doesn’t enforce any heavyweight overhead such as extra branching or serialisation on the process, you can pretty much work as you always have.

Conclusion

So, put your effort into a high quality build and listen to what it is telling you. Don’t get hung up on absolute guarantees of unbreakability: with the right response to broken builds you’re better off with an optimistic approach.

—
1 OK, I am glossing over spurious failures here, but I would like to deal with that in another post.
2 As opposed to a textual merge conflict which must be fixed prior to checking in.

Once when I was working for a large enterprise, we where taken  to an off-site team building program. We where divided into groups of 5 and the group had a task of creating 5 squares out of paper cuttings of different shapes and sizes. Each team-player got a few pieces and the rules where:

1. No talking
2. You can only give a piece (no taking)
3. The game ends when every player has a complete square

The trick in this session was that one team-player got just the right amount of pieces to make a rectangle, that team-player would finish his rectangle and then just leave the game feeling that he finished his part. In may cases the team-player would physically leave and go have a coffee or just to chat with other ‘rectangle’ team-players (normally boasting how they finished first not even considering the fact the they don’t have a square but a rectangle).

The point is that the team as a whole cannot finish its task. In order for every team-player to have a square the rectangle must be dismantled by the ‘rectangle’ team-player (as no-one can take any piece only give) and give away some of his pieces. It normally takes the team a few minutes to understand this but as they cannot talk between each other - they are in a deadlock. The ‘rectangle’ team player is sure that he has finished the task and has switched off, no one is allowed to talk or touch his pieces and most of the time the ‘rectangle’ team player is no where to be found. (Bonus: How can you break this deadlock)

The lesson learnt is that although you think that you have done your job - it might not help the team as a whole to complete their job, you might even delay them!

Although one development team are sure that by using the best process (Agile) they have done there job, they might be delaying the rest of the teams and the company as a whole.

The solution is in Communication. More about that later.

Bookmark at:
          

Folks, I've got a ton of ideas to take Creative Chaos. I've been spending more and more time lately involved in discussions of KanBan - which is an evolving method that is slowly competing with Scrum.

I could write about the aspects of software quality that are hard to define requirements for - scalability, performance, reliability, usability - and what to do about it.

I could put more meat on the bones of "when should a test be automated?" (or "run unattended?")

I could talk about the business of software development, negotiating for nerds, or the dynamics of maintenance and legacy software.

Or I could just post more stories from my youth as a military cadet.

I'm probably most interested in exploring the idea of uncertainty. After all, we can't predict the future - yet we have these things called project plans with dates and deadlines and deliverables.

Those are just some of my the ideas bouncing around in my head. Still, at this point, after three years of evolution, you - the audience of Creative Chaos, might know better than me what to do next, because you know what appeals to you and why.

I covet your feedback, and promise to respond and take it seriously.

And, if you've ever just wanted a chance to heckle ... now's you chance.

Please consider leaving a comment for this post; it's your chance to steer the ship.

In LoadRunner SAP monitor, it requires the SAP CCMS (Computer Center Management System) for monitoring to be effective.  This is unlike the typical monitoring mechanism that is used for Windows or...

Visit LoadRunnerTnT for full links and contents!!

The Third Test Management Summit will take place on Wednesday 28 January 2009 at the sumptuous Institute of Directors at 116 Pall Mall, London.

The Summit is an annual gathering of Test Managers who meet as a COMMUNITY.

The Booking Form can be found here.

read more