I guess I assumed that creating a foreign key constraint would automatically create a non-clustered index.  I mean, a primary key is a clustered index, why wouldn’t a foreign key also create an index?

A batch process today slowed from completing in about 4 hours to an estimated completion time of around 3 days.  Since this was a process that needed to complete daily, well obviously it should complete in less than a day.  First order of business was looking at the profiler to see what was going on.

It was exactly one SELECT statement per transaction, no joins, and one predicate in the WHERE clause, on a foreign key.  And it took an average of 23 seconds to execute.  Yikes.

Next up was checking the execution plan and statistics.  This was on a table that doubled in size in one day, to 7 million rows.  The culprit naturally was a table scan, and the 230K reads showed why the query took so long.  A coworker pointed out that the statistics plan also provided a hint for performance tuning, to add an index.

I really couldn’t believe the foreign key didn’t have an index, but sure enough, it didn’t.  Added the index on the foreign key, and the query time was reduced from 23 seconds to nearly instantaneous, 2-3 orders of magnitude better.

And that 4 hour batch process?  It finished in about 5 minutes.

Yes, database indexes are your friend.  Assuming database indexes are already in place, not your friend.

*sigh*

In one picture:

In Hg, branches are metadata included in each changeset.  In the operation above, I created a branch, but that only marked the current directory with a branch name.  The branch won’t actually show up unless I make a commit.  In fact, if I checkout to another revision, it’s as if the “hg branch” command never happened.  When I list the branches, you won’t see this new branch name.  The name in the PS window just comes from the PowerShell-Hg extension being smart.

One implication is that our team has to make phantom commits for the branch to officially “show up”.  You start to see commits like “phantom commit” early on, then later they start saying “stupid commit”.  It’s the only way we could get a branch to show up locally and on the server.

In Git, a branch is nothing more than a pointer to a commit, and the commit itself carries no information about a branch name.  A much more flexible model in my experience.

This hit me today when I wanted to move a set of commits to a different branch, which turned out to be very difficult as the branch name was embedded in to the commit names.  Rebasing helped, except that the commits were already pushed to the server.  We wound up having to back out of the commits, even though these were on a different logical branch, simply because our build server got confused about these extra commits that were marked with the wrong branch name.

Perhaps Hg branches aren’t broken and this design is intentional.  But it’s annoying and less powerful and flexible than Git’s more simple model, of just a pointer to a commit.

Side note for the Git folks – I’ve also wasted 2 hours of my life on Git when my .git folder suddenly went empty – and not from me accidentally deleting anything.  Lost all my dangling topic branches on that one.

On a recent large project, we had a goal early on that we didn’t want to have a lot of QA folks manually testing our software.  Finding bugs through manual testing is incredibly time consuming and expensive, so we opted to try and build as much quality in to the product.  That’s not to say that manual testing doesn’t have its place, as humans are fantastic about using software in ways you didn’t expect.

This was a long project, around 18 months, and will continue to have active development in the future.  Very early on we found that a good testing strategy was critical to the success of the project, especially for our team to be able to 1) continue to increase our velocity over time and 2) have the confidence to make both small and large changes in our application.

It took quite a while for us to settle on an effective strategy.  This was mostly because we had to learn how to design our application for testability, in all layers of the application.  Our team were all experienced in TDD before starting the project, but that wasn’t the only skill we needed to create an effective testing strategy.

Levels of tests

Categorizing tests can get annoying.  You have functional tests, integration tests, unit tests, acceptance tests, slow tests, fast tests, UI tests, and on and on.  We found that our tests belonged in three main categories:

• Full-system tests
• Subcutaneous tests
• Unit tests

Each of these differs in the scope of what’s being tested.  A full-system test exercises the application through the external interface, whether that’s a browser, file drop, queue, WinForms app or whatnot.

Subcutaneous tests work at the layer directly below the external user interface.  In the context of a web application, a subcutaneous test in our case would be a form object sent through a command processor, with all the real classes and implementations in place.  We bypassed the Controller, which only contained UI layer logic, straight to the domain layer.  Send in form object, out pops success/failure.

Finally, we had unit tests.  Unit tests are designed to test one class, and can either be fast or slow tests.  Fast tests are the normal TDD tests, used to build out class design.  Test doubles are used as needed, but strictly interaction-based tests have less value unless we find the interactions very interesting.  We also have slow unit tests, which could also be classified as integration tests.  These would be things like repository tests, persistence tests, etc.

Our ratio of unit:subcutaneous:full-system tests hovered around something like 10:2:1.  We ended the project with something around 5000 unit tests, 1000 subcutaneous tests, and 500 full-system tests that used WatiN and Gallio to drive a browser.  The 6000 unit/subcutaneous tests executed in about 10 minutes, while the 500 UI tests completed in about 50 minutes.

Unit testing strategy

Unit tests were developed in a pretty strict TDD manner.  We write tests before any implementation is in place, and use the tests to drive the design of the code.  These tests help identify design issues, encapsulation problems, code smells and so on.

We strived to not write code that existed solely to enable testing.  That often meant that we had a design issue, and responsibilities were misplaced or encapsulation was violated.

As we got further down the pipeline in our project, we started to value interaction tests less and less.  Interaction testing through mocking is only really interesting if you’re truly interested about interactions.  But more often, we were more interested in side-effects, and interactions were just an implementation detail.  What we often did instead is mock out slow or untestable pieces, like repositories, facades over external services, configuration classes, etc.  Otherwise, we limited mocking only to places where mocks were the only observation points for what we were interested in.

At some point in large projects, it can become obvious that your design needs a large-level refactoring, to extract out concepts to enable quicker delivery of features.  On our last project, some concepts unearthed included:

• AutoMapper
• Processing forms as individual command messages
• Input builders

With each of these, unit tests were actually a barrier to these refactorings.  But the barrier only existed because we had relied on these tests to capture all of the interesting behavior in our application.  To effectively allow large- and mid-size refactorings, we needed an additional level of testing.

Subcutaneous testing strategy

Subcutaneous tests, like their name implies, test everything just below the surface of the user interface.  In an MVC application, these would be tests for everything just below the controller.  For a web service, everything just below the endpoint.  The idea is that the topmost layer in an application does not perform any actual business logic, but just connect the external interfaces with the underlying services.

Subcutaneous tests are important because we want to be able to test business logic with the entirety of the system in play, with the exception of external connection points such as the user interface and external services.  While a unit test focuses on small-scale design, a subcutaneous test does not address design, but instead tests basic inputs and outputs of the system as a whole.

To build effective subcutaneous tests, we can try and build uniform pinch points through which common logic flows.  For example, we might build a command message handling system, or a common query interface.  In a recent project that processed batch files, each row in the file was transformed into a message.  We could then craft a message, send it through the system, and then verify all the side effects of processing that message.

Because subcutaneous tests address high-level behavior, rather than design, they are ideal for scenario-based testing strategies such as BDD or the Testcase Class per Fixture pattern.  If we want to be able to perform large refactorings, we need these high-level tests to create that wide-cast safety net for business behavior.  Subcutaneous tests are also great target points for calling features done, as they focus on more end-to-end logic.

While subcutaneous tests allow us to safely perform larger refactorings, they still do not provide a satisfactory level of confidence that our system will work in production.

Full system testing strategy

Our team originally called these tests “UI tests”, until more and more of our projects entailed integration strategies where the inputs to our system weren’t a browser, but instead messages, a REST endpoint, or FTP drops and batch file processing.  UI testing is a subset of full system testing.  The idea behind a full system test is that we want to test our software as it might be used in production.  For an MVC application, these would be browser-based tests.  For batch files, we would use actual files.  REST, actual HTTP requests.  Messages, real queues and messages.

If we want to know if our application works as expected, before it goes to production, one effective and efficient way to do so is to create an automated test that exercises the full system.  If my UI tests logs in to the application, places an order and I can verify that an order request was generated, I’m feeling pretty good about things.

One common misconception about full system tests is that they are black box tests.  While these have their merit, full system tests should have intimate knowledge about what’s going on underneath the covers.  In fact, full system tests can even take advantage of the domain model to build up data, instead of a back-door system built solely for testing purposes.  One big mistake teams run in to is not following the same code paths in testing as they do in production, leading to wacky invalid, impossible states of the system.

In our projects, a full system test is the last code we write before we call a feature/story done, done, done.  Manual testing is just too expensive and unreliable for characterizing “done-ness” of a feature, but if I can do the exact same actions as would happen in production through the exact same external interfaces, that’s success.

A holistic approach

In an application without tests, we’ve actually found it most valuable to start with full system testing, moving down towards unit tests as a means of a strategy for coverage.  We cast the widest net possible, but the simplest assertions first, then slowly move down towards unit-level logic.  In new applications, we tend not to focus on any one area, as all of these tests are critical to us for long-term maintainability of a system.

This testing strategy does require a level of investment.  We’ve found this holistic approach especially effective when we know that this application is critical to our client’s business.  If an application is critical to business, it’s going to require change.  If it’s going to require change, we better be able to safely change it without affecting our client’s business.

In the last post, I posed the question on what to do with .NET framework upgrades and OSS support.  I wanted to upgrade AutoMapper to .NET 4, but I didn’t want to leave a lot of folks behind because their project isn’t .NET 4.  I had a couple of choices:

• Call the last release (1.1) the last official .NET 3.5 release
• Branch it and allow both to live side-by-side

Since option #2 didn’t really cause me much extra overhead, I went for that one.  To upgrade, I first needed to create a remote branch on github:

Creating a remote branch in git is a little esoteric, as for the most part, git wants to be local.  To create this remote branch, I just ran:

• git push origin origin:refs/heads/NET35

More than a little esoteric, but whatever.  To then start using this remote branch, I needed to create a remote tracking branch to make it easier to push and pull:

• git checkout –track –b NET35 origin/NET35

And now my local repository has both remote branches being tracked:

With both branches going, I now set about upgrading the AutoMapper to .NET 4.

Upgrading the project

Upgrading the AutoMapper code to VS 2010 and .NET 4 was easy enough.  I just opened the VS 2008 solution in VS 2010, and the upgrade wizard properly upgraded everything.  Upgrading the solution files does not change the target framework, so I still needed to modify all of the project files to target .NET 4:

That was easy enough, and all tests immediately passed.  I decided to leave the Silverlight 3.0 projects out of the equation, I still need a better way to support multiple runtimes easily.  Some of the app.config files were automatically modified, but there were really no problems upgrading the actual projects to .NET 4.

Upgrading the build, however, was a completely different story.

Upgrading the build

Most of the build tools I used, including NUnit and psake, were still targeting .NET 3.5.  To upgrade these, I first needed to download the latest psake and include the following configuration in my build script:

$framework = '4.0'

This told psake to use the .NET 4 framework.  Next, I needed to tell NUnit to target the .NET 4 framework.  I was able to do this by adding the “supportedRuntime” element:

<startup>
    <requiredRuntime version="v4.0.20506" />
</startup>

I stuck this in my “nunit-console-x86.exe.config” file, where it already had a commented-out section for me to edit.

The last part I needed to fix was the ILMerge call in my build script.  This was much trickier.  I needed to modify the call to ILMerge to include the path to the .NET framework.  Previously, I only needed to include the version.  I created a PS function to get the framework directory:

function Get-FrameworkDirectory()
{
    $([System.Runtime.InteropServices.RuntimeEnvironment]::GetRuntimeDirectory()
        .Replace("v2.0.50727", "v4.0.30319"))
}

Quite lame, as I get the current runtime path and replace the folder name of the .NET 2 version number with .NET 4.  I’m sure there are better ways, but hey, it works.

Finally, I just created a property to hold this value:

$framework_dir = Get-FrameworkDirectory

And modified the call to ILMerge to include this value:

ilmerge.exe /targetplatform:"v4,$framework_dir"

Once all that was done, I created a second build on the TeamCity CodeBetter build server:

That’s pretty much it.  It was a little hairy upgrading the build, but that’s only because of the tools I use.  If I didn’t have Stack Overflow, it would have been much more difficult :)

Support

The idea going forward is to apply pull requests and bug fixes to the .NET 3.5 branch, but not any new feature work.  I’m not going to delete that branch, so anyone wanting to add new features just needs to fork that branch and send pull requests.  Because branches are so easy to manage in github, there’s really no reason for me to just kill the .NET 3.5 version.

Otherwise, I’m getting started on the master branch on version 2.0 work.  For those that have forked master and want to now point to the NET35 branch, you can just update your local upstream remote to point to the different branch.

I’ve hit a bit of a dilemma recently, I want to use features in C# 4.0 and .NET 4.0 to enhance AutoMapper, but this would eliminate the possibility of .NET 3.0 projects from being able to use the new version.

One option is to say that the current version is the last 3.0 version, and if you want 3.0, use that one.  Another option is to fork, and keep a 3.0 version going forward, where I can apply bug fixes etc. to it.

This is the first OSS project I’ve been involved in that actually survived to another .NET Framework release, so I’m not entirely sure what the community expectations would be around this area.

My initial thought is just to upgrade the library to VS 2010 and .NET 4, but that might leave a few people in a lurch.  Luckily I’m on Git, so it’s not really a problem to support multiple branches.

How have other projects dealt with this?

A lot of times an operation on a single aggregate root needs to result in side effects that are outside the aggregate root boundary.  There are several ways to accomplish this, such as:

• A return parameter on the method
• A collecting parameter
• Domain events

We’ve used the default implementation of domain events for quite a while, but with some recent applications I’ve worked on, we’ve noticed one small design issue:

    public static class DomainEvents

It’s that big ol’ “static” piece.  Domain events are then raised explicitly by calling this static method, Raise:

public static IHandlerContainer Container { get; set; }

public static void Raise<TEvent>(TEvent args) where TEvent : IDomainEvent
{
    if (Container != null)
        Container.GetAllHandlers<TEvent>()
            .Cast<IHandler<TEvent>>()
            .ForEach(x => x.Handle(args));

Where Container in this case is just a facade over an IoC container.  The silly Cast is from this being C# 3.0 code, the contravariance of C# 4.0 would fix this.  Again, another design issue here.  The reference to the container is static.  This means that more powerful container patterns such as nested containers are out of the picture.  This is too bad, because nested containers are another great tool in the toolbox that lets us delete a lot of code that sets up contexts for things.

The problem here is that I still want to raise events in a static manner from an entity.  I don’t want to have to reference some event pipeline object thingy in my domain objects, and I’m really not keen to start injecting things.  Instead, I want a true, fire-and-forget event.

Contextual containers and disposable actions

What I need to do is allow this static method to work with a contextual, scoped piece of code.  But that’s exactly what the “using” statement allows us to do – create a scoped piece of code, that executes something at the beginning (whatever creates the IDisposable) and something at the end (the Dispose method).

To help us create this scoped context to slip in our nested container, we can take advantage of Ayende’s most brilliant piece of code ever written, the DisposableAction:

public class DisposableAction : IDisposable
{
    private readonly Action _callback;

    public DisposableAction(Action callback)
    {
        _callback = callback;
    }

    public void Dispose()
    {
        _callback();
    }
}

I can then just implement a simple method on the DomainEvents class to allow me to swap out – and then restore – the container reference:

public static IDisposable CreateContext(IHandlerContainer container)
{
    var existingContainer = Container;
    Container = container;

    return new DisposableAction(() =>
    {
        Container = existingContainer;
    });
}

I keep a reference around to the previous container, then swap out the DomainEvents’ container for the one passed in.  When this CreateContext is finished, the DisposableAction restores the previous container with a handy closure.

So how do I use this in real code?  Something like:

public void Process<T>(T message) where T : IMessage
{
    using (var nestedContainer = _container.GetNestedContainer())
    using (var unitOfWork = new UnitOfWork(_sessionSource))
    using (DomainEvents.CreateContext(new HandlerContainer(nestedContainer)))

I have several scoped items I’m using to process a message (part of a batch processing program).  Each line in a file gets processed as a single message, with its own unit of work, its own container etc.  It’s now very plain to see the context I create to process the message because I just use the C# feature that creates bounded, self-cleaning contexts: the “using” statement.

This method still isn’t thread-safe, as it still has static elements.  I’ve just allowed a scoped, nested container to be used instead of a single, global static ontainer.  Some folks mentioned patterns like event aggregators, so there are likely other patterns that can help out with the static nature of this domain events pattern.  But for now, I can harness the power and simplicity of nested containers, and keep my handy domain events around as well.

Like many github users, I often create forks for projects whenever I want to pull down their code, rather than cloning from the source directly.  This is pretty much the default way of working on github, as the site encourages collaboration through individualized repositories.  On my github page, this is what you’d see for my repositories:

It’s front and center, right on my home page.  Only two of those repositories were ones that I created completely fresh, and the rest are forks.

The other day, I posted a question on the StructureMap mailing list, but wound up getting a response in a github pull request!

This is because it’s just as easy to fork a user’s repository as it is the central repository.  Even the concept of a “central repository” isn’t ingrained in git (or github), there are only “blessed” repositories that the OSS project’s organizers agree upon.  For example, the “blessed” AutoMapper repository is just the AutoMapper repository in my github.

Pretty sweet, I can pull in this request locally, as well as any upstream StructureMap changes (as Git allows me to have as many remotes as I want), and see if it works for me.  If it does, I can then issue a pull request to the blessed StructureMap repository.  But if it doesn’t go through, no worries, I have my own StructureMap repository :)

Comparison to CodePlex

CodePlex is fantastic for a public-facing project hub, but it’s still not close to github on the OSS collaboration side.  Here’s an example, my homepage for the AutoMapper repository looks like this:

I have a wealth of information at my disposal here.  For new users, it’s a one-click operation to watch the project, fork it, create a pull request.  I can see how many folks follow the project or forked it.  All the information here is about the repository itself, rather than a project.  Github has a separate page for that, but by default, it’s about collaboration rather than documentation.

On CodePlex, I get a list of checkins:

The list of forks isn’t that much better.  CodePlex is definitely making strides, but you can definitely tell the difference between an OSS project hosting site built around DVCS versus one around centralized version control.  In github, its design is built around distributed collaboration, centered around individual commiters.  In CodePlex, its design is built around centralized projects.

Both definitely have their benefits, as it’s super-easy to get a URL to the CodePlex AutoMapper project page:  http://automapper.codeplex.com.  In fact, I kept the project on CodePlex because its support for project-centric activities I felt was better.  However, its support for collaboration-centric workflows still is pretty far away from github.  Unfortunately, CodePlex has to support both centralized (TFS) and distributed (Hg) source control systems, so I’m not sure how it’ll all shake out in the end…

In the last post, we looked at a workflow very common in the Git sphere: utilizing local branches to create segregated workspaces for individual topic branches.  As far as I can tell, this seems to be the preferred day-to-day workflow for Git users, as its first-class local branching support makes it completely seamless to create segregated areas of work.

This workflow is completely possible in Hg, but does take little more set up and one or two extra steps that Git doesn’t force you to make.  The idea behind this workflow is that I can’t predict when unrelated work comes in.  Because local branches and local commits are so cheap, there’s really no reason to throw any code away, ever.  With local topic branches, I can keep mini-spikes around without affecting anyone.

However, there are scenarios where the likelihood of unrelated work is small, so the need for topic branches tends to diminish.  With Hg, the limitations of its model of local branches, bookmarks and rebasing tends to lessen the full benefits of local topic branches.

For those cases where I’m truly confident that I’m working on a continuous main line of work, I’ll use a slightly different workflow than one that uses topic branches.

The simplified workflow

In the simplified workflow, it is nearly identical to the normal centralized workflows (except most operations are local).  When I want to start work for the day, I’ll:

• work work work
• hg commit –Am “Working on some stuff”
• work work work
• hg ci –Am “Working on some more stuff”
• work work work
• hg ci –Am “Finished my stuff”

At this point, I’ve finished some logical set of work, and I’d like to push my work upstream.  My local repository now looks like:

Unlike the previous workflow, my “master” bookmark moves along, instead of always pointing at the latest pulled commit.  It’s still important that this bookmark sticks around, as we’ll see soon.  Now that I want to push, I want to first pull down incoming commits.  Let’s suppose that someone else also made some commits on another repository and already pushed.  The server repository shows this:

Note that we don’t see our bookmarks here, as by default bookmarks don’t get pushed upstream.  When we pull from upstream, we’ll get the commit from the “otherdude” developer.  So, I’ll:

• hg pull --rebase

Instead a “pull/merge/update” workflow, which generates noisy merge commits, I’ll rebase my three commits against the upstream changes.  Rebase simply replays my three commits against the incoming tip.  That would mean that I expect to see that the parent of “Working on some stuff” to be the “otherdude” commit instead of the “Finishing work on a feature” commit.  After the pull and rebase, my local repository is now:

This is what we expected, our commits that originally came after the “Finishing work on a feature” commit got moved AFTER the “otherdude” commit.  This produces a nice clean timeline that makes localizing bugs and merging changes a lot easier.  With a regular pull/merge workflow, you’re merging all 3 commits at once.  With a rebase, I merge one commit at a time, making the potential merges much smaller.  Each merge also modifies each commit, instead of one gigantic merge commit with all changes coming in at once.

Anyway, I’ve pull upstream changes, so now I’m ready to push:

• hg push –b master

I only want to push that mainline branch, “master”, just like my previous workflows.  By pushing only my “master” branch, I can transfer back and forth between my simplified, mainline workflow and topic branch workflow very easily, with neither conflicting with the other.  In Git, only the current branch is ever pushed by default, but in Hg, it’s the opposite, so I have to a bit more explicit.

Comparing rebase and merge

Just to show what a merge looks like in comparison, let’s say “otherdude” doesn’t rebase before he commits his additional work.  He has some more commits:

Now he wants to push these two commits up.  However, the other user already pushed rebased commits, so now the server looks like:

Instead of doing a rebase on pull, he does a regular pull and update.  Because other commits have gone in, he’ll need to do a merge:

• hg pull –u
• hg merge
• hg ci –Am “Stupid merge commit”

He tries to pull and update, but since there were commits there, two heads get created and he needs to merge in his changes.  This causes an extra merge AND commit step, and now uglies up the history:

So the silly thing about this is the stupid merge commit contains ALL changes from the other two outgoing commits, yet all three commits get pushed!  This also becomes really ugly over time, especially when you have overlapping commits and merges:

I’m not sure human beings are meant to comprehend this picture, so I’ll take the rebased workflow with its clean, linear history any day of the week.  With the simplified workflow, rebasing is actually simpler than the pull/merge/commit workflow.  Rebase is good, whether we work in topic branches or not.

Yesterday, I gave a talk at the virtual ASP.NET MVC conference, mvcConf on “Putting your controllers on a diet.  You can find the slides and code at the Headspring Labs CodePlex site:

http://headspringlabs.codeplex.com/

Just clone the Hg repository, and you’ll find all the code and slides.  In the talk, I showed how to incrementally refactor this:

public class ConferenceControllerBefore : DefaultController
{
    private readonly IConferenceRepository _repository;

    public ConferenceControllerBefore()
    {
        _repository = new ConferenceRepository(Sessions.Current);
    }

    public ActionResult Index(int? minSessions)
    {
        minSessions = minSessions ?? 0;

        var list = (from conf in _repository.Query()
                    where conf.SessionCount >= minSessions
                    select new ConferenceListModel
                    {
                        Id = conf.Id,
                        Name = conf.Name,
                        SessionCount = conf.SessionCount,
                        AttendeeCount = conf.AttendeeCount
                    }).ToArray();

        return View(list);
    }

    public ViewResult Show(string eventName)
    {
        var conf = _repository.GetByName(eventName);

        var model = new ConferenceShowModel
        {
            Name = conf.Name,
            Sessions = conf.GetSessions()
                .Select(s => new ConferenceShowModel.SessionModel
                {
                    Title = s.Title,
                    SpeakerFirstName = s.Speaker.FirstName,
                    SpeakerLastName = s.Speaker.LastName,
                }).ToArray()
        };

        return View(model);
    }

    public ActionResult Edit(string eventName)
    {
        var conf = _repository.GetByName(eventName);

        var model = new ConferenceEditModel
        {
            Id = conf.Id,
            Name = conf.Name,
            Attendees = conf.GetAttendees()
                .Select(a => new ConferenceEditModel.AttendeeEditModel
                {
                    Id = a.Id,
                    FirstName = a.FirstName,
                    LastName = a.LastName,
                    Email = a.Email,
                }).ToArray()
        };

        return View(model);
    }


    [HttpPost]
    public ActionResult Edit(ConferenceEditModel form)
    {
        if (!ModelState.IsValid)
        {
            return View(form);
        }

        var conf = _repository.GetById(form.Id);

        conf.ChangeName(form.Name);

        foreach (var attendeeEditModel in form.Attendees)
        {
            var attendee = conf.GetAttendee(attendeeEditModel.Id);

            attendee.ChangeName(attendeeEditModel.FirstName, attendeeEditModel.LastName);
            attendee.Email = attendeeEditModel.Email;
        }

        return this.RedirectToAction(c => c.Index(null), "Default");
    }
}

Into this:

public class ConferenceAfterController : DefaultController
{
    public ActionResult Index(int? minSessions)
    {
        return Query<ConferenceListModel[]>(View(new ListConferences(minSessions)));
    }

    public ActionResult Show(Conference eventName)
    {
        return AutoMapView<ConferenceShowModel>(View(eventName));
    }

    public ActionResult Edit(Conference eventname)
    {
        return AutoMapView<ConferenceEditModel>(View(eventname));
    }

    [HttpPost]
    public ActionResult Edit(ConferenceEditModel form)
    {
        var success = this.RedirectToAction(c => c.Index(null), "Default");
        
        return Form(form, success);
    }
}

The recordings for the talks will be posted soon, so stay tuned to the mvcConf website for details.  Thanks again to the organizers (Eric Hexter, Javier Lozano, Jon Galloway, and some others I know I’m forgetting).  The conference went more smoothly than a lot of non-virtual conferences I’ve been to, so hopefully we have another one soon (well, not TOO soon…)

The nice thing about distributed version control systems (DVCS) such as Git and Hg is that they both allow me to basically decide how my source control should fit with my short and long-term development workflows.  A while back, I wrote what was basically a stream of consciousness post on getting my Git workflow working in Hg.  Well, a teammate tried to follow those steps…and found that I missed a few important nuggets.

My local workflow revolves around local branches and rebasing.  There are plenty of good articles out there on why this is an interesting and valuable workflow to know and understand, so I won’t rehash all the arguments.  I will say that I like this workflow because it:

• Gives me a clean, linear, understandable timeline
• Allows me to keep lines of work separate from each other, until it’s ready to push back upstream
• Works well in the face of unpredictable work
• Is light, quick, and does not leak into the public, mainline work

I tried a few other options, such as real branches, patch queues, and so on, but none had the same flavor that I was looking for with local topic branches.  With local topic branches, I don’t use different commands for committing (as I would with patch queues), nor do my branches leak metadata into the public timeline, as it would with normal Hg branches.

First, let’s get our local environment set up appropriately.

Prepping our environment

The cornerstone of my local development workflow include the Rebase and Bookmarks extension.  To enable these extensions, just modify your hgrc file.  You’ll also want to enable tracking the current commit for bookmarks.  This ensures that our bookmark gets moved up with each commit, instead of getting just stuck on one.  Your hgrc file would now include:

[extensions]
rebase = 
bookmarks =

[bookmarks]
track.current = True

I’ve enabled the Rebase and Bookmarks extension, and configured bookmarks to track the current commit.  Tools like TortoiseHg have features that light up when bookmarks are enabled, so you’ll have all sorts of things help you in those tools.

Now that we have our extensions enabled, we need to create our local marker for a master branch.  This bookmark represents the last pushed commit, so you can execute the “outgoing” command to make sure that you have nothing to push:

If everything’s good, we’ll create the “master” bookmark:

Git will create a “master” branch by default when you clone, but we’ll need to do this manually.  You can think of “master” as trunk.  It represents the mainline of the code we’re working on, and everything will be pulled into this line, both from upstream and from our local branches.  Our repository explorer, as seen in TortoiseHg, now looks like:

Note here that “master” is orange.  This indicates that “master” is the current bookmark being tracked.  Now that we have our local repository set up, we can walk through making local changes.

Scenario 1: One local branch, no upstream changes

First, let’s create a local topic branch and make some changes:

• hg bookmark SomeTopic
• --work work work
• hg commit –Am “Simple Change”
• -- work work work
• hg commit –Am “Some other change”

At this point, we decide we want to push our changes up.  We first want to synchronize our local repository with upstream, but we want to do this on master.  So we:

• hg checkout master
• hg pull --rebase

We switch back to the master branch and synchronize with upstream.  But since there aren’t any upstream changes, we want to now fold our SomeTopic branch back to master.  Here’s what the picture looks like right now:

Since there are no other local branches, we follow a special workflow as Mercurial’s rebase extension does not do a fast-forward merge by default.  That is, if I tell Hg to rebase or merge SomeTopic, I really just want to move master up to SomeTopic, and not perform some merge.  So I:

• hg checkout SomeTopic
• hg bookmark -f master
• hg bookmark -d SomeTopic
• hg push -b master

I switch back to the SomeTopic branch, and move the master bookmark up to SomeTopic.  I could have done this with “hg bookmark –f master –r SomeTopic”, but I want to switch to SomeTopic instead.  I delete the SomeTopic bookmark, as “master” is now moved up to SomeTopic.  Finally, I push master, and ONLY master up.  I don’t want any other local topic branches to get pushed until they’re integrated with my mainline master branch.  When we’re finished, this is what our local repository looks like:

Scenario 2: Need to work on unrelated items but not push unfinished work

aka, the whole reason for topic branches.  In this case, we’ve created our local topic branch, but now some other work comes in that we need to do.  We need to work on something else unrelated to our feature work, and don’t want to push the feature work until it’s done.  Our local repository first starts out like this:

Just to review what we’re seeing here, the “master” bookmark points to the last pulled commit from upstream.  We’re currently on the TopicOne bookmark, indicated here because it’s orange.  The two up arrows indicate that I have not yet integrated and pushed my TopicOne branch.  You can also execute “hg bookmark” at the command line to view the bookmarks and your current tracked one:

So we’re working on TopicOne, which might represent some feature we’re working on.  Some other work comes up, maybe it’s to fix some CSS or a batch script that has a higher priority than this feature.  But we don’t want to push our TopicOne changes yet, it’s not ready to deploy, the tests are broken, it’s just not finished.  So, we’ll start a new topic by:

• hg checkout master
• hg bookmark TopicTwo

At this point we can start committing as need be:

• work work work
• hg ci –Am “Critical work”
• work work work
• hg ci –Am “More critical work”

Once we’ve done that first commit, Hg will tell you that a new head was created.  This is because we first switched back to master, created a new bookmark, and started committing.  Here’s what our repository looks like right now:

Again, master sits back as our last pulled commit.  It’s the critical placeholder that helps us know where to start new topic branches from.  It’s not required, as I could look at these arrows to know what the last pulled commit was to start from, but it’s a lot easier when doing rebases and merges.

Now that TopicTwo is finished, I want to integrate TopicTwo into master and push it back upstream.  Because master is a direct ancestor of TopicTwo, I only need to follow the Scenario #1 workflow:

• hg co master
• hg pull --rebase
• hg co TopicTwo
• hg bookmark –f master
• hg bookmark –d TopicTwo
• hg push –b master

It’s very important that I only push master, as that lets my local repository now look like:

Now that my critical work is done, I can go back to working on TopicOne:

• hg co TopicOne
• work work work
• hg ci –Am “Finishing work on a feature”

Once I do this, my local repository looks a little changed now:

Here we see the master branch hanging off to the side, and my un-pushed changes in the TopicOne timeline.  But now master is no longer in the ancestry of TopicOne.  Now that I want to integrate TopicOne into master, I can either merge this branch, or rebase it.  I prefer rebase, so I’ll follow the normal rebase workflow.  But first, whenever we’re about to push changes, we ALWAYS:

• hg co master
• hg pull --rebase

Now that we’re sure we have the latest and greatest, we can rebase our TopicOne onto master:

• hg co TopicOne
• hg rebase –b TopicOne –d master

We switch to the TopicOne branch, then rebase from the base of TopicOne to the destination of master.  This command replays the commits from TopicOne onto master, then deletes the TopicOne commits.  Because the timeline changes, these are entirely new commits with new hashes, but containing the exact same changes/commit messages/commit times:

Even though I committed two of my changes before the original TopicTwo branch, after a rebase, these commits show up after the TopicTwo.  This is because a rebase replays the commits one at a time on top of the destination (master).  At this point, I can run the build to make sure everything works, and then follow the normal workflow when master is a direct ancestor of my topic branch, skipping the steps of pulling (we already did that):

• hg bookmark –f master
• hg bookmark –d TopicOne
• hg push –b master

Finally, here’s what my repository looks like:

Because I’ve always rebased, no one ever needs to know about my topic branches until I integrate.  The pushed timeline is always a clean, linear progression for the mainline master branch (in this case, it’s “dev” as the actual Hg branch).  With topic branches, each topic is independent of each other, and I decide when that topic is ready to be integrated into the mainline.  I might never integrate back, and switching topic branches is a VERY VERY fast “hg checkout” command away.  This workflow is fast, cheap, flexible, and allows me to have one working directory and one repository that contains all the work I’m doing, no matter what its state.

When working with other people, it’s inevitable that file conflicts arise.  In the next post, I’ll dive in to how to deal with conflicts along each step of the way and how this workflow functions in a team environment.

In my recent adventures with massive bulk processing, there are some times when I want to pull bulk loaded tables from SQL, but don’t want to go through all the trouble of building a mapping in NHibernate.  For example, one recent project had an intermediate processing table of something like:

This table is used in a bulk copy scenario, so it’s very string-based to ease the burden of bulk loading.  Later, we’ll transactionally process this table to update our actual customer table.  In the meantime, we want to use this data in a .NET application.  We have a few options:

• Load into a DataSet
• Stream from an IDataReader
• Map using NHibernate
• Ad-hoc map using NHibernate

Many times, I like to go with the last option.  DataSets and data readers can be a pain to deal with, as most of the code I write has nothing to do with dealing with the data, but just getting it out in a sane format.

NHibernate supports transformers, which are used to transform the results of the query into something useful.  To make things easy, I’ll create a simple representation of this table:

public class BulkLoadCustomer
{
    public string CustomerId { get; set; }
    public string RegisteredDate { get; set; }
}

I’ll create some generic query:

var sql = "SELECT CustomerId, RegisteredDate FROM BulkLoad.Customer";

var sqlQuery = _unitOfWork.CurrentSession.CreateSQLQuery(sql);

I just have the NHibernate ISession object exposed through a Unit of Work pattern.  With the ISqlQuery object that gets created with the CreateSQLQuery() method, I can then specify that I want the results projected into my custom DTO:

var results = sqlQuery
    .SetResultTransformer(Transformers.AliasToBean(typeof(BulkLoadCustomer)))
    .List<BulkLoadCustomer>();

The AliasToBean method is a factory method on the static Transformers class.  I tell NHibernate to build a transformer to my DTO type, and finally use the List() method to execute the results.  I don’t have to specify any additional mapping file, and NHibernate never needs to know about that BulkLoadCustomer type until I build up the query.

The name “AliasToBean” is a relic of the Java Hibernate roots, which is why it didn’t jump out at me at first.  But it’s a great tool to use when you want to just map any table into a DTO, as long as the DTO matches up well to the underlying query results.

On a recent project, much of the application integration is done through bulk, batch processing.  Lots of FTP shuffling, moving files around, and processing large CSV or XML files.  Everything worked great with our normal NHibernate usage, until recently when we had to process historical transactional data.

The basic problem is that we had to calculate historical customer order totals.  Normally, this would be rather easy to do with a SQL-level bulk update.  However, in our case, we had to process the items one-by-one, as each transaction could potentially trigger an event, “reward earned”.  And naturally, the rules for this trigger are much too complex to attempt to do in straight T-SQL.

This meant that we still had to run our historical feed through the domain model.

A normal, daily drop of customer orders is processed fairly easily.  However, historical data over just the past year totaled close to 5 million transactions, each transaction being a single line in a CSV file.

Since a normal daily file would take about 2 hours to process, we simply could not wait the projected time to process all these transactions.  However, with some handy tips from the twitterverse, we were able to speed things up considerably.

Bulk import and export

The first thing we found is that if you have to do bulk, set-based processing, it was important to determine if this data is a bulk import, or bulk process.  Bulk import is extremely quick with tools like SQL Bulk Copy.  If you need to do a bulk import or export, use the right tool.  A bulk import of these rows into a single table takes about 2.5 minutes, versus days and days one transaction at a time.

In another process we needed to bulk load customer data.  The customer data matched fairly closely to our existing customer table, so we crafted a process that basically followed:

• Create table to match shape of CSV file
• Load CSV into table
• Issue single UPDATE statement to target table with the WHERE clause joining to the CSV-imported table

In this manner we were able to very, very quickly import millions of customer records very quickly.  However, if it’s not straight bulk import or export, we have to go through other channels.

Optimizing NHibernate for bulk processing

It was a lot of work tinkering with several different ideas, but ultimately the churn was worth it.  Here’s a few tips I picked up along the way:

Utilize the Unit of Work pattern and explicit transactions

When I first started, all processing was done with implicit transactions, and copious use of the Flush() method on ISession.  This meant that every. single. save. was in its own transaction.  When you look at the number of roundtrips this entailed, our database was just getting completely hammered.

Additionally, an ISession instance was disposed after every write to the database.  This meant that we could not take advantage of any of the first-level-cache support (aka, identity map) inherent in ISession.

Instead, I switched the codebase to use an actual Unit of Work, where I created a class that controlled the begin, commit and rollback of a unit of work:

public interface IUnitOfWork : IDisposable
{
    ISession CurrentSession { get; }
    void Begin();
    void Commit();
    void RollBack();
    bool IsActive { get; }
}

Before, we really had no control or understanding of the lifecycle of the ISession object.  With this pattern, its lifecycle is tied to the IUnitOfWork, allowing me to take advantage of other NHibernate features.

Use MultiCriteria/MultiQuery to batch SELECTs

MultiCriteria and MultiQuery allow you to send multiple SELECTs down the wire.  For each row in our table, we had to issue a SELECT, as processing a single transaction meant I needed to potentially affect the customer record as well as any previous order transaction records.  Doing this one SELECT at a time can be quite chatty, so I batched several together using MultiCriteria.

Just going from 1 at a time to 10 at a time, while insignificant for <100 records, can really add up once you get into the millions.

Use statement batching

In addition to SELECTs, we can also batch together INSERTs and UPDATEs.  In our case, we parameterized the processing of the file to a certain batch size (say, 250).  We then enabled NHibernate’s statement batching in the hibernate.cfg file:

<property name="adonet.batch_size">250</property>

And now instead of one INSERT being sent down the line at a time, we send a whole messload at once.  Profiling showed us that statement batching alone dropped the time by 50%.

NHibernate is very, very smart about knowing when and in what order to save things, so as long as items are persistent, we only really need to commit the transaction for the bulk processing to go through.

Process bulk updates in batches

Finally, once we had a proper Unit of Work implementation in place, we could now process the giant file as if it were many, smaller files.  We split the incoming work into batches of 250, then created a Unit of Work per batch.  This meant that an entire set of 250 was processed in a single transaction, instead of 5 million individual transactions.

Without a proper Unit of Work in place, we would not be able to do this.

Profiling is your friend

Finally, we needed a way to test our processing and the resulting speed improvements.  A simple automated test with stop watches in place let us tinker with the internals and observer the result.  With tools like NHProf, we could also observe what extra fetching we might need to do along the way.  Its suggestions also keyed us in to the various improvements we added along the way.

Wrapping it up

Bottom line is, if you can reduce the operation to a bulk import or export, the SQL tools will be orders of magnitude faster than processing one at a time.  But if you’re forced to go through your domain model and NHibernate, just be aware that your normal NHibernate usage will not scale.  Instead, you’ll need to lean on some of the built-in features that you don’t normally use to really squeeze as much performance as you can.

Back in the day, I used to develop scheduled tasks by writing my own task scheduler and batch execution program.  I don’t think at the time I knew about the Task Scheduler service built in to Windows.  It support just about any scheduling algorithm I could throw at it, outside of building dependent or cascading tasks.

For build automation, I often want to create, stop and start these scheduled tasks.  Luckily, there’s a handy command-line tool to do so: schtasks.exe.  Not only can you administer tasks on your own machine, but other machines as well.  This is perfect for build and deployment automation, where I need to not only copy files, but execute SQL migration scripts, start/stop services, and power down/up scheduled tasks.

The command-line tool lets you do quite a few things:

• Create a task
• Delete a task
• Run a task
• End a task
• Query a task for status/information
• Modify a task

I typically don’t create the tasks, as that really only needs to happen once.  However, it would be fairly trivial for me to do so, and have all of the task setup driven through automation.

One thing I like to do is disable batch jobs before the deployment happens, then turn them all back on.  I don’t want to do this with all the tasks there, so I use NAnt to pull from a text file of the scheduled tasks I’m interested in.  However, I don’t want to disable running tasks, as I want to just let them finish and have the build stop:

<echo message="Disabling batch jobs..." />
<foreach item="Line" in="batchjobs.txt" property="taskName">
    <exec program="schtasks.exe" 
                commandline="/Query /TN ${taskName} /FO TABLE /NH" 
                output="task.txt" />
    <loadfile file="task.txt" property="batchjob.info" />
    <exec program="schtasks.exe" 
                commandline="/CHANGE /TN ${taskName} /DISABLE" 
                if="${string::contains(batchjob.info, 'Ready')}"/>
</foreach>

In this NAnt snippet, I loop through the batch jobs I care about.  I then call the “schtasks.exe”, querying the tasks status by name and outputting the result in the form of a table to a “task.txt” file.  Next, I load that file into a NAnt property.  Finally, I use the “/CHANGE” switch to disable the scheduled task, but ONLY IF its status is “Ready” and not “Running” or something else.

Next, I’ll run through the batch jobs file again, this time querying for any task that hasn’t been disabled.  If there are any, I’ll just fail the build.  I could sleep the build script, and poll until the task completes.

Once the build is done, I’ll enable all the configured scheduled tasks:

<echo message="Enabling batch jobs..." />
<foreach item="Line" in="batchjobs.txt" property="taskName">
    <exec program="schtasks.exe" commandline="/CHANGE /TN ${taskName} /ENABLE" />
</foreach>

Build automation can really cut down on those launch-night headaches and uncertainty.  Having a good command-line tool goes a long way to enabling easy build automation scripts.

On a recent long project, our team’s commitment to continuous improvement led to some interesting results.  Originally, we started with quite a bit of the Scrum ceremony, such as sprint planning meetings, time-boxed iterations, and daily stand-ups.  However, since we practiced “whole team”, the daily stand-ups included around 25 people or so.  As a means of communicating information, these meetings tended to drag quite a bit.

As time went on, the importance of our daily stand-ups waned, to the point where in many cases we stopped doing them altogether.  It wasn’t that we didn’t find value in these meetings, but that instead we found better, more efficient ways to communicate information.

Waiting to communicate

In the daily stand-up, you answer three simple questions:

• What did I accomplish yesterday?
• What will I do today?
• What is preventing me from accomplishing today’s goals?

It took me a while to understand, but a conversation Scott Bellware a couple years back drove a simple point home.  Why are you waiting for a once-a-day meeting to bring up a blocking issue?  Unless this blocking issue happened in the previous 15 minutes before the meeting, how much did you waste by keeping the blocking issue internal?

Scott recommended that each of these questions could be more efficiently answered, and answered in a JIT fashion.  When you need to know the answer, just ask!  When you have a blocking issue, just raise it!  Don’t wait for the next day to raise the flag for help.

Examining the goals

The purposes of a daily stand-up include:

• Share commitment
• Communicate status, progress and plans
• Identify obstacles
• Set direction and focus
• Build a team

The problem I have with a daily stand-up is that no matter how short it is, it still stops everyone’s workflow for 10-20 minutes.  That’s 10-20 minutes of downtime, plus whatever time it takes to prepare, plus whatever time it takes to resume activities started before the meeting.

Often, the standup meeting takes place well after the time I’ve arrived at work.  It can take quite a bit of time to “prime the pump” and work efficiently.  Human beings do not deal well with interruptions and context switching.  While these goals are good, are there different, more efficient ways of dealing with them?

Sharing commitment

When I’ve worked with a strong, motivated team, shared commitment came naturally.  Everyone had a shared goal of moving stories to production in the shortest time possible.  Everything else that did not add value to that process was superfluous to that goal.

When our team did not have shared commitment, there were often larger issues at play.  It could be that the goals were not effectively communicated from management.  It could be that work habits were not in tune.  But often I’ve seen people shy away from the individual commitments that a stand-up brings.  You can say it’s a shared commitment, but that’s tough to do psychologically when the questions answered are centered around “I”.  What did “I” do yesterday.  What will “I” accomplish today, and so on.

Instead, I’ve seen that when a commitment made to the team isn’t met, when a mini-deadline is given, the person is often left in a negative, defensive position when the commitment isn’t met for whatever reason.

Communicate status, progress and plans

This can ALL be accomplished with a single, visible, shared and open story wall.  A story wall is a place where you can visually see where a story is in the overall process.  Ideally, the story wall is physical (or at least very visible to everyone).

All three of these items can be seen from a single story board.  Status is just where the story is in the pipeline.  Progress is how far it is along the journey, along with how long it’s been in the current bucket.  Planned work is simply the stories queued up.

Whenever status changes on a story, we move the story from phase to phase.  Whenever anyone wants to know where a story is, they can just look at the wall.  We initial our stories for who’s working on them, so if anyone wants to know more detailed status, they can go ask the individual person currently working on the story.

Identify obstacles

Again, why wait to bring up a blocking issue?  In our story board, we visually indicated a story was blocked by placing it either in a “blocked” bucket at the bottom, or placing a red Post-It note on it.  It was up to the person with the blocked story to communicate with the correct people, as soon as possible.

Set direction and focus

This could be another one of those cases where management should take care of this one.  The direction is to finish stories.  The focus is to finish the story you’re working on.  If you’re done with a story, pick up a new one.

If the process flow is clearly laid out, there really shouldn’t be a question of direction or focus.  For things like product direction, is this something that’s supposed to be communicated on a daily basis?

Build a team

I do admit, daily stand-ups do help build a team.  But for us, it was the shared conversation of how little value we were getting out of our stand-ups any more, besides just observing patterns in other’s daily reports.

I’m just not sure it’s worth stopping the entire line for 10-20 minutes every day to achieve this goal.  There are much more targeted ways for building a team other than a daily stand-up.  Like office Nerf wars, for example.

Achieving the goals

Daily stand-ups are a good way to meet the goals of a daily stand-up.  But that doesn’t mean it’s the only way.  As with any process, it’s important to constantly re-evaluate what the goals and values are, and how they relate back to the ultimate goal of the project.  If the goals are important, we should try to work to the most efficient means of achieving those goals.

For us, moving towards a lean-oriented approach, with a pull-based system, big visible story board, and keeping information visible to all achieved these goals in a more efficient manner.  That’s not to say that daily stand-ups are never a good idea, but I think it’s important to always evaluate the project activities to determine if there are other ways to meet the same goals.  Often, daily stand-ups are the most efficient means to achieve these goals.  In many other cases, it’s not.

As a quick review, a test fixture has inputs and outputs.  We set up our test fixture by configuring inputs.  We observe the results through the fixture’s outputs.

Inputs and outputs can be grouped into direct and indirect variants.  Direct inputs include:

• Constructor arguments
• Method arguments
• Property/field values

Indirect inputs are things we can’t directly set on our fixture.  An example would be a ShippingCalculatorService, that returns a shipping cost.  An order processor might use this service to calculate the full cost of an order.  However, we don’t directly set this shipping cost through the direct use of our order processor.  Instead, the shipping cost is an indirect input to our method through this shipping calculator.

On the other side of the coin are direct outputs, which include:

• Return values
• Mutated inputs
• Mutated fixture

Often, we just look at the return value of a method for the direct output.  But we might also look at one of the inputs, which might have mutated as the result of an operation.  We also have indirect outputs, which again are services whose methods are void.

If we properly follow Command-Query Separation, we can group our Rhino Mock usage into only two buckets:

• Stubbing indirect inputs
• Capturing indirect outputs

The first case is easy.  The second one can get ugly, as it requires the use of rather funky looking call.  Let’s say we have an order processor that reserves shipping spots through a service:

public interface IShippingReservationService
{
    void Reserve(int orderId, decimal totalWeight);
}

If we want to capture ALL arguments made and want to assert things, we must do something like:

IList<object[]> argsMade = shipper.GetArgumentsForCallsMadeOn(
    s => s.Reserve(0, 0m), 
    opt => opt.IgnoreArguments());

argsMade[0][0].ShouldEqual(batch1.Id);

Not too pretty.  I have to make sure I use the “IgnoreArguments” configuration option, so that I get all calls.  Next, I get an IList<object[]>, which is basically a jagged array of objects.  Everything is an object, so I have to cast if I want to do any special assertions.

But if we’re in C# 4.0, we can do better.

Capturing one argument-methods

I’d really like to capture all the arguments made as a single list of items. First, let’s look at the simple case where I have an indirect output with only ONE argument:

public interface IOrderProcessor
{
    void Process(OrderBatch batch);
}

In that case, I only want to capture the arguments made as a list of OrderBatch.  No real need to wrap that with anything else at this point.  To do this, I first set up my objects as I normally would with Rhino Mocks:

var processor = Stub<IOrderProcessor>();
var shipper = Stub<IShippingReservationService>();

var batchProcessor = new OrderBatchProcessor(processor, shipper);

The Stub method merely wraps MockRepository.GenerateStub().  From here, I want to then capture the arguments made.  In previous versions, I would do this by passing in a closure for the Do() method of Rhino Mocks.  I can extend this to a general case:

IList<OrderBatch> args = processor
    .Capture()
    .Args<OrderBatch>((p, batch) => p.Process(batch));

batchProcessor.ProcessBatches(new[]
{
    batch1, batch2, batch3
});

args.Count().ShouldEqual(3);
args.First().ShouldEqual(batch2);

I capture the arguments made as a list of OrderBatch, then call the ProcessBatches as normal.  I have some logic where express batches are processed first, which is why I assert that “batch2” came first.

The CaptureMethod is an extension method that begins a chain for me to start capturing arguments:

public static class MockExtensions
{
    public static CaptureExpression<T> Capture<T>(this T stub) 
        where T : class
    {
        return new CaptureExpression<T>(stub);
    }
}

I have to return a CaptureExpression<T> as a trick so that I don’t have to specify the stub’s type in my test.  My CaptureExpression<T> then lets me capture the args:

public class CaptureExpression<T>
    where T : class
{
    private readonly T _stub;

    public CaptureExpression(T stub)
    {
        _stub = stub;
    }

    public IList<U> Args<U>(Action<T, U> methodExpression)
    {
        var argsCaptured = new List<U>();

        Action<U> captureArg = argsCaptured.Add;
        Action<T> stubArg = stub => methodExpression(stub, default(U));

        _stub.Stub(stubArg).IgnoreArguments().Do(captureArg);

        return argsCaptured;
    }

In the Args method, I accept an expression detailing the method I want to call.  I create a list of the arguments, which is where I’ll stash them as they come in.  In the next couple of lines, I build the delegates that Rhino Mocks uses to both stub a method, and provide a stand-in callback.

Finally, I use the Stub() and Do() methods from Rhino Mocks to provide a replacement closure for calling the original method.  When the stubbed method is called, Rhino Mocks passes control to my CaptureArg method.  This method is a closure that adds the method argument to the “argsCaptured” list.

Initially, the list returned is empty.  But as the stub is used in the fixture, this list will be populated with the items used.

That’s the easy case of a single argument, let’s look at multiple arguments.

Capturing multiple arguments with tuples

I’d also like group the arguments made from each call into a single object, instead of just an array that I have to then poke around in.  In earlier versions of C#, I would need to craft an object to hold these values.  In C# 4.0, I have the Tuple classes.

I’ll follow the same pattern here as the method above, except this time use the Tuple class to group the method arguments together:

public IList<Tuple<U1, U2>> Args<U1, U2>(Action<T, U1, U2> methodExpression)
{
    var argsCaptured = new List<Tuple<U1, U2>>();

    Action<U1, U2> captureArg = (u1, u2) => argsCaptured.Add(Tuple.Create(u1, u2));
    Action<T> stubArg = stub => methodExpression(stub, default(U1), default(U2));

    _stub.Stub(stubArg).IgnoreArguments().Do(captureArg);

    return argsCaptured;
}

I return a Tuple<U1, U2>, which are the types of the method parameters of the stubbed method, but now just grouped together in a strongly-typed bucket.  I can now create strongly-typed assertions about my indirect outputs:

var processor = Stub<IOrderProcessor>();
var shipper = Stub<IShippingReservationService>();

var args = shipper
    .Capture()
    .Args<int, decimal>((s, orderId, weight) => s.Reserve(orderId, weight));

var batchProcessor = new OrderBatchProcessor(processor, shipper);

batchProcessor.ProcessBatches(new[]
{
    batch1, batch2
});

args.Count.ShouldEqual(2);
args[0].Item1.ShouldEqual(batch1.Id);
args[0].Item2.ShouldEqual(batch1.TotalWeight);
args[1].Item1.ShouldEqual(batch2.Id);
args[1].Item2.ShouldEqual(batch2.TotalWeight);

Additionally, because I don’t have to guess at the number of arguments made, the Tuple returned is linked to the number of arguments to the stubbed method.  This provides a much stronger link between the arguments I capture and the method being stubbed.

From here, it’s trivial to extend this approach to as many arguments as I need.  And as long as I stick to CQS, and my methods either do something or answer a question, these are the only mocking requirements I’ll need.  As always, you can find this example on my github.

It took me a while to really settle in to a Git workflow I like to use on a daily basis.  It’s a pretty common workflow, and is centered around local topic branches and rebasing.  It’s not actually much different than the workflow I used with SVN, except that I prefer rebase over merge.  My typical Git workflow starts out with:

• git checkout –b “SomeTopic”
• <work work>
• git add .
• git commit –am “Commit message”
• <repeat last 2 steps as necessary>
• git checkout master
• git pull --rebase origin master
• git checkout SomeTopic
• git rebase master
• git checkout master
• git rebase SomeTopic (or merge, same thing)
• git push origin master
• git branch –d SomeTopic

In a nutshell, all work, and I mean ALL WORK, is done in a local branch.  Every time.  This is because I can never predict when other, unrelated work might come up.  I do work in a local branch.  When I want to bring that work back to master (basically, trunk), I first do a pull from origin back to master, rebasing my existing commits.

At this point, I should note that I rarely, rarely ever need to rebase upstream changes.  If I pull work back to master, that means that I’m about to push.  Otherwise, it can just stay in a local topic branch.

Anyway, I make sure that my master reflects the absolute latest upstream changes.  When I’m ready to bring the local branch back in, INSTEAD OF MERGING, I rebase the branch on to master.  All this means in practice is that the branch’s commits are re-played onto the master branch.  Merging instead squashes all the branch’s work in to one single merge commit, which I’d rather avoid.

Side note – The git “rebase considered harmful” article is 3 years old.  A lot of opinions have changed since then, so while its core arguments do apply (never rebase a pushed commit), rebase is a sharp, useful tool that does great things when used right.

Finally, I do a merge/rebase from master to the branch, which really just does a fast-forward merge.  Because the SomeTopic pointer is a descendant of the master pointer, a merge/rebase is really just moving the master pointer up to the SomeTopic pointer.  Once this is done, I push and I’m finished.

Translating this to Hg has been a little more difficult, however.

Combining Rebase and Bookmarks

Right now, I’m trying to use Bookmarks and Rebase to achieve this same workflow.  The basic workflow I want is:

• All work is always isolated from any other work
• Pulling latest does not affect isolated work
• Isolated work, when rolled back in, has linear history preserved

I don’t really care how this is accomplished.  So, I’m going from this article on the different ways of doing branching in Mercurial.

So my first try was using the Hg extensions Rebase and Bookmarks, which have both been included with Mercurial for a while now.  On the surface, Rebase and Bookmarks seem very similar to Git’s rebase and branching model.

Mimicking git, I try:

• hg bookmark SomeTopic
• <work work>
• hg commit –Am “Some message”

Now I want to pull those changes back to master…but wait, there is no master!  I don’t have a pointer to when I first made the branch, and even worse, the “default” branch is now sitting on my SomeTopic branch.

What I’d really like to do is do “hg checkout default”, then do an update, so that “default” always represents the upstream current state.  But “default” has moved!

My next attempt was to create a “master” bookmark right before I created the “SomeTopic” bookmark, so that “master” mimics the Git master branch – that is, it’s merely a named pointer, nothing special.

I now want to do some more, unrelated work, so I:

• hg update master
• hg bookmark SomeOtherTopic
• <work work>
• hg commit –Am “Some other message”

At this point, here’s my tree:

As we can see here, I have my master bookmark marking where I diverged my bookmarks.  This now more or less matches what I see in Git, with the exception of that “default” branch.

Let’s say that we now want to bring “SomeTopic” back to “master”.  Really, I want to rebase “SomeTopic” on to “master”.  Because “master” is a parent of SomeTopic, this should really just be a fast-forward merge.  The master should just be moved up to SomeTopic.

If there were upstream changes, that would change the story a bit.  Master would move up to those upstream changes, and all changes from where SomeTopic and master diverged would be replayed on top of master.  One thing to note is that Git is very smart about fast-forward merges.  If I rebase SomeTopic on to master, and master is still where it was, nothing would happen.  I could then rebase master on to SomeTopic (or merge), and master would just move up.

In the picture above, I really just want “master” to move up to “SomeTopic”, then I can push “master” up.  So let’s try to do an hg rebase:

• hg rebase –b SomeTopic –d master

I want to rebase SomeTopic on top of master.  I get a message “nothing to rebase”.  That’s fine, as I have nothing to do here anyway.  “master” is in the direct ancestry of “SomeTopic”.

The next thing I want to do is move master to SomeTopic, which at this point should just be a fast-forward merge.  But nothing I seem to do will allow me to move “master” up to “SomeTopic”.  I try all of these:

• hg update master/hg merge –r SomeTopic <- “nothing to merge”
• hg rebase –b master –d SomeTopic <- “nothing to rebase”

Blarg.  Even though I’ve configured my bookmark to automatically move forward, there doesn’t seem to be a way to do this myself.  What I can do is:

• hg bookmark –d master
• hg update SomeTopic
• hg bookmark master
• hg bookmark –d SomeTopic

This basically fast-foward merges the master bookmark to SomeTopic, by…deleting it and re-creating it.  If Mercurial supported a fast-forward merge here, that would be GREAT, but it doesn’t, so I have to jump through a bunch of hoops here.  All of which I could batch up in to a “fast-forward” alias, but is still annoying, as Git just handles this automatically.

Anyway, this is now the state of things:

And now I want to push “master” up.  But this will be interesting, I don’t want “SomeOtherTopic” to go out.  So, I use:

• hg push –b master

Now only the “master” piece got pushed up.  Let’s say we now want to get the SomeOtherTopic back in to the fold, AND, that there were upstream changes.  In this case, we want to update our master to be include new changes, but without affecting “SomeOtherTopic”.  We do this to update our master:

• hg update master
• hg pull --rebase –b default

This makes our local repository tree now:

Exactly what we wanted! The “master” bookmark got moved up, past “SomeOtherTopic”.  Now, we just need to rebase SomeOtherTopic onto master:

• hg rebase –b SomeOtherTopic –d master

This means I’m replaying the SomeOtherTopic on top of master, resulting in the following tree:

Looking good!  I now just follow the FF-merge-master-to-branch routine:

• hg update SomeOtherTopic
• hg bookmark –f master <- basically moves master to current location, better than delete/add
• hg bookmark –d SomeOtherTopic <- delete the bookmark (the work is integrated now)
• hg push –b master

Well, that’s it.  It’s not completely like git branching, there are some caveats here and there, as git handles remotes different than Hg.  Git also automatically handles fast-forward merges, but in practice, I don’t think that it’ll be a big deal.

I need to run with this with a team to really make sure it doesn’t corrupt things, but it seems to work so far.

Other posts in this series:

• Controllers
• Contextual controller injection
• Filters
• Action Results
• Views

In this series, we’ve looked on how we can go beyond the normal entry point for dependency injection in ASP.NET MVC (controllers), to achieve some very powerful results by extending DI to filters, action results and views.  We also looked at using the modern IoC container feature of nested containers to provide injection of contextual items related to both the controller and the view.

In my experience, these types of techniques prove to be invaluable over and over again.  However, not every framework I use is built for dependency injection through and through.  That doesn’t stop me from getting it to work, in as many places as possible.

Why?

It’s really amazing how much your design changes once you remove the responsibility of locating dependencies from components.  But instead of just talking about it, let’s take a closer look at the alternatives, from the ASP.NET MVC 2 source code itself.

ViewResult: Static Gateways

One of my big beefs with the design of the ActionResult concept is that there are two very distinct responsibilities going on here:

• Allow an action method to describe WHAT to do
• The behavior of HOW to do it

Controller actions are testable because of the ActionResult concept.  I can return a ViewResult from a controller action method, and simply test its property values.  Is the right view name chosen, etc.

The difficulty comes in to play when it becomes harder to understand what is needed for the HOW versus the pieces describing the WHAT.  From looking at this picture, can you tell me which is which?

Offhand, I have no idea.  The ViewName member I’m familiar with, but what about MasterName in the ViewResult class?  Then you have a “FindView” method, which seems like a rather important method.  The other pieces are all mutable, that is, read and write.  Poring over the source code, none of these describe the WHAT, that’s just the ViewName and MasterName.  Those are the pieces the ViewEngineCollection uses to find a view.

Then you have the View property which can EITHER be set in a controller action, or is dynamically found by looking at the ViewEngineCollection.  So let’s look at that property on the ViewResultBase class:

[SuppressMessage("Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly",
    Justification = "This entire type is meant to be mutable.")]
public ViewEngineCollection ViewEngineCollection {
    get {
        return _viewEngineCollection ?? ViewEngines.Engines;
    }
    set {
        _viewEngineCollection = value;
    }
}

Notice the static gateway piece.  I’m either returning the internal field, OR, if that’s null, a static gateway to a “well-known” location.  Well-known, that is, if you look at the source code of MVC, because otherwise, you won’t really know where this value can come from.

So why this dual behavior?

Let’s see where the setter is used:

The setter is used only in tests.  All this extra code to expose a property, this null coalescing behavior for the static gateway (referenced in 7 other places), all just for testability.  Testing is supposed to IMPROVE design, not make it more complicated and confusing!

I see this quite a lot in the MVC codebase.  Dual responsibilities, exposition of external services through lazy-loaded properties and static gateways.  It’s a lot of code to support this role of exposing things JUST for testability’s sake, but is not actually used under the normal execution flow.

So what you wind up happening is a single class that can’t really make up its mind on what story it’s telling me.  I see a bunch of things that seem to help it find a view (ViewName, MasterName), as well as the ability to just supply a view directly (the View property).  I also see exposing through properties things I shouldn’t set in a controller action.  I can swap out the entire ViewEngineCollection for something else, but really, is that what I would ever want to do?  You have pieces exposed at several different conceptual levels, without a very clear idea on how the end result will turn out.

How can we make this different?

Separating the concerns

First, let’s separate the concepts of “I want to display a view, and HERE IT IS” versus “I want to display a view, and here’s its name”.  There are also some redundant pieces that tend to muddy the waters.  If we look at the actual work being done to render a view, the amount of information actually needed becomes quite small:

public class NamedViewResult : IActionMethodResult
{
    public string ViewName { get; private set; }
    public string MasterName { get; private set; }

    public NamedViewResult(string viewName, string masterName)
    {
        ViewName = viewName;
        MasterName = masterName;
    }
}

public class ExplicitViewResult : IActionMethodResult
{
    public IView View { get; private set; }

    public ExplicitViewResult(IView view)
    {
        View = view;
    }
}

Already we see much smaller, much more targeted classes.  And if I returned one of these from a controller action, there’s absolutely zero ambiguity on what these class’s responsibilities are.  They merely describe what to do, but it’s something else that does the work.  Looking at the invoker that handles this request, we wind up with a signature that now looks something like:

    public class NamedViewResultInvoker : IActionResultInvoker<NamedViewResult>
    {
        private readonly RouteData _routeData;
        private readonly ViewEngineCollection _viewEngines;

        public NamedViewResultInvoker(RouteData routeData, ViewEngineCollection viewEngines)
        {
            _routeData = routeData;
            _viewEngines = viewEngines;
        }

        public void Invoke(NamedViewResult actionMethodResult)
        {
            // Use action method result
            // and the view engines to render a view

Note that we only use the pieces we need to use.  We don’t pass around context objects, whether they’re needed or not.  Instead, we depend only on the pieces actually used.  I’ll leave the implementation alone as-is, since any more improvements would require creation of more fine-grained interfaces.

What’s interesting here is that I can control how the ViewEngineCollection is built, however I need it built.  Because I can now use my container to build up the view engine collection, I can do things like build the list dynamically per request, instead of the singleton manner it is now.  I could of course build a singleton instance, but it’s now my choice.

The other nice side effect here is that my invokers start to have much finer-grained interfaces.  You know exactly what this class’s responsibilities are simply by looking at its interface.  You can see what it does and what it uses.  With broad interfaces like ControllerContext, it’s not entirely clear what is used, and why.

For example, the underlying ViewResultBase only uses a small fraction of ControllerContext object, but how would you know?  Only by investigating the underlying code.

New Design Guidelines

I think a lot of the design issues I run into in my MVC spelunking excursions could be solved by:

1. Close attention to SOLID design
2. Following the Tell, Don’t Ask principle
3. Favoring composition over inheritance
4. Favoring fine-grained interfaces

So why doesn’t everyone just do this?  Because design is hard.  I still have a long ways to go, as my current AutoMapper configuration API rework has shown me.  However, I do feel that careful practice of driving design through behavioral specifications realized in code (AKA, BDD) goes the farthest to achieving good, flexible, clear design.

While driving design with unit tests, I often break behaviors out into separate classes, both to increase cohesion, and as a side effect, increase testability.  Occasionally, I run into situations where I have some sort of environmental variable that never changes.  Or, if it does change, it would take an act of Congress.

When designing these environmental “astronomical constants”, I take the “JFHCI” approach.  These are values that have never changed, and the customer has told us don’t need to change.  One case I ran in to recently is a reward limit.  If you purchase over X dollars, you earn a reward.  The value X does not change, so I made it a constant:

public const decimal GadgetSpendRewardLimit = 200m;

My “testability” hat on says that constants are bad, and that I need to be able to properly set up all environmental variables.  However, in this case, allowing this value to change in ANY way produces a design that does not match reality.  In reality, this reward limit does not need to change.

Instead of going through a bunch of hoops to allow this value to change solely for testing, I’ll leave the value alone.  However, in the off chance that this value DOES change, my tests make the assumption that this value is this constant.

So, I’ve introduced testing assumptions made in my tests with a set of preconditions in the setup portion of the test:

[SetUp]
public void SetUp()
{
    Debug.Assert(Customer.GadgetSpendRewardLimit == 200m, "Assumes threshold is $200");

I use a Debug.Assert mostly just to signify that I don’t need to test this every time, so I don’t really need a regular test assertion.  Instead, this is just a fail-safe, and matches the intent of assumption validation, rather than behavior validation.

With assumption checking through preconditions, I keep the correct design (an immutable constant), while providing explicit callouts of assumptions made about the “correctness” of my tests.  If my assumptions are wrong, I won’t bother trying to continue the test, as the base set of assumptions the test was built around are no longer valid.

A quick way to explain a set of differing opinions is to label it as “a religious argument”.  In a post about using MEF on NerdDinner, Scott Hanselman showed an example on using poor man’s DI versus regular DI.  Now, the post wasn’t about that topic, but more on how to integrate MEF with ASP.NET MVC.  I do get rather annoyed at comments like this however (emphasis mine):

The second constructor takes an IDinnerRepository, allowing us to make different implementations, but the default constructor says, "well, here's a concrete implementation if you don't give one." It's a slippery slope and by adding the default implementation I get to sidestep using dependency injection while making the controller testable, but I've tied my controller down with a direct dependency to the DinnersController. This is sometimes called "Poor Man's IoC" and many would say that this is a very poor man. That's a religious argument, but Hammett takes a stand by removing the default constructor.

I see a religious argument is an argument whose opposite positions aren’t based in facts, but opinions.  It’s reasoning based on assumptions that are grounded in either faith, ignorance or a matter of opinion.

Something like poor man’s DI versus actual DI is different.  Let’s compare the code.  First, poor man’s DI:

public class DinnersController
{
    private readonly IDinnerRepository dinnerRepository;

    public DinnersController() : this(new DinnerRepository())
    {
    }

    public DinnersController(IDinnerRepository repository)
    {
        dinnerRepository = repository;
    }

Now, regular DI:

public class DinnersController
{
    private readonly IDinnerRepository dinnerRepository;

    public DinnersController(IDinnerRepository repository)
    {
        dinnerRepository = repository;
    }

In comparison, the poor man’s DI example:

• Has more code
• Is coupled to a specific implementation
• Decides its component’s lifecycle

This isn’t just in the controller, every component used must use this technique.  Anything that DinnerRepository uses also would have to employ this technique, as we’re using the no-argument constructor for DinnerRepository.

I don’t know about you, but if I can write less code and gain the benefits of looser coupling and externalizing component lifecycle, that’s a win.

Let’s review.

Poor man’s DI: more code, more coupled, no flexibility

Regular DI: less code, less coupled, high flexibility

It was a failure in teaching dependency injection that the argument was made based on testability.  It only helps those already doing TDD to shape design, rather than just writing tests.

Instead, DI is about creating highly flexible components, both in lifecycle and component selection.  DI removes component resolution responsibilities from classes, therefore removing code.  And less code is ALWAYS a good thing.

Other posts in this series:

• Controllers
• Contextual controller injection
• Filters
• Action Results

And now for a bit more controversial shift.  While most folks doing DI in ASP.NET MVC see the benefit of the ability to provide injection around the controller-side of things (filters, action results, controllers etc.), I’ve also seen a lot of benefit from injection on the view side.  But before delve into the how, let’s first look at the why.

The responsibility of a view is to render a model.  Pretty cut and dry, until you start to try and do more interesting things in the view.  Up until now, the CODE extensibility points of a view included:

• XyzHelper extensions (UrlHelper, AjaxHelper, HtmlHelper)
• Custom base view class with custom services

I’m leaving out markup extensibility points such as MVC 2 templated helpers, master pages, partials, render action and so on.  If we want our custom helper extension method to use a service, such as a custom Url resolver, localization services, complex HTML builders and so on, we’re left with service location, looking something like this:

public static MvcHtmlString Text<TModel>(this HtmlHelper<TModel> helper, string key)
{
    var provider = ObjectFactory.GetInstance<ILocalizationProvider>();

    var text = provider.GetValue(key);

    return MvcHtmlString.Create(text);
}

We started seeing this sort of cruft all over the place.  It became clear quite quickly that HtmlHelper extensions are only appropriate for small, procedural bits of code.  But as we started building customized input builders (this was before MVC 2’s templated helpers and MVC Contrib’s input builders), the view started becoming much, much more intelligent about building HTML.  Its responsibilities were still just “build HTML from the model”, but we took advantage of modern OO programming and conventions to drastically reduce the amount of duplication in our views.

But all of this was only possible if we could inject services into the view.  Since MVC isn’t really designed with DI everywhere in mind, we have to use quite a bit of elbow grease to squeeze out the powerful designs we wanted.

Building an injecting view engine

Our overall strategy for injecting services into the view was:

• Create a new base view class layer supertype
• Expose services as read/write properties
• Utilize property injection to build up the view

Since we’re using the WebFormsViewEngine, we don’t really have any control over view instantiation.  We have to use property injection instead.  That’s not a big hurdle for us here as it was in other place.  We’re not instantiating views in unit tests, which are a big source of confusion when doing property injection.

First, we need to subclass the existing view engine and plug in to its existing behavior:

public class NestedContainerViewEngine : WebFormViewEngine
{
    public override ViewEngineResult FindView(
        ControllerContext controllerContext, 
        string viewName, string masterName, bool useCache)
    {
        var result = base.FindView(controllerContext, viewName, masterName, useCache);

        return CreateNestedView(result, controllerContext);
    }

    public override ViewEngineResult FindPartialView(
        ControllerContext controllerContext, 
        string partialViewName, bool useCache)
    {
        var result = base.FindPartialView(controllerContext, partialViewName, useCache);

        return CreateNestedView(result, controllerContext);
    }

We’re going to use the base view engine to do all of the heavy lifting for locating views.  When it finds a view, we’ll create our ViewEngineResult based on that.  The CreateNestedView method becomes:

private ViewEngineResult CreateNestedView(
    ViewEngineResult result, 
    ControllerContext controllerContext)
{
    if (result.View == null)
        return result;

    var parentContainer = controllerContext.HttpContext.GetContainer();

    var nestedContainer = parentContainer.GetNestedContainer();

    var webFormView = (WebFormView)result.View;

    var wrappedView = new WrappedView(webFormView, nestedContainer);

    var newResult = new ViewEngineResult(wrappedView, this);

    return newResult;
}

We want to create a nested container based on the calling controller’s nested container.  Our previous controller factory used a static gateway to store the outermost nested container in HttpContext.Items.  To make it visible to our view engine (which is SINGLETON), we have no choice but to build a little extension method in GetNestedContainer for HttpContextBase to retrieve our nested container.

Once we have the outermost nested container, we create a new, child nested container from it.  Containers can nest as many times as we like, inheriting the parent container configuration.

From there, we then need to build up our own IView instance, a WrappedView object.  Unfortunately, the extension points in the WebFormView class do not exist for us to seamlessly extend it to provide injection.  However, since MVC is open source, we have a great starting point.

After we build our WrappedView, we create our ViewEngineResult and our custom view engine is complete.  Before we look at the WrappedView class, let’s look at how our views will be built.

Layer supertype to provide injection

To provide injection of services, we’ll need a layer supertype between our actual views and the normal MVC ViewPage and ViewPage<T>:

public abstract class ViewPageBase<TModel> : ViewPage<TModel>
{
    public IHtmlBuilder<TModel> HtmlBuilder { get; set; }
}

public abstract class ViewPageBase : ViewPageBase<object>
{
}

Here, we include our custom IHtmlBuilder service that will do all sorts of complex HTML building.  We can include any other service we please, but we just need to make sure that it’s a mutable property on our base view layer supertype.  The HtmlBuilder implementation does nothing interesting, but includes a set of services we want to have injected:

public class HtmlBuilder<TModel> : IHtmlBuilder<TModel>
{
    private readonly HtmlHelper<TModel> _htmlHelper;
    private readonly AjaxHelper<TModel> _ajaxHelper;
    private readonly UrlHelper _urlHelper;

    public HtmlBuilder(
        HtmlHelper<TModel> htmlHelper, 
        AjaxHelper<TModel> ajaxHelper, 
        UrlHelper urlHelper)
    {
        _htmlHelper = htmlHelper;
        _ajaxHelper = ajaxHelper;
        _urlHelper = urlHelper;
    }

When we configure our container, we want any service used to be available.  By configuring the various helpers, we allow our helpers to be injected instead of passed around everywhere in method arguments.  This is MUCH MUCH cleaner than passing context objects around everywhere we go, regardless if they’re actually used or not.

Wrapping WebFormView to provide injection

As I mentioned before, we can’t subclass WebFormView directly, but we can instead wrap its behavior with our own.  Composition over inheritance, but we still have to duplicate more behavior than I would have liked.  But, it’s about the cleanest and lowest-impact implementation I’ve come up with, and gets around any kind of sub-optimal poor-man’s dependency injection.

First, our WrappedView definition:

public class WrappedView : IView, IDisposable
{
    private bool _disposed;

    public WrappedView(WebFormView baseView, IContainer container)
    {
        BaseView = baseView;
        Container = container;
    }

    public WebFormView BaseView { get; private set; }
    public IContainer Container { get; private set; }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    protected virtual void Dispose(bool disposing)
    {
        if (_disposed)
            return;

        if (Container != null)
            Container.Dispose();

        _disposed = true;
    }

We accept the base view (a WebFormView created from the original WebFormsViewEngine), as well as our nested container.  We need to dispose of our container properly, so we implement IDisposable properly.

Now, the next part is large, but only because I had to duplicate the existing MVC code to add in what I needed:

public void Render(ViewContext viewContext, TextWriter writer)
{
    if (viewContext == null)
    {
        throw new ArgumentNullException("viewContext");
    }

    object viewInstance = BuildManager.CreateInstanceFromVirtualPath(BaseView.ViewPath, typeof(object));
    if (viewInstance == null)
    {
        throw new InvalidOperationException(
            String.Format(
                CultureInfo.CurrentUICulture,
                "The view found at '{0}' was not created.",
                BaseView.ViewPath));
    }

    ////////////////////////////////
    // This is where our code starts
    ////////////////////////////////
    var viewType = viewInstance.GetType();
    var isBaseViewPage = viewType.Closes(typeof (ViewPageBase<>));

    Container.Configure(cfg =>
    {
        cfg.For<ViewContext>().Use(viewContext);
        cfg.For<IViewDataContainer>().Use((IViewDataContainer) viewInstance);

        if (isBaseViewPage)
        {
            var modelType = GetModelType(viewType);
            var builderType = typeof (IHtmlBuilder<>).MakeGenericType(modelType);
            var concreteBuilderType = typeof (HtmlBuilder<>).MakeGenericType(modelType);

            cfg.For(builderType).Use(concreteBuilderType);
        }
    });

    Container.BuildUp(viewInstance);
    ////////////////////////////////
    // This is where our code ends
    ////////////////////////////////

    var viewPage = viewInstance as ViewPage;
    if (viewPage != null)
    {
        RenderViewPage(viewContext, viewPage);
        return;
    }

    ViewUserControl viewUserControl = viewInstance as ViewUserControl;
    if (viewUserControl != null)
    {
        RenderViewUserControl(viewContext, viewUserControl);
        return;
    }

    throw new InvalidOperationException(
        String.Format(
            CultureInfo.CurrentUICulture,
            "The view at '{0}' must derive from ViewPage, ViewPage<TViewData>, ViewUserControl, or ViewUserControl<TViewData>.",
            BaseView.ViewPath));
}

I’m going to ignore the other pieces except what’s between those comment blocks.  We have our ViewPageBase<TModel>, and we need to configure various services for our views, including:

• ViewContext
• Anything needed by the helpers (IViewDataContianer)
• Custom services, like IHtmlBuilder<TModel>

Just like our previous nested container usage, we take advantage of StructureMap’s ability to configure a container AFTER it’s been created.  We first configure ViewContext and IViewDataContainer (needed for HtmlHelper).  Finally, we determine the TModel model type of our view, and configure IHtmlBuilder<TModel> against HtmlBuilder<TModel>.  If TModel is of type Foo, we configure IHtmlBuilder<Foo> to use HtmlBuilder<Foo>.

Finally, we use the BuildUp method to perform property injection.  Just as we explicitly configured property injection for our filter’s services, we need to do the same for the view’s services:

SetAllProperties(c =>
{
    c.OfType<IActionInvoker>();
    c.OfType<ITempDataProvider>();
    c.WithAnyTypeFromNamespaceContainingType<ViewPageBase>();
    c.WithAnyTypeFromNamespaceContainingType<LogErrorAttribute>();
});

The view services are all contained in the namespace of the ViewPageBase class.  With that in place, we just have one more piece to deal with: services in master pages.

Dealing with master pages

In the RenderViewPage method, we add a piece to deal with master pages and enable injection for them as well:

private void RenderViewPage(ViewContext context, ViewPage page)
{
    if (!String.IsNullOrEmpty(BaseView.MasterPath))
    {
        page.MasterLocation = BaseView.MasterPath;
    }

    page.ViewData = context.ViewData;

    page.PreLoad += (sender, e) => BuildUpMasterPage(page.Master);

    page.RenderView(context);
}

Because master pages do not flow through the normal view engine, we have to hook in to their PreLoad event to do our property injection in a BuildUpMasterPage method:

private void BuildUpMasterPage(MasterPage master)
{
    if (master == null)
        return;

    var masterContainer = Container.GetNestedContainer();

    masterContainer.BuildUp(master);

    BuildUpMasterPage(master.Master);
}

If we needed any custom configuration for master pages, this is where we could do it.  In my example, I don’t, so I just create a new default nested container from the parent container.  Also, master pages can have nesting, so we recursively build up all of the master pages in the hierarchy until we run out of parent master pages.

Finally, we need to hook up our custom view engine in the global.asax:

protected void Application_Start()
{
    AreaRegistration.RegisterAllAreas();

    RegisterRoutes(RouteTable.Routes);

    StructureMapConfiguration.Initialize();

    var controllerFactory = new StructureMapControllerFactory(ObjectFactory.Container);

    ControllerBuilder.Current.SetControllerFactory(controllerFactory);

    ViewEngines.Engines.Clear();
    ViewEngines.Engines.Add(new NestedContainerViewEngine());
}

And that’s it!  With our nested container view engine in place, we can now inject complex UI services in to our views, allowing us to create powerful UI content builders without resorting to gateways or service location.

Conclusion

It was a bit of work, but we were able to inject services into not only views, but partials, master pages, even MVC 2 templated helpers!  By using nested containers, we were able to configure all of the contextual pieces so that services built for each view got the correct contextual item (the right HtmlHelper, ViewContext, IViewDataContainer, etc.)

This is a quite powerful tool now, we don’t need to resort to ugly usage of static gateways or service location.  We can now build UI services that depend on an HtmlHelper or ViewContext, and feel confident that our services get the correct instance.  In the past, we’d need to pass around our ViewContext EVERYWHERE in order to get back at these values.  Not very fun, especially when you start to see interfaces that accept everything under the sun “just in case”.

For those folks that don’t want to inject services in to their views, it’s all about responsibilities.  I can create encapsulated, cohesive UI services that still only create HTML from a model, but I’m now able to use actual OO programming without less-powerful static gateways or service location to do so.

So looking back, we were able to inject services into our controllers, filters, action results and views.  Using nested containers, we were able to provide contextual injection of all those context objects that MVC loves to use everywhere.  But now we can let our services use them only when needed through dependency injection, providing a much cleaner API throughout.

You can find code for this example on my github:

http://github.com/jbogard/blogexamples