Skip to content

Jimmy Bogard
Syndicate content
Strong opinions, weakly held
Updated: 15 min 50 sec ago

Saga Implementation Patterns: Singleton

Fri, 04/17/2015 - 17:38

NServiceBus sagas are great tools for managing asynchronous business processes. We use them all the time for dealing with long-running transactions, integration, and even places we just want to have a little more control over a process.

Occasionally we have a process where we really only need one instance of that process running at a time. In our case, it was a process to manage periodic updates from an external system. In the past, I’ve used Quartz with NServiceBus to perform job scheduling, but for processes where I want to include a little more information about what’s been processed, I can’t extend the Quartz jobs as easily as NServiceBus saga data. NServiceBus also provides a scheduler for simple jobs but they don’t have persistent data, which for a periodic process you might want to keep.

Regardless of why you’d want only one saga entity around, with a singleton saga you run into the issue of a Start message arriving more than once. You have two options here:

  1. Create a correlation ID that is well known
  2. Force a creation of only one saga at a time

I didn’t really like the first option, since it requires whomever starts to the saga to provide some bogus correlation ID, and never ever change that ID. I don’t like things that I could potentially screw up, so I prefer the second option. First, we create our saga and saga entity:

public class SingletonSaga : Saga<SingletonData>,
    IAmStartedByMessages<StartSingletonSaga>,
    IHandleTimeouts<SagaTimeout>
{
    protected override void ConfigureHowToFindSaga(
    	SagaPropertyMapper<SingletonData> mapper)
    {
    	// no-op
    }

    public void Handle(StartSingletonSaga message)
    {
        if (Data.HasStarted)
        {
            return;
        }

        Data.HasStarted = true;
        
        // Do work like request a timeout
        RequestTimeout(TimeSpan.FromSeconds(30), new SagaTimeout());
    }
    
    public void Timeout(SagaTimeout state)
    {
    	// Send message or whatever work
    }
}

Our saga entity has a property “HasStarted” that’s just used to track that we’ve already started. Our process in this case is a periodic timeout and we don’t want two sets of timeouts going. We leave the message/saga correlation piece empty, as we’re going to force NServiceBus to only ever create one saga:

public class SingletonSagaFinder
    : IFindSagas<SingletonSagaData>.Using<StartSingletonSaga>
{
    public NHibernateStorageContext StorageContext { get; set; }

    public SingletonSagaData FindBy(StartSingletonSaga message)
    {
        return StorageContext.Session
            .QueryOver<SingletonSagaData>()
            .SingleOrDefault();
    }
}

With our custom saga finder we only ever return the one saga entity from persistent storage, or nothing. This combined with our logic for not kicking off any first-time logic in our StartSingletonSaga handler ensures we only ever do the first-time logic once.

That’s it! NServiceBus sagas are handy because of their simplicity and flexibility, and implementing something a singleton saga is just about as simple as it gets.

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

Categories: Blogs

Clean Tests: Database Persistence

Tue, 04/07/2015 - 18:13

Other posts in this series:

A couple of posts ago, I walked through my preferred solution of isolating database state using intelligent database wiping with Respawn. Inside a test, we still need to worry about persisting items.

This is where things can get a bit tricky. We have to worry about transactions, connections, ORMs (maybe), lazy loading, first-level caches and more. When it comes to figuring out which direction to go in setting up a test environment, I tend to default to matching production behavior. Too many times I’ve been burned by bizarre test behavior, only to find my test fixture/environment doesn’t match against any plausible or possible production scenario. It’s one thing to simply and isolate, it’s another to operate in a bizzaro world.

In production environments, I deal with a single unit of work per request, whether that request is a command in a thick client app, a web API call, or a server-side MVC request. The world is built up and torn down on every request, creating a lovely stateless environment.

The kicker is that I often need to deal with ORMs, and barring that, some sort of unit of work mechanism even if it’s a PetaPoco DB object. When I set up state, I want nothing shared between that setup part and the Execute step of my test:

Each of these steps is isolated from the other. With my apps, the Execute step is easy to put inside an isolated unit of work since I’m using MediatR, so I’ll just need to worry about Setup and Verify.

I want something flexible that works with different styles of tests and not have something implicit like a Before/After hook in my tests. It needs to be completely obvious “these things are in a unit of work”. Luckily, I have a good hook to do so with that Fixture object I use to have a central point of my test setup.

Setup

At the setup portion of my tests, I’m generally only saving things. In that case, I can just create a helper method in my test fixture to build up a DbContext (in the case of Entity Framework) and save some things:

public void Txn(Action<MyContext> action)
{
    var dbContext = new MyContext();
    DbContextTransaction txn  = null;
    try
    {
        txn = dbContext.Database.BeginTransaction();
        action(dbContext);
        dbContext.SaveChanges();
        txn.Commit();
    }
    catch (Exception)
    {
        txn?.Rollback();
        throw;
    }
}

We create our context, open a transaction, perform whatever action and commit/rollback our transaction. With this method, we now have a simple way to perform any action in an isolated transaction without our test needing to worry about the semantics of transactions, change tracking and the like. We can create a convenience method to save a set of entities:

public void Save(params object[] entities)
{
    Txn(dbContext =>
    {
        foreach (var entity in entities)
        {
            var entry = dbContext.ChangeTracker
                .Entries()
                .FirstOrDefault(entityEntry => entityEntry.Entity == entity);

            if (entry == null)
            {
                dbContext.Set(entity.GetType()).Add(entity);
            }
        }
    });
}

And finally in our tests:

public InvoiceApprovalTests(Invoice invoice,
    [Fake] IApprovalService mockService,
    IInvoiceApprover invoiceApprover,
    SlowTestFixture fixture)
{
    fixture.Save(invoice);

We still have our entities to be used in our tests, but they’re now detached and isolated from any ORMs. When we get to Verify, we’ll look at reloading these entities. But first, let’s look at Execute.

Execute

As I mentioned earlier, for most of the apps I build today requests are funneled through MediatR. This provides a nice uniform interface and jumping off point for any additional behaviors/extensions. A side benefit are the Execute step in my tests are usually just a Send call (unless it’s unit tests against the domain model directly).

In production, there’s a context set up, a transaction started, request made and sent down to MediatR. Some of these steps, however, are embedded in extension points of the environment, and even if extracted out, they’re started from extension points. Take for example transactions, I hook these up using filters/modules. To use that exact execution path I would need to stand up a dummy server.

That’s a little much, but I can at least do the same things I was doing before. I like to treat the Fixture as the fixture for Execute, and isolate Setup and Verify. If I do this, then I just need a little helper method to send a request and get a response, all inside a transaction:

public TResult Send<TResult>(IRequest<TResult> message)
{
    var context = Container.GetInstance<MyContext>();
    var mediator = Container.GetInstance<IMediator>();
    DbContextTransaction txn = null;
    TResult result;

    try
    {
        txn = context.Database.BeginTransaction();
        result = mediator.Send(message);
        txn.Commit();
    }
    catch (Exception)
    {
        txn?.Rollback();
        throw;
    }

    return result;
}

It looks very similar to the “Txn” method I build earlier, except I’m treating the child container as part of my context and retrieving all items from it including any ORM class. Sending a request like this ensures that when I’m done with Send in my test method, everything is completely done and persisted:

public InvoiceApprovalTests(Invoice invoice, 
    [Fake] IApprovalService mockService,
    SlowTestFixture fixture)
{
    fixture.Save(invoice);

    A.CallTo(() => mockService.CheckApproval(invoice.Id)).Returns(true);

    fixture.Send(new ApproveInvoiceCommand {InvoiceId = invoice.Id});

My class under test now routes through this handler:

public class ApproveInvoiceHandler : RequestHandler<ApproveInvoiceCommand>
{
    private readonly MyContext _context;
    private readonly IInvoiceApprover _invoiceApprover;

    public ApproveInvoiceHandler(MyContext context,
        IInvoiceApprover invoiceApprover)
    {
        _context = context;
        _invoiceApprover = invoiceApprover;
    }

    protected override void HandleCore(ApproveInvoiceCommand message)
    {
        var invoice = _context.Set<Invoice>().Find(message.InvoiceId);

        _invoiceApprover.Approve(invoice);
    }
}

With my Execute built around a uniform interface with reliable, repeatable results, all that’s left is the Verify step.

Verify

Failures around Verify typically arise because I’m verifying against in-memory objects that haven’t been rehydrated. A test might pass or fail because I’m asserting against the result from a method, but in actuality a user makes a POST, something mutates, and a subsequent GET retrieves the new information. I want to reliably recreate this flow in my tests, but not go through all the hoops of making requests. I need to make a fresh request to the database, bypassing any caches, in-memory objects and the like.

One way to do this is to reload an item:

public void Reload<TEntity, TIdentity>(
    ref TEntity entity,
    TIdentity id)
    where TEntity : class
{
    TEntity e = entity;

    Txn(ctx => e = ctx.Set<TEntity>().Find(id));

    entity = e;
}

I pass in an entity I want to reload, and a means to get the item’s ID. Inside a transaction and fresh DbContext, I reload the entity and set it as the ref parameter in my method. In my test, I can then use this reloaded entity as what I assert against:

public InvoiceApprovalTests(Invoice invoice, 
    [Fake] IApprovalService mockService,
    SlowTestFixture fixture)
{
    fixture.Save(invoice);

    var invoiceId = invoice.Id;

    A.CallTo(() => mockService.CheckApproval(invoiceId)).Returns(true);

    fixture.Send(new ApproveInvoiceCommand {InvoiceId = invoiceId});

    fixture.Reload(ref invoice, invoiceId);

    _invoice = invoice;
}

In this case, I tend to prefer the “ref” argument rather than something like “foo = fixture.Reload(foo, foo.Id)”, but I might be in the minority here.

With these patterns in place, I can rest assured that my Setup, Execute and Verify are appropriately isolated and match production usage as much as possible. When my tests match reality, I’m far less likely to get myself in trouble with false positives/negatives and I can have much greater confidence that my tests actually reduce bugs.

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

Categories: Blogs

Clean Tests: Isolation with Fakes

Tue, 03/24/2015 - 17:58

Other posts in this series:

So far in this series, I’ve walked through different modes of isolation – from internal state using child containers and external state with database resets and Respawn. In my tests, I try to avoid fakes/mocks as much as possible. If I can control the state, isolating it, then I’ll leave the real implementations in my tests.

There are some edge cases in which there are dependencies that I can’t control – web services, message queues and so on. For these difficult to isolate dependencies, fakes are acceptable. We’re using AutoFixture to supply our mocks, and child containers to isolate any modifications. It should be fairly straightforward then to forward mocks in our container.

As far as mocking frameworks go, I try to pick the mocking framework with the simplest interface and the least amount of features. More features is more headache, as mocking frameworks go. For me, that would be FakeItEasy.

First, let’s look at a simple scenario of creating a mock and modifying our container.

Manual injection

We’ve got our libraries added, now we just need to add a way to create a fake and inject it into our child container. Since we’ve built an explicit fixture object, this is the perfect place to put our code:

public T Fake<T>()
{
    var fake = A.Fake<T>();

    Container.EjectAllInstancesOf<T>();
    Container.Inject(typeof(T), fake);

    return fake;
}

We create the fake using FakeItEasy, then inject the instance into our child container. Because we might have some existing instances configured, I use “EjectAllInstancesOf” to purge any configured instances. Once we’ve injected our fake, we can now both configure the fake and use our container to build out an instance of a root component. The code we’re trying to test is:

public class InvoiceApprover : IInvoiceApprover
{
    private readonly IApprovalService _approvalService;

    public InvoiceApprover(IApprovalService approvalService)
    {
        _approvalService = approvalService;
    }

    public void Approve(Invoice invoice)
    {
        var canBeApproved = _approvalService.CheckApproval(invoice.Id);

        if (canBeApproved)
        {
            invoice.Approve();
        }
    }
}

In our situation, the approval service is some web service that we can’t control and we’d like to stub that out. Our test now becomes:

public class InvoiceApprovalTests
{
    private readonly Invoice _invoice;

    public InvoiceApprovalTests(Invoice invoice,
        SlowTestFixture fixture)
    {
        _invoice = invoice;

        var mockService = fixture.Fake<IApprovalService>();
        A.CallTo(() => mockService.CheckApproval(invoice.Id)).Returns(true);

        var invoiceApprover = fixture.Container.GetInstance<IInvoiceApprover>();

        invoiceApprover.Approve(invoice);
        fixture.Save(invoice);
    }

    public void ShouldMarkInvoiceApproved()
    {
        _invoice.IsApproved.ShouldBe(true);
    }

    public void ShouldMarkInvoiceLocked()
    {
        _invoice.IsLocked.ShouldBe(true);
    }
}

Instead of using FakeItEasy directly, we go through our fixture instead. Once our fixture creates the fake, we can use the fixture’s child container directly to build out our root component. We configured the child container to use our fake instead of the real web service – but this is encapsulated in our test. We just grab a fake and start going.

The manual injection works fine, but we can also instruct AutoFixture to handle this a little more intelligently.

Automatic injection

We’re trying to get out of creating the fake and root component ourselves – that’s what AutoFixture is supposed to take care of, creating our fixtures. We can instead create an attribute that AutoFixture can key into:

[AttributeUsage(AttributeTargets.Parameter)]
public sealed class FakeAttribute : Attribute { }

Instead of building out the fixture items ourselves, we go back to AutoFixture supplying them, but now with our new Fake attribute:

public InvoiceApprovalTests(Invoice invoice, 
    [Fake] IApprovalService mockService,
    IInvoiceApprover invoiceApprover,
    SlowTestFixture fixture)
{
    _invoice = invoice;

    A.CallTo(() => mockService.CheckApproval(invoice.Id)).Returns(true);

    invoiceApprover.Approve(invoice);
    fixture.Save(invoice);
}

In order to build out our fake instances, we need to create a specimen builder for AutoFixture:

public class FakeBuilder : ISpecimenBuilder
{
    private readonly IContainer _container;

    public FakeBuilder(IContainer container)
    {
        _container = container;
    }

    public object Create(object request, ISpecimenContext context)
    {
        var paramInfo = request as ParameterInfo;

        if (paramInfo == null)
            return new NoSpecimen(request);

        var attr = paramInfo.GetCustomAttribute<FakeAttribute>();

        if (attr == null)
            return new NoSpecimen(request);

        var method = typeof(A)
            .GetMethod("Fake", Type.EmptyTypes)
            .MakeGenericMethod(paramInfo.ParameterType);

        var fake = method.Invoke(null, null);

        _container.Configure(cfg => cfg.For(paramInfo.ParameterType).Use(fake));

        return fake;
    }
}

It’s the same code as inside our context object’s “Fake” method, made a tiny bit more verbose since we’re dealing with type metadata. Finally, we need to register our specimen builder with AutoFixture:

public class SlowTestsCustomization : ICustomization
{
    public void Customize(IFixture fixture)
    {
        var contextFixture = new SlowTestFixture();

        fixture.Register(() => contextFixture);

        fixture.Customizations.Add(new FakeBuilder(contextFixture.Container));
        fixture.Customizations.Add(new ContainerBuilder(contextFixture.Container));
    }
}

We now have two options when building out fakes – manually through our context object, or automatically through AutoFixture. Either way, our fakes are completely isolated from other tests but we still build out our root components we’re testing through our container. Building out through the container forces our test to match what we’d do in production as much as possible. This cuts down on false positives/negatives.

That’s it for this series on clean tests – we looked at isolating internal and external state, using Fixie to build out how we want to structure tests, and AutoFixture to supply our inputs. At one point, I wasn’t too interested in structuring and refactoring test code. But having been on projects with lots of tests, I’ve found that tests retain their value when we put thought into their design, favor composition over inheritance, and try to keep them as tightly focused as possible (just like production code).

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

Categories: Blogs

Clean Tests: Isolating the Database

Mon, 03/02/2015 - 19:35

Other posts in this series:

Isolating the database can be pretty difficult to do, but I’ve settled on a general approach that allows me to ensure my tests are built from a consistent starting point. I prefer a consistent starting point over something like rolled back transactions, since a rolled back transaction assumes that the database is in a consistent state to begin with.

I’m going to use my tool Respawn to build a reliable starting point in my tests, and integrate it into my tests. In my last post, I walked through creating a common fixture in which my tests use to build internal state. I’m going to extend that fixture to also include my Respawn project:

public class SlowTestFixture
{
    private static IContainer Root = IoC.BuildCompositionRoot();
    private static Checkpoint Checkpoint = new Checkpoint
    {
        TablesToIgnore = new[]
        {
            "sysdiagrams",
            "tblUser",
            "tblObjectType",
        },
        SchemasToExclude = new[]
        {
            "RoundhousE"
        }
    };

    public SlowTestFixture()
    {
        Container = Root.CreateChildContainer();
        Checkpoint.Reset("MyConnectionStringName");
    }

Since my SlowTestFixture is used in both styles of organization (fixture per test class/test method), my database will either get reset before my test class is constructed, or before each test method. My tests start with a clean slate, and I never have to worry about my tests failing because of inconsistent state again. The one downside I have is that my tests can’t be run in parallel at this point, but that’s a small price to pay.

That’s pretty much all there is – because I’ve created a common fixture class, it’s easy to add more behavior as necessary. In the next post, I’ll bring all these concepts together with a couple of complete examples.

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

Categories: Blogs