PhatBoyG.com

It’s been a couple of busy weeks! The weekend before my presentations in Arkansas, I decided to forego sleep and go to Wakarusa! The Saturday lineup included two of my favorite electronica acts, Shpongle and Ott, and introduced to another band STS9. Staying up all night dancing to heavy techno music and getting no sleep was one heck of a prelude to the week. After the seriously fun times I stopped in on the Fort Smith and Northwest Arkansas .NET user groups and shared my thoughts on Event Driven Architecture. Both nights went well, with some great conversations afterwards about the how and why that took me towards that style of architectural system design.

Tomorrow morning at 9:00 AM (hey, if you aren’t first, you’re last — this is Texas after all) I’ll be presenting at Dallas TechFest on the same subject in the .NET track. If you want to learn more about Event Driven Architecture and how we are making it easier to implement with MassTransit, come by and check it out. I promise to deliver a demo that includes all sorts of cool things like NHibernate, FluentNHibernate, StructureMap, Castle, Topshelf, Magnum, and of course, MassTransit.

Also, if you’re in the Tulsa area and weren’t able to make it to Dallas TechFest, I’ll be presenting at the Tulsa .NET User Group on the 29th of June as well. If you do come by, be sure to stop afterwards and say hello. I’ll be at the event throughout the day and am more than happy to talk to anyone interested in learning more about MassTransit. I’m sure I’ll be around the functional programming open space playing with crazy monadic parsers written in C#.

In the past year, I’ve been learning about Event Driven Architecture and how using it in the enterprise can make adding functionality over time easier through a loosely-coupled event-based architecture. With Event Driven Architecture, business components can subscribe and react to events, using correlation and orchestration to combine those events to represent business processes. The loose-coupled nature makes it easier to add functionality without modifying existing services — reducing the risk of deploying new services over time.

When I first added saga support to MassTransit, the focus was purely on allowing for long-lived transactions and the orchestration of a series of events that related to a specific process. Earlier this year, the first release of a new declarative programming style for sagas using a fluent interface was put into the trunk, making it possible to design sagas using a more natural structure and language. This feature alone is one of the more compelling things about sagas in MassTransit.

Over the past month or so, I’ve been working on extending the capabilities of sagas in MassTransit beyond the orchestration of directly correlated events (represented as messages). With the latest updates to the trunk, messages that are not directly correlated to the saga can now be observed by the saga using either the interface-based or declarative state machine based saga syntax. This makes for some very compelling functionality as sagas can now observe other events and react to them!

Let’s take a quick look at the syntax using the state machine syntax using a simple message indicating a user has signed out.

public class UserSignedOut
{
	public string Username { get; set; }
}

This message only contains the username of the user that signed out of the system. If we had a saga that tracked the state of an operation on a web site based on the current user, we could observe this message and release any resources that were being tracked by the saga. For example:

public class ShoppingCartSaga :
	SagaStateMachine< ShoppingCartSaga >
{
	static ShoppingCartSaga()
	{
		Define(()=>
		{
			Correlate(UserSignedOut).By((saga,message) => saga.Username == message.Username);

			During(Active,
				When(UserSignedOut)
					.Then(saga => saga.ReturnCartItemsToInventory())
					.Complete());
		};
	}

	public static Event< UserSignedOut > UserSignedOut { get; set; }
}

The Correlate() method is used to define the correlation expression used to match a message to one or more sagas in the saga repository. With this addition, the ISagaRepository has been changed to make it more integrated into the message pipeline, along with the new pipeline message sinks to support this enhanced functionality. The pipeline viewer has also been updated to show the contents of the message pipeline, including all the messages handled by the saga. The Correlate() method can also be used to change how a CorrelatedBy message is also correlated to the saga, making it possible to override the logic on how messages are used to create/use existing saga instances.

The first use case for this new functionality was to improve the subscription service. By making Subscription and SubscriptionClient sagas observe messages from each other, the sagas are able to now complete when they see a node coming back online after a crash or going offline, etc. Since the saga repositories themselves are used to keep track of all the clients and subscriptions in the system, putting that responsibility within the declaration of the saga state machine itself makes for a really clean implementation.

These bits are currently in the trunk and will be part of the next release drop of MassTransit. I encourage you to check them out and provide some feedback on any issues. Some of this may be extended soon to also allow for the custom definition of the policy associated with how messages should be applied to sagas, such as always creating a new instance or using an existing instance of a saga. Some conventions are currently in place based on the Initial state that select a default saga policy (check out ISagaPolicy and the classes that implement it for more details there).

So check it out and keep the great feedback and comments coming on the message group. The new syntax additions should be added to the wiki soon.

One of the really cool features that is available in the 0.6 release of MassTransit is the ability to declaratively define a saga/workflow using a nested-closure syntax in combination with a fluent builder. While some wonder if fluent interfaces are going to become the “aluminum wiring” of the current generation of software development, I find them incredible valuable for building an expressive representation of logic.

A saga is a way of defining a long-lived transaction, typically involving multiple transactional actors. A workflow is a way of orchestrating a number of actors into a process. Both sagas and workflows can be modeled using the new declarative syntax. First, let’s look at what this new language looks like in code:

static DrinkPreparationSaga()
{
	Define(() =>
		{
			Initially(
				When(NewOrder)
					.Then((saga, message) => saga.ProcessNewOrder(message))
					.TransitionTo(WaitingForPayment)
				);

			During(WaitingForPayment,
			       When(PaymentComplete)
			       	.Then((saga, message) =>
			       		{
			       			Console.WriteLine("Payment Complete for '{0}' got it!", saga.Name);
			       			saga.ServeDrink();
			       		})
			       	.Complete()
				);
		});
}

(I’ve removed the class structure around the constructor to keep the code short)

If we read from the top down, it is easy to understand the behavior for this saga. Initially (which signifies the creation of a new saga based on an event in this block), when a NewOrder is received, then we’re going to call the ProcessNewOrder method on the saga passing it the message that was received. Once that method completes, the saga will transition to the WaitingForPayment state. During the WaitingForPayment state, if a PaymentComplete is received, the saga will output a message, call the ServeDrink method on the saga, and complete.

The states and events of the saga are defined by adding some static properties to the class:

public static State Initial { get; set; }
public static State Completed { get; set; }
public static State PreparingDrink { get; set; }
public static State WaitingForPayment { get; set; }

public static Event< NewOrderMessage > NewOrder { get; set; }
public static Event< PaymentCompleteMessage > PaymentComplete { get; set; }

The static constructor for the state machine initializes these properties during the definition of the behavior as part of the Define() call. It is nice to be able to see all the states and events for the saga in one place, as well as the messages that are related to the events.

Another great feature allows us to ask the saga state machine engine to tell us what it is going to do based on how we defined the behavior of the saga. The StateMachineInspector class is a tool that will output a trace of the defined behavior of the state machine. For the above saga, the output looks like:

	During Initial
		When NewOrder Occurs Containing NewOrderMessage
			(custom action)
			Transition To WaitingForPayment
	During PreparingDrink
	During WaitingForPayment
		When PaymentComplete Occurs Containing PaymentCompleteMessage
			(custom action)
			Transition To Completed
	During Completed

This information is generated by using expressions and reflecting over the class and types contained. The above example is taken from the Starbucks example in the MassTransit trunk. For a service to subscribe and handle sagas like this, the service only needs to call:

var unsubscribe = bus.Subscribe< DrinkPreparationSaga >();

Like all component-based consumers in MassTransit, the container should know how to build the saga and needs to have a ISagaRepository< T > implementation for the saga as well.

I hope as more people start to use this syntax that it evolves in a really rich was of doing code-first workflow and saga definitions. One thing we definitely want to do is take the text-based visualization of the saga and output it in a format that can be used by GraphViz or MSGLEE to get a nice diagram of the behavior of the saga. Hopefully that will be coming sooner rather than later!

So that’s one of the really cool features of MassTransit that have been added.

I’m proud to announce that we are getting close to a final 0.6 release of MassTransit. We’ve been working on this release since early December and have added a lot of new functionality. We’re also glad to have integrated a number of patches that the community has contributed — thanks for your help!

If you are currently developing with MassTransit, please check out the new bits and give us feedback. The trunk contains all of the latest code. A snapshot of the trunk, along with a 0.6 RC1 build have been added to the GoogleCode site. If you have any problems, comments, suggestions, or patches please post them to the Google Group for MassTransit. We’re interested in all experiences, good, bad, or otherwise.

A quick run down of the changes includes:

• Message Pipeline
  The message pipeline is a replacement for the message dispatcher, as well as the cache-based publishing and subscribing bits. This new pipeline is built using a pipes-and-filter approach and is dynamically adjusted as subscriptions are added and removed — both locally and by remote services.
• Saga State Machine
  With the all new method of defining sagas using a state/event fluent interface, it is now even easier to describe the business logic of a process and the events from which it is built. With the new state machine, it is not necessary to have message consumers and interfaces on your saga, you can just define the events in the class and they are wired automatically by the subscription framework. The early version of the syntax was discussed in this post.
• Better Request/Response Support
  The old bus.Request() syntax has been replaced with a new fluent style for making requests and waiting for responses. Both synchronous and asynchronous calling conventions are supported. Look for the new bus.MakeRequest extension method for more information on this slick new syntax.
• Code-based Configuration Syntax
  A new, code-based method of configuring and running a service bus has been added. This syntax significantly simplifies getting started with MassTransit and has been used throughout the unit tests. The facilities for Castle have also been updated to transparently use this new syntax with the same XML-style configuration in Windsor. More details on this syntax can be found here.
• Message Headers
  Messages are now wrapped in an envelope and headers are associated with the message. The pipeline will add certain headers, and additional headers can be added by the message publisher using an extension method to the IServiceBus.Publish and IEndpoint.Send methods. Header properties such as ResponseAddress, FaultAddress, DestinationAddress, and SourceAddress make some more advanced routing options available. All three message serializers support the message headers. Which brings me to…
• Binary, XML, and JSON Message Serializers
  All three serializers can now be configured for use as the wire message format.
• Topshelf Integration
  The old MassTransit.Host has been replaced with the easier to use Topshelf service hosting framework.
• Unsubscribe via Delegate
  The Unsubscribe methods on IServiceBus have been removed. When Subscribe is called, an UnsubscribeAction is returned instead. This delegate should be called when the subscription is no longer needed. This makes it easier to modify the state of the service bus in response to subscribe/unsubscribe actions by maintaining all of the information needed to unsubscribe in a lambda expression. The UnsubscribeAction is a delegate, and multiple unsubscribe handlers can be added together via the += operator. You can also use the extension method to get an IDisposable version of the subscription.
• Redesigned Subscription Service
  The subscription service and client have been rewritten to take advantage of the new saga syntax. This reduces the number of specialized components needed for each service, as well as leveraging the features of MassTransit internally. Check out the SubscriptionManagerGUI in the WinFormSample for a demonstration of the service, as well as the pre-built services for use in your production environment.
• Starbucks Sample Updated
  The Starbucks example has been updated to use the new saga state machines as well as some housekeeping updates to use the new features. Be sure to check it out as a starting point for understanding sagas and how to interact with them. Details on how to get the Starbucks sample running are on the wiki.

We’re hoping to get an official 0.6 release in the next couple of weeks, so please provide any feedback you have to help us wrap it up. You can get the bits, including a source and binary distribution here.

MassTransit supports sagas, which are long-lived transactions consisting of multiple events. The saga support makes it easy to orchestrate the events into a process, but it doesn’t do much to help with state management. Since state management is fairly common, I felt it necessary to add some support for a state machine.

In MassTransit, a saga is defined by creating a class and attaching some interfaces for the messages that will be consumed. The messages are the events, and the class is the saga. An example saga is shown below.

   public class DrinkPreparationSaga :
        InitiatedBy < NewOrderMessage >,
        Orchestrates < PaymentCompleteMessage >,
        ISaga
    {
        public void Consume(NewOrderMessage message)
        {
        }

        public void Consume(PaymentCompleteMessage message)
        {
        }
    }

I’ve omitted the code for each consumer to keep it short, but the business logic defined would handle the fact that once the drink is prepared, it should not be served until the PaymentCompleteMessage has been received. Storing this state is an application concern.

While I was at QCon San Francisco, I attended a tutorial session on DSLs. It was here that I got the idea to somehow make a DSL for defining the behavior of a state machine that could be used with sagas in MassTransit. It’s taken a couple of months, but I think I’ve managed to put something together that simplifies state management in sagas. With the addition of a new state machine in Magnum, along with persistence support for NHibernate, it is now easy to manage this state automatically.

To demonstrate this, below is the state machine for the above process.

	public class DrinkPreprationStateMachine :
		StateMachine < DrinkPreprationStateMachine >,
		ISaga,
		InitiatedBy < NewOrderMessage >,
		Orchestrates < PaymentCompleteMessage >,
		Orchestrates < DrinkPreparedMessage >
	{
		static DrinkPreprationStateMachine()
		{
			Define(() =>
				{
					Initially(
						When(NewOrder)
							.Then(saga =>
								{
									// start preparing the drink
								})
							.TransitionTo(PreparingDrink));

					During(PreparingDrink,
						When(DrinkPrepared)
					       	.TransitionTo(WaitingForPayment),
						When(PaymentComplete)
					       	.TransitionTo(WaitingForDrink));

					During(WaitingForPayment,
					    When(PaymentComplete)
					       	.Then(saga =>
					       		{
					       			// publish drink ready message
					       		})
					       	.Complete());

					During(WaitingForDrink,
						When(DrinkPrepared)
					       	.Then(saga =>
					       		{
					       			// publish drink ready message
					       		})
					       	.Complete());
				});
		}

		// event and state definitions not shown, but are simple properties
	}

As shown above, all the logic of states, events, and transitions is wrapped into a clean fluent interface. The During() blocks define what is to be done when events are received during a particular state. The When() blocks define the behavior when the specified events occur. All state transitions are explicit, allowing them to be captured as part of the state machine.

There are two types of events. BasicEvent supports an event without any accompanying data. The DataEvent < V > allows data to be associated with an event. In MassTransit, each DataEvent would have a matching Consume() handler. For example, Event < NewOrder > would have a public void Consume(NewOrder message) that would call RaiseEvent(NewOrderEvent, message) to pass the event to the state machine along with the message.

The state machine also supports inspection, allowing the definition to be output for verification that the intent was properly conveyed by the interface. This not only makes it possible to verify the flow between states, but also could allow the creation of a graph to display the states, events, and transitions in a visual manner. The provided state machine inspector currently only outputs to the trace window, but could easily be enhanced by somebody with some skills.

Also, not shown above, is the ability to specify an expression using the .And() method to evaluate the data associated with an event to determine if that event is handled by that state event action. This expression is kept as an expression, allowing the details to be output using the inspector as well.

Currently, the saga must still implement the Consume() method for each message and call RaiseEvent() on the state machine to pass the message and trigger the event. I plan to add some new message sinks to the message pipeline to make those methods unnecessary, mapping the messages directly into the event handlers within the state machine. This isn’t done yet, but it is planned.

The StateMachine also provides an IUserType implementation for NHibernate in the Magnum.Infrastructure assembly. This makes it easy to persist the state using NHibernate, storing the current state as a string. This is just the default implementation, any other could be built if your needs are different.

I originally presented this syntax during the Virtual ALT.NET meeting last week. It was a last minute presentation, so I wasn’t sure I covered all the details. Hopefully this will help provide some guidance on how it is used, along with plans for the future.

Last weekend, Dru Sellers and I (along with 100 or so others) attended the Continuous Improvement in Software Development (referred to as KaizenConf, likely due to the URL) conference in Austin, TX. We left Tulsa late Friday night after taking my girls trick-or-treating for Halloween and drove all night arriving in Austin around 5:30 AM. After a quick nap and a shower, we went to the first day of the conference.

The night before at the opening, those who were there put up a series of topics that they wanted to discuss. One of those talks was on Enterprise Service Bus (ESB) patterns and implementations using MassTransit. I think it was pretty obvious where Dru and I were going to spend our time right after lunch. There were a number of people that we have had conversations with that also attended and they were excited to learn more about distributed application designs and how to implement them using open-source tools like MassTransit.

The First Session: Alternative Architectures

After the morning announcements, Dru and I joined Ben Scheirman and Ayende Rahien in a conversations about alternatives to the RDBMS. The conversation started a bit rough, but quickly opened up into other ways to store data in our applications. Ayende brought up CouchDB and the things he learned about it. There was also some in depth discussions about proper use of databases and separating transactional data from reporting to avoid transaction blocks. Some concepts on how to achieve the appropriate separation, including asynchronous ETL (extract-transform-load) were discussed. The Map/Reduce algorithm was also covered and some examples were given in the discussion of how to map data into dimensions for reporting. The proceedings from this talk can be found on the wiki.

The Second Session: Lean Architecture

My second session of the day was on Lean Architecture. This discussion was forward looking and related to how development teams could branch out and work on features with the intent of releasing individual features as they are complete (instead of waiting for a big release with many unrelated features). There was a lot of talk about how the teams work, how version control and build processes would need to adapt to handle the complexity of dealing with multiple development versions of a single code base. I think one of my biggest take aways from this session was the need for an integration branch that is created from the trunk before each merge. It is then possible to integrate a branch into the trunk without immediately impacting the trunk. Once the simulation branch merge is complete and tested, the simulation is merged into the trunk (which should be easy, since it was created from the trunk originally). This alone would help to ensure a solid trunk that can be delivered on demand. The proceedings from this talk are also on the wiki.

Over lunch we enjoyed some amazing Austin weather while talking about projects and recent events. We then went to setup for the discussion on ESB/MassTransit. Upon arrival, we were surprised at the number of people in attendance. There was a crowd of at least 20 people and the video crew was setup with remote microphones and the works. Once we had gotten the projector setup and demos loaded, we started into the discussion.

The Third Session: ESB Patterns and MassTransit

The conversation started with a general discussion about messaging patterns. A reading list was presented, along with the major patterns that are used in a publish/subscribe system like an ESB. Applications for this type of system including everything from a command/query interactive system to migrating a batch processing application to a more real-time asynchronous process. We touched on where MassTransit was in the development lifecycle and some of the things we learned over the last year of development. We showed some of the sample applications, including the new web service bridge sample for connecting external customer web services to an internal domain.

Needless to say, we were amazed at the response we got related to MT and ESB patterns in general. It was great to have Ayende and Jeremy Miller bouncing ideas around regarding the project and integration with other existing systems. The notes from the discussion are located on the KaizenConf Wiki.

The Last Session: A Mixed Bag

My last session of the day was split between two completely different sessions. One was on Advanced IOC usage, and the other was on moving from Project to Product. I bounced between these two sessions, picking up some interesting bits from each one. Both were late in the day, so I was fading pretty fast considering I only had 90 minutes of sleep the previous night. Good concepts were captured in the notes, so be sure to check those out (I’ll have to just to remember them).

After the sessions for the day, we went to the Hyatt with Ben (B#) and provided input as he started writing a new sample for MassTransit. The context for this new sample was the Gregor Hohpe article about how Starbucks Does Not Use Two-Phase Commit. This was mostly a learning exercise for Ben, however, we gained some insight watching him build it out. I think Dru refactored the host quite a bit to remove some extraneous ceremony that was just unnecessary. Once we had the first bits working, it was time to find some dinner and the rest of the gang. We had some great Tex-Mex at Trudy’s and then went back to the hotel bar to watch Texas Tech put the smack down on U-Texas Longhorns.

The next morning we met up at Starbucks (go figure) to complete the new sample Ben was building (yes, the irony of working on a sample about Starbucks @ Starbucks didn’t get past us). We ended up converting from using regular message consumers to a saga-based approach given the requirements and it took another 30 minutes to get it all working. Oren had some great ideas on how we could tweak little things here and there to eliminate some friction as well. Once we were done, we headed on over to the conference to see what was in store for the morning sessions.

This year, the focus of the conference was on continuous improvement. Therefore, instead of just having additional sessions on Sunday morning the attendees voted on sessions that they wanted to help improve. At least, that’s how I understood it, we were working on the Starbucks sample, remember? Anyway, when we arrived we found that the ESB Patterns session was one of the topics. The room was full with something like 22 votes on the page. I was surprised to see the response of the people, truly surprised.

Continuous Improvement: ESB Patterns and MassTransit

The actions are summarized on the wiki, and included things like documentation, more sample applications, and help on how to add messaging to an existing application. We also dug into the things needed to help people build and understanding messaging components in their applications. Things like diagnostics, conventions to avoid common missteps, hands-on labs to walk through building a message-based component, and a few other getting started items were brought up. From a technical side, we talked about additional patterns such as an overall conversation id to correlate multiple sagas (like a saga of sagas), as well as built-in support for compensation.

A large part of the discussion was on how functional programming features could be used to enhance the system. An example in Erlang of a message exchange was drawn up, along with how it might be written in C#. Matthew Podwysocki was consulted on how this example might be written in F#, and there was discussion about how the threading model of F# doesn’t really support the style used by Erlang. Some work is still needed there I suppose but we’ll continue to be open about the possibilities offered by functional languages. Glenn Block also talked about getting in touch with the connected systems group to see what they could offer.

County Line BBQ

With that, the conference was closed. Once again, the open space technology used for the event was awesome. The rules of open space dictate that what happened is the only thing that could have happened, and I agree with that completely. It was great to be a part of yet another excellent event and I look forward to future installments in other locations. Be sure to keep up with the wiki, as videos from most of the sessions will be made available online soon. One of our improvements was to better document our proceedings, and I think we’ve managed to succeed on that one for sure.

This past Thursday and Friday, the Tulsa TechFest was held at OSU Tulsa in Tulsa (could I write Tulsa one more time, I knew I could). Attendance was high and most of the sessions I attended were in rooms full of people. The day started off early Thursday morning, but that’s not the start of the story.

The night before, Dru and Rob came by the house for a little pre-conference warmup (and by warmup, I don’t mean this). Rob went through his presentation on continuous integration one last time while Dru and I worked through our presentation on messaging (and how to do it with MassTransit, of course). The night ended early for me, but Rob and Dru met up with Ben at the hotel and closed the bar (and by closed, I mean walked in at last call and got one beer).

The next morning we all met up and caught up on things since the last gathering. Since the last time we saw Ben, he’d been through a hurricane and granted the MVP Award from Microsoft. We then planned out our day of sessions based on the information currently available to us.

The speaker for the CSS talk was unable to attend, so the four of us convened an open-space session on CSS. The discussion in the fishbowl was good with a lot of interesting topics. Ben gave an on-screen demonstration of CSS from the ground up for those in the room that were new to it, providing context for the audience. CSS is extremely important considering it is the best (only?) way to layout and style websites consistently across browsers. I think everyone brought up how much of a turd IE 6 is when it comes to CSS compatibility.

After lunch, it was time for Dru and I to present our session on message-driven architecture (using MassTransit). You can see the first hour of the session on video here. The crowd really got into it, asked a lot of questions, and hopefully came away with an understanding of asynchronous application design and messaging.

After that session, we sat down with a guy that works for Sun and talked about enterprise application architecture. It was interesting comparing the mature open-source nature of Java to the budding open-source landspace in .NET. After the closing session and prize giveaway, there was a speakers dinner (Rib Crib, good stuff). Once we had eaten, we went to the hotel and did a little code sharing and Dru and Ben went through ASP.NET MVC some more. Then we went over to Dirty’s Tavern for some post-day fun. I was worn out, so I went back to my car and called it a night.

The next day was full of interesting stuff. A nice introduction to ASP.NET MVC by Ben, some extensive coverage of log4net by Dru, and I gave a presentation on iPhone development. Outside of the actual sessions there were a lot of great conversations about development and tools in general. We also recorded Ray Lewallen’s session on Behavior Driven Development, which can be viewed here.

My iPhone development session was purely introductory to show the tools and how they are used to build and deliver applications for the iPhone. The room was absolutely packed and hopefully everyone walked away with some good information. I know at least one guy did, he left two seconds after I said that building iPhone applications requires a Mac!

To wrap it up, the event was a huge success. There were a ton of people there, the vendor room was always alive with activity (likely due to Chris Koenig and his Rock Band setup giving some much needed ADHD relief between sessions). Chris also had a couple of great sessions on Silverlight and the new features in 2.0 that should really improve the use of Silverlight for Rich Internet Applications (RIAs).

I’m happy to say that we’ve just tied a bow around the latest release of MassTransit. Release 0.4 includes a number of new features and some tweaks to the internals as well. I’m going to describe a few of those features below, but you can grab the latest from the trunk or download the 0.4 release.

Building MassTransit

Since Visual Studio 2008 has been out for almost a year, it is now required to open the updated solution for MassTransit. In the main folder, the MassTransit-2008.sln is the one to use to build and run the unit tests. Many of the samples solutions are also 2008 solutions. The assemblies, however, are still targeting the .NET 2.0 framework, making them usable on both 2005 and 2008 projects. With only the .NET 3.5 framework installed, you should be able to run the build.bat to build the project without Visual Studio (our CI server does this).

Timeout Service

To enable automated support for timeouts in sagas, a new timeout service is available. This is a general service that can be used to schedule timeouts for whatever purpose may be needed. To schedule a timeout, the application should publish a ScheduleTimeout message with the duration or time when a response should be sent. The application/service can then consume the TimeoutExpired message, which will be published by the timeout service when the timeout period expires.

Message Deferral Service

One of the scenarios I often find in our systems is the need to poll a remote resource to determine if an operation has completed. To support this behavior without custom code in each instance, a new message deferral service has been added. This can be used to defer the delivery of a message until a period of time expires. The deferral services leverages the timeout service for scheduling and we republish a message after that timeout expires.

For example, we have a CheckRemoteResponseStatus message in one of our systems. This is initially published after a request is submitted to a remote system and a remote transaction id is returned. The first time the consumer gets the message, it checks the remote system for a response. In most cases, the response is immediately available and the saga continues. However, sometimes the remote system is too busy to respond and returns a pending status. In this case, the same CheckRemoteResponseStatus message is published within a DeferMessage. The deferred message service handles that message and will republish the original CheckRemoteResponseStatus message when the timeout expires. The saga will then handle the message to see if a response is now available. The saga keeps track of how many times the remote status has been checked and uses a sliding interval that increases as the retry count increases. Eventually, the final retry results in a failed transaction and is handle appropriately.

The nice thing about this is there was no custom retry logic required, and a common timeout and message deferral service were used. There are likely other cases within the application that will benefit from this shared functionality.

Transactional Queue Support

With 0.4, the entire method of reading from the endpoints has been redesigned. Previously, a single receive thread was used to receive from the endpoint which then dispatched the message handling to the dispatcher inside the service bus. This has been redesigned to use the dispatcher threads to perform the actual receive from the endpoint, using a transaction (ala System.Transactions) to handle the message reception. This keeps the transaction to a single thread while at the same time allowing concurrent message reception.

The transaction carries over into actions that are part of the message consumer. If a database update is part of the consumer, that database update can cause the entire message to rollback if it fails. If any exception is thrown, the entire reception of the message, any additional database operations, new messages sent, etc. will all be rolled back with the transaction.

Performance Improvements

Dru spent some time in NYC with Ayende Rahien reviewing the MT source code and Oren recommended changing from using locks to ReaderWriterLocks to improve concurrency. The changes in the threading system, along with the elimination of a lot of locking in favor of reader/writer locks has nearly doubled the throughput of messages when using a multi-core system using MSMQ. There have been a number of other internal tweaks as well to improve the concurrency of the bus dispatcher.

Control Bus

To enable competing consumer in a publish/subscribe environment, the control messages need to be on a separate bus from the data messages. To allow multiple services to compete against a single data channel (single MSMQ) in order to load balance and handle failure scenarios, the services cannot compete on the control messages such as subscriptions. The subscription client has been tweaked to allow it to operate on a separate bus from the data bus, at the same time notifying the subscription service of messages handled by the local endpoints of the service.

It’s easy to setup a single service that consumes messages from multiple buses (which in turn each have a specific endpoint being serviced). When a component is created to consume a message, the specific bus/endpoint that received the message is injected into the component (via setter injection) so that any subsequent messages can be published to the appropriate bus.

Health Service

The health service has been added making it easy to monitor endpoints and identify when an endpoint goes down. Periodic heartbeats are sent to the service and when a heartbeat hasn’t been received in a while, it marks that endpoint as down and attempts to directly ping it to get a response. The heartbeats can be subscribed, so a monitoring tool can keep track of which endpoints are there and what they are handling.

Configuration Model

To make it easier to use the bus in different containers, a new configuration model has been added to build and configure a service bus instance. This will ultimately result in moving a lot of the code used by build a service bus out of the container-specific facilities (such as the Windsor container).

Host Improvements

The ability to create and deploy Windows services has gotten easier with the updates to the Host assembly in MassTransit. With only a few files to define the lifecycle of a service, it is easy to get the ability to run, test, install and deploy a service. This includes services that are using the service bus. There is little to no coupling between the host and service bus, making it usable for a variety of purposes.

Learning MassTransit

A lot of requests have come for information on how to learn to use MassTransit. During Tulsa TechFest this week, we’re going to record our presentation and make it available online within a few days. This should give at least some introduction on how to use MassTransit (the presentation is mainly on distributed architecture, but we’re using MT for the demo bits). We’re also talking about doing a couple of podcasts on how to use it as well. Depending upon how that goes, we’ll try to do a couple of screencasts on “creating your first project with MT.”

The best way to discover how to use the code is to review the samples. The WinFormSample gives an overall example of how a variety of features are used. The HeavyLoad shows how many of the pieces work as well. The samples folder has a few others that demonstrate how to use MT in other scenarios.

So, check out the new release and give us some feedback on how the new features are working. We’ve already got a few backlog items that we’re slating for 0.5 based on some other contexts that have come up in our applications. Feel free to post on the message group or send either Dru or I an e-mail or tweet if you have any questions.