PhatBoyG.com

Just over a week ago, Dru Sellers and I recorded a podcast with Elegant Coders Jarod Ferguson and David Starr. We spent a couple of hours discussing event driven architecture (EDA), messaging, and MassTransit. Through the magic of editing, they managed to end up with a solid eighty minutes (most likely to achieve a straight up G rating).

Check it out on their web site (where you can listen, or subscribe via iTunes/PodCastRSS).

MassTransit supports sagas, which are long-lived transactions consisting of multiple events. The saga support makes it easy to orchestrate the events into a process, but it doesn’t do much to help with state management. Since state management is fairly common, I felt it necessary to add some support for a state machine.

In MassTransit, a saga is defined by creating a class and attaching some interfaces for the messages that will be consumed. The messages are the events, and the class is the saga. An example saga is shown below.

   public class DrinkPreparationSaga :
        InitiatedBy < NewOrderMessage >,
        Orchestrates < PaymentCompleteMessage >,
        ISaga
    {
        public void Consume(NewOrderMessage message)
        {
        }

        public void Consume(PaymentCompleteMessage message)
        {
        }
    }

I’ve omitted the code for each consumer to keep it short, but the business logic defined would handle the fact that once the drink is prepared, it should not be served until the PaymentCompleteMessage has been received. Storing this state is an application concern.

While I was at QCon San Francisco, I attended a tutorial session on DSLs. It was here that I got the idea to somehow make a DSL for defining the behavior of a state machine that could be used with sagas in MassTransit. It’s taken a couple of months, but I think I’ve managed to put something together that simplifies state management in sagas. With the addition of a new state machine in Magnum, along with persistence support for NHibernate, it is now easy to manage this state automatically.

To demonstrate this, below is the state machine for the above process.

	public class DrinkPreprationStateMachine :
		StateMachine < DrinkPreprationStateMachine >,
		ISaga,
		InitiatedBy < NewOrderMessage >,
		Orchestrates < PaymentCompleteMessage >,
		Orchestrates < DrinkPreparedMessage >
	{
		static DrinkPreprationStateMachine()
		{
			Define(() =>
				{
					Initially(
						When(NewOrder)
							.Then(saga =>
								{
									// start preparing the drink
								})
							.TransitionTo(PreparingDrink));

					During(PreparingDrink,
						When(DrinkPrepared)
					       	.TransitionTo(WaitingForPayment),
						When(PaymentComplete)
					       	.TransitionTo(WaitingForDrink));

					During(WaitingForPayment,
					    When(PaymentComplete)
					       	.Then(saga =>
					       		{
					       			// publish drink ready message
					       		})
					       	.Complete());

					During(WaitingForDrink,
						When(DrinkPrepared)
					       	.Then(saga =>
					       		{
					       			// publish drink ready message
					       		})
					       	.Complete());
				});
		}

		// event and state definitions not shown, but are simple properties
	}

As shown above, all the logic of states, events, and transitions is wrapped into a clean fluent interface. The During() blocks define what is to be done when events are received during a particular state. The When() blocks define the behavior when the specified events occur. All state transitions are explicit, allowing them to be captured as part of the state machine.

There are two types of events. BasicEvent supports an event without any accompanying data. The DataEvent < V > allows data to be associated with an event. In MassTransit, each DataEvent would have a matching Consume() handler. For example, Event < NewOrder > would have a public void Consume(NewOrder message) that would call RaiseEvent(NewOrderEvent, message) to pass the event to the state machine along with the message.

The state machine also supports inspection, allowing the definition to be output for verification that the intent was properly conveyed by the interface. This not only makes it possible to verify the flow between states, but also could allow the creation of a graph to display the states, events, and transitions in a visual manner. The provided state machine inspector currently only outputs to the trace window, but could easily be enhanced by somebody with some skills.

Also, not shown above, is the ability to specify an expression using the .And() method to evaluate the data associated with an event to determine if that event is handled by that state event action. This expression is kept as an expression, allowing the details to be output using the inspector as well.

Currently, the saga must still implement the Consume() method for each message and call RaiseEvent() on the state machine to pass the message and trigger the event. I plan to add some new message sinks to the message pipeline to make those methods unnecessary, mapping the messages directly into the event handlers within the state machine. This isn’t done yet, but it is planned.

The StateMachine also provides an IUserType implementation for NHibernate in the Magnum.Infrastructure assembly. This makes it easy to persist the state using NHibernate, storing the current state as a string. This is just the default implementation, any other could be built if your needs are different.

I originally presented this syntax during the Virtual ALT.NET meeting last week. It was a last minute presentation, so I wasn’t sure I covered all the details. Hopefully this will help provide some guidance on how it is used, along with plans for the future.

Today marks the one year anniversary of the first commit to the MassTransit GoogleCode repository. While Dru Sellers and I initially did some proof of concept work, this day marks the decision to go forward with a standalone .NET messaging framework.

MassTransit was started as a collaborative effort to provide a lightweight messaging framework on MSMQ for .NET applications. Both Dru and I needed a framework for asynchronous messaging to address some work-related application requirements. While MSMQ is provided out of the box, it doesn’t directly encourage some good distributed application practices such a loose coupling. Our goal was to abstract the messaging aspects so the services could be built to deal with plain old objects (POCOs) instead of lower level transport messages.

Originally, we both looked at NServiceBus as a way to make this happen. I’ve followed Udi’s blog for a while and have really gained a lot of knowledge from his posts and presentations. However, our lack of experience in Spring.NET, along with a general lack of understanding of all the complexity of such a framework led us down the path of building our own framework.

So that’s how MassTransit got started. A year later we’re still improving the code and providing useful functionality for developers to build loosely-coupled asynchronous applications. We’ve even harvested some common functionality (such as the service host, which is now a standalone project Topshelf), and plan to harvest out some additional features in the near future. The great thing about building open source software is the cooperative nature — even on similar projects. We all learn from each other and build that knowledge into tools that other developers can use and learn from as well.

One of the things I’ve missed since we integrated container support is the ability to quickly and easily create an instance of the ServiceBus. After Ayende agreed, I decided it was time to do something about it.

Behold the minimum amount of code necessary to create a service bus:

	IObjectBuilder objectBuilder = YourContainer;

	IEndpointFactory endpointFactory = EndpointFactoryConfigurator.New(x =>
		{
			x.SetObjectBuilder(objectBuilder);
			x.RegisterTransport< MsmqEndpoint >();
		});

	// add the endpointFactory to your container

	IServiceBus bus = ServiceBusConfigurator.New(x =>
		{
			x.SetObjectBuilder(objectBuilder);
			x.ReceiveFrom("msmq://localhost/mt_client");
		});

That’s it. In fact, you can easily mock out the container with some nifty Rhino.Mocks usage:

	ObjectBuilder = MockRepository.GenerateMock< IObjectBuilder >();

	ISubscriptionCache subscriptionCache = new LocalSubscriptionCache();
	ObjectBuilder.Stub(x => x.GetInstance< ISubscriptionCache >())
		.Return(subscriptionCache);

	ITypeInfoCache typeInfoCache = new TypeInfoCache();
	ObjectBuilder.Stub(x => x.GetInstance< ITypeInfoCache >())
		.Return(typeInfoCache);

	ObjectBuilder.Stub(x => x.GetInstance< IEndpointFactory >())
		.Return(endpointFactory);

I’m starting to convert the tests to use the mocked container approach to reduce the runtime of the tests. But so far the HeavyLoad, Starbucks and WinFormSample samples have been verified to work with the new model. The Windsor facility is also using the new model (Dru is going to update the other two tomorrow).

I’m pretty happy with the new configuration syntax, it certainly makes it easier to setup a bus with zero XML abuse. Look for more of this style of configuration/extension in MT in the near future.

The main conference for QCon started Wednesday, opening with a keynote by Martin Fowler and Rebecca Parsons from Thoughtworks. The topic was Architects and Agilists – Allies not Adversaries. As you can surmise from the title, this talk focused on building a collaborative relationship between ivory tower architects and the teams responsible for delivering software. With a scene from the Matrix where Neo meets The Architect as the backdrop, a number of solutions were presented after highlighting the key disconnects between the two roles. I’ve always been a firm believer that architects who do not code (NCA’s) are not nearly as effective as those who understand the issues of those actually writing code.

MERB

Since the presentation by Dan North was overflowing with humanity, we opted to attend the session on MERB. MERB is a high-performance, scalable MVC framework for Ruby. It is in the same space as Rails, but with a lot of optimizations to increase performance. It also supports slices, allowing features to be added as simple gems. This is one to keep an eye on as it grows.

Google App Engine and the Google Data APIs

My next stop was to learn about Google’s AppEngine and how it handles scalability and performance. It’s currently free, and developers can build and deploy applications into the cloud using Python and Bigtable. It’s an interesting engine with a completely transparent scaling infrastructure. You worry about your application, the system takes care of the rest. Large applications have been built and Google has allowed them to scale beyond the standard quota for free accounts. While final pricing has yet to be discussed, it’s likely to be very competitive. There is also a planned introduction of new language support early next year, but the exact language runtime being added was kept secret.

Responsive Design

The next stop was a talk by the legendary Kent Beck and his thoughts on what he calls responsive design. This talk was way up the clouds, and I’m talking the real ones. I think it’s great that we have people that think that far out, but I found little to take away from this session other than some looks of befuddlement.

Coupling, Messages, and Conversations

Since I started working on MassTransit, I’ve used the Enterprise Integration Patterns book by Gregor Hohpe as a reference manual for building distributed message-based systems. In this talk, Gregor laid down some fundamentals and set the stage for a sequel to the book that will be titled Conversation Patterns. But first, the challenges of message-based systems were presented. The levels of failure are quite involved, and include things like lost request, lost response, slow response and retry duplication. A lot of these are covered in texts online, so much of this material was review for me. A new acronym for ACID was also declared: Associative, Commutative, Idempotent, and Distributed.

With large scale distributed systems where consistency is of a more eventual rather than immediate nature, it’s important to recognize that the future if flexible and redundant rather than predictable and accurate. Building distributed transactions that are durable and that support compensation is crucial to having a success, scalable application.

During the final session slot, a few of us sat down and had an in-depth discussion about what we had learned that day. Realizing that this was only the first and lightest day, we recognized quickly that we were in for a couple of dense days of exposure. We went over some things in MassTransit, and thought about how we could offer a more developer-friendly method of managing the state of distributed transactions (sagas in MT). We also delved into the hierarchical structure of conversations on top of sagas, a topic we hoped to dig into more as we saw multiple concurrent sagas running in parallel as part of an overall conversation.

Attendee Party

That evening, there was an attendee party at a nearby pool hall. After some snacks and a quick game of pool, Dru and I sat down with Gregor to discuss some messaging concepts. After a quick exchange in Japanese, we earned a seat at the table and started to talk about various conversation patterns that we had identified in our own work. I was surprised that we were thinking along the same lines and that several of the message patterns we had used were going to be covered in the book. We also talked a little bit about Google Protocol Buffers (a platform and language independent format for serializing data in an efficient, binary stream). We started digging into GPB a few weeks ago as a way to build more system interoperability between Java and .NET when using MassTransit. Based on our process so far, this is likely to end up in a future release of MT.

With that, the evening was pretty much done with my head pounding. I retired for the night recognizing that the time zone change was probably going to result in me waking up at 4:00 AM again (which, of course, I did).

Last weekend, Dru Sellers and I (along with 100 or so others) attended the Continuous Improvement in Software Development (referred to as KaizenConf, likely due to the URL) conference in Austin, TX. We left Tulsa late Friday night after taking my girls trick-or-treating for Halloween and drove all night arriving in Austin around 5:30 AM. After a quick nap and a shower, we went to the first day of the conference.

The night before at the opening, those who were there put up a series of topics that they wanted to discuss. One of those talks was on Enterprise Service Bus (ESB) patterns and implementations using MassTransit. I think it was pretty obvious where Dru and I were going to spend our time right after lunch. There were a number of people that we have had conversations with that also attended and they were excited to learn more about distributed application designs and how to implement them using open-source tools like MassTransit.

The First Session: Alternative Architectures

After the morning announcements, Dru and I joined Ben Scheirman and Ayende Rahien in a conversations about alternatives to the RDBMS. The conversation started a bit rough, but quickly opened up into other ways to store data in our applications. Ayende brought up CouchDB and the things he learned about it. There was also some in depth discussions about proper use of databases and separating transactional data from reporting to avoid transaction blocks. Some concepts on how to achieve the appropriate separation, including asynchronous ETL (extract-transform-load) were discussed. The Map/Reduce algorithm was also covered and some examples were given in the discussion of how to map data into dimensions for reporting. The proceedings from this talk can be found on the wiki.

The Second Session: Lean Architecture

My second session of the day was on Lean Architecture. This discussion was forward looking and related to how development teams could branch out and work on features with the intent of releasing individual features as they are complete (instead of waiting for a big release with many unrelated features). There was a lot of talk about how the teams work, how version control and build processes would need to adapt to handle the complexity of dealing with multiple development versions of a single code base. I think one of my biggest take aways from this session was the need for an integration branch that is created from the trunk before each merge. It is then possible to integrate a branch into the trunk without immediately impacting the trunk. Once the simulation branch merge is complete and tested, the simulation is merged into the trunk (which should be easy, since it was created from the trunk originally). This alone would help to ensure a solid trunk that can be delivered on demand. The proceedings from this talk are also on the wiki.

Over lunch we enjoyed some amazing Austin weather while talking about projects and recent events. We then went to setup for the discussion on ESB/MassTransit. Upon arrival, we were surprised at the number of people in attendance. There was a crowd of at least 20 people and the video crew was setup with remote microphones and the works. Once we had gotten the projector setup and demos loaded, we started into the discussion.

The Third Session: ESB Patterns and MassTransit

The conversation started with a general discussion about messaging patterns. A reading list was presented, along with the major patterns that are used in a publish/subscribe system like an ESB. Applications for this type of system including everything from a command/query interactive system to migrating a batch processing application to a more real-time asynchronous process. We touched on where MassTransit was in the development lifecycle and some of the things we learned over the last year of development. We showed some of the sample applications, including the new web service bridge sample for connecting external customer web services to an internal domain.

Needless to say, we were amazed at the response we got related to MT and ESB patterns in general. It was great to have Ayende and Jeremy Miller bouncing ideas around regarding the project and integration with other existing systems. The notes from the discussion are located on the KaizenConf Wiki.

The Last Session: A Mixed Bag

My last session of the day was split between two completely different sessions. One was on Advanced IOC usage, and the other was on moving from Project to Product. I bounced between these two sessions, picking up some interesting bits from each one. Both were late in the day, so I was fading pretty fast considering I only had 90 minutes of sleep the previous night. Good concepts were captured in the notes, so be sure to check those out (I’ll have to just to remember them).

After the sessions for the day, we went to the Hyatt with Ben (B#) and provided input as he started writing a new sample for MassTransit. The context for this new sample was the Gregor Hohpe article about how Starbucks Does Not Use Two-Phase Commit. This was mostly a learning exercise for Ben, however, we gained some insight watching him build it out. I think Dru refactored the host quite a bit to remove some extraneous ceremony that was just unnecessary. Once we had the first bits working, it was time to find some dinner and the rest of the gang. We had some great Tex-Mex at Trudy’s and then went back to the hotel bar to watch Texas Tech put the smack down on U-Texas Longhorns.

The next morning we met up at Starbucks (go figure) to complete the new sample Ben was building (yes, the irony of working on a sample about Starbucks @ Starbucks didn’t get past us). We ended up converting from using regular message consumers to a saga-based approach given the requirements and it took another 30 minutes to get it all working. Oren had some great ideas on how we could tweak little things here and there to eliminate some friction as well. Once we were done, we headed on over to the conference to see what was in store for the morning sessions.

This year, the focus of the conference was on continuous improvement. Therefore, instead of just having additional sessions on Sunday morning the attendees voted on sessions that they wanted to help improve. At least, that’s how I understood it, we were working on the Starbucks sample, remember? Anyway, when we arrived we found that the ESB Patterns session was one of the topics. The room was full with something like 22 votes on the page. I was surprised to see the response of the people, truly surprised.

Continuous Improvement: ESB Patterns and MassTransit

The actions are summarized on the wiki, and included things like documentation, more sample applications, and help on how to add messaging to an existing application. We also dug into the things needed to help people build and understanding messaging components in their applications. Things like diagnostics, conventions to avoid common missteps, hands-on labs to walk through building a message-based component, and a few other getting started items were brought up. From a technical side, we talked about additional patterns such as an overall conversation id to correlate multiple sagas (like a saga of sagas), as well as built-in support for compensation.

A large part of the discussion was on how functional programming features could be used to enhance the system. An example in Erlang of a message exchange was drawn up, along with how it might be written in C#. Matthew Podwysocki was consulted on how this example might be written in F#, and there was discussion about how the threading model of F# doesn’t really support the style used by Erlang. Some work is still needed there I suppose but we’ll continue to be open about the possibilities offered by functional languages. Glenn Block also talked about getting in touch with the connected systems group to see what they could offer.

County Line BBQ

With that, the conference was closed. Once again, the open space technology used for the event was awesome. The rules of open space dictate that what happened is the only thing that could have happened, and I agree with that completely. It was great to be a part of yet another excellent event and I look forward to future installments in other locations. Be sure to keep up with the wiki, as videos from most of the sessions will be made available online soon. One of our improvements was to better document our proceedings, and I think we’ve managed to succeed on that one for sure.

One of the more common scenarios I hear about when talking about building a service-oriented architecture has to do with providing a web service to external customers. This is often in the context of providing information from an internal system. The problem comes with how many of these systems are built.

In many situations, a web service is created (perhaps with ASP.NET Web services, an .asmx) which accepts a request from a client, performs some query against the production database, and returns the results. This is a very tightly-coupled way of meeting the requirements. How does one go about reducing coupling? Well, all too often the answer is “just add another web service.” You can see where this is going. The control still follows a chain of web service requests up until the final connection to the database.

The new WebServiceBridge sample in MassTransit demonstrates how to break this coupling using the publish/subscribe architecture. When the external web service receives a request, a message is asynchronously published. A subscription to the requested information is added at the same time. A completely insulated service then handles the request, publishing the response to the bus. The web service request that was pending is completed and the results of the query are returned to the client.

In the sample, this all happens with the least amount of load on the web server by using the asynchronous nature of web methods in ASP.NET. This reduces the load on the web servers, making it easier to handle multiple requests without extensive hardware requirements.

Last night I made a screencast that goes through this new sample and shows how it works. You can download the screencast here (please Save As to conserve bandwidth). Check it out and let me know what you think. Questions and comments are always welcome!