PhatBoyG.com

Last week I presented at Codemash on ASP.NET MVC. I received a request for the demo code, which you can download here.

Codemash was an awesome event, one that hope to attend again in the future!

The goal of this article is simple: a clean installation of Windows 7 that can be used to test new versions of Visual Studio 2010 until we reach RTM. Since a new beta drops every couple of months (and maybe even an interim release or two in between for those lucky enough to get access to it), I don’t want to have to repave the machine for each new release. Since Windows 7 has the ability to boot into a VHD, I thought this would be the perfect opportunity to try this functionality myself.

Getting Started

The first thing I did was slide the Windows 7 DVD into the drive and rebooted the machine. Once the setup window appeared, I pressed Shift+F1 to open a command prompt and created a new VHD by typing the commands shown below.

diskpart
create vdisk file="d:\vhds\w7base.vhd" type=expandable maximum=80000
select vdisk file="d:\vhds\w7base.vhd"
attach vdisk

Once this was finished, I switched back to the setup window and clicked Install Now. I chose the advanced install option, after which I selected my newly created partition as the installation target and formatted it so the installation could continue. While waiting for the setup process to complete (which included at least two reboots, both of which properly loaded into the VHD without any issues, so that’s cool), I started working on this blog post.

After the install completed, I wanted to make sure I didn’t get confused by the boot manager now having two “Windows 7″ options at started. So I quickly opened a command prompt (as Administrator), and typed:

bcdedit /set description "Windows 7 Baseline"

This will change the name of the boot manager entry for the current machine to “Windows 7 Baseline,” distinguishing it from the normal Windows 7 install. Once I’ve completed all the setup tasks, I’ll set the regular Windows 7 option to be the default when I’m not working with the beta software installations.

Next I ran Windows Update, along with installing the Microsoft Security Essentials to have a basic level of virus/malware protection.

I almost always need a database server to do any sort of distributed application testing, so I wanted to have a SQL Server 2008 available. However, I didn’t want to have it setup on each virtual disk. So I installed it to the base installation, along with a few other tools that I use often (and are not really updated regularly). I also decided that since Visual Studio 2010 and Visual Studio 2008 run nicely side-by-side that I would go ahead and install Visual Studio 2008 on the base image since I’ll most likely be working primarily with 2008 through the next year.

When you boot from a VHD, your regular Windows system drive is available as drive D. This is really useful since you can keep a single checkout of all your projects and work on them from different systems! On my regular Windows 7 install, I created a “Shared” folder off the root (OMG not a Library?). Within that folder I have a Home folder which I use for SVN checkouts, and I also configured SQL Server to use a directory under Shared for all the data storage. This will be handy for ensuring that data in the database is updated regardless of which system is using SQL – very cool.

Making a Difference

With the above steps behind us, we now have a machine with two Windows 7 installations on it. The first, a regular disk-based install, is our primary installation that we use for whatever. The second, a VHD-based install, is our clean Windows 7 with our minimum set of software installed on it. It’s this clean version that we want to keep “pristine” to avoid having to redo all the steps above with each new release of Visual Studio 2010.

A quick note, the size of the VHD after all the software is installed is about 20 gigabytes, which is pretty substantial considering that we now don’t need to copy this space for each branch of the baseline we create — a nice savings!

To create the differential image, I restarted the machine and selected my regular disk-based Windows 7 install. Once it was loaded, I opened up a command prompt and created a new VHD based off the existing one I just finished creating.

diskpart
create vdisk file="D:\vhds\w7-2008.vhd" parent="D:\vhds\w7base.vhd"
create vdisk file="D:\vhds\w7-2010.vhd" parent="D:\vhds\w7base.vhd"

Once the partition is created, create a boot entry for it using:

bcdedit /v (this displays the GUID we'll use next for our copy)
bcdedit /copy {3F2504E0-4F89-11D3-9A0C-0305E82C3301} /d "Windows 7 - 2008"
bcdedit /v (to get the GUID of our copy, which is also displayed after the copy command as well)
bcdedit /set {3F2504E1-4F89-11D3-9A0C-0305E82C3301} device vhd="[C:]\vhds\w7-2008.vhd"
bcdedit /set {3F2504E1-4F89-11D3-9A0C-0305E82C3301} osdevice vhd="[C:]\vhds\w7-2008.vhd"
bcdedit /v (to verify our work)
bcdedit /set {3F2504E2-4F89-11D3-9A0C-0305E82C3301} device vhd="[C:]\vhds\w7-2010.vhd"
bcdedit /set {3F2504E2-4F89-11D3-9A0C-0305E82C3301} osdevice vhd="[C:]\vhds\w7-2010.vhd"
bcdedit /v (again, to verify our work)
bcdedit /displayorder {current} {3F2504E1-4F89-11D3-9A0C-0305E82C3301} {3F2504E2-4F89-11D3-9A0C-0305E82C3301}

In this case, {3F2504E0-4F89-11D3-9A0C-0305E82C3301} is my original VHD-based install, {3F2504E1-4F89-11D3-9A0C-0305E82C3301} is my copy for Visual Studio 2008, and {3F2504E2-4F89-11D3-9A0C-0305E82C3301} is my copy for Visual Studio 2010.

I now have three boot options when the system starts, my disk-based Windows 7 install, and my two newly created “differential” installs — one for Visual Studio 2008 and one for Visual Studio 2010. Since both of these are based off the VHD we created above, we want to make sure we don’t accidentally boot into the w7base VHD and change it — that would be like crossing the streams in Ghostbusters (read: bad). The /displayorder command takes care of this by having a side effect of removing the Windows 7 Baseline option from the list of boot configurations.

One thing to keep in mind, and another benefit of having the physical disk drive available as drive D, is that I do all of my work on the D drive (in the Shared folder). Since this drive is persistent I can be sure that I don’t lose my files as I boot between different installations of Windows 7. It also makes it so I don’t have to checkout projects on each VHD, which would waste valuable space.

Cleaning Up

So now that we have a nice baseline and a couple of working boot options (both of which don’t impact each other as changes are made), we can install our latest BETA/CTP/whatever on our 2010 install. When an update is released (in the form of a subsequent CTP, beta, etc.), we can take a few simple steps to revert our 2010 to a clean install.

First, we need to boot into our disk-based installation of Windows 7. Once there, there are a couple of options available. We could create an entirely new configuration by repeating the steps above. This would be helpful if we were in the middle of something on a current version of 2010 and wanted to try it in the update to check for breaking changes. The more likely option, however, is to just replace the installation with a clean baseline to install fresh.

To install fresh, we can just delete the existing w7-2010.vhd and create it again using:

diskpart
create vdisk file="D:\vhds\v7-2010.vhd" parent="D:\vhds\w7base.vhd"

The boot configuration is still pointing to that file, so it will find it after a restart and you can install the latest update without fear of some bad configuration/files from a previous beta/CTP getting in the way.

Conclusion

The new boot to VHD features of Windows 7 (which also work just fine for Windows 2008 Server R2 by the way) are a really slick addition to the Windows platform. Being able to take advantage of the VHD features, differential disks in particular, really makes it a great host for the various incarnations of Visual Studio. I hope this guide has helped ease some of the CTP/BETA pain that comes with running on the bleeding edge.

Over the past couple of years I have been traveling more, both for work and to attend and/or present at a variety of technical events. While I have yet to earn the esteemed grizzled road veteran merit badge, I have managed to learn a few things along the way. When presenting at a technical event, whether it is a user group meeting, a code camp, or a full blown conference, being prepared is critical to delivering a successful presentation (hey, you aren’t paying for this advice, so don’t expect it to be groundbreaking).

In this post, I would like to share with you a few things that I have found that can make the difference between a good event and a great event, at least as far as your presentation is concerned.

Before You Leave (a.k.a. Pack Lightly, But Carry An Armored Tank Division)

If you pack a multiple outlet power strip, you will make friends. Power outlets are like gold at any technical event and people will covet them to charge up their power hungry devices. If you encounter a full tap and plan to stay for more than a few minutes, pull it out and share the juice with your five new friends. I have a very compact one that turns 2 outlets into 6, as well as a slightly larger unit that provides 3-to-1 expansion along with 2 USB ports for charging iPhone, iPod, or other USB-powered devices.

If you will be driving during any part of the trip, pack a car charger for your devices. Small and compact usually wins, so I prefer something that gives me at least two USB connections from the 12V plug. That is usually enough to charge my phone and headset, and can help charge a friends phone at the same time (since they likely forgot their charger unless you forwarded them a link to this article).

Don’t forget to bring plenty of cables. I always pack at least one 14-foot network cable, two USB-to-mini-USB cables, one USB-to-micro-USB cable, one firewire 800 cable, and a 800/400 adapter. The power supply for your laptop is pretty important too, so bring that long as well.

Bring an additional mobile phone battery or a battery powered top-up charger to get you through the trip. Nothing drains an iPhone like watching a movie during the flight, firing up the GPS to find your way to the hotel, checking your email and flight status, etc. I bought an inexpensive unit from Monoprice (less than $20) that connects to the bottom and tops off an iPhone that is under 20%. It has another dock connector on the bottom and charges from the same cable as the iPhone.

Scope out your network availability and plan accordingly. If you are traveling alone and cannot tether your phone to your laptop, make sure wireless networking is available. If it is not available, be sure to have all the software you need installed and make sure your demo doesn’t require an internet connection. If you are traveling with others and the hotel charges for internet, considering bringing along a wireless router (the Airport Express from Apple is small and easy to setup) to share the connection with your friends!

A quick note about travel gear – carry on only. Do not check a bag, it will only slow you down. If last minute flight changes are needed when travel issues occur, being quick and unencumbered by checked baggage will help you out. Also, the 22” roller bags will not fit in smaller aircraft, so get yourself a smaller 17” one that fits under the seat (and in the overhead on the smaller regional jets). You can easily make a two day trip with a smaller bag, particularly if you avoid different styles of clothes.

En Route (cue Vacation theme music)

First things first — if you don’t have an iPhone (or some other type of reliable smartphone), get one. I have found that having an always available internet connection, full web browser, instant email access, real-time flight tracking, and live Google maps with GPS to be the most important tools when traveling.

Now that you have your iPhone, some great apps that will really help your trip are listed below.

• AutoPilot from USA TODAY is a great, free application for managing your travel. It’s not the fastest or best, but it is free and includes TripIt integration for loading your flight information.
• Flight Update Pro is a air travel focused app that is highly accurate in keeping you up-to-date on flight status and arrival/departure times. While expensive ($9.99), it also includes TripIt integration (saving you data entry) and an easy to read status.
• The FlightCaster web is fairly accurate at predicting delays, and they also have an iPhone application. Again, integration with TripIt eliminates the duplicate entry of information.

By the way, if you aren’t using TripIt to consolidate all of your booking confirmations into a single site yet — do it. TripIt can parse the details from most booking confirmation emails and automatically updates your travel calendar (which you can subscribe to using iCal). I setup a filter in Gmail to forward my confirmation emails to plans@tripit.com, which parses the message and updates your travel calendar.

And hopefully this goes without saying, but when traveling — particularly by air — dress respectable, be courteous, and don’t lose your temper. Things happen during travel — delayed flights, road construction, missed exits, engine fires — most of these are out of your control. The last thing you want to do is be an ass to a person already under pressure. Being polite can mean the difference between getting the last seat on the last flight and sleeping on the airport floor. Oh, and when at all possible, use your phone since there are more agents at the call center who can help get you on the right flight then there are at the airport helping the people in line.

After Arriving (the night before)

Do not let any friends, co-workers, acquaintances, or otherwise get you drunk because presenting with a hangover is just not cool. And that’s all I’ve got to say about that.

Presenting (drum roll, please)

If you are presenting to a Microsoft-centric audience and you are using a Mac, make sure you have everything you need to peacefully coexist in a Windows world. You will get the inevitable comments about running a Mac, so the last thing you want is to fail due to hardware or software issue. To help you out, here are a few tips:

Almost every projector you come across will use an analog VGA (d-sub) connector, so be sure to keep a Mini DisplayPort to VGA (or DVI to VGA for older Macs) in your backpack. It doesn’t hurt to have a Mini DisplayPort to DVI as well in case you encounter a more modern project (although I’ve seen more HDMI than DVI projectors this past year — all of which also supported VGA). I also pack actual video cables as well in case you end up doing an ad-hoc session in your hotel room.

Many groups like to record or broadcast their sessions so that members unable to participate in person can view the presentation. To make things easy, be sure to have LiveMeeting already setup and tested on your machine. If you are on a Mac, this means converting your presentation from Keynote to either PowerPoint (using the free PowerPoint viewer if necessary) or Quicktime and presenting the slides from Windows. If you are not doing any Windows-based demos (like if you were giving an iPhone development talk), well, the remote folks are out of luck!

If your talk is less about code and more about practices or principles, use the presenter display from Keynote or Powerpoint on your laptop. This helps you keep track of time, review your slide notes, and preview the next slide (which helps you avoid talking ahead of your deck). On the other hand, with code focused sessions stick to a mirrored display in the native resolution of the projector.

When presenting code it is best to use a white background with black text. While the dark themes are great for large screen monitors, they look terrible on projectors with monospaced fonts. Many editors allow you to save and load settings, so build yourself a presentation mode settings file. With a set of high contrast colors (the default Visual Studio colors are a bit too hard to read at 25 feet) and a font size of 12-14 you’ll be all set to show your code to a large room. Font choice is another thing. While Consolas is my favorite, at larger font sizes it looks terrible – particularly in bold. Consider an alternate font for your presentation settings that looks good at a large font size.

When presenting with slides, a remote is a great way to get out from behind the podium. I like to present while standing in front of the screen, so I always pack my Apple remote. I tend to avoid the higher tech solutions such as the Keynote Remote app for the iPhone as they tend to just get in the way. If you need to use the remote to control PowerPoint, there are some utilities that will make that work as well.

Choose Your Own Adventure (n, n, open door, n, e, melt wizard)

Most of the time, your presentation is going to be perfect for the audience. Assuming that your session description was accurate and a level of learning was indicated you should be set. However, your audience might not be prepared for the material being covered. In these situations it is best to have an alternative path that your presentation can take, one that is more appropriate for a general audience. In some cases this might not be possible, but it can really make the difference between an engaged audience and a bunch of people cleaning out their inbox (because there is likely was too much caffeine in their blood to actually sleep).

Wrap It Up (I’ll take it)

I hope the tips I’ve shared with you here will be useful if you decide to step up to the podium. There are many other posts that talk about slide design, demo subject matter, travel tips, and more but the items above are some of the ones I’ve had to learn on my own (and thankfully not always the hard way — well except for one).

This post is the third in a series on building a highly available service gateway. The implementation will be built in C# using MassTransit, StructureMap, ASP.NET MVC, and NHibernate.

Did somebody say code?

The past two posts began to explain how to build a service gateway using MassTransit. In this post, I’m going to share some of the initial code that makes up the gateway service. The gateway itself consists of two components. The first implements the communication to the external service with a set of messages that are only used internally by the service. The second is the saga that provides the interface to the service gateway.

Service Contract

The interface exposed to the application consists of two messages, the first for the command and the second for the response. The message contracts are defined using interfaces, allowing the class for the message to be an internal implementation detail.

The message contract representing a request for order details includes the CustomerId and the OrderId.

public interface RetrieveOrderDetails
{
	string OrderId { get; }
	string CustomerId { get; }
}

When the order details are received, the following message is published.

public interface OrderDetailsReceived
{
	string OrderId { get; }
	string CustomerId { get; }
	DateTime Created { get; }
	OrderStatus Status { get; }
}

public enum OrderStatus
{
	Unknown = 0,
	Submitted = 1,
	Accepted = 2,
	InProcess = 3,
	Complete = 4,
}

The CustomerId and OrderId are the same as the values passed in the request. Created is when the order was created, and Status is an enum representing the status of the order.

Notice that no internal values are included — no primary key from the order table and no primary key from the customer table. The request and response are correlated on identifiers that make sense in the application domain. While SQL purists will point out that numeric primary keys are quicker for retrieving rows in a database, they make for a very fragile interface with other components in the system. Reliance on a primary key outside of the context of the system storing the order details is a path to friction or outright failure.

Time To Make The Saga

At this point in the design of our service, the need for a saga to manage the request state is not entirely obvious. While the TDD purists might want to call YAGNI at this point, let me assure you that “it will all be… revealed!” So for now, let us take a look at the first pass of our saga definition.

public class OrderDetailsRequestSaga :
	SagaStateMachine<OrderDetailsRequestSaga>,
	ISaga
{
	static OrderDetailsRequestSaga()
	{
		Define(Saga);
	}

Our saga state machine uses a static initializer to define the states, events, and transitions of a saga. The previous code merely defines our class as a saga and calls our saga initialization method (shown below).

	private static void Saga()
	{
		Correlate(RequestReceived)
			.By((saga, message) => saga.CustomerId == message.CustomerId &&
			                       saga.OrderId == message.OrderId &&
			                       saga.CurrentState == WaitingForResponse);

		Correlate(ResponseReceived)
			.By((saga, message) => saga.CustomerId == message.CustomerId &&
			                       saga.OrderId == message.OrderId &&
			                       saga.CurrentState == WaitingForResponse);

Since our request criteria include our customer id and our order id, we use those to correlate the message to the saga. We also include the state of the saga to ensure that we do not match to a request that has already completed. We will look at some other ways we can enhance the performance of the service later on by using some additional states.

	public static State Initial { get; set; }
	public static State WaitingForResponse { get; set; }
	public static State Completed { get; set; }

The three states we have defined, including an initial state when a new saga instance is created, a waiting for response state one our request has been sent to the service, and a completed state once the response has been received and published.

	public static Event<RetrieveOrderDetails> RequestReceived { get; set; }
	public static Event<OrderDetailsResponse> ResponseReceived { get; set; }
	public static Event<OrderDetailsRequestFailed> RequestFailed { get; set; }

The three events that we have defined, including the message contract that maps to the event. The subscription logic for the saga will automatically map message handlers for these events that will invoke the actions depending upon the current state of the saga.

		Initially(
			When(RequestReceived)
				.Then((saga, request) =>
					{
						saga.OrderId = request.OrderId;
						saga.CustomerId = request.CustomerId;
					})
				.Publish((saga, request) => new SendOrderDetailsRequest
					{
						RequestId = saga.CorrelationId,
						CustomerId = saga.CustomerId,
						OrderId = saga.OrderId,
					})
				.TransitionTo(WaitingForResponse));

The first event handler, RequestReceived, is invoked when the saga is created in response to the RetrieveOrderDetails message. The handler copies the properties of the request, and then publishes the request message to the proxy that will call the external web service. After the message is published, the state of the saga transitions to the waiting for response state. When using transactional queues, the receipt of the message, creation of the saga in the database, sending of the command message to the proxy, and saving the saga are all part of a single distributed transaction. This ensures that everything completes as a single operation to ensure no requests are lost.

		During(WaitingForResponse,
			When(ResponseReceived)
				.Then((saga, response) =>
					{
						saga.OrderCreated = response.Created;
						saga.OrderStatus = response.Status;
					})
				.Publish((saga, request) => new OrderDetails
					{
						CustomerId = saga.CustomerId,
						OrderId = saga.OrderId,
						Created = saga.OrderCreated.Value,
						Status = saga.OrderStatus,
					})
				.TransitionTo(Completed));
	}

The second event handler, ResponseReceived, is invoked when the OrderDetailsResponse message is received. The results of the request are stored in the saga and a message is published containing the details of the order back to the original requestor. Another approach would be to capture the requestor address (via the ResponseAddress header from the original message) and then resolve that address using the endpoint factory to send the response directly to the requestor. I don’t really encourage this approach without having a truly unique identifier for each request.

	public OrderDetailsRequestSaga(Guid correlationId)
	{
		CorrelationId = correlationId;
	}

	protected OrderDetailsRequestSaga()
	{
	}

	public virtual string CustomerId { get; set; }
	public virtual string OrderId { get; set; }
	public virtual OrderStatus OrderStatus { get; set; }
	public virtual DateTime? OrderCreated { get; set; }

	public virtual Guid CorrelationId { get; set; }
	public virtual IServiceBus Bus { get; set; }
}

The rest of the saga class is shown above for completeness. The properties are part of the saga and get saved when the saga is persisted (using the NHibernate saga persister, or in the case of the sample the in-memory implementation). The constructor with the Guid is used to initialize the saga when a new one is created, the protected one is there for NHibernate to be able to persist the saga.

The Service Proxy

The saga uses the external service proxy to perform the actual work, which is shown in the proxy class below.

public class OrderDetailsWebServiceProxy :
	Consumes<SendOrderDetailsRequest>.All
{
	public void Consume(SendOrderDetailsRequest request)
	{
		// make the call to the service to get the order details here

		var details = new OrderDetailsResponse
			{
				OrderId = request.OrderId,
				CustomerId = request.CustomerId,
				Created = (-1).Days().FromUtcNow(),
				Status = OrderStatus.InProcess,
			};

		CurrentMessage.Respond(details, x => x.ExpiresAt(5.Minutes().FromNow()));
	}
}

The message handler uses the criteria from the SendOrderDetailsRequest message (which was published by the saga) to call the external service and retrieve the order details. The details are then returned to the saga in the form of an OrderDetailsResponse message which is internal to the service (and therefore not part of the interface assembly that is provided to applications that want to use the order details service).

Test That Thang

Now that our saga has been developed, we need to be able to test it. A unit test will be created that creates a testing instance of the service bus (see the sample for the implementation details) and verifies that the saga responds properly to the request. To request order details, a very simple client would subscribe to the response message and then publish the request.

const string orderId = "ABC123";
const string customerId = "12345";

LocalBus.Subscribe<OrderDetailsReceived>(message =>
	{
		response.Set(message);
	},
	x => x.OrderId == orderId && x.CustomerId == customerId);

RetrieveOrderDetails request = new RetrieveOrderDetailsRequest(customerId, orderId);
LocalBus.Publish(request, x => x.SendResponseTo(LocalBus.Endpoint));

The subscribe method used above specifies that when a message of type OrderDetailsReceived is received, if the contents of the message match the predicate specified (which in this case, is checking the OrderId and CustomerId contained in the message) then the statement specified should be called. Our example is from the integration test (built using NUnit) that verifies the service performs from end-to-end.

The syntax above is functional, and it will work, but it does not represent the most scalable approach. In the next post, I’ll start to explain how to build a much more scalable method of handling thousands of concurrent requests on a single machine using IIS.

In the meantime, the sample code is available in the MassTransit trunk as a standalone solution. You can find it in the trunk\src\Samples\ServiceGatewaySample folder. There are unit tests that verify the calling syntax shown above and a service that hosts the services (both the saga, and the proxy service).

This post is the second in a series on building a highly available service gateway. The implementation will be built in C# using MassTransit, StructureMap, ASP.NET MVC, and NHibernate.

Continued…

In part one, I discussed two exchange patterns that are often exposed as a web service. In this installment, I’m going to cover a more complex exchange pattern that including makes a request to an external system in response to a request on our web service.

One of the comments on the previous post raised the question of using messaging for queries. Udi, Ayende, and I agree that this so-called “Data-SOA” is a bad thing. The queries I am presenting in this article are made against an external service and not an internal database. In many cases, this service might have limited availability and limited throughput, making it important to isolate our service to increase availability.

Complex Request

In the simple request, order status is a small enough data point that caching it at the web service makes sense. The details of an order, however, are much more involved and it is not practical to keep the details cached in case they are requested (which in this case, is much less often that the order status). To support the order detail request, we will apply a pattern that separates the inbound request from the outbound request, ensures that requests are performed only once (minimizing the click, click, click refresh mentality of some users), and even retain requests if the outbound service is unavailable until the request expires or the outbound service becomes available.

The first thing we want to build is the service that will be responsible for calling the external web service. To start, we’ll define two messages representing the happy path of calling the service. The first message contains the criteria of the request (such as the order id and perhaps the customer id) and the second message contains the details of the order that were returned by the web service. In addition to the messages on the happy path, we may also define additional messages used to publish exception information related to the request.

Once the message contracts are defined to call the web service, we need to build a service that will handle the request messages. To do this, we’ll build a message consumer for the request message, host it inside Topshelf, and subscribe to a service bus bound to the input queue of the service.
The message handler will use the request message properties to prepare the request to the external web service and produce the response message once the request returns. In this example, the external web service only supports synchronous requests (in a later article I will try to cover calling remote services that support asynchronous requests).

At this point, we have a service that responds to a command message (the request) and produces a result depending upon the outcome of the request. What we need now is a way to coordinate the requests to the service to ensure that requests are not lost due to an unavailable service, errors are retried once a failed service becomes available, and duplicate requests are ignored.

To coordinate the service requests, a saga will be created to manage the state of each request. A new message will be created to initiate the saga containing the same data that is needed to produce the request message that is sent to our gateway service. And as with the service, messages will be created to return the results of the request to the consumer of the external service. The saga will use our state machine driven saga syntax, making the business logic understandable at a glance. The saga will also define the retry parameters that should be used to recover from service outages by specifying an exception policy. The messages that are used by the requester to interact with the saga will become the interface that is used by our system to make requests to the external service. This includes the web service we are in turn providing to our customers. The internal messages used to communicate between the saga and the gateway service are not intended for use outside of our gateway service component.

A nice side benefit of this architecture involves the actual call made by the gateway service. Since the interface is defined by the messages that are orchestrated by the saga, the backing implementation can be changed without impacting the consumers of the saga-based front end. This can be a huge benefit when it comes time to change service providers or bring an external service in-house either through acquisition or new product developments.

With the saga in place (and for this example, we will go ahead and host it in the same process that we are hosting the gateway service), we are now ready to build the web service request handler. Rather than go straight into that now, I think I’m going to stop here and save that for part three. After part three is finished, I’ll start posting some of the code for the examples and include it in the samples folder to make it easy to pull down and experiment with on your own machine.

This post is the first in a series on building a highly available service gateway. The implementation will be built in C# using MassTransit, StructureMap, ASP.NET MVC, and NHibernate.

Introduction

A common way of applying a messaging solution to an existing system is to find a tightly coupled service reference in the system and insulate the system from that dependency. For example, the system may utilize a web service via a call from the user interface, blocking the user interface until the web service request completes. Another situation might involve a web service provided by the system which in turn calls another web service to complete the request. While these two examples are common in systems today, it is important consider how the system is using a dependent service when determining how best to insulate the system from that service.

To help make that decision, allow me to share some common exchange patterns implemented using a web service. While most web services are modeled after the request/response pattern, there are actually several types of exchanges that can occur within a web service method.

Drop Box

If the web service is called by the client to provide information to the application, availability is likely the most important concern. Clients may have limited connectivity (such as a wireless client), limited resources (such as an embedded system that is unable to store data for future delivery), or a combination of these and other factors. To deal with these limitations, the web service should be designed for maximum availability to avoid failed requests due to an outage behind the web service boundary.

To support this high availability, the web service should count on the only thing that is available at the time web service method is invoked — the local machine. If the web service attempts to write the information to a database located on a server across the network that is not available, the information in the request may be lost. In this case, it would be a better choice to write a message that contains the request information to a local queue. By doing this, the request information is retained and can be processed separately from the web service request. This allows allows the web service method to return to the caller, keeping resources available for other clients to report.

Once the data from the request is safely stored in the queue, a separate service is built that consumes the messages from the queue and sends the information to the internal system, which in this case might be a database server. If the database server is unavailable, the message is left in the queue until the database is available. This new service can also be stopped and even updated without disrupting the web service from receiving requests.

Simple Request

If we look at another scenario, one where the request contains search criteria and the response includes the data specified matching the request. In order to break this down in different way, we will look at two different types of requests. First, we will define a request to retrieve the status of an order. Second, we will define a request to retrieve the details of an order.

When only the status of the order is requested, we are dealing with a relatively small amount of data per order — in this case, the order id and the status of the order. If this consisted of maybe 50 bytes of data per order, we could easily store the status of 100,000 orders in only 5 megabytes of RAM. Since users might check the status of an order often, at least once an hour, this could generate thousands of requests an hour. Since the status of an order may only change once or twice a day, the need to query the database each time the order status is checked can put an unnecessary strain on the database server. In addition, the order status request service is now tightly coupled to the database, making the availability of the service dependent upon the availability of the database. This dependency chain can get even longer as more complex systems are designed, so limiting the dependencies of a service is a key parameter in increasing availability.

To reduce database load, eliminate the dependency on the database, and increase the availability of our order status web service, we can design our service to subscribe to order status updates. When the status of an order is updated, the update will publish a message containing the new status. As the status of orders are updated, our service would update an in-memory cache containing the status of every order in the system (well, every is not necessarily every — it could just be the orders placed over the last week that have not yet been received by the customer). On startup, the web service would query the database for the status of all orders placed and use the results of that query to seed the cache. Once the cache is seeded, as new orders are added and order status updates occur, the cache would update dynamically in response to the update messages. Since the cache is local to the web service, requests need only check the status by querying the cache and immediately returning the status of the order to the caller. In the case of an status request for an order that does not exist in the cache, the service could queue a request to get the order status from the database which would then publish that orders status so that it could be returned. If the order is not found, the service could return an unknown order response to the caller with instructions to perhaps try their request again later.

Up Next

The two exchanges described above are relatively easy to implement using messaging (and likewise, using MassTransit). The next exchange pattern I’m going to cover is the more complex request where the dependent service must be called to complete the request. Due to that complexity, I’m going to wait until the next installment to describe that in greater detail. After that, I’ll start to share some code as we build a solution to these exchanges.

One feature that is often overlooked in software development is the output of information that can be observed by operations once the application is in production. Fortunately, many open source projects are leveraging log4net to provide a configurable level of runtime information that can be useful in figuring out why a system is behaving a certain way (and face, if you’re looking, it’s more than likely behaving badly). Logging, however, is only one view into an application — one that might not deliver the appropriate information in a useful way.

Anyone who has used a computer with any interest is familiar with system monitoring tools. Task Manager (or if you’re really cool Process Explorer) is the first place Windows users look when their system starts to crawl, Mac users turn to Activity Monitor, and I’m sure Linux users have some really obscure command-line tool as well. These coarse grained tools are usually enough for users, however, an operations team needs a higher degree of visibility into application — particularly if they are expected to determine how to tune the application for better performance.

For operations on Windows, Performance Monitor provides detailed information for running applications in real-time. On a web server, it is easy to find out how many threads your ASP.NET application is using, as well as how many requests are queued. That information can be correlated with processor utilization to help determine if the bottleneck is the CPU, the network, or possibly even the database server. When it comes to troubleshooting issues on a live system, more information is always helpful to determine the source of the problem.

To support this level of visibility in MassTransit, performance counter support has been added. Performance counters in .NET are part of the System.Diagnostics namespace. There are various counter types that can be defined, including counts, rates, and averages. When an application wants to output performance counters, it has to create a category and specify the counters that are included in the category. For instance:

ConsumerThreadCount = new RuntimePerformanceCounter("Consumer Threads",
	"The current number of threads processing messages.",
	PerformanceCounterType.NumberOfItems32);

ReceiveRate = new RuntimePerformanceCounter("Received/s",
	"The number of messages received per second",
	PerformanceCounterType.RateOfCountsPerSecond32);

These are two of the counters defined by the MassTransit category. The first is a count that is updated when the number of threads in use changes. The second is a rate which gets incremented once for every message received. The actual calculation and display of the rate is handled by the performance monitoring tools – the application does not need to calculate it.

ConsumerDuration = new RuntimePerformanceCounter("Average Consumer Duration",
	"The average time a consumer spends processing a message.",
	PerformanceCounterType.AverageCount64);

ConsumerDurationBase = new RuntimePerformanceCounter("Average Consumer Duration Base",
	"The average time a consumer spends processing a message.",
	PerformanceCounterType.AverageBase);

This counter is used to report the average consumer duration of a message. For an average, two counters are used. One is the base which is incremented for each occurrence and the counter is the actual count that is added. So for each message, the base is incremented once and the duration is incremented by the amount of time spent executing the consumer.

In adding performance counter support, I wanted to do it in a way that didn’t leak the details of updating performance information throughout the framework. It was at this point that I turned to the Magnum Pipeline. Using the pipeline to publish the metrics allowed me to isolate the actual performance counter interface to a single method in a single class for the service bus. So instead of passing interfaces around all the components that make up the bus, a single event aggregator is passed instead. When you start up the bus, the performance counter code subscribes to the events as shown:

_eventAggregatorScope.Subscribe<MessageReceived>(message =>
	{
		_counters.ReceiveCount.Increment();
		_counters.ReceiveRate.Increment();
		_counters.ReceiveDuration.IncrementBy((long) message.ReceiveDuration.TotalMilliseconds);
		_counters.ReceiveDurationBase.Increment();
		_counters.ConsumerDuration.IncrementBy((long) message.ConsumeDuration.TotalMilliseconds);
		_counters.ConsumerDurationBase.Increment();
	});

Now, when the bus receives a message, it sends the event to the event aggregator (an instance of the Magnum Pipeline).

var message = new MessageReceived
	{
		MessageType = messageType,
		ReceiveDuration = receiveTime,
		ConsumeDuration = consumeTime,
	};

	_eventAggregator.Send(message);

Since the Magnum Pipeline is publish/subscription, additional consumers could also opt-in to the MessageReceived event and perform other actions as well. I also plan to add counters per message type, allowing a finer grained view at message counts and consumer durations.

While the main story behind this post is the new counters available in MassTransit, my hope is that this brief introduction to performance counters was useful as well. You can learn more about performance counters from various articles that have been posted (such as a good one on CodeProject). You can check out the Magnum Pipeline in the Magnum project which is hosted at GoogleCode.

So you bought a Mac, and you are staring at a fairly lonely dock wondering what other goodies you can add. Sure, the Mac comes with a lot of great software, including Snow Leopard, iTunes, iPhoto, iMovie, GarageBand, iWeb, iChat, and more but there has to be some other great stuff too right?

Here is my list of (mostly free) software to add to your Mac to take it up a notch:

• Skype
  The best way to make phone calls on your Mac
• Quicksilver
  Free fast application launcher and more
• HandBrake
  Don’t let those old DVDs go to waste, use HandBrake to rip them to your iTunes library (and play them on your TV or iPhone)
• Tweetie
  If Tweetie for the iPhone is good, Tweetie for the Mac is an even better Twitter client
• Evernote
  Keep track of everything and sync it automatically between Mac, Windows, iPhone, and web. Includes character recognition on images for searching text on pictures taken with your iPhone
• Transmission
  The best torrent client for the Mac
• Cyberduck
  FTP, WebDAV, iDisk, whatever this client is fast, free, and easy to use
• Colloquy
  IRC isn’t dead yet, and this free client works great on the Mac and iPhone
• Adium
  The best IM client on the Mac, works with just about anything that chats

And then there are the less often useful, but sometimes helpful ones:

• KisMAC
  Because sometimes you really, really need a hot spot
• MacTheRipper
  For those stubborn DVDs that won’t rip with HandBrake

Those are all the free ones, I’ll post another list of all the paid items that I find myself using nearly every day soon.