Load Balancing vs. Failover Clustering

A pending post from long time!!! At distance both look quite similar and are point of confusion for many. Rationale though is making Load Balancing address scalability while Failover Clustering address high availability. Load Balancing is all about improvising performance while Failover Clustering is improvising uptimes mitigating system failures. Another difference is, you would find Load Balancing happening at web/application servers (stateless hopefully) and failover clustering at database servers (state full). Industry seems to be using word “Cluster” (set of connected nodes) for both – but with different intents of Load & Failover.

Both are also separate things in terms of configuration & setup. For instance, Windows 2003 (currently that’s what we have in our production) has separate options for Load Balancing & Clustering. Windows recommendation is not to mix both, i.e. you shouldn’t cluster machines for failover which are already configured for load balancing.

Setting up load balancing is simple – you need couple of machines connected to a common network and an additional IP where clients would connect to. This Virtual IP where the requests are made by clients, is in turn is used for Load Balancing nodes that part of this cluster (load balancing cluster).

Setting up failover clustering on the other hand is little complex. You need 2 networks a public and private (hear beat), a shared drive (called Quorum), and an additional Public IP (in addition to minimum – 2 public and 2 private IPs that 2 systems will have). Remember, creating a failover cluster at Windows level is a primary requirement to build a failover SQL Server cluster. Reason to create a Windows level cluster is install required cluster services and create cluster groups (logical collection of nodes). You can select a cluster group (obviously at least 2 nodes should be part of this group) and configure SQL Server Cluster or anything else on top it. SQL Server Cluster would require an additional IP, another shared disk for installation & database files (this disk is a shared resource for chosen cluster group), domain account (that has administrative privileges on all nodes) & group on that domain which that account has complete access. Shared Quorum and Shared Disk are normally part of SAN storage. I have also come across quite a few implementations using Starwind or similar tools to create these shared iSCSI targets in form of virtual disks (.img). It might be helpful to know that Windows 2003 doesn’t have the iSCSI initiator built-in and you can download the same from here.

Hope above helps to some extent .

Implementing Audit Trail for your application

I recently came across Davy’s post where he talks about implementing auditing. Davy talks about 3 requirements which I guess are quite common for audit trail:

1) We have to be able to easily retrieve what user X changed in the database on day Y or during a certain time frame.
2) We need to know what record X looked like at point Y in time, and be able to show a historic list of changes for a specific record.
3) We need to configure the duration for which auditing data must be kept in the database on a per-table (or per-schema) level

In this post I would like to talk about the way we did it for one of our recent projects. We had prepared a generic sort of table which would hold the audit data. This table mainly consists of AuditID, EntityID, EntityTypeID, UserID, AuditTime, AuditLevel & AuditData. AuditData is a TEXT column (we support SQL 2000) holding XML structure, for those changes made to an entity by a given user in form of old and new values, at a level specified. This would cater to the first requirement as it’s quite easy to search for activities of a given user.

Second requirement is little tough, as with this model we don’t allow direct queries against record attributes. But this can be achieved by 2 variants. First simple solution could be you have to search by entity type & a specific entity in turn for a date range & browse through records to see changes done on them. Second option is take this AuditData out and restructure its layout with help of OLAP cubes to produce meaningful quick search.

Third requirement is very much there for us in order to control size of this massive table but we for simplicity purge entire database at a given time interval & do not make it adjustable for each entity. Though I feel providing that flexibility shouldn’t be difficult considering the design we have laid out.

A final thing I wanted to highlight is about generating the XML data itself. It’s always better to do such processing in .NET code & if you have been using DataSets you get old values extraction feature out of box. We are yet to see a significant performance degradation due to this design but as a contingency plan I was thinking of making this XML conversion & insertion into AuditDB an asynchronous activity with help of Queue like data structure.

Look forward to read your thoughts on above.

Whoever saves one life, saves the world entire

Title of this post happens to be from one of my all time favourite movies – Schindler’s List, but post is about my newly created cloud application using Azure Platform. My cloud application – EmergencyBloodBank is finally up. Here’s the direct link to application. This application was created was Cloud App Contest.

There were 2 major driving factors for this application – one to create an application covering all offerings of Azure platform that can help community members to boot start their Azure efforts and second was coming up with scenario that effectively leverages cloud services displaying no elements of over engineering (which can make solution look artificial). My initial thought was to create a shared calendar service or streaming live video but there were few existing compelling solutions already there. So, I decided to focus on healthcare segment and immediately could see the impact on blood bank field. An initial search on internet landed me here and that was it, I was all set to create a communication platform for sharing blood related requirements that leverages cloud infrastructure and services.

First step was to create a site with 99.999% uptime which is a necessary requirement when talking about blood bank sort of service. Site is built on ASP.NET 3.5 with C# as programming language, and this is where one can post their blood related requests. Next was using Live Services for authentication. How many times you have turned away from an excellent site due to its lengthy login form? I have and I never wanted the end users of this site to do that. So using Live Services was a natural fit with a large user base – apprx. 500 million users. So challenge was how we make these requests reach potential donors? Pull model is not effective in this case (imagine the load it would put on site or a web service) and .NET Service Bus provides the push model I was looking for. So I went ahead with it and created a WPF client which gets the relayed messages from Azure worker role via .NET Services Bus. Few things that I opted out of due to time constraints were providing a SMS service, porting WPF client to Windows mobile, & integration with web 2.0 applications like FaceBook, Twitter, etc. Finally task was to store the user details so they don’t have to key them everytime they visit site & also audit all blood requests they place. SQL Azure with its SaaS offering was a natural fit because the data here is nothing more than a KB per request, so one can easily forgo buying the product and use the available SaaS model (add to that almost zero percent of installation & maintenance effort). This would help maintain the site cost within 100$ per month providing a unique high available solution. Below is the architecture diagram for the application

You can access the acutal solution document here.

Let me know your thoughts if I could have done things differently. I plan to cover the entire architecture, design & development process in one of upcoming talks.

Till then, Happy Clouding .

(P.S. My Application has been selected as winner of the Azure Cloud App Contest. Thanks to everybody who voted and provided their valuable feedback).

MVC vs. MVP vs. MVVM

An important FAQ. The answer actually depends on where the person is coming from. MVC is a fundamental pattern which has been tweaked quite a bit to fit into various platforms. For instance if you had asked anybody how to implement an MVC in ASP.NET (prior to release of ASP.NET MVC framework) you would get very different answers. So let’s start with basic. The common motivation behind all 3 is separation of concerns, cutting flab from UI (good for UI designers), swapping UIs (for instance windows to web), make UI easy for Unit Testing, etc. Have a look at the below diagram, I have taken it from CAB documentation.

MVC: Three components – View (your UI), Model (your business entities / data – that view is displaying) & Controller (contains the logic that alters the model depending on the action triggered by UI, typically implementing a Use Case). It’s widely known that MVC is a compound pattern (View and Controller have Strategy implementation, View itself can be a Composite implementation & View and Model are synched through Observer). In this case Controller doesn’t know anything about View, and the idea is that a View can switch Controllers (for instance depending upon who has logged to the system) & a single controller can be used by multiple Views. View subscribes to the changes done to the model & hence both are sync from the data perspective. One of the disadvantages of MVC is that it’s difficult to unit test. Controller manipulates the data but how about asserting those changes from a view perspective. For instance on click of a button you raise an event to controller, and controller modifies the value in model. This value modification changes the font size / color in View. Unit testing this scenario is slightly difficult in MVC.

MVP: Again three components. But dependencies change (look at arrows in the diagram). Over here we replace Controller with Presenter (one which presents the changes done in model back to view). The main difference between both is that Presenter refers back to the view while Controller doesn’t. Normal pattern found here is to create an abstraction of the View (in terms of properties / events) & Presenter refers to it. This makes the mocking of View much easier (also model sync is with presenter) & hence the Unit Testing aspect. Presenter here hence takes the responsibility of not only manipulating model but also updating the view. Of course the implementations of MVP differ in real world in terms of how much thin the view is, some prefer keeping basic logic still inside view & taking complex logic in presenter, while others prefer keeping the entire logic in Presenter. Martin fowler describes 2 variations on MVP on these lines namely – Supervising Controller & Passive View described below

(A Passive View handles this by reducing the behavior of the UI components to the absolute minimum by using a controller that not just handles responses to user events, but also does all the updating of the view. This allows testing to be focused on the controller with little risk of problems in the view.

Supervising Controller uses a controller both to handle input response but also to manipulate the view to handle more complex view logic. It leaves simple view behavior to the declarative system, intervening only when effects are needed that are beyond what can be achieved declaratively.)

MVVM: Model–View-ViewModel talks of creating a new model (in addition to your domain model). This model normally adds additonal properties from the prespective of View (as we understand that View has controls in addition to data which it’s displaying). For instance if View had a property IsChecked and Presenter was setting in classic MVP, in MVVM Presenter will have that IsChecked Property which View will sync up with (doesn’t it look like Startegy pattern has been replaced with Observer?). So now a Presenter becomes more like a combo of – View Properties & Model properties which would be synchronized with View. So why not rename Presenter to ViewModel? Do that and you get MVVM. MVVM is attractive for platforms which support bi-directional binding with less effort. Also a minor tradeoff is ViewModel unlike Presenter can stand on its own (Presenter normally requires a View’s interface). Martin fowler describes similar pattern called Presentation Model & Josh Smith captures MVVM implementation for WPF / Silverlight in this article.

ASP.NET MVC: So what has MVC got to do with ASP.NET MVC? First, Web works on a different model. Here, user interacts with HTML in browser and send a request back to the server for processing (for client side Ajax you might go just for data). As the interaction is normally stateless, when the request comes back to the server we need to recreate our View, load the model back & manipulate both of them as required. There are 2 variations on how handle this recreation – Page Controller & Front Controller. Make Page the decision maker – in this widely implemented pattern HTTP request is specific to physical page on server (.aspx for instance) & page in turn creates itself (builds the view from postback data) decides what model it needs and triggers the manipulation (events in codebehind file) it requires. As you see here the distinction between View & Controller becomes blur & is little difficult to separate. This where ASP.NET MVC comes in which behaves like a Front Controller – where Controller is the decision maker. Here all HTTP requests are mapped to methods on the Controller Class. Controller class recreates the model & view as required and does the manipulations. This makes unit testing easier as we can directly instantiate the front controller class & invoke methods on it to perform the assertions.

I can add code snippets to above explanations if you feel they would help you understand things better. I will look forward to your comments .

How EBay does it?

I stumbled on this excellent presentation by Randy Shoup. A must watch.

Enhancing Team’s Productivity – .NET, VS.NET

Software projects normally have a scary deadline. One way to meet them is to increase your team’s productivity. I am going to jot down few practices I have been following, & look forward to read the ones you follow:

1) Using Macros for VS.NET repetitive tasks: I normally have a complex VS.NET folder Structure & this structure gets repeated for every module added. For instance creating a new module in my current solution requires you to create about 10 folders, starting from DTO, Factories, Domain Model, Exceptions, etc. This took quite a bit of time of developers about start a new module. So I created a VS.NET macro to automate this task.

2) Code Generators: I have seen many companies having a framework wherein they take care of common tasks starting from transactions to Workflows, etc. Their implementation normally revolves around Factory method or Template method patterns. If you follow this approach, using code generator you can create “Fill-In-The-Blanks” template for your developers. I don’t have a framework like this, but I have still created few code generators. I normally use NHibernate for my projects. Given it’s a fantastic tool, it takes sometime to generate mappings & corresponding classes which again might be error prone. I have created a code generator which not allows you to create mappings & classes but also creates Factories & Repositories for entities that are generated. (N.B. You can find one alternative approach to mapping files here).

3) Extracting Code Snippets: I use Snippet Designer a lot to extract common code patterns & insert them with zero effort. This definitely saves a lot of valuable time.

4) Tools: Though they cost we can’t program without them – ReSharper, CodeRush, etc. I personally use ReSharper. You can also check out plugins available for these tools. You can find a good plugin here.

5) Design Techniques: I normally prefer making cross cutting concerns oblivious to developers. This includes things are Security, Transactions, Logging, etc. They help in reducing lot of developer’s key strokes. You catch my recent article on same here. You can also use techniques like Visual Inheritance, though it’s tough to get it right with WPF (for a way out you can look here).

6) Keyboard Shortcuts: Although minor this can save lot of mouse clicks. The ones I use most frequently are CTRL + K + D (Alignment), CTRL + K + C (Comment), CTRL + K + U (UnComment), F12 (Navigate to a type) (though I prefer ReSharper shortcuts over above). An awesome way to speed up with XAML editing in VS.NET is this.

7) Proper Training: I guess all of us understand the importance of this but the level to which we get it is always less. I would also recommend productivity trainings once you master the basics of a technology / framework. I have written about one such technique here.

8 ) Appropriate Hardware: I pity the developers running on 1 GB Ram with VS.NET 2008, SQL Server, Oracle, etc. (even 2 GB is less).

9) Shared knowledge base: When working with a team you find many issues are recurring. A issue solved today by one developer is faced tomorrow by another developer. Keeping a shared knowledge base for team definitely boosts productivity.

10) Builds that always work: Integration of local working copies can consume a lot of valuable time of your projects. Normally developers have a tendency to get away with their check in. Have an hour per day where developers can integrate their work (or you can have couple per week). You can automate the build to save more time, but I leave it as a personal choice considering competency & cost involved.

11) Holidays/Working hours/Re-creation: Hmm… May be I am getting into aspects which I shouldn’t be not to mention salary. So I will stop here.

Let me know what you do to gear up your team’s producitivity .

HAPPY NEW YEAR

Happy new year, Guys . Sorry for being late on this one. I intended to first publish a long pending article as a small new year gift. Finally the article is up & you can see it here. It talks about applying AOP with Unity on NHibernate. It doesn’t contain ground breaking stuff, but it was exciting personally to connect these diverse pieces. Hope Unity would bring DI & AOP into the mainstream of .NET development (I have used Spring.NET in past for these techniques).

As part of new year I have chosen new hobbies, Photography & Automobiles to be precise. I guess its imperative to start each year with a new productive hobby which stimulates your mind towards a new joy. I will have few posts on my new hobbies soon.

Though I started this blog only in June (previous was here) below are my top posts (going by number of views) for last year:

Total Posts: 27 ( – Improvisation required)

1) Debugging an XBAP / WCF application – Windows Vista + VS.NET 2008
2) Issues using Workflow with Code Separation (.xoml)
3) Installing Team Foundation Server (TFS) on Windows 2008
4) WCF Certificate Security with XBAP / IIS Issues
5) Logging Best Practices

Keep Rocking .

Capacity Planning vs. Hardware Sizing

I have been recently working on business proposals which require me to architect solutions starting from layers to deployment. This is where I crashed into these 2 terms: Capacity Planning & Hardware Sizing. I came across an accurate difference here which I would like to quote straight away:

“decide whether you need to do capacity planning or hardware sizing, you can’t do both, at least not at the same time. In capacity planning the software and hardware are constant while the workload varies (ie, given a particular system, how much work can it do?). In hardware sizing the software and workload are constant while the hardware varies (ie, given a particular amount of work, what’s the least-costly system that can handle the workoad in the specified performance constraints?).”

I am going to edit this space as my thoughts mature.

Food for thought – Designing Presentation Layer

I was having discussion with a fellow architect where he shared an idea with me. It was pertaining to automating forms creation in presentation layer. Normally there are techniques like visual inheritance relevant here. But what he did is created few custom controls. Specialty of these controls is they take an object graph and automatically build the UI depending on the properties and associations of the object.

Benefits -

1) If tomorrow you update your object model with additional properties you don’t have to alter your UI as its auto generated.

2) You update your custom control UI changes throughout the application.

This would normally require you to map every primitive type to a UI control, but what he did is created his own custom types by inheriting from primitive types & these custom types would then map to the corresponding UI control. Though he has a successful running application build on this framework I am yet to try it. Let me know if you have similar ideas (can I call this a pattern ???) for UI or if you use this one in your project.