Explicit Interface Implementation and WCF

You are inheriting from 2 contracts to create a contract. Both contracts that you are inheriting from have a method with same name and signature. You don’t have much control over these interfaces as they are tightly coupled with your subsystem. You create an implementation class with help of explicit interface implementations and dispatch calls to subsystem. Now you have a challenge of invoking the right implementation from client side? Will WCF work even when we can’t have any config files? Fortunately YES, WCF supports such scenarios. Below is a code sample on same, hope it helps you when you have a sharp knife against your throat :

//Contracts
[ServiceContract]
public interface IIPV4Location
{
[OperationContract]
string GetLocationId();
}

[ServiceContract]
public interface IIPV6Location
{
[OperationContract(Name = "IIPV6Location")] //A must for WCF to distinguish
string GetLocationId();
}

[ServiceContract]
public interface ILocation : IIPV4Location, IIPV6Location
{ }

//Server Side
class LocationImpl : ILocation
{
string IIPV4Location.GetLocationId()
{
return "Ipv4";
}

string IIPV6Location.GetLocationId()
{
return "Ipv6";
}
}

class Program
{
static void Main(string[] args)
{
ServiceHost host = new ServiceHost(typeof(LocationImpl), new Uri("http://localhost:9999/"));
var behavior = new ServiceMetadataBehavior();
behavior.HttpGetEnabled = true;
host.Description.Behaviors.Add(behavior);

host.AddServiceEndpoint(typeof(ILocation), new BasicHttpBinding(), "Location");

host.Open();
Console.WriteLine("Running...");
Console.ReadLine();
host.Close();
}
}
//Client Side
class Program
{
static void Main(string[] args)
{
ILocation location = new ChannelFactory(new BasicHttpBinding(),
"http://localhost:9999/Location")
.CreateChannel();

var ipv4Location = ((IIPV4Location)location).GetLocationId();
Console.WriteLine(ipv4Location);

var ipv6Location = ((IIPV6Location)location).GetLocationId();
Console.WriteLine(ipv6Location);
}
}

Load Balancing vs. Failover Clustering

A pending post from long time!!! At distance both look quite similar and are point of confusion for many. Rationale though is making Load Balancing address scalability while Failover Clustering address high availability. Load Balancing is all about improvising performance while Failover Clustering is improvising uptimes mitigating system failures. Another difference is, you would find Load Balancing happening at web/application servers (stateless hopefully) and failover clustering at database servers (state full). Industry seems to be using word “Cluster” (set of connected nodes) for both – but with different intents of Load & Failover.

Both are also separate things in terms of configuration & setup. For instance, Windows 2003 (currently that’s what we have in our production) has separate options for Load Balancing & Clustering. Windows recommendation is not to mix both, i.e. you shouldn’t cluster machines for failover which are already configured for load balancing.

Setting up load balancing is simple – you need couple of machines connected to a common network and an additional IP where clients would connect to. This Virtual IP where the requests are made by clients, is in turn is used for Load Balancing nodes that part of this cluster (load balancing cluster).

Setting up failover clustering on the other hand is little complex. You need 2 networks a public and private (hear beat), a shared drive (called Quorum), and an additional Public IP (in addition to minimum – 2 public and 2 private IPs that 2 systems will have). Remember, creating a failover cluster at Windows level is a primary requirement to build a failover SQL Server cluster. Reason to create a Windows level cluster is install required cluster services and create cluster groups (logical collection of nodes). You can select a cluster group (obviously at least 2 nodes should be part of this group) and configure SQL Server Cluster or anything else on top it. SQL Server Cluster would require an additional IP, another shared disk for installation & database files (this disk is a shared resource for chosen cluster group), domain account (that has administrative privileges on all nodes) & group on that domain which that account has complete access. Shared Quorum and Shared Disk are normally part of SAN storage. I have also come across quite a few implementations using Starwind or similar tools to create these shared iSCSI targets in form of virtual disks (.img). It might be helpful to know that Windows 2003 doesn’t have the iSCSI initiator built-in and you can download the same from here.

Hope above helps to some extent .

PPTX – Leveraging Windows Azure – Virtual Tech Days

Dear All, thanks for attending my session. Hope I could do some justice to your precious time . For my past attendees, session could have been slightly plain as VS.NET was not involved at all, but I still believe attending it should have kick started your thought process on leveraging Cloud Computing. You can download the presentation from Skydrive. A small confession – there were few demos (including EmergencyBloodBank) planned for the session, but may be during those hours Azure was getting ready for PDC and everything was quite flaky . In fact all my .NET Service Bus solutions (Namespaces) were unavailable. So, I took the decision of discussing more scenarios and in hindsight it also looks more appropiate too. You might want to keep a eye on VTD Site as session recordings would be shortly available. I will look forward to hear your thoughts on the session. Till next time, Phir milenge Chalte Chalte. Love you all!!!

Leveraging Windows Azure – @VTD

It’s becoming difficult for me to keep posting about my talks as there have been many recently . In Last 2 – 3 weeks I gave a talk on cloud computing for a local college in Bangalore, then did WCF / WF 4.0 for Community Tech Days, and .NET Service Bus Architecture session for Microsoft Tech Days @Wipro. Today, I will be talking about leveraging Windows Azure for your enterprise @Virtual Tech Days. Allow me to provide a brief about this session. Many of us have a decent understanding of what Cloud is all about, but are still stuck on where and how we can get started with it. This session helps you build a practical approach, making you familiar with those migration patterns that other are leveraging. Session also talks about Azure practicies and challenges involved around these patterns. To catch this session live logon to VTD site. I hope this would help you with your cloud efforts.

Implementing Audit Trail for your application

I recently came across Davy’s post where he talks about implementing auditing. Davy talks about 3 requirements which I guess are quite common for audit trail:

1) We have to be able to easily retrieve what user X changed in the database on day Y or during a certain time frame.
2) We need to know what record X looked like at point Y in time, and be able to show a historic list of changes for a specific record.
3) We need to configure the duration for which auditing data must be kept in the database on a per-table (or per-schema) level

In this post I would like to talk about the way we did it for one of our recent projects. We had prepared a generic sort of table which would hold the audit data. This table mainly consists of AuditID, EntityID, EntityTypeID, UserID, AuditTime, AuditLevel & AuditData. AuditData is a TEXT column (we support SQL 2000) holding XML structure, for those changes made to an entity by a given user in form of old and new values, at a level specified. This would cater to the first requirement as it’s quite easy to search for activities of a given user.

Second requirement is little tough, as with this model we don’t allow direct queries against record attributes. But this can be achieved by 2 variants. First simple solution could be you have to search by entity type & a specific entity in turn for a date range & browse through records to see changes done on them. Second option is take this AuditData out and restructure its layout with help of OLAP cubes to produce meaningful quick search.

Third requirement is very much there for us in order to control size of this massive table but we for simplicity purge entire database at a given time interval & do not make it adjustable for each entity. Though I feel providing that flexibility shouldn’t be difficult considering the design we have laid out.

A final thing I wanted to highlight is about generating the XML data itself. It’s always better to do such processing in .NET code & if you have been using DataSets you get old values extraction feature out of box. We are yet to see a significant performance degradation due to this design but as a contingency plan I was thinking of making this XML conversion & insertion into AuditDB an asynchronous activity with help of Queue like data structure.

Look forward to read your thoughts on above.

Whoever saves one life, saves the world entire

Title of this post happens to be from one of my all time favourite movies – Schindler’s List, but post is about my newly created cloud application using Azure Platform. My cloud application – EmergencyBloodBank is finally up. Here’s the direct link to application. This application was created was Cloud App Contest.

There were 2 major driving factors for this application – one to create an application covering all offerings of Azure platform that can help community members to boot start their Azure efforts and second was coming up with scenario that effectively leverages cloud services displaying no elements of over engineering (which can make solution look artificial). My initial thought was to create a shared calendar service or streaming live video but there were few existing compelling solutions already there. So, I decided to focus on healthcare segment and immediately could see the impact on blood bank field. An initial search on internet landed me here and that was it, I was all set to create a communication platform for sharing blood related requirements that leverages cloud infrastructure and services.

First step was to create a site with 99.999% uptime which is a necessary requirement when talking about blood bank sort of service. Site is built on ASP.NET 3.5 with C# as programming language, and this is where one can post their blood related requests. Next was using Live Services for authentication. How many times you have turned away from an excellent site due to its lengthy login form? I have and I never wanted the end users of this site to do that. So using Live Services was a natural fit with a large user base – apprx. 500 million users. So challenge was how we make these requests reach potential donors? Pull model is not effective in this case (imagine the load it would put on site or a web service) and .NET Service Bus provides the push model I was looking for. So I went ahead with it and created a WPF client which gets the relayed messages from Azure worker role via .NET Services Bus. Few things that I opted out of due to time constraints were providing a SMS service, porting WPF client to Windows mobile, & integration with web 2.0 applications like FaceBook, Twitter, etc. Finally task was to store the user details so they don’t have to key them everytime they visit site & also audit all blood requests they place. SQL Azure with its SaaS offering was a natural fit because the data here is nothing more than a KB per request, so one can easily forgo buying the product and use the available SaaS model (add to that almost zero percent of installation & maintenance effort). This would help maintain the site cost within 100$ per month providing a unique high available solution. Below is the architecture diagram for the application

You can access the acutal solution document here.

Let me know your thoughts if I could have done things differently. I plan to cover the entire architecture, design & development process in one of upcoming talks.

Till then, Happy Clouding .

(P.S. My Application has been selected as winner of the Azure Cloud App Contest. Thanks to everybody who voted and provided their valuable feedback).

.NET Worker Threads, I/O Threads And Asynchronous Programming

Before I talk about Asynchronous programming, I will outline type of threads available in .NET environment. CLR maintains a pool of threads (to amortize thread’s creation cost) and it should be the place to look for when you need to create more threads. Thread pool maintains 2 types of threads – Worker & I/O. As name implies Worker threads are computational threads while I/O are used for wait (block) of long duration (e.g. when invoking a remote WCF service). A good rule to follow is to ensure your all waits are on I/O thread, provided they are long enough otherwise you would end up degrading your performance due to a context switch. Let’s understand what above statement means with a bit of code. Your entry point to CLR’s thread pool is ThreadPool class. Code below helps you find how many threads are there in thread pool

int wt, iot;
ThreadPool.GetAvailableThreads(out wt, out iot);
Console.WriteLine(“Worker = ” + wt);
Console.WriteLine(“I/O = ” + iot);

One can only request for a worker thread by ThreadPool.QueueUserWorkItem. If all worker threads are occupied your request will be blocked till time a worker thread is available. Below sample shows how you can consume & monitor worker threads.

int wt, iot;
for (int i = 0; i < 10; i++)
{
ThreadPool.QueueUserWorkItem(Dummy);
}
Console.ReadLine();
ThreadPool.GetAvailableThreads(out wt, out iot);
Console.WriteLine("Worker = " + wt);
Console.WriteLine("I/O = " + iot);

static void Dummy(object o)
{
Thread.Sleep(50000000);
}

Unlike worker threads you don’t have any direct API to request for I/O threads. But .NET leverages I/O threads automatically when we use asynchronous programming. Below code uses asynchronous operations on client side to invoke a WCF Service

for (int i = 0; i < 10; i++)
{
client.BeginGetData(10, null, null); /*Put some Thread.Sleep in server side code*/
}
ThreadPool.GetAvailableThreads(out wt, out iot);
Console.WriteLine("Worker = " + wt);
Console.WriteLine("I/O = " + iot);

Contradictory to expectations of many, the output of the above program shows that only one I/O thread is in use instead of 10. This optimization makes I/O so important and something you should care about. As a good designer of your application you want to ensure that minimal threads are blocked and I/O threads just give you that. Also note when call returns another I/O thread is picked up from pool and callback method gets executed, hence we should avoid touching UI from callback method.

So why should one use Asynchronous programming? There are quite a few reasons but what I also see is many programmers miss on server side asynchronous programming & focus just on client aspect of it. Also irrespective of whether you are doing client or server side I strongly recommend you set timeouts for your I/O operations (WCF defaults this to 60 seconds).

Client Side:

1) You don’t want to block UI while processing the request with server (though I have found that this creates other issues like handling UI parts that are only to be activated post request, accessing UI from callback, exception handling, etc.)

2) You want to issue simultaneous requests to your server so that all of them are processed in parallel and your effective wait time is the wait time of longest request (though in case you are making requests to same server you can try the DTO pattern to create a chunky request & break the request up on server for parallel processing)

3) You just want to queue up a request (message queue)

Server Side:

1) Normally there is throttle placed on server side handling. If you sever side code is I/O oriented (Calling long processing DB query / remote web service) you are better off doing that on an asynchronous thread. For more information you can refer to ASP.NET link and WCF link.

I haven’t mentioned about asynchronous ADO.NET in this post and you can find more information about it here. So what are your scenarios for using asynchronous programming?

Performance Testing – Response vs. Latency vs. Throughput vs. Load vs. Scalability vs. Stress vs. Robustness

Normally I find quite a bit of ambiguity when people talk about performance tests, some restrict it to response time whereas some use it to cover a gamut of things they are testing or measuring. In this post, I will put across few thoughts on contrasting between them. Ideally a lot depends on what you are trying to measure. The terms that you will frequently hear in this arena are – Response Time, Latency, Throughput, Load, Scalability, Stress, Robustness, etc. I will try explaining these terms below also throwing some light on how can you measure them.

Response Time – Amount of time system takes to process a request after it has received one. For instance you have API and you want to find how much time that API takes to execute once invoked, you are in fact measuring response time. So how do we measure them? Simple use a StopWatch (System.Diagnostics) – start it before calling API & stop it after API returns. The duration arrived here is quite small so a preferred practice is to call that API in sequential loops say 1000 times, or pass variable load to the API if possible (input/output varies from KBs/MBs/GBs e.g. returning customer array of varied lengths).

Latency – In simplest terms this is Remote Response time. For instance, you want to invoke a web service or access a web page. Apart from the processing time that is needed on the server to process your request, there is a delay involved for your request to reach to server. While referring to Latency, it’s that delay we are talking about. This becomes a big issue for a remote data center which is hosting your service/page. Imagine your data center in US, and accessing it from India. If ignored, latency can trigger your SLA’s. Though it’s quite difficult to improve latency it’s important to measure it. How we measure Latency? There are some network simulation tools out there that can help you – one such tool can be found here.

Throughput – transactions per second your application can handle (motivation / result of load testing). A typical enterprise application will have lots of users performing lots of different transactions. You should ensure that your application meets the required capacity of enterprise before it hits production. Load testing is the solution for that. Strategy here is to pick up a mix of transactions (frequent, critical, and intensive) and see how many pass successfully in an acceptable time frame governed by your SLAs. How to measure it? You normally require a high end professional tool here like Visual Studio Team System (Load Testing feature). Of course, you can try to simulate load through your custom made applications /code but my experience says custom code are good to test for response times; whereas writing custom code for load testing is too much of work. A good load testing tool like VSTS allows you to pick a mix of transactions, simulate network latency, incorporate user think times, test iterations, etc. I would also strongly recommend this testing to be as close as possible to real world with live data.

Scalability – is the measure of how your system responds when additional hardware is added. Does it take new increased load by making use of added resources? This becomes quite important while taking into consideration the growth projections for your application in future. Here we have two options – scale vertically/up (better machine) or horizontally/out (more machines), latter is usually more preferred one. A challenge to scale out is to ensure that your design doesn’t have any server affinity, so that a Load balancer can adjust load across servers. Measuring scalability can be done with help of load balancing tools with a software/hardware NLB in place ensuring system is able to take on new load without any issues. One can monitor performance counters to see whether actual request load has been balanced/shared across servers (I plan to cover NLB in a future post).

Stress testing – Many people confuse this or relate it to load testing. My take which I have found easy to explain is, if you find yourself running tests for more than 24 hours you are doing a stress test. Motivation behind stress test is to find out how easily your system can recover from over loaded (stressed) conditions. Does it limp back to normalcy or gives up completely? Robustness an attribute that is measured as part of stress testing relates to long running systems with almost negligible down time. A simple example here could be memory leak. Does your system release memory after working at peak loads? Another, what happens if a disk fails due to constant heavy I/O load? Does your system lose data? Finding and addressing such concerns is motivation behind stress testing.

I will look forward to read your thoughts on above .

WCF Serializers – XmlSerializer vs. DataContratSerializer vs. NetDataContractSerializer

I talked about encodings in a previous blog post, in this one I will talk about Serializers. Have a look at the below diagram which depicts WCF architecture in its simplest form.

As you can see serializers convert a .NET Object to WCF Message (XML Infoset) whereas Encoder convert that WCF Message into byte stream. Serializers are governed by Service Contracts whereas Encoders are specified through endpoint’s binding. There are 3 serializers supported by WCF – XmlSerializer, DataContractSerializer & NetDataContractSerializer. DataContractSerializer is the default one and should be used always unless there are backward compatibilities required with ASMX / Remoting. Let’s explore each of serializers in turn.

XmlSerializer, I guess most of us are familiar with it coming from ASMX world, is an opt-out serializer. By default this serializer takes only public fields, properties of a given type & sends them over wire. Any sensitive data must be explicitly opted out using XmlIgnore attribute. The advantage of XmlSerializer is the amount of flexibility it gives in controlling layout of XML Infoset (schema driven), and this sometimes might be required due to compatibility requirements with existing clients. Choosing XmlSerializer over default DataContractSerializer is quite easy, just use XmlSerializerFormat attribute along with your ServiceContract or OperationContract.

[ServiceContract]
//[XmlSerializerFormat] – Could be applied here for all contracts
interface IBank
{
[XmlSerializerFormat]
[OperationContract]
Customer GetCustomerById(int id);
}

DataContractSerializer – When to moving to WCF world, Microsoft seems to have decided to give more focus on versioning contracts then creating contracts. DataContractSerializer which is default serializer doesn’t allow us the flexibility of XmlSerializer for layout of XML Infoset (though you can still serialize a type implementing IXmlSerializable), but provides good versioning support with help of Order, IsRequired attributes & IExentsibleDataObject interface. Also there is a myth that DataContractSerializer only supports types decorated with DataContract/DataMember & Serializable attributes in reality though there is a programming model supporting a range of types outlined here including Hashtables, Dictionary, IXmlSerializable, ISerializable and POCO’s. There is an attribute DataContractFormat which can allow a mix with XmlSerializer. Note that if you don’t apply any attributes at all, DataContractFormat is applied by default.

[ServiceContract]
[XmlSerializerFormat] //Serialize everything using XmlSerializer
interface IBank : IExtensibleDataObject
{
[DataContractFormat] //Override with DataContractSerializer
[OperationContract]
Customer GetCustomerById(int id);
}

Finally NetDataContractSerializer. To many this is a distant concept. Let me try explaning this with a example:

[ServiceContract]
public interface IAdd /*AddService is the implementation class*/
{
[OperationContract]
int Add(int i, int j);
ISub Sub { [OperationContract] get; } /* return new SubService() */
}

[ServiceContract]
public interface ISub /*SubService is the implementation class*/
{
[OperationContract]
int Sub(int i, int j);
}

The problem with above is that it won’t work with either of serializers, why? WCF only shares contracts not types and here you are trying to send a type ‘SubService’ back. To make this work with DataContractSerializer we need to use KnownType or ServiceKnownType attribute:

[ServiceContract]
[ServiceKnownType(typeof(SubService))]
public interface IAdd { …

Or you can use NetDataContractSerializer. WCF team discourages use of NetDataContractSerializer and hence there are no straight attributes to apply it, luckily it’s quite easy to create one as shown here. Hence with NetDataContractSerializer there is no need to declare the sub types, actual type information travels over the wire & same will be automatically loaded.

[ServiceContract]
public interface IAdd /*AddService is the implementation class*/
{
[OperationContract]
int Add(int i, int j);
ISub Sub { [NetDataContractFormat][OperationContract] get; }
}

Note that both of the above approaches require that your implementation assembly is present on client side (there is no mapping for MarshalByRefObject of .NET Remoting in WCF).

Few more items you should care about WCF Serialization, DataContractSerializer in particular, all of which you can control through DataContractSerializer’s constructor:

1) maxItemsInObjectGraph which controls maximum items your object graph can have. You can also control it through behaviors in configuration file or Service behavior attribute (you don’t have attributes for endpoint behaviors as they are not mapped to static programming constructs, so you will have to wire maxItems explicitly by code or through configuration on client side) – default is 65536

<behaviors>
      <serviceBehaviors>
        <behavior name=”largeObjectGraph”>
          <dataContractSerializer maxItemsInObjectGraph=”100000″/>
        </behavior>
      </serviceBehaviors>
      <endpointBehaviors>
        <behavior name=”largeObjectGraph”>
          <dataContractSerializer maxItemsInObjectGraph=”100000″/>
        </behavior>
      </endpointBehaviors>
</behaviors>

2) preserveObjectReferences which help you deal with circular references like Person having Children or bidirectional references between Order and line items.

3) IDataContractSurrogate which I would let you figure out yourself (post has already crossed my normal word limit – MSDN link here).

Look forward to read your experiences with WCF Serializers.

SBTeamRules!!! YouRules!!! NirajRules!!!

Dear All, Thank you so much for being such a wonderful audience. I have uploaded my Virtual Tech Days presentation on “.NET Service Bus – Cloud for Communication Channel” here, and samples were mostly SDK samples that were tweaked by me for Kiosk scenario. To reiterate my prescription: 10 times a day – “Talk to any endpoint sitting out there on internet”. And don’t forget my millions for the billions you make . Love you all.

(P.S. Session available on VTD site)