WCF Serializers – XmlSerializer vs. DataContratSerializer vs. NetDataContractSerializer

I talked about encodings in a previous blog post, in this one I will talk about Serializers. Have a look at the below diagram which depicts WCF architecture in its simplest form.

As you can see serializers convert a .NET Object to WCF Message (XML Infoset) whereas Encoder convert that WCF Message into byte stream. Serializers are governed by Service Contracts whereas Encoders are specified through endpoint’s binding. There are 3 serializers supported by WCF – XmlSerializer, DataContractSerializer & NetDataContractSerializer. DataContractSerializer is the default one and should be used always unless there are backward compatibilities required with ASMX / Remoting. Let’s explore each of serializers in turn.

XmlSerializer, I guess most of us are familiar with it coming from ASMX world, is an opt-out serializer. By default this serializer takes only public fields, properties of a given type & sends them over wire. Any sensitive data must be explicitly opted out using XmlIgnore attribute. The advantage of XmlSerializer is the amount of flexibility it gives in controlling layout of XML Infoset (schema driven), and this sometimes might be required due to compatibility requirements with existing clients. Choosing XmlSerializer over default DataContractSerializer is quite easy, just use XmlSerializerFormat attribute along with your ServiceContract or OperationContract.

[ServiceContract]
//[XmlSerializerFormat] – Could be applied here for all contracts
interface IBank
{
[XmlSerializerFormat]
[OperationContract]
Customer GetCustomerById(int id);
}

DataContractSerializer – When to moving to WCF world, Microsoft seems to have decided to give more focus on versioning contracts then creating contracts. DataContractSerializer which is default serializer doesn’t allow us the flexibility of XmlSerializer for layout of XML Infoset (though you can still serialize a type implementing IXmlSerializable), but provides good versioning support with help of Order, IsRequired attributes & IExentsibleDataObject interface. Also there is a myth that DataContractSerializer only supports types decorated with DataContract/DataMember & Serializable attributes in reality though there is a programming model supporting a range of types outlined here including Hashtables, Dictionary, IXmlSerializable, ISerializable and POCO’s. There is an attribute DataContractFormat which can allow a mix with XmlSerializer. Note that if you don’t apply any attributes at all, DataContractFormat is applied by default.

[ServiceContract]
[XmlSerializerFormat] //Serialize everything using XmlSerializer
interface IBank : IExtensibleDataObject
{
[DataContractFormat] //Override with DataContractSerializer
[OperationContract]
Customer GetCustomerById(int id);
}

Finally NetDataContractSerializer. To many this is a distant concept. Let me try explaning this with a example:

[ServiceContract]
public interface IAdd /*AddService is the implementation class*/
{
[OperationContract]
int Add(int i, int j);
ISub Sub { [OperationContract] get; } /* return new SubService() */
}

[ServiceContract]
public interface ISub /*SubService is the implementation class*/
{
[OperationContract]
int Sub(int i, int j);
}

The problem with above is that it won’t work with either of serializers, why? WCF only shares contracts not types and here you are trying to send a type ‘SubService’ back. To make this work with DataContractSerializer we need to use KnownType or ServiceKnownType attribute:

[ServiceContract]
[ServiceKnownType(typeof(SubService))]
public interface IAdd { …

Or you can use NetDataContractSerializer. WCF team discourages use of NetDataContractSerializer and hence there are no straight attributes to apply it, luckily it’s quite easy to create one as shown here. Hence with NetDataContractSerializer there is no need to declare the sub types, actual type information travels over the wire & same will be automatically loaded.

[ServiceContract]
public interface IAdd /*AddService is the implementation class*/
{
[OperationContract]
int Add(int i, int j);
ISub Sub { [NetDataContractFormat][OperationContract] get; }
}

Note that both of the above approaches require that your implementation assembly is present on client side (there is no mapping for MarshalByRefObject of .NET Remoting in WCF).

Few more items you should care about WCF Serialization, DataContractSerializer in particular, all of which you can control through DataContractSerializer’s constructor:

1) maxItemsInObjectGraph which controls maximum items your object graph can have. You can also control it through behaviors in configuration file or Service behavior attribute (you don’t have attributes for endpoint behaviors as they are not mapped to static programming constructs, so you will have to wire maxItems explicitly by code or through configuration on client side) – default is 65536

<behaviors>
      <serviceBehaviors>
        <behavior name=”largeObjectGraph”>
          <dataContractSerializer maxItemsInObjectGraph=”100000″/>
        </behavior>
      </serviceBehaviors>
      <endpointBehaviors>
        <behavior name=”largeObjectGraph”>
          <dataContractSerializer maxItemsInObjectGraph=”100000″/>
        </behavior>
      </endpointBehaviors>
</behaviors>

2) preserveObjectReferences which help you deal with circular references like Person having Children or bidirectional references between Order and line items.

3) IDataContractSurrogate which I would let you figure out yourself (post has already crossed my normal word limit – MSDN link here).

Look forward to read your experiences with WCF Serializers.

SBTeamRules!!! YouRules!!! NirajRules!!!

Dear All, Thank you so much for being such a wonderful audience. I have uploaded my Virtual Tech Days presentation on “.NET Service Bus – Cloud for Communication Channel” here, and samples were mostly SDK samples that were tweaked by me for Kiosk scenario. To reiterate my prescription: 10 times a day – “Talk to any endpoint sitting out there on internet”. And don’t forget my millions for the billions you make . Love you all.

(P.S. Session available on VTD site)

It’s Cloudy Again!!!

After my session “Reaching Cloud in 60 minutes” @ TechEdOnRoad, I will be presenting one more related session on Cloud computing tomorrow @ Virtual Tech Days – “.NET Services – Cloud For Communication Channel”. Session would focus on why bi-directional connectivity matters to Enterprise today & Issues around it. Session then evaluates the challenges that go in building in a Relay Service, and how .NET Service Bus bridges that barrier with Naming, Registry, and Messaging (all patterns). So book your slots for tomorrow 3:45 to 5:00 p.m. IST. See you there .

(P.S. I will post the PPT & Demos in a separate blog post, after my session).

MTOM vs. Streaming vs. Compression – Large attachments over WCF

Above question pops up when one is about to do a large transfer of data (images for instance) using WCF. Let me try answer this starting with basics.

Bandwidth & Buffer – There are 2 considerations to large transfers. First – you want to transfer as minimal as possible in terms of size (bytes) to avoid bandwidth cost which normally matters a lot when you are over WAN paying for it & Second – whether you want to transfer the entire message (read the entire image in memory on client & send it to server) or you want to stream it byte by byte. Streaming sometimes is necessary as buffering can adversely affect performance of your server in case of multiple clients (e.g. 20 clients concurrently transferring a 100 MB image, which would take up to 2 GB of your server’s RAM). So coming to title of this post – MTOM is related to Bandwidth while Streaming is related to Buffering. Let’s dig in bit more.

MTOM (Message Transmission Optimization Mechanism) – WCF supports 3 encodings (in context of WCF, encoding means converting a WCF message (serialized XML InfoSet) into bytes) – Text, MTOM & Binary (JSON & POX are also possible – webHttpBinding). All Http Bindings (Basic, WS, Dual, etc.) support Text / MTOM encoding, Text being the default one. Text/MTOM are preferred in WS-* interoperability scenarios. To switch to MTOM encoding all you need to do is just select it as shown below:

<wsHttpBinding>
        <binding messageEncoding=”Mtom” />
</wsHttpBinding>

why MTOM? Problem with Text Encoding is it uses base 64 encoding format which can inflate the message size by 30%. This can be a heavy penalty while carrying large binary attachments. Enter MTOM!!! MTOM avoids base 64 encoding for binary attachments keeping the overall size of message in control. Moreover, MTOM is based on open specifications & hence is largely interoperable. Coming to binary encoding of WCF (TCP/Pipe/MSMQ) though it’s best in terms of performance it’s not interoperable. Some people are also averse to TCP etc. because of firewall constraints & need of Sticky Sessions (Load balancing with transport sessions). I would strongly recommend to do a performance test on all of them in your environment and then take a decision.

Streaming – Streaming (BasicHttp, Tcp, Pipe) can be a good solution when you don’t want to increase the load on your servers though unlike buffering this doesn’t allow you to leverage on WCF’s message based security & reliability (how do you ensure that entire stream is transferred and not broken in between?). In case, latter two are your requirements and you want to limit the memory usage on Server, there is a chunking channel sample on MSDN. When you want to use streaming though, your OperationContract should use only one instance of Stream class (details here) in parameter list or as return type.
E.g. Stream PlaySong();
Unfortunately above still uses a buffered mode. PlaySong API is as good as returning a ‘Byte array’ in buffered mode. To enable the Streamed mode, you need to select it at Binding level, as shown below:

<basicHttpBinding>
        <binding name=”streamedHttp” transferMode=”Streamed” />
</basicHttpBinding>

Compression – WCF’s extensible channel architecture allows us to easily plug-in a compression channel. So, how about not using MTOM or binary, and just applying compression on what we are about to transfer? First compression doesn’t come for free, it costs a lot in terms of CPU. You need to weigh the CPU cost of compression / decompression vs. Latency cost (i.e. is bandwidth a bottleneck?). For Binary encoding, I think it doesn’t make sense (I would encourage you to do your own test, but it didn’t show me much difference), for MTOM encoding I would prefer sending an already offline compressed attachment (i.e. a compressed .bmp instead of .bmp) & for Text encoding, yes, it may make sense. Say, you want to send 10000 customers over WAN (though you shouldn’t be doing that) and you need to use Text for interoperability reasons. I recommend to use compression by all means for such scenarios.

Below are the important Knobs one might have to configure depending on their message transfer requirements.

<customBinding>
        <binding name=”LargeMessageOverHttp”>
<!–Encoders–>
          <textMessageEncoding>
            <readerQuotas maxStringContentLength=”" maxArrayLength=”"
  maxBytesPerRead=”" maxDepth=”" maxNameTableCharCount=”" />
          </textMessageEncoding>
<!–Transport–>
          <httpTransport maxBufferPoolSize=”" maxBufferSize=”" maxReceivedMessageSize=”" />         
        </binding>
</customBinding>

maxArrayLength
The maximum allowed array length. The default is 16384.

maxBytesPerRead
The maximum allowed bytes returned for each read. The default is 4096.

maxDepth
The maximum nested node depth. The default is 32.

maxNameTableCharCount
The maximum characters allowed in a table name. The default is 16384.

maxStringContentLength
The maximum string length returned by the reader. The default is 8192.

maxBufferPoolSize
The maximum size of the buffer pool. The default is 524,288 bytes.

maxBufferSize
The maximum size, in bytes, of the buffer. defaults to 65536.

maxReceivedMessageSize
The maximum allowable message size that can be received. The default is 65,536 bytes.

Hope above gives some clarification .

Getting into my skin – SOA with Biztalk

I will be talking today at BDOTNET on how we can drive our Organization’s SOA efforts using Biztalk. When I started with Biztalk, I was struggling to apply it into my solutions (when to when not to – Biztalk?). I had heard of SOA but wasn’t quite sure what it stood for (was it just a web service?). Apart from that, Session will showcase few down to earth samples – why Biztalk makes sense for Enterprise SOA efforts. If you feel the above is what you are struggling with or need clarity on, be sure to occupy seats by 11:00 A.M. I will make best of my efforts to provide value for your time and petrol .