And quickly ran into problems.

For DNS TXT records, DnsQuery returns a DNS_TXT_DATA structure in the Data field of the DNS_RECORD structure. DNS_TXT_DATA is declared like this:

typedef struct {
    DWORD dwStringCount;
    PWSTR pStringArray[1];
} DNS_TXT_DATA,
    *PDNS_TXT_DATA;

Using the very handy P/Invoke Interop Assistant, we see that this struct can be represented like this in managed code:

[StructLayout(LayoutKind.Sequential)]
public struct DNS_TXT_DATA {
 
    /// DWORD->unsigned int
    public uint dwStringCount;
 
    /// PWSTR[1]
    [MarshalAs(UnmanagedType.ByValArray,
            SizeConst=1,
            ArraySubType=UnmanagedType.SysUInt)]
    public IntPtr[] pStringArray;
}

There is a problem with pStringArray, unfortunately. The System.Runtime.InteropServices.Marshal class cannot marshal a variable length array, as it needs to know in advance how big the array is in order to allocate memory. That’s why the managed structure needs SizeConst specified in the MarshalAs attribute.

However, if the DNS TXT record data contains multiple quoted strings separated by whitespace, DnsQuery will return a structure with a variable number of elements in pStringArray. Since SizeConst is set at compile-time, when we marshal this into the managed struct defined above, we only get the first element in our single-element array. Rats.

More googling turned up very little info on dealing with this, though I found indications that others had run into the same problem without finding a satisfactory conclusion. DnsQuery is not the only Win32 function that returns variable-length arrays, and p/invoking any of the others has the same issue.

Simply declaring SizeConst to be bigger than we need – “hey, I know I’ll never get more than 10 or so strings back, so why not declare SizeConst to be 128?” – is inelegant (hardcoded upper limits, ugh) and doesn’t work properly anyway. Since the struct layout is sequential the marshaller will copy over (e.g.) 128*sizeof(IntPtr) sequential bytes (a total of 512 bytes, in this case). That much memory was never allocated on the unmanaged side, so we end up with a load of junk in the tail of pStringArray, and more often than not the marshaller chokes on this junk and throws an AccessViolationException. Fun.

There IS a way to get round the problem, though. I’m not sure it’s the best way, but it works and seems stable, so I thought I’d throw it out there in case anyone else can use it (or maybe explain to me why it’s an unsafe stupid thing to do…)

Basically, since we’re dealing with sequential memory, we can use Marshal.PtrToStructure to marshal the DNS_TXT_DATA structure as defined above, then use pointer arithmetic to gain access to any further data that needs marshalling.

Pointer arithmetic? Oh yes, even in the safe and secure world of managed code it’s sometimes still necessary to get our hands dirty, and situations like this illustrate that it will always be valuable to have some hard-earned Assembly/C/C++ war wounds.

So, assuming we have valid p/invoke declarations and data structures (I’ve included a complete source program below), DnsQuery is called like so:

var pServers = IntPtr.Zero;
var ppQueryResultsSet = IntPtr.Zero;
var ret = DnsQuery(domain,
        DnsRecordType.TEXT,
        DnsQueryType.STANDARD,
        pServers,
        ref ppQueryResultsSet,
        IntPtr.Zero);
if (ret != 0)
    throw new ApplicationException("DnsQuery failed: " + ret);

If we examine the memory location of ppQueryResultsSet (Ctrl-Alt-M,1 or Debug->Windows->Memory->Memory1 in Visual Studio) we’ll see something like the following (actual address locations may vary – just copy the int value of ppQueryResultsSet to the Address bar of the memory window):

0x049E0878  00 00 00 00  ....
0x049E087C  b8 09 9e 04  ¸.ž.
0x049E0880  10 00 20 00  .. .
0x049E0884  19 30 00 00  .0..
0x049E0888  00 00 00 00  ....
0x049E088C  00 00 00 00  ....
0x049E0890  06 00 00 00  ....
0x049E0894  b8 08 9e 04  ¸.ž.
0x049E0898  d8 08 9e 04  Ø.ž.
0x049E089C  f8 08 9e 04  ø.ž.
0x049E08A0  28 09 9e 04  (.ž.
0x049E08A4  68 09 9e 04  h.ž.
0x049E08A8  88 09 9e 04  ˆ.ž.

I’ve set the column size to 4 here, as most of the values we are dealing with are 4 bytes in size. This effectively shows one value per line.

The first 6 rows (24 bytes) correspond to the DNS_RECORD structure up until (but not including) the DNS_TXT_DATA structure in DNS_RECORD’s Data union. We can marshal this first structure without problem:

var dnsRecord = (DnsRecord) Marshal.PtrToStructure(
        ppQueryResultsSet, typeof (DnsRecord));

The DNS_TXT_DATA structure starts at address 0x049E0890 in my example. Having already marshalled the DNS_RECORD structure, now I want a pointer to the DNS_TXT_DATA structure. I can do this by creating a new pointer at the address of ppQueryResultsSet plus 24 bytes, and marshalling again:

var ptr = new IntPtr(
        ppQueryResultsSet.ToInt32() + Marshal.SizeOf(dnsRecord));
var txtData = (DNS_TXT_DATA) Marshal.PtrToStructure(
        ptr, typeof (DNS_TXT_DATA));

Because of the definition of DNS_TXT_DATA, this only marshals 8 bytes – 4 bytes for dwStringCount, and 4 bytes for the single element in pStringArray (an IntPtr). Since we know the memory is sequential, however, this gives us everything we need – we now know how many strings have been received (6 in this case, as indicated at 0x049E0890), and the location of the pointer to the first string (0x049E0894).

With this info, we can marshal all the pointers into an array with a length of dwStringCount:

ptr = new IntPtr(ptr.ToInt32() + sizeof (uint)); // move to first
var ptrs = new IntPtr[txtData.dwStringCount]; // dest array
Marshal.Copy(ptr, ptrs, 0, ptrs.Length);

And finally we iterate through those pointers, marshalling the string pointed at by each:

var strings = new List<string>();
for (var i = 0; i < ptrs.Length; ++i)
{
    strings.Add(Marshal.PtrToStringAnsi(ptrs[i]));
}

While the example I’ve presented here is specific to DnsQuery, the general approach should be applicable to any situation where you need to marshal a data structure containing a variable-length array.

Source code

“Find the greatest product of five consecutive digits in the 1000-digit number.

73167176531330624919225119674426574742355349194934
96983520312774506326239578318016984801869478851843
85861560789112949495459501737958331952853208805511
12540698747158523863050715693290963295227443043557
66896648950445244523161731856403098711121722383113
62229893423380308135336276614282806444486645238749
30358907296290491560440772390713810515859307960866
70172427121883998797908792274921901699720888093776
65727333001053367881220235421809751254540594752243
52584907711670556013604839586446706324415722155397
53697817977846174064955149290862569321978468622482
83972241375657056057490261407972968652414535100474
82166370484403199890008895243450658541227588666881
16427171479924442928230863465674813919123162824586
17866458359124566529476545682848912883142607690042
24219022671055626321111109370544217506941658960408
07198403850962455444362981230987879927244284909188
84580156166097919133875499200524063689912560717606
05886116467109405077541002256983155200055935729725
71636269561882670428252483600823257530420752963450″

The first step here is to find a representation for that fairly humungous number. Obviously it’s not going to fit into a paltry 32-bit int…but then we don’t need it to. The problem description requires us to think in terms of smaller (5-digit) numbers, not one giant 1000-digit number.

So, it is sufficient for us to consider the number as an enumerable stream of single digits, which we can conveniently represent as IEnumerable. I could use a macro to convert the number into a collection initialiser, but it’s much easier to treat the string as an IEnumerable<char> and let LINQ do the heavy lifting.

var nums = Enumerable.AsEnumerable(
        "73167176531330624919225119674426574742355349194934" +
        "96983520312774506326239578318016984801869478851843" +
 
        // ..... etc etc ......
 
        "05886116467109405077541002256983155200055935729725" +
        "71636269561882670428252483600823257530420752963450"
        ).Select(x => Convert.ToInt32(x.ToString()));

This gives us an IEnumerable<int> containing every digit in the 1000-digit number. Now, the ‘obvious’ way to solve the problem is to iterate through the collection, and at each index multiply the value against the next four indexes. A simple loop should deal with it:

private static int SimpleSolver(int[] ints)
{
    int max = 0;
    for (int i = 0; i < ints.Length - 4; i++)
    {
        int tmp = ints[i] * ints[i + 1] * ints[i + 2]
            * ints[i + 3] * ints[i + 4];
        max = Math.Max(max, tmp);
    }
 
    return max;
}

As ever, though, that’s pretty ugly – the loop condition and product calculation is tied to the sequence size of 5, and messing with an index variable is tedious.

An alternative approach is to take advantage of LINQ’s Skip and Take methods to split the problem domain into overlapping ‘slices’. Similar to the for loop above, the core of the approach is to iterate through the digits, and at each digit grab a number of subsequent digits and calculate the product.

Lets look at the 5-digit slices available from the first 10 digits:

  7   3   1   6   7   1   7   6   5   3
|       73167       |
    |       31671       |
        |       16717       |
            |       67176       |
                |       71765       |
                    |       17653       |

We can use Skip to progressively move the starting index forward, and Take to grab the 5 digits we need. So, starting with i=0, each successive slice can be sliced from the whole with:

var slice = ints.Skip(i++).Take(5);

To calculate the product of the digits in the slice, we can use the Aggregate operation:

slice.Aggregate(1, (curr, next) => curr*next);

We’ve met Aggregate before – it’s basically a fold, which collapses a sequence to a single item by repeatedly applying an operation to an accumulating result.

This can all be wrapped up as an iterator block, like so:

private static IEnumerable<int> EnumerateSlices(
        IEnumerable<int> ints, int sliceSize)
{
    int i = 0;
    while (true)
    {
        var slice = ints.Skip(i++).Take(sliceSize);
 
        if (slice.Count() < sliceSize)
            yield break; // end
 
        yield return slice.Aggregate(1,
                (curr, next) => curr*next);
    }
}

Note the termination condition – when we have enumerated every slice, our next slice will contain only 4 elements (3, 4, 5, and 0 from the end of the sequence) – that’s our cue to exit the loop.

Also note that this approach makes the algorithm trivial to parameterize – it will work just as well with slice sizes other than 5.

This iterator will produce an IEnumerable containing the products of all slices, so the final step is to select the largest:

var result = EnumerateSlices(nums, 5).Max();

I don’t know why macros are so underused. It might be a mindset thing – it can take a little while to develop the ability to spot repetitive editing tasks quickly (i.e. not when you’re 75% of the way through thinking dang, I have to do this again?), so maybe many people never quite make the leap.

It’s worth it though, because once you get your eye in you see chances to use macros everywhere.

I had a useful example just yesterday, in which I needed to make a change to a colossal switch statement (220 branches! Run the cyclomatic-complexity doohickey on THAT!) and had no unit tests to fall back on.

If I had to modify (and hopefully refactor) such a huge construct I wanted to be able to compare before-and-after test results, but I didn’t much fancy hand-cranking a few hundred unit tests.

By recording a temporary macro, however, it took just a couple of minutes to cover every branch. I’ve decided to post a detailed walkthrough of the process here in the hopes that a fairly simple example will be illustrative for those that don’t already lean heavily on macros.

Note that this is not an advanced tutorial. Please refrain from leaving snarky comments about how macros are so much more powerful than this – I’m just doing some introductory material here

Here is a representative snippet of the C# source. It’s part of a legacy permissioning system that, under certain circumstances, needs to check for the existence of a permission represented by an enum against a permission table containing a string-based hierarchy (application/role/permission). The code I was modifying did the appropriate conversion:

    case PermissionKey.SecurityParameterManagementAdd:
    {
        return new string[] {"Security", "ParameterManagement", "Add"};
    }
    case PermissionKey.SecurityRoleManagementAdd:
    {
        return new string[] {"Security", "RoleManagement", "Add"};
    }
    case PermissionKey.SecurityRoleManagementModify:
    {
        return new string[] {"Security", "RoleManagement", "Modify"};
    }
    case PermissionKey.SecurityUserManagementDelete:
    {
        return new string[] {"Security", "UserManagement", "Delete"};
    }
    case PermissionKey.SecurityPermissionManagementAdd:
    {
        return new string[] {"Security", "PermissionManagement", "Add"};
    }

I needed to add a couple of branches to this, but I also wanted to tidy up the code by removing the superfluous braces, as a precursor to converting it into something a bit more robust and maintainable. I wanted unit test coverage to give me confidence that I hadn’t mucked up some logic and inadvertantly granted admin access to the helpdesk trainee role or something.

So, I copied the entire switch body into Notepad++ (well, vim really, but I’ll pretend it’s Notepad++ for the sake of making this post a bit more accessible) and set to work^[1].

Before recording my macro, I needed to do a bit of preprocessing to trim the code down to just the data I wanted to work with. The following steps show the ‘find’ regexes I used (in each case, the value of the replace field was empty, so these are effectively deletes), and the effect on the first switch branch from the list above:

1) Remove opening and closing braces from every switch branch:

^\s+[\{\}]$

    case PermissionKey.SecurityParameterManagementAdd:
 
        return new string[] {"Security", "ParameterManagement", "Add"};

2) Remove blanks – TextFX/Edit/Delete Blank Lines

    case PermissionKey.SecurityParameterManagementAdd:
        return new string[] {"Security", "ParameterManagement", "Add"};

3) Remove case statements and leading whitespace:

^\s+case\s+

PermissionKey.SecurityParameterManagementAdd:
PermissionKey.SecurityParameterManagementAdd:
        return new string[] {"Security", "ParameterManagement", "Add"};

4) Remove colon from end of case statement:

:$

5) Remove return statement and leading whitespace:

^\s+return new string\[\]\s*

I ended up with a sequence of couplets looking similar to this one:

PermissionKey.SecurityParameterManagementAdd
{"Security", "ParameterManagement", "Add"};

Now the fun starts – lets walk through the process.

We want to convert the first couplet into a simple unit test fixture, and record the process. This will be our macro – the instructions for converting one couplet into one unit test. We can then play the macro multiple times to convert all the others effortlessly.

Start by moving the cursor to the start of the line, before the ‘P’ of PermissionKey. This is the start point of the macro, so for the macro to be repeatable we must make sure that we finish recording the macro in perfect position to run it again, i.e. before the ‘P’ of PermissionKey for the next couplet (column 0 line 3). Hit Ctrl-Shift-R to start recording.

It is important not to use the mouse when editing – stick to the keyboard. It’s also important not to record keystrokes that are too specific to one bit of code. For instance, don’t use the arrow keys to move left and right character-by-character, because it won’t work on longer or shorter lines.

Instead, use the Home and End keys to jump to the start or end of the line, and hold Ctrl whilst arrowing left or right to move a word at a time instead of a character at a time (this is one of the areas where vim’s movement commands really differentiate it from wannabes like Notepad++…but I digress). See the ‘Detailed Instructions’ section below for more information.

Assume the original switch body is in a method called ‘LookupEnumPermission’. The couplet should be edited to look like this (without the linewrap…):

[Test]
public void TestSecurityParameterManagementAdd()
{
    string[] result = LookupEnumPermission(
            PermissionKey.SecurityParameterManagementAdd);
    Assert.AreEqual("Security", result[0]);
    Assert.AreEqual("ParameterManagement", result[1]);
    Assert.AreEqual("Add", result[2]);
}

Make sure you finish by moving the cursor into position for the next couplet, and hit Ctrl-Shift-R again to stop recording.

Now, hit Ctrl-Shift-P to play back the macro. If you’ve done everything right, the next couplet should magically format itself into a unit test. Hit Ctrl-Shift-P again, and the next couplet will change too. Under the Macro menu, select ‘Run a macro multiple times…’ and you can enter a fixed number of iterations, or just apply the macro over and over again until the end of the file is reached.

Finally, you can copy the unit tests into a new or existing test fixture, and you’re done! In much less time (hopefully) and with fewer errors than if the tests had been written one-by-one.

Detailed Instructions:

These are ley-by-key instructions in Notepad++, in case something in the description above is unclear. Visual Studio should be similar. Vim will be faster once you’ve learned how, but I’ll assume if you use vim you’re already au fait with this sort of editing

1. Type [Test], and hit enter to start a new line.
2. Type ‘public void Test’ and hit Enter.
3. Type ‘{‘ and hit Enter, then Tab.
4. Hold Ctrl and tap the right arrow twice to jump over a couple of words and place the cursor at the start of the word SecurityParameterManagementAdd, then hold Ctrl-Shift and right arrow again to select the word. Ctrl-C to copy, then arrow up two lines and paste it after the word ‘Test’ to create the full function name TestSecurityParameterManagementAdd. Type () for the empty parameter list.
5. Arrow down two lines and hit Home to jump to the start of the line. Type ‘string[] result = LookupEnumPermission(‘, then hit End to jump to the end of the line and type ‘);’.
6. Arrow down one line, hit Home, then Tab. Type ‘Assert.Equals(‘ then hit Delete to remove the ‘{‘. Hold Ctrl and move right three times (to move the cursor just past the comma) and type ‘result[0]);’ and hit Enter.
7. Repeat variations of step 7 a couple of times to convert the next two lines. Remember to use the correct indexes (result[1] and result[2]). Hit Enter after the last line and type ‘}’ to close the function body.
8. Arrow down one line and hit Home to place the cursor at the correct start position for the next couplet, and end the macro by hitting Ctrl-Shift-R again.

^[1]I could have just done this in a new file in Visual Studio, but for some reason I find VS intolerably slow at running macros once recorded. So slow, in fact, that you can watch the cursor laboriously complete each step – I wind up thinking it would have been quicker to do it manually. That might just be something odd about my VS installation though, as no-one else seems to think it’s slow.

A couple of days ago Coding Horror carried a fluff piece about how all developers should be marketers too. Predictably, the article soon got posted to proggit where it was ripped on by reddit’s resident Jeff-haters, and even more predictably the comments were a mix of interesting insight and barely-concealed hate.

Apparently some of them got up Jeff’s nose a bit, and today he responded. The core of his rebuttal seems to be that you shouldn’t trust what you read on blogs, and should verify everything yourself. True enough, I guess, if perhaps a bit impractical given the sheer amount of information out there.

Then, however, Jeff goes on to give an example by referencing a compression benchmark he’d read on a blog and providing counter-analysis to show that the benchmark was wrong in claiming Deflate is faster than gzip. In doing so, much knowledge was gained.

Or so we are told.

The comment thread quickly becomes a goldmine of humour. Bugs in Jeff’s benchmarking code (not resetting the stopwatch) meant that the durations were cumulative, not independent, with inevitable distortion of the results. Another commenter pointed out that gzip cannot possibly be faster than Deflate, since the gzip algorithm IS the Deflate algorithm plus some additional computation.

“gzip” is often also used to refer to the gzip file format, which is:

• a 10-byte header, containing a magic number, a version number and a timestamp
• optional extra headers, such as the original file name,
• a body, containing a DEFLATE-compressed payload
• an 8-byte footer, containing a CRC-32 checksum and the length of the original uncompressed data

http://en.wikipedia.org/wiki/Gzip

With the benchmarking code fixed, we see that Deflate is indeed slightly faster than gzip.

All of which leads to repeated quotations from Jeff about the community being smarter than him, and some drastic toning down of language in post-publication edits to the article. I read a cached version of the RSS feed, which is markedly different to the article currently live on codinghorror.com – “on my box, GZip is twice as fast as Deflate” becomes “on my box, GZip is just as fast as Deflate”, “Deflate is way slower. It’s not even close” becomes “Deflate is nowhere near 40% faster”, etc.

Anyone who’s tackled a major performance problem will likely agree that profiling is a tremendously valuable technique that should always be applied before attempting to optimise (i.e. look before you leap). I think this little episode has highlighted a couple of important things to bear in mind, however:

1. Profiling isn’t a magic wand – if you use buggy profiling code, you are leading yourself up the garden path.
2. Profiling is less useful when you can reason (in the mathematical sense) about the code. That involves understanding the algorithms you are dealing with. Gzip is Deflate plus a bit more processing – so unless that extra processing has a negative duration gzip must necessarily take longer. You don’t need a profiler to work that out. Look before you look before you leap.

Anyway, enough hatcheting from me, normal service will be resumed shortly.

I’ve been playing around a bit recently with the June CTP in the context of batching up web service calls, as that’s something I do quite a lot. One particular problem that comes up occasionally is a two-stage series of requests to download a complete set of paged data. I might do this if I wanted to download an entire discussion thread, for instance, or a large account statement from my online bank.

Typically in this situation the web service will limit the number of records I can retrieve in one request, and allow me to specify start and count parameters to the request. The response will also include a total record count, so I know how much data there is.

The normal use case for this is to request the first page of data, and use the total record count to display a list of page links that my user can click on to navigate the data or jump to any page. In my case, however, I want ALL the data as quickly as possible.

So, imagine a situation where I am using a service that lets me download a maximum of 200 records per request. My first step is to request the maximum 200 records starting from index 0, i.e. the first page of data. In the response will be a total record count – if that number is equal to the number of records I got back (i.e. <= 200) I’ve got everything in one hit and can stop. But what if the total record count is, say, 1000? I need to make four more requests (since I’ve already got records 1-200, I have 800 more to get in batches of 200 each).

Naturally I want to do this asynchronously, using as few resources as I can. This means all webservice calls should be using the APM pattern (thus using IO completion ports, and not consuming worker threads from the thread pool or creating my own threads) and, preferably, not blocking anywhere except when I actually need some data before continuing.

The two-stage process can be successfully captured asynchronously by combining a future and a continuation. I encapsulate the initial request in a Future object (which is a subclass of Task), and handle the check-record-count-and-get-more-records-if-required logic in the continuation. The code for this basically looks as follows:

public Future<List<Item>> GetAllItemsAsync()
{
    var f = Create<GetItemsResponse>(
            ac => Service.BeginGetItems(0, ac, null),
            Service.EndGetItems);

    var start = 200;

    var resultFuture = f.ContinueWith(
        r =>
            {
                // Batch retrieval here...
            });

    return resultFuture;
}

In order to support the APM pattern neatly, I’m using the following method from the PFX blog:

private static Future<T> Create<T>(
        Action<AsyncCallback> beginFunc,
        Func<IAsyncResult, T> endFunc)
{
    var f = Future<T>.Create();
    beginFunc(iar =>
        {
            try
            {
                f.Value = endFunc(iar);
            }
            catch (Exception e)
            {
                f.Exception = e;
            }
        });
    return f;
}

This could be coded as an extension method, though I haven’t bothered yet as I’m hopeful this immensely useful snippet will be integrated into the library itself.

Now I need to make a number of calls to get the rest of the data, so I loop until I’ve made the required number of async service calls:

var resultFuture = f.ContinueWith(r =>
    {
        var items = new ConcurrentQueue<Item>();
        var handles = new List<WaitHandle>();

        while (start < r.Value.TotalRecordCount)
        {
            var asyncResult = Service.BeginGetItems(200,
                ar => Service.EndGetItems(ar).Items
                    .ForEach(items.Enqueue), null);

            handles.Add(asyncResult.AsyncWaitHandle);
            start += 200;
        }

        handles.ForEach(h => h.WaitOne());
        return items.ToList();
    });

I’m about 85% happy with this as an approach. I’m not completely happy, however, because of the WaitOne calls, which mean that I’m blocking on a threadpool thread until all the calls complete. Given that this is all wrapped up in a future, I may not actually need to access the data until well after the calls have completed, in which case I am wastefully consuming a threadpool thread for some period of time. So the $64,000 question is, how do I get rid of it? I’m sure there’s a way to do it, but my brain has gone on a protest march about all the time I’m forcing it to spend thinking about this stuff.

Still, there’s some interesting discussion in the comments about whether .Net’s closures are proper first-class lexically-scoped closures. The answer is yes – but with a fun twist.

The twist has been around for a long time – Brad Abrams blogged about it way back in 2004, for instance – but it’s probably worth going over it again, since the recent arrival of LINQ and lambda syntax in C# 3.0 will presumably lead to more people being bitten by this as the use of closures becomes more mainstream.

A key thing to remember is that C# lambdas are just anonymous delegates in skimpy syntax. Behind the scenes the compiler turns them into classes – if you were looking at disassembled MSIL you wouldn’t be able to tell whether the code was written with lambda syntax or anonymous delegate syntax. Therefore, the issue discussed by Brad has not gone anywhere.

Lets revisit the problem, with a 2008 sheen applied (i.e. I’ll use lambda syntax rather than anonymous delegate syntax). What does the following code display?

Func<int>[] funcs = new Func<int>[10];
for (int i = 0; i < 10; ++i)
{
    funcs[i] = () => i * i;
}

funcs.ForEach(f => Console.WriteLine(f()));

If you answered something along the lines of “prints the square of every number between 0 and 9″ you’d be…wrong. Really, try it out. See?

Now, a lexical closure is supposed to capture its environment, meaning that the lambda stored on the first loop would capture i when i==0, the second loop would capture i when i==1, and so on. If this happened, then executing all the lambdas would indeed result in the squares of the numbers 0-9 being printed. So what gives?

The problem stems from the fact that the lambda is binding itself to a variable that is accessible outside the closure, which is being changed in every iteration of the loop. The closure doesn’t capture the value of i, it captures a reference to i itself, which is mutable.

You could actually make a case that this is bad code anyway, since it gives two responsibilities to the loop index – control the loop, and act as data in the closures. If we were being pedantic, we could split the responsibilities by creating a new variable, j, to be the closure data each iteration, and let i concentrate on being an index:

for (int i = 0; i < 10; ++i)
{
    int j = i;
    funcs[i] = () => j * j;
}

Lo and behold, the code now works! Pedantry rules! Take a look with Reflector or ildasm to see what’s going on here. The executive summary is that the compiler captures the environment (i in the first example, j in the second) by creating a member variable within the class it generates for the closure. Previously, since the same instance of i lived for the entire duration of the loop, only one instance of the generated class was created and shared. Now, however, a new instance of the generated class is created in each iteration of the loop (since j is scoped within the loop body and thus we have a new j every time round). Thus, the data is not shared and we get the expected output.

There are two important points to consider here:

1. The problem goes away if you write code more declaratively. Do away with the clunky for loop and everything works OK.

   Enumerable.Range(0, 10).Select(x => x * x);

2. It isn’t always bad that multiple closures can capture a reference – since one closure can ‘see’ updates made to the shared data by another closure, you could use this as a coordination mechanism.

This is not an issue that’s going to crop up every day – the example above is fairly contrived – but knowing about it will save some painful debugging sessions when inevitably you do run into it. The fix is always to take a local copy of the mutable data to coerce the compiler into generating code that creates multiple instances of the class generated to represent the closure.

Simple, yes?

The Problem

A key goal for the kind of async work I do is to batch multiple calls up, so that I get all the responses at once. This is important for keeping the rest of the code clean. To illustrate, imagine you are writing an application against the Betfair API, and you have a screen that displays a market, your current profit and loss on that market, and your unmatched bets on that market. To populate this screen will require four API calls – getMarket(), getMarketPrices(), getMarketProfitAndLoss(), and getCurrentBets().

Now, the worst (though easiest) thing to do is make the four calls sequentially on the UI thread. The problem with this is it’s slow, and the UI freezes during the process (since you’re blocking on the UI thread), which is a lousy user experience.

A slightly better approach is to spin off a thread, and make the four calls there, raising an event on completion. This gets all the work off the UI thread and therefore keeps the application responsive, but it’s still slow as the calls are still sequential.

To speed it up, you can create a thread per call (so four threads in this case). There’s a whole lot of complexity around working out the optimum number of threads to use (depending on how many processors you have, how many simultaneous connections you are allowed to open, etc) but that’s a bit beyond the scope of this post, so for now we’ll go with the one-thread-per-task approach and assume it’s optimal.

So, each thread makes one webservice call, and raises an event to signify that it’s finished. Simple, right? Unfortunately, this can lead to some real headaches in collating the data.

Imagine a user has hundreds of bets on the market, and therefore the getCurrentBets() call takes a bit longer to execute than the other three. The user clicks on a market, and the threads responsible for getting market data and P&L raise their events quickly, so you display the screen with the data you have and plan to display the bets as and when they arrive.

Before the bets are received, however, the user clicks on another market. Again, the market data and P&L come back quickly and you display them. Then, finally, the original getCurrentBets() call completes. But wait! You’ve moved onto another market now, so you don’t care about those bets any more! So you have to write some code to make sure that each piece of data received is still relevant. This can become very onerous very quickly, as you struggle to determine your UI state and work out what data you want and what should be discarded.

Now imagine that your application has timers firing all over the place to update prices and P&L on the market every second or two, so you have events being raised all the time.

I’ve worked with code that ventured down this path, and believe me, you don’t want to go there.

The Solution

The best approach is to batch these calls up, so that each happens on a separate thread, but only one event is raised – when all of the data has been received. That way, you can be sure that when you handle the event, all the data is consistent.

Since this is one of the things that PLINQ does for you, it seems like a good candidate for kicking the tyres, so to speak. First, though, I’ll do a quick run through of how to do this without PLINQ, for comparison’s sake. The task will be to display a list of all the Premiership matches available on Betfair at the time the code runs.

Take Out The Old

Betfair list Premiership matches grouped by fixture date, under the Barclays Premiership node in the event tree. It looks something like this:

Soccer
    English Soccer
        Barclays Premiership
            Fixtures 23 February
                Fulham v West Ham
                Liverpool v Middlesbrough
                ...
            Fixtures 24 February
                Blackburn v Bolton
                Reading v Aston Villa
            Fixtures 25 February
                Man City v Everton

The Barclays Premiership event node has an ID that doesn’t change (2022802), so I can jump straight to that node and save myself the bother of having to navigate the Soccer and English Soccer parent nodes.

I’ll assume you already know how to create Service References for Betfair’s global WSDL, and skip straight on to creating some useful helper methods. I need to be able to call getEvents(), obviously:

private GetEventsResp GetEvents(int parentEventID)
{
    return m_global.getEvents(
            MakeEventRequest(parentEventID)).Result;
}

private getEventsIn MakeEventRequest(int parentEventID)
{
    return new getEventsIn(new GetEventsReq()
        {
            header = new APIRequestHeader()
            {
                sessionToken = m_sessionToken
            },
            eventParentId = parentEventID
        });
}

If you’re not used to C# 3.0, this is taking advantage of type initialisation to create nested objects without having to create a bunch of extra local variables. You can write the exact same method without type initialisation like this:

private getEventsIn MakeEventRequest(int parentEventID)
{
    APIRequestHeader header = new APIRequestHeader();
    header.sessionToken = m_sessionToken;
    GetEventsReq req = new GetEventsReq();
    req.header = header;
    req.eventParentId = parentEventID;
    return new getEventsIn(req);
}

The first thing I need to do is get a list of fixture nodes. I can do this by asking for child events of the Premiership node, and filtering for the events that start with the word ‘Fixture’. This can be achieved with a simple regex and a bit of normal LINQ:

private List<BFEvent> GetPremiershipFixtureEvents()
{
    return GetEvents(PREMIERSHIP).eventItems.Where(
        (ev, idx) => Regex.IsMatch(ev.eventName, "^Fixtures.*")
        ).ToList();
}

Assume PREMIERSHIP is a const int with the value 2022802. The Where() method works as a filter – you pass it a delegate, and it executes that delegate against each member of the list and returns a new list containing only the elements for which the delegate returned true.

In this case, I’m creating the delegate with a lambda expression, which returns true for elements with an event name that is matched by the regex.

Now I’ve got the fixture events, I need to get the child events of each, which correspond to the actual matches. I want each call to be asynchronous so that they happen in parallel, rather than sequentially. I also want to wait for all calls to complete before continuing, so I use the WaitHandle.WaitAll() method:

private List<BFEvent> GetMatchEvents(
    List<BFEvent> fixtureDateEvents)
{
    List<BFEvent> matchEvents = new List<BFEvent>();
    var callbacks = (
        from ev in fixtureDateEvents
        select StartGetEvents(ev.eventId, matchEvents)
        ).ToList();
    WaitHandle.WaitAll(callbacks.ConvertAll(
                ar => ar.AsyncWaitHandle).ToArray());
    return matchEvents;
}

Here, the LINQ expression and the ConvertAll() method call are doing similar things – converting all elements of a list into another type. In the case of the LINQ expression, I am effectively obtaining a list of IAsyncResult objects by calling StartGetEvents() on each event in my list and storing the return value of each call. In the case of the ConvertAll() call, I am obtaining a list of WaitHandle objects by accessing the AsyncWaitHandle property of each IAsyncResult object in the list.

It is perfectly possible to replace the LINQ expression with a call to ConvertAll(), or the ConvertAll() call with another LINQ expression. Which one you use in cases like this is largely a matter of preference.

The StartGetEvents() method needs to make an asynchronous webservice call and append the results to the provided list. Since multiple threads are accessing the list, the write must be protected with a lock:

private IAsyncResult StartGetEvents(int parentEventID,
    List<BFEvent> matchEvents)
{
    return m_global.BegingetEvents(MakeEventRequest(parentEventID),
        delegate(IAsyncResult ar)
        {
            lock (matchEvents)
            {
                matchEvents.AddRange(
                    m_global.EndgetEvents(ar).Result.eventItems);
            }
        },
        m_global);
}

I am using an anonymous delegate for the callback here. All it does is lock the list and add the events contained in the response. Note that in production code you might want to be a bit more diligent about locking strategies and so on – I’ve written the code like this for conciseness, not necessarily for production-grade correctness.

Now the whole shebang can be invoked very simply:

var fixtures = GetPremiershipFixtureEvents();
GetMatchEvents(fixtures).ForEach(
        e => Console.WriteLine(e.eventName));

Note that the calling code is very clean and simple, and doesn’t care about threads or anything like that – all that async plumbing is nicely contained in the GetMatchEvents() and StartGetEvents() methods.

Bring In The New

So how can PLINQ help with this? Well, it lets me get rid of those GetMatchEvents() and StartGetEvents() methods, which contain all the fiddly async code and are easily the most complex methods in the code above.

First, I’ll create a simple task class which represents the task of getting events for a particular ID:

public class GetEventsTask
{
    private int m_parentEventID;
    private string m_sessionToken;

    public GetEventsTask(string sessionToken,
            int parentEventID)
    {
        m_sessionToken = sessionToken;
        m_parentEventID = parentEventID;
    }

    public List<BFEvent> GetEvents()
    {
        BFGlobalService svc = new BFGlobalServiceClient();
        APIRequestHeader header = new APIRequestHeader()
            { sessionToken = m_sessionToken };
        return new List<BFEvent>(svc.getEvents(
            new getEventsIn(new GetEventsReq()
            {
                eventParentId = m_parentEventID,
                header = header
            })).Result.eventItems);
    }
}

Once I’ve instantiated an instance of this class, a call to GetEvents() will get me all the child events for the specified parent node.

To use PLINQ, all I have to do is create an array of these task objects – one per fixture date – and use the AsParallel() extension method to specify that I want the task processing done in parallel:

    GetEventsTask[] tasks = (
            from ev in fixtureDateEvents
            select new GetEventsTask(m_sessionToken, ev.eventId)
            ).ToArray();
    var taskResults = (
            from t in tasks.AsParallel()
            select t.GetEvents()
            ).ToList();

Neat, eh? Note that PLINQ will also take care of deciding the optimal number of threads, neatly sidestepping the work I alluded to earlier.

One wrinkle is that my PLINQ statement results in a list of lists, so I need to flatten it out before returning.

List<BFEvent> matchEvents = new List<BFEvent>();
taskResults.ForEach(results => matchEvents.AddRange(results));

Obviously this is only scratching the surface, not only of PLINQ but of LINQ itself. Much more powerful expressions can be created with a little tweaking of the objects generated from the Betfair WSDL – but that’s a topic for another article.

The problem is that achieving the magical 100% is, in many cases, neither practical nor particularly useful. It can even be a problem, if the warm fuzzy feeling you get when you see “Coverage: 100%” leads to complacency. Even with 100% coverage and pass-rate, you don’t necessarily have quality software.

Our high-level project architecture involved a .Net client talking to a suite of web services written in Java. The .Net client, as an application with a GUI and a web service proxy, contained a great deal of generated code and was my main concern when the targets were set.

Now, it’s my belief that in most cases there’s no benefit to writing tests for generated code (unless you also wrote the generator). Unless you have a very, very good reason not to, you should trust that the tools are doing their job and generating sane code. That’s what they’re there for. If the tools are flaky, you probably shouldn’t use them at all – though I suppose that if you sometimes fell foul of a particular bug you could write a test to detect it^[1].

The cause of my concern was that the UI and web reference code accounted for about 30-35% of the SLOC in the application, and so any coverage report that covered the whole app would be way short of the targets we were set. There are a number of ways to deal with this:

1. Bite the bullet and write tests for everything. That includes InitializeComponent(), drag ‘n’ drop handlers, and the sync and async versions of every web service stub. Best of luck, and see you in 2017^[2].
2. Explain patiently that some code does not need testing (or at least, is on the wrong side of the productivity bell curve and subject to massively diminishing returns in terms of effort/value). Of course, then you’ll be asked to prove that you’re not pulling a fast one and that the delta of your target and actual coverage percentage can be accounted for entirely by generated code. This will be tricky if you count SLOC for the generated code and use decision points for your test coverage, and maintaining this is another administrative task that you probably don’t want to do.
3. Separate your code such that some assemblies contain only generated code, and the rest contain only business logic. Then exclude the former from your test suite so they don’t show on the coverage report. This is probably achievable, though it can lead to some fairly hideous contortions to maintain the boundary, and can even result in sensible design decisions being discarded in favour of wacky ones that have no redeeming feature other than supporting your arbitrary separation rules.
4. Swear indiscriminately and refuse. Then clear your desk, probably.

None of those appealed, so we set out to find another approach. What we wanted was a more flexible variant of option 3, where we could exclude methods or classes without having to exclude the whole assembly. If we could exclude code at a fairly granular level, then it became both more realistic and useful to aim for 100% coverage of our actual business code, using all the normal techniques.

It turns out that code exclusion isn’t so tough – NCover will ignore methods and classes tagged with an attribute named CoverageExclude in the global namespace^[3].

This still requires a little discipline – for example making sure that if Joe marks a class as excluded, Jim doesn’t add some business logic to that class a week later without removing the attribute – but nothing that can’t easily be dealt with in regular code reviews.

The Powers That Be are wily, alas, and when we pitched the idea to them they approved in principle but were wary of allowing bits of code to be arbitrarily dropped off the coverage reports. If a class was excluded, who excluded it and why?

This seemed reasonable for accountability – the information would be available in the source check-in notes, but that’s a bit fiddly since you don’t know when the attribute was added; our source control system doesn’t have anything analogous to subversion’s ‘blame’ so you have to go rummaging through a potentially very long version history. A better solution would be to find a way to add the information directly to the coverage report, so that it’s right there for all to see. So, how?

The first step was to get the appropriate metadata into the code. The reference implementation for the CoverageExclude attribute is as follows:

public class CoverageExcludeAttribute : Attribute { }

We wanted to capture additional information when the attribute was used, however, so we added a couple of read-only properties and did away with the default constructor.

public class CoverageExcludeAttribute : Attribute
{
    private string m_author;
    private string m_reason;

    public CoverageExcludeAttribute(string reason,
        string author)
    {
        this.m_reason = reason;
        this.m_author = author;
    }

    public string Author
    {
        get { return this.m_author; }
    }

    public string Reason
    {
        get { return this.m_reason; }
    }
}

Now, when anyone uses the attribute, the compiler forces them to add some additional data.

[CoverageExclude("John Q Dev", "No testable code here, buster")]
public void MethodToBeExcluded(int x, int y)
{
    // ...
}

NCover can be told to pay attention to this attribute with the excludeAttributes parameter, as explained here.

With the easy bit out of the way, the next task was to report on these exclusions. Our build system, after running the test suite, used NCoverExplorer to generate a summary report. You can tell NCoverExplorer to list exclusions in reports, so we figured that would be a good place to start. The appropriate NAnt incantation is:

<ncoverexplorer failonerror="false"
  program="C:\NCoverExplorer\NCoverExplorer.Console.exe"
  projectName="Atmosphere Processor::LV426"
  reportType="4"
  xmlReportName="Report.xml"
  mergeFileName="CoverageMerge.xml"
  showExcluded="True"
  satisfactoryCoverage="80" >

  <fileset>
    <include name="Coverage.xml"/>
  </fileset>
  <exclusions>
    <exclusion type="Assembly" pattern="*Tests" />
    <exclusion type="Assembly" pattern="*Fixtures*" />
  </exclusions>
</ncoverexplorer>

Note the reportType and showExcluded attributes, which specify the summary report we want, with details of excluded code appended to the report. Note also the exclusion nodes, which specify that we want our test assemblies excluded from coverage metrics. The report will include a table like this:

Our goal was to somehow get our custom properties (Author and Reason) into this report. To do so, firstly we needed to modify the above table with two extra columns to hold this custom data. NCoverExplorer ships with stylesheet called CoverageReport.xsl; the table modification was achieved by tweaking the ‘exclusions summary’ section as follows:

<!-- Exclusions Summary -->
<xsl:template name="exclusionsSummary">
  <tr>
    <td colspan="5">&#160;</td>
  </tr>
  <tr>
    <td class="exclusionTable mainTableHeaderLeft"
      colspan="1">
      Excluded From Coverage Results</td>
    <td class="exclusionTable mainTableGraphHeader"
      colspan="1">
      All Code Within</td>
    <td class="exclusionTable mainTableGraphHeader"
      colspan="2">
      Reason For Exclusion</td>
  <td class="exclusionTable mainTableGraphHeader"
      colspan="1">
      Developer</td>
  </tr>
  <xsl:for-each select="./exclusions/exclusion">
    <tr>
      <td class="mainTableCellBottom exclusionTableCellItem"
          colspan="1">
        <xsl:value-of select="@name" /></td>
      <td class="mainTableCellBottom mainTableCellGraph"
          colspan="1">
        <xsl:value-of select="@category" /></td>
      <td class="mainTableCellBottom mainTableCellGraph"
          colspan="2">
        <xsl:value-of select="@reason" /></td>
      <td class="mainTableCellBottom mainTableCellGraph"
          colspan="1">
        <xsl:value-of select="@author" /></td>
    </tr>
  </xsl:for-each>
</xsl:template>

The next step was to actually inject our custom data into the report. This was a two-stage process:

1. Use reflection to iterate through the application assemblies, looking for anything tagged with our attribute
2. Open the report data file generated by NCoverExplorer and shoehorn our new data into it.

We created a simple little post-processor application to perform this work. To complete stage 1, we needed to iterate through a directory of assemblies, loading each one in turn. In each assembly, we iterated through the types contained therein, and looked for our custom attribute on each one. Then, we iterated through the methods on each type, and looked for the custom attribute there too. This is actually very simple – the code skeleton looks like this:

foreach (FileInfo assemblyFile in assemblies)
{
    try
    {
        // Attempt to load the file as an assembly, and grab  
        // all the types defined therein
        Assembly assembly = Assembly.LoadFrom(
            assemblyFile.FullName);
        Type[] types = assembly.GetTypes();

        // Spin through the types, looking for classes and 
        // methods tagged with CoverageExclude
        foreach (Type type in types)
        {
            object[] classAttributes = type.GetCustomAttributes(
                typeof(CoverageExcludeAttribute), false);
            foreach (CoverageExcludeAttribute classAttribute
                    in classAttributes)
            {
                // ...
            }

            MethodInfo[] methods = type.GetMethods(
                BindingFlags.Public |
                BindingFlags.NonPublic |
                BindingFlags.Instance |
                BindingFlags.Static);
            foreach (MethodInfo method in methods)
            {
                object[] methodAttributes =
                    method.GetCustomAttributes(
                    typeof(CoverageExcludeAttribute), false);
                foreach (CoverageExcludeAttribute methodAttribute
                    in methodAttributes)
                {
                    // ...
                }
            }
        }
    }
    catch (Exception ex)
    {
        // Probably not a .Net assembly, do some appropriate 
        // complaining to the user
    }
}

In the loops, we cached the fully-qualified names of the types and methods tagged with the attribute.

Stage 2 was implemented by tweaking the XML data file NCoverExplorer generates for the report. This is straightforward too – suck the report into an XmlDocument, grab the exclusion nodes, and add a couple of attributes to each one. All the types and methods were already listed since we’d set the excludeAttributes parameter in the NAnt configuration (see above).

Therefore, all we needed to do was match up the FQNs we cached in stage 1 with the nodes already in the report:

XmlDocument doc = new XmlDocument();
doc.Load(coverageFile.FullName);

// Go through all the exclusion nodes and try to match 
// them up against the exclusion data we have sucked 
// out of the assemblies
foreach (XmlNode node in doc.SelectNodes("//exclusion"))
{
    switch (node.Attributes["category"].Value)
    {
        case "Class":
            // Find and remove the first exclusion reason for 
            // this class
            FindExclusionAndModifyNode(exclusions.ClassExclusions,
                node);
            break;
        case "Method":
            // Find and remove the first exclusion reason for
            // this method
            FindExclusionAndModifyNode(exclusions.MethodExclusions,
                node);
            break;
        default:
            // Exclusion at either assembly or namespace level
            break;
    }
}

The implementation of FindExclusionAndModifyNode simply loops through the cached FQNs to see if we have data that corresponds to the current node, and if so it creates two new attributes – one containing the name of the developer that added the CoverageExcludeAttribute to the code, and another containing their justification for doing so. Then the modified XmlDocument is written out to disk, overriding the original.

The end result is a report that looks something like this, with all the excluded code neatly documented to keep suspicious managers happy.

Since the post-processor was written as a simple command-line application, we could create a custom NAnt task for it and integrate the whole process seamlessly with our continuous integration setup.

[1] I’ve seen it happen a few times before, for example an XML generator (which shall remain nameless) that occasionally ‘forgot’ our custom namespace and used a default, which caused parsers of that XML to scream in agony. It’s rare though, unless you regularly dig up tools from CodeProject and use them in your production code, in which case you deserve everything you get

[2] Written in 2008. So if you’re reading this on December 31st 2016, adjust accordingly and don’t come crying to me.

[3] Yes, I know that NCover 2.x has built-in regex-based exclusions that do all this, but a) not everyone has an NCover 2.x pro licence, and b) we weren’t using NCover 2.x as it hadn’t been released at the time.

One of the areas in which things get somewhat less straightforward is high performance. Granted, most web services don’t have particularly eye-popping requirements in terms of hits or transactions, but occasionally you find an exception. Betfair, for instance, have an API that has peak rates in excess of 13,000 requests per second, many of which hit the database, with individual users making tens or even hundreds of requests per second. Betfair’s data changes at a breathtaking rate and there is a perceived advantage to getting hold of up-to-the-millisecond information.

To put that in context, the Digg Effect is estimated to peak at around 6K-8K hits per hour as I write in December 2007, which translates to a piffling couple of hits per second (8000 / 60 / 60 = 2.2). Even a more extreme prediction of 50K hits per hour is only around 14 hits per second, so we’re talking about handling three orders of magnitude more requests than Digg generates.

If you are writing a .Net client and want to get the best out of this sort of situation, you are hamstrung unless you learn a few tricks. Read on to learn five of the best. I’ll be using the Betfair API as an example throughout, but the techniques apply to any high-performance web service where the usage profile involves frequent small (<1KB) SOAP requests.

For those who normally turn to the back of the book for answers and don’t really care about the whys and wherefores, here’s the executive summary:

1. Switch off Expect 100 Continue. This should be done in your App.config file (see below).
2. Switch off Nagle’s Algorithm. This should be done in your App.config file (see below).
3. Use multithreading. Unfortunately this is not a simple configuration file setting, it is a fundamental part of your application design.
4. Remove the maximum connection bottleneck. This should be done in your App.config file (see below). This is vital if your application is multithreaded.
5. Use gzip compression. .Net 2 has this built-in if you switch it on; .Net 1.1 needs a helping hand.

The XML snippet that configures these settings is shown below, and should be added to your App.config file:

<system.net>
    <connectionManagement>
        <add address="*" maxconnection="20" />
    </connectionManagement>
    <settings>
        <servicePointManager
            expect100Continue="false"/>
        <servicePointManager
            useNagleAlgorithm="false"/>
    </settings>
</system.net>

For those who share my distrust of unexplained sorcery, here’s the gory details.

Expect-100 the Unexpected

RFC 2616 (the specification for HTTP 1.1) includes a request header and response code that together are known as ‘Expect 100 Continue’. When using the Expect header, the client will send the request headers and wait for the server to respond with a response code of 100 before sending the request body, i.e. splitting the request into two parts with a whole round-trip in-between.

Why would you ever want to do this? Well, imagine you are trying to upload a 101MB file to a web server with a 100MB file size limit. If you just submit the whole thing as one request, you’ll sit there for ages waiting for the upload to complete, only to have it fail right at the last minute when you hit the 100MB limit. Using Expect 100 Continue, you submit just the headers initially, and wait for the server’s permission to continue; in our example, this gives the server a chance to look at the Content-Length parameter and identify that your file is too big, and return an error code instead of permission to continue. This way you know the upload will fail, without needlessly transmitting a single byte of the large file.

By default, the .Net Framework uses the Expect 100 Continue approach. Most SOAP requests are not, however, anywhere near 101MB in size, and a server designed to deal with thousands of requests per second is not likely to be returning anything other than 100 Continue if the Expect header is sent. Depending on latency, the round-trip penalty may be unacceptable.

For example, Betfair’s Australian exchange (physically located in Australia) contains all their Australian markets, so if you’re in the UK and want to trade on the Australian Open tennis you’re subject to a round-trip time of about 350ms. If you allow your requests to be split into two, then you have two round-trips, so your request will take 700ms plus processing time.

You don’t want that, so switch it off. Note that the request headers and body are still sent separately, but the latter is no longer dependent on the response to the former (so they are shown as sent together in this diagram).

The setting corresponding to this in the XML snippet above is:

<servicePointManager
            expect100Continue="false"/>

Nagle’s Algorithm

Nagle’s algorithm is a low-level algorithm that tries to minimise the number of TCP packets on the network, by trying to fill a TCP packet before sending it. TCP packets have a 40-byte header, so if you try to send a single byte you incur a lot of overhead as you are sending 41 bytes to represent 1 byte of information. This 4000% overhead causes chaos on congested networks. A TCP packet size is configurable, but is often set to 1500 bytes, and so Nagle’s algorithm will buffer outgoing data in an attempt to send a small number of full packets rather than a huge amount of mostly empty packets.

Nagle’s algorithm can be paraphrased in English like this:

If I have less than a full packet’s worth of data, and I have not yet received an acknowledgement from the server for the last data I sent, I will buffer new outbound data. When I get a server acknowledgement or have enough data to fill a whole packet, I will send the data across the network.

Now, with our use case (frequent SOAP requests, each <1KB) a request doesn’t fill a packet, so requests are subject to buffering. Furthermore, as explained above, even with Expect 100 Continue switched off the request headers are still sent separately from the request body, so the body of the first request is buffered until the request headers reach their destination and the server sends back a TCP ACK.

Let us again consider a UK client communicating with Betfair’s Australian API. Your application issues two requests one for current prices and one for your current market position (Req1 and Req2 in the diagram below). Together, both these requests are less than 1500 bytes. The headers for the first request are transmitted, but Nagle’s algorithm buffers the request body, plus the whole second request, since they don’t fill a TCP packet.

Due to the round trip latency, the acknowledgment of the first request’s headers is not received for 350ms, so that is how long the requests are buffered for. When the requests do get sent, they too are subject to 350ms latency around the world, so again you end up with around 700ms added to each of your Australian API calls.

Without Nagle, we dispense with the buffering and send out our smaller packets immediately, saving a round-trip.

One word of warning – disabling Nagle can cause problems if you are on a highly-congested network or have overworked routers and switches, since it increases the number of network packets flying around. If you don’t own your network, or are deploying your web service client to a large number of users, you might want to think about this carefully as you won’t be popular if your software saturates the network. The setting corresponding to this in the XML snippet above is:

servicePointManager
            useNagleAlgorithm="false"/>

Happenin’ Threads

This one should be pretty obvious. A single-threaded application can only do one thing at a time. A multi-threaded application can do multiple things at a time. So if, for example, you want to display some information and need to collate the results from four web service requests to build your view, calling them one after another is going to be slower than calling them all simultaneously (not to mention freezing your UI if you’re writing a WinForms application and making calls on the UI thread).

Explaining how to design and write multi-threaded .Net applications is way out of scope for this blog post, but any time you spend reading and learning about it either online or in books is going to be time well-spent. Go on, do it now. Also read the next section, otherwise your application will not get as much benefit as you expect.

Walk and Chew Gum

.Net, by default, has a bottleneck on simultaneous web service calls against the same host. This one catches loads of people out – they write clever multi-threaded applications that issue many, many requests, and never realise that beneath the application layer a lot of their requests are called in sequence rather than in parallel.

Tsk, damn Microsoft for unnecessarily crippling their framework, right? Well no, actually, since this is an example of Microsoft following standards and recommendations and is to be applauded, lest they go back to their old ways of “embrace and extend”. The recommendation in question is from RFC 2616 again, and states:

A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy.

(RFC 2616, section 8.1.4)

.Net uses HTTP 1.1 persistent connections by default (this is a good thing – you don’t want to incur the cost of establishing a TCP connection with every request, especially if you have long round-trip times as well since it involves multiple round-trips), so MS have done the right thing and restricted processes to two simultaneous connections by default.

What does this mean for your application? It means that if you want to make 20 requests, and do so on 20 threads as recommended above, under the hood .Net only makes two at a time and queues the rest up. Therefore your 20 threads are wasted, as your throughput is no better than if you’d only created two threads, and innocent web servers are protected from overzealous clients.

When we know that we’re talking to an insane city-flattening Godzilla-on-steroids of a server, however, the two-connection limit is unreasonable and we can ramp-up safely. The corresponding setting in the XML snippet above is:

<connectionManagement>
        <add address="*" maxconnection="20" />

Sorted. You can, of course, change the number 20 to whatever is appropriate for your application.

Warning: Do not, under any circumstances, get into the habit of configuring this for all your web service applications. With all 20 threads issuing one request per second, you are exceeding the 14-request-per-second Digg Effect example I used above; with this technique and enough bandwidth your application will be quite capable of taking a lot of websites down, and those that manage to weather the storm will probably blacklist your IP address and/or close your account. Only use this if you are absolutely sure the server is up to it and such aggressive behaviour is permitted by the owners.

Small is Beautiful

The last step is to enable compression on your responses. Depending on the nature of the service you are using, the value of this tip may vary, but it’s likely to be of some benefit given the number of requests per second we are issuing. Of course, it is dependent on the web service actually supporting compression, but it’s been a standard feature of just about every web server for years, so this shouldn’t be a problem.

Lets look at a worked example. The getMarketPrices response from the Betfair API is about 30KB of XML. If, as above, we have 20 threads issuing one request per second, and each thread is interested in prices from a different market, that’s about 600KB of data per second, which will quite easily saturate a lot of home broadband connections.

With gzip compression, however, each response comes down to about 5KB (XML compresses very well, generally, since it is just text with a lot of repetition and whitespace), so the 20 threads now demand a more manageable 100KB per second.

Great. So how do we use it? In .Net 2.0 and above it’s very easy – just set the EnableDecompression property of your web service proxy object (ignore IntelliSense, which incorrectly claims the value is true by default; it’s actually false by default, as stated on MSDN). For example, to get compressed responses from Betfair’s global server:

BFGlobalService service = new BFGlobalService();
service.EnableDecompression = true;

If you’re still using .Net 1.1, you have a bit more work to do, since support for gzip was inexplicably left out of the framework. First, you need to subclass the generated BFGlobalService proxy class, and override some key methods so you can a) include the Accept-Encoding header to tell the server that you understand gzip, and b) decompress the gzipped response before the XML deserializer sees it, otherwise it’ll choke.

public class BFGlobalWithGzip : BFGlobalService
{
    /// <summary>
    /// Adds compression header to request header
    /// </summary>
    protected override System.Net.WebRequest
         GetWebRequest(Uri uri)
    {
         HttpWebRequest request = 
             (HttpWebRequest)base.GetWebRequest(uri);

         // Turn on compression
         request.Headers.Add("Accept-Encoding",
             "gzip, deflate");
         return request;
    }

    /// <summary>
    /// Decompress response before the Xml serializer gets
    /// its hands on it
    /// </summary>
    protected override WebResponse GetWebResponse(
        WebRequest request)
    {
        return new HttpWebResponseDecompressed(request);
    }

    /// <summary>
    /// Need to override this method if performing
    /// asynchronous calls, otherwise de-compression
    /// will not be performed and will throw an error
    /// </summary>
    protected override WebResponse GetWebResponse(
        WebRequest request, IAsyncResult result)
    {
        return new HttpWebResponseDecompressed(
            request, result);
    }
}

Next, implement the HttpWebResponseDecompressed class. This subclasses .Net’s WebResponse class and knows how to decompress a response if it has ContentEncoding ‘gzip’:

public class HttpWebResponseDecompressed : WebResponse
{
    private HttpWebResponse m_response;
    private MemoryStream m_decompressedStream;

    public HttpWebResponseDecompressed(WebRequest request)
    {
        m_response = (HttpWebResponse)request.GetResponse();
    }

    public HttpWebResponseDecompressed(WebRequest request,
        IAsyncResult result)
    {
        m_response = (HttpWebResponse)
            request.EndGetResponse(result);
    }

    public override long ContentLength
    {
        get
        {
            if (m_decompressedStream == null
                || m_decompressedStream.Length == 0)
                    return m_response.ContentLength;
            else return m_decompressedStream.Length;
        }
        set { m_response.ContentLength = value; }
    }

    public override string ContentType
    {
        get { return m_response.ContentType; }
        set { m_response.ContentType = value; }
    }

    public override Uri ResponseUri
    {
        get { return m_response.ResponseUri; }
    }

    public override WebHeaderCollection Headers
    {
        get { return m_response.Headers; }
    }

    public override System.IO.Stream GetResponseStream()
    {
        Stream compressedStream = null;

        if (m_response.ContentEncoding == "gzip")
            compressedStream = new GZipInputStream(
                m_response.GetResponseStream());
        else if (m_response.ContentEncoding == "deflate")
            compressedStream = new InflaterInputStream(
                m_response.GetResponseStream());

        if (compressedStream != null)
        {
            m_decompressedStream = new MemoryStream();
            int size = 4096;
            byte[] buffer = new byte[size];
            while (true)
            {
                size = compressedStream.Read(
                    buffer, 0, size);

                if (size > 0)
                    m_decompressedStream.Write(
                        buffer, 0, size);
                else
                    break;
            }

            m_decompressedStream.Seek(0, SeekOrigin.Begin);
            compressedStream.Close();
            return m_decompressedStream;
        }
        else return m_response.GetResponseStream();
    }
}

To decompress the data, we need a decompression library since .Net 1.1 doesn’t provide one. In most cases, SharpZipLib will do the business:

using ICSharpCode.SharpZipLib.GZip;
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;

Now, when creating an instance of BFGlobalService you can use the gzip-supporting subclass and everything else happens automatically.

BFGlobalService service = new BFGlobalServiceWithGzip();

Fin

Jebus, that went on for a bit. Now, go forth and write high-performance clients at will – but heed the warnings about only doing this when you know the server is on-the-ball, because these tips really can cause havoc if used irresponsibly.