The new Task parallel library has continuations that allow one task to chain onto the end of a previous task but you still a way to track all the tasks currently active so you can find the other task to chain onto it. In a multi-threaded asp.net environment that’s not so easy.

Below is a TaskFactory that gives you constrained parallelism allowing you to queue up tasks in such a way that no two tasks with the same key will execute in parallel. To use it you simply create a new TaskFactorySequentiallyByKey and then call StartNewChainByKey() with a suitable key, e.g. “RENDERimage12345.jpg”. This method returns a normal Task object that you can Wait on or add more continuations. All the usual TaskFactory constructor options are provided so you can have a different TaskScheduler, common cancellation token, and other options.

Note also that it expects an Action<CancellationToken> not just a plain Action. This is so your Action can be polite and monitor the cancellation token to know when to stop early. If you don’t need that you can always pass in a closure that tosses the CancellationToken, i.e. (token) => MyAction().

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
using System.Diagnostics;

namespace Utility
{
    /// <summary>
    /// The TaskFactorySequentiallyByKey factory limits concurrency when actions are passed with the same key.  Those actions are executed sequentially
    /// and never in parallel.
    /// </summary>
    /// <remarks>
    /// For example, you have an action to fetch an image from the web to a local hard drive and then render a specific size of thumbnail for it.
    ///   The action includes code to check if the original image is already on disk, if not it fetches it.
    ///   It then checks if the correct size thumbnail has been rendered, if not it renders it.
    /// You want to be able to fire off requests for thumbnails from multiple different asp.net web pages and ensure that any two requests for the
    /// same original image are executed sequentially so that the image is only fetched once from the web before both thumbnail renders run.
    /// </remarks>
    public class TaskFactorySequentiallyByKey : TaskFactory
    {
        /// <summary>
        /// Tasks currently queued based on key
        /// </summary>
        Dictionary<string, Task> inUse = new Dictionary<string, Task>();

        public TaskFactorySequentiallyByKey()
            : base()
        {
        }

        public TaskFactorySequentiallyByKey(CancellationToken cancellationToken)
            : base(cancellationToken)
        { }

        public TaskFactorySequentiallyByKey(TaskScheduler scheduler)
            : base(scheduler)
        { }

        public TaskFactorySequentiallyByKey(TaskCreationOptions creationOptions, TaskContinuationOptions continuationOptions)
            : base(creationOptions, continuationOptions)
        { }

        public TaskFactorySequentiallyByKey(CancellationToken cancellationToken, TaskCreationOptions creationOptions, TaskContinuationOptions continuationOptions, TaskScheduler scheduler)
            : base(cancellationToken, creationOptions, continuationOptions, scheduler)
        { }

        protected virtual void FinishedUsing(string key, Task taskThatJustCompleted)
        {
            lock (this.inUse)
            {
                // If the key is present AND it point to the task that just finished THEN we are done
                // and can clear the key for the next task that comes in ...
                if (this.inUse.ContainsKey(key))
                    if (this.inUse[key] == taskThatJustCompleted)
                    {
                        this.inUse.Remove(key);
                        Debug.WriteLine("Finished using " + key + " completely");
                    }
                    else
                    {
                        Debug.WriteLine("Finished an item for " + key);
                    }

            }
        }

        /// <summary>
        /// Queue an action but prevent parallel execution of items having the same key.  Instead, run them sequentially.
        /// </summary>
        /// <remarks>
        /// This allows you to, for example, queue up tasks to fetch an image from the web to a cache and render a thumbnail for it at different sizes
        /// while ensuring that the image is only fetched to the cache once before each different size thumbnail is generated
        /// </remarks>
        public Task StartNewChainByKey(string key, Action<CancellationToken> action)
        {
            return StartNewChainByKey(key, action, base.CancellationToken);
        }

        /// <summary>
        /// Queue an action but prevent parallel execution of items having the same key.  Instead, run them sequentially.
        /// </summary>
        /// <remarks>
        /// This allows you to, for example, queue up tasks to fetch an image from the web to a cache and render a thumbnail for it at different sizes
        /// while ensuring that the image is only fetched to the cache once before each different size thumbnail is generated
        /// </remarks>
        public Task StartNewChainByKey(string key, Action<CancellationToken> action, CancellationToken cancellationToken)
        {
            CancellationToken combined = cancellationToken == base.CancellationToken ? base.CancellationToken :
                                CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, base.CancellationToken).Token;

            lock (inUse)
            {
                Task result;
                if (inUse.TryGetValue(key, out result))
                {
                    // chain the supplied action after it ...
                    result = result.ContinueWith((task) => action(combined), combined);

                    // And then schedule a completion check after that
                    result.ContinueWith((task) => FinishedUsing(key, task));

                    // Update the dictionary so that it tracks the new LAST task in line, not any of the earlier ones
                    inUse[key] = result;

                    Debug.WriteLine("Chained onto " + key);

                    return result;
                }

                // otherwise simply create it and start it after remembering that the key is in use
                result = new Task(() => action(combined), combined);

                inUse.Add(key, result);

                // queue up the check after it
                result.ContinueWith((task) => FinishedUsing(key, task));

                Debug.WriteLine("Starting a new action for " + key);

                // And finally start it
                result.Start(this.Scheduler);

                return result;
            }
        }
    }
}

The new Task parallel library doesn’t have anything that can handle these ‘run-only-one’ actions directly but it does have all the necessary building blocks to build one by creating a new TaskFactory and using Task continuations.

Below is a TaskFactory that gives you ‘run-only-one’ actions. To use it you simply create a new TaskFactoryLimitOneByKey and then call StartNewOrUseExisting() with a suitable key, e.g. “FETCH/cache/image12345.jpg”. This method returns a normal Task object that you can Wait on or add more continuations. All the usual TaskFactory constructor options are provided so you can have a different TaskScheduler, common cancellation token, and other options.

Note also that it expects an Action<CancellationToken> not just a plain Action. This is so your Action can be polite and monitor the cancellation token to know when to stop early. If you don’t need that you can always pass in a closure that tosses the CancellationToken, i.e. (token) => MyAction().

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
using System.Diagnostics;

namespace Utility
{
    /// <summary>
    /// A task factory where Tasks are queued up with a key and only one of that key is allowed to exist either in the queue or executing
    /// </summary>
    /// <remarks>
    /// This is useful for tasks like fetching a file from backing store, or updating local information from a remote service
    /// You want to be able to queue up a Task to go do the work but you don't want it to happen 5 times in quick succession
    /// NB: This does not absolve you from using file locking and other techniques in your method to handle simultaneous requests,
    /// it just greatly reduces the chances of it happening.  Another example would be updating a user's last logged in data in a
    /// database.  Under heavy load the queue to write to the database may be getting long and you don't want to update it for the same
    /// user repeatedly if you can avoid it with a single write.
    /// </remarks>
    public class TaskFactoryLimitOneByKey : TaskFactory
    {
        /// <summary>
        /// Tasks currently queued based on key
        /// </summary>
        Dictionary<string, Task> inUse = new Dictionary<string, Task>();

        public TaskFactoryLimitOneByKey()
            : base()
        {
        }

        public TaskFactoryLimitOneByKey(CancellationToken cancellationToken)
            : base(cancellationToken)
        { }

        public TaskFactoryLimitOneByKey(TaskScheduler scheduler)
            : base(scheduler)
        { }

        public TaskFactoryLimitOneByKey(TaskCreationOptions creationOptions, TaskContinuationOptions continuationOptions)
            : base(creationOptions, continuationOptions)
        { }

        public TaskFactoryLimitOneByKey(CancellationToken cancellationToken, TaskCreationOptions creationOptions, TaskContinuationOptions continuationOptions, TaskScheduler scheduler)
            : base(cancellationToken, creationOptions, continuationOptions, scheduler)
        { }

        protected virtual void FinishedUsing(string key, Task taskThatJustCompleted)
        {
            lock (this.inUse)
            {
                // If the key is present AND it point to the task that just finished THEN we are done
                // and can clear the key so that the next task coming in using it will get to execute ...
                if (this.inUse.ContainsKey(key))
                    if (this.inUse[key] == taskThatJustCompleted)
                    {
                        this.inUse.Remove(key);
                        Debug.WriteLine("Finished using " + key + " completely");
                    }
                    else
                    {
                        Debug.WriteLine("Finished an item for " + key);
                    }

            }
        }

        /// <summary>
        /// Queue only one of a given action based on a key.  A singleton pattern for Tasks with the same key.
        /// </summary>
        /// <remarks>
        /// This allows you to queue up a request to, for example, render a file based on the file name
        /// Even if multiple users all request the file at the same time, only one render will ever run
        /// and they can all wait on that Task to complete.
        /// </remarks>
        public Task StartNewOrUseExisting(string key, Action<CancellationToken> action)
        {
            return StartNewOrUseExisting(key, action, base.CancellationToken);
        }

        /// <summary>
        /// Queue only one of a given action based on a key.  A singleton pattern for Tasks with the same key.
        /// </summary>
        /// <remarks>
        /// This allows you to queue up a request to, for example, render a file based on the file name
        /// Even if multiple users all request the file at the same time, only one render will ever run
        /// and they can all wait on that Task to complete.
        /// </remarks>
        public Task StartNewOrUseExisting (string key, Action<CancellationToken> action, CancellationToken cancellationToken)
        {
            CancellationToken combined = cancellationToken == base.CancellationToken ?  base.CancellationToken :
                                            CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, base.CancellationToken).Token;
            lock (inUse)
            {
                if (inUse.ContainsKey(key))
                {
                    Debug.WriteLine("Reusing existing action for " + key);
                    return inUse[key];  // and toss the new action away
                }

                // otherwise, make a new one and add it ... with a continuation on the end to pull it off ...
                Task result = new Task(() => action(combined), combined);
                inUse.Add(key, result);

                // queue up the check after it
                result.ContinueWith((finished) => this.FinishedUsing(key, result));

                Debug.WriteLine("Starting a new action for " + key);

                // and finally start it
                result.Start(this.Scheduler);
                return result;
            }
        }
    }
}

If you are running a high volume web site, and your images are on a SAN you’ll find this technique necessary to prevent an eventual exhaustion of filehandles.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Drawing;
using System.IO;
using System.Data;

namespace Utility
{
public static class ImageLoader
{
// This isn’t going to help much – you’ll run out of memory anyway on very large images – but if you are keeping several in memory it might …
public const int MaximumImageDimension = 10000;

///
/// Method to safely load an image from a file without leaving the file open,
/// also gets the size down to a manageable size in the case of HUGE images
///
/// An Image – don’t forget to dispose of it later
public static Image LoadImage (string filePath)
{
try
{
FileInfo fi = new FileInfo(filePath);

if (!fi.Exists) throw new FileNotFoundException(“Cannot find image”);
if (fi.Length == 0) throw new FileNotFoundException(“Zero length image file “);

// Image.FromFile is known to leave files open, so we use a stream instead to read it
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
if (!fs.CanRead) throw new FileLoadException (“Cannot read file stream”);

if (fs.Length == 0) throw new FileLoadException(“File stream zero length”);

using (Image original = Image.FromStream(fs))
{
// Make a copy of the file in memory, then release the one GDI+ gave us
// thus ensuring that all file handles are closed properly (which GDI+ doesn’t do for us in a timely fashion)
int width = original.Width;
int height = original.Height;
if (width == 0) throw new DataException(“Bad image dimension width=0″);
if (height == 0) throw new DataException(“Bad image dimension height=0″);

// Now shrink it to Max size to control memory consumption
if (width > MaximumImageDimension)
{
height = height * MaximumImageDimension / width;
width = MaximumImageDimension;
}
if (height > MaximumImageDimension)
{
width = width * MaximumImageDimension / height;
height = MaximumImageDimension;
}

Bitmap copy = new Bitmap(width, height);
using (Graphics graphics = Graphics.FromImage(copy))
{
graphics.DrawImage(original, 0, 0, copy.Width, copy.Height);
}
return copy;
}
}
}
catch (Exception ex)
{
ex.Data.Add(“FileName”, filePath);
throw;
}
}
}

Click to enlarge

On the weather page you can view all of the forecast information it collects. Of interest the graph in the lower right compares the forecast from NOAA with the actual recorded temperature. In this case you can see just how accurate the forecast has been for the last 24 hours. Normally it runs almost this close but with occasionally there is a significant discrepancy caused by the bizarre ‘convergence zone’ we live in where Pacific weather patterns split around the Olympic mountains and then recombine over Seattle in somewhat unpredictable ways.

Unlike most home automation systems that have fairly limited if-the-else or table-driven approaches to defining the home logic (often limited to only the current value of any variable), my system has control structures that include statistical functions over temporal data allowing analysis of past data (e.g. average temperature in the last day), or in the case of the weather forecast, future values, e.g. expected temperature one hour from now found using interpolation.

                var oneHourForNow = DateTime.Now.AddHours(1);
                double outsideForecastOneHourFromNow = weatherService.ApparentTemperatureHourly.ValueAtTimeWithLinearInterpolation(oneHourForNow);

This forward looking view at the weather allows for features like garden sprinklers that don’t turn on when it’s going to rain or HVAC that skips heating cycles when the forecast predicts warm weather later today.

In addition the house is able to issue a detailed local forecast over the speakers when it wakes you up in the morning. Somewhat uniquely the forecast it generates is a relative weather forecast comparing yesterday with today, or today with tomorrow. It might for example say “Today will be much warmer that yesterday” which is a whole lot less words than a normal weather forecast!

The house also has Natural Language Generation (NLG) features which are able to summarize a group of temporal series into distinct ranges allowing it for instance to highlight which the best times of the day are to be outside:-

Monday : excellent from dawn at 5:34 AM until 11:36 AM; hot from 11:36 AM to 2:00 PM; too hot from 2:00 PM to 7:00 PM; hot until sunset at 8:53 PM.

ASP.NET Charts

Incidentally, all of the graphs are rendered using the .NET Charting Control. The graph object is instantiated on the server, all the lines and axes are added to it, then it is serialized and sent over TCP to the web server using WCF. On the web server it is rendered as a PNG file using an Action method that takes size parameters allowing any size graph to be shown on any page. Here’s the MVC code that takes the stream from WCF, loads the Chart and then delivers it as a PNG.

            Chart chart = new Chart();
            chart.Serializer.Load(ms);

            MemoryStream ms2 = new MemoryStream();
            chart.SaveImage(ms2);

            return File(ms2.GetBuffer(), @"image/png");

But, before explaining why, let’s first look at an example:-

            FirstFloor.Kitchen.GoesOff.Provided((dt) => Entrance.VisitorCountThisEvening.Count < 2).TurnOff(Aquarium.Light);

This means that if the kitchen goes not occupied (off) provided we have less than two visitors this evening, then turn off the aquarium lights.

Here’s a more complex example where we create an intermediate logic element called ‘activityInKitchen’ using the .Or method, and then based on that activity we decide whether to announce that the fish have not been fed. The decision uses a combination of pulse stretching, repeats (Every) and Then which fires only if a particular sequence is followed within a given time window. Finally it uses the Do method which fires off an Action.

            SensorDevice activityInKitchen =
                Kitchen.KitchenFloor.Or(Kitchen.MotionSensor, Kitchen.BackDoorToGarage, Kitchen.BreakfastBarFloor, Kitchen.Phone);

            activityInKitchen
                .ProvidedNot(Home.DinnerGuests)
                .PulseStretch(16 * 60)       // Make it continuous
                .Every(60 * 60)              // Once every 1 hour
                .Then(activityInKitchen, 15*60)
                .Do("Announce fish are hungry", (sender) =>
                {
                    Kitchen.Aquarium.AnnounceFishHungry(Kitchen.MediaZone);
                });

As you can see the use of a fluent syntax allows for a very natural, almost english-like definition of complex temporal expressions. But that’s not all it allows…

The sequential logic blocks created when you call one of the many methods like .Then, .Or, .Every, … are actual objects created within the hierarchical structure of the house and the objects used to represent it. This means that they take part (automatically) in all of the features provided by a base house object including logging, browsing, and most importantly persistence.

Because these sequential logic blocks are persistent you can have very long running events like .EveryDays(2) and even if the entire system is shut down and restarted it will come back in the correct state and all the necessary delays and timers will still be working.

Another key benefit of these sequential logic blocks compared to, say, the .NET Reactive Framwork is that they form a chain that can be traversed in either direction: forward as an event propagates through, or in reverse to determine why a particular action was taken. This means that my home automation system is able to explain why it turned the lights on or off. Currently all of these decisions are logged on the per-object logging system which means you can inspect any room, appliance, light, etc. and see everything that happened to it and the reasons why.

Hallway became occupied, caused by Motion sensor
Set brightness on Aisle lights in Kitchen to 20% because Leaving office late at night

Compare this to other home automation systems from major vendors. They contain fairly simple logic, often expressed in a tabular or grid format with limited if-the-else type logic and when something happens there is often no record as to what happened and, worse, there is absolutely no record as to why it happened.

Any sufficiently complex system will have unexpected behavior, the difference here is that when it does something odd I can easily look at the logging and see the explanation as to why it happened.

        private string UrlLengthen(string url)
        {
            string newurl = url;

            bool redirecting = true;

            while (redirecting)
            {

                try
                {
                    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(newurl);
                    request.AllowAutoRedirect = false;
                    request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
                    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                    if ((int)response.StatusCode == 301 || (int)response.StatusCode == 302)
                    {
                        string uriString = response.Headers["Location"];
                        Log.Debug("Redirecting " + newurl + " to " + uriString + " because " + response.StatusCode);
                        newurl = uriString;
                        // and keep going
                    }
                    else
                    {
                        Log.Debug("Not redirecting " + url + " because " + response.StatusCode);
                        redirecting = false;
                    }
                }
                catch (Exception ex)
                {
                    ex.Data.Add("url", newurl);
                    Exceptions.ExceptionRecord.ReportCritical(ex);
                    redirecting = false;
                }
            }
            return newurl;
        }

The store is currently hooked up to an Euler reasoner that can reason against it, lazily fetching just the necessary records from the SQL database that backs the Entity Model.

Here’s the EDMX showing how I modeled the Ontology Store:

The operation failed: The relationship could not be changed because one or more of the foreign-key properties is non-nullable. When a change is made to a relationship, the related foreign-key property is set to a null value. If the foreign-key does not support null values, a new relationship must be defined, the foreign-key property must be assigned another non-null value, or the unrelated object must be deleted.

The reason for this was a simple override to GetHashCode(). Entity Framework seems somewhat pedantic about how hash codes work.

Here for example is a way to do a permanent redirect when a given route is matched. To use it you might, for example, do:-

            routes.Add(new Route("sample.aspx", new RedirectRouteHandler("/home/start")));

Here is the RedirectRouteHandler that can turn any request into a 301 redirect for you:-

    /// <summary>
    /// Redirect Route Handler
    /// </summary>
    public class RedirectRouteHandler : IRouteHandler
    {
        private string newUrl;

        public RedirectRouteHandler(string newUrl)
        {
            this.newUrl = newUrl;
        }

        public IHttpHandler GetHttpHandler(RequestContext requestContext)
        {
            return new RedirectHandler(newUrl);
        }
    }

    /// <summary>
    /// <para>Redirecting MVC handler</para>
    /// </summary>
    public class RedirectHandler : IHttpHandler
    {
        private string newUrl;

        public RedirectHandler(string newUrl)
        {
            this.newUrl = newUrl;
        }

        public bool IsReusable
        {
            get { return true; }
        }

        public void ProcessRequest(HttpContext httpContext)
        {
            httpContext.Response.Status = "301 Moved Permanently";
            httpContext.Response.StatusCode = 301;
            httpContext.Response.AppendHeader("Location", newUrl);
            return;
        }
    }

Note: I’m not saying this is the best or only way to handle this. You’ll want to look at Url Rewriting and the Application and Request Routing module for IIS7 in particular.