Programming
GDI+ Image.FromFile has a problem – here’s how to fix it
Jul 30th
In GDI+ you can call Image.FromFile to load an image from a file. BUT there are several issues with this call, the biggest being that GDI+ will keep the file open long after you are done with it. Here is an image loader that gets around this issue.
If you are running a high volume web site, and your images are on a SAN you’ll find this technique necessary to prevent an eventual exhaustion of filehandles.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Drawing;
using System.IO;
using System.Data;
namespace Utility
{
public static class ImageLoader
{
// This isn’t going to help much – you’ll run out of memory anyway on very large images – but if you are keeping several in memory it might …
public const int MaximumImageDimension = 10000;
///
/// Method to safely load an image from a file without leaving the file open,
/// also gets the size down to a manageable size in the case of HUGE images
///
/// An Image – don’t forget to dispose of it later
public static Image LoadImage (string filePath)
{
try
{
FileInfo fi = new FileInfo(filePath);
if (!fi.Exists) throw new FileNotFoundException(“Cannot find image”);
if (fi.Length == 0) throw new FileNotFoundException(“Zero length image file “);
// Image.FromFile is known to leave files open, so we use a stream instead to read it
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
if (!fs.CanRead) throw new FileLoadException (“Cannot read file stream”);
if (fs.Length == 0) throw new FileLoadException(“File stream zero length”);
using (Image original = Image.FromStream(fs))
{
// Make a copy of the file in memory, then release the one GDI+ gave us
// thus ensuring that all file handles are closed properly (which GDI+ doesn’t do for us in a timely fashion)
int width = original.Width;
int height = original.Height;
if (width == 0) throw new DataException(“Bad image dimension width=0″);
if (height == 0) throw new DataException(“Bad image dimension height=0″);
// Now shrink it to Max size to control memory consumption
if (width > MaximumImageDimension)
{
height = height * MaximumImageDimension / width;
width = MaximumImageDimension;
}
if (height > MaximumImageDimension)
{
width = width * MaximumImageDimension / height;
height = MaximumImageDimension;
}
Bitmap copy = new Bitmap(width, height);
using (Graphics graphics = Graphics.FromImage(copy))
{
graphics.DrawImage(original, 0, 0, copy.Width, copy.Height);
}
return copy;
}
}
}
catch (Exception ex)
{
ex.Data.Add(“FileName”, filePath);
throw;
}
}
}
Weather Forecasting for Home Automation
Jul 25th
In the USA we are lucky to have the NOAA and their excellent web service that can provide a detailed weather forecast for any location (specified by latitude and longitude). Using this service my home automation system maintains a detailed, regularly updated local weather forecast object which can be queried easily by any other object in the home.
On the weather page you can view all of the forecast information it collects. Of interest the graph in the lower right compares the forecast from NOAA with the actual recorded temperature. In this case you can see just how accurate the forecast has been for the last 24 hours. Normally it runs almost this close but with occasionally there is a significant discrepancy caused by the bizarre ‘convergence zone’ we live in where Pacific weather patterns split around the Olympic mountains and then recombine over Seattle in somewhat unpredictable ways.
Unlike most home automation systems that have fairly limited if-the-else or table-driven approaches to defining the home logic (often limited to only the current value of any variable), my system has control structures that include statistical functions over temporal data allowing analysis of past data (e.g. average temperature in the last day), or in the case of the weather forecast, future values, e.g. expected temperature one hour from now found using interpolation.
var oneHourForNow = DateTime.Now.AddHours(1);
double outsideForecastOneHourFromNow = weatherService.ApparentTemperatureHourly.ValueAtTimeWithLinearInterpolation(oneHourForNow);
This forward looking view at the weather allows for features like garden sprinklers that don’t turn on when it’s going to rain or HVAC that skips heating cycles when the forecast predicts warm weather later today.
In addition the house is able to issue a detailed local forecast over the speakers when it wakes you up in the morning. Somewhat uniquely the forecast it generates is a relative weather forecast comparing yesterday with today, or today with tomorrow. It might for example say “Today will be much warmer that yesterday” which is a whole lot less words than a normal weather forecast!
The house also has Natural Language Generation (NLG) features which are able to summarize a group of temporal series into distinct ranges allowing it for instance to highlight which the best times of the day are to be outside:-
Monday : excellent from dawn at 5:34 AM until 11:36 AM; hot from 11:36 AM to 2:00 PM; too hot from 2:00 PM to 7:00 PM; hot until sunset at 8:53 PM.
ASP.NET Charts
Incidentally, all of the graphs are rendered using the .NET Charting Control. The graph object is instantiated on the server, all the lines and axes are added to it, then it is serialized and sent over TCP to the web server using WCF. On the web server it is rendered as a PNG file using an Action method that takes size parameters allowing any size graph to be shown on any page. Here’s the MVC code that takes the stream from WCF, loads the Chart and then delivers it as a PNG.
Chart chart = new Chart();
chart.Serializer.Load(ms);
MemoryStream ms2 = new MemoryStream();
chart.SaveImage(ms2);
return File(ms2.GetBuffer(), @"image/png");
Sequential Logic Blocks – compared to the Reactive Framework
Jul 19th
One of the features of my home automation system is extensions to the C# language that make it easy to define complex logical and temporal behaviors. These behave somewhat like the new Reactive Extensions in .NET but with some key differences which I will explain below. I developed these extensions before Reactive Framework was released and have recently been looking at Rx to see if I could combine my ideas with Rx but because of the differences explained below I haven’t been able to do that.
But, before explaining why, let’s first look at an example:-
FirstFloor.Kitchen.GoesOff.Provided((dt) => Entrance.VisitorCountThisEvening.Count < 2).TurnOff(Aquarium.Light);
This means that if the kitchen goes not occupied (off) provided we have less than two visitors this evening, then turn off the aquarium lights.
Here’s a more complex example where we create an intermediate logic element called ‘activityInKitchen’ using the .Or method, and then based on that activity we decide whether to announce that the fish have not been fed. The decision uses a combination of pulse stretching, repeats (Every) and Then which fires only if a particular sequence is followed within a given time window. Finally it uses the Do method which fires off an Action.
SensorDevice activityInKitchen =
Kitchen.KitchenFloor.Or(Kitchen.MotionSensor, Kitchen.BackDoorToGarage, Kitchen.BreakfastBarFloor, Kitchen.Phone);
activityInKitchen
.ProvidedNot(Home.DinnerGuests)
.PulseStretch(16 * 60) // Make it continuous
.Every(60 * 60) // Once every 1 hour
.Then(activityInKitchen, 15*60)
.Do("Announce fish are hungry", (sender) =>
{
Kitchen.Aquarium.AnnounceFishHungry(Kitchen.MediaZone);
});
As you can see the use of a fluent syntax allows for a very natural, almost english-like definition of complex temporal expressions. But that’s not all it allows…
The sequential logic blocks created when you call one of the many methods like .Then, .Or, .Every, … are actual objects created within the hierarchical structure of the house and the objects used to represent it. This means that they take part (automatically) in all of the features provided by a base house object including logging, browsing, and most importantly persistence.
Because these sequential logic blocks are persistent you can have very long running events like .EveryDays(2) and even if the entire system is shut down and restarted it will come back in the correct state and all the necessary delays and timers will still be working.
Another key benefit of these sequential logic blocks compared to, say, the .NET Reactive Framwork is that they form a chain that can be traversed in either direction: forward as an event propagates through, or in reverse to determine why a particular action was taken. This means that my home automation system is able to explain why it turned the lights on or off. Currently all of these decisions are logged on the per-object logging system which means you can inspect any room, appliance, light, etc. and see everything that happened to it and the reasons why.
Hallway became occupied, caused by Motion sensor
Set brightness on Aisle lights in Kitchen to 20% because Leaving office late at night
Compare this to other home automation systems from major vendors. They contain fairly simple logic, often expressed in a tabular or grid format with limited if-the-else type logic and when something happens there is often no record as to what happened and, worse, there is absolutely no record as to why it happened.
Any sufficiently complex system will have unexpected behavior, the difference here is that when it does something odd I can easily look at the logging and see the explanation as to why it happened.
Lengthening short Urls in C#
Jul 5th
If you deal with Twitter APIs, sooner or later, you’ll probably want to lengthen those short URLs back to long URLs. There’s a couple of services you can use to do this but it’s really quite easy so you might as well do it yourself. The code below follows each redirect until it reaches a real page (or an error), it then returns the last URL it finds.
private string UrlLengthen(string url)
{
string newurl = url;
bool redirecting = true;
while (redirecting)
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(newurl);
request.AllowAutoRedirect = false;
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if ((int)response.StatusCode == 301 || (int)response.StatusCode == 302)
{
string uriString = response.Headers["Location"];
Log.Debug("Redirecting " + newurl + " to " + uriString + " because " + response.StatusCode);
newurl = uriString;
// and keep going
}
else
{
Log.Debug("Not redirecting " + url + " because " + response.StatusCode);
redirecting = false;
}
}
catch (Exception ex)
{
ex.Data.Add("url", newurl);
Exceptions.ExceptionRecord.ReportCritical(ex);
redirecting = false;
}
}
return newurl;
}
Recent reading material on .NET and software development in general
May 22nd
Links I’m reading up on at the moment: Memory Mapped Files in .NET 4.0
An ontology triple (quad) store for RDF/OWL using Entity Framework 4
May 12th
This weeks side-project was the creation of an ontology store using Entity Framework 4. An ontology store stores axioms consisting of Subject, Predicate, Object which are usually serialized as RDF, OWL, N3, … Whereas there’s lots of details about these serialization formats, the actual mechanics of how to store and manipulate them was somewhat harder to come by. Nevertheless, after much experimentation I came up with an Entity Model that can store Quads (Subject, Predicate, Object and Meta) or Quins (Subject, Predicate, Object, Meta, Graph). The addition of Meta allows one Axiom to reference another. The addition of Graph allows the store to be segmented making it easy to import some N3 or RDF into a graph, then flush that graph if it is no longer needed or if a newer version becomes available.
The store is currently hooked up to an Euler reasoner that can reason against it, lazily fetching just the necessary records from the SQL database that backs the Entity Model.
Here’s the EDMX showing how I modeled the Ontology Store:
The relationship could not be changed because one or more of the foreign-key properties is non-nullable
May 9th
Today’s cryptic Entity Framework message …
The operation failed: The relationship could not be changed because one or more of the foreign-key properties is non-nullable. When a change is made to a relationship, the related foreign-key property is set to a null value. If the foreign-key does not support null values, a new relationship must be defined, the foreign-key property must be assigned another non-null value, or the unrelated object must be deleted.
The reason for this was a simple override to GetHashCode(). Entity Framework seems somewhat pedantic about how hash codes work.
Error 3032: Problem in mapping fragments : mapped to the same rows in table
Apr 27th
Today I tried creating a two-level hierarchy using Entity Framework 4 but using the same discriminator column for both layers of the hierarchy. It seems you cannot do this but instead must have a separate column for each layer of the hierarchy. Makes some sense but the error message wasn’t very helpful.
I have a table-per-hierarchy model, one abstract class (Node), a column that differentiates the sub-classes (NodeType), and three sub-classes (NodeClass, NodeProperty, NodeIndividual) with conditions (NodeType=1, 2, 3), it’s working just fine.
I now want to add another sub-class which itself has three sub-classes. So I create the new sub-class marking it abstract (NodeLiteral), I create the three derived types inheriting from it (NodeiteralInt, NodeLiteralDouble, NodeLiteralDateTime) and add conditions on them based on the same discriminator column used in the first level of inherited classes (NodeType=101,102,103).
When I compile I get:-
Error 3032: Problem in mapping fragments starting at lines … (classes) … are being mapped to the same rows in table Node. Mapping conditions can be used to distinguish the rows that these types are mapped to.
Since I DO have mapping conditions that are distinct for each of these classes the error message is somewhat confusing.
If I add another column and use that as the discriminator for the second level of inheritance it works.
So if you are trying to model a two or more layer hierarchy in EF don’t try to use a single column for all your conditions, create a column for each layer of the hierarchy instead.
A simple redirect route handler for ASP.NET 3.5 routing
Apr 20th
ASP.NET 3.5 Routing is a very powerful tool not just for registering routes for newer ASP.NET MVC applications but also for adding SEO friendly routes to older Webforms (ASPX) applications, or for routing multiple URLs to a single page. But that’s not all it can do. You can create your own IRouteHandler and then have complete control over what to do with any incoming HttpRequest.
Here for example is a way to do a permanent redirect when a given route is matched. To use it you might, for example, do:-
routes.Add(new Route("sample.aspx", new RedirectRouteHandler("/home/start")));
Here is the RedirectRouteHandler that can turn any request into a 301 redirect for you:-
/// <summary>
/// Redirect Route Handler
/// </summary>
public class RedirectRouteHandler : IRouteHandler
{
private string newUrl;
public RedirectRouteHandler(string newUrl)
{
this.newUrl = newUrl;
}
public IHttpHandler GetHttpHandler(RequestContext requestContext)
{
return new RedirectHandler(newUrl);
}
}
/// <summary>
/// <para>Redirecting MVC handler</para>
/// </summary>
public class RedirectHandler : IHttpHandler
{
private string newUrl;
public RedirectHandler(string newUrl)
{
this.newUrl = newUrl;
}
public bool IsReusable
{
get { return true; }
}
public void ProcessRequest(HttpContext httpContext)
{
httpContext.Response.Status = "301 Moved Permanently";
httpContext.Response.StatusCode = 301;
httpContext.Response.AppendHeader("Location", newUrl);
return;
}
}
Note: I’m not saying this is the best or only way to handle this. You’ll want to look at Url Rewriting and the Application and Request Routing module for IIS7 in particular.
How to stop IIS7 from handling 404 errors so you can handle them in ASP.NET
Apr 17th
IIS7 has lots of places you could look to make this change: you might start off looking to see if it’s an advanced option on your application pool, no, so then you try looking at the web site itself and the option .NET Error Pages. That has to be it, surely! So you try every option there Mode=On, Mode=Off, Mode=Remote Only. Nothing works so you consult the help for those items only learn that “Mode” is to “Select a mode for the error pages: On, Off, or Remote Only.” You can see now why help writers at Microsoft are so well paid – who would have guessed that Mode = Remote Only sets the Mode to Remote Only!
Now you are really frustrated but luckily you landed on my blog post here where you learned that the true path to 404 happiness is a simple change to your web.config:
<system.webServer> <httpErrors errorMode="Detailed" />

