If you are running a high volume web site, and your images are on a SAN you’ll find this technique necessary to prevent an eventual exhaustion of filehandles.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Drawing;
using System.IO;
using System.Data;

namespace Utility
{
public static class ImageLoader
{
// This isn’t going to help much – you’ll run out of memory anyway on very large images – but if you are keeping several in memory it might …
public const int MaximumImageDimension = 10000;

///
/// Method to safely load an image from a file without leaving the file open,
/// also gets the size down to a manageable size in the case of HUGE images
///
/// An Image – don’t forget to dispose of it later
public static Image LoadImage (string filePath)
{
try
{
FileInfo fi = new FileInfo(filePath);

if (!fi.Exists) throw new FileNotFoundException(“Cannot find image”);
if (fi.Length == 0) throw new FileNotFoundException(“Zero length image file “);

// Image.FromFile is known to leave files open, so we use a stream instead to read it
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
if (!fs.CanRead) throw new FileLoadException (“Cannot read file stream”);

if (fs.Length == 0) throw new FileLoadException(“File stream zero length”);

using (Image original = Image.FromStream(fs))
{
// Make a copy of the file in memory, then release the one GDI+ gave us
// thus ensuring that all file handles are closed properly (which GDI+ doesn’t do for us in a timely fashion)
int width = original.Width;
int height = original.Height;
if (width == 0) throw new DataException(“Bad image dimension width=0″);
if (height == 0) throw new DataException(“Bad image dimension height=0″);

// Now shrink it to Max size to control memory consumption
if (width > MaximumImageDimension)
{
height = height * MaximumImageDimension / width;
width = MaximumImageDimension;
}
if (height > MaximumImageDimension)
{
width = width * MaximumImageDimension / height;
height = MaximumImageDimension;
}

Bitmap copy = new Bitmap(width, height);
using (Graphics graphics = Graphics.FromImage(copy))
{
graphics.DrawImage(original, 0, 0, copy.Width, copy.Height);
}
return copy;
}
}
}
catch (Exception ex)
{
ex.Data.Add(“FileName”, filePath);
throw;
}
}
}

photo credit: dynamosquito

Technology to heat or cool buildings naturally and without expending huge quantities of energy has existed for thousands of years.  In Iran this ‘badgir’ has a natural cooling system made with mud bricks and Adobe. It uses the air circulation between two towers passing through a dome refreshed by the flow of water into an underground channel named Qanat.

By contrast, typical American home construction affords few opportunities to use nature to help heat or cool the spaces we live in.  Homes here are built with thin walls making them poor insulators and although modern homes are well insulated with fiberglass insulation in the walls and roof spaces that is done primarily to keep the heat in; it provides little thermal inertia and has the unintended consequence of trapping heat in the house during summer months when there is plenty of sunlight streaming through large windows but no way out.  Worse still, in modern construction, windows and doors are kept tightly closed and the building itself is built so tight that it needs a fan to bring in outside air regularly to improve the air quality in the building.  That fan uses energy and runs on a dumb timer, sucking in potentially cold air in winter and hot air in the summer.

Having already reduced my total electricity consumption by over 40% and made inroads in how much gas we use for heating I’ve recently begun to look at how we can reduce the amount of cooling needed to keep our house comfortable in the summer.

In a location where there is a significant variation between daytime and night time temperatures there ought to be an opportunity to heat or cool a house naturally using free energy from the environment. Here near Seattle for several months each year we have just such an environment as you can see on the graph to the right (click to enlarge). The nighttime lows are currently below 70°F and the daytime highs are well above 70°F.

Since we already have a fan connected up that’s forcing external air into the house why not connect that fan to the home automation system and dispense with the dumb timer that was driving it.  Now the house has control of that fan it can change the time of day when fresh air is brought into the house to use warmer air in winter (around 3PM) and cooler air in summer (around 3AM).  It can also use this fan in conjunction with the air conditioning system.  For example, it knows you are upstairs and that it’s too warm up there tonight, the air conditioning has been running but it’s now past midnight and although it’s still 72 inside it’s dropped below 70 outside.  In this situation it can simply open the external damper, turn on the fan and turn off the air conditioning.  Cool air flows in and the compressor is idle.

All this seems like a good theory but because I’ve only had it installed for a few days it’s too early to say how well it will work.

But what about houses with no circulation fan?  Could we simply use doors and windows to improve comfort and reduce costs by telling the occupants when to open and close them?  Today for example I was up early and it was cool outside so I opened up all the doors to the deck.  The graph below shows what happened: a much bigger temperature drop than the day before even though it’s a much warmer day today overall.  What I failed to do today, however, was to close them at the right time so the early gains in ‘coolness’ were soon offset by the rapidly rising outdoor temperature and before lunch it was already warmer inside than the day before.  But what if the house calculated what to do and told you so that you could do an optimum adjustment to doors and windows to achieve free cooling?

My home automation system tracks the temperature in each zone in the house using an Aprilaire communicating thermostat with RS485.  It can display graphs for any variable or collection of variables using the ASP.NET charting control.  These graphs and experiments like the one this morning are helping me understand the dynamics of our house and figure out the best ways to achieve passive cooling (or heating).

Click to enlarge

On the weather page you can view all of the forecast information it collects. Of interest the graph in the lower right compares the forecast from NOAA with the actual recorded temperature. In this case you can see just how accurate the forecast has been for the last 24 hours. Normally it runs almost this close but with occasionally there is a significant discrepancy caused by the bizarre ‘convergence zone’ we live in where Pacific weather patterns split around the Olympic mountains and then recombine over Seattle in somewhat unpredictable ways.

Unlike most home automation systems that have fairly limited if-the-else or table-driven approaches to defining the home logic (often limited to only the current value of any variable), my system has control structures that include statistical functions over temporal data allowing analysis of past data (e.g. average temperature in the last day), or in the case of the weather forecast, future values, e.g. expected temperature one hour from now found using interpolation.

                var oneHourForNow = DateTime.Now.AddHours(1);
                double outsideForecastOneHourFromNow = weatherService.ApparentTemperatureHourly.ValueAtTimeWithLinearInterpolation(oneHourForNow);

This forward looking view at the weather allows for features like garden sprinklers that don’t turn on when it’s going to rain or HVAC that skips heating cycles when the forecast predicts warm weather later today.

In addition the house is able to issue a detailed local forecast over the speakers when it wakes you up in the morning. Somewhat uniquely the forecast it generates is a relative weather forecast comparing yesterday with today, or today with tomorrow. It might for example say “Today will be much warmer that yesterday” which is a whole lot less words than a normal weather forecast!

The house also has Natural Language Generation (NLG) features which are able to summarize a group of temporal series into distinct ranges allowing it for instance to highlight which the best times of the day are to be outside:-

Monday : excellent from dawn at 5:34 AM until 11:36 AM; hot from 11:36 AM to 2:00 PM; too hot from 2:00 PM to 7:00 PM; hot until sunset at 8:53 PM.

ASP.NET Charts

Incidentally, all of the graphs are rendered using the .NET Charting Control. The graph object is instantiated on the server, all the lines and axes are added to it, then it is serialized and sent over TCP to the web server using WCF. On the web server it is rendered as a PNG file using an Action method that takes size parameters allowing any size graph to be shown on any page. Here’s the MVC code that takes the stream from WCF, loads the Chart and then delivers it as a PNG.

            Chart chart = new Chart();
            chart.Serializer.Load(ms);

            MemoryStream ms2 = new MemoryStream();
            chart.SaveImage(ms2);

            return File(ms2.GetBuffer(), @"image/png");

But, before explaining why, let’s first look at an example:-

            FirstFloor.Kitchen.GoesOff.Provided((dt) => Entrance.VisitorCountThisEvening.Count < 2).TurnOff(Aquarium.Light);

This means that if the kitchen goes not occupied (off) provided we have less than two visitors this evening, then turn off the aquarium lights.

Here’s a more complex example where we create an intermediate logic element called ‘activityInKitchen’ using the .Or method, and then based on that activity we decide whether to announce that the fish have not been fed. The decision uses a combination of pulse stretching, repeats (Every) and Then which fires only if a particular sequence is followed within a given time window. Finally it uses the Do method which fires off an Action.

            SensorDevice activityInKitchen =
                Kitchen.KitchenFloor.Or(Kitchen.MotionSensor, Kitchen.BackDoorToGarage, Kitchen.BreakfastBarFloor, Kitchen.Phone);

            activityInKitchen
                .ProvidedNot(Home.DinnerGuests)
                .PulseStretch(16 * 60)       // Make it continuous
                .Every(60 * 60)              // Once every 1 hour
                .Then(activityInKitchen, 15*60)
                .Do("Announce fish are hungry", (sender) =>
                {
                    Kitchen.Aquarium.AnnounceFishHungry(Kitchen.MediaZone);
                });

As you can see the use of a fluent syntax allows for a very natural, almost english-like definition of complex temporal expressions. But that’s not all it allows…

The sequential logic blocks created when you call one of the many methods like .Then, .Or, .Every, … are actual objects created within the hierarchical structure of the house and the objects used to represent it. This means that they take part (automatically) in all of the features provided by a base house object including logging, browsing, and most importantly persistence.

Because these sequential logic blocks are persistent you can have very long running events like .EveryDays(2) and even if the entire system is shut down and restarted it will come back in the correct state and all the necessary delays and timers will still be working.

Another key benefit of these sequential logic blocks compared to, say, the .NET Reactive Framwork is that they form a chain that can be traversed in either direction: forward as an event propagates through, or in reverse to determine why a particular action was taken. This means that my home automation system is able to explain why it turned the lights on or off. Currently all of these decisions are logged on the per-object logging system which means you can inspect any room, appliance, light, etc. and see everything that happened to it and the reasons why.

Hallway became occupied, caused by Motion sensor
Set brightness on Aisle lights in Kitchen to 20% because Leaving office late at night

Compare this to other home automation systems from major vendors. They contain fairly simple logic, often expressed in a tabular or grid format with limited if-the-else type logic and when something happens there is often no record as to what happened and, worse, there is absolutely no record as to why it happened.

Any sufficiently complex system will have unexpected behavior, the difference here is that when it does something odd I can easily look at the logging and see the explanation as to why it happened.

It’s a fun camera to have around because it can literally go anywhere: you can take shots above the water, underwater, or just close to water without any worries about your equipment getting wet. These pictures were all taken using my Powershot D10 over the last month, from Australia to North America. Click to enlarge any of them.

In iPhoto (or any other photo management application) simply tag the photo with the email address (or group name) of the people you want to share it with!

Yes, that’s it! That’s all you would need to do to share an image with someone.

What happens behind the scenes is more interesting …

The sharing software makes a regular sweep over the iPhoto library and finds any newly tagged photos with email addresses or group names on them. If then looks up each individual and your preferred method of reaching them (email, Facebook, …) and it delivers the pictures to them. A management screen allows you to define groups and to manage the list of people with whom you share photos. For each person or group you can define how to reach them (Facebook, Twitter, email, Flickr, web site, CD-ROM, …) and you can also set a limit on how large a photo to send them and a limit on how many to send per day. Thus for a friend on a slow connection you might chose to only email them a medium sized version and limit it to one per day. They would thus get a daily email from you every day until all the photos tagged with their name (or a group that they belong to) have been delivered to them. It would also enable you to easily set up a photo of the week post to Facebook and to queue up images for future picture of the week posts.

I think this approach would be much easier than my current photo sharing process. Thoughts? Comments?

I also see that a local startup PhotoRocket has announced plans to revolutionize how we share photos. I wonder if there are any similarities between my ideas and what they are planning … we shall see soon.

1. Reduced heating/cooling when the house isn’t occupied
The house automatically drops back to a lower (or higher in the case of the air-conditioning set-point) setting whenever a zone in the house is unoccupied for a set period.

2. Further reduction in heating/cooling when the house isn’t occupied in the case of a vacation

If the house is totally unoccupied for the majority of the day (i.e. excluding brief visits from cleaners and pet sitters) it will automatically flip into an even lower power consumption mode using less heating and no cooling at all.

3. Heat-point / cool-point variation by time of day (for each zone)

Instead of aiming for a single fixed temperature for a 24 hour period the house has target temperatures for different times of day – at night for example it lowers the heat-point substantially, during the evening it lowers it subtly in preparation for nighttime (except if we have visitors (which it knows)).  On hot summer evenings it will cool the bedrooms in advance of bedtime (as shown in the graph here) but during the night it will allow the temperature to creep up slightly.  Each zone has it’s own target temperatures curve because what’s right for the bedrooms isn’t right for the kitchen.

4. Optimum start in the morning

A traditional thermostat with a timer typically simply slams the thermostat up to 68F at, say, 5AM every morning to get the house to the right temperature by the time we wake up.  My smart house instead follows an ‘optimum start’ routine whereby it gradually increases the set-point every five minutes along a predefined curve that matches the house’s thermal characteristics for heating (which varies according to the outside temperature).  This means it heats the house for the absolute minimum duration necessary to arrive at the correct temperature by the desired time of day.  A traditional thermostat by contrast may have been holding the house at 68 for an hour or more before it was really necessary.

5. No heating of cooling at all if the weather forecast says it’s not needed

If the house is going to get warm all on its own today because it’s forecast to be a hot sunny day then the house will automatically skip all heating in the morning even if it means the house will be a few degrees cooler than desired for an hour or so in the morning.  When it’s sunny and the forecast is for a hot day there’s no point heating just to increase comfort for such a short period and besides when it’s sunny outside people don’t feel as cold anyway.  This also means that the house will not need as much cooling later should it be a really hot day.

Another example of this can be seen in the graph above where the house decided to stop cooling the upper floor because the forecast indicated that it would soon be cool enough outside to not require any A/C and it would be cheaper and more effective to just suck in air from outside.

6. Manual override and the subtle shift back to computer control

There are unfortunately many people in the world who don’t understand thermostats let alone thermodynamics.  For some there is a perception that the higher you set it, the faster it will get warm.  Rather than try to reason with such people, or offend them with rude alerts or announcements over the speakers to explain how thermostats work, my home takes a more subtle approach: you can set the thermostat to whatever value you like but within an hour it will have taken back control and the set point will be back to where it should be.

7. Thermostats are part of the occupancy sensing network in the house

If you adjust a thermostat that counts as an occupancy trigger in the room in which the thermostat is located.  In order to have the densest possible network of occupancy sensors any device that receives input is treated as an occupancy sensor, so thermostats, light switches, TV remotes all act as occupancy sensors just like the more traditional door sensors, motion sensors, floor sensors etc.

8. The house refuses to attempt to cool the entire world – Close the doors!

If the house is in cooling mode and external doors are left open it will wait for 5 minutes, then issue a verbal warning over the speakers, and then if the doors are still left open it will simply stop trying to cool that zone until the doors are closed.

9. The house logs what happened in each zone and can explain why it made changes

Any complex system will have unexpected behaviors, but unlike other home automation systems this one keeps a detailed individual log for each zone and in that log you can see what happened and equally importantly why it happened because the house leaves a trail including explanations as to what it was doing and why.

10. The house makes graphs for each zone

These graphs show how the temperature varied and what changes it was making to the set points during the course of the day

These cooling and heating features together with all of the other home automation features directed at energy saving mean that our total electricity consumption is down 40% from where it was five years ago and comfort has if anything improved along with convenience – it’s very rare now that we ever need to adjust a thermostat.

11. Future improvements

Although the house can tell if we are home or away, or if we have visitors for the evening or visitors stopping over all without being told, it still can’t figure out when we will get home.  But that’s about to change, the house now knows where we are when we aren’t at home (more about that later) and it will soon be able to predict a return time.  Armed with that I hope to improve it so it can have the house ready for our return after we’ve been away without having to explicitly tell it anything!

12. A more detailed example
Click the image on the right to enlarge it. You’ll see a detailed example of how the house manages the cool point to achieve optimum comfort with minimal energy consumption.

The UI is complex and despite numerous attempts with different options it still wasn’t finding the duplicates – sometimes it would spin for minutes and then declare no duplicates, sometimes it would find some but not all. I finally found a setting that seemed to find all my duplicates and I launched it into deleting them. iPhoto began flickering wildly as it did its work but it gave no indication as to how far it had progressed or when it would finish, so I left it running. In the morning it seemed to have stopped running but iPhoto would not quit. I reluctantly did a force quit and restarted it. Disaster – it had deleted the images but left the thumbnails in iPhoto for each of the duplicates it had found. Worse still, there were still lots of duplicate images left in my library. Undeterred I tried the option to find images with thumbnails but no backing files. It found 1,900 of them so I asked it to delete them all. That failed so I tried deleting just a few at a time from the list of 1,900 images. After about 40 operations it claimed they were all gone and my library was clean. Except it wasn’t, I still have missing images and I still have duplicates.

Next I tried an option to find photos by similarity in the time they were taken except the developer used the date instead of the date plus time so it lumped all the photos found on a single date and called them duplicates!

At this point I emailed to get my money back. The developer send me a form which I dutifully filled in, signed, scanned and sent back. But, oh no, that wasn’t good enough for him, he wanted me to mail the letter to Italy. At this point I sent a strongly worded email and contacted Paypal to get a refund which came through. The developer claimed that because it worked for thousands of other people there was nothing wrong with it and that he shouldn’t have to refund me, a classic case of “it works on my machine”! It’s also a lesson that “because most customers can’t be bothered to call doesn’t mean they are happy”, for $33 most will just write it off as a failed investment. A quick look at the forum showed just a handful of people using (or attempting to use) the product.

Based on my experience with the developer I would not recommend this product. If you could try it and then pay if it worked that would be one thing, but having to pay to find out that it doesn’t work and then having to deal with the hassle of getting a refund makes this not worthwhile.

So now I’m back to cleaning up iPhoto by hand to remove duplicates and fix the thumbnails that still have no backing photo.

        private string UrlLengthen(string url)
        {
            string newurl = url;

            bool redirecting = true;

            while (redirecting)
            {

                try
                {
                    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(newurl);
                    request.AllowAutoRedirect = false;
                    request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 4.0.20506)";
                    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                    if ((int)response.StatusCode == 301 || (int)response.StatusCode == 302)
                    {
                        string uriString = response.Headers["Location"];
                        Log.Debug("Redirecting " + newurl + " to " + uriString + " because " + response.StatusCode);
                        newurl = uriString;
                        // and keep going
                    }
                    else
                    {
                        Log.Debug("Not redirecting " + url + " because " + response.StatusCode);
                        redirecting = false;
                    }
                }
                catch (Exception ex)
                {
                    ex.Data.Add("url", newurl);
                    Exceptions.ExceptionRecord.ReportCritical(ex);
                    redirecting = false;
                }
            }
            return newurl;
        }

I was fortunate to be in the right place at the right time to capture this sequence of images of the lunar eclipse. Shot from Cooloongatta, Queensland, Australia with a Canon 70-200 f2.8 lens the composite and some of my individual shots were wused by the BBC on their web site here and here.