Smartest House

The home as a user interface

Recently I’ve spent a lot of time thinking about the future of home automation.  There’s a lot of chatter about the internet of things, most of which relates to having lots of sensors reporting data back to servers so you can see what’s happening at home.  That’s all very well, but personally I think that sensors in the home can also act like a gigantic input device.  In effect the sensors you have in your home become a giant keyboard, and as you ‘type’ by simply moving around, opening doors and such like you are providing input to a home automation system that will then act and respond.  The interface of the future isn’t fancy touch-screen keypads in every room, it’s an invisible network of sensors that know you walked upstairs, opened a door, are standing near the fridge etc.

For example, in our house, you don’t look at a screen to find out who called while you were out, instead the mere act of coming home, opening the garage door and then moving into the kitchen triggers an announcement over the speakers telling you that you missed a call while you were out.  You didn’t need to press anything, you didn’t need to look at anything, the house simply responds like a human butler would do and speaks to you at an appropriate time.  The algorithm is actually rather more complex than I implied as it doesn’t blurt it out right away as you stagger through the back door with arm fulls of groceries, instead it waits until you’ve moved around in the kitchen for a while and then it makes the announcement.

Another example is the link between open doors and air conditioning. If you leave a door open to the outside, after five minutes it will warn you and then after a few more it will turn the A/C off.

Yet another example if the warning it issues in the evening if you left the garage doors open.  A simpler system might just repeat an annoying warning over the speakers every half hour after 8PM. My system waits until you are up and moving about before telling you so you get the alert when you are ready to act on it and it never disturbs you while you are sat still watching TV.


The house is the user interface.




A traffic service that answers “which way should I go?”

Which way should I go?  Traffic

Most traffic reports (on the radio or in text message alerts) are fairly useless. Like weather reports they contain lots of irrelevant information that could be eliminated with just a bit of extra context. In fact, most of the information they deliver is completely irrelevant to you as an individual located in one spot and hoping to get to another spot. Furthermore they aren’t actionable – telling me the traffic is slow on SR-520 and on I-90 isn’t interesting unless you can tell me which is the best way to go given where I am now and where I want to be.

So this weekend I added a new feature to the home automation that uses the WSDOT’s excellent traffic feed API to calculate a traffic report just for me. Recently I’ve started driving from the north end of Bellevue to the south end of Sammamish during rush hour. There are two very different paths I can take: SR-520 or I-405 to I-90. If either route has a problem I should take the other. So now I get an XMPP (chat) message from 4PM to 6PM whenever the optimal path changes from one route to the other. It’s the absolute minimum information I need and it’s 100% actionable.

For the moment the calculation is fairly simple, I simply maintain a list of the FlowDataID values along each route and then calculate a total ‘slowness’ factor based on the sum of those segments. If one way is much better than the other it generates an alert. If it goes back to being roughly equal the alert is cleared.

Since the calculation is purely relative (route A vs route B) it’s also fairly immune to day-of-week / school-holidays and other factors that have a significant impact on traffic but no impact on the only actionable decision I need to consider.

One other interesting point from the graph is just how spiked the traffic is on SR-520 compared to I-90.

Programming a smart home with a fluent, domain-specific language

In response to a question I received recently, here is an example of the fluent extensions to C# that my home automation system uses to define how it should behave. In effect this is a domain specific language for modeling home automation behavior as a finite state machine. Note how you can use either a purely declarative sequence or you can use procedural code within a Do() block.

                .Then(Entrance.HallBeam, 30)
                .PulseStretch(10 * 60)      // don't announce it too often - every 10 minutes
                .Do("Garage doors warning", () =>
                if (Garage.GarageDoors.AreAnyOpen)
                    FirstFloor.Kitchen.AllMediaZones.AnnounceString("Excuse me, I think you may want to close the garage doors.");

In many ways this is similar to the Reactive Framework from Microsoft except my work on this predates the availability of Reactive Framework and unlike the Reactive Framework my ‘sequential logic blocks’ include persistence logic so the entire state can be saved to disk (as it is every second). This is important because some transitions in the state machine might be several hours or even days long and you need to be able to restart the system and carry on exactly where you left off.

One key benefit of the declarative approach over the procedural approach is that the declarative approach can explain itself. So, the log entry for a light going on can show that the light was turned on because ‘it was dark’ and ‘we had visitors’ and ‘someone came into the room’. Compared to traditional home automation systems where you either have no logging at all or you have a log of what happened, this kind of logging is invaluable for figuring out what went wrong when the logic gets complicated. So in this example I should have moved the test for Garage.GarageDoors.AreAnyOpen out to a Provided clause which would allow it to be part of the reverse-chain logic explanation.

Partial results can be captured at any point in these logic chains and then reused in other chains because each partial result is itself a Sensor device that implements the full fluent interface.

Ultimately I plan to hook the logging for what happened back up to the NLP engine which will allow users to ask the home ‘Why did you put the driveway light on last night around 9pm?’ and ultimately I plan to allow the logic itself to be defined using natural language.

Holiday Season (Christmas) in our Smart Home

Smart Home lighting for the Christmas Tree

So what does a smart home do at Christmas time?

Well, obviously it controls the Christmas lights, both the ones on the exterior and the ones on the Christmas tree and around the house. The indoor lights come on automatically at dusk and stay on provided the room they are in is occupied. Leave the house and they go off automatically. Walk back in and a strain-gauge under the living room floor detects your arrival and the tree lights and rope lights come right back on. Why waste energy lighting your Christmas tree if nobody can see it and why press light switches if you don’t need to?

The exterior lights come on at dusk and go off around 9PM. During holiday seasons the permanent Christmas lights along the eaves of the house come on automatically too. But the house also understands whether it has visitors. This isn’t something you need to tell it, it just figures it out by counting cars arriving and people coming in through the front door. If the house thinks it has visitors it will leave all the outside and Christmas lights on until the visitors have left. Again, there’s nothing you need to tell it, this is a ‘smart’ home not a ‘dumb’ home, and this is real ‘home automation’ not ‘home control’.

Another change that happens automatically during the holiday season is that the alert for a car coming down the drive changes to a subtle jingle bells sound. Normally the sound is the distant tweeting of birds and it’s played quietly so that visitors don’t even notice it. The residents of course know that there isn’t some bird tweeting outside but that a car is approaching the house. The dogs know the sound too and go running to the door barking. With the change to jingle bells sound the dogs quickly figure that one out too. The only problem is that now, when jingle bells are heard on TV they both run off barking to the front door! If only home automation systems were as easy to train as dogs! The driveway sensor by the way is one of those magnetic detectors buried about 120′ away to the side of the drive. It gives sufficient warning to prepare for a visitor before they even get to the front door.

One final change that happens automatically at this time of year concerns the music system in our house. My home automation software includes multi-channel audio playback through a zoned-audio switch. This allows any one of three sound cards to be connected to any set of speakers in the home. This means you can have exactly the same music playing across a whole floor, or indeed throughout the house, without any lag between rooms. In effect this is a poor man’s Sonos as the cost of each additional source is about $7.50 for a USB sound card plus about $150 for a single zone amplifier. You instruct the music system to play back music either by chatting to it on Instant Messenger (it has its own XMPP address), by emailing it, or by putting an entry on the house’s own Google calendar telling it when to start playback. For example, ‘play songs added this week in office’ would begin playback of any new music added to the system this week. And, the change that happens at this time of year is that the music playback subsystem allows Christmas music to play. That’s right, for the rest of the year, no matter what random selection of music you ask it to play (e.g. every song with the word ‘Bing’ in the title, artist or album) you will not hear any Christmas music. This is one feature iTunes and every other music player should adopt!

Home Automation Top Features

Home lighting

Lighting probably uses more electricity than anything else in your home

Someone recently asked me what the top features are in my home automation system. That’s a tough question, I have several favorites and there are so many features in there already or under development. But, here’s a current list of some of my favorites:

  • Lights turn themselves off automatically – saves 40% on electricity usage
  • Lights turn on ahead of you at night as you walk around
  • Intelligent heating and A/C control (based on actual and expected occupancy, ‘optimum start’, weather forecast and local thermostat control)
  • Monitor house from anywhere, see at a glance what’s happening
  • Play music, news, podcasts, … in any room or zone
  • House speaks: caller ID, alarm clock, reminders, missed calls, weather forecast, … with professional quality ducking of background music
  • Automatic phone book learns the name of everyone who calls
  • Chat interface with natural language control: understands complex sentences like “who called last week on Friday after 4pm”, tells you what’s happening (calls, cars, …)
  • Calendar integration to record what happened and accept future instructions
  • Turn TV down or off remotely (“dinner’s ready” feature)
  • House can sense what’s happening and act accordingly: different behavior for away on vacation, not-occupied, occupied, visitors staying over, party
  • House generates a local, sports-specific weather forecast and alerts when there’s fresh snow, pass closures, really bad traffic
  • House can explain what it did and why it did it
  • Programming features for defining new behavior based on events, times, past history, forecasts, …

Passive Air Conditioning to reduce Energy Consumption

Creative Commons License photo credit: dynamosquito

Technology to heat or cool buildings naturally and without expending huge quantities of energy has existed for thousands of years.  In Iran this ‘badgir’ has a natural cooling system made with mud bricks and Adobe. It uses the air circulation between two towers passing through a dome refreshed by the flow of water into an underground channel named Qanat.

By contrast, typical American home construction affords few opportunities to use nature to help heat or cool the spaces we live in.  Homes here are built with thin walls making them poor insulators and although modern homes are well insulated with fiberglass insulation in the walls and roof spaces that is done primarily to keep the heat in; it provides little thermal inertia and has the unintended consequence of trapping heat in the house during summer months when there is plenty of sunlight streaming through large windows but no way out.  Worse still, in modern construction, windows and doors are kept tightly closed and the building itself is built so tight that it needs a fan to bring in outside air regularly to improve the air quality in the building.  That fan uses energy and runs on a dumb timer, sucking in potentially cold air in winter and hot air in the summer.

Having already reduced my total electricity consumption by over 40% and made inroads in how much gas we use for heating I’ve recently begun to look at how we can reduce the amount of cooling needed to keep our house comfortable in the summer.

Temperature Variation Near Seattle In a location where there is a significant variation between daytime and night time temperatures there ought to be an opportunity to heat or cool a house naturally using free energy from the environment. Here near Seattle for several months each year we have just such an environment as you can see on the graph to the right (click to enlarge). The nighttime lows are currently below 70°F and the daytime highs are well above 70°F.

Since we already have a fan connected up that’s forcing external air into the house why not connect that fan to the home automation system and dispense with the dumb timer that was driving it.  Now the house has control of that fan it can change the time of day when fresh air is brought into the house to use warmer air in winter (around 3PM) and cooler air in summer (around 3AM).  It can also use this fan in conjunction with the air conditioning system.  For example, it knows you are upstairs and that it’s too warm up there tonight, the air conditioning has been running but it’s now past midnight and although it’s still 72 inside it’s dropped below 70 outside.  In this situation it can simply open the external damper, turn on the fan and turn off the air conditioning.  Cool air flows in and the compressor is idle.

All this seems like a good theory but because I’ve only had it installed for a few days it’s too early to say how well it will work.

Passive Cooling - Open The Doors

But what about houses with no circulation fan?  Could we simply use doors and windows to improve comfort and reduce costs by telling the occupants when to open and close them?  Today for example I was up early and it was cool outside so I opened up all the doors to the deck.  The graph below shows what happened: a much bigger temperature drop than the day before even though it’s a much warmer day today overall.  What I failed to do today, however, was to close them at the right time so the early gains in ‘coolness’ were soon offset by the rapidly rising outdoor temperature and before lunch it was already warmer inside than the day before.  But what if the house calculated what to do and told you so that you could do an optimum adjustment to doors and windows to achieve free cooling?

My home automation system tracks the temperature in each zone in the house using an Aprilaire communicating thermostat with RS485.  It can display graphs for any variable or collection of variables using the ASP.NET charting control.  These graphs and experiments like the one this morning are helping me understand the dynamics of our house and figure out the best ways to achieve passive cooling (or heating).

Weather Forecasting for Home Automation

In the USA we are lucky to have the NOAA and their excellent web service that can provide a detailed weather forecast for any location (specified by latitude and longitude). Using this service my home automation system maintains a detailed, regularly updated local weather forecast object which can be queried easily by any other object in the home.

Home Automation Weather Forecasting

Click to enlarge

On the weather page you can view all of the forecast information it collects. Of interest the graph in the lower right compares the forecast from NOAA with the actual recorded temperature. In this case you can see just how accurate the forecast has been for the last 24 hours. Normally it runs almost this close but with occasionally there is a significant discrepancy caused by the bizarre ‘convergence zone’ we live in where Pacific weather patterns split around the Olympic mountains and then recombine over Seattle in somewhat unpredictable ways.

Unlike most home automation systems that have fairly limited if-the-else or table-driven approaches to defining the home logic (often limited to only the current value of any variable), my system has control structures that include statistical functions over temporal data allowing analysis of past data (e.g. average temperature in the last day), or in the case of the weather forecast, future values, e.g. expected temperature one hour from now found using interpolation.

                var oneHourForNow = DateTime.Now.AddHours(1);
                double outsideForecastOneHourFromNow = weatherService.ApparentTemperatureHourly.ValueAtTimeWithLinearInterpolation(oneHourForNow);

This forward looking view at the weather allows for features like garden sprinklers that don’t turn on when it’s going to rain or HVAC that skips heating cycles when the forecast predicts warm weather later today.

In addition the house is able to issue a detailed local forecast over the speakers when it wakes you up in the morning. Somewhat uniquely the forecast it generates is a relative weather forecast comparing yesterday with today, or today with tomorrow. It might for example say “Today will be much warmer that yesterday” which is a whole lot less words than a normal weather forecast!

The house also has Natural Language Generation (NLG) features which are able to summarize a group of temporal series into distinct ranges allowing it for instance to highlight which the best times of the day are to be outside:-

Monday : excellent from dawn at 5:34 AM until 11:36 AM; hot from 11:36 AM to 2:00 PM; too hot from 2:00 PM to 7:00 PM; hot until sunset at 8:53 PM.

ASP.NET Charts

Incidentally, all of the graphs are rendered using the .NET Charting Control. The graph object is instantiated on the server, all the lines and axes are added to it, then it is serialized and sent over TCP to the web server using WCF. On the web server it is rendered as a PNG file using an Action method that takes size parameters allowing any size graph to be shown on any page. Here’s the MVC code that takes the stream from WCF, loads the Chart and then delivers it as a PNG.

            Chart chart = new Chart();

            MemoryStream ms2 = new MemoryStream();

            return File(ms2.GetBuffer(), @"image/png");

Is a 24hr energy consumption graph really useful?

Home Energy Consumption Chart

I’m somewhat surprised to see how much effort and excitement Google and Microsoft are putting into the ability to show graphs of home energy consumption minute-by-minute from a smart meter. I’ve had that ability in my house for several years now (not directly measured but a close enough proxy based on knowing which devices are currently on or off).  My conclusion after staring at the graph on and off for several years now is that it’s really not that interesting!

Here’s one of my 24 hour graphs showing the peak, minimum and average electricity consumption in five minute intervals during the day. Fascinating huh?  So what are you going to do with it?

The fact of the matter is that you still need to take a shower and the fridge still needs to stay cool – those peaks really aren’t interesting. If you do want to see how much energy any device is using you can get one of the many Wattage Monitoring Products. They can come in handy in deciding whether to replace your television with a modern LED TV but they probably aren’t going to change the way you behave.

Home Automation Energy Saving

It turns out that the real problem here is the base-load rather than the occasional short-lived peak. Some of that base load is unavoidable but one of the top controllable loads is lights that stay on for hours at a time, so, in your quest to reduce energy consumption, start with them. Replacing them with compact fluorescents or LED lights is an obvious step, but not without drawbacks (lack of dimming, harsher white light, flickering, …).  A better solution is to have a smart home; one that can turn lights off when people aren’t in a room, or can lower the brightness according to the time of day. That’s what my home automation system does and it has produced some fairly dramatic energy savings as a result: over the past 5 years it now uses 40% less electricity than it used to! Unfortunately most home automation systems aren’t smart enough to do this – they will happily plunge you into darkness because you sat still for too long in one room. After years of refinement (both software and hardware) my own system can now accurately assess which rooms are occupied and very rarely does it make a mistake that results in darkness when there should be light. As I mentioned earlier it also prolongs bulb life by running them at less than 100% saving more energy and dramatically reducing how often you need to go up a ladder to change those high-up lights.

This then is really the graph you want to pay attention to and it’s something you can make yourself using the utility bills you get every month. I also think that the energy companies could do more to promote green-envy: simply show people how much less energy their neighbors are using! Shame them into action. Rank them … “you are #1 in your street for energy conservation”. I know some places have started to do something like this but it needs to be widespread.

How to save energy through lighting control with home automation

Home lighting

Lighting probably uses more electricity than anything else in your home

Over the years as I’ve advanced my home automation system I’ve gained a good picture as to how power is used within a home and number one on the list of energy hogs is home lighting.  Maybe in a small house this would be different but in America houses tend to be large and they have lots of lights and those lights are left on for long periods of time.

In addition to replacing some light bulbs with compact fluorescents and some with LED lights (Cree LED’s only since I don’t like flickery-blue light) the house also strives to shut down the lights in any area of the home that’s not occupied.  It aggressively shuts down lights in rooms that have had no motion in them for a while and if it detects someone leaving it looks for more opportunities to shut off lights sooner.

Another unexpected saving that you can make with home automation is to run your incandescent lights at less than 100% brightness.  Since most home automation controlled light switches are dimmable it’s easy to set them to come on at 90% or less.  Based on the time of day my house will use different lighting levels – just 20% if you head to the bathroom late at night, 60% early evening while there’s still natural light and 90% for ‘full-on’.  Not only does this save energy but it prolongs the life of the light bulbs themselves dramatically.  Couple with running at less than 100%, the soft-start that most smart light switches offer provides less thermal shock on the filament and it runs cooler.  And since it takes energy to make light bulbs, and it’s tiresome to go around changing them all the time, this is one saving that pays multiple dividends.

Using Home Automation to Monitor Cable Modem

Monitoring a cable modem

Click to enlarge

Some time ago I was having an issue with my cable modem connection.  The connection was intermittent at certain times of day but fine otherwise.  Calling the cable company with an intermittent problem is pointless: you get the standard answer “But it’s working fine now”.

So, since I have the world’s smartest home automation system running in my closet it was trivial to add a quick HTML scraper to read the cable modem statistics every few minutes, persist them to storage and then add some graphs showing how they vary over time.

The .NET Chart Control is what I’m using now to render these charts.  I’ve tried Google charts, Open Flash Charts and even my own pure-CSS chart generator.  The .NET Chart control has the best options at the moment and it’s a plain JPG file so it will work on any browser.

After a few days I had a better picture as to how the signal strength and signal to noise ratio was varying over time and was quickly able to realize that it was in fact weather related.  Waterproofing the cable connections cured the problem.