Posts tagged Home Automation
Weather Forecasting for Home Automation
Jul 25th
In the USA we are lucky to have the NOAA and their excellent web service that can provide a detailed weather forecast for any location (specified by latitude and longitude). Using this service my home automation system maintains a detailed, regularly updated local weather forecast object which can be queried easily by any other object in the home.
Click to enlarge
On the weather page you can view all of the forecast information it collects. Of interest the graph in the lower right compares the forecast from NOAA with the actual recorded temperature. In this case you can see just how accurate the forecast has been for the last 24 hours. Normally it runs almost this close but with occasionally there is a significant discrepancy caused by the bizarre ‘convergence zone’ we live in where Pacific weather patterns split around the Olympic mountains and then recombine over Seattle in somewhat unpredictable ways.
Unlike most home automation systems that have fairly limited if-the-else or table-driven approaches to defining the home logic (often limited to only the current value of any variable), my system has control structures that include statistical functions over temporal data allowing analysis of past data (e.g. average temperature in the last day), or in the case of the weather forecast, future values, e.g. expected temperature one hour from now found using interpolation.
var oneHourForNow = DateTime.Now.AddHours(1);
double outsideForecastOneHourFromNow = weatherService.ApparentTemperatureHourly.ValueAtTimeWithLinearInterpolation(oneHourForNow);
This forward looking view at the weather allows for features like garden sprinklers that don’t turn on when it’s going to rain or HVAC that skips heating cycles when the forecast predicts warm weather later today.
In addition the house is able to issue a detailed local forecast over the speakers when it wakes you up in the morning. Somewhat uniquely the forecast it generates is a relative weather forecast comparing yesterday with today, or today with tomorrow. It might for example say “Today will be much warmer that yesterday” which is a whole lot less words than a normal weather forecast!
The house also has Natural Language Generation (NLG) features which are able to summarize a group of temporal series into distinct ranges allowing it for instance to highlight which the best times of the day are to be outside:-
Monday : excellent from dawn at 5:34 AM until 11:36 AM; hot from 11:36 AM to 2:00 PM; too hot from 2:00 PM to 7:00 PM; hot until sunset at 8:53 PM.
ASP.NET Charts
Incidentally, all of the graphs are rendered using the .NET Charting Control. The graph object is instantiated on the server, all the lines and axes are added to it, then it is serialized and sent over TCP to the web server using WCF. On the web server it is rendered as a PNG file using an Action method that takes size parameters allowing any size graph to be shown on any page. Here’s the MVC code that takes the stream from WCF, loads the Chart and then delivers it as a PNG.
Chart chart = new Chart();
chart.Serializer.Load(ms);
MemoryStream ms2 = new MemoryStream();
chart.SaveImage(ms2);
return File(ms2.GetBuffer(), @"image/png");
Sequential Logic Blocks – compared to the Reactive Framework
Jul 19th
One of the features of my home automation system is extensions to the C# language that make it easy to define complex logical and temporal behaviors. These behave somewhat like the new Reactive Extensions in .NET but with some key differences which I will explain below. I developed these extensions before Reactive Framework was released and have recently been looking at Rx to see if I could combine my ideas with Rx but because of the differences explained below I haven’t been able to do that.
But, before explaining why, let’s first look at an example:-
FirstFloor.Kitchen.GoesOff.Provided((dt) => Entrance.VisitorCountThisEvening.Count < 2).TurnOff(Aquarium.Light);
This means that if the kitchen goes not occupied (off) provided we have less than two visitors this evening, then turn off the aquarium lights.
Here’s a more complex example where we create an intermediate logic element called ‘activityInKitchen’ using the .Or method, and then based on that activity we decide whether to announce that the fish have not been fed. The decision uses a combination of pulse stretching, repeats (Every) and Then which fires only if a particular sequence is followed within a given time window. Finally it uses the Do method which fires off an Action.
SensorDevice activityInKitchen =
Kitchen.KitchenFloor.Or(Kitchen.MotionSensor, Kitchen.BackDoorToGarage, Kitchen.BreakfastBarFloor, Kitchen.Phone);
activityInKitchen
.ProvidedNot(Home.DinnerGuests)
.PulseStretch(16 * 60) // Make it continuous
.Every(60 * 60) // Once every 1 hour
.Then(activityInKitchen, 15*60)
.Do("Announce fish are hungry", (sender) =>
{
Kitchen.Aquarium.AnnounceFishHungry(Kitchen.MediaZone);
});
As you can see the use of a fluent syntax allows for a very natural, almost english-like definition of complex temporal expressions. But that’s not all it allows…
The sequential logic blocks created when you call one of the many methods like .Then, .Or, .Every, … are actual objects created within the hierarchical structure of the house and the objects used to represent it. This means that they take part (automatically) in all of the features provided by a base house object including logging, browsing, and most importantly persistence.
Because these sequential logic blocks are persistent you can have very long running events like .EveryDays(2) and even if the entire system is shut down and restarted it will come back in the correct state and all the necessary delays and timers will still be working.
Another key benefit of these sequential logic blocks compared to, say, the .NET Reactive Framwork is that they form a chain that can be traversed in either direction: forward as an event propagates through, or in reverse to determine why a particular action was taken. This means that my home automation system is able to explain why it turned the lights on or off. Currently all of these decisions are logged on the per-object logging system which means you can inspect any room, appliance, light, etc. and see everything that happened to it and the reasons why.
Hallway became occupied, caused by Motion sensor
Set brightness on Aisle lights in Kitchen to 20% because Leaving office late at night
Compare this to other home automation systems from major vendors. They contain fairly simple logic, often expressed in a tabular or grid format with limited if-the-else type logic and when something happens there is often no record as to what happened and, worse, there is absolutely no record as to why it happened.
Any sufficiently complex system will have unexpected behavior, the difference here is that when it does something odd I can easily look at the logging and see the explanation as to why it happened.
Is a 24hr energy consumption graph really useful?
May 28th
I’m somewhat surprised to see how much effort and excitement Google and Microsoft are putting into the ability to show graphs of home energy consumption minute-by-minute from a smart meter. I’ve had that ability in my house for several years now (not directly measured but a close enough proxy based on knowing which devices are currently on or off). My conclusion after staring at the graph on and off for several years now is that it’s really not that interesting!
Here’s one of my 24 hour graphs showing the peak, minimum and average electricity consumption in five minute intervals during the day. Fascinating huh? So what are you going to do with it?
The fact of the matter is that you still need to take a shower and the fridge still needs to stay cool – those peaks really aren’t interesting. If you do want to see how much energy any device is using you can get one of the many Wattage Monitoring Products
It turns out that the real problem here is the base-load rather than the occasional short-lived peak. Some of that base load is unavoidable but one of the top controllable loads is lights that stay on for hours at a time, so, in your quest to reduce energy consumption, start with them. Replacing them with compact fluorescents or LED lights is an obvious step, but not without drawbacks (lack of dimming, harsher white light, flickering, …). A better solution is to have a smart home; one that can turn lights off when people aren’t in a room, or can lower the brightness according to the time of day. That’s what my home automation system does and it has produced some fairly dramatic energy savings as a result: over the past 5 years it now uses 40% less electricity than it used to! Unfortunately most home automation systems aren’t smart enough to do this – they will happily plunge you into darkness because you sat still for too long in one room. After years of refinement (both software and hardware) my own system can now accurately assess which rooms are occupied and very rarely does it make a mistake that results in darkness when there should be light. As I mentioned earlier it also prolongs bulb life by running them at less than 100% saving more energy and dramatically reducing how often you need to go up a ladder to change those high-up lights.
This then is really the graph you want to pay attention to and it’s something you can make yourself using the utility bills you get every month. I also think that the energy companies could do more to promote green-envy: simply show people how much less energy their neighbors are using! Shame them into action. Rank them … “you are #1 in your street for energy conservation”. I know some places have started to do something like this but it needs to be widespread.
Applying the Semantic Web to Home Automation
Apr 26th
Recently I’ve been considering how the Semantic Web will impact home automation.
Technologies like the Web Ontology Language (OWL) and RDF allow for the construction of complex ontologies that define what things are, and how they relate. Using these ontologies automated reasoning can be applied to generate new facts or to prove or disprove assertions.
This sounds like the ideal companion to the Natural Language Processing (NLP) Engine that I have already created for the my home automation system. With reasoning powers added to the natural language engine and the ability to augment the knowledge base by adding new assertions the whole system will be much more powerful. One day it might even be possible to create the entire home definition using a natural language text file and to query the system using rich natural language queries.
So, the first step is to find an existing ontology store and reasoning engine. A quick web search reveals that most are built in Java. There were a couple of links I came across later for .NET: http://razor.occams.info/code/semweb/ and http://www.intellidimension.com/products/semantics-server/. There’s also an interesting Q&A site at http://semanticoverflow.com which has lots of useful information on it.
But rather than starting with some existing library I really wanted to understand more deeply how an ontology store works and how a reasoning engine functions, so over the course of a couple of evenings I created my own. I now have a triple store and a simple reasoning engine. Here’s an actual conversation so you can see what it’s capable of so far and can perhaps get a glimpse at how powerful this concept could be:-
house is a class
contains is a property
contain is a property
contains is the same as contain
contain is the same as contains
contains is transitive
contain is transitive
first floor is a class
room is a class
kitchen is a room
first floor contains kitchen
house contains first floor
does house contain kitchenHouse: Yes, house contain kitchen because [house contains first floor] -> [first floor contains kitchen]
As you can see my semantic store can already represent classes, relationships between classes, new relationships (‘contains’), relationships between relationships (‘same as’). For such a small amount of code it’s quite surprising what this system can now handle in terms of knowledge representation and simple reasoning.
Next time I get some spare time I’ll hook it up to the actual home model so you can start to query that in much more powerful ways than before.
Stay tuned!
How to save energy through lighting control with home automation
Apr 9th
Lighting probably uses more electricity than anything else in your home
Over the years as I’ve advanced my home automation system I’ve gained a good picture as to how power is used within a home and number one on the list of energy hogs is home lighting. Maybe in a small house this would be different but in America houses tend to be large and they have lots of lights and those lights are left on for long periods of time.
In addition to replacing some light bulbs with compact fluorescents and some with LED lights (Cree LED’s only since I don’t like flickery-blue light) the house also strives to shut down the lights in any area of the home that’s not occupied. It aggressively shuts down lights in rooms that have had no motion in them for a while and if it detects someone leaving it looks for more opportunities to shut off lights sooner.
Another unexpected saving that you can make with home automation is to run your incandescent lights at less than 100% brightness. Since most home automation controlled light switches are dimmable it’s easy to set them to come on at 90% or less. Based on the time of day my house will use different lighting levels – just 20% if you head to the bathroom late at night, 60% early evening while there’s still natural light and 90% for ‘full-on’. Not only does this save energy but it prolongs the life of the light bulbs themselves dramatically. Couple with running at less than 100%, the soft-start that most smart light switches offer provides less thermal shock on the filament and it runs cooler. And since it takes energy to make light bulbs, and it’s tiresome to go around changing them all the time, this is one saving that pays multiple dividends.
Using Home Automation to Monitor Cable Modem
Apr 8th
Click to enlarge
Some time ago I was having an issue with my cable modem connection. The connection was intermittent at certain times of day but fine otherwise. Calling the cable company with an intermittent problem is pointless: you get the standard answer “But it’s working fine now”.
So, since I have the world’s smartest home automation system running in my closet it was trivial to add a quick HTML scraper to read the cable modem statistics every few minutes, persist them to storage and then add some graphs showing how they vary over time.
The .NET Chart Control is what I’m using now to render these charts. I’ve tried Google charts, Open Flash Charts and even my own pure-CSS chart generator. The .NET Chart control has the best options at the moment and it’s a plain JPG file so it will work on any browser.
After a few days I had a better picture as to how the signal strength and signal to noise ratio was varying over time and was quickly able to realize that it was in fact weather related. Waterproofing the cable connections cured the problem.
The NOAA feed is great for generating custom weather forecasts
Apr 7th
- Click to enlarge
Although my home automation system is mostly an invisible assistant, carrying out its duties in unseen and silent fashion there is a web interface to it too and here’s one of the screens from that web interface. It shows the weather forecast for the area around my house. This information comes from a NOAA feed that is specific to the geographic location (lat/long) that you give it.
Uses for Weather Information by Home Automation
1. In the morning the house will announce the weather for the day. In addition to the usual ‘showers with a high of blah and a low of blah’ type of forecast you can get on any radio or TV station my home automation system gives you a comparative, e.g. ‘it will be much warmer today than yesterday’. Since you know exactly what the weather was like yesterday this simple summary is in many ways more useful than the full forecast and can be more concise.
2. The algorithm that runs the thermostats in the house performs an optimum start in the morning which means it continually calculates what temperature the house needs to be at on 5 minute intervals in order to achieve the target temperature at the target time of day – i.e. 69 degrees when you get up. But some mornings it will see that the weather forecast for the day is such that the heating would be on for a short period of time and after that the sun will provide all the heating you need. On such days it will skip heating the house completely and rely on the sun coming up and the fact that you won’t notice if it’s 67 degrees for an hour in the morning.
A strongly-typed natural language engine (C# NLP)
Feb 28th
Here is an explanation of the natural language engine that powers my home automation system. It’s a strongly-typed natural language engine with tokens and sentences being defined in code. It currently understands sentences to control lights, heating, music, sprinklers, … You can ask it who called, you can tell it to play music in a particular room, … it tells you when a car comes down the drive, when the traffic is bad on I-90, when there’s fresh snow in the mountains, when it finds new podcasts from NPR, … and much more.
The natural language engine itself is a separate component that I hope one day to use in other applications.
Existing Natural Language Engines
- Have a large, STATIC dictionary data file
- Can parse complex sentence structure
- Hand back a tree of tokens (strings)
- Don’t handle conversations
C# NLP Engine
- Defines strongly-typed tokens in code
- Uses type inheritance to model ‘is a’
- Defines sentences in code
- Rules engine executes sentences
- Understands context (conversation history)
Sample conversation
Goals
- Make it easy to define tokens and sentences (not XML)
- Safe, compile-time checked definition of the syntax and grammar (not XML)
- Model real-world inheritance with C# class inheritance:
- ‘a labrador’ is ‘a dog’ is ‘an animal’ is ‘a thing’
- Handle ambiguity, e.g.
play something in the air tonight in the kitchen remind me at 4pm to call john at 5pm
C# NLP Engine Structure
Tokens – Token Definition
- A hierarchy of Token-derived classes
- Uses inheritance, e.g. TokenOn is a TokenOnOff is a TokenState is a Token. This allows a single sentence rule to handle multiple cases, e.g. On and Off
- Derived from base Token class
- Simple tokens are a set of words, e.g. « is | are »
- Complex tokens have a parser, e.g. TokenDouble
A Simple Token Definition
public class TokenPersonalPronoun : TokenGenericNoun
{
internal static string wordz { get { return "he,him,she,her,them"; } }
}
- Recognizes any of the words specified
- Can use inheritance (as in this example)
A Complex Token
public abstract class TokenNumber : Token
{
public static IEnumerable<TokenResult> Initialize(string input)
{
…
- Initialize method parses input and returns one or more possible parses.
TokenNumber is a good example:
- Parses any numeric value and returns one or more of TokenInt, TokenLong, TokenIntOrdinal, TokenDouble, or TokenPercentage results.
The catch-all TokenPhrase
public class TokenPhrase : Token
TokenPhrase matches anything, especially anything in quote marks
e.g. add a reminder "call Bruno at 4pm"
The sentence signature to recognize this could be
(…, TokenAdd, TokenReminder, TokenPhrase, TokenExactTime)
This would match the rule too …
add a reminder discuss 6pm conference call with Bruno at 4pm
TemporalTokens
A complete set of tokens and related classes for representing time
- Point in time, e.g. today at 5pm
- Approximate time, e.g. who called at 5pm today
- Finite sequence, e.g. every Thursday in May 2009
- Infinite sequence, e.g. every Thursday
- Ambiguous time with context, e.g. remind me on Tuesday (context means it is next Tuesday)
- Null time
- Unknowable/incomprehensible time
TemporalTokens (Cont.)
Code to merge any sequence of temporal tokens to the smallest canonical representation,
e.g.
the first thursday in may 2009
->
{TIMETHEFIRST the first} + {THURSDAY thursday} + {MAY in may} + {INT 2009 -> 2009}
->
[TEMPORALSETFINITESINGLEINTERVAL [Thursday 5/7/2009] ]
TemporalTokens (Cont.)
Finite TemporalClasses provide
All TemporalClasses provide
Existing Token Types
- Numbers (double, long, int, percentage, phone, temperature)
- File names, Directories
- URLs, Domain names
- Names, Companies, Addresses
- Rooms, Lights, Sensors, Sprinklers, …
- States (On, Off, Dim, Bright, Loud, Quiet, …)
- Units of Time, Weight, Distance
- Songs, albums, artists, genres, tags
- Temporal expressions
- Commands, verbs, nouns, pronouns, …
Rules – A simple rule
/// <summary>
/// Set a light to a given state
/// </summary>
private static void LightState(NLPState st, TokenLight tlight, TokenStateOnOff ts)
{
if (ts.IsTrueState == true) tlight.ForceOn(st.Actor);
if (ts.IsTrueState == false) tlight.ForceOff(st.Actor);
st.Say("I turned it " + ts.LowerCased);
}
Any method matching this signature is a sentence rule:- NLPState, Token*
Rule matching respects inheritance, and variable repeats … (NLPState st, TokenThing tt, TokenState tokenState, TokenTimeConstraint[] constraints)
Rules are discovered on startup using Reflection and an efficient parse graph is built allowing rapid detection and rejection of incoming sentences.
State – NLPState
- Every sentence method takes an NLPState first parameter
- State includes RememberedObject(s) allowing sentences to react to anything that happened earlier in a conversation
- Non-interactive uses can pass a dummy state
- State can be per-user or per-conversation for non-realtime conversations like email
- Chat (e.g Jabber/Gtalk)
- Web chat
- Calendar (do X at time Y)
- Rich client application
- Strongly-typed natural language engine
- Compile time checking, inheritance, …
- Define tokens and sentences (rules) in C#
- Strongly-typed tokens: numbers, percentages, times, dates, file names, urls, people, business objects, …
- Builds an efficient parse graph
- Tracks conversation history
- Company names, locations, documents, …
- From TimeExpressions
User Interface
Works with a variety of user interfaces
Summary
Future plans
Expanded corpus of knowledge
Generate iCal/Gdata Recurrence
Demonstration of my smart house home automation
Feb 1st
You can now a demonstration of my smart house here: http://blog.abodit.com/wp-content/uploads/2010/01/Smartest-House-by-Ian-Mercer.html
How can I tell if my house is smart?
May 27th
Posted by Ian in Commentary
No comments
So, you’ve been sold on the idea of a smart home and are looking at your options. There’s one that has gorgeous LCD keypads that you could put in every room for just a few hundred dollars a pop, and look! it says in the brochure that it’s “smart”. Well, guess, what, if that LCD keypad has options on it like “Home”, “Away”, or “Entertaining” you are about to purchase what I call a “dumb” or “stupid” home.
If you have to repeatedly tell your house to do an action that would be obvious to a human being then it’s not smart.
So how is a real smart home different from a dumb home?
Well for starters it has no flashy LCD keypads – they simply aren’t necessary – it figures out what to do and does it without being told. Lights come on ahead of you and go off behind you. When you have visitors if knows that and adjusts its behavior accordingly, keeping the house slightly warmer, leaving lights on in areas frequented by visitors, suppressing audible warnings it would normally give, …
Secondly, it has a much more dense sensor network than a dumb house. To reliably detect occupants and distinguish between homeowners, dinner guests, relatives stopping over, and a full-blown party it needs motion sensors in every room, contacts on every door, a sensor on the driveway to detect cars approaching, … and more. With this dense sensor network and algorithms that can track each sensor over long periods of time and perform statistical calculations your home can become a whole lot smarter.
Thirdly, it has a whole new programming paradigm. The traditional if-then-else or the even more lame table driven approach to home automation is not going to be sufficient to make your home smart. It needs a whole new language with the expressive power of what I call ‘sequential logic blocks’ that allow for events to be wired up easily using a fluent, almost natural language approach that hides the complex statistical and timing code that is required by the scenes to make your home smart.
Fourthly, it can remember what it did last night, and the night before than and the month before that and the same time last year! A smart house tracks all of these hundreds of sensors over long periods of time so it can detect trends and can determine what is normal and what is not normal.
Fifthly, it can explain why it did it! Any complex system is going to have unpredictable behavior, that’s almost guaranteed in a Goedel-esque kind of way. But when your smart home does something crazy it’s no good calling the author and saying ‘it went wrong last night, why?’ unless the author has what I have in my house which is a log of what happened and an explanation by the house as to why it happened. My house can, for example, explain that it turned the driveway lights off because it was 9PM and there were no visitors at the house and all of the people who lived there appear to be home.
So give your house the test, figure out how smart it is, and if it’s anywhere above “really dumb”, leave me a note in the comments below.