The Blog of Ian Mercer.

Remember Everything ... a long-term project

Here's a working example of the kind of semantic and mathematical summarization possible.

One of my long-term project ideas that is gradually coming together piece-by-piece is a system I call "Remember Everything" (for want of a better name).

"Remember Everything" connects nearly all of my projects into one giant solution that, well, remembers everything and has a natural language interface over it.

As inputs it will take information from my home automation system, my whole-network storage crawler, Google calendar, email, Twitter, blog, web crawler, an address-monitoring browser add-on I plan to write, the weather and traffic feeds, and, of course my natural language engine.

All this data will be put into MongoDB and can then be queried. Relationships between entities will be created using a semantic-web triple store and reasoner.

Together these capabilities will allow queries like:-

  • Copy all the photos I took last week onto c:\vacationPhotos
  • Send img_0938.jpg to mum.
  • Who called last Monday?
  • Show pictures from last month taken on sunny days.
  • What was happening two weeks ago when X called?
  • Who called yesterday when I was in a meeting?
  • What song was playing around 9pm last night?
  • How long did I spend on the phone to my accountant last week?
  • What web pages did I read last week about the Semantic Web?
  • Send the web page I tweeted about last night to my Kindle.
  • We need butter and olives.
  • What do I need to buy from QFC? (a semantic shopping list concept, more on that later ...)

In addition to the shopping lists concept (that's already in my home automation system but lacks the semantic reasoning) the system will take any subject-verb-object phrase and remember it and then allow you to query it back later, e.g.

  • My son read 20 pages tonight (making the weekly reading report easier)
  • How many pages did he read this week?
  • I took the red pill at 10AM
  • I walked 2 miles this morning
  • I ran 4 miles
  • How much exercise did I do this week when it wasn't raining? (summarizing values semantically and mathematically)
  • The Audi was serviced this week (remembering schedules so you can check if an item is overdue)
  • My BA frequent flyer number is #### (remembering numbers you need to look up often)
  • I took the day off on friday (vacation reporting)
  • I spent $12.95 on lunch (expense reporting) ...

Whenever you have anything you need to remember the system will be able to remember it, recall it, and where possible aggregate or summarize it using math and/or semantic reasoning (e.g. running subClassOf exercise, butter subClassOf dairy product, dairy products areSoldAt QFC, ...).

By linking my natural language engine to a triple store I can even allow users to teach it new concepts:

Using Natural Language to populate a triple store

By silently monitoring your email, Twitter stream, calendar, activity in the house, ... it will be able to answer questions based on the context not just on the content in ways that we take for granted as humans but which are not possible for computers today.

Related Stories

Natural Language Processing

I could not find a Natural Language Processing engine when I needed one for my home automation system so I developed my own. After 10 years of on and off development I now have a unique NLP engine for .NET that is easy to configure but incredibly powerful for precise command and control applications. It doesn't use a tokenizer so it doesn't care if you run words together.

Ian Mercer
Ian Mercer

Neo4j Meetup in Seattle - some observations

Some observations from a meetup in Seattle on graph databases and Neo4j

Ian Mercer
Ian Mercer

MongoDB substring search with a difference

Ian Mercer
Ian Mercer

MongoDB - Map-Reduce coming from C#

Ian Mercer
Ian Mercer

MongoDB Map-Reduce - Hints and Tips

Ian Mercer
Ian Mercer

A great video explaining the Semantic Web

Ian Mercer
Ian Mercer

World's Smartest House

Over 15 years of experimentation with home automation

Ian Mercer
Ian Mercer