# Algorithm Complexity and the 'Which one will be faster?' question

Over and over on Stackoverflow.com people ask a question of the form 'If I have an algorithm of complexity A, e.g. O(n.log(n)), and an algorithm of complexity B, which one will be faster?' Developers obsess over using the cleverest algorithm available with the lowest theoretical bound on complexity thinking that they will make their code run faster by doing so. To many developers the simple array scan is anathema, they think they should always use a SortedCollection with binary search, or some heap or tree that can lower the complexity from O(n squared) to O(n.log(n)).

To all these developers I say "You cannot use the complexity of an algorithm to tell you which algorithm is fastest on typical data sets on modern CPUs".

I make that statement from bitter experience having found and implemented a search that could search vast quantities of data in a time proportional to the size of the key being used to search and not to the size of the data being searched. This clever suffix-tree approach to searching was beaten by a simple dumb array scan! How did that happen?

First you need to understand that there are constants involved. An algorithm might be O(n) but the actual instruction execution time for each step might be A microseconds. Another algorithm of O(n.log(n)) might take B microseconds to execute each step. If B >> A then for many values of n the first algorithm will be faster. Each algorithm may also involve a certain set up time and that too can dwarf execution time when n is small.

But here's the catch: even if you think the number of add and multiply instructions between the two algorithms is similar, the CPU you are executing them on may have a very different point of view because modern x86 CPUs have in effect been optimized for dumb programmers. They are incredibly fast at processing sequential memory locations in tight loops that can fit in on-chip RAM. Give them a much more complex tree-based O(n log(n)) algorithm and they now have to go off-chip and access scattered memory locations. The results are sometimes quite surprising and can quickly push that O(n log(n)) algorithm out of contention for values of n less than several million.

For most practical algorithms running on typical datasets the only way to be sure that algorithm A is faster than algorithm B is to profile it.

The other huge catch is that even when you have profiled it and found that your complex algorithm is faster, that's not the end of the story. Modern CPUs are now so fast that memory bottlenecks are often the real problem. If your clever algorithm uses more memory bandwidth than the dumb algorithm it may well affect other threads running on the same CPU. If it consumes so much memory that paging comes into play then any advantage it had in CPU cycles has evaporated entirely.

An example I saw once involved someone trying to save some CPU cycles by storing the result in a private member variable. Adding that member variable made the class larger and as a result less copies of it (and all other classes) could fit in memory and as a result the application overall ran slower. Sure the developer could claim that his method was now 50% faster than before but the net effect was a deterioration in the overall system performance.

## Related Stories

### Time Series Data Compression

This new technique to compress the time series data collected by my home automation system seems to be working really well.

Ian Mercer

### Digital Twins are never identical

Digital Twin are an online representation of a real world object, a copy of its properties in the digital world and a way to send updated and commands to it. In effect I've been making them for years but now they have a trendy name.

Ian Mercer

### Why smarthomes are hard

Why automated learning is hard for a smart home. The perils of over-fitting, under-fitting and how the general unpredictable nature of life makes it hard to build a system that learns your behavior.

Ian Mercer

### Collinearity test for sensor data compression

One way to reduce the volume of sensor data is to remove redundant points. In a system with timestamped data recorded on an irregular interval we can achieve this by removing co-linear points.

Ian Mercer

### Event blocks

Home automation systems need to respond to events in the real world. Sometimes it's an analog value, sometimes it's binary, rarely is it clean and not susceptible to problems. Let's discuss some of the ways to convert these inputs into actions.

Ian Mercer

### Logistic function - convert values to probabilities

Another super useful function for handling sensor data and converting to probabilities is the logistic function 1/(1+e^-x). Using this you can easily map values onto a 0.0-1.0 probability range.

Ian Mercer

### ATAN curve for probabilities

In a home automation system we often want to convert a measurement into a probability. The ATAN curve is one of my favorite curves for this as it's easy to map overything onto a 0.0-1.0 range.

Ian Mercer

Several years ago we did a major remodel. I did all of the finish electrical myself and supervised all of the rough-in electrical. I also put in all of the electrical system and water in our barn. I have opinions ...

Ian Mercer

### T-Mobile home internet

I'm testing a T-Mobile Home Internet device as a backup to XFinity and a way to offload half our monthly traffic to avoid the XFinity 1.2TB cap

Ian Mercer

### My love/hate relationship with Stackoverflow

Stackoverflow is a terrific source of information but can also be infuriating.

Ian Mercer

### iBeacon meetup in Seattle - January 2015

My notes on the iBeacon meetup in Seattle held in January 2015

Ian Mercer

Ian Mercer

### Xamarin Forms Application For Home Automation

Building a Xamarin Forms application to control my home automation system

Ian Mercer

Ian Mercer

Ian Mercer

### JSON Patch - a C# implementation

Ian Mercer

A slightly radical idea to eliminate passwords from many of the websites you use just occasionally

Ian Mercer

### Dynamically building 'Or' Expressions in LINQ

How to create a LINQ expression that logically ORs together a set of predicates

Ian Mercer

### VariableWithHistory - making persistence invisible, making history visible

A novel approach to adding history to variables in a programming language

Ian Mercer

### Neo4j Meetup in Seattle - some observations

Some observations from a meetup in Seattle on graph databases and Neo4j

Ian Mercer

### Updated Release of the Abodit State Machine

A hierarchical state machine for .NET

Ian Mercer

### My first programme [sic]

At the risk of looking seriously old, here's something found on a paper tape

Ian Mercer

### Building a better .NET State Machine

A state machine for .NET that I've released on Nuget

Ian Mercer

### The Internet of Dogs

Connecting our dog into the home automation

Ian Mercer

### A simple state machine in C#

State machines are useful in many contexts but especially for home automation

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

### Exception: An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key.

Help with this exception

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

### Continuous Integration -> Continuous Deployment

What is "quality" in terms of a released software product or website?

Ian Mercer

Ian Mercer

### Making a bootable Windows 7 USB Memory Stick

Here's how I made a bootable USB memory stick for Windows 7

Ian Mercer

### Tip: getting the index in a foreeach statement

A tip on using LINQ's Select expression with an index

Ian Mercer

### Future proof your home with a new conduit system?

Running conduit can be expensive but maybe you don't need one to every room

Ian Mercer

### SQL Server - error: 18456, severity: 14, state: 38 - Incorrect Login

A rant about developers using the same message for different errors

Ian Mercer

### WCF and the SYSTEM account

Namespace reservations and http.sys, my, oh my!

Ian Mercer

Ian Mercer

### Mixed mode assembly errors after upgrade to .NET 4 Beta 2

Fixing this error was fairly simple

Ian Mercer

Ian Mercer

### The EntityContainer name could not be determined

How to fix the exception "the entitycontainer" name could not be determined

Ian Mercer

### Shortened URLs should be treated like a Codec ...

Expanding URLs would help users decide whether or not to click a link

Ian Mercer

### Tagging File Systems

Isn't it time we stopped knowing which drive our file is on?

Ian Mercer

### A great site for developing and testing regular expressions

Just a link to a site I found useful

Ian Mercer

### WMPnetwk.exe started using 50% of my CPU

Uninstalling Windows Media Player - the end of an era

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer

### A better Tail program for Windows

A comparison of tail programs for Windows

Ian Mercer

### Measuring website browser performance

Found this great resource on website performance

Ian Mercer

### Amazon Instance vs Dedicated Server comparison

Some benchmark performance for Amazon vs a dedicated server

Ian Mercer

Ian Mercer

### System.Data.EntitySqlException

Hints for dealing with this exception

Ian Mercer

### Agile Software Development is Like Sailing

You cannot tack too often when sailing or you get nowhere. Agile is a bit like that.

Ian Mercer

### Exception Handling using Exception.Data

My latest article on CodeProject covers the lesser known Exception.Data property

Ian Mercer

### Javascript error reporting

Sending client-side errors back to a server for analysis

Ian Mercer

### AntiVirus Software is the Worst Software!

When your anti-virus software starts stealing your personal data, it's time to remove it!

Ian Mercer

### ASP.NET Custom Validation

How to solve a problem encountered with custom validation in ASP.NET

Ian Mercer

Ian Mercer

LinqKit came in handy back in 2009

Ian Mercer

### It's all about disk speed

Why disk speed is the most critical aspect for most modern PCs and servers

Ian Mercer

Ian Mercer

### Cache optimized scanning of pairwise combinations of values

Using space-filling curves to optimize caching

Ian Mercer

Ian Mercer

### Take out the trash!

Why Windows shutdown takes so long

Ian Mercer

Ian Mercer

Ian Mercer

Ian Mercer