Posts tagged MongoDB

In an earlier post I explained a technique to create a class-free persistence layer using MongoDB. [Read that post first, then come back here.]

Since then I’ve refined the techniques involved and created a cleaner implementation that does away with the `.props` collection on each object. Now when you add an interface to an object you get exactly what you expected in the persisted data.

To use it you first need to register the serialization code somewhere in your startup code…

            BsonSerializer.RegisterSerializationProvider(new MongoDynamicSerializationProvider());

The Serialization provider is quite simple:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MongoDB.Bson.Serialization;

namespace MongoData.Dynamic
{
    public class MongoDynamicSerializationProvider : IBsonSerializationProvider
    {

        public IBsonSerializer GetSerializer(Type type)
        {
            if (typeof(MongoDynamic).IsAssignableFrom(type))
                return MongoDynamicBsonSerializer.Instance;
            return null;
        }
    }
}

The serializer is a bit more involved. It uses an interface map to decide what type to return for each serialized object. This is critical because many different .NET types can map onto the same BSon serialized value and only by maintaining this map can we get back to the original type. It’s also
critical for handling nested object graphs containing different types.

using System;
using System.Collections.Concurrent;
using System.Dynamic;
using System.Linq;
using System.Linq.Expressions;
using System.Runtime.CompilerServices;
using Microsoft.CSharp.RuntimeBinder;
using MongoDB.Bson.IO;
using MongoDB.Bson.Serialization;
using MongoDB.Bson.Serialization.Serializers;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.IdGenerators;
using System.Collections.Generic;
using ImpromptuInterface;

namespace MongoData.Dynamic
{
    public class MongoDynamicBsonSerializer : BsonBaseSerializer
    {
        private static MongoDynamicBsonSerializer instance = new MongoDynamicBsonSerializer();

        public static MongoDynamicBsonSerializer Instance
        {
            get { return instance; }
        }

        public override object Deserialize(BsonReader bsonReader, Type nominalType, IBsonSerializationOptions options)
        {
            var bsonType = bsonReader.CurrentBsonType;
            if (bsonType == BsonType.Null)
            {
                bsonReader.ReadNull();
                return null;
            }
            else if (bsonType == BsonType.Document)
            {
                var os = new ObjectSerializer();
                MongoDynamic md = new MongoDynamic();
                bsonReader.ReadStartDocument();

                Dictionary<string, Type> typeMap = null;

                // scan document first to find interfaces
                {
                    var bookMark = bsonReader.GetBookmark();
                    if (bsonReader.FindElement(MongoDynamic.InterfacesField))
                    {
                        md[MongoDynamic.InterfacesField] = BsonValue.ReadFrom(bsonReader).AsBsonArray.Select(x => x.AsString);
                        typeMap = md.GetTypeMap();
                    }
                    else
                    {
                        throw new FormatException("No interfaces defined for this dynamic object - can't deserialize it");
                    }
                    bsonReader.ReturnToBookmark(bookMark);
                }

                while (bsonReader.ReadBsonType() != BsonType.EndOfDocument)
                {
                    var name = bsonReader.ReadName();

                    if (name == "_id")
                    {
                        md[name] = BsonValue.ReadFrom(bsonReader).AsObjectId;
                    }
                    else if (name == MongoDynamic.InterfacesField)
                    {
                        // Read it and ignore it, we already have it
                        BsonValue.ReadFrom(bsonReader);
                    }
                    else
                    {
                        if (typeMap == null) throw new FormatException("No interfaces define for this dynamic object - can't deserialize");
                        // lookup the type for this element according to the interfaces
                        Type elementType;
                        if (typeMap.TryGetValue(name, out elementType))
                        {
                            var value = BsonSerializer.Deserialize(bsonReader, elementType);
                            md[name] = value;
                        }
                        else
                        {
                            // This is a value that is no longer in the interface, maybe a column you removed
                            // not really much we can do with it ... but we need to read it and move on
                            var value = BsonSerializer.Deserialize(bsonReader, typeof(object));
                            md[name] = value;

                            // As with all databases, removing elements from the schema is always going to cause problems ... 
                        }
                    }
                }
                bsonReader.ReadEndDocument();
                return md;
            }
            else
            {
                var message = string.Format("Can't deserialize a {0} from BsonType {1}.", nominalType.FullName, bsonType);
                throw new FormatException(message);
            }
        } 
    

        public override bool GetDocumentId(object document, out object id, out Type idNominalType, out IIdGenerator idGenerator)
        {
            MongoDynamic x = (MongoDynamic)document;
            id = x._id;
            idNominalType = typeof(ObjectId);
            idGenerator = new ObjectIdGenerator();
            return true;
        }

        public override void SetDocumentId(object document, object id)
        {
            MongoDynamic x = (MongoDynamic)document;
            x._id = (ObjectId)id;
        }

        public override void Serialize(BsonWriter bsonWriter, Type nominalType, object value, IBsonSerializationOptions options)
        {
            if (value == null)
            {
                bsonWriter.WriteNull();
                return;
            }
            var metaObject = ((IDynamicMetaObjectProvider)value).GetMetaObject(Expression.Constant(value));
            var memberNames = metaObject.GetDynamicMemberNames().ToList();
            if (memberNames.Count == 0)
            {
                bsonWriter.WriteNull();
                return;
            }

            bsonWriter.WriteStartDocument();
            foreach (var memberName in memberNames)
            {
                // ToDo: handle all those _id Id id variants?
                bsonWriter.WriteName(memberName);

                object memberValue;
                if (memberName == "_id") memberValue = ((MongoDynamic)value)._id;
                else if (memberName == "int") memberValue = ((MongoDynamic)value).@int;
                else memberValue = Impromptu.InvokeGet(value, memberName);

                if (memberValue == null)
                    bsonWriter.WriteNull();
                else
                {
                    var memberType = memberValue.GetType();
                    var serializer = BsonSerializer.LookupSerializer(memberType);
                    serializer.Serialize(bsonWriter, memberType, memberValue, null);
                }
            }
            bsonWriter.WriteEndDocument();
        }
    }
}

And finally, the actual MongoDynamic class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Dynamic;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using ImpromptuInterface;

namespace MongoData.Dynamic
{
    /// <summary>
    /// All MongoDynamic objects support this interface because every object needs an _id in MongoDB
    /// </summary>
    public interface IId
    {
        ObjectId _id { get; set; }
    }

    /// <summary>
    /// MongoDynamic is like an ExpandoObject that also understands document Ids and uses Improptu interface
    /// to act like any other collection of interfaces ...
    /// It can be serialized and deserialized from BSon and thus stored in a MongoDB database.
    /// </summary>
    /// <remarks>
    /// This simple class gives you the ability to define database objects using only .NET interfaces - no classes!
    /// Those objects can be dynamically extended to support any interface you want to add to them - polymorphism!
    /// When loaded back from the database the object will support all of the interfaces that were ever applied to it.
    /// Adding a new field is easy.  Removing one works too.
    /// All fields must be nullable since they may not be present on earlier instances of an object type.
    /// </remarks>
    public class MongoDynamic : DynamicObject, IId
    {
        [BsonId(Order=1)]
        public ObjectId _id { get; set; }

        // Dumb name for a property - which is why I chose it - very unlikely it will ever conflict with a real property name
        public const string InterfacesField = "int";

        /// <summary>
        /// Interfaces that have been added to this object
        /// </summary>
        /// <remarks>
        /// We always begin by supporting the _id interface
        /// Order is important, we need to see this field before we can deserialize any others
        /// </remarks>
        [BsonElement(InterfacesField, Order=2)]
        internal HashSet<string> @int = new HashSet<string>(){ typeof(IId).FullName };

        /// <summary>
        /// A text version of all interfaces - mostly for debugging purposes, stored in alphabetical order
        /// </summary>
        [BsonIgnore]
        public string InterfacesAsText
        {
            get { return string.Join(",", this.@int.OrderBy(i => i)); }
        }

        /// <summary>
        /// Add support for an interface to this document if it doesn't already have it
        /// </summary>
        public T AddLike<T>()
            where T : class
        {
            @int.Add(typeof(T).FullName);
            // And also act like any interfaces that interface implements (which will include ones they represent too)
            foreach (var @interface in typeof(T).GetInterfaces())
                @int.Add(@interface.FullName);
            return Impromptu.ActLike<T>(this, this.GetAllInterfaces());
        }

        /// <summary>
        /// Add support for multiple interfaces
        /// </summary>
        public T AddLike<T>(Type[] otherInterfaces)
            where T : class
        {
            var allInterfaces = otherInterfaces.Concat(new[] { typeof(T) });
            var allInterfacesAndDescendants = allInterfaces.Concat(allInterfaces.SelectMany(x => x.GetInterfaces()));
            foreach (var @interface in allInterfacesAndDescendants)
                @int.Add(@interface.FullName);
            return Impromptu.ActLike<T>(this, this.GetAllInterfaces());
        }

        /// <summary>
        /// Cast this object to an interface only if it has previously been created as one of that kind
        /// </summary>
        public T AsLike<T>()
            where T : class
        {
            if (!this.@int.Contains(typeof(T).FullName)) return null;
            else return Impromptu.ActLike<T>(this, this.GetAllInterfaces());
        }

        // A rather large cache of all interface types loaded into the App Domain
        private static List<Type> cacheOfTypes = null;

        // A cache of the interface types corresponding to a given 'key' of interface names
        private static Dictionary<string, Type[]> cacheOfInterfaces = new Dictionary<string, Type[]>();

        public Type[] GetAllInterfaces()
        {
            // We always behave like an object with an Id plus any other interfaces we have
            var key = string.Join(",", this.@int.OrderBy(i => i));
            if (!cacheOfInterfaces.ContainsKey(key))
            {
                if (cacheOfTypes == null)
                {
                    var assemblies = AppDomain.CurrentDomain.GetAssemblies();
                    cacheOfTypes = assemblies.SelectMany(ass => ass.GetTypes()).Where(t => t.IsInterface).ToList();
                }
                var interfaces = cacheOfTypes.Where(t => this.@int.Any(i => i == t.FullName));

                // Could trim the interfaces to remove any that are inherited from others ...
                cacheOfInterfaces.Add(key, interfaces.ToArray());
            }
            return cacheOfInterfaces[key];
        }

        /// <summary>
        /// Get a mapping from a field name to a type according to the interfaces on this object
        /// </summary>
        /// <returns></returns>
        public Dictionary<string, Type> GetTypeMap()
        {
            Dictionary<string, Type> typeMap = new Dictionary<string, Type>();
            var interfaces = this.GetAllInterfaces();
            foreach (var mi in interfaces.SelectMany(intf => intf.GetProperties()))
            {
                typeMap[mi.Name] = mi.PropertyType;
            }
            return typeMap;
        }


        /// <summary>
        /// Becomes a Proxy object that acts like it implements all of the interfaces listed as being supported by this Entity
        /// </summary>
        /// <remarks>
        /// Because the returned object supports ALL of the interfaces that have ever been added to this object
        /// you can cast it to any of them.  This enables a type of polymorphism.
        /// </remarks>
        public object ActLikeAllInterfacesPresent()
        {
            return Impromptu.DynamicActLike(this, this.GetAllInterfaces());
        }

        [BsonIgnore]
        // BsonIgnore because Bson serialization will happen on the dynamic interface this class exposes not on this dictionary
        private Dictionary<string, object> children = new Dictionary<string, object>();

        /// <summary>
        /// Fetch a property by name
        /// </summary>
        public override bool TryGetMember(GetMemberBinder binder, out object result)
        {
            if (binder.Name == "_id") { result = this._id; return true; }
            else if (binder.Name == InterfacesField) { result = this.@int; return true; }
            else 
            {
               children.TryGetValue(binder.Name, out result); 
               result = null;                         // we hope that it's nullable!  If not you have an issue 
               return true;                           // when you do a database migration or query a nullable field it won't be in 'children'
            }
        }

        /// <summary>
        /// Set a property (e.g. person1.Name = "Smith")
        /// </summary>     
        public override bool TrySetMember(SetMemberBinder binder, object value)
        {
            if (binder.Name == "_id") { this._id = (ObjectId)value; return true; }      // you shouldn't need to use this
            if (binder.Name == InterfacesField) throw new AccessViolationException("You cannot set the interfaces directly, use AddLike() instead");
            if (!this.GetTypeMap().ContainsKey(binder.Name)) throw new ArgumentException("Property '" + binder.Name + "' not found.  You need to call AddLike to specify the interfaces you want to support."); 
            children[binder.Name] = value;
            return true;
        }

        public override IEnumerable<string> GetDynamicMemberNames()
        {
            return new[]{"_id", InterfacesField}.Concat(children.Keys);
        }

        /// <summary>
        /// An indexer for use by serialization code
        /// </summary>
        internal object this[string key]
        {
            get
            {
                if (key == "_id") return this._id;
                else if (key == InterfacesField) return this.@int;
                else return children[key];
            }

            set
            {
                if (key == "_id" && value is BsonObjectId) this._id = ((BsonObjectId)value).Value;
                else if (key == "_id") this._id = (ObjectId)value;
                else if (key == InterfacesField) this.@int = new HashSet<string>((IEnumerable<string>)value);
                else children[key] = value;
            }
        }
    }
}

You’ll need Impromptu interface (from Nuget) to build this. To use it, you write code like this to save to MongoDB:

            MongoDynamic entity = new MongoDynamic();
            var user = entity.AddLike<IUser>();         // *** Add the IUser fields to it ...
            user.Name = name;                           // Use it as if it were an IUser
            // save it to the database as normal

And to retrieve an object you create a query as normal and then query for MongoDynamic objects like so …

            var user = database.GetCollection<MongoDynamic>("***collectionName***").FindOne(query);
            if (user == null) return null;
            return user.AsLike<IUser>();

Typically you will want your query to reference the field called int (where all the interfaces are stored) so you can query for objects that support a specific type (if you do, you’ll want to add an index on that field). [NB the name was chosen to be one you were unlikely to ever use in .NET]

MongoDynamic objects are polymorphic – you can morph them to support any other interface at any time like so …

            user.AddLike<ISomeOtherInterface>();