Serializing Null Objects

Nov 20, 2011 at 6:02 AM

Hi There,

I was looking to create a custom converter for ICollection and Array objects that converted null Collection or Array into an empty array in JSON.  After writing up the converter I was surprised to see that my converter wasn't fired when null collections or arrays were encountered.  After dipping into the Json.Net source code I think I found the culpret in Newtonsoft.Json.Serialization.JsonSerializerInternalWriter:

private void SerializeValue(JsonWriter writer, object value, JsonContract valueContract, JsonProperty member, JsonContract collectionValueContract)
{
      JsonConverter converter = (member != null) ? member.Converter : null;

      // NOTE: This handling of NULL appears before the converters are called.
      if (value == null)
      {
        writer.WriteNull();
        return;
      }

      // The Converters are called here...
}

 

So it seems nulls are handled before the converters are called.  Is this by design?  It seems to me this takes some of the control of the serialization process away from the developer. I had a look through some of the standard stock converters that were shipped with Json.Net and I noticed the StringEnumConverter handles null values in it's WriteJson overload, so I suspect this isnt the inteded behaviour?

Thanks!

Coordinator
Nov 20, 2011 at 9:33 AM

It is. All the converters work by matching the value's type in the CanConvert method. If the value is null there is no value to pass into CanConvert.

I have some code to handle nulls just in case someone calls the converter outside of the serializer with a null value.

Nov 20, 2011 at 1:36 PM

Hey James,

Yep, that makes sense; although wouldn't it be better to retrieve the type information from the Property that holds reference to the object rather than retrieving the type information from the object itself?  I know you couldn't really do this with the root of the object graph itself, but surely you could do that with all properties within the object graph?  Having the ability to handle nulls in my custom converters would be quite useful to me.

Thanks for the response mate.

Nov 21, 2011 at 11:40 PM

I'm considering modifying JSON.Net so it passes NULL values to the converters, and get's it's type information from the MemberInfo instead of the object itself.  Can anyone give me an indication on how difficult this would be?  If I achieve this, would you consider it as a patch for JSON.Net in the future?

Thanks!

Coordinator
Nov 22, 2011 at 1:36 AM

It could be as simple as using the MemberInfo (or collection contents) type when calling CanConvert if the value is null.

I might include it as a patch if you can do it simply. The only case where it is useful is for null values, and for such a narrow situation (I don't think anyone has ever asked for it before) I don't want to complicate things.

Nov 22, 2011 at 2:12 AM
Edited Nov 22, 2011 at 2:15 AM

Ok, no worries James.  I'll look into it.

I'm currently working on a system that tightly integrates MVC and Knockout JS together.  I'm using JSON.Net to serialize my server side ViewModel into a JSON structure which Knockout JS then turns into an observable javascript view model.  When Knockout JS traverses the JSON object graph it looks at each property and figures out if it should be converted to an observable object, or an observableArray.  Since JSON is not strongly typed, when Knockout JS encounters a NULL where you would normally expect a collection, it converts it to a observable object instead of an observable array.  To solve this problem, I need to make sure all of my collections are empty and not null so knockout JS can see that it's suppose to convert them to observableArrays on the client side.

Nov 24, 2011 at 7:46 PM

http://json.codeplex.com/SourceControl/list/patches

ID: 10926

I have submitted a patch that should handle null, 0 and 1 sized arrays and collections. Hopefully you find it useful.

Nov 24, 2011 at 11:26 PM

Thanks for that mate, I'll check it out. 

As for the handling of NULL objects, I've managed to make a few tweaks to JsonSerializerInternalWriter allow JSON.Net to pass null values through to Converters.  Instead of pulling the type information the Converters need from the value being converted, I'm pulling the type information from the Member being serialized.  I'll create a patch for this and publish it soon.

Lucas

Jun 14, 2012 at 1:21 AM

Hi duckaroy,

Did you ever get this resolved? I'm trying to do exactly the same thing (using KO as well) and am having some trouble getting a custom converter to work. This is what I've got so far:

using System;
using System.Collections;
using System.Linq;
using Newtonsoft.Json;

namespace blah
{
   /// <summary>
   /// A convert for IEnumerable types used to force null values to be serialized as empty arrays.
   /// </summary>
   public class NullCollectionConverter : JsonConverter
   {
      public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
      {
         if (value != null)
         {
	    // Trying to signal that writing has failed here - doesn't work
            throw new JsonSerializationException();
         }

         // The passed IEnumerable is null so serialize it as an empty array.
         writer.WriteStartArray();
         writer.WriteEndArray();
      }

      public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
      {
         return serializer.Deserialize(reader, objectType);
      }

      public override bool CanConvert(Type objectType)
      {
         return objectType.GetInterfaces().Any(t => t == typeof(IEnumerable));
      }
   }
}
Any help would be much appreciated.
Cheers, Andrew.
Jun 14, 2012 at 1:53 AM

Heya Andrew,

I ended up forking JSON.Net and hacking it myself to get it working.  I'd like to be able to submit my changes as a patch as soon as I can be sure it is working 100%.  It's kind of a background task at the moment while I work on a project of mine. 

You can check out my code at https://github.com/CogShift/Json.Net.  I've checked in about 4 commits which solve this problem.

Also, the converter I wrote is as follows:

    /// <summary>
    /// A write only JSON.Net converter that converts null collections and arrays into empty JSON arrays.
    /// </summary>
    /// <remarks>
    /// This json converter will only work with hacks done to JSON .Net that allows null values to be processed by converters.  The
    /// CogShift customised version of JSON.Net supports this.
    /// </remarks>
    public class JsonNullCollectionConverter : JsonConverter
    {

        public override bool CanRead { get { return false; } }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            writer.WriteStartArray();

            if (value != null)
            {
                var collection = value as IEnumerable;

                foreach (var item in collection)
                    serializer.Serialize(writer, item);
            }

            writer.WriteEndArray();
        }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            throw new NotSupportedException();
        }

        public override bool CanConvert(Type objectType)
        {
            return objectType.IsImplementationOf(typeof(ICollection<>)) || objectType.IsImplementationOf(typeof(ICollection)) || objectType.IsArray;
        }
    }

Lucas

Jun 14, 2012 at 2:30 AM
Hey Lucas,

Thanks for the reply, that looks like just what I need. Just one
thing: where are you getting IsImplementationOf from?

Cheers, Andrew.

On Thu, Jun 14, 2012 at 12:53 PM, duckaroy <notifications@codeplex.com> wrote:
> From: duckaroy
>
> Heya Andrew,
>
> I ended up forking JSON.Net and hacking it myself to get it working.  I'd
> like to be able to submit my changes as a patch as soon as I can be sure it
> is working 100%.  It's kind of a background task at the moment while I work
> on a project of mine.
>
> You can check out my code at https://github.com/CogShift/Json.Net.  I've
> checked in about 4 commits which solve this problem.
>
> Also, the converter I wrote is as follows:
>
> /// <summary>
> /// A write only JSON.Net converter that converts null collections and
> arrays into empty JSON arrays.
> /// </summary>
> /// <remarks>
> /// This json converter will only work with hacks done to JSON .Net that
> allows null values to be processed by converters. The
> /// CogShift customised version of JSON.Net supports this.
> /// </remarks>
> public class JsonNullCollectionConverter : JsonConverter
> {
>
> public override bool CanRead { get { return false; } }
>
> public override void WriteJson(JsonWriter writer, object value,
> JsonSerializer serializer)
> {
> writer.WriteStartArray();
>
> if (value != null)
> {
> var collection = value as IEnumerable;
>
> foreach (var item in collection)
> serializer.Serialize(writer, item);
> }
>
> writer.WriteEndArray();
> }
>
> public override object ReadJson(JsonReader reader, Type objectType,
> object existingValue, JsonSerializer serializer)
> {
> throw new NotSupportedException();
> }
>
> public override bool CanConvert(Type objectType)
> {
> return objectType.IsImplementationOf(typeof(ICollection<>)) ||
> objectType.IsImplementationOf(typeof(ICollection)) || objectType.IsArray;
> }
> }
>
> Lucas
>

--
Andrew Groom  <[email removed]>  http://andrewgroom.com/
Mobile: +64 274 992 569
Skype: andrewgee
Twitter: http://twitter.com/andrewchch
Jun 14, 2012 at 3:39 AM

Ah, that would be one of my custom Type extension methods.  Here's the code for it:

public static class TypeExtensions
{
    /// <summary>
    /// Checks to see if the type implements a given interface.
    /// </summary>
    /// <param name="baseType">The type to be extended.</param>
    /// <param name="interfaceType">The interface to check for on the type.</param>
    /// <returns>Returns <code>true</code> if the type implements the given interface.</returns>
    public static bool IsImplementationOf(this Type baseType, Type interfaceType)
    {
        if (baseType == interfaceType || (baseType.IsGenericType && baseType.GetGenericTypeDefinition() == interfaceType))
            return true;

        return baseType.GetInterfaces().Any(e => (e.IsGenericType && e.GetGenericTypeDefinition() == interfaceType) || (!e.IsGenericType && e == interfaceType));
    }
}

Lucas

Jun 14, 2012 at 3:48 AM

Thanks again, Lucas. I tried your converter without any changes to JSON.Net but had the same problem, plus a new one - some of my collections want to serialize themselves in a certain way, so iterating over the collection and serializing each item gave me a different result.

I want to avoid patching JSON.Net so I guess I'll have to implement a workaround until your patches are adopted. Thanks again for your help.

Cheers, Andrew.

Jun 14, 2012 at 4:53 AM

Yeah, I know what you mean - I really didn't want to hack JSON.Net either. 

From memory it seems JSON.Net makes the asumption for you that a NULL collection property in your CLR class = a Null value in resulting JSON string. Because of this, JSON.Net wont invoke any of your custom converters when it encounters a null collection property.  Unfortunately modifying the JSON.Net source code is the only way I've found to fix issue. 

Happy coding!

Lucas

Jun 14, 2012 at 5:45 AM

Thanks, Lucas. After all that, it turns out I have a workaround of setting null collection properties on my data model to empty properties before they get serialised so the problem is solved for now. 

Jun 14, 2012 at 7:01 AM

Ah yes, that also works :)

Feb 18, 2013 at 8:48 AM
Hi Lucus

I download your code from https://github.com/CogShift/Json.Net and try to test with
    /// <summary>
    /// A write only JSON.Net converter that converts null collections and arrays into empty JSON arrays.
    /// </summary>
    /// <remarks>
    /// This json converter will only work with hacks done to JSON .Net that allows null values to be processed by converters.  The
    /// CogShift customised version of JSON.Net supports this.
    /// </remarks>
    public class JsonNullCollectionConverter : JsonConverter
    {

        public override bool CanRead { get { return false; } }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            writer.WriteStartArray();

            if (value != null)
            {
                var collection = value as IEnumerable;

                foreach (var item in collection)
                    serializer.Serialize(writer, item);
            }

            writer.WriteEndArray();
        }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            throw new NotSupportedException();
        }

        public override bool CanConvert(Type objectType)
        {
            return objectType.IsImplementationOf(typeof(ICollection<>)) || objectType.IsImplementationOf(typeof(ICollection)) || objectType.IsArray;
        }
    }
but it doesn't work to me.

Please could you give me some suggestion
Feb 18, 2013 at 9:11 AM
Sorry for my mistake the null array I should test. It must be a property of object.
Aug 2, 2013 at 4:36 PM
Just ran into this problem as well. It seems to me that JSON.NET should either:
(a) pass a Type of null to CanConvert; this is likely to cause issues for existing custom JsonConverters when upgrading but seems reasonable; or
(b) add a separate CanConvertNull method that a custom JsonConverter can override but that will return null by default if they don't override it
Aug 3, 2013 at 11:32 PM
Filed as issue [enhancement]: https://json.codeplex.com/workitem/24577
Aug 3, 2013 at 11:47 PM