How do I read a list of json objects from a stream one at a time without them all being in memory.

Mar 31, 2010 at 9:51 PM
Edited Mar 31, 2010 at 10:00 PM

I am writing multiple json objects to a stream using the following code:


_serializer = new JsonSerializer
                                      DefaultValueHandling = DefaultValueHandling.Ignore,
                                      MissingMemberHandling = MissingMemberHandling.Ignore,
                                      NullValueHandling = NullValueHandling.Ignore
_writer = new JsonTextWriter( new StreamWriter(stream ));



// Then this line is called multiple times.
_serializer.Serialize( _writer, message );


How can I read these objects out one at a time without without consuming all my ram. Note, this stream goes to a file that is 1.6gb of json objects. Ideally I would use a yield return.

When I try to use:


_serializer.Deserialize< T >( _reader ); 

it reads to the end of the stream. I only want it to read ONE object, then ONE the next time, etc.



Apr 1, 2010 at 2:27 AM

You could try combining JsonTextReader, a while loop, LINQ to JSON and the Json.NET serializer. It is reasonably complicated and I've never done it myself using Json.NET but I'm pretty sure it would be possible. I did something similar a number of years ago using LINQ to XML. Same concept, just different APIs :)