How do I read a list of json objects from a stream one at a time without them all being in memory.

Mar 31, 2010 at 10:51 PM
Edited Mar 31, 2010 at 11:00 PM

I am writing multiple json objects to a stream using the following code:

 

_serializer = new JsonSerializer
                              {
                                      DefaultValueHandling = DefaultValueHandling.Ignore,
                                      MissingMemberHandling = MissingMemberHandling.Ignore,
                                      NullValueHandling = NullValueHandling.Ignore
                              };
_writer = new JsonTextWriter( new StreamWriter(stream ));

 

 

// Then this line is called multiple times.
_serializer.Serialize( _writer, message );

 

How can I read these objects out one at a time without without consuming all my ram. Note, this stream goes to a file that is 1.6gb of json objects. Ideally I would use a yield return.

When I try to use:

 

_serializer.Deserialize< T >( _reader ); 

it reads to the end of the stream. I only want it to read ONE object, then ONE the next time, etc.

 

 

Coordinator
Apr 1, 2010 at 3:27 AM

You could try combining JsonTextReader, a while loop, LINQ to JSON and the Json.NET serializer. It is reasonably complicated and I've never done it myself using Json.NET but I'm pretty sure it would be possible. I did something similar a number of years ago using LINQ to XML. Same concept, just different APIs :)