incremental parsing?

May 18, 2011 at 9:26 PM

Hi - does Json.NET support incremental parsing? I need to receive a continuous stream of data, and as such I won't know whether I have a full object's worth of data at any given moment. The deserialized data does not correspond to object instances, rather just passing messages around, so I don't necessarily need to validate it against a specific contract.

Is there a recommended approach to this using Json.NET? It looks like I could try to use JsonReader directly, but I haven't found any examples that provide a best practices example. Thanks for any info!

May 19, 2011 at 6:46 PM

OK now I've got a nice incremental parser built up around a JsonTextReader, but it seems as though a single JsonReader instance can only read a single Json object/message. Feeding JsonReader a stream with multiple consecutive messages (I also tried delimiting them with \0) results in the following exception: CreateJsonReaderException("After parsing a value an unexpected character was encountered: {0}. Line {1}, position {2}.", currentChar, _currentLineNumber, _currentLinePosition);

If my stream contains consecutive messages, do I need to create a new JsonReader for each message, or have I missed something in the API? Seems like it would be nicer if it could continue reading as many messages as are available in the stream.

May 20, 2011 at 5:49 PM

OK - I've actually got another little problem with this approach. If the incoming JSON stream does not contain a full message, JsonTextReader will throw a generic Newtonsoft.Json.JsonReaderException: After parsing a value an unexpected character was encountered: 8. Line 1, position 22." error. It would be much nicer if it could indicate normally that it's out of data (ideally without throwing an exception, if possible). Even if it did throw an exception, a more specific "I'm out of data" exception would be better, so it could be handled separately.

For example, from a sample python script, I can send a single message in two separate writes, separated by a 1 second sleep:

sock.send("{\"hi there\": 4")
time.sleep(1)
sock.send("5678} ")

Right now, I need to use something like the following to handle this case:

while (!mStreamReader.EndOfStream) {
        try {
          ParseData(); // uses a JsonTextReader to incrementally build up an object out of tokens/values
        }
        catch (Newtonsoft.Json.JsonReaderException e) {
          Console.WriteLine("boo " + e);
        }
      }
which fortunately seems to work, but might mask other errors, so it seems like it wouldn't disrupt things too much to provide support for this kind of behavior.
Please let me know if I'm missing anything - thanks!