BSON Binary Bug

Dec 14, 2010 at 1:42 PM
Edited Dec 14, 2010 at 1:43 PM

We are using the BSON functionality of Json.NET 3.5 Release 8 in our project and we are very happy with the performance! However, there is a problem with the representation of binary data in the output of this library. According to the BSON specification at http://bsonspec.org/#/specification there are two subtypes for binary data, 0x00 and 0x02, of which 0x00 is the recommended format.

The BSON specification has the following to say on the 0x02 format:

Old generic subtype - This used to be the default subtype, but was deprecated in favor of \x00. Drivers and tools should be sure to handle \x02 appropriately. The structure of the binary data (the byte* array in the binary non-terminal) must be an int32 followed by a (byte*). The int32 is the number of bytes in the repetition.

Json.NET uses 0x02 but fails to add the required extra length field to the binary data. This results in errors when trying to read the BSON data with another library like the MongoDB Java driver.

Preferably, Json.NET should start using 0x00 as the binary format, which it is already implicitly doing by omitting the extra length field, and it should support reading the correct 0x02 format. This will probably pose problems with backward compatibility...

Regards,

Benny van Reeven
The Netherlands 

Dec 17, 2010 at 6:02 PM

I just encountered the same problem.

Coordinator
Dec 18, 2010 at 11:12 PM

Hrm this sucks. Compatibility will be an issue. Maybe I'll do a special build for people that reads the bad JSON correctly and writes good JSON.

Coordinator
Dec 19, 2010 at 12:36 AM

I've made a couple of changes.

When writing BSON binary values Json.NET now always uses the new binary subtype. Json.NET also reads the old subtype correctly. For compatibility I've added a switch to BsonReader: JsonNet35BinaryCompadibility. When it is true it will read the old wrong BSON correctly.

Because newly written BSON will use the new subtype, bad BSON and good BSON can co-exist. Even so, I'd recommend writing a console app or something that reads all old BSON with the flag set to true and then writing it back out again.

 

Let me know whether everything is working as expected now.

Dec 20, 2010 at 9:18 AM

Thanks for your quick reply. Everything is working as expected now.

P.S. The compatibility flag is misspelled and should read "JsonNet35BinaryCompatibility".