This patch changes how the streams are created to avoid the size
limitation on the coded streams. As we only read/write a single
message at a time, there is never any message larger than a few
bytes. However, the coded stream eventually complains that its
internal counter reaches 64+ MByte if the total file size exceeds this
value.
Based on suggestions in the protobuf discussion forums, the coded
stream is now created for every message that is read/written. The
result is that the internal byte count never goes about tens of bytes,
and we can read/write any size file that the underlying file I/O can
handle.
This patch adds support for inputting protobuf messages through a
ProtoInputStream which hides the internal streams used by the
library. The stream is created based on the name of an input file and
optionally includes decompression using gzip.
The input stream will start by getting a magic number from the file,
and also verify that it matches with the expected value. Once opened,
messages can be read incrementally from the stream, returning
true/false until an error occurs or the end of the file is reached.
This patch adds support for outputting protobuf messages through a
ProtoOutputStream which hides the internal streams used by the
library. The stream is created based on the name of an output file and
optionally includes compression using gzip.
The output stream will start by putting a magic number in the file,
and then for every message that is serialized prepend the size such
that the stream can be written and read incrementally. At this point
this merely serves as a proof of concept.
