Streaming Writer

[sandstrom]

Just wanted to raise the idea of a streaming writer, where data is sequentially committed to a file (or other Ruby IO object).

Basically, instead of building a complete in-memory model of the XML file, before calling Ox.dump on it, one would write directly to the underlying IO object (Ruby IO has built-in buffering).

Ox would still need to keep in memory the number of open tags. For some types of documents this would be a lot of data (very deep XML trees). But for most XML documents, you are rarely more than 5-10 layers deep at any one point in time, so in these cases the amount of data held in memory should be minimal.

Just wanted to put this idea out there.

I'm not saying this is something that should be added right now, but if others also find it useful, it could be explored further.

The primary use-case would be large XML exports, e.g. in the hundreds of mb, where you'd prefer to not hold everything in memory.

[ohler55]

For such a feature I'd suggest the Ox.Builder could either be extended to support streams or a new class with the same methods could be used. What do you think of the Builder API?

[sandstrom]

Yeah, I think that'll probably do the job! 😄

Haven't looked at it since we ended up writing it ourselves, but most likely.

[dvandersluis]

@ohler55 where can I find the documentation for Ox::Builder?

[ohler55]

http://ohler.com/ox/Ox/Builder.html

[dvandersluis]

Thanks! 🙌 How do I use it to stream to a file? (Is there more to it than Ox::Builder.file?)

[ohler55]

http://ohler.com/ox/Ox/Builder.html#method-c-file