[Go] Refactored Stream() API to be more ergonomic.

[apascal07]

The previous API is not ideal because you have to pass in a function into an iterator-like interface that then chooses between stream value, final value, or error. This change resembles a classic iterator.

Checklist (if applicable):

• Tested (manually, unit tested, etc.)
• Changelog updated
• Docs updated

[jba]

The general feeling is that designs that only report the error at the end are error-prone (no pun intended).
The classic example is https://pkg.go.dev/bufio#Scanner, where to make the iteration convenient, you have to call a function at the end to get the error. We know empirically, from looking at a lot of code, that people often forget to make that call to Err.

You are in a better place here, because at least FinalOutput is returning something useful as well as the error. If you think everyone will want that final value, then this is fine. If you think some people might not need it, you may have a problem. If you do go down this road (I'm not going to try and stop you beyond this comment), make sure docs and examples emphasize the need to check that error even if the final output isn't wanted.

[jba]

One other thing to think about: Go 1.23 is about to drop, and with it support for iterators, or as we call it, range-over-func. See https://eli.thegreenplace.net/2023/preview-ranging-over-functions-in-go and https://tip.golang.org/doc/go1.23.
Now, the Go team promises support for the last two versions of Go, so I recommend you do too. That means you shouldn't use this feature until Go 1.24 comes out in about six months. But it's worth thinking about how you'd design your iterators with range-over-func.

[jba]

What does it do if the flow hasn't completed?

[jba]

Ah, I actually wrote this with range-over-func in mind. Its return type is exactly an iter.Seq2 (see https://pkg.go.dev/iter#Seq2). So think about keeping it. It seems weird now but won't in a few months.

[apascal07]

The general feeling is that designs that only report the error at the end are error-prone (no pun intended). The classic example is https://pkg.go.dev/bufio#Scanner, where to make the iteration convenient, you have to call a function at the end to get the error. We know empirically, from looking at a lot of code, that people often forget to make that call to Err.

You are in a better place here, because at least FinalOutput is returning something useful as well as the error. If you think everyone will want that final value, then this is fine. If you think some people might not need it, you may have a problem. If you do go down this road (I'm not going to try and stop you beyond this comment), make sure docs and examples emphasize the need to check that error even if the final output isn't wanted.

My first stab at this (f62be7a) included returning an error from Next() so that the caller can be notified if it fails mid-stream. In many cases, users will want the final output so I don't find it that risky or strange to have to call it to get the final result, whether it's good output or an error. I agree with you it should be well documented that it needs to be checked.

[apascal07]

One other thing to think about: Go 1.23 is about to drop, and with it support for iterators, or as we call it, range-over-func. See https://eli.thegreenplace.net/2023/preview-ranging-over-functions-in-go and https://tip.golang.org/doc/go1.23. Now, the Go team promises support for the last two versions of Go, so I recommend you do too. That means you shouldn't use this feature until Go 1.24 comes out in about six months. But it's worth thinking about how you'd design your iterators with range-over-func.

@pavelgj

For us it would look like this, which is pretty clean:

iter := DefineStreamingFlow(...)
for s, err := range iter {
    if err != nil {
        // handle err
    }
    if s.Done {
        output := s.Output
    } else {
        chunk := s.Stream
    }
}