Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Propose stream-based Replication of Log Entries #1029

Open
1 task done
drmingdrmer opened this issue Feb 24, 2024 · 2 comments
Open
1 task done

Propose stream-based Replication of Log Entries #1029

drmingdrmer opened this issue Feb 24, 2024 · 2 comments
Labels
A-replication Area: replication protocol A-store Area: storage

Comments

@drmingdrmer
Copy link
Member

drmingdrmer commented Feb 24, 2024

Stream-based Replication of Log Entries

Rationale

The existing request-response style API offers limited parallelism and pipelining capabilities and does not allow for dynamic adjustment of request sizes.

By introducing a Stream as an argument to RaftNetwork::append_entries(), we can empower the method to replicate log entries in a more flexible and potentially optimized way.

Accordingly, the storage layer should also expose a stream-based API. We propose adding RaftLogReader::try_get_log_entries() -> impl Stream<Item=Entry>, which will enable the streaming of log entries as needed.

Proposed new API:

trait RaftLogReader<C: RaftTypeConfig> {
    async fn get_log_entries(&mut self, since: u64)
               -> impl Stream<Item=C::Entry>;
}

trait RaftNetwork<C: RaftTypeConfig> {
    async fn entries(&mut self, 
                   prev: Option<LogId>, 
                   entries: impl Stream<Item=C::Entry>) 
              -> Result<
                       impl Stream<Item=Result<Response, RPCError>, 
                       RPCError<_,_,RaftError<>>
                    >;`
}

RaftNetwork::entries() now takes a stream of log entries for transmission and returns a stream of replication responses. Should a replication attempt result in an Err, such as the absence of prev on the remote peer or the detection of a higher term vote, the caller should immediately discard the stream.

TODO:

@drmingdrmer drmingdrmer added A-store Area: storage A-replication Area: replication protocol labels Feb 24, 2024
Copy link

👋 Thanks for opening this issue!

Get help or engage by:

  • /help : to print help messages.
  • /assignme : to assign this issue to you.

@drmingdrmer drmingdrmer changed the title Stream-based Replication of Log Entries Propose stream-based Replication of Log Entries Feb 24, 2024
@schreter
Copy link
Collaborator

Nice!

drmingdrmer added a commit to drmingdrmer/openraft that referenced this issue Apr 11, 2024
Doc `log_stream.md` discusses the core aspects of streaming log replication.

- Part of databendlabs#1029
drmingdrmer added a commit that referenced this issue Apr 11, 2024
Doc `log_stream.md` discusses the core aspects of streaming log replication.

- Part of #1029
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-replication Area: replication protocol A-store Area: storage
Projects
None yet
Development

No branches or pull requests

2 participants