Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Traverse pipeline topology looking for the flow control stuck vunerability #873

Open
FelonEkonom opened this issue Sep 26, 2024 · 0 comments

Comments

@FelonEkonom
Copy link
Member

FelonEkonom commented Sep 26, 2024

Every randomized time interval check if there is a diamond structure in the pipeline topology with a root having :auto flow control and pull effective flow control (and proper flow control on both paths to the "leaf").

If it occurs, add a debug/warning log informing about possibility of getting stuck, if the data will be unbalanced and possible solutions (setting auto_demand_size to a bigger number, modifying pipeline topology).

Another possible approach is to auto-magically-bump auto_demand_size in case of getting stuck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Development

No branches or pull requests

1 participant