Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create docker container environment for Kafka-Toolbox repo #15

Open
ritikBhandari opened this issue Mar 7, 2023 · 2 comments
Open

Create docker container environment for Kafka-Toolbox repo #15

ritikBhandari opened this issue Mar 7, 2023 · 2 comments

Comments

@ritikBhandari
Copy link

ritikBhandari commented Mar 7, 2023

@romain-fontugne I was planning to create an entire docker environment for our kafka server. But what I have observed is we have around 6-7 producers and a bunch of consumers in psql/consumers. Do you have any existing approach as to how to club these producers and consumers? I have 2 options:

  1. Creating separate containers for each producer and consumer which will allow us to scale specific components if they generate data with high velocity and have fine-grained control over the components. But this can create a little bit of chaos as well since we'll have to deploy 20+ containers.

  2. If we want a simpler deployment and management experience, we could combine all of the producers into one container and all of the consumer into another container. Just the fact I feel is it would limit our ability to scale individual components independently.

But entirely your call. We can move forward accordingly.

@romain-fontugne
Copy link
Member

I think the first approach would be better, especially when things goes wrong we can restart only the failing producer/consumer

@ritikBhandari
Copy link
Author

Makes sense!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants