Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

relational.write - deduplication mechanism #239

Open
spicy-sauce opened this issue Jan 3, 2023 · 0 comments
Open

relational.write - deduplication mechanism #239

spicy-sauce opened this issue Jan 3, 2023 · 0 comments
Assignees
@vd2org
Labels
block bug Something isn't working

Comments

@spicy-sauce
Copy link
Member

When we get the same key in the same batch, the whole batch will probably fail.
First, add a test that verifies it and fails.
Then, add a dedup mechanism that groups the records by the key in the relational.write block and takes only the latest record per key.
@spicy-sauce spicy-sauce added bug Something isn't working block labels Jan 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vd2org vd2org

Labels
block bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vd2org @spicy-sauce