Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/dlq #59

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Feature/dlq #59

wants to merge 3 commits into from

Conversation

souvik-databricks
Copy link
Collaborator

Work for #53 going on this branch and will be addressed as part of this PR
CC: @robertkossendey

@souvik-databricks souvik-databricks linked an issue Dec 31, 2022 that may be closed by this pull request
@souvik-databricks
Copy link
Collaborator Author

@robertkossendey I will be adding the ability to write the invalid records to a separate DLQ table but I need help for this two

  • Ability to automatically mark rows as invalid in target table
  • Ability to fail on invalid records

Please feel free to jump in on this branch and add in your logic for that functionality

@robertkossendey
Copy link
Collaborator

@souvik-databricks sure, will do! could you provide an example for each method to show how it is supposed to be used?

@MrPowers
Copy link
Owner

@souvik-databricks - can you add some tests for these WIP functions? Think that'll make it easier for me to see what functionality you're going for here. Thanks!

@robertkossendey
Copy link
Collaborator

@MrPowers I think we can close this PR now, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Brainstorm data quality features
3 participants