Add schema support for load_csv
#87
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bradlo
requested changes
Oct 11, 2022
examples/load_csv.py
Outdated
| @@ -57,6 +57,13 @@ def run(database: str, engine: str, fname: str, relation: str, | |||
| help="relation name (default: file name)") | |||
| p.add_argument("-p", "--profile", type=str, default="default", | |||
| help="profile name") | |||
| p.add_argument( | |||
There was a problem hiding this comment.
please format this like the surrounding code
There was a problem hiding this comment.
The sdk now uses flake8 for linting, so that formatting changed. I autoformatted using pep8 now.
railib/api.py
Outdated
| inputs = {'data': data} | ||
| command = _gen_syntax_config(syntax) | ||
| command += "".join( |
There was a problem hiding this comment.
comprehension nested in a join behind a plusop is not ideal expression complexity.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #25
I added support for specifying a schema dict when loading CSV. I adapted the existing integration test to include a schema definition.
I did not validate whether the passed schema dictionary is valid in the sense that it only specifies valid types, since I did not find a policy in the code how validations of input values are handled in general. We may create a follow-up issue on that so that the SDK yields meaningful errors on non-valid inputs.