Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: benchmarking #10

Open
aljazerzen opened this issue Feb 22, 2024 · 2 comments
Open

perf: benchmarking #10

aljazerzen opened this issue Feb 22, 2024 · 2 comments

Comments

@aljazerzen
Copy link
Owner

ATM I have no idea how performant this connector is. Surly it is slower than using plain connections to the database and not converting to Arrow at all.

But how does it compare to:

  • pandas.read_sql,
  • polars.read_database,
  • connectorx,
  • ADBC,
  • dyplr?

Ideally, I would reuse the benchmarks from the connector-x project, but I'm not sure how portable they are.

@aljazerzen
Copy link
Owner Author

This is also needed because right now, it doesn't make sense making any performance optimizations, since I have no way of verifying that they are even any faster.

@aljazerzen
Copy link
Owner Author

There is something similar here: https://github.com/pola-rs/tpch

Ideally, I would be able to benchmark "getting X amount of data from a data store Y into memory", for a few different amounts of data X and for all supported data stores Y.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant