Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sort node deserialization from proto #12626

Merged
merged 6 commits into from
Sep 27, 2024

Conversation

palaska
Copy link
Contributor

@palaska palaska commented Sep 25, 2024

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added logical-expr Logical plan and expressions proto Related to proto crate labels Sep 25, 2024
Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Thanks @palaska

LogicalPlanBuilder::from(input).sort(sort_expr)?.build()
let fetch: usize = sort.fetch.try_into().map_err(|_| {
DataFusionError::Internal(String::from(
"Protobuf deserialization error, invalid limit value'",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to show the invalid value and/or the error message to ease debugging

@andygrove
Copy link
Member

Also, could you add/update a unit test so that we prevent future regressions?

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @palaska and @andygrove

@@ -341,6 +341,32 @@ async fn roundtrip_logical_plan_aggregation() -> Result<()> {
Ok(())
}

#[tokio::test]
async fn roundtrip_logical_plan_sort() -> Result<()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

@alamb
Copy link
Contributor

alamb commented Sep 26, 2024

I took the liberty of fixing the clippy error and pushing the fix to this branch to get CI to pass (example of failing CI)

@palaska
Copy link
Contributor Author

palaska commented Sep 26, 2024

I guess when no LIMIT is specified with ORDER BY, our default behavior was to fallback to -1 since fetch is not an optional arg in our proto definition. I made the conversion into usize best effort so things should work as expected now

@@ -704,6 +730,7 @@ async fn roundtrip_logical_plan_distinct_on() -> Result<()> {
let plan = ctx.sql(query).await?.into_optimized_plan()?;

let bytes = logical_plan_to_bytes(&plan)?;
println!("plan {:?}", plan);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could you remove this debug println?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops

@andygrove andygrove merged commit 9b4f90a into apache:main Sep 27, 2024
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logical-expr Logical plan and expressions proto Related to proto crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants