Skip to content

Commit

Permalink
[SPARK-46652][SQL][TESTS] Remove Snappy from TPCDSQueryBenchmark
Browse files Browse the repository at this point in the history
…benchmark case name

### What changes were proposed in this pull request?

This PR aims to remove a hard-coded compression codec name from `benchmark case name` in `TPCDSQueryBenchmark`.

### Why are the changes needed?

`GenTPCDSData` can generate dataset with the other compression codecs than `snappy`. So, we had better remove the hard-coded `Snappy` because it's misleading.

```
$ JDK_JAVA_OPTIONS='spark.sql.parquet.compression.codec=zstd' \
build/sbt "sql/Test/runMain org.apache.spark.sql.GenTPCDSData --dsdgenDir /Users/dongjoon/DATA/tpcds-kit/tools --location /Users/dongjoon/DATA/tpcds-sf-1-zstd --scaleFactor 1 --numPartitions 1"
```

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manual test.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#44657 from dongjoon-hyun/SPARK-46652.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
dongjoon-hyun committed Jan 10, 2024
1 parent bda9957 commit a73ff66
Show file tree
Hide file tree
Showing 3 changed files with 541 additions and 543 deletions.
Loading

0 comments on commit a73ff66

Please sign in to comment.