Skip to content

Latest commit

 

History

History
105 lines (82 loc) · 2.87 KB

README.md

File metadata and controls

105 lines (82 loc) · 2.87 KB
Raw

Crystal GPU Library (extended)

This is an extention built on top of the original Crystal library (description below) and Crystal-opt.

The extention includes the following:

  • "Compiled" version of queries.
  • Two parallelisation models for compiled queries: Batch-to-GPU and Batch-to-SM.
  • Prefetching, streaming handles for int values in device code.
  • Plots for comparing parallelisation models and selected metrics gathered from ncu.

Usage (extended)

Example to create ssb dataset, run all queries and plot metrics:

python3 run_tests.py --data-dir=/home/user/TUM/crystal/test/ssb/data/ --sf=10 --mode=PROFILE --create-dataset=/home/user/TUM/crystal/test/ssb/

Example to run all queries and plot metrics:

python3 run_tests.py --data-dir=/home/user/TUM/crystal/test/ssb/data/ --sf=10 --mode=PROFILE

Example to run all queries and skip metrics:

python3 run_tests.py --data-dir=/home/user/TUM/crystal/test/ssb/data/ --sf=10

The Crystal library implements a collection of block-wide device functions that can be used to implement high performance implementations of SQL queries on GPUs.

The package contains:

  • Crystal: crystal/ contains the block-wide device functions
  • Implementations: src/ contains SQL query operator implementations and implementations of 13 queries of the Star Schema Benchmark

For full details of the Crystal, see our paper

@inproceedings{shanbhag2020crystal,
  author = {Shanbhag, Anil and Madden, Samuel and Yu, Xiangyao},
  title = {A Study of the Fundamental Performance Characteristics of GPUs and CPUs for Database Analytics},
  year = {2020},
  url = {https://doi.org/10.1145/3318464.3380595},
  doi = {10.1145/3318464.3380595},
  booktitle = {Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data},
  pages = {1617–1632},
  numpages = {16},
  location = {Portland, OR, USA},
  series = {SIGMOD ’20}
}

Usage

To use Crystal:

  • Copy out the crystal directory into your project.
  • Include Crystal
#include "crystal/crystal.cuh"
  • Add the crystal directory to your include path

To run the operator implementations:

  • Compile and run the operator. E.g.,
make bin/ops/project
./bin/ops/project

To run the Star Schema Benchmark implementation:

  • Generate the test dataset
cd test/

# Generate the test generator / transformer binaries
cd ssb/dbgen
make
cd ../loader
make 
cd ../../

# Generate the test data and transform into columnar layout
# Substitute <SF> with appropriate scale factor (eg: 1)
python util.py ssb <SF> gen
python util.py ssb <SF> transform
  • Configure the benchmark settings
cd src/ssb/
# Edit SF and BASE_PATH in ssb_utils.h
  • To run a query, say run q11
make bin/ssb/q11
./bin/ssb/q11