Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

…parsed by ddlog-sql

Changes:
- char(N) --> varchar(N) because the DDlog language doesn't support char(N)
- Remove FK constraints, as neither Calcite nor Presto support this
- Add types to arrays so they are parseable by Calcite

----------

All uncommented views in DBSchema pass both H2 and Presto translation

Edits required:
- select --> select distinct, necessary for both Calcite and Presto
- remove camelcase column names, as Presto make everything case-insensitive
- DDlog cannot support LIMIT, which will need to be addressed in the future
- Remove ';' at the end of some statements, as Presto doesn't like this

TODO:
- Fix commented-out views in DBSchema, which are a combination of sql->DDlog compiler not supporting some statements, or Calcite doesn't like some of the complex SQL

----------

All views other than affinity/anti-affinity pass

- Add all columns that are using in the HAVING clause to GROUP BY clause
- Add array_length functionality to DDlog
- DDlog only supports array_agg for now, instead of array_agg_distinct

TODO:
- array_agg_distinct
- Affinity views

-----------

Move all DDlog SQL updates to separate files, so that we can still run our H2 tests

-----------

Signed-off-by: Amy Tai <[email protected]>

Changes to make SchedulerTest::testSchedulerNodePredicates pass

Summary of changes:
- New IConnectionPool interface so that we can seamlessly swap DBConnectionPool out for DDlogDBConnectionPool
- Temporarily comment out OnDuplicateKeyUpdate queries in PodEventsToDatabase and NodeResourceEventHandler, as DDlog backend cannot handle MERGE SQL queries at the moment
- Inserts into DDlog must be in order of the column declarations in the CREATE TABLE statement
- Any SQL queries that will eventually hit DDlog must use unqualified names
- Wherever there is a dslContext.meta.getTables() call (or any dslContext.meta.* function call), for the DDlog backend this must be called against the dslContext that is a class variable within the DDlogJooqProvider. Every other dslContext call (i.e. ones that are used to execute select/insert/etc. queries), should be made against the DSLContext held inside the DDlogDBConnectionPool. These two use cases are differentiated by the two different accessor functions inside IConnectionPool.

Signed-off-by: Amy Tai <[email protected]>

Migrate modifications to scheduler_tables.sql and DBView.java to DDlog versions

testSchedulerNodePredicates and testSpareCapacity both pass.

Signed-off-by: Amy Tai <[email protected]>

testHostPorts passes

Goes with corresponding DDlog patch on vectors (array_agg column type)

Signed-off-by: Amy Tai <[email protected]>

…spare capacity views that Lalith has

Signed-off-by: Amy Tai <[email protected]>

…ables on the DDlog side

Signed-off-by: Amy Tai <[email protected]>

… calcite packages

…ts seem to work at this commit

Signed-off-by: Amy Tai <[email protected]>

- "_augmented" views still don't work because they require preemption views from DDlogDBViews, which were commented out hitherto

Signed-off-by: Amy Tai <[email protected]>

… to work

Signed-off-by: Amy Tai <[email protected]>

… use a stale DDlog API.

Instead, don't compileAndLoad until the very last time (after Autoscope)

Signed-off-by: Amy Tai <[email protected]>

Signed-off-by: Amy Tai <[email protected]>

Signed-off-by: Amy Tai <[email protected]>

…ptimized DDlog program file. In that case, /tmp/program.dl is not compiled using sql-ddlog compiler. Instead, we just copy over contents of the hand-optimized file into /tmp/program.dl.

Signed-off-by: Amy Tai <[email protected]>

Signed-off-by: Amy Tai <[email protected]>

Signed-off-by: Amy Tai <[email protected]>

…kend

Signed-off-by: Amy Tai <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

… Views once, so that we never have to compile it again with each of the CartesianProductTests.

We also temporarily change the Scheduler constructor to add AutoScoped views to DDlog, even if the Scheduler isn't running in AutoScoped mode. This is so that we can use the same DDlog program (with scoping) for non-scoped tests as well.

Need to fix this eventually, as for real tests/workloads, we might want to turn off Autoscoping.

Signed-off-by: Amy Tai <[email protected]>

Signed-off-by: Amy Tai <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

…t-joins are brewing

Signed-off-by: Lalith Suresh <[email protected]>

…th identity views

Signed-off-by: Lalith Suresh <[email protected]>

…passes.

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Amy Tai <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Lalith Suresh <[email protected]>

Signed-off-by: Athinagoras Skiadopoulos <[email protected]>

Signed-off-by: Athinagoras Skiadopoulos <[email protected]>

Signed-off-by: Athinagoras Skiadopoulos <[email protected]>

Signed-off-by: Athinagoras Skiadopoulos <[email protected]>