Building

Building the connectors from source is only necessary when you want to use or contribute to the latest (not yet released) version of the Pravega Flink connectors.

Note: There has not been a release of Pravega deployed to Maven Central, so currently all users have to build the connector from source.

Build and install Pravega

To build and install the latest version of the Flink connector for Pravega libraries, you need to build the latest version of the main Pravega project as well. This step installs Pravega libraries into your local Maven repository. See Pravega documentation for more information on how to start a standalone server.

$ git clone https://github.com/pravega/pravega.git
$./gradlew install

Build and install the Flink Connector library

Build and install the latest version of the Pravega Flink Connector library into your local Maven repository.

$ git clone https://github.com/pravega/flink-connectors.git
$ ./gradlew clean install

The resulting jar file will be called pravega-connectors-flink_2.11-<version>.jar.

Building against a custom Flink version

You can check and change the Flink version that Pravega builds against via the flinkVersion variable in the gradle.properties file. Note that you can only choose Flink versions that are compatible with the latest connector code.

Building against another Scala version

This section is only relevant if you use Scala in the stream processing application in with Flink and Pravega.

Parts of the Apache Flink use the language or depend on libraries written in Scala. Because Scala is not strictly compatible across versions, there exist different versions of Flink compiled for different Scala versions. If you use Scala code in the same application where you use the Apache Flink or the Flink connectors, you typically have to make sure you use a version of Flink that uses the same Scala version as your application.

By default, the dependencies point to Flink for Scala 2.11. To depend on released Flink artifacts for a different Scala version, you need to edit the build.gradle file and change all entries for the Flink dependencies to have a different Scala version suffix. For example, flink-streaming-java_2.11 would be replaced by flink-streaming-java_2.10 for Scala 2.10.

In order to build a new version of Flink for a different Scala version, please refer to the Flink documentation (the link refers to Flink 1.3).

IDE Setup (IntelliJ IDEA)

Import the project into IDEA as a Gradle-based project.

Please install the Lombok plugin to be able to build from within the IDE. The source code uses Lombok annotations to generate various helper code.