Launch a Drillbit From Eclipse

Sometimes it helps to launch Drill in distributed mode from within Eclipse. Here is how I was able to do so for a "single node" cluster.

Create a Site Directory

Create a directory, outside of your normal Drill install and outside of the Drill build location. Let's call it ~/site.

1. Copy the drill-override.conf file from a Drill build conf folder.
2. Edit this file to create a new Drill root location in ZK:

drill.exec: {
  zk.root: "drill-debug", # For debug only
  cluster-id: "drillbits1",
  zk.connect: "localhost:2181"
}

Go ahead and add any other config you might need for your tests.

3. Copy the drill-env.sh and logback.xml files from the build conf directory to your site directory.
4. If you like, add more in-depth logging by editing logback.xml:

  <logger name="org.apache.drill" additivity="false">
    <level value="debug" /> <!-- Change this debug level -->
    <appender-ref ref="FILE" />
  </logger>

Your site directory should now look like this:

ls ~/site
drill-env.sh		drill-override.conf	logback.xml

Prepare a Debug Launch Configuration

In Eclipse, create a new debug configuration.

On the Main tab:

• Project: drill-java-exec. (This launches the "core" Drill services. A bit more fiddling will be needed to use contrib storage plugins.)
• Main class: org.apache.drill.exec.server.Drillbit

On the Arguments tab, add the following as VM arguments:

-Xms4G 
-Xmx4G 
-XX:MaxDirectMemorySize=8G 
-XX:ReservedCodeCacheSize=1G 
-Ddrill.exec.enable-epoll=false 
-XX:MaxPermSize=512M 
-XX:+CMSClassUnloadingEnabled 
-XX:+UseG1GC 
-Dlog.path=/tmp/drill/log/drillbit.log 
-Dlog.query.path=/tmp/drill/log/drillbit_queries.json

Feel free to change the log directory location or memory sizes as needed. Hint: if you have a working Drill you want to emulate, invoke drillbit.sh debug to dump the environment and launch command. You can grab the information from there. (That is, in fact, how the above was created.)

In the Classpath tab:

• Add a new User Entry, external folder, that points to your ~/site directory.
• Move this entry up so that it is the first one under User Entries.

In the Environment tab, add:

• DRILL_HOST_NAME = <IP address of your machine>

This last bit is important: without it Drill dies a quick death. On my machine, Drill can't resolve the reported host name. Using localhost results in the infamous message about Drill not being allowed to bind to the loopback address.

Optionally, set

• DRILL_LOG_DIR = /path/to/logs/and/query/profiles

if you'd like to be able to access Drill logs or query profiles.

Start ZK

If you have not yet done so, start ZK on your development machine.

Launch Drill

If all has worked correctly, you can launch Drill. Once all the threads start, you can use the Web UI to submit queries, or log in using SqlLine.

Now, why have we done all this when we could have built with Maven and launched Drill from the command line? It is because we can now easily debug. More important, Eclipse can dynamically replace code. If you are editing UI templates, Eclipse will immediately put the new copies in the class path to avoid the need to rebuild. And, if you do need to restart Drill, doing so takes seconds rather than minutes.

Enjoy!