Static vs. JIT compilation

Most compilers with which you are probably familiar are standalone programs which take as input some source code text and compile it into machine code (or some other target representation).

Halide is an embedded language in C++ (or Python, or other future front-ends). That means that the Halide compiler is actually C code that runs inside a program you yourself write. Your Halide program ("pipeline") is not passed to the Halide compiler as source code text, but rather as a set of C++ data structures which you construct using the types and operators/functions define in the Halide namespace. Rather than feeding text source to a standalone compiler program, which parses the text to build its own data structures from which to compile, you, the Halide programmer, directly construct these data structures and hand a pointer to them directly to the Halide compiler logic, right within your own program. This is why Halide is provided as a library (libHalide.a and Halide.h), rather than an executable program; the executable which runs the Halide compiler is your own program.

In this world, there are two ways the compiler can be used—really, two places to which it can output its compiled machine code:

1. It can output code to a file on disk. The compiled output can then be linked together with other code into a separate program which will actually run the compiled code. This is how compilers like GCC, javac, or most other things you probably think of, work. (The actual files the Halide compiler outputs in this mode are specifically equivalent to those output by a C compiler and assembler: an object file exporting a C ABI-compatible function which implements the pipeline, just like if you'd written it in C and compiled it with gcc -c.) We call this mode "static compilation," and it's triggered by Func::compile_to_file() and related functions.

2. The Halide compiler can also output machine code directly into memory inside the process in which it's running. Remember, the Halide compiler is running inside your own program, so, after you have triggered the compiler to generate code for your pipeline, you may want to immediately run it inside the same program. We call this model "JIT (or just-in-time) compilation," and it can be triggered with Func::compile_jit() and the resulting compiled code easily run with Func::realize(). (Realizing a function implicitly requires it to be JIT-compiled in memory before it is run, so realize() will internally call compile_jit() if the function hasn't been previously compiled and cached in the same session.)