Project Goals

High level Goals

Design a modern Unix shell that can do everything bash/zsh/etc. can do, and more:

• System Administration
  • Building Linux distributions (e.g. Arch Linux uses bash for PKGBUILD).
  • Startup scripts
  • Configure and build scripts. Reproducible and distributed Builds.
• Distributed Computing -- Building containers, specifying remote jobs. Feedback and Monitoring.
  • Performance measurement
  • Security testing
• Scientific Computing -- heterogeneous "big data" and small data pipelines.
  • The language should scale down as well as scale up, i.e. low startup latency for small jobs.
  • Incorporate features of "workflow languages" and systems in the MapReduce family.
  • Reproducible Research.
• Interactive Computing -- a general purpose REPL. May hook into Jupyter at first.

Oil Language Design Goals

• Easy upgrade path from bash, the most popular shell in the world. To do this, I've written a very compatible bash parser, which will allow automatic conversion of bash (osh) to oil. So the language has a different syntax and a superset of bash semantics.

• Consistent syntax. POSIX sh and bash have evolved many quirks.

• Semantic fixes, such as fixing arrays.

• Treat the shell as a real programming language. Example: Completion functions in bash are fairly difficult have a bad API involving globals. It should feel more like writing completion in Python or JavaScript.

  • Fill in obvious gaps, like abspath, etc.
  • selected influences: Python, R, Ruby, Perl 6, Lua (API), ML, C and C++. Power Shell.

• Reduce language cacophony in shell programming. Example: combine shell, awk, and make.

  • Also combine tools like find (which has its own expression parser and starts processes), and xargs/GNU parallel, which both start processes. GNU parallel is actually mentioned in the bash manual.

• Security
  • in emitting strings: escaping
  • in reading strings: error checking should be easy, better control over "read" delimiters, etc.
  • fix issues with globs and flags, i.e. untrusted file system and untrusted variables

Implementation Goals

• Proper error messages like Clang/Swift. Static Parsing.
• Provide end-to-end tracing and profiling tools (e.g. for pipelines that run for hours)

Longer Term Goals

• Expose our toolkit for little languages -- lexing, parsing, AST representation, etc. So that other languages can be built in the same way.
• Metaprogramming with ASTs as first class data structures.