Bug fixes

Fixes bug in #57 regarding executing local tasks (like sparse run) with
pty=True.

Major release: streamparse runner, component API updates, Python logging support and more

This is a major release that introduces several potentially breaking API
changes, hence the major version advancement to 1.0.0. Please read the changes
below as well as our migration wiki
guide before upgrading.

streamparse runner

• Added a new runner for topology components and as a result, a new way to
  define topologies (extension to the existing Clojure DSL). More info can be
  found in the docs and in the 1.0.0 migration guide.
• Quickstart projects have been updated to now have nested directories
  "src/bolts" and "src/spouts".

Component API updates

• Added auto_anchor, auto_ack and auto_fail flags to base Bolt class. See
  docs for detailed descriptions of these flags and migration page for info on
  how to safely upgrade your bolts.
• BasicBolt is now deprecated.
• Class var BatchingBolt.SECS_BETWEEN_BATCHES renamed to
  secs_between_batches since this isn't a constant, just a setting.
• Spout.emit and Bolt.emit now returns a list of task IDs a tuple was
  emitted to unless need_task_ids kwarg is set to False.
• Spout.emit_many and Bolt.emit_many now return a two-dimensional list of
  the task IDs each emit tuple was sent to unless need_task_ids kwarg is set
  to False.
• BatchingBolt does not return task IDs due to concurrency issues that will
  be addressed in a future release.
• Spouts and bolts now have the following instance variables which users are
  free to use. These are initialized before the call to initialize():
  • _topology_name - name of the topology when submitted to Storm.
  • _task_id - task ID of the current component in the topology.
  • _component_name - the name of the current component as defined in the
    Clojure definition (e.g. "my-bolt").
  • _debug - the topology.debug setting (configured using the sparse
    --debug flag).
  • _storm_conf - the entire config dict recieved on initial handshake.
  • _context - the entire context dict receieved on initial handshake.
• BatchingBolt threads (main and _batcher) now have more descriptive names.

IPC

• A lot of code cleanup around how we read from and write to stdin and stdout.
  Much more stable here but no breaking changes for users since they should
  never interface with these methods directly.

Logging

• Full support for Python logging has now been added. Lots of info in the docs
  on this and the migration guide. Logging config settings added to new
  quickstart projects.
• print statements are now properly sent to the component's log file - feel
  free to add print statements for handy debugging or better yet, create a
  logger for nicely formatted messages.
• Full support for Storm log levels once STORM-414 is merged in.

Administration

• Added pre and post submit hooks in fabric and invoke for topologies to
  enable users to run arbitrary code (e.g. send IRC message) after topologies
  are submitted (info in docs).\
• sparse tail command now requires -n <topology> argument as it will tail
  only the logs for a specific topology and environment.
• Added remove_logs fab task which users can optionally hook up to a
  pre_submit hook to clear out Python logs.

Testing

• Better test support added for all our components, more improvements to come
  here.

v0.0.13 - Python 3 support and classpath sync to Storm 0.9.2-incubating

This release will work with Storm 0.9.2 (recently released). It also includes merged support for Python 3 (thanks @dan-blanchard!)

v0.0.9 - Topology options supported

This release includes changes to sparse submit and sparse run that allow you to tune topology workers and ackers and their parallelism levels. There is also a mechanism now to override any Storm topology option per-submit or per-run, e.g. topology.message.timeout or topology.max.spout.pending.

The JVM interop library was also re-released, and now bundles a fork of org.clojure/tools.cli in it to avoid the classpath/dependency issues that were plaguing users with this setup.

v0.0.8 - cluster mgmt bugfixes

This release fixes an issue where sparse submit limited to 3 workers and always submitted with "debug mode" by default. This would slowdown production clusters needlessly. This release bumps the version of the Clojure-based interop library, so you should update your project.clj file for existing projects appropriately.

v0.0.7 - initial remote cluster management support

Our 0.0.5 release started with sparse quickstart and sparse run. This release adds:

• sparse submit for topology building and sending to remote cluster
• sparse list and sparse kill management commands
• sparse tail for tailing logs on remote workers

Available on PyPI.