This feature is now released in the stable channel of Bonobo.
Bonobo-Docker is a lightweight package that adds Docker-related capabilities to Bonobo.
To use it, you'll need the bonobo[docker] extra, or the bonobo-docker package.
You also need a working docker installation, and a correctly setup docker client accessible from PATH (if you can run docker ps, you should be good to go).
The extension defines a new runc command for bonobo to run a (local) ETL job within a docker container.
bonobo runc uses the same syntax as bonobo run.
It currently supports both running a file (with bonobo runc <filename></filename>) or a directory (using bonobo runc <path></path>), but not a python module refered as by python dot notation (bonobo run -m foo.bar.baz has no container equivalent).
Using bonobo runc without arguments will spawn a python read-eval-print-loop (REPL) in a docker container, with bonobo pre-installed and pre-loaded (you can use the bonobo symbol without importing it first.
To explore a bonobo-enabled container using a bash shell, you can run bonobo runc --shell. Of course, bonobo is pre-installed.
There are a few candies available that includes mounting volumes in the containers run using bonobo runc.
To ease the process of building images for bonobo, the extension also contains a wrapper for docker build that will correctly tag the images with the right version numbers, using metadata retrieved from PyPI package page.
You should not normally need to use this, as we will build and publish images after each bonobo release (and probably in a scheduled way too on old versions, so we can include security updates from the base operating system).
As this feature is less user-land and more release-land, it's not published as a bonobo command, but is only useable using the following command:
For now, bonobo-docker does not support running docker within a docker container. This is something we want to look into in the future, especially for bonobo-docker continuous integration processes.