How to bootstrap ETL jobs with Bonobo?

There are two kind of jobs you can bootstrap using bonobo init.

• Either you just want to create a one file based job, and you can run bonobo init somefile.py (the .py extension is important, bonobo will refuse to create a python file without it).
• Or you want to create a python package, that is installable using regular python packaging tools (pip, wheel, ...).

Create a single file ETL job

The basic command to create a skeleton job is bonobo init. It only requires jinja2, which is a bonobo dependency.

Kind of simple.

If you don't like the verbosity of the default template, you can use the "bare" template, that contains less comments and less boilerplate, just the bare necessary code for a job.

The generated jobs contains enough to be executed right away.

Create a package-based ETL job

Once you start to have a bunch of code in your job, the recommended way forward is to migrate this single file to a regular python package, to leverage the import infrastructure that comes with it and be able to factor your code in different modules and packages.

Bonobo provides a helper to do it, using Medikit to bootstrap the package.

After answering a few basic questions, your job will be generated in a subdirectory.

You can install it, as an "editable" package (which means it won't be copied to your python's site-package directory, but instead symbolically linked, so you can edit the code in place).

You can now run the main entry point using the -m flag of python interpreter.

That's it! And it's just standard python packaging, so you can do whatever is normal in python.

For more informations, you should read the Python Packaging User Guide.