My fabric deployment script (fabric2)
Since 2018, we’ve had a new version of
fabric, also known as
It comes with an updated API, and is incompatible with the old
It’s also split up some functionality into a few different libraries, including
There was a rant on Reddit about the changes that the new version introduced.
I chimed in with some of my own complaints, which were mainly due to what I believed was inadequate documentation.
After that I regretted it. It wasn’t very nice to the developers who obviously put in a lot of work to release the new version, and obviously it wasn’t very productive either.
So instead of just complaining, I decided to make a blog post that would actually help the community and encourage more people to use it.
My confusion with the documentation
As mentioned, some functionality was split out into several other libraries. That means you’ll need to look at the documentation for each library separately.
For example, anything to do with CLI and task running that isn’t “strictly SSH” is now in a library called Invoke.
So if you’re used to the old primary API of
fabric which was to define a task, then run it in the command line using
fab <task-name>, then you probably want to be looking at the documentation for invoke. Fabric has a thin wrapper on top of invoke for tasks, and Fabric’s documentation simply refers you to Invoke.
Unfortunately, there were a few things that took me way too long to figure out.
Chief among them is how fabric and invoke work together. For example, a lot of
fabric’s documentation deals with using
from fabric.connection import Connection connection = Connection("username@remote-ip") print(connection.run("ls"))
If you’re used to some of the old
fabric methods like
cd(), well, you can’t find them in the
Connection object. What you get is
run() and not much else.
But when I look at
invoke’s documentation, they clearly have methods like
cd() on an object passed in as the first argument in a task. This is called a
For example here’s an invoke task:
from invoke import task @task def some_task(c): with c.cd("some-directory"): c.run("ls")
Well, that’s what I need. But that’s a
Context object, not a fabric
Looking at the “Getting Started” documentation, it first says (emphasis mine):
Defining and running task functions
The core use case for Invoke is setting up a collection of task functions and executing them. This is pretty easy – all you need is to make a file called tasks.py importing the task decorator and decorating one or more functions. You will also need to add an arbitrarily-named context argument (convention is to use c, ctx or context) as the first positional arg. Don’t worry about using this context parameter yet.
Okay. Eventually they’ll explain what it is, right?
Sure enough, if you scroll down you see this bit:
Aside: what exactly is this ‘context’ arg anyway?
A common problem task runners face is transmission of “global” data - values loaded from configuration files or other configuration vectors, given via CLI flags, generated in ‘setup’ tasks, etc.
Some libraries (such as Fabric 1.x) implement this via module-level attributes, which makes testing difficult and error prone, limits concurrency, and increases implementation complexity.
Invoke encapsulates state in explicit Context objects, handed to tasks when they execute . The context is the primary API endpoint, offering methods which honor the current state (such as Context.run) as well as access to that state itself.
Maybe I’m unfamiliar with task runners in general, but I don’t really understand what these paragraphs mean.
Let’s try some stuff out and see if it works
So I decided to try and play around with the
Context. The convention is to use
c as its name, and a fabric
Connection also starts with a
Could I just pass the
Connection object into a task….?
from fabric import Connection from fabric.tasks import task @task def sub_task(c): with c.cd("some-folder"): c.run("ls") @task def main_task(c): con = Connection("username@some-host") print(sub_task(con))
I’ll be damned, that actually worked!
That seems like a pretty important detail about using
fabric, and fundamental in using
invoke together – but it was really strange that it wasn’t documented anywhere. I had to discover it more or less by accident!
So if you’re curious where the
cd() method went, the way to do it is to pass your
Connection object into a task.
Once that was cleared up, the API wasn’t actually that difficult to work with. In fact, it was pretty great! Some things that changed in the new release actually makes a lot of sense.
What use cases am I interested in?
Ultimately, all I want to do is to use it as a way to automate deploying Django apps. Instead of manually SSH-ing and doing a
git pull, or doing an
rsync to copy files to the application server.
I think it would be helpful to the community if we had some cookbooks or example
fabfiles that we could use and modify quickly.
This seems to be missing both in the official docs and the community, so I decided to publish my own one.
My deployment file
You can find it here in my GitHub repository:
This is a script that works for me personally, and hopefully it helps other people as well.