My fabric deployment script (fabric2)
Since 2018, there’s been a new version of fabric, also known as fabric2.
It comes with an updated API, and is incompatible with the old fabric 1.x fabfiles.
It’s also split up some functionality into a few different libraries, including invoke and patchwork.
There was a rant on Reddit about the changes that the new version introduced.
I chimed in with some of my own complaints, which were mainly due to what I believed was inadequate documentation.
After that I regretted it. It wasn’t very nice to the developers who obviously put in a lot of work to release the new version, and obviously it wasn’t very productive either.
So instead of just complaining, I decided to make a blog post that would actually help the community and encourage more people to use it.
My confusion with the documentation
As mentioned, some functionality was split out into several other libraries. That means you’ll need to look at the documentation for each library separately.
For example, anything to do with CLI and task running that isn’t “strictly SSH” is now in a library called Invoke.
So if you’re used to the old primary API of fabric which was to define a task, then run it in the command line using fab <task-name>, then you probably want to be looking at the documentation for invoke. Fabric has a thin wrapper on top of invoke for tasks, and Fabric’s documentation simply refers you to Invoke.
Unfortunately, there were a few things that took me way too long to figure out.
Chief among them is how fabric and invoke work together. For example, a lot of fabric’s documentation deals with using Connection objects:
from fabric.connection import Connection
connection = Connection("username@remote-ip")
print(connection.run("ls"))
If you’re used to some of the old fabric methods like cd(), well, you can’t find them in the Connection object. What you get is run() and not much else.
But when I look at invoke’s documentation, they clearly have methods like cd() on an object passed in as the first argument in a task. This is called a Context object.
For example here’s an invoke task:
from invoke import task
@task
def some_task(c):
with c.cd("some-directory"):
c.run("ls")
Well, that’s what I need. But that’s a Context object, not a fabric Connection object.
What’s a Context object?
Looking at the “Getting Started” documentation, it first says (emphasis mine):
Defining and running task functions
The core use case for Invoke is setting up a collection of task functions and executing them. This is pretty easy – all you need is to make a file called tasks.py importing the task decorator and decorating one or more functions. You will also need to add an arbitrarily-named context argument (convention is to use c, ctx or context) as the first positional arg. Don’t worry about using this context parameter yet.
Okay. Eventually they’ll explain what it is, right?
Sure enough, if you scroll down you see this bit:
Aside: what exactly is this ‘context’ arg anyway?
A common problem task runners face is transmission of “global” data - values loaded from configuration files or other configuration vectors, given via CLI flags, generated in ‘setup’ tasks, etc.
Some libraries (such as Fabric 1.x) implement this via module-level attributes, which makes testing difficult and error prone, limits concurrency, and increases implementation complexity.
Invoke encapsulates state in explicit Context objects, handed to tasks when they execute . The context is the primary API endpoint, offering methods which honor the current state (such as Context.run) as well as access to that state itself.
Maybe I’m unfamiliar with task runners in general, but I don’t really understand what these paragraphs mean.
Let’s try some stuff out and see if it works
So I decided to try and play around with the Context. The convention is to use c as its name, and a fabric Connection also starts with a c.
Could I just pass the Connection object into a task….?
from fabric import Connection
from fabric.tasks import task
@task
def sub_task(c):
with c.cd("some-folder"):
c.run("ls")
@task
def main_task(c):
con = Connection("username@some-host")
print(sub_task(con))
I’ll be damned, that actually worked!
That seems like a pretty important detail about using fabric, and fundamental in using fabric and invoke together – but it was really strange that it wasn’t documented anywhere. I had to discover it more or less by accident!
So if you’re curious where the cd() method went, the way to do it is to pass your Connection object into a task.
Once that was cleared up, the API wasn’t actually that difficult to work with. In fact, it was pretty great! Some things that changed in the new release actually makes a lot of sense.
What use cases am I interested in?
Ultimately, all I want to do is to use it as a way to automate deploying Django apps. Instead of manually SSH-ing and doing a git pull, or doing an rsync to copy files to the application server.
I think it would be helpful to the community if we had some cookbooks or example fabfiles that we could use and modify quickly.
This seems to be missing both in the official docs and the community, so I decided to publish my own one.
My deployment file
You can find it here in my GitHub repository:
https://github.com/yoongkang/fabric-deployment
This is a script that works for me personally, and hopefully it helps other people as well.