Do-nothing scripting: the key to gradual automation

Every ops team has some manual procedures that they haven’t gotten around to automating yet. Toil can never be totally eliminated.

Very often, the biggest toil center for a team at a growing company will be its procedure for modifying infrastructure or its procedure for provisioning user accounts. Partial instructions for the latter might look like this:

  1. Create an SSH key pair for the user.
  2. Commit the public key to Git and push to master.
  3. Wait for the build job to finish.
  4. Find the user’s email address in the employee directory.
  5. Send the user their private key via 1Password.

This is a relatively short example. Sometimes there are 20 steps in the process. Sometimes there are branches and special cases to keep track of as you go. Over time, these procedures can become unmanageably large and complex.

Procedures like this are frustrating because they’re focus-intensive yet require very little thought. They demand our full attention, but our attention isn’t rewarded with interesting problems or satisfying solutions – just another checkbox checked. I have a word for a procedure like this: a slog.

We know that this procedure is ripe for automation. We can easily see how to automate any given step. And we know that a computer could carry out the instructions with far greater speed and accuracy than we can, and with less tendency toward practical drift.

However, automating slogs sometimes feels like an all-or-nothing proposition. Sure, we could write a script to handle step 2, or step 5. But that wouldn’t really make the procedure any less cumbersome. It would lead to a proliferation of single-purpose scripts with different conventions and expectations, and you’d still have to follow a documented multi-step procedure for using those scripts.

This perception of futility is the problem we really need to solve in order to escape from these manual slogs. I’ve found an approach that works pretty reliably: do-nothing scripting.

Do-nothing scripting

Almost any slog can be turned into a do-nothing script. A do-nothing script is a script that encodes the instructions of a slog, encapsulating each step in a function. For the example procedure above, we could write the following do-nothing script:

import sys

def wait_for_enter():
    raw_input("Press Enter to continue: ")

class CreateSSHKeypairStep(object):
    def run(self, context):
        print("Run:")
        print("   ssh-keygen -t rsa -f ~/{0}".format(context["username"]))
        wait_for_enter()

class GitCommitStep(object):
    def run(self, context):
        print("Copy ~/new_key.pub into the `user_keys` Git repository, then run:")
        print("    git commit {0}".format(context["username"]))
        print("    git push")
        wait_for_enter()

class WaitForBuildStep(object):
    build_url = "http://example.com/builds/user_keys"
    def run(self, context):
        print("Wait for the build job at {0} to finish".format(self.build_url))
        wait_for_enter()

class RetrieveUserEmailStep(object):
    dir_url = "http://example.com/directory"
    def run(self, context):
        print("Go to {0}".format(self.dir_url))
        print("Find the email address for user `{0}`".format(context["username"]))
        context["email"] = raw_input("Paste the email address and press enter: ")

class SendPrivateKeyStep(object):
    def run(self, context):
        print("Go to 1Password")
        print("Paste the contents of ~/new_key into a new document")
        print("Share the document with {0}".format(context["email"]))
        wait_for_enter()

if __name__ == "__main__":
    context = {"username": sys.argv[1]}
    procedure = [
        CreateSSHKeypairStep(),
        GitCommitStep(),
        WaitForBuildStep(),
        RetrieveUserEmailStep(),
        SendPrivateKeyStep(),
    ]
    for step in procedure:
        step.run(context)
    print("Done.")

This script doesn’t actually do any of the steps of the procedure. That’s why it’s called a do-nothing script. It feeds the user a step at a time and waits for them to complete each step manually.

At first glance, it might not be obvious that this script provides value. Maybe it looks like all we’ve done is make the instructions harder to read. But the value of a do-nothing script is immense:

  • It’s now much less likely that you’ll lose your place and skip a step. This makes it easier to maintain focus and power through the slog.
  • Each step of the procedure is now encapsulated in a function, which makes it possible to replace the text in any given step with code that performs the action automatically.
  • Over time, you’ll develop a library of useful steps, which will make future automation tasks more efficient.

A do-nothing script doesn’t save your team any manual effort. It lowers the activation energy for automating tasks, which allows the team to eliminate toil over time.

51 thoughts on “Do-nothing scripting: the key to gradual automation

  1. Pingback: New top story on Hacker News: Do-Nothing Scripts – Outside The Know

  2. Pingback: New top story on Hacker News: Do-Nothing Scripts – protipsss

  3. Pingback: New top story on Hacker News: Do-Nothing Scripts – Golden News

  4. Pingback: New top story on Hacker News: Do-Nothing Scripts – World Best News

  5. Pingback: New top story on Hacker News: Do-Nothing Scripts – News about world

  6. Pingback: New top story on Hacker News: Do-Nothing Scripts – Latest news

  7. Pingback: New top story on Hacker News: Do-Nothing Scripts – Hckr News

  8. Wojtek Bażant

    I run a complicated release process. It has to be as complicated as it is, because involves long computationally intensive analyses on the primary data. Each cycle takes a few months.

    I’ve never used your approach but I like the idea. I’ve been a fan of really good instructions that I copy from the instruction page and execute at the right time. I pay attention to keep them all working, to keep them readable, and to make them mention important places – e.g. a dump script takes the dump location as a parameter, probably involving an environmental variable for the release number.

  9. Pingback: Do-Nothing Scripts – INDIA NEWS

  10. We use this hybrid process effectively in a script. I never thought of it as a design pattern, so thanks for that. Design patterns help with communication of an idea within a team, and it’ll be interesting to explore where else this can be applied.

  11. Pingback: Do-Nothing Scripts – Hacker News Robot

  12. Pingback: Do-nothing scripting: the key to gradual automation - Latest News,Jobs,Notifications.Gk,Telugu News,Results,MCQs,Science,Tips and Tricks and More.

      1. I send an email to my AWS Account Rep quarterly to inquire about ed25519 support. I can’t imagine what amount of cruft they have sitting around that’s making it take so long for them to adopt support.

        (Great post, by the way! The checklist aspect of it reminded me a lot of Atul Gawande’s “Checklist Manifesto”. I have several processes here that could benefit from this type of thing.)

        1. Keep fighting the good fight.

          I love _Checklist Manifesto_. I hadn’t thought about it, but that book probably inspired me to have this idea in the first place.

  13. Pingback: Do-nothing scripting: the key to gradual automation - RAM NETWORK

  14. Pingback: Do-nothing scripting: the key to gradual automation - LazyProgrammer

  15. Stephen Dowdy

    Nice! I do something similar, w/o the prompting bit. The script (mostly POSIX shell), doesn’t do something (because i think there may be a failure mode i didn’t account for, or just to ensure a real person is sanity checking what’ll happen), by dumping out informational/warning text interspersed with actions tagged:
    : CUTANDPASTE :; command-to-exec args…
    (making it easy to identify the steps that need to be cut-pasted, and also allowing triple-click line-select on it (POSIX Shell effectively ignores the CUTANDPASTE and execs the command after ‘;’ ) )
    Use of variables are tested by POSIX emtpy/null check: “${var:?You Forgot To Set var}”
    There’s also the CYA of : ” *My Script* didn’t bork the system, *You* blindly taking my script’s advice borked the system ;)”

  16. Pingback: Wordpress Web Site Disaster Recovery On The Cheap - Developer Coach

  17. Pingback: Do-nothing scripting: the key to gradual automation | Pratikar

  18. Pingback: New best story on Hacker News: Do-nothing scripting: the key to gradual automation – Golden News

  19. Pingback: Do-nothing scripting: the key to gradual automation – Infinity News

  20. Pingback: New best story on Hacker News: Do-nothing scripting: the key to gradual automation – protipsss

  21. Pingback: Do-nothing scripting: the key to gradual automation

  22. Pingback: Do-nothing scripting: the key to gradual automation | toppertrick

  23. Jörg Strebel

    Congratulations on re-inventing a (simple) workflow management system. You could easily add logging and auditing for legal/safety reasons, so the do-nothing script would at least add some value.

  24. Ironman

    Your approach is really appreciated, but how about the logic used like this:

    import sys

    def wait_for_enter():
    raw_input(“Press Enter to continue: “)

    class CreateSSHKeypairStep(object):
    def run(self, context):
    print(“Run:”)
    print(” ssh-keygen -t rsa -f ~/{0}”.format(context[“username”]))

    class GitCommitStep(object):
    def run(self, context):
    print(“Copy ~/new_key.pub into the `user_keys` Git repository, then run:”)
    print(” git commit {0}”.format(context[“username”]))
    print(” git push”)

    class WaitForBuildStep(object):
    build_url = “http://example.com/builds/user_keys”
    def run(self, context):
    print(“Wait for the build job at {0} to finish”.format(self.build_url))

    class RetrieveUserEmailStep(object):
    dir_url = “http://example.com/directory”
    def run(self, context):
    print(“Go to {0}”.format(self.dir_url))
    print(“Find the email address for user `{0}`”.format(context[“username”]))
    context[“email”] = raw_input(“Paste the email address and press enter: “)

    class SendPrivateKeyStep(object):
    def run(self, context):
    print(“Go to 1Password”)
    print(“Paste the contents of ~/new_key into a new document”)
    print(“Share the document with {0}”.format(context[“email”]))

    class DoSequanceTask(task_number i) {
    def run(self, context):
    if task_number==1:
    CreateSSHKeypairStep()
    elif task_number==2:
    GitCommitStep()
    elif task_number==3:
    WaitForBuildStep()
    elif task_number==4:
    RetrieveUserEmailStep()
    elif task_number==5:
    SendPrivateKeyStep()

    if task_number<5:
    wait_for_enter()
    DoSequanceTask(task_number+1)
    else:
    print("Done.")
    }

    if __name__ == "__main__":
    context = {"username": sys.argv[1]}
    DoSequanceTask(1)

  25. Awesome post!

    I created a Ruby DSL specifically designed for this use case. It also incorporates a number of other helpful features for working with your do-nothing script such as generating a representation in a yaml format and resuming your script at the same step if it fails. Check it out at: https://github.com/braintree/runbook

  26. Pingback: === popurls.com === popular today

  27. Pingback: Quick Update | SwimGeek

  28. Pingback: Collective #536 | Web Design News from Dubado

  29. Pingback: Collective #536 – Unsorted

  30. Pingback: Collective #536 – Find Right Software

  31. Pingback: Collective #536 – Slacker News

  32. Pingback: Collective #536 - News 20h

  33. Pingback: 2019_07_31 News of Note? – DC3IT Helpful Links

  34. Pingback: Android-VPN mit Wireguard - Mitch’s Manga Blog

  35. Great post. Do you know of any frameworks that help with writing these “do nothing scripts”? It would be nice to factor out the common parts.

  36. j2

    Some of these steps could be executed by the script – why not do that? (Some can’t of course).

    Also, a Jenkins pipeline might be an alternative way to achieve this with some accountability of who executed which step plus it can be replayed from a middle step if it fails (well if it can fail).

  37. Pingback: Links: 9-3-2019 | Aaron Johnson

  38. Pingback: Get started with automation by writing scripts that do nothing – logandonley.com

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s