10 April 2023
Three traits of a task you can automate
What: Thought leadership blog post
Written for: Gradient AI (formerly known as Preemo), freelance
Links: Original Preemo Medium post, current Gradient AI blog post
My role: Independently conceived of and wrote the piece in one day.
Do you trust a washing machine?
Think about your washing machine. You put clothes in, set the dial, pour in detergent, and bam—clean clothes. Your washing machine just saved you an hour of scrubbing (and probably did it better than you could have).
Now, let’s go back in time. Imagine you’re looking at the world’s first washing machine. Do you want to use it?
Now, you might stop to ask:
How do I use it? Do I need to be an electrical engineer to turn it on, or change the settings? (How easy is it to specify what I want?)
How do I know it worked? Maybe a quick check for previous stains. And if I set it to cold water but got hot, I’ll feel the temperature difference. (How easy is it to check I got what I wanted?)
How often does it work, and what happens when it doesn’t? Even if it’s once in a while, it’s really bad if it shreds my clothes, but ok if I just have to run it again. (How reliable is it, and what’s the cost of subpar performance?)
Today, your washing machine meets all these criteria for you, whether or not you think about it.
The three traits
You can identify any task to automate using a similar line of questioning. The task must:
Be cheap to specify.
Be cheap to check.
Have minor consequences if not done to specification. Unless the automation is much more reliable than a human.
The three traits, for coding automation
Let’s apply these traits to software. We’ll dig one level deeper to draw corollaries.
Cheap to specify
We used “easy” above to describe the washing machine, but “easy” is not the only way to be useful. We use “cheap” here because it’s more general. For software to be cheap to specify, it can either be:
Quick and easy for an engineer to specify, OR
Possible for a non-engineer to specify
Cheap to check
We use the word “cheap” here for similar reasons. In order for it to be cheap to check code:
The checker must have the skills to check code. Note, they do not need to be able to write code. (Economically speaking, this point is inherently easier to satisfy.)
The software must be easy to check. That means either:
The specs are loose/imprecise, OR
The output is small in scope
Have minor consequences if not done to specification
Unless the automation is much more reliable than a human.
If there’s a type of automation that may cause severe problems, but works a lot better than a human, it might be acceptable. We’re not going to name any names, but you probably know of at least one widely debated use case.
For a lot of software, it’s not life-or-death. Especially in the world of B2B SaaS. (Sorry not sorry.) Even so, there are certain parts of software where we need high reliability and accuracy.
As a general rule of thumb:
Internal-facing software is less critical than external-facing software.
A piece of product is less critical than a piece of infrastructure.
Example coding tasks: today, and tomorrow
Now that we’ve established the traits we’re looking for, let’s put our knowledge to use. Let’s use the traits to predict where we’ll see coding automation.
Example 1: today
Let’s first take the perspective of an engineer. Let’s consider what can satisfy these constraints:
Cheap to specify—Quick and easy for an engineer to specify.
Cheap to check—The output is small in scope.
Have minor consequences if not done to specification.
These bullets describe the realm of coding assistants like Github Copilot. Some possible outputs:
The body of a function, given the function signature.
A small function, given a comment docblock.
Example test data, given a natural language description of the data structure.
Example 2: tomorrow
Now, let’s take the perspective of a non-engineer. What can satisfy these constraints?
Cheap to specify—Possible for a non-engineer to specify.
Cheap to check—The specs are loose/imprecise.
Have minor consequences if not done to specification.
Here are some ideas we thought of:
A starter home page for marketing.
A modified version of checkers to play with friends.
A bot that finds and books a reservation for dinner tonight, somewhere “interesting”—“interesting” loosely defined by the user.
A bot that books desks for employees when they visit an office.
Take a minute. What other ideas can you come up with?
Impact on software development
In this post we’ve focused on tasks, and the traits of tasks that make them candidates for automation.
But inevitably, what we can automate changes how we shape the world. After the washing machine was invented, we designed clothes to be machine-washable.
It might occur to you that the three traits roughly map to different parts of the software development cycle: specification (problem understanding and definition), checking (QA), consequences (impact analysis).
How will this impact software development workflows? Which skills and roles will we need? For example, QA engineers exist today. Conventional wisdom predicts that the number of QA engineers will decline. But could the role of the QA engineer not only persist, but grow because of generative AI?
We’ll explore this question in a next post in this series.
Closing thoughts
Cheap to specify, cheap to check, minor consequences. What exists in your day-to-day work that fits this bill?
More interestingly: What can you dream of, that doesn’t exist yet, but fits this bill?
Think about coding automation today like that very first washing machine. That’s where we are.