80th Percentile Pizza: On AI and Employment
Just because forecasting is useless doesn't mean it is not worth thinking things through
TL;DR - this essay includes, at its end, a simplistic framework for thinking about the impact of artificial intelligence on employment in different industries. I don’t make any predictions, just give what I think is a reasonable basis to start thinking about the issue from before you get into complex math, economic studies, and selling books with prophecies of AI induced unemployment.
There is, to be sure, a lot of hype about artificial intelligence these days. I don’t need a citation for that - you’ve probably already seen at least two or three stories in your news feeds just this morning. The hype comes in a lot of forms, but in the English speaking world - maybe things are different in Asia - the hype has three foci:
Machine learning and large-language models are the greatest investment opportunity since Bitcoin!
Artificial intelligence is going to kill us all, and we all need to be very, very concerned about this - more than the existential risks of anthropogenic climate change, nuclear war, plagues, or literally anything else.
Artificial intelligence is going to cause mass unemployment and social disruption.
Foci 3 is the topic of this essay.
Zero Accountability: The Life of a Forecaster
Human beings, as the only animal we know of that lives beyond the present moment, remembering the past and anticipating the future, have an inherent weakness for prophecy. Confidently given predictions about the future - particularly if very positive or very negative - always attract attention, media coverage, book deals, speaking tours, and the like. Particularly when the prophet can weave together lots of the current intellectual and political topics into one coherent narrative.
It has been said, accurately, that the key lesson of history is that no one learns anything from history - except for possibly that lesson. One of the lessons no one learns is that prophecy has an incredibly poor track record. Strangely, though, prophets are still listened to, though I think for the most part they are not taken seriously. Listening to prophets, buying their books, discussing their prophecies, seems to be a kind of social ritual. After every financial and economic crisis journalists flocks to interview economists and professional investors who “predicted” the current moment. Few journalists bother to check the prior prophecies. Many of them consistently predict economic doom year after year, always confident of the date of the stock market melt down (6 to 18 months from ‘now’). They are not right anywhere near as often as a broken clock, but no one holds them to account for it.
It doesn’t help that none of these prophets act on their prophecies: they are not shorting the NASDAQ, stockpiling food, water, and ammunition, or running around in the street telling people to repent because the end times are near. If you are living in the wilderness, eating locusts and wild honey, baptizing people at the Jordan River, I may not believe in your Messiah, but I will definitely believe that you believe. These other prophets, not so much.
The Robots are Coming For Whose Jobs?
For a while now, at least since the current wave of deep learning applications that began in the early 2010s, some economists have been regularly forecasting effects of these technologies on the labor market. These forecasts range from “some people will lose their jobs, but new jobs will be created” to “we need Universal Basic Income now because everyone from short order cooks and Wal-Mart greeters to physicists and investment bankers are going to be begging in the streets in 5-7 years.”
Forecasting, however, rather than an attempt to predict the future, can be useful to think about what factors we ought to pay attention to. By which I mean that the exercise of thinking about how to forecast accurately, to even just map out the extent of our uncertainty, can be valuable in itself.
Erik Brynjolfsson and Tom Mitchell, economists at Stanford and Carnegie Mellon University respectively, published an article in Science in 2017 with just such a framework for thinking through the potential impacts of machine-learning driven automation. They identified the following factors (keep in mind the piece was published before the recent breakthroughs in large-language models from OpenAI):
Tasks that involve learning a function that maps well-defined inputs to outputs - classification and prediction tasks, in other words ”this is/is not a hot dog.”
Large data sets exist or can be created to obtain input-output pairs - there are enough photos of hot dogs and things that can be mistaken for hot dogs to give the model enough pieces to work from
The task provides clear feedback with clearly definable goals and metrics - a human would agree that the picture is a hot dog.
No long chains of logic or reasoning that depends on diverse background knowledge or common sense - if looks like a hot dog, it is probably a hot dog. [Chain-of-Thought prompting may have turned this factor upside down]
No need for detailed explanation of how the decision was made - I cannot tell you how I understand, from looking at the hot dog, that it is a hot dog, and no one expects a machine learning model to be able to say so either.
Tolerance for error and no need for provably correct or optimal solutions - if a non-hot dog slips through the model’s classifier, its not the end of the world (though someone may have a very bad lunch experience…)
The phenomenon or function being learned should not change rapidly over time - hot dogs, being hot dogs, tend to stay hot dogs. No supervillains are introducing fake hot dogs into the environment.
No specialized dexterity, physical skills, or mobility is required - no part of the classification process involves a robot picking up a hot dog
I’m being flippant but Brynjolfsson and Mitchell’s essay is quite good (the link above is behind, but get the DOI and download it from Sci-Hub - wink-wink).
I’m not going to delve deep into Brynjolfsson and Mitchell’s subsequent work on professions that are most vulnerable to automation-induced unemployment, or any of the recent flood of predictions - it gets the institutions name in the news! - but instead suggest another perspective to look at the issue from
Pareto Principles
I’m not going to delve deep into the Pareto Principle here. Go ask ChatGPT or Bing Chat about it - they’ll hook you up.
For our purposes, just consider it the 80/20 rule:
20% of the families control 80% of the land
20% of the scientists publish 80% of the research
20% of a stock portfolio delivers 80% of the growth
20% of your clients give you 80% of your income
Of interest here is what I call the “value of perfection” problem. This is any situation where to go from 80% of perfection to 100% - the final 20% of the work - takes 80% of the resources. Writers know this - the initial idea, the note taking, the research, the first draft, all of these take a lot of time, but not most of the effort. Editing is where the sweat is.
How does this apply to thinking about AI/ML and unemployment? Think about what recent machine learning innovations have brought:
We can effortlessly summarize, modify, and create text,
We can effortlessly generate images in any style or specification, in a quality that is superior to 90% of human artists
Let’s leave aside whether any of this counts as “creativity.” What matters is that these technologies are (a) good enough for a huge range of applications, and (b) blindingly fast.
Rather than delve deep into all the factors that our two economists propose, I propose a Pareto principle model. Take it for granted that AI is not as “good” at generating text, painting pictures, classifying hot dogs from non-hot dogs, as the very best human at those tasks. Allow, for the near future (realistically, 6-12 months given how fast the machine learning field is advancing) that AI systems will be, at best, equivalent to 80th percentile humans at the task. As a friend of mine described ChatGPT, “we have automated B+ students.”
In how many industries, in how many professions, in how many product categories, in how much of life is the 80th percentile just (or more than just) “good enough?” compared to the 99th percentile? When you stop to think about it, it is quite a lot. Now how much better would it be if the 80th percentile cost 1/5th the price of the 99th?
Viscerally, and tastily, think about how adequate 80th percentile pizza (Panago?) is compared to 99th percentile pizza (maybe the kind your Italian grandma makes?)
Now imagine that pizza cheaper, ubiquitously available. That's my analogy for the potential impact of AI systems. Professions and jobs with this character are left as an exercise for the reader.