Quote
We were so sure But we were so wrong But now it's over But there's no one left to see And there's no one left to die There's only m.e.
-Gary Numan, “M.E.” from The Pleasure Principle (1979)
If you’re inclined to worry about AI-induced human extinction, this song - a meditation by the last thinking machine, one that cannot flip off its own off switch - should be on one of your playlists. For myself, I just think it is an awesomely eerie song.
Links
DeepMind - Open-Endedness is Essential for Artificial Superhuman Intelligence - a profound paper about the nature of greater than human intelligences, examining a quality that we would both want them to have AND that would help us recognize them as superintelligent. The profound part is the expansion on the quality of “open-endedness”, which the researchers formalize as the novel creations of a system that are (a) highly novel compared to its past creations from the standpoint of an observer but (b) learnable by that observer. Learnable means that you can understand the creation as in sequence with, related to, and as an expansion upon past creations. Looking at the whole sequence of creations, you’d observe a constant quantity of surprise and astonishment at each new creation, but the difference between the first creation and the latest is astronomical, and completely unforeseeable from the beginning of the sequence. I’m still trying to think of a way to visualize this, but it’s creations that are consistently surprising, consistently at the edge of the expected distribution of productions - no matter how much you think you understand the distribution of possible outputs, the next creation is always at the extreme edge.
One way to think about this is by analogy with difference between art produced by a human being, versus the world’s that our current best procedural-content generation algorithms generate. A great fantasy illustrator will keep making new paintings, new digital artworks, that will astound you, but which are clearly by the same person, in the same style. If you study the artwork, you can produce paintings like it, but you can’t produce the next thing that a Frank Frazetta or a Boris Vallejo will create. Procedural content generation in a game like Minecraft, by contrast, eventually bottoms out in terms of creativity and uniqueness - Minecraft’s PCG is well-tuned, but if you explore Minecraft worlds for real-time months on end, you’ll eventually see everything it has to offer. Similarly, a human being is an open-ended system for interaction - you never quite know what even someone you’ve known intimately for years is going to say, going to do, going to feel. But when they speak with you, you recognize them in their actions, in their tone of voice, their body language, how they choose their words.
I find this prospect of open-ended AI immensely exciting: a continuing, never-ending encounter with things just at the edge of the unknown, things coming into view in a never-ending sequence of learning, growth, and transformation. Something to look forward to.Jason Crawford - How to Talk to Stupid People - wise advice about dealing with people you find difficult. Crawford argues for intellectual charity - very likely, the person you are having issues with, the person you want to dismissively label ‘stupid’, is seeing things from a different perspective, has less situational knowledge, is a poor communicator, is facing different incentives. And maybe you’re the “stupid” one - those things actually apply to you. Well worth the read - it will save you countless hours of frustration.
Avital Balwit - My Last Five Years of Work - An intriguing reflection on work and what value it brings to people’s lives, and whether it is worth preserving in a world where Artificial General Intelligence is a fact. The author is the Chief of Staff to the CEO of Anthropic, and thus as knowledgeable a person as any to weigh in on this. She focuses on how people will feel about being made redundant.
Nick Bostrom has written a much deeper because book-length dive into just this issue, in his Deep Utopia: Life and Meaning in a Solved World, but I am currently in the midst of reading it. Also, it is primarily concerned with the Nth Day After the end of employment, not the immediate transition period that is the subject of Balwit’s essay.Dorian Taylor - P(Dumb) - a refreshing alternative to AI Doomerism and Boosterism, by an AI mehptic: its important, its impactful, but you will have had lots of warning from the world changing radically around you before human extinction from AI is a possibility. Your P(Doom) in any ~5-year timeframe should be = 0. He digs deep into the physical computing infrastructure that runs deployed large-language models like ChatGPT, and how the things AI Safety researchers worry about are impractical or flat out impossible with the current architecture.
The essay is useful for understanding the barriers that need to be overcome to ever more powerful and impactful AI systems.Ben Landau-Taylor - The Academic Culture of Fraud - a fine rant, but a justified one, on the culture of fraud, and the lack of consequences following from it, in academia. I don’t want to make any global judgments, but it rather undercuts the finger-waving at students for using ChatGPT to write their take home essays when prominent academics plagiarize even their own PhD dissertations. The essay has an amusing comparison of how fraud is handled in academia - depending on how prominent you are, potentially zero consequence - versus finance, where there is a government bureaucracy dedicated to hunting down market manipulators and fraudsters, jailing them, and giving gigantic payouts to whistleblowers. How many graduate students, rather than feeling they have to “go along to get along” by generating massaged or outright fraudulent results, would prefer to walk away from academia with 30% of the funding ever granted to their supervisors?
One of the hopes I have for advancing artificial intelligence is the creation of knowledge-seeking agents that are motivated by discovery, rather than career, financial security, fame, and power. It’s an interesting question how to define an objective function for such agents. Perhaps the quality of open-endedness, referenced above, could help guide the search for novel hypotheses, if even in an approximating way. Optimizing for publication surely can’t be a good one, nor can citations - a quick enough agent could just flood the academic literature with publications that cite its own works. Much like academics do today!Girl Talk - Feed the Animals (YouTube full album video) - As a rule, I don’t like rap or hip-hop music. The samples used in most songs only makes me want to listen to those original songs, or reflect on what a step backward musically the song I’m listening to is compared to the one sampled. But I was surprised to find how differently I felt when I stumbled upon Girl Talk’s mashups of Top 40, hip-hop, rap, and classic rock. Samples of songs built on samples are a different order of thing altogether. Like the difference in how you perceive broken glass from a derelict building lying on the street, and that same glass used in a mosaic. There’s a pseudo-profound bullshitting some academic type could make up about remix culture, but I just think the album is awesome. If you like it, check out Girl Talk’s All Day.