I recently read this book for the first time, as it is a mainstay in AI, and I had never actually read it when it first appeared. The book has been discussed as a leading thought experiment in what it means for AI, how that will impact us as humans, and what that might mean.

Tegmark starts with a fictional scenario about an “Omega Team” building the first superintelligent AI, and plays out how that might work. It isn’t presented as the only option, just an option, and is informative to explain how a superintelligent AI will provide an unfair advantage to the first people to create it, but then later, what happens when that AI escapes. 

This book is best when it is most tangible. The near term themes around AI causing mass economic disruption and job displacement are exactly the reasons why I created BRAIN in the first place, to investigate the impact of AI on the Ballarat region. Tegmark beat me to the idea by about six years, so it is impressive how accurate these themes were, given the book was published in 2017. While the idea of a singularity (where an AI starts self improving past the ability for humans to catch up, understand or control) was not developed by Tegmark, this book covers the impact this would have on us as humans.

The core chapter of the book is Chapter 7 on alignment. Given a superintelligent AI, what will it try to do? How can we guarantee this would benefit us, or at least not harm us? This goal-alignment problem is one that people have been struggling with for decades in this field of AI, and I don’t know if we are much closer to understanding what this means or what we can do about it. Eventually an AI would get smart enough to outsmart the people locking it in - what then?

I did find the book starts to get a bit too speculative. Tegmark looks much more forward to the future, talking about energy requirements from Dyson Spheres (where you surround a star and capture all the energy), and people getting asteroids to themselves. These are potential options for the future, but I feel the overly speculative nature here provides little value. Here Tegmark is testing the thinking to extremes, seeing what physics permits as possible extremely long term outcomes.

The real value of this book is the more near term tangible outcomes. Given it was developed before ChatGPT, before the current LLM boom, it is incredible how accurate it was on the main issues, which we are grappling with as a society today as “now” problems, not “future” problems. Overall a great book to read, but if you are like me, you probably want to skim over the overly futuristic parts.

The link has been copied!