superintelligence paths dangers and strategies

Thus, Chapters 11–13 explain the difficulty of programming an AI to pursue ends that we would approve of, and propose several methods for solving this ‘value-loading problem’ (Bostrum 2014, 226).

Bostrum’s conclusion is that there is little hope of constraining an unfriendly superintelligence, and thus that the solution must involve making sure that the superintelligence wants to pursue ends that we value. (More on this question below.) Chapters 8–10 cover what Bostrum calls the ‘control problem’, the problem of keeping a superintelligent intellect from destroying humanity. Bostrum argues that a superintelligence could conceivably have just about any possible final goal. Chapter 7 discusses what we can surmise about the will of a superintelligent being. These chapters also discuss the advantages a superintelligence would have against its competitors. Chapters 4–6 explore some possible paths that a roughly human-level AI might take in developing into a superintelligent AI. Chapter 3 elaborates on the concept of superintelligence, suggesting three (in practice equivalent, Bostrum thinks) ways an agent could qualify as superintelligent. Chapter 2 begins the main thread of the book by discussing various ways in which we might build AI, and how it might eventually become superintelligent. Chapter 1 surveys the history of AI and presents some polling data from AI researchers on their anticipations for its future. Superintelligence is certainly worth a read for anyone interested in artificial intelligence (henceforth AI), existential risks, and the prospects for humanity’s future. The result is a book that will have something of interest for technical and non-technical, philosophical and non-philosophical readers alike. Bostrum has done a remarkable job of balancing readability with rigor. The book is commendable both for its scope and innovation. The book explores the ways in which such an intellect might come to be, and what may happen to us - and to the world - if one did. ‘Superintelligence’ as Bostrum uses the term, refers to ‘ny intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest’ (Bostrum 2014, 22).

Nick Bostrum’s book, Superintelligence: Paths, Dangers, Strategies, is the culmination of a small, but quickly expanding body of literature on the prospects for the rise of extremely advanced artificial general intelligence.

Warning: This review is more academic than the average Practical Philosopher article.

Superintelligence paths dangers and strategies