Superintelligence
Nick Bostrom
This deep dive into the mechanics of artificial cognition reveals how a machine-led "intelligence explosion" could rapidly and irreversibly surpass human control. You will learn to navigate the complex ethical and strategic hurdles of the AI frontier, gaining the vital foresight needed to ensure that the birth of a superior mind preserves, rather than ends, the human legacy.
The Road to Superintelligence
Mapping the technological paths to machine intelligence and the distinct forms superintelligence might take.
The Multi-Path Ascent
The Trinity of Superintelligence
The Intelligence Explosion
Analyzing the speed of the transition to superintelligence and the resulting shifts in global power.
Kinetics of the Explosion
The Decisive Strategic Advantage
Singletons and Global Order
The Motivational Profile
Investigating why a superintelligent machine will not naturally share human values or ethics.
The Orthogonality Thesis
Instrumental Convergent Drives
The Treacherous Turn
Capability Control
Methods designed to limit what a superintelligent agent can do or how it interacts with the world.
Boxing and Containment
Incentives and Capability Limiting
Motivation Selection
Strategies for ensuring an AI wants to act in humanity's best interests.
Direct Goal Specification
Indirect Normativity
The Oracle and the Sovereign
Coherent Extrapolated Volition
Risks and Value Alignment
Deep diving into the specific ways the transition could fail and the difficulty of loading values.
The Value Alignment Problem
Malignant Failure Modes
Strategy and Policy
Navigating the global political landscape and setting research priorities for a safe future.
The Common Good Principle
Collaboration vs. Arms Race
Strategic Research Priorities
The Ultimate Choice
Start reading with AI
Interactive Socratic dialogue, level by level