ملخص Human Compatible

In Human Compatible, Stuart Russell argues that the standard model of AI development—giving machines fixed objectives to maximize—is fundamentally flawed and potentially catastrophic. Russell, a UC Berkeley computer scientist and co-author of the leading AI textbook, contends that the real risk isn't malevolent robots but competent machines pursuing poorly specified goals with superhuman efficiency, a problem he illustrates with the "King Midas problem" and the story of the sorcerer's apprentice. His central proposal is to rebuild AI around uncertainty about human preferences. Rather than optimizing a fixed objective, machines should be designed to remain uncertain about what humans want, learn those preferences through observation and feedback, and defer to humans when confidence is low. Russell calls this approach "provably beneficial AI," structuring the book around three principles: the machine's only objective is to maximize the realization of human preferences, the machine is initially uncertain about what those preferences are, and the ultimate source of information about human preferences is human behavior. The book traces AI's history from early symbolic systems through the deep learning revolution, explains why techniques like inverse reinforcement learning and assistance games offer a technical path toward safe machines, and examines near-term harms such as algorithmic bias and autonomous weapons. Russell also confronts the "control problem"—how to prevent a superintelligent system from resisting correction—and proposes that machines should learn to accept being switched off because they are uncertain whether doing so serves human interests. Drawing on economics, philosophy, and computer science, he makes a case that building AI that is compatible with human values is both a technical challenge and an urgent civilizational priority.