By Eliezer Yudkowsky
This book has two main parts.
Teaching you how systems, organizations, societies, etc, full of intelligent and moral people can produce unintelligent, immoral outcomes. Some common factors that produce these outcomes:
Decision makers who are not beneficiaries - Cases where the decision lies in the hands of people who would gain little personally, or lose out personally, if they did what was necessary to help someone else.
Asymmetric information - Cases where decision makers can’t reliably learn the information they need to make decisions, even though someone else has that information.
Nash equilibria that aren’t even the best Nash equilibrium - Systems that are borke in multiple places so that no one actor can make them better, even though, in principle, some magically coordinated action could move to a new stable state.
How to spot and act on areas where you as an individual can personally out perform these broken systems. two tidbits of advice:
Its way easier to determine which sprinter will win than run the race yourself and win. Don’t fall into the trap that in order to out perform expert X you need to be able to personally be superior to expert X when you can just follow expert Y.
There are lots of societal and peer pressures which will make you want to be modest. Remember the more broken the institution the lower the bar you need to clear to out perform.