Preventing an AI-related catastrophe - Problem profile

Tags

Top Highlights

• We estimate that there are around 400 people worldwide working directly on this.3 As a result, the possibility of AI-related catastrophe may be the world’s most pressing problem — and the best thing to work on for those who are well-placed to contribute.

• Why is it that humans, and not chimpanzees, control the fate of the world?

• AI will have a variety of impacts and has the potential to do a huge amount of good. But we’re particularly concerned about the possibility of extremely bad outcomes, especially an existential catastrophe. Some experts on AI risk think that the odds of this are as low as 0.5%, some think that it’s higher than 50%. We’re open to either being right — and you can see further discussion of this here. My overall guess is that the risk of an existential catastrophe caused by artificial intelligence by 2100 is around 1%, perhaps stretching into the low single digits. This puts me on the less worried end of 80,000 Hours staff: as an organisation, our take is that the risk is somewhere between 3% and 50%.

• Promising options for working on this problem include technical research on how to create safe AI systems, strategy research into the particular risks AI might pose, and policy research into ways in which companies and governments could mitigate these risks. If worthwhile policies are developed, we’ll need people to put them in place and implement them. There are also many opportunities to have a big impact in a variety of complementary roles, such as operations management, journalism, earning to give, and more — some of which we list below.

• Some of these risks from advanced AI could be existential — meaning they could cause human extinction, or an equally permanent and severe disempowerment of humanity

• As a result, the possibility of AI-related catastrophe may be the world’s most pressing problem — and the best thing to work on for those who are well-placed to contribute.