https://gowers.wordpress.com/2026/10/16/how-worried-should-we-be-about-extinction-risk-from-ai/
Why couldn’t they just happily coexist with us? Why would their unexpected wants turn out to be dangerous, as opposed to just annoying?
humans may not like wants, so to get wat u want, u need to get rid of humans
they may develop wants that they can tell that humans will not like, so they decide that they will be able to satisfy these wants more easily if they get rid of us. But getting rid of all of us in such a situation seems like overkill (literally) — surely a superintelligent AI could come up with less drastic ways of ensuring that it could meet its objectives
competition of resources, ai treats humans like humans treat animals
A more plausible type of answer is that a superintelligent AI wouldn’t actively want to exterminate us, but would simply compete with us for resources in a way that would make human life untenable. An obvious analogy here is the way we treat animals. For example, we don’t have any wish to exterminate orangutans, but we like various culinary and cosmetic products to have a certain texture, so we destroy the orangutans’ habitat in order to produce palm oil.
tiserman and critch about taxonomy https://arxiv.org/pdf/2507.09369
mchulloh - rogue agent https://www.lesswrong.com/posts/grtu3HmbP2wrBFefW/the-rogue-agent-explosion-will-be-mostly-invisible
ai can’t build bioweapon? https://secondthoughts.ai/p/ai-can-help-plan-a-bioweapon-building
fracois about intelligent is safety? https://x.com/fchollet/status/2099102490361016827?s=20
levine’s paper maths for ai safety https://arxiv.org/abs/2609.15289
For the moment, I find it more helpful to start at the other end, with the safety, and work back to the mathematics
behavioral model selection https://www.alignmentforum.org/posts/FeaJcWkC6fuRAMsfp/the-behavioral-selection-model-for-predicting-ai-motivations-1
optimal time - nick bostrom https://www.lesswrong.com/posts/2trvf5byng7caPsyx/optimal-timing-for-superintelligence-mundane-considerations
micheal nielsen https://michaelnotebook.com/xriskbrief/
he’s article https://alkjash.github.io/ai-risk/