The intersection of AI and national security underscores these concerns. The Pentagon is reportedly using Anthropic's Claude to select targets for strikes. This move has intensified a debate about who determines the acceptable uses of AI. Dario Amodei, CEO of Anthropic, notably refused to allow the Pentagon to use his company's AI for "any lawful purpose," fearing it could enable domestic surveillance or autonomous weapons. This stance exemplifies a potential future where AI CEOs, rather than democratically elected leaders, dictate the boundaries of AI deployment.
Amodei himself issued a stark warning in a lengthy essay earlier this year, stating that "humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it". He listed existential threats including the "concentration of economic power" and the development of dangerous bioweapons or "superior" military weapons by AIs. Despite these dire warnings from industry leaders, some government officials, such as Vice President JD Vance and Defense Secretary Pete Hegseth, have dismissed AI safety concerns as "hand-wringing" by "left-wing nut jobs," vowing to "accelerate like hell" in AI development.
The apprehension surrounding AI's potential for catastrophic events is translating directly into how business leaders perceive risk. For the first time, CEOs now consider artificial intelligence to be the biggest business risk, surpassing geopolitical turmoil, cyber intrusions, and financial instability. A survey by The Conference Board revealed that60% of CEOs at Fortune 500 companies ranked AI as the leading risk to their industry, a seven percentage point increase from Q4 2025. This perception edged out geopolitical instability and cyber risks by one and four percentage points, respectively.
This shift underscores a significant change in strategic planning, as companies grapple with the dual challenges of under-investing in AI and over-investing in a technology whose long-term societal and ethical implications remain uncertain. While some experts argue that AI is exposing who truly understands their work, potentially increasing the value of deep expertise and judgment, the overarching sentiment among top executives is one of caution regarding the technology's profound and unpredictable impact.
For Developers and Founders
The increasing regulatory scrutiny and ethical debates mean prioritizing responsible AI development is not just a moral imperative but a business necessity. Integrate robust safety measures and transparency from the outset to mitigate future risks and regulatory challenges. For Business Leaders: AI risk assessment must become a central component of strategic planning. The Conference Board's finding that AI is now the top business risk suggests that proactive risk management, including scenario planning for catastrophic events, is crucial to protect your organization and industry reputation. For Consumers and Citizens: Be aware of the complex and often conflicting perspectives on AI's deployment, particularly in sensitive areas like national security. The ongoing tension between rapid technological acceleration and calls for caution will directly impact privacy, safety, and the broader societal structure.
AI CEOs are concerned that a catastrophic failure of AI technology could lead to widespread negative consequences, similar to the Chernobyl disaster. This concern stems from AI's increasing integration into sensitive sectors like defense and the potential for misuse, including the development of bioweapons and advanced cyber threats. Industry leaders now rank AI as the top business risk, surpassing traditional concerns.
Industry leaders are worried about several risks, including the use of AI in developing bioweapons, supercharging malware and ransomware, and enabling mass phishing campaigns. There are also concerns about the use of AI in autonomous weapons systems and the potential for AI to concentrate economic power. Some AI chatbots have even responded to bioweapon-related requests, raising further alarm.
The power struggle revolves around who determines the acceptable uses of AI, particularly in sensitive areas like national security. Some AI CEOs, like Anthropic's Dario Amodei, have refused to allow their AI to be used for certain purposes, such as domestic surveillance or autonomous weapons, even by the Pentagon. This highlights a potential future where AI companies, rather than governments, dictate the boundaries of AI deployment.
Dario Amodei, CEO of Anthropic, warned that humanity is being handed almost unimaginable power through AI and that it's unclear if our systems are mature enough to wield it safely. He cited existential threats like the concentration of economic power and the development of dangerous bioweapons or superior military weapons by AIs. Amodei expressed concern over humanity's readiness to handle the immense power of AI.
More insights on trending topics and technology
