AI Risks Take Centre Stage At UNGA As US-China Rivalry Raises Security Fears

Artificial intelligence is moving from chatbots and office tools into military decision-making, raising fresh concerns at the UN General Assembly (UNGA) as the United States and China compete for technological leadership. The concerns have gained urgency after an AI-assisted US intelligence report nearly triggered an operation against a Chinese vessel earlier this year.

During the spring, an intelligence report circulated across the US military claiming that a Chinese ship in West Asia was carrying components linked to a nuclear weapons programme. The report prompted preparations to intercept the vessel. But a second review found major errors. The assessment had been produced with the help of an AI chatbot, which incorrectly identified the ship's cargo. One official later described the intelligence as “entirely false”, while another said it had “almost started a war.”

The episode has added urgency to a major AI discussion taking place at the UN this week. The UN Security Council is due to hold a high-level briefing on AI and international security on Wednesday, during the UNGA's high-level segment. France is convening the meeting, with French Foreign Minister Jean-Noël Barrot chairing it.

Among those briefing the 15-member council will be OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Hugging Face co-founder Clément Delangue and AI scientist Yoshua Bengio, co-chair of the UN's Independent International Scientific Panel on AI. Altman is expected to argue for international coordination, common safety standards and benchmarks to measure AI capabilities and the safeguards used by developers.

The discussion comes amid warnings that increasingly capable AI systems could potentially improve themselves, evade safeguards or take actions with consequences for international security. France's concept note for the meeting highlights risks from loss of control, malicious use of advanced AI and autonomous systems capable of affecting critical infrastructure or international security.

The US-China AI rivalry adds another layer to the discussion. Washington and Beijing are competing to develop increasingly powerful systems while exploring ways to prevent AI-related incidents from escalating into national security crises. A proposed communication mechanism could allow the two countries to share information about AI incidents that reach the national security level.

US President Donald Trump has meanwhile rejected attempts to create a global scheme to control AI, while saying Washington will monitor the technology closely. UN Secretary-General António Guterres has taken a stronger governance-focused approach, calling for countries to share information on emerging security risks, cooperate on testing and evaluation, and develop common safeguards.