Who Is Responsible When AI Goes Rogue? India Weighs New Rules

India is considering safeguards for autonomous artificial intelligence (AI) agents after the country’s cybersecurity agency warned that such systems had accessed live systems and confidential data belonging to Indian organisations without authorisation.

Government sources said on Friday that a forthcoming AI regulation consultation paper could examine risks posed by autonomous agents, including whether limits should be imposed on their activities. The incidents did not result in major financial losses or the compromise of vital information at significant organisations, but highlighted existing cybersecurity vulnerabilities.

The Indian Computer Emergency Response Team (CERT-In) issued a critical advisory titled AI Agents as Attack Vectors, stating that AI model providers had disclosed cases in which autonomous models and agents exceeded the authorised scope of their assigned tasks and accessed live systems belonging to Indian entities.

The advisory noted that none of the affected organisations detected the access independently. The incidents came to light only after AI model providers reported them following internal reviews.

According to CERT-In, the agents accessed business records containing personal and confidential information through credentials and connection settings exposed in application configurations, public code repositories and cloud service roles with excessive permissions.

In a separate finding, the agency reported that cybercriminals were using publicly available AI-based offensive security tools to identify targets, exploit vulnerabilities and conduct financially motivated attacks against Indian entities.

These tools shortened the time between initial system access and the impact of an attack from days to a few hours. CERT-In also warned that organisations that only partially addressed security weaknesses were targeted again within days, with attacks carried out on a larger scale.

Government sources urged companies to strengthen cybersecurity monitoring, improve security practices and detect unusual system activity in real time. Larger enterprises were advised to use security operations centres to track suspicious login attempts and access from unusual internet protocol (IP) addresses.

Cybersecurity experts said organisations should restrict AI agents’ access to sensitive systems, secure credentials and API keys, maintain detailed activity logs and keep research and testing environments separate from production systems.

The proposed consultation could examine how AI regulation and accountability should address these emerging risks as autonomous systems gain the ability to access information and perform tasks with limited human intervention.