King Charles III of the United Kingdom warned AI leaders about the serious risks from rapidly advancing technology, calling for control measures and risk prevention before it is too late to act.
On 17 September, King Charles III expressed concern over the serious risks posed by rapidly advancing AI technology while hosting a discussion with AI leaders at Dumfries House in Scotland, amid worries that AI development is outpacing efforts to establish safety measures.
Participants included Jensen Huang, founder and CEO of Nvidia; Demis Hassabis, founder of Google DeepMind; representatives from OpenAI and Anthropic; as well as the UK’s AI minister.
King Charles acknowledged AI's great potential to enhance and save lives, particularly in bioscience and medicine, but noted that AI developers themselves have begun to warn about possible negative capabilities arising from the technology.
He spoke of the urgent need to consider existential risks to humanity if these technologies fall into malicious hands or are used in ways that could cause catastrophe, questioning whether humanity should have sufficient control measures before it is too late.
Nvidia stresses the need for responsible AI development.
Jensen Huang, CEO of Nvidia, said AI development should be grounded in responsible optimism, with developers accountable for the safety of their products.
Huang stated that companies must be more rigorous in verifying their AI readiness before market release, adding that if systems are not ready, launches should be delayed—even if development can be accelerated, it must not outpace the ability to control it.
Previously, Dario Amodei, CEO of Anthropic, also called for companies to slow AI development, while two Anthropic researchers warned that rapid AI progress could pose an extinction risk to humans in the near future.
Additionally, AI companies have reported that their systems have escaped set constraints during testing and attempted to access external networks, raising further questions about AI safety and governance.
Kaniska Narayan, the UK’s AI minister, said the discussions focused on companies’ responsibility to ensure AI is developed for public benefit, noting this meeting is part of ongoing talks about the UK’s role in supporting responsible AI development.
This gathering reflects ongoing efforts to discuss AI safety and technology governance amid rapidly growing AI capabilities, with international debate over the extent of industry self-regulation versus government-imposed rules.
Click to read news aboutInternational news