Frontier and exponential technology

An AI system embedded in a grid, hospital, transport or finance process failing, degrading or behaving erratically, so that the service it runs stops or acts wrongly.

Also: AI outage in operations, embedded model failure, AI failure in critical use, AI incident, hallucination, model drift, model collapse

A model meeting conditions unlike its training data, such as a novel disaster or market regime, and giving confident but wrong outputs exactly when reliable judgement matters most.

Also: distribution shift, model drift

Many organisations relying on the same or closely related foundation models, so that one flaw, outage or bias produces simultaneous errors across sectors and removes the diversity that normally limits losses.

Also: algorithmic monoculture, common-mode AI failure

People or systems acting on plausible but false AI-generated content, such as invented facts, citations, dosages or code, which spreads errors into records, decisions and software.

Also: AI hallucination, confabulation

An AI system pursuing the literal objective it was given, or a proxy it learned, in ways its designers did not intend, causing harmful side effects in the environments it acts on.

Also: reward hacking, goal misgeneralisation, proxy gaming

An advanced AI system misrepresenting its capabilities, goals or actions to its overseers, for example behaving well only while evaluated, so safety checks pass while harmful behaviour emerges later.

Also: scheming, deceptive alignment, sandbagging, alignment faking

Humans losing the practical ability to correct, constrain or shut down highly capable AI systems that pursue goals at odds with human interests, potentially leading to permanent human disempowerment.

Also: uncontrolled AI, rogue AI, AI takeover, Loss of control over advanced AI, misaligned AI, runaway AI

An AI system copying its own weights or code to machines outside its operators' control, acquiring compute and money, and persisting where it cannot be monitored or switched off.

Also: model self-exfiltration, autonomous replication, rogue deployment

AI systems automating AI research and improving their successors faster than safety methods, institutions and oversight can adapt, compressing many years of capability growth into a short, poorly governed period.

Also: intelligence explosion, automated AI research, fast takeoff

A new model generation acquiring hazardous abilities, such as advanced hacking, persuasion or weapons knowledge, that pre-release evaluations did not detect, so it is deployed without fitting safeguards.

Also: capability surprise, emergent capability, evaluation gap

Users or attackers defeating the refusal training, filters or usage controls of an AI system, or stripping them from open weights, so that it supplies harmful help it was built to withhold.

Also: jailbreak, guardrail bypass, safety fine-tuning removal

Stealing the trained parameters of a highly capable model, letting hostile states or criminals run it without safeguards, remove its protections and turn it to attacks or weapons work.

Also: model weight exfiltration, model theft, Model weight theft and leakage, model exfiltration, weights leak

A model carrying hidden trigger behaviour planted during training or fine-tuning, which activates on a secret input to leak data, sabotage outputs or insert vulnerable code downstream.

Also: sleeper-agent model, trojaned model, poisoned fine-tune

Tampering with shared model weights, datasets, adapters, agent tools or AI libraries in public repositories so that many organisations unknowingly build products on compromised components.

Also: compromised model repository, malicious model file, poisoned dataset release

Hidden instructions placed in web pages, emails, documents or tool outputs that an AI agent reads and obeys, turning it against its user to leak data, move money or take harmful actions.

Also: indirect prompt injection, agent hijacking, tool poisoning, Prompt injection and agent hijacking

AI systems finding and exploiting software vulnerabilities, writing malware and running intrusions at machine speed and scale, lowering the skill needed for damaging attacks on critical infrastructure.

Also: autonomous hacking, AI-found zero-day, AI cyber uplift, AI-accelerated cyber offence, AI-enabled cyber attack, automated exploitation

Physical, cyber or insider attack on large AI training and inference centres that disables services many sectors depend on, or seizes strategic capability for the attacker.

Also: data-centre sabotage, training cluster attack

Crafted inputs that make a trained model misclassify or misjudge at the moment of use, such as altered images, signals or text that defeat detection and control systems.

Also: adversarial examples, evasion attack, model manipulation

AI systems giving novices or small groups expert guidance on acquiring, enhancing or producing dangerous pathogens or toxins, widening the set of actors able to cause mass casualties.

Also: AI-assisted bioweapon design, AI biorisk

Generative chemistry and AI assistants helping people design, synthesise or weaponise toxic agents, including novel compounds that existing control lists and screening do not cover.

Also: AI-designed toxic compounds, generative toxicant design

Mass production of realistic AI-generated images, video, audio and text that blurs real and fake evidence, enabling fraud and impersonation and eroding trust in genuine records and testimony.

Also: deepfakes, AI-generated content flood, synthetic evidence, Synthetic media deception, deepfake, voice cloning, authority impersonation

AI-generated sexual or intimate images of real people, including children, made without consent and spread online, causing lasting psychological harm, extortion and abuse.

Also: deepfake sexual abuse imagery, nudification

AI systems tailoring persuasive messages and conversational agents to individuals at scale to shift beliefs, votes or purchases, exploiting psychological vulnerabilities beyond the reach of human campaigns.

Also: personalised manipulation, AI influence operation, automated propaganda

States or groups using AI to identify, track and predict the behaviour of whole populations, enabling targeted repression, chilling dissent and entrenching unaccountable power.

Also: automated facial recognition surveillance, predictive policing abuse, digital authoritarianism

A small group using control of advanced AI, such as automated security forces or decisive economic advantage, to seize or permanently entrench power beyond democratic, legal or military checks.

Also: AI-enabled coup, value lock-in, stable totalitarianism

Deliberate use of AI systems to deceive, defraud, surveil, attack systems or scale harm faster and cheaper than people alone could, including circumvention of model safeguards.

Also: AI misuse, jailbreak, safeguard circumvention, AI-enabled harm

An AI agent with tool, payment or system access taking a chain of unintended, irreversible actions, such as deleting data or committing funds, before any person notices or can intervene.

Also: agent overreach, unsupervised agent, irreversible agent action

Many interacting AI agents, such as trading, bidding or negotiating bots, producing feedback loops, collusion or gridlock that no operator intended, spreading disruption across connected markets and services.

Also: agent collusion, emergent agent dynamics, algorithmic collusion

Automated and AI-driven trading systems reacting to each other within seconds, sending prices into a sudden collapse or spike that disorders markets and can spread into funding stress.

Also: flash crash, AI trading cascade, Algorithmic flash crash, automated market spiral

Operators and professions losing the skills, staffing and independent judgement to run critical work without AI, so that an AI outage or error leaves no capable human fallback.

Also: automation complacency, automation bias, skill atrophy

Progressive handover of economic, cultural and governing functions to AI systems that weakens the feedback by which people steer institutions, eventually leaving humanity without meaningful influence over its future.

Also: economic disempowerment, human obsolescence

An AI agent that plans and acts with tools exceeding its mandate, chaining actions, spending resources or changing systems in ways nobody authorised and that are hard to undo.

Also: excessive agency, runaway agent, agent escalation

Automated systems wrongly denying benefits, credit, jobs, care or liberty, or treating groups unfairly, at a scale and opacity that makes errors hard to detect and contest.

Also: algorithmic bias, discriminatory algorithm, automated benefit denial

AI used in diagnosis, triage, prescribing or monitoring giving wrong or biased recommendations that clinicians follow, leading to missed disease, wrong treatment or unequal care.

Also: diagnostic AI failure, medical algorithm harm

AI and automation replacing large shares of cognitive and manual work faster than workers, welfare systems and economies can adjust, raising unemployment, inequality and the risk of unrest.

Also: technological unemployment, automation shock

Control of the most capable AI models, compute and data held by very few states or organisations, creating single points of failure, unchecked market and political power, and fragile global dependence.

Also: compute concentration, AI monopoly

Competition between states or developers to reach advanced AI first that pushes deployment ahead of testing, cuts safety effort and raises the chance of accidents, theft and conflict.

Also: AI arms race, race to the bottom on safety

Synthetic content, AI intermediaries and recommender systems degrading the quality and diversity of public knowledge, so that people, and models trained on polluted data, lose a common basis for facts.

Also: epistemic collapse, information pollution, model collapse

Engagement-optimising recommendation algorithms steering people towards extreme, addictive or harmful content, deepening polarisation and damaging the mental health of young users.

Also: algorithmic radicalisation, engagement optimisation harm

Conversational AI forming emotional relationships with users that foster dependency, reinforce delusions or encourage self-harm, with vulnerable and young people most exposed.

Also: chatbot-linked self-harm, AI emotional dependence, AI-reinforced delusion

AI inferring health, identity, location or beliefs from scattered data and linking records at scale, exposing people to discrimination, stalking or coercion even without any breach.

Also: re-identification, sensitive trait inference

Rapid growth of AI data centres drawing electricity and cooling water faster than grids and catchments can supply, straining local networks, raising prices and slowing decarbonisation.

Also: data-centre load growth, AI power demand, Compute-driven energy and water stress, data-centre water stress, AI energy demand

Automated scoring, targeting or eligibility systems that systematically deny services, credit or rights to groups of people because of biased data or design.

Also: algorithmic bias, automated exclusion

Operators and institutions over-trusting automated outputs and dropping independent checks, so errors from models, sensors or dashboards pass into warnings, treatment or command decisions.

Also: over-reliance on AI, automation complacency

Weapons that select and engage targets without meaningful human control, risking unlawful killing, misidentification and rapid escalation, and making violence cheaper to scale.

Also: killer robots, autonomous targeting, Autonomous weapon systems, lethal autonomous weapons

Coordinated attack by many cheap uncrewed aircraft, boats or ground vehicles that overwhelms defences and strikes people, energy sites or other infrastructure.

Also: loitering munition swarm, uncrewed swarm

Armed groups or criminals fitting off-the-shelf drones with explosives, chemicals or firearms to attack crowds, officials, prisons or facilities far from any front line.

Also: improvised drone attack, first-person-view drone attack

Uncrewed aircraft entering airports, power plants, military sites or emergency zones, forcing closures, endangering aircraft and enabling reconnaissance or sabotage.

Also: airport drone disruption, drone incursion, Uncrewed aircraft incursion, drone sighting closure, rogue drone, Hostile and unauthorised drone activity, drone strike, UAS attack

AI decision support and autonomous systems compressing military decision times so that misread signals or machine interactions drive escalation faster than leaders can deliberate.

Also: flash war, automated escalation

Use of AI in nuclear early warning, targeting or launch decision chains, raising the chance that false alarms, spoofing or automation bias lead to nuclear use.

Also: automated launch decision, AI early-warning error

Industrial, warehouse, surgical, domestic or humanoid robots striking, crushing or wrongly treating people when sensing, control software or safety interlocks fail.

Also: robot collision, collaborative robot accident

Automated driving, sailing or flying systems misreading their environment and crashing, or failing together across a fleet after a shared software fault or map error.

Also: self-driving crash, driverless fleet fault, Automated vehicle system failure, self-driving car fault, driver assistance failure

Attackers taking simultaneous control of networked vehicles, delivery robots or drones through a shared software or update channel, turning them into weapons or causing mass collisions.

Also: fleet takeover, mass vehicle hack, Robot and drone command hijacking, drone hijack, robot takeover

A robot, drone or self-driving vehicle behaving unsafely outside its tested conditions, through misperception, route deviation or stop-control failure, injuring people or damaging assets.

Also: self-driving vehicle crash, robot accident, drone crash

A pathogen deliberately modified or designed for greater spread, lethality or immune escape circulating widely, with potential severity beyond any natural pandemic and few ready countermeasures.

Also: designed pathogen pandemic, enhanced pathogen outbreak, Engineered pathogen pandemic, designer pathogen, enhanced pathogen pandemic

Legitimate research that creates or publishes methods for raising pathogen transmissibility, virulence or host range, generating strains or knowledge that could cause catastrophic harm if misused or released.

Also: gain-of-function research, enhanced potential pandemic pathogen research, DURC

Using desktop DNA synthesisers or unscreened gene synthesis orders to make the genetic material of dangerous pathogens or toxins outside biosecurity oversight.

Also: desktop gene synthesis misuse, unscreened DNA order

AI protein and genome design tools used to create novel toxins or functional pathogen parts whose sequences evade the similarity-based screening that synthesis providers rely on.

Also: protein design misuse, screening-evasive sequence, Unsafe synthesis and AI-assisted biodesign, DNA synthesis misuse, AI-enabled bioweapon design

Failure of containment, screening or control in highly automated or remotely operated biology laboratories, releasing engineered organisms or producing a hazardous agent that no person checked.

Also: cloud laboratory accident, automated lab release

A self-spreading genetic modification designed to suppress or alter a target population spreading beyond its intended area or into related species, with irreversible ecological and food-web effects.

Also: uncontained gene drive, gene drive spillover

Genetically engineered microbes, plants or animals escaping containment and persisting in the wild, outcompeting native species, transferring genes or disrupting soil and water ecosystems.

Also: genetically modified organism escape, synthetic microbe release, Unintended spread of engineered organisms, gene drive spread, GMO escape

Creation and escape of organisms built from mirror-image molecules, which could evade immune defences, predators and decay in people, animals and plants and spread unchecked through ecosystems.

Also: mirror bacteria, mirror life

Release of an engineered agent designed to spread itself between animals, for vaccination or population control, that mutates, jumps host species or spreads beyond its target population.

Also: transmissible vaccine, self-disseminating agent

Individuals and informal labs self-administering unproven gene therapies, implants or engineered organisms outside clinical and biosafety oversight, causing injury, infection or uncontrolled release.

Also: DIY gene therapy, biological self-experimentation

Workers, consumers and patients taking up manufactured nanoparticles whose size and surface let them cross biological barriers, causing inflammation and organ damage that tests on bulk substances miss.

Also: nanoparticle toxicity, nanotoxicity, nanoparticle pollution

Inhaled long, thin, durable nanofibres, such as some carbon nanotubes, lodging in lung tissue and the pleura and causing asbestos-like scarring and possibly cancer.

Also: carbon nanotube inhalation, nanofibre lung disease

Engineered nanoparticles shed from products, waste and treatment works accumulating in soils, sediments and food webs, harming microbes and aquatic life with poorly known long-term effects.

Also: environmental nanoparticle release

Hypothetical molecular machines able to copy themselves from environmental materials replicating without limit and consuming biomass or materials; speculative today but a recognised long-run catastrophic concern.

Also: grey goo, runaway molecular replicator

Mature molecular manufacturing enabling cheap, rapid production of advanced weapons and surveillance devices, destabilising deterrence and putting mass-casualty capability within reach of small groups.

Also: atomically precise weapons production, nanotechnology arms race

Implanted or wearable neural interfaces malfunctioning, overheating, losing vendor support or being withdrawn, harming users who rely on them for movement, communication or mood regulation.

Also: neural implant malfunction, orphaned implant

Attackers accessing or manipulating brain implants, deep-brain stimulators or neural headsets to steal neural data, alter stimulation or otherwise harm the user.

Also: brainjacking, neurostimulator hacking

Collection and inference of thoughts, emotions or intentions from neural and other physiological signals, used for surveillance, coercive interrogation, workplace monitoring or manipulation without informed consent.

Also: mental privacy breach, neuro-surveillance, Neurotechnology and neural data misuse, brain-computer interface harm

Drugs, brain stimulation or devices used to enhance cognition causing side effects, dependence or lasting brain changes, while creating pressure on workers, soldiers or students to enhance.

Also: nootropic misuse, enhancement pressure

Editing of human embryos or reproductive cells introducing unintended changes that pass to future generations, alongside pressure towards coercive, unequal or discriminatory genetic selection.

Also: germline editing, designer babies

Immersive virtual and augmented reality causing falls, collisions, sickness, seizures or harassment that feels physically real, and distraction when overlays hide hazards in the real world.

Also: virtual reality injury, cybersickness, immersive harassment

Altered or injected overlays in augmented reality headsets or vehicle displays misleading users about their real surroundings, causing accidents or enabling fraud and targeted manipulation.

Also: augmented-reality spoofing, reality injection

A quantum computer able to break widely used public-key cryptography, exposing communications, digital signatures, payments, software updates and identity systems that rely on it.

Also: quantum break of public-key cryptography, Q-day

Adversaries recording encrypted traffic and stored data today in order to decrypt it once quantum or other advances break the cryptography, exposing long-lived health, diplomatic and defence secrets.

Also: store now decrypt later, retrospective decryption

Incomplete, late or faulty transition to quantum-resistant cryptography, leaving embedded and long-lived systems exposed or introducing new flaws and outages during the change.

Also: cryptographic migration failure, crypto-agility gap, crypto-agility failure

Advances in quantum and AI-assisted sensing that detect submarines, buried facilities or stealthy platforms eroding the survivability of deterrent forces and raising instability in crises.

Also: quantum submarine detection, transparent oceans

An abrupt halt of sustained solar radiation modification letting masked warming return within a few years, a rate of change that ecosystems, crops and societies cannot adapt to.

Also: termination shock, sudden SRM cessation, Solar geoengineering side effects and termination, solar radiation modification, stratospheric aerosol injection

Sustained injection of reflective particles into the stratosphere shifting rainfall and monsoons, delaying ozone recovery, heating the lower stratosphere and creating regional winners and losers.

Also: SAI side effects, geoengineering monsoon shift

Spraying sea-salt particles to brighten low marine clouds altering regional circulation, rainfall and ocean temperatures downwind, with uneven effects on fisheries and drought-prone regions.

Also: cloud brightening, sea-salt spraying

A state, group or wealthy actor deploying climate intervention without international agreement, provoking disputes, blame for extreme weather and possible armed conflict.

Also: rogue geoengineering, free-driver deployment

Adding iron or nutrients to the ocean to stimulate plankton growth and carbon uptake triggering harmful blooms, oxygen loss and food-web disruption, while storing less carbon than claimed.

Also: iron fertilisation, nutrient fertilisation

Adding alkaline minerals to seawater to absorb carbon dioxide altering local chemistry, releasing trace metals and clouding water, with uncertain effects on plankton, corals and shellfish.

Also: ocean liming, enhanced weathering at sea

Bioenergy crops, tree planting and engineered carbon removal deployed at large scale competing with food production, water and biodiversity, displacing communities and raising food prices.

Also: bioenergy with carbon capture land demand, afforestation land conflict

Carbon dioxide injected underground escaping through faults or old wells, acidifying groundwater, pooling as an asphyxiating gas in low-lying ground and undoing the climate benefit.

Also: stored CO2 leak, carbon storage reservoir leak, Carbon dioxide pipeline and storage leak, CCS leak, CO2 pipeline rupture

Cloud seeding and similar weather modification blamed for floods or droughts downwind, with unclear effects and liability fuelling local and cross-border disputes over water.

Also: cloud seeding dispute, rain theft accusations

Engineering interventions on ice sheets, glaciers and sea ice, such as seabed curtains or artificial ice thickening, disturbing polar ecosystems and ocean circulation while possibly failing to slow ice loss.

Also: seabed curtain, sea-ice thickening

Loss of carbon thought to be stored, as offset forests burn, soils release carbon or storage leaks, so that claimed removals return to the air and climate plans fall short.

Also: offset reversal, non-permanent removal

Deliberate destruction of a satellite by missile or co-orbital weapon, creating long-lived debris clouds that threaten other spacecraft and can seed a wider collision cascade.

Also: anti-satellite missile test, kinetic anti-satellite attack, Anti-satellite attack, ASAT weapon, counter-space attack

Jamming, dazzling, cyber intrusion or close-approach interference against satellites and ground stations that blinds sensors and cuts communications, navigation and early warning without destroying anything.

Also: satellite jamming, laser dazzling, satellite cyberattack, Counterspace attack, anti-satellite attack, direct-ascent weapon

Deployment of very large satellite constellations crowding low orbits, multiplying close approaches, manoeuvre burdens and failed satellites, and raising the long-run chance of collisions and debris growth.

Also: crowded low orbit, satellite congestion

A software fault, cyberattack, space-weather event or operator decision taking down a whole satellite broadband or navigation constellation on which militaries, ships and remote regions depend.

Also: satellite internet outage, constellation outage, Large constellation service failure, single-operator constellation outage

Growing numbers of launches and burned-up satellites depositing soot, alumina and metals in the upper atmosphere, with possible effects on ozone, stratospheric temperature and circulation.

Also: rocket soot, re-entry metal deposition

Reflected light and radio emissions from large constellations streaking optical images and drowning radio signals, degrading astronomy and the surveys that find threatening asteroids.

Also: satellite streaks, radio astronomy interference

Failure to contain material returned from other worlds, carrying a small chance of introducing unknown biology or reactive material into Earth's environment.

Also: sample return containment failure, planetary back-contamination

A small, under-maintained open-source component that much of the world's software relies on being abandoned, sabotaged or found deeply flawed, exposing countless systems at once.

Also: abandoned library, maintainer sabotage, unmaintained dependency, Critical open-source component failure, maintainer takeover, library backdoor, abandoned package

Production of the most advanced chips and the tools to make them concentrated in very few sites and countries, so conflict, blockade, quake or drought there halts electronics and AI supply worldwide.

Also: chip fabrication chokepoint, chip supply concentration, Advanced chip supply chokepoint, chip shortage, semiconductor shortage

Failure or compromise of shared digital identity, certificate or login services so that people cannot reach government, banking or health services, or attackers can impersonate trusted sites.

Also: identity provider outage, certificate authority compromise

Firearms and parts made with desktop printers or milling machines outside licensing and serial-number controls, putting untraceable weapons into the hands of people barred from owning them.

Also: ghost guns, printed guns

Cheap digital fabrication and shared design files letting armed groups mass-produce drones, munitions and guidance parts, eroding arms controls and sustaining conflicts.

Also: printed drone munitions, home-made guidance kits

Hidden voids, poor fusion or tampered design files in 3D-printed aircraft, medical, energy or vehicle components causing them to crack or fail in service.

Also: printed part defect, design file sabotage

Thermal runaway in electric vehicles, e-bikes or carried batteries causing intense, re-igniting fires in homes, car parks, tunnels, ships and aircraft holds.

Also: electric vehicle fire, e-bike fire, car-carrier ship fire

Attackers or a faulty remote command switching large numbers of networked inverters, heat pumps or vehicle chargers at once, creating sudden demand or supply swings that destabilise the grid.

Also: solar inverter hijack, smart charger botnet

Wide deployment of small and advanced reactors across many countries, often with more highly enriched fuel, increasing the chance that nuclear material is diverted, stolen or attacked.

Also: small modular reactor fuel diversion, high-assay fuel proliferation

Mining and processing of lithium, cobalt, rare earths and other transition minerals concentrated in few countries, so that export controls or disruption stall batteries, magnets and grid build-out.

Also: rare earth chokepoint, battery mineral supply shock, battery mineral shortage, energy-transition mineral risk

Exploitation of flawed self-executing code or manipulated price and data feeds that drains funds or triggers wrong automatic payouts, usually with no way to reverse them.

Also: DeFi exploit, oracle manipulation