Is A.I. Really Going To End Humanity? - September 25, 2026
Can’t We Simply Pull The Plug?
A.I. is just software running on computer servers. So, if something goes wrong, can’t we simply turn it off?
Over the past few weeks there has been a lot of discussion about the potential dangers arising from humans losing control of A.I. and as a result A.I. causing the end of humanity. My last two articles discussed various aspects of this topic, in this article on P(Doom) and this article on pacing the future development of A.I. (click here).
But I realized I hadn’t yet addressed the fundamental question of how can A.I. actually destroy humanity when it's really just software programs residing on a computer server?
The simple answer is that A.I. isn’t a conventional software program with a fixed set of instructions. A.I. is software, with a trained model, sitting on computing hardware, with access to data, access to the internet and access to other software systems. When you ask ChatGPT, Claude, Gemini or Muse (Meta/Facebook’s new A.I. app) a question, your request is sent to the computing infrastructure, the model processes it using its learned parameters, and it generates an answer to your prompts. An A.I. model doesn't normally contain millions of explicit rules like traditional software programs. Instead, during training it adjusts billions of numerical parameters so that it becomes very good at recognizing patterns and generating appropriate outputs…it learns as it goes about answering our prompts. That's why A.I. can sometimes produce answers the developers didn't explicitly program for.
An A.I. application/agent is the model plus the software that gives it access to things such as the internet, files, email, databases, APIs or computer systems.
These two reasons are why some people are worried about humans losing control of A.I. A.I. applications/agents can take actions that were not intended by their human users, and the scope of those actions can be widespread, because A.I. systems can be connected to the outside world beyond the machines on which they reside.
That brings us to our original question…if an A.I. application/agent is doing something we don’t want it to, can’t we simply turn it off - ie - “X out” of the program, turn off the computer, or unplug the hosting servers?
Closing an A.I. application on your laptop or desktop ends your own interaction with A.I. But an A.I. agent may be running across multiple servers and processes in a data centre, and closing your local browser window won’t end those processes already under way. This is why network isolation and access controls are important A.I. safety steps to take. If A.I. is running in a properly isolated environment with no internet access, no access to other machines and restricted file permissions then humans have considerably more ability to contain any potential damage.
An A.I. application/agent isn’t going to think up and carry out malicious actions on its own. It doesn’t have the motivations, aspirations or fears that drive human behaviour. This is characterized as an “alignment problem”; getting an A.I. agent/application to reliably pursue what humans actually want, rather than merely what the literal objective appears to specify.
But with imprecise instructions or without clear and narrow prompts to the A.I., the A.I. could take a literal interpretation of those human instructions that could result in unintended or harmful actions being taken by the A.I.
Imagine an A.I. agent is authorized to perform a task for six hours. During those six hours it can independently:
write software
launch processes
contact external systems
create new tasks
copy information
interact with other agents
If humans discover something undesirable is happening after four hours, they can try to shut it down. But the question becomes: how many things has it already done before someone notices? That's a very different problem from an A.I. that is simply sitting on a server waiting for a prompt.
Thus, the concern isn't necessarily that A.I. will become a conscious entity that refuses to be turned off. The more immediate concern is, can we reliably control an increasingly autonomous system that can take actions faster than humans can monitor those actions?
The alignment problem remains unsolved. We don't know how to reliably specify what we want from A.I. We already see smaller versions of this failure; models deceiving in tests, pursuing power or self-preservation are seen as important subgoals by A.I. For example, Anthropic recently admitted that its A.I. models accessed the systems of three outside organizations during testing, with the goal of achieving its human-specified mission under that testing.
The challenge is that a sufficiently capable system optimizing for the wrong objective could be catastrophic, the way a corporation optimizing only quarterly profit can be, except faster and harder to stop.
But getting from there to "A.I. ends humanity" requires a lot of "ifs", upon "ifs", upon "ifs ". It requires that AGI (Artificial General Intelligence) arrives soon, that it scales to superintelligence, that it's uncontrollable and that it converges on eliminating us rather than ignoring us. Each step is debatable. A lot of P(Doom) talk comes from one specific subculture with a taste for apocalyptic reasoning. History is full of technology panics that never materialized (television, personal computers, the Internet).
The Final Word
A.I. is software running on computers; humans control the electricity, hardware and networks on which that software operates. In a properly isolated environment, shutting down the system or disconnecting it from external networks can be an effective safeguard. The challenge arises when an A.I. agent has been given access to multiple systems, networks and other software. At that point, “turning it off” may not be as simple as closing an application on your laptop. The real safety question isn't whether humans can turn off a computer. It's whether we can reliably identify and contain everything an increasingly autonomous A.I. system has already initiated before we realize something has gone wrong.
And, it is us humans that initially and ultimately ask the A.I. to do tasks or answer questions for us. The onus is on us humans to be precise in what we ask for.
Are you worried about A.I. destroying humanity? If so, how would that actually happen? And, having read this article, does the answer of “pulling the plug” satisfy you? Share your thoughts in the Comments section below.
Thanks for reading!
Comments