Sometime around early June, I found a Japanese beetle on my raspberry bush, munching away on the large green leaves. In response, I did what any rational gardener would do. I pointed outdoor Reolink cameras at the garden, installed Frigate NVR to detect the presence of beetles, threw in an LLM to identify the perpetrators, and tied the whole thing into a WWII-era air raid siren. Now I have beetles and pissed-off neighbors.
The issue here isn’t really the beetles. It’s the architecture and the ease with which it can be developed and deployed. More importantly, it can be done privately for a relatively small investment. The technologies are inexpensive. The computer vision is readily available. The LLM required is tiny. So what’s the problem?
Replace “Japanese beetle” with any social identifier: Person. Vehicle. Protester. Gun owner. Employee. Undesirable.
Next, replace “rational gardener” with any well-funded social-action group, political-action group, or corporation: MIRI. Everytown for Gun Safety. The Heritage Foundation. Granola. Flock.
Yesterday I wrote about MIRI’s argument that increasingly capable AI systems could eventually become autonomous threats. The underlying assumption is that sufficiently capable AI becomes the dangerous actor. I think that focuses our attention in the wrong direction.
My concern has never primarily been government surveillance, particularly in the United States. Government can be monitored, audited, protested, investigated, sued, voted out of office, and written about. Public-records laws exist. Courts exist. Elected oversight exists. None of those mechanisms is perfect, but we have spent a very long time building tools specifically to restrain government power.
Even when police abuse surveillance technology, the abuse can become a matter of public record. In Milwaukee, an officer was charged after allegedly using Flock license-plate data to track someone he was dating and that person’s former partner.
Private surveillance is a different problem. A private organization doesn’t answer to voters. Its internal databases aren’t generally subject to public-records requests. Its algorithms don’t have to be debated at a city council meeting. Its surveillance infrastructure may sit entirely on private property, purchased with private money, operating under private policies. And increasingly, it doesn’t require extraordinary resources. A camera. A drone. Computer vision. A small language model. A database. Some automation. That’s enough.
This is where I think MIRI gets it wrong. The danger is not necessarily AGI turning into SkyNet. The danger is when a well-funded organization decides it needs a database of every gun owner, every woman who entered an abortion clinic, or everyone who attended the wrong political protest. None of those systems requires artificial general intelligence. They require identification. Classification. Correlation. Storage. Automation. And motive.
The dystopian bureaucracy in Terry Gilliam’s Brazil depended on an enormous Ministry of Information. It required buildings full of workers, records, machinery, paperwork, ducts, terminals, and bureaucrats to maintain the surveillance apparatus. The Ministry of Information already exists. It just isn’t monolithic. It is a thousand small ministries in the hands of people we don’t know, operating systems we cannot see, building databases we may never know exist. Some are corporations. Some are advocacy organizations. Some are data brokers. Some are private individuals. Some may consist of little more than a camera, a computer, and some open-source software. They don’t need AGI to deliver a dystopian nightmare. They need a $100 camera, Frigate, an LLM, a database, and a motive.
A sufficiently powerful stupid system can be far more dangerous than a hypothetical intelligent one.