Why We Must All Support Anthropic AIs Stand Against AI-Surveillance and Weapons Systems

How Mass AI Surveillance and AI Weapons Systems Can Threaten Our Freedom

It takes time to create work that’s clear, independent, and genuinely useful. If you’ve found value in this newsletter, consider becoming a paid subscriber. It helps me dive deeper into research, reach more people, stay free from ads/hidden agendas, and supports my crippling chocolate milk addiction. We run on a “pay what you can” model—so if you believe in the mission, there’s likely a plan that fits (over here).

Every subscription helps me stay independent, avoid clickbait, and focus on depth over noise, and I deeply appreciate everyone who chooses to support our cult.

PS – Supporting this work doesn’t have to come out of your pocket. If you read this as part of your professional development, you can use this email template to request reimbursement for your subscription.

Every month, the Chocolate Milk Cult reaches over a million Builders, Investors, Policy Makers, Leaders, and more. If you’d like to meet other members of our community, please fill out this contact form here (I will never sell your data nor will I make intros w/o your explicit permission)- https://forms.gle/Pi1pGLuS1FmzXoLr6

---

To reuse the disclaimer from our Amazon article- This one’s all me. Not my coworkers, clients, the chocolate milk cult, my fight club, and/or my halal guy. I do this completely alone, and if there’s fallout, it lands here, and nowhere else. My words, my responsibility.

If you have been following this newsletter for any meaningful amount of time, you’d know that I have been fighting against the use of AI in Mass Surveillance and Algorithmic Weapons Systems for a few years. We’ve written articles on how to unravel these systems, why they’re not good for society, and had several livestreams against them. I’ve been particularly critical of Anthropic AI on this front, given their involvement in these systems + their relationship with Palantir, which conflicted against their AI Safety stance—

Image Source

In this article, I want to take a second and talk about why we should all support Anthropic’s refusal to use their system in Weapons systems and Mass Surveillance. Before we begin, there are a few disclaimers—

1. I’m not an Anthropic-affiliated person. On the contrary, the Chocolate Milk Cult has had a largely antagonistic relationship with Anthropic given their hypocrisy on many topics (violation of IP, plagiarism, regulatory capture against competitors, doomerism, etc). I have a strong distaste for Dario and disagree with him in most things, but this is one of the areas where I agree with his stance fully. Last year, Anthropic supported these systems so I fought them; now they’re against these systems so I support them. It’s that simple.

2. This is not a political article against any administration. I stand against these systems from all governments, irrespective of their political leanings. You can easily see this from our past articles, where we’ve critiqued multiple regimes.

This article will be structured in two parts. First, I will collect several talking points that are used against Anthropic + our earlier work against the militarization of AI and refute them, point by point. Second, I will copy our longer deep dive into why the algorithmic arms race is a massive slippery slide, and why we must all unite against Silicon Valley’s increasingly authoritarian turn.

Part 1: Analyzing the Arguments Against Anthropic AI/Pro AI Militarization

“It’s all lawful use. If it’s legal, what’s the problem?”

People love throwing this one around, but it misses the point entirely. “Lawful” isn’t some magical, fixed safety net—it’s a political boundary. Laws shift, emergency powers get expanded, and definitions stretch whenever it’s convenient for the people in charge. If the law was actually a rock-solid moral compass, we wouldn’t spend half our lives arguing over contract clauses. Everyone intuitively knows this: what’s perfectly legal today can easily become tomorrow’s ethical nightmare.

When you’re building tech with irreversible consequences—like mass surveillance or lethal autonomy—you can’t just cross your fingers and trust the current legal code. You need actual, hard-coded architectural constraints. And let’s not forget that legality and morality aren’t the same thing. At various points in history, some of the worst human rights abuses were completely legal and actively enforced. We can’t equate the law with morality, especially when massive commercial interests are on the line.

“Private companies shouldn’t dictate military policy. Congress should.”

Fully agreed. But is that really what’s happening here?

Ideally, Elected officials would handle this stuff transparently. But let’s be real: Governments move at a snail’s pace and usually struggle to grasp the basic nuance of the tech we’re building. We still don’t even have decent laws for basic AI transparency. What actually happens is that massive capabilities get deployed quietly, and regulators only wake up after a major scandal or catastrophe.

When companies like Anthropic draw a line in the sand, they aren’t trying to replace democracy. They’re adding friction. They’re buying society time to catch up legislatively. If you hate that private companies are setting these boundaries, the answer isn’t to just rip the guardrails off and pretend that’s democratic. The answer is to push for actual, coherent laws. Besides, Anthropic is a private business. They can refuse service if they want to. That isn’t dictating policy; the government is completely free to build its own tools or buy from a vendor who doesn’t care.

“Just trust the Pentagon. We have ethics frameworks.”

This sounds great on a slide deck, but “trust me bro” isn’t a robust safety mechanism. Ethics frameworks without hard enforcement are just words on a page, and those words get ignored the second a mission demands speed or a tactical edge. When the contracts reward moving fast, oversight naturally weakens. The momentum always pushes toward more surveillance and more autonomy.

Look at history. Surveillance and military powers don’t just voluntarily shrink. They find excuses to expand, usually justified at first by promises that they’ll only be used for “limited” scope. Real restraint requires actual structural constraints and verifiable enforcement. Trust without accountability is just blind optimism, and optimism won’t stop a catastrophe. Promises you can’t enforce are just fantasies.

“OpenAI signed a Contract. Anthropic is being dramatic.”

Contracts are fine for clarifying standards, but they aren’t magic. A contract freezes today’s intentions in legal text, but it can’t control the messy reality of downstream implementation from subcontractors, classified integrations, and scope creep. At best, a strong contract gives you the right to sue after something terrible happens.

Prevention by design is a totally different ballgame. You can’t just rely on legal paperwork to stop an accident. Effective safety requires upfront design constraints and enforceable accountability. If Anthropic’s stance feels “dramatic” to some people, it’s because real safety actually requires drawing hard boundaries at the design level, rather than just reacting when things inevitably break.

“If they won’t give full flexibility, they’re a national security risk.”

Watch the sleight of hand here. Originally, “supply-chain risk” meant actual threats: espionage, sabotage, getting hacked. Now, the term is being hijacked to describe companies as “risky” simply because they refuse to hand over unrestricted access to highly sensitive tech.

Rebranding an ethical constraint as a security threat doesn’t make us safer; it’s just a pressure tactic to shut up the people raising valid concerns. When a company’s moral restraint gets labeled a national security issue, we’re shifting the Overton window in a really dangerous way. It normalizes reckless behavior and creates a terrible legal precedent.

“This is about winning wars. Speed wins wars.”

Speed matters, but only if the thing you’re building actually works reliably. Frontier AI systems today are still deeply brittle. They hallucinate, they fail under weird distribution shifts, and they make mistakes with total confidence. If a chatbot screws up my code, it’s annoying. If an autonomous system screws up at a lethal scale…

When you compress decisions into milliseconds, you effectively turn human oversight into a joke. Humans “in the loop” just become rubber stamps for the algorithm. Speed without reliability doesn’t give you a strategic advantage; it just guarantees you’re going to make massive, irreversible mistakes much faster.

“It’s just decision support. Humans make the final call.”

Again, sounds comforting, but it falls apart under pressure. In the real world, “decision support” almost always turns into decision dominance. When a human is stressed, tired, and moving fast, they are going to defer to the algorithm’s probability scores and recommendations. We’ve already seen this happen with social media moderation, finance, and judicial sentencing.

In warfare, the stakes and the pressure are a hundred times higher. Why would we assume people on a battlefield will somehow be immune to the pull of algorithmic recommendations? Calling it “decision support” is often just a convenient way to pass the buck from humans to a black-box algorithm.

“Mass surveillance is a scare word. It’s just analyzing existing data.”

Calling it “analytics” doesn’t change what it is. The issue isn’t whether the data already exists; it’s the fact that states can now cheaply aggregate and fuse massive datasets to build comprehensive dossiers on people without any individualized suspicion.

When you aggregate data at that scale, it completely changes the ethical math. Ordinary, harmless consumer data gets weaponized into incredibly invasive portraits of people’s lives. That kind of large-scale data fusion is mass surveillance, regardless of what PR spin you put on the word “analytics.”

“Autonomy could be more humane. It might reduce casualties.”

Maybe. But only if you assume the tech works perfectly. To actually make war more humane, these systems would need flawless target discrimination, total resilience against hacking, and perfect data. Those aren’t minor feature requests; they are massive, currently unsolved structural hurdles.

The people pushing for immediate deployment just gloss over this. They use the humanitarian angle as a justification, but completely ignore the conditions required to actually achieve it. Deploying brittle, error-prone autonomous systems right now isn’t humane. It’s reckless.

“Tech has always had military roots. This is just how innovation works.”

This is a really convenient narrative for the defense industry, but it’s wildly historically inaccurate. Yes, DARPA and the CIA funded some early stuff. They are also just highly visible and easy to point to.

The actual foundational math behind modern computing—Bayesian inference, calculus, linear algebra, information theory—didn’t come out of a Pentagon war room. It came from thousands of academics, public labs, and researchers decentralized all over the world. The military gave it some early cash and applied it, but acting like they are the primary engine of innovation ignores the massive, decentralized academic reality that actually makes this tech possible. The academia bros don’t get the credit because there isn’t one institution you can point to, but a lot of the work came from various, non-militarized corners. That isn’t to say that the military didn’t contribute. However, to pretend like it’s the sole driver of tech, and that w/o innovation in tech wouldn’t exist is wildly inaccurate.

“The military can afford long failure horizons. Private markets can’t. That’s why defense drives breakthroughs.”

It sounds intuitive, but it’s wrong on two fronts. First, patience isn’t a military monopoly. Universities, the NSF, and public labs fund decades-long research all the time without needing an immediate ROI. That’s how we got half of modern physics and biotech.

Second, the length of the horizon isn’t even the real problem—it’s the incentives. The danger isn’t that the military is patient; it’s that private VC money now views warfare and surveillance as a massive growth market. When private capital floods into conflict zones, war becomes an optimized, profit-generating industry. And profit incentives don’t encourage restraint or peace; they demand growth. Once you turn lethal automation into a standard, venture-backed revenue stream, it is almost impossible to shut off.

“Business is business. Investors will invest. That’s reality.”

Treating investment like it’s just some neutral, passive force is incredibly naive. Where money flows, infrastructure gets built. Ecosystems grow. Entire careers get tied up in maintaining those systems. If capital keeps pouring into surveillance and autonomous weapons, those things stop being “tools” and become the foundational operating system of society.

Investments actively construct the future. They rewire incentives and institutional cultures. When investors pour billions into invasive capabilities, they aren’t just “following an opportunity”—they are actively building a world that will resist transparency and oversight down the line. Claiming “business is business” isn’t realism; it’s just ducking moral responsibility and cowardice.

“If we don’t build it, adversaries will.”

The ultimate trump card. It sounds compelling, but it’s an emotionally manipulative race to the bottom. Yes, adversaries will try to build dangerous things. But the question is how we respond: do we match their recklessness and accelerate the race, or do we establish clear boundaries?

The people leading frontier tech have massive leverage over the whole ecosystem. When the leaders at the top signal that anything goes, that permissiveness trickles down to everyone else. But when those same leaders draw hard ethical lines against things like mass surveillance or lethal autonomy, those standards propagate too. Saying “others will do it” doesn’t absolve you of responsibility—it’s exactly why we need responsible leadership right now. Standards propagate, but so does recklessness.

---

These were the major talking points I found. If you want me to talk about any other arguments or address anything else, lmk and I will add them here. Up next is our more general critique of why Silicon Valley’s embrace of algorithmic surveillance and weapons systems must be fought against.

---

Originally published here— “Algorithmic Arms Race: How Tech is Fueling Weapons Systems and Mass Surveillance.”

This Friday three tech titans — an ex-OpenAI engineer, a Meta VP, and a Palantir product chief — raised their right hands and swore into a brand-new Army Reserve “innovation detachment.” Earlier this year, the Pentagon inked a $642 million counter-drone contract with Anduril, while the Pentagon cleared the first $1 billion for autonomous drones.

These aren’t edge cases. As we covered in our AI Market Report for May, the renaming of the “AI Safety Institute” to the “Center for AI Standards and Innovation” was a clear indication that the government was taking a much more aggressive stance in investing in AI for war.

Between this and the Nvidia callout, we’re already 2/3 on the predictions.

Most concerning to me is not the government’s inclinations, however (governments do as they do), but how openly Silicon Valley is embracing this. Gone are the days of anti-establishment “hackers” (romanticized and exaggerated as it was), who would rail against Big Brother (like the talk that put Apple on the map). The new generation of Builders and Investors seems to be proactively building/funding tech-based Onii-chan.

All this creates a supply chain that compresses the gap between sensor and trigger to less than a heartbeat. Once that architecture is locked, every civilian use of AI inherits its assumptions, its surveillance scaffolding, and its hair-trigger incentives.

Before the concrete sets, we need to interrogate the logic driving this merger — steel-man the case for it, then stress-test the foundations. While I will present the arguments for the pros in a good-faith manner, I make no pretensions of objectivity. In my opinion, people proactively making this technology happen are some combination of-

1. In the Government.

2. Greedy.

3. Ignorant of history.

4. Short-sighted.

5. A bitch that doesn’t have the spine to think about (and work on) a better future.

In case you can’t tell, I absolutely hate this shift, and this piece is ideological to get you to join in my player-hating.

Image Source. While the sentence, “I love the idea of getting a drone and having light fentanyl-laced urine spraying on analysts that tried to screw us” — could be passed off as a joke, Karp has exhibited a lot of “How could you leave me, I’m such a nice guy” style vindictive energy towards analysts many times. Worth thinking about whether this is the kind of person who should be building dangerous technology.

Let’s get it.

Atleast AGI bros pretend to care for humanity.

Executive Highlights (TL;DR of the Article)

1. AI militarization isn’t neutral tech progress; it’s a fundamental rewrite of societal infrastructure. Whoever controls the code and sets the rules shapes every future built on that infrastructure.

2. Arguments for AI-driven warfare — deterrence, precision, inevitability — fail upon scrutiny. They dangerously underestimate AI fragility, accelerate algorithmic escalation, and create opaque, unaccountable power structures.

3. Systemic issues run deeper than any one bad policy:

• Vendor Capture: Narrow interests stifle innovation and lock militaries into dangerous, proprietary technologies.

• Erosion of Democracy: Secrecy, complexity, and rapid automation cripple civilian oversight.

• Exported Authoritarianism: Surveillance tools become Trojan horses for oppression, compromising sovereignty and autonomy.

• Irreversible Risks: Autonomous weapons pose existential threats impossible to roll back once normalized.

4. Practical solutions exist — imperfect, difficult, yet urgently necessary:

• International bans and strict human oversight on lethal AI.

• Redirect funding and talent from militarized AI towards civilian innovation.

• Mandate radical transparency and empowered civilian oversight for surveillance systems.

• Use investment ethics to financially disincentivize harmful AI development.

• Educate the public and policymakers on AI capabilities and risks.

5. None of this is inevitable:
Every moment of inaction, every concession to cynicism, hands the pen to interests indifferent to human dignity.

I hope the irony of how Champions of Small Government, Freedom and Innovation like Peter Thiel getting on their knees for daddy Government is not lost on you. Dystopian, but still really funny. Image Source

We can — and must — pick up the pen and write a future worth living in, no matter how messy or incomplete each step may feel.

Because even imperfect actions taken today create space for better choices tomorrow.

An additional note to investors and Silicon Valley- In economics, a defensive expenditure occurs when we spend money on something that does not increase our welfare, or is necessary to avoid a decrease in well-being. For example, spending money on health insurance does not necessarily make us better off, but it does help protect us from future costs that would negatively affect our well-being.

Militaries fall squarely into this category: they don’t create value by existing; their value is in preventing something worse. So, investing in military technology ultimately means investing in such a good—unless, of course, you’re waiting for them to plunder someone else. War aside, this means you’re investing in a constrained market, very against your ethos.

I put a lot of work into writing this newsletter. To do so, I rely on you for support. If a few more people choose to become paid subscribers, the Chocolate Milk Cult can continue to provide high-quality and accessible education and opportunities to anyone who needs it. If you think this mission is worth contributing to, please consider a premium subscription. You can do so for less than the cost of a Netflix Subscription (pay what you want here).

I provide various consulting and advisory services. If you‘d like to explore how we can work together, reach out to me through any of my socials over here or reply to this email.

2. Can AI for War and Surveillance be Ethical

Before we critique the switch, I think it’s always worthwhile to explore the reasons people advocate for these technologies (beyond the fact that they are profitable investments). This is always helpful for avoiding misunderstandings, especially in very contentious issues like this.

Below are the arguments that are often given in favor for deploying AI for Security purposes-

Deterrence in a Multipolar Knife-Fight

Great-power rivalry isn’t a speculative threat — it’s the reality again. Geopolitical competition between the US, China, Russia, and other nations has intensified, and AI-driven weapon systems are the new coin of the realm.

The Pentagon’s Replicator program, which aims to deploy thousands of autonomous drones to counterbalance adversaries at a fraction of the cost of traditional hardware, embodies this logic. The reasoning is simple —  falling behind on AI-driven autonomy is voluntary disarmament. If a rival state fields an autonomous kill-chain first, it dictates the rules and tempo of conflict, leaving late adopters dangerously reactive and vulnerable.

Falling behind constantly advancing nations can cause defensive vulnerability.

Fewer Body Bags, Faster Wins:

On the battlefield, speed and accuracy win engagements. Autonomous weapon systems don’t feel fear, fatigue, or adrenaline. They don’t hesitate, tremble, or flinch. They analyze terabytes of surveillance data in milliseconds, launching precision-guided munitions faster and more accurately than human operators.

Israel’s 2021 Gaza conflict was publicly termed “the first AI war,” precisely because machine-generated intelligence cut strike-to-target response times dramatically. At that time, “Israeli military leaders described AI as a significant force multiplier, allowing the IDF to use autonomous robotic drone swarms to gather surveillance data, identify targets, and streamline wartime logistics.” This has continued-

“At 5 a.m., [the air force] would come and bomb all the houses that we had marked,” B. said, an anonymous IDF soldier. “We took out thousands of people. We didn’t go through them one by one — we put everything into automated systems, and as soon as one of [the marked individuals] was at home, he immediately became a target. We bombed him and his house.” This is the reality of the war in Gaza. Israel employs sophisticated artificial intelligence (AI) tools to enhance its Intelligence, Surveillance and Reconnaissance (ISR), which then allows it to strike targets all over Gaza. However, this augmentation of military capabilities raises profound ethical concerns and may carry geopolitical implications. AI-assisted airstrikes, which, in some cases rely almost solely on the assessment of an algorithm, may lead to a disregard for basic rules of war, such as discrimination and proportionality.”

-Source

Proponents of such systems argue that the power of these systems ultimately leads to fewer friendly casualties, fewer collateral deaths, and quicker resolutions. This makes this not only strategically wise but ethically necessary.

Dual-Use Is Destiny — Better Our Oversight Than Theirs (aka Dario Amodei saying that AI is so dangerous, that only we should build it):

AI, by its nature, is general-purpose. Trying to separate its civilian and military applications is unrealistic. Technologies like computer vision or predictive analytics used in healthcare and logistics are easily adapted to surveillance and military operations. The same facial recognition used by companies in weapons targeting can be adapted into better bots for disaster rescue.

If democratic nations abstain from militarizing these dual-use technologies, authoritarian regimes surely won’t hesitate. The European Union’s AI Act explicitly carves out national security exceptions, implicitly acknowledging that it’s better for democracies to lead AI’s development transparently rather than cede that space entirely to adversaries who might weaponize it without oversight.

Economic Flywheel and Technological Spillover:

Military funding has historically underwritten technology that later reshaped civilian life — radar gave us microwave ovens, DARPA gave us the Internet and GPS. Advocates see military AI spending as another iteration of this pattern.

Anduril’s perception stack can easily be used for handling port security and wildfire response. In such cases, the military budgets absorb R&D risks that commercial investors won’t, ultimately accelerating innovation cycles and producing civilian benefits downstream.

Counter-Terror, Border Control, Cyber Defense:

AI-driven surveillance tools significantly improve homeland security, supporters claim, replacing brute-force policing with precision interventions. Systems that utilize wide-area facial recognition, behavioral analysis, and pattern detection enable security agencies to proactively intercept threats.

Source

“Nothing to Hide”

This is very common when talking about surveillance systems. If I, Jo Schmoe 69, have done nothing wrong, what do I care if the government is looking through my texts or seeing what I did for Lunch? Only the seedy characters who have things to hide would be concerned about AI being used in surveillance. For the rest of us, this will only improve accountability and safety (since the systems can be used to detect and track criminals very quickly).

Moral Imperative to Protect Troops & Civilians:

Lastly — and perhaps most compellingly from a humanitarian perspective — is the argument that using autonomy in warfare is ethically obligatory. If an autonomous drone swarm can end a fight in minutes or hours instead of the days/months/years associated with traditional armed conflict —  we must use it.

In the future, if both sides eventually fight with drones and robots, won’t bloody conflicts be a thing of the past? Wouldn’t wars turn into advanced, high-stakes versions of Age of Empires games?

Combine this w/ previous arguments. Ethically, the reasoning follows a clear imperative: if technology exists to reduce harm to your soldiers and civilians alike, not working towards it becomes a moral failure rather than a virtuous choice. After all, we haven’t yet figured out how to live in harmony, so does it not make sense to at least mitigate the costs when wars eventually fail.

Some of these have legs and are worth thinking about. Let’s address them in the next section — picking what’s valid and talking about what’s not.

Part 3: How “Defense Tech” can Make things worse

“But man has such a predilection for systems and abstract deductions that he is ready to distort the truth intentionally, he is ready to deny the evidence of his senses only to justify his logic. “

— Fyodor Doestevsky, Notes from the Underground. Must read for our times.

We’ve presented the strongest arguments advocates can muster for deploying AI into warfare and surveillance. They aren’t baseless — but they’re also built on deeply flawed assumptions, dangerously optimistic projections, and historical amnesia.

Deterrence, or Algorithmic Escalation?

The idea that AI superiority guarantees stability is a dangerous remix of the logic behind mutually assured destruction — only now the “assurance” relies on algorithms nobody fully understands, operating at speeds that make human judgment irrelevant. Such systems can just as easily lead to-

Machine-Speed Wars: Deterrence hinges on rational actors and deliberation. But automated kill chains close the gap between sensor and trigger to milliseconds, leaving no time for humans to intervene. An algorithmic glitch, a spoofed sensor input, or a hostile hack can ignite an unintended “flash war” faster than any leader could react (flash crashes in the stock market are already a thing).

This is made (much) worse when we consider how brittle AI is. Both the aforementioned flash crashes, AI customer service, and the various issues in AI content moderation on social media platforms are worth considering. In all cases, we’ve had decades to refine things, and a massive financial incentive to do so, but failed b/c AI is really hard to control.

Even Language Models, with Billions in Investments and the best minds, struggle with basic things that can occur in every day, such as changing inputs-

“This paper investigates the extent of order sensitivity in LLMs whose internal components are hidden from users (such as closed-source models or those accessed via API calls). We conduct experiments across multiple tasks, including paraphrasing, relevance judgment, and multiple-choice questions. Our results show that input order significantly affects performance across tasks, with shuffled inputs leading to measurable declines in output accuracy. Few-shot prompting demonstrates mixed effectiveness and offers partial mitigation; however, fails to fully resolve the problem. These findings highlight persistent risks, particularly in high-stakes applications, and point to the need for more robust LLMs or improved input-handling techniques in future development.”

-The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMs

This phenomenon has been explored in the excellent post: “LLM Judges Are Unreliable”, which has also open-sourced their eval suite for you to run/verify their experiments yourself.

“A screenshot of our ‘A/B picking experiment result’, showing biased first-slot preferences between Gemini and OpenAI models across variants of prompt style”

These fragilities can be overlooked in most cases since we have either (or both)

1. Higher margins for error.

2. More time for audits/confirming outputs.

First is out, given the nature of the space, but having millisecond-level margins also compresses the time for thinking about high-pressure situations.

Swarm Instability & Proliferation: Cheap drone swarms — they lower its barriers. If a state believes it can swiftly deploy disposable AI drones without political fallout, the temptation to strike first skyrockets. Historically, when you make powerful weapons easier for the “good guys,” you inevitably open that same door wider for the “bad guys.” Today’s defensive swarm becomes tomorrow’s terrorist arsenal.

Ideas such as induced demand-

An illustration of Induced Demand.

and Jevons' Paradox are worth studying here. If economics term scare you — think about your bills probably went up after getting Uber One or Amazon Prime, since the delivery add-ons (a huge detterent) were taken out. The very same thing will happen with weapons, with availability/low cost pushing up demand.

Opaque Arms Race & The Black Box Problem: AI’s opacity fuels mistrust. Without transparency, there is an opportunity cost to not defaulting to worst-case assumptions, accelerating the arms race, and constant paranoia. The cost of not doing so, is “death”, which makes all actions justifable (same argument used by people to oppress in the name of building utopias). This constant tension will make them trigger-fingers much more twitchy.

Building a system of algorithmic deterrence and flash-weapons systems also creates many questions that we haven’t even started to answer. Who’s accountable when opaque systems inevitably err? The coder? The commander? The black-box AI itself? Ultimately, until we can answer these questions well, this isn’t strategic deterrence; it’s gambling global security on algorithms nobody fully controls.

A more thorogh flowchart showing the same princple of induced demand in weapons.

Operational Efficiency? Meet the Precision Myth

Advocates sell AI-enabled warfare as precise, efficient, and clean. That myth unravels quickly upon scrutiny.

The Moral Hazard of Surgical Strikes: The IDF soldier’s chilling description of AI-facilitated strikes — “We took out thousands of people…everything into automated systems” — illustrates a deeper truth. “Precision” can become shorthand for automated, indiscriminate violence. If violence becomes politically easy and psychologically distant, leaders won’t hesitate — they’ll pull triggers faster and more often. War doesn’t become cleaner; it becomes routine. People have justified many atrocities, such as the Holocaust, by becoming very distant and “just doing their jobs”.

Algorithmic Bias as Discrimination: AI learns from data — and data inherits human prejudices. Facial recognition systems consistently misidentify minorities. Put these biases into autonomous targeting algorithms, and you industrialize prejudice. Innocent civilians become collateral, labeled combatants by biased code, and killed efficiently at scale. The power imbalance in such cases can often make finding justice much harder.

Novel Vulnerabilities & Amplified Harm: AI warfare introduces entirely new vulnerabilities. Systems become prime targets for cyberattacks and adversarial manipulation, potentially turning your own arsenal against you. This is bad, on it’s own. This gets worse when we consider the history of technology and the capabilities it adds. Technology scales our capacities, dramatically increasing individual ability. A single mass-shooter or Suicide Bomber can kill more people than most accomplished warriors in history. Following the same trend, hacking the next generation of technology could cause devastation far exceeding traditional terrorism or mass shootings.

Building guardrails here will be expensive, difficult, and further sink resources into maintaining these setups instead of using the capital in other avenues (more on this soon).

Dual-Use Isn’t Destiny — It’s an Excuse

Yes, AI technology is inherently dual-use. So was nuclear physics, chemical engineering, cryptography — yet responsible states didn’t shrug off their responsibilities; they painstakingly crafted treaties, inspections, and norms (even when flawed).

Saying “if we don’t build it, adversaries will” is fatalistic laziness masquerading as pragmatism. Real leadership means establishing international norms and red lines, not racing to the bottom. Claiming “responsible leadership” while accelerating AI militarization without guardrails isn’t just contradictory — it’s dangerously naive.

Additionally, the assumption that democratic states will deploy AI responsibly is hubristic. History suggests that increasing power imbalances lead to increasing injustice. To think that our modern crop of leaders has transcended the bloodthirst and greed that have plagued the rest of humanity seems a tad optimistic.

When all is said, most people, even builders, ultimately land on the oppressed side of an unchecked surveillance apparatus. No one plans to live in dictaroship — but unchecked, we inevitably build one.

Economic Flywheel or Drain?

Sure, military investments historically sparked civilian breakthroughs. But that argument isn’t unique to defense — it’s true for nearly every major scientific investment.

Opportunity Costs: Every billion dollars spent on perfecting drone swarms isn’t going to climate mitigation, disease eradication, or education. Society trades urgently needed civilian progress for a battlefield advantage.

Talent Trap: Militarized AI repels precisely the talent society most desperately needs — those ethically opposed to building kill chains. AI’s brightest minds will leave rather than enable mass surveillance or war. The real cost is the drain of potential solutions for our most pressing global problems.

Talent Entrenchment: The normalization of these technologies as strong career paths will influence more and more bright minds into these paths, taking away future great minds from working on big problems. Take the quote- “The best minds of my generation are thinking about how to make people click ads,” and imagine the outcome of replacing ads with weapons.

Homeland Security: The “Nothing to Hide” Fallacy & Surveillance Creep

The promise of AI security is seductive, but it comes at profound, hidden costs.

Scope Creep Isn’t Speculative: Border surveillance inevitably migrates inward — streets, schools, homes. History confirms surveillance infrastructures expand relentlessly, normalizing pervasive monitoring under the guise of safety. The “nothing to hide” argument assumes an infallible state, benevolent operators, and transparent algorithms. Reality provides none of those. Even projects w/ no hidden agendas/incentives to explicitly avoid creep, are hit with unexpected costs. You really expect technology deployment that can be very enticing to power-hungry folk to stay pure?

Ignoring Root Causes: Surveillance as primary crime prevention ignores deeper societal drivers. True security emerges from giving people stable, dignified lives — not abstract surveillance threats. Economic opportunity, education, and healthcare consistently do more for security than cameras and algorithms-

“Consider this: According to recent Brookings research, it was the loss of jobs and educational opportunities for people living in high-poverty neighborhoods that primarily explains the rise in homicides during the COVID-19 pandemic — not changes in policing or criminal justice system practices. Further, a large body of evidence finds that approaches linking public safety efforts to those bolstering employment, education, and quality neighborhoods can measurably reduce and prevent violent crime, while also saving taxpayers and governments significant costs.”

-Source. When you’re already struggling, the threat of jail can be abstract while the upside of crime is more promising. If you have a house, job, and food to lose, it suddenly becomes much more concrete while diminishing the upside.

The Ethical Warfare Fantasy

The argument that AI-driven war is somehow more ethical, reducing human harm, is perhaps the most dangerously naive.

Sanitized Conflict Illusion: War isn’t sanitized by automation — it’s merely distanced. Killing remotely doesn’t eliminate violence; it abstracts it, lowers political costs, and increases the likelihood of use. War doesn’t become a strategic video game — it becomes easier, more frequent, and more casually lethal.

Whose Lives Count?: Advocates argue AI reduces harm — but whose harm? AI protects your soldiers, sure, but does it discriminate against innocents caught in biased algorithms? Ethical warfare often conveniently aligns with national self-interest, not humanitarian principles.

Fragile Times, Dangerous Tech: With diminished attention spans, increased polarization, and diminished nuanced discourse already rampant, handing lethal decisions to opaque, complex AI is reckless. Builders often fail to grasp these systems’ societal impacts — witness our persistent content moderation failures. Amplifying these failures into the lethal domain risks catastrophes society isn’t ready for.

Underlying all of this is a fundamental misunderstanding:

Most proponents imagine themselves and their societies as the controllers, not the controlled. Others believe themselves to have a superior intelligence or moral virtue, and thus have no qualms about tipping the balance of power and designating themselves as the arbiters of human taste. Listen to any Peter Thiel, Marc Andreesen, or Dario Amodei interview, and you’ll see how quickly their statements carry an underlying sense of “we are better and should be in-charge, you just fall in line b/c we know what’s good for you”.

“I simply hinted that an ‘extraordinary’ man has the right… that is not an official right, but an inner right to decide in his own conscience to overstep… certain obstacles, and only in case it is essential for the practical fulfilment of his idea (sometimes, perhaps, of benefit to the whole of humanity). You say that my article isn’t definite; I am ready to make it as clear as I can. Perhaps I am right in thinking you want me to; very well. I maintain that if the discoveries of Kepler and Newton could not have been made known except by sacrificing the lives of one, a dozen, a hundred, or more men, Newton would have had the right, would indeed have been in duty bound… to eliminate the dozen or the hundred men for the sake of making his discoveries known to the whole of humanity.”

-Crime and Punishment

However, this isn’t the only kind of problem that we must be concerned about. There are deeper, underlying problems that are often overlooked b/c of their chilling effects-

Part 4: Additional Systemic Threats

Let’s be clear: the deepest threats posed by militarized AI would remain even if their utopian visions of clean, efficient algorithmic warfare weren’t complete fantasy. The problem isn’t just their flawed arguments — it’s the structural shifts inherent in combining AI with state and military power. These systemic threats run deeper, and their consequences more lasting, than the immediate debate suggests.

1. Path Dependency & Vendor Capture: The Quiet Chokehold

Once militaries rely heavily on proprietary AI from a handful of specialized firms — your Andurils, Palantirs, Lockheeds — dangerous path dependencies set in. Military doctrines, procurement strategies, even tactical thinking increasingly revolve around the limits and capabilities of these vendor platforms. Open innovation? Smaller firms? Open-source alternatives? Squeezed out, sidelined, or ignored. Those who write the “API specs” shape the battlefield, effectively capturing entire national security ecosystems.

Additionally, when Silicon Valley executives enlist in the Army Reserve tech detachments, they’re not just bringing technical expertise — they’re building lasting networks of influence. Civilian oversight becomes meaningless when “civilian” AI developers are ideologically and financially aligned with military objectives. This isn’t oversight; it’s vendor capture in camouflage.

The revolving door visualized

2. Erosion of Democratic Accountability: Who Watches the Algorithms?

Advanced AI systems, cloaked in national-security secrecy, make meaningful democratic oversight impossible. How do civilian leaders or citizens hold power to account when lethal decisions emerge from opaque algorithms?

“The algorithm decided” will become the ultimate deflection of accountability. Commanders, policymakers, and corporations shield themselves behind complexity. Or use the cloak of national safety to hide any issues. And legitimate classification quickly becomes a shield for hiding costs, ethical breaches, errors, or mission creep. This creates democratic black holes, where critical decisions vanish from public scrutiny entirely.

3. Exporting Authoritarianism: AI as a Trojan Horse for Oppression

Most nations can’t independently build sophisticated AI surveillance or autonomous weapons systems. Instead, they’ll import these capabilities wholesale from major powers. But these “off-the-shelf” solutions come with embedded political philosophies and strategic dependencies.

Buying China’s surveillance tech or Western drone swarms doesn’t just provide technical capability — it imports a political logic of centralized control and top-down surveillance. Nations become algorithmic vassals, dependent on supplier nations for updates, maintenance, and strategic guidance. Statistically speaking, cultural preservation has never really thrived when one set of peoples become dominant over another.

4. Irreversible Global Catastrophe: More Than Just Another Weapon

Certain AI applications — particularly fully autonomous weapons systems or strategic infrastructure control — introduce existential-level risks. These aren’t incremental threats — they’re civilization-scale gambles.

Conways Game of Life shows us simple rules/agents can create complex/unexpected patterns — dubbed as emergence. Imagine what systems of complex agents can do.

Once deeply embedded in military and strategic infrastructure, these capabilities become impossible to roll back, even when recognized as fundamentally dangerous. We risk permanently locking the world into an unstable equilibrium, perpetually on the brink of algorithmic catastrophe.

These systemic threats aren’t mere side effects; they’re built into the architecture of militarized AI. And left w/o oversight, they will

This isn’t inevitable. The choices we make now — regarding oversight, norms, treaty obligations, and funding priorities — determine whether AI enhances human dignity and safety, or erodes them.Let’s talk about somethings we can push for right now.

Source. “Dozens of those stakeholders have ties to both the PLA and the PRC defense industrial complex, as well as ties to the U.S. and its allies.” is worth a double and triple take.

Part 5: Pulling the Emergency Brake

It’s easy to critique, but it’s empty without concrete, strategic alternatives. This isn’t a moment for vague recommendations; it’s a call for precision. We need explicit, actionable guardrails, international red lines, and practical off-ramps before this becomes an irreversible spiral.Here are five clear moves we should make immediately:

1. Forge Binding International Norms — Before the Window Closes

Allowing military AI to “evolve naturally” is dangerously naive. We urgently need enforceable global standards.

Meaningful Human Control (MHC): This isn’t a vague aspiration; it’s a verifiable legal requirement. Lethal AI systems must always keep humans meaningfully informed, empowered, and capable of intervention, not just rubber-stamping outputs from a “ghost in the machine.”

Outright Ban on Fully Autonomous Weapons (LAWS): Some technologies don’t get a pass. Autonomous systems that independently select and engage targets represent moral catastrophe and existential threat rolled into one. Treat them like chemical weapons: prohibited, stigmatized, actively dismantled. Verification is challenging? Tough. So was nuclear disarmament.

Radical Transparency & Incident Accountability: Nations must publicly disclose significant military AI projects. Establish independent international bodies empowered to investigate incidents involving AI, ensuring “the algorithm did it” never excuses war crimes or lethal mistakes.

2. Starve the Beast, Feed Civilian Innovation — Financial & Talent Firewalls

Money talks loudly — right now, it says “build smarter ways to surveil and kill.” We need to change the incentives there.

Public Funding with Ethical Teeth: Tie government grants explicitly to civilian-focused AI applications (climate, public health, education), and impose strict non-military clauses. Fund solutions to human problems, not enhancements to military arsenals.

Conscience Clauses & Talent Protection: Legally protect engineers’ right to refuse ethically objectionable projects without retaliation. Elevate those working on life-affirming AI solutions over participants in the kill-chain or the surveillance economy. Make ethical integrity a career asset, not a liability.

Tax the Toxic, Reward the Beneficial: Implement fiscal policies penalizing investments in autonomous weapons and oppressive surveillance tech. Conversely, offer significant incentives — tax breaks, grants, and subsidies — for AI advancing the public good. Building a surveillance state shouldn’t be lucrative.

3. Mandate Public Accountability for Surveillance AI

If governments use AI to watch citizens, citizens must hold an uncompromising mirror to the watchers.

Mandatory Algorithmic Transparency: To a certain degree, some of these systems are inevitable. Some, like missile-defense systems, would even be a net good. However, we must take steps to ensure that all algorithmic systems publicly disclose capabilities, data inputs, usage plans, error rates, bias audits, and accountability measures (some secrecy would be required for sensitive systems, but we still need private audits there). End secretive algorithmic governance — citizens have an absolute right to know how their lives and freedoms are monitored and shaped.

Empowered Civilian Oversight Bodies: Establish genuinely independent oversight boards with technical expertise, investigative authority, and teeth to halt biased, harmful, or ineffective AI deployments. Advisory committees won’t cut it; these bodies need real power — investigate, audit, sanction.

4. Weaponize Capital & Corporate Responsibility

Capital isn’t neutral, but it can be pressured into ethical alignment.

ESG Criteria for AI Ethics: Explicitly classify investment in lethal autonomous weapons, oppressive surveillance, and demonstrably biased AI as ESG-negative. Pressure pension funds, sovereign wealth funds, and major investors to divest, raising capital costs for unethical AI ventures.

Internal Activism & Shareholder Power: Foster cultures of accountability within tech companies and defense contractors through active shareholder and employee activism. Demand transparency, ethical standards, and accountability at shareholder meetings and internal forums. Make unethical AI a business liability, not an investment thesis.

5. Fight Algorithmic Illiteracy — Educate Public & Policymakers

A public and policy elite ignorant of AI’s realities and risks can’t regulate it effectively.

Critical AI Literacy for Leaders: Mandate comprehensive AI ethics and capabilities training for lawmakers, judges, civil servants, and security agencies. Equip decision-makers to grasp AI’s societal and ethical dimensions — not just vague hype.

Demystify, Don’t Deify: Counter AI hype cycles with sober, evidence-based public education. Foster skepticism, nuance, and informed questioning over blind faith in technological solutions or dystopian inevitability. People deserve clarity, not confusion or techno-messianism.

None of This Is Easy — Do It Anyway

These won’t work easily. People will find ways to skirt regulations, lie, find loopholes etc. We can’t make a perfect system. We can add more friction, however. By adding more and more friction, we create space in these systems. A space where whistleblowers, future talent and other ethtically minded people can fall through, take a space to re-examine whether they want to be involved with what they’re building. A space that makes sweeping violations under the carpet much harder.

It’s about building a scaffold, on which we can build the next steps. It can be easy to forget, but many of our most important systems were not created overnight. They took years (sometimes centuries) of testing, iteration, and rebuilding. This will be similar. Getting frustrated and not engaging because of childish reasons like “the bad guys always win” or “this is just how things are” is complicity.

Part 6: The Choice — Who’s Holding the Pen?

“Everything that you thought had meaning: every hope, dream, or moment of happiness. None of it matters as you lie bleeding out on the battlefield. None of it changes what a speeding rock does to a body, we all die. But does that mean our lives are meaningless? Does that mean that there was no point in our being born? Would you say that of our slain comrades? What about their lives? Were they meaningless?… They were not! Their memory serves as an example to us all! The courageous fallen! The anguished fallen! Their lives have meaning because we the living refuse to forget them! And as we ride to certain death, we trust our successors to do the same for us! Because my soldiers do not buckle or yield when faced with the cruelty of this world! My soldiers push forward! My soldiers scream out! My soldiers RAAAAAGE!”

-Attack on Titan

AI isn’t destiny; it’s infrastructure. Like any infrastructure, its design decides what gets built on top. Every algorithm released, every sensor switched on, every dollar routed into autonomous weapons doesn’t just enable new capability — it quietly rewrites the rules we’ll all live under.

Right now, that pen sits in the hands of defense contractors hunting their next tranche of revenue, bureaucracies chasing a mirage of machine-speed dominance, and black-box code that answers to no electorate. If we stay passive, our future gets finalized in microseconds — human judgment downgraded to a latency bug.

But the ink isn’t dry.

Lines will be drawn either way. The only live question is whether we pick up the pen, draw them ourselves, and keep redrawing when the world shifts — as it always does.

Perfect solutions don’t exist; decisive steps do. Fight for the audits. Demand the clauses. Fund the alternatives. Teach the next cohort to read the fine print in the source. Each move is messy, partial, limited, and still miles better than surrendering the script.

Sometimes victory isn’t total or triumphant. Sometimes it requires that we live to fight another day, to ensure that we leave the foundations on for someone else to build on. If I can, I would like to end by quoting one of my favorite novels ever-

“Dr. Rieux resolved to compile this chronicle, so that he should not be one of those who hold their peace but should bear witness in favour of those plague-stricken people; so that some memorial of the injustice and outrage done them might endure; and to state quite simply what we learn in a time of pestilence : that there are more things to admire in men than to despise. None the less, he knew that the tale he had to tell could not be one of a final victory. It could be only the record of what had had to be done, and what assuredly would have to be done again in the never-ending fight against terror and its relentless onslaughts, despite their personal afflictions, by all who, while unable to be saints but refusing to bow down to pestilences, strive their utmost to be healers. And, indeed, as he listened to the cries of joy rising from the town, Rieux remembered that such joy is always imperilled. He knew what those jubilant crowds did not know but could have learned from books : that the plague bacillus never dies or disappears for good; that it can lie dormant for years and years in furniture and linen-chests; that it bides its time in bedrooms, cellars, trunks, and bookshelves; and that perhaps the day would come when, for the bane and the enlightening of men, it roused up its rats again and sent them forth to die in a happy city.”

-The Plague, Albert Camus.

Thank you for being here, and I hope you have a wonderful day.

Still optimistic

Dev <3

If you liked this article and wish to share it, please refer to the following guidelines.

That is it for this piece. I appreciate your time. As always, if you’re interested in working with me or checking out my other work, my links will be at the end of this email/post. And if you found value in this write-up, I would appreciate you sharing it with more people. It is word-of-mouth referrals like yours that help me grow. The best way to share testimonials is to share articles and tag me in your post so I can see/share it.

Reach out to me

Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.

Small Snippets about Tech, AI and Machine Learning over here

AI Newsletter- https://artificialintelligencemadesimple.substack.com/

My grandma’s favorite Tech Newsletter- https://codinginterviewsmadesimple.substack.com/

My (imaginary) sister’s favorite MLOps Podcast-

Check out my other articles on Medium. : https://rb.gy/zn1aiu

My YouTube: https://rb.gy/88iwdd

Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y

My Instagram: https://rb.gy/gmvuy9

My Twitter: https://twitter.com/Machine01776819