AI Vendors are Silently Installing Things For You.

Alexander Hanff, "the privacy guy," has been publishing a number of pieces on ... well, questionable behavior by some AI vendors. The latest one is Google: it silently installs a 4GB AI model as part of Chrome, without consent or notification, on any device that meets a hardware eligibility check. Prior to that, it's Claude Desktop registering native messaging bridges in Chromium-based browsers on every Mac it touches, including ones that Anthropic says it does not support.

Mr. Hanff does a good job documenting the claims; he shows what the assertion is, then shows how he verified it, with commands. You can run the commands yourself; I did, and I was particularly concerned because I use Claude Desktop myself.

This is an unflattering set of incidents for everyone involved: the companies, of course, but most significantly, to users like you and me.

Both of these installations are bad, but they're bad in different ways and they're bad in a way that matters for anyone building software on top of LLMs - and they're bad in a way that matters for anyone who uses software built on top of LLMs.

The Anthropic install is ... legible. It's a Native Messaging bridge. The protocol is public, the helper binary's argument surface is bounded, the three pre-authorized extension IDs are listed in the manifest. You can reason about the capability set. The answer to "what is this doing on my machine?" is "Claude wants to drive my browser when one of three named extensions calls it." It's possible it stretches beyond that kind of usage, but at least there's a primary derivative reason on the surface of the feature. It's icky but one can understand... if the assumption is that Claude is well-behaved and doesn't abuse the capability¹.

The Chrome install doesn't have a similar assumption available. A 4GB LLM weights file has no defined capability surface. It has a model surface, which is whatever the calling code chooses to ask of it, plus whatever new things the calling code chooses to ask of it after the next Chrome auto-update. What Nano does for you is determined by Chrome's runtime decisions, not by anything in the file you can audit. Hanff observes that the visible "AI Mode" pill in the Chrome 147 omnibox doesn't currently invoke the local model at all, which means today's invocation path is one of many possible invocation paths, and the others can be lit up later without anyone re-asking².

There is also a passive-use aspect. On-device scam detection, one of the things Google has marketed Nano as powering, means the model is running over content you view regardless of whether you have invoked an AI feature. It is not a tool you can choose to use. It is a classifier whose feedback loop is whatever Google chooses to make it, and whose data goes where Google chooses to send it.

The security practitioner reading of "what is this thing actually doing?" is that nobody can fully answer, including Google, really. The capability boundary of an embedded LLM is not characterizable the way an remote API call is. When you ship the model, you ship as a side effect the slate of capabilities any future runtime update can put on top of it without re-prompting the user. Anthropic installed a bridge without permission or acknowledgement. Google installed an open possibility space.

Both installs violate the same baseline norm: don't write things to user machines the user didn't ask for, don't reach across vendor trust boundaries, don't auto-restore after the user manually deletes the artifact. But the AI overreach is harder to bound than Claude's overreach. A stealth install of setup.exe exposes a bounded threat model; you could always just ... not run it. And at least the OS has permissions to say "are you sure you want to grant this .exe capability to write to your disk?"

A stealth install of weights.bin exposes whatever the vendor decides to wire it to, when they decide to do so, indefinitely. If your browser can do things, then so can the model... and the browser can send data to someone else, data that you might want concealed - or data you need concealed.

If you are shipping software that includes a local model, the design lesson writes itself: the features you ship today might not be the features your users have agreed to. A generalizable LLM is an unbounded set of features whose contents are determined by the next decision your team makes and the deployment after that. That is not a posture you can hand-wave with "we would never use it for X." (That's what Google used to say, right?) Hanff has now twice documented a vendor doing X, in writing, with code-signed and notarized binaries, six months after they would have asserted that they'd never.

People have been worrying about stealth installs for decades, and the privacy-paranoid have been correct often enough that the burden of proof has flipped. What is new with AI is that the silently-installed thing doesn't typically have an enumerable scope. Even if it's built with very tight constraints on the model, there's little preventing an update with wider scope... and nobody would be any wiser.

Hopefully both Anthropic and Google will consider the behaviors Hanff pointed out - if they fix it, they'll say how seriously they take responsible AI, you can trust them, here's evidence that they're doing the Right Thing. But it's not just doing the right thing that matters - it's the fact that it's so easy for them to trivially do the wrong thing. How does that get changed?