Warrantless Surveillance and the End of the Expectation of Privacy

How Apple and Meta Are Killing What Expectation of Privacy We Have Left

Last fall I gave a talk at OWASP Global on the hidden risks of AI, and afterward someone came up to ask questions. We talked for a few minutes before I noticed the glasses. I asked if they were filming. They said yes, they had been the whole time, and that they used it to remember things from the conference.

I didn’t say anything that I wouldn’t say publicly, but that isn’t the point. The point is that a stranger recorded a conversation with me, gave me no indication they were doing it, and only admitted it when I asked. This left me deeply disturbed.

Now Apple wants to put that same capability on your wrist. And the courts have already started reasoning that the more of these devices exist, the less privacy any of us can reasonably expect.

In the U.S., we don’t have a lot of privacy protections. The best we have is in the healthcare realm where HIPAA ensures that doctors and hospitals are responsible about who they disclose your health data to. This doesn’t restrict them from sharing your data with service providers as long as those service providers also enforce the same downstream disclosure rules. These days, that includes AI companies who get your data and can use it however they wish as long as they aren’t publishing it or sharing it to individuals without your permission.

The other major protection in the US is enshrined into the 4th amendment of the constitution:

The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.

The third-party doctrine

I’ve written before about how much the 4th Amendment has been eroded over time, particularly with the 3rd-party Doctrine, which stemmed first from a 1967 supreme court case. That case is where they redefined the 4th Amendment to be constrained to only cover places where a person has a “reasonable expectation of privacy.” Subsequently, Miller (bank records) and Smith v. Maryland (phone records) established that you forfeit your 4th amendment protections when you “voluntarily” give information to a third party. Never mind that you have absolutely no choice when it comes to bank and phone services.

Though a third party can refuse a law enforcement request without a warrant, they don’t have to. AT&T, for example, famously sells any data to law enforcement with no warrant required under their Hemisphere project, which continues to this day under a new name.

It’s not all bleak, though. In 2018, the Supreme Court ruled in Carpenter that cell-site location data is protected by the fourth amendment. No doubt AT&T’s profits took a bit of a hit from that. And this past June, the Court held in Chatrie that you have a reasonable expectation of privacy in your Google Location History, too. All this means that law enforcement now needs a judge’s approval before they can know who happened to be near a crime scene. Which seems like it should always have been the case, but wasn’t.

The erosion of the 4th Amendment

Your data, when stored with service providers (and here I’m not just talking about Big Tech, but banks, telecom, credit reporting agencies, and much more), has no expectation of privacy. But what about your conversations?

Historically, if the government wanted to listen in to a private conversation, they’d need a warrant to use microphones. But where can you expect privacy? In a 2016 court case in California, the FBI warrantlessly bugged the area around a courthouse, indiscriminately eavesdropping on all conversations, including those between attorneys and their clients. Then they used that information in court. The judge denied a motion to suppress the conversations as follows:

Defendants contend that they had a reasonable expectation of privacy in their communications outside the courthouses, citing cases recognizing a privacy right in communications made in a public place. None of the cases are directly on point, to hold that one has a reasonable expectation of privacy in communications at or near a courthouse entrance. … it is equally unrealistic for anyone to believe that open public behavior including conversations can be private given that there are video cameras on many street corners, storefronts and front porches, and in the hand of nearly every person who owns a smartphone. … Accordingly, the court finds that the warrantless recording of defendants’ conversations did not violate their rights under the Fourth Amendment or under the wiretap statute.

This was ten years ago now and it wasn’t appealed.

To be fair, days later a different judge in the same district looked at the same FBI program at a different courthouse and ruled the other way. Judge Breyer found that those defendants did have a reasonable expectation of privacy, precisely because they spoke in hushed tones, stopped talking when strangers approached, and kept the conversation among themselves. The first judge, Judge Hamilton, ruled the other way because, in part, the defendants’ spoke “in conversational or loud tones, and not hushed or whispered voices.” The law is not settled in this area, but in practice, it is important to show that you expect or even strive for privacy.

Maybe I should make t-shirts that say “this conversation is private” on them? Nah, they’d probably need little speakers that say that out loud every minute.

The microphones in the Breyer case recorded for more than 200 hours and they caught women discussing salacious personal relationship details, a judge and a prosecutor rating the attorneys who appeared before them, and much more. But though all those people had their privacy invaded via warrantless wide-net surveillance, none of them have standing to protest it. They either had to be named in a warrant (and there wasn’t one) or charged with a crime before they could fight back. You only have standing when you can move to suppress evidence taken from you.

The rise of surreptitious recording

Now we’re in a position where you may not have a reasonable expectation of privacy simply because smartphones are prevalent and because they could be used to record audio or video.

Fast-forward to today. Meta’s AI Glasses are turning up all over the place, including in bathrooms. I wonder if the judge would think there’s an expectation of privacy in a public bathroom? Even private footage of private acts gets reviewed by humans at Meta. There’s a class action lawsuit now over this, but only the owners of the smart glasses are a party to it. What about the victims being filmed? Nope. No standing.

DEF CON and B-Sides Las Vegas in August banned surreptitious recording and didn’t allow “pervert glasses” (the apt term coined by Eva Galperin) because of how they get used. I wish more places would adopt this policy. The person filming me at OWASP wasn’t breaking any rule, because there wasn’t one to break.

And now Apple. Oh Apple, et tu?

The new Apple watches have a feature for continuously capturing audio, transcribing it, then using AI to summarize it as a memory jog. They aren’t storing the audio recording, but are storing transcriptions and summaries.

It’s constant, 24/7 monitoring of everything you do and say.

Of course, if you own an Apple watch, you don’t have to enable that feature. But what if someone you work with or live with enables it?

Apple’s taken the position that they will protect the privacy of device owners by encrypting that data and making the AI workflows private. This is good, of course, because they can’t spy on their users and listen to their recordings. But they’re making their watches into a weapon of mass surreptitious recording and their privacy posture doesn’t take into account the people around the person doing the recording. It’s Meta glasses all over again, but minus the video. Do I now need to ask people who come up to me if their watch is recording?

Apple watches are basically ubiquitous (Tim Cook said it’s the best selling watch in the world and in 2019 it outsold the entire Swiss watch industry combined). I wear one. But I would never use it to secretly record conversations.

Unfortunately, I expect I’m part of an ethical minority.

The next time the Feds choose to record people without a warrant, can they just point to Meta glasses and Apple watches and say, regardless of location, there can be no expectation of privacy anymore? I’m not really asking. That’s a rhetorical question. Of course they’ll say that.

The normalization of constant surveillance

In the Breyer case, the defendants lowered their voices, stopped talking when strangers approached, and kept the conversation among themselves. Taking visible steps to protect a conversation is the thing courts still credit, and it’s the most reliable way left to prove you expected privacy at all.

An ambient AI microphone defeats the one privacy measure the law still recognizes.

Now think about the impact of an always-on microphone to that expectation: lowering your voice when the microphone is already in your personal space does nothing to stop the recording. Can you reasonably expect your conversation won’t be recorded if one of the participants wears a smart watch? These devices attack the only privacy measure that has worked in courts, regardless of your actions to keep the conversatoin from others.

On the bright side, at least one Supreme Court Justice thinks the underlying test is broken. In his Chatrie concurrence this past June, Justice Gorsuch argued the Court should abandon the “reasonable expectation of privacy” standard in favor of a property-based approach. Judges would just have to know whose data it is to determine if it’s protected. When the standard depends on normalized expectations, every new gadget or feature can lower the bar.

The normalization of privacy invasions has been happening for years, especially in tech. Now, with AI, the problem is getting worse. The only way to have an expectation of privacy may be to ban devices from places, or like BSides’ Skytalks, to force people to show their devices have been turned off before they’re allowed to enter a private space.

This is about to be your problem too

If you run a company, this can be an operational problem. You sign NDAs or make promises about the privacy of your customers’ data, but your employees are walking into deal reviews, customer sites, incident calls, and clinics wearing devices that record everything they see or hear. But I bet you didn’t sign a BAA for your employee’s watches, and I bet you didn’t negotiate a DPA for an employee’s “smart” glasses.

Apple does a reasonably good job of protecting the privacy of the data flowing to its LLMs, but they’re alone in this. As more copycats build devices that are always recording, you can bet that the audio and video captured by these devices will flow through various frontier LLM companies. If you’re using Meta’s glasses, all that data certainly flows to Meta servers and employees.

Once that happens, the data is out of your control. You can’t even know what potentially problematic bits various service providers are holding. The big question then is: if one of these companies holding video or audio recordings or transcripts (or even logs, as will be the case for frontier AI companies) is hacked, are you then culpable?

I think your employee recording a conversation without consent and sending it to a company with whom you don’t have any sort of agreement probably means you are.

We can’t help you or society with the problems created by pervert glasses and eavesdropping watches. Those problems require policy changes since a technical solution doesn’t exist. But if you’re building an app that holds sensitive data, we can help you secure it, build trust with your customers, meet data sovereignty laws, keep AI workflows private, and more. Take a look at our products and reach out if we can help.