Posted on Oct 30, 2015 by Rick Falkvinge

So Google Records All The Microphone Audio All The Time, After All?

It seems Google does record audio from microphones all the time, despite attempts to play down the situation. The “hotword” searching – when you initiate a search by saying “Ok Google” – has been criticized before, when it was downloaded to open-source browsers running Chromium. However, major privacy concerns remain as Google doesn’t start recording when you say “Ok Google”; it was recording before you said the hotword.

Back when Google drive-by-downloaded a black box of recording code onto the Chromium open-source browser, there was a general outrage about Google code listening in to your room. The objections mainly fell into one of two categories:

  • Google is downloading a black box of code to open-source and free-software systems without permission, compromising them.
  • Google is using proprietary code to listen in to your room.

Of these, the first was considered the most grave by far, as any proprietary running code will have access to the device’s sensors – including microphone – almost by definition, and therefore it wasn’t really considered strange that proprietary code had access to black-box recording. However, it was still a privacy issue and a concern that Google had the capability to listen in to any and every room where there was a Chrome or an Android running, which – frankly – is a rather large portion of the available rooms on the planet.

Nevertheless, Google kind of roundabout apologized for having drive-by-downloaded black-box proprietary code to the open-source Chromium browser, and people attempting to shoot down the story (there are always those, and usually with the top-voted comment…) insisted that Google didn’t open the audio recording until after you had said the magic words “Ok Google”, something that is termed a hotword to begin recording, analyzing, and transcribing.

As part of Google’s transparency initiative, you can see (some of?) the data Google has stored about you. It turns out that all audio searches are saved, permanently, and you can listen to your own previous voice commands and voice searches. They’re listed in chronological order.

A screenshot from my Google Audio History. I can listen to the recordings as well as read the transcriptions. If you've been using voice search, you have a page like this, too.
A screenshot from my Google Audio History. I can listen to the recordings as well as read the transcriptions. If you’ve been using voice search, you have a page like this, too.

You’ll recall from the previous privacy discussion, that Google having access to microphones in pretty much every room gives Google the ability to eavesdrop on those each and every rooms continuously. It should be pointed out that this is a technical ability; there’s no indication this is happening, but the presence of the capability is cause for serious privacy concerns.

In that discussion, people said – no, insisted and shouted – that the fear was overblown, tinfoilhattery, and mental. “Obviously, Google only records audio after you say Ok, Google“, pundits said from all directions. But when listening to this recorded audio search off of Google’s audio search history page, something peculiar sticks out, that nobody seems to have noticed. Listen to this:

 

Did you hear it? The recording starts with me saying “Ok, Google”. That means that the above assertion – that recording starts after those words are said – is incorrect. Recording happened before those words were said. And if recording happened before those words were said, which must be the case since they’re part of the very recording, then audio recording from the microphone(s) is always active to some unknown degree. We see what Google claims is sent to Google – but of course, we have no way whatsoever of verifying this other than blind trust, and that’s in a corporation whose motto has shifted over the years from “Don’t be evil”, to “Privacy is overrated”, to “Whee, we make military robots”.

But here’s the thing. Even if Google isn’t intentionally recording at unknown cues in addition to the hotword searches – and I can’t emphasize enough that there’s no way to know this but blind trust is required – Google will still start recording audio at random times and send it to Google’s servers, when it picks up something it thinks sounds like “Ok, Google” from a conversation. This happened to me a week ago in Seoul, when I was discussing privacy concerns with EFF activists Parker Higgins and Maira Sutton (at least I believe it was that part of the conversation, but that’s beside the point here). All of a sudden, I noticed that my phone was transcribing what I was saying on its screen: Google’s audio detection had kicked in mid-conversation and it was recording the room. I was able to download that part of my conversation with the EFF activists off of Google’s Voice History page later:

 

So in summary;

We know now that Google’s audio recording does not start after you say “Ok Google”, but was active before you said it, suggesting that it’s always active to some degree.

Regardless of intentions, Google can start recording audio from a room mid-conversation anyway, as illustrated above.

This has all sorts of nasty privacy implications, and they’re not easy to work out. The way to do this properly for privacy is to make sure that all audio interpretation must happen locally, but even then, you can’t know if something is sent to a remote server if you’re searching for the wrong things as long as proprietary code is running (for instance, if you’re searching for something like methods for acquiring substances used to manufacture thermonuclear weapons, a search I dare say is never used to actually manufacture one in your basement, but which might still raise a flag somewhere).

Privacy remains your own responsibility.

About Rick Falkvinge

Rick is Head of Privacy at Private Internet Access. He is also the founder of the first Pirate Party and is a political evangelist, traveling around Europe and the world to talk and write about ideas of a sensible information policy. Additionally, he has a tech entrepreneur background and loves good whisky and fast motorcycles.


VPN Service

Leave a Reply

Your email address will not be published. Required fields are marked *

32 Comments

  1. Matt H

    If you’re a real privacy nerd then you’re probably best routing your device and putting a cyanogenmod flash on it. Then only ever using the F-Droid app store for apps. Although I’m actually quite dependent on Google apps and services now.

    2 years ago
    Reply
    1. Stop Bush and Clinton

      You don’t need Java. The only 2 tools you need to flash stuff on a device made for Android are fastboot and adb, both of which are written in C.

      They don’t even use any obscure shared libraries, just libc itself.

      $ ldd /opt/android-sdk-linux/platform-tools/fastboot
      linux-vdso.so.1 (0x00007ffc3194e000)
      libdl.so.2 => /lib64/libdl.so.2 (0x00007f48129a1000)
      libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f4812787000)
      libm.so.6 => /lib64/libm.so.6 (0x00007f4812494000)
      libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f4812481000)
      libc.so.6 => /lib64/libc.so.6 (0x00007f4812122000)
      /lib64/ld-linux-x86-64.so.2 (0x000056511c67f000)

      $ ldd /opt/android-sdk-linux/platform-tools/adb
      linux-vdso.so.1 (0x00007fff8dbc1000)
      librt.so.1 => /lib64/librt.so.1 (0x00007f66791cc000)
      libdl.so.2 => /lib64/libdl.so.2 (0x00007f6678fc9000)
      libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6678daf000)
      libm.so.6 => /lib64/libm.so.6 (0x00007f6678abc000)
      libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f6678aa9000)
      libc.so.6 => /lib64/libc.so.6 (0x00007f667874a000)
      /lib64/ld-linux-x86-64.so.2 (0x0000560841d52000)

      1 year ago
      Reply
    2. diy crafts

      Nevertheless, Google kind of roundabout apologized for having drive-by-downloaded black-box proprietary code to the open-source Chromium browser, and people attempting to shoot down the story (there are always those, and usually with the top-voted comment…) insisted that Google didn’t open the audio recording until after you had said the magic words “Ok Google”, something that is termed a hotword to begin recording, analyzing, and transcribing.

      5 months ago
      Reply
    3. pumpkin design

      Google is downloading a black box of code to open-source and free-software systems without permission, compromising them. Google is using proprietary code to listen in to your room.

      5 months ago
      Reply
    4. Inspirational Believe Tattoos

      Of these, the first was considered the most grave by far, as any proprietary running code will have access to the device’s sensors – including microphone – almost by definition, and therefore it wasn’t really considered strange that proprietary code had access to black-box recording. However, it was still a privacy issue and a concern that Google had the capability to listen in to any and every room where there was a Chrome or an Android running, which – frankly – is a rather large portion of the available rooms on the planet.

      5 months ago
      Reply
    5. Kawaii style

      You’ll recall from the previous privacy discussion, that Google having access to microphones in pretty much every room gives Google the ability to eavesdrop on those each and every rooms continuously. It should be pointed out that this is a technical ability; there’s no indication this is happening, but the presence of the capability is cause for serious privacy concerns.

      3 weeks ago
      Reply
  2. Justus Ranvier

    At least until Chromium starts shipping with VM escape exploits, it’s possible to sandbox Chromium in a virtual machine that doesn’t even have any audio hardware at all.

    I’ve managed to set up an environment on my PC such that I run zero network-accessing applications in the host OS itself, with everything else like web browsers, instant messaging clients, IRC clients, Thunderbird, etc is running in its own dedicated VM where I can control exactly what hardware it sees and control exactly which packets it’s allowed to send and receive.

    It seemed excessive back when I started the project, but every time something like this comes out I’m glad I invested the time to do it.

    2 years ago
    Reply
  3. fredy franklel

    I don’t see to much of a problem im not a hot target to be gay for the nsa I watch my network traffic with alliance rom and dont see KB going out for no reason.

    2 years ago
    Reply
  4. Elmar Veerman

    This may be a dumb question, but is there a real difference between ‘listening’ and ‘recording’ if it is done by a device? To make sense of what is said, the sound will always have to be stored for at least a short while. That doesn’t necessarily mean storage for a long time, or in a place outside of the device, but it does mean that it’s always possible to include ‘OK Google’ in the recording. Right?

    1 year ago
    Reply
    1. Cyberchip

      Right, I’m surprised it didn’t pick up the beep… they must have deleted that part.

      9 months ago
      Reply
  5. Cyr4x

    That’s probably why Gmail Android app yells for giving way too much permissions for Google Play Services when i tap new massage. Maybe Google wants to spy everything? It says, Gmail will work properly only then. In fact I use it without glitches, giving no permissions to GPS. Google Keep sometimes puts a notification that it cannot contact with GPS, but works well too (and synchronizes!). GPS is a closed source apk and we really don’t know what does it do and contain. Curious that GPS is using so much permissions since maybe 2-3 yrs. AFAIR some update on KitKat bringed it. So how Google apps and services worked well before without them?

    1 year ago
    Reply
  6. Stop Bush and Clinton

    Of course it listens all the time – there’s no way it can detect the “magic words” unless the microphone is on and something is analyzing what’s happening there.

    Of course we can only speculate what that binary blob does – but if it’s benign, it listens and starts acting the moment it recognizes the “magic words” – by recording (and transmitting) what it heard from the moment the recognition started returning a positive result, starting with the segment that triggered the match.

    It’s possible (and likely, simply because they can) that they transmit other stuff when they hear other interesting keywords. But this recording isn’t sufficient proof.

    1 year ago
    Reply
    1. elKaiC

      Exactly my thoughts. It doesn’t need to hit a server to recognize ‘ok google’ that is being done on the device and happens when you don’t have a network connection. After the recognition occurs it might send the full recording that caused it. I’m not a privacy nut, but lately I’ve felt more and more concerned with how much access google is asking for.

      1 year ago
      Reply
    2. Cyberchip

      OK google sometimes picks up stuff. I agree, as I said before I read this… How could anyone think that a black box that responds to a keyword isn’t listening for that keyword no matter how it analyses the input to see if the keyword is said. You’re right, it says nothing about what google is really doing; but, if I had no Trust relationship with them I wouldn’t be using it. It’s that simple. It’s like change the channel if you don’t like what you’re watching. I’m more worried about the stuff that might come out of a paranoid administration. As the saying goes, “Just because you’re paranoid it doesn’t mean they’re not out to get you.” It doesn’t mean they are either. Right now I just noticed a forced app in Windows 10 onenote, has a process called onenoteim (Did they say OneNote Instant Messenger?) I don’t know, but it’s always connected and listening to cloudflare in California… since they say they’re an internet protection company, what are they listening for… me to become infected… hah!

      9 months ago
      Reply
  7. Johannes Bols

    I thought OK Google was the NSA’s identifying the voice of the ISP user. When I was a contract worker for US Customs I had a security clearance. My phone was tapped more than Jim Price’s trumpt pegs during the 1972 Rolling Stones North American tour.
    When my friends and I would hear the ‘click’, meaning the line was being listened to, we launched into highly detailed gay pornographic stories. I mean REALLY detailed. Not raunch, just details, details, details. The line went ‘click’ shortly after this would start. Checkmate, fuckers…
    https://uploads.disquscdn.com/images/accc3149cd4cc2bb5a193e48d91d51c84353d58c0265c000358f333519f14c52.png

    11 months ago
    Reply
  8. John Dearing

    I’m like this if someone can earn a paycheck by sitting or standing at a work station listening to every conversation everywhere. Which has to number in the 100’s of millions at any given time. Then good for them. I’m not sure what they can do with most of our conversation as it is. Most of it wouldn’t be admisable in a court in any case. I don’t it’s as nefarious as it is being made out in this instance.

    9 months ago
    Reply
    1. Cyberchip

      Google already translates it, they’d just have to scan the text looking for words like kill president, or bomb something. lol This will probably get picked up… lol

      9 months ago
      Reply
    2. a female faust

      silly netizen! they have machines for that, not humans. software. that’s how come ‘collecting’ isn’t’ ‘listening’ — *technically* —l

      4 months ago
      Reply
      1. Carl Frederik

        Ha ! I known (or felt) that for a long time.

        Why does Google track what you`re searching then ?
        Also for `collecting` and `listening`.

        Damn machines. Maybe i should “go Amish”….

        2 months ago
        Reply
        1. a female faust

          sorry to take so long to get back to you, especially because that is my response exactly – but with a different cutoff date. the cutoff date is really the question, or if we want one at all: the tech is not the problem, its the transparency, or rather the unilateral or asymmetric deployment of the tech.

          2 weeks ago
          Reply
  9. Cyberchip

    Turn it off, so that it only responds when you touch it, if you think it’s a problem. OF COURSE any device that listens for a keyword has to be listening… I thought it was obvious. Google made so many false starts when I left it on for listening, I had to turn it off. So, I did. Now it have to touch the microphone first, every time.

    9 months ago
    Reply
    1. Jessica Eberl

      How do I turn it off?

      8 months ago
      Reply
      1. Cyberchip

        These are generally where they’re located, depending on phone brand. If you don’t find the settings as listed below, they should be in a similar part of the device for your phone. Otherwise look up ‘yourbrand Android turn of voice recognition’ to find the settings.

        Open the Google app.
        In the top left corner of the page, touch the Menu icon.
        Tap Settings > Voice > “OK Google“ Detection.
        From here, you can choose when you want your phone to listen when you say “Ok Google.”

        3 months ago
        Reply
  10. a female faust

    very interesting. indeed. i would agree with the person who commented that perhaps its a streaming/caching issue,

    — except for the fact that that is giving a lot of benefits of doubt to a company caught, for instance, colluding to impersonate my operating system so as to secure my authorization, BEHIND MY BACK and via local user account password, in order to set cookies for corporate big Data web vampires like DoubleClick.

    (wrote about it here, and yes i realize Google – i mean AlphaBet, owns them now).

    which may not be, to quote a warning famously softpedaled so as not to incite unnecessary freaking out, “specifically too good.”

    4 months ago
    Reply
  11. Robert Cook

    It’s high time the public sue the hell out of every company, including associates, and third parties for invasion of privacy!!! Even cops need warrants. On second thought, I guess not with all the legal spy/malware available being held ransomed by denial of service unless we agree to the ever changing fine print in terms and agreements when offered. All the while companies are increasing profits at the expense of our privacy and wallets.

    3 months ago
    Reply
  12. Mrs. Robert Cook

    Did I just give Google the authority to spy on us by playing the above recording? I revoke all permissions to Google, associates, and third parties to spy on us with programs or devices.

    3 months ago
    Reply
  13. aiota

    Recently turned off Voice Search as I would often find my phone randomly listening to audio, and this having made me curious did a search to find out how to access my voice search history; what I discovered was random snippets of my conversations recorded almost *daily* for the last year. I then turned off Voice search, but I still get “Google needs your permission to record audio” notifications often. Weird.

    3 months ago
    Reply
  14. Gabriel Coleman

    When I chose to disable the “feature ” there was a message that said it will continue to record ananomously. There is no way to stop it….

    2 months ago
    Reply