Mic Jammer examines the hidden methods that the Internet-of-Things uses to communicate with mobile phones and how its unregulated spread can challenge the way we think about public and private space.
This hidden method is the broadcast of inaudible sounds from IoT devices and services using near-ultra high frequency (NUHF) audio between 18 kHz to 24 Khz. The purpose of using such hidden sounds is to identify the person and their phone without our awareness.
Another method, increasingly popular, is to use Audio Content Recognition (ACR) systems. These SDKs are embedded in apps and utilise the microphone to record television adverts, programs and other “ambient” sounds.
Today’s urban landscape is filled with creepy teddy bears, billboards that identify you, televisions that watch you, buildings and shops that track you and phones that will always listen to what you say. Combined they form part of a massive evolution in the way technology defines the economy and mediates social relations.
The deriving of profit from digitised identities, behaviours and actions has seen the rise of outsourcing of labour at the forefront of the digital economy. From the isolated and speculative income of the Uber driver and the unprotected food delivery rider, to the access of vital social services via online apps; the sharing economy socialises the risks, debts and running costs while privatising the profits.
One method to extract profit from human digital labour is to digitise the Time and Motion surveillance of the factory into software known as trackers. This type of software is typically made available for developers to include within their apps. It can provide a source of revenue via the exchange of gleaned statistics about who is using the app and how. Companies profit from this type of information by connecting the capabilities of tracker software with massive relational databases. These databases exist for the specific purpose of identifying people by their behaviours, their traits, their habits and their possessions. And from here the professed intention is to anticipate “your needs” before you are aware of them.
Other than overt surveillance, an app with an SDK in it that uses the microphone for NUHF recordings can be used for contactless payments. While it may offer a secure method with encrypted data, the reliance on audio signals from a potentially unknown source that cause payment transactions directly on the mobile phone may not be the most sensible use of this technology. Several of the companies involved in this area also have partnerships with advertising/marketing tracker companies or are directly involved in advertising. In some ways the use of the mobile phone’s microphone could viewed as if it were a bluetooth radio: something that needs to be turned on manually and has an indication that it is running and can potentially record any audio.
Understanding this process may reveal the current scale of internet based surveillance and why privacy and the right to privacy is important. And why your phone doesn’t need to record your conversations to know anything about you.
ACR Audio Tracker demo, an Audio Record capture demonstration.
To determine an answer to the question “is my device listening to me?”, a demonstration is examined that uses an Android mobile phone, a game app downloaded for free from Google Play and some analysis software.
Link: Audio Tracker demo
Known types of NUHF signals (also Audio Content Recognition, and Voice Interactive Adverts)
SDK package pronouns: acrcloud, actv8 (cifrasoft), alphonso(ACR), axwave(ACR, NUHF), beatgridmedia(ACR), bitsound (soundlly), chirp, chromecast(NUHF), cifrasoft(ACR, NUHF), copsonic, cueaudio, digimarc(ACR), dv (dov-e), fanpictor, fidzup, fluzo(ACR), gracenote(ACR), hotstar(ACR)(zapr), hound(ACR), inscape(ACR), instreamatic (VIA), kantarmedia(ACR), lisnr, moodmedia, mufin (ACR), prontoly (now sonarax), realitymine(ACR), redbricklane(ACR)(zapr), runacr, shopkick, signal360, silverpush, sonarax, soniccode, sonicnotify (now Signal360), soundpays, tonetag, trillbit, uber, zapr(ACR)
The list below is not complete:
IoT beacons – Actv8, Bitsound, Cue Audio, Fidzup, Shopkick, Signal360(SonicNotify), Soundpays, ToneTag, Trillbit
Voice Assistants – Chromecast (Google Home), Lisnr, Trillbit
Web – Cifrasoft, Copsonic, Digimarc, Dov-e, Infosonic(soniccode), Lisnr, Sonarax, Soundpays, ToneTag
Internet streaming – ACRcloud, Alphonso, Cifrasoft, Instreamatic, Signal360, Trillbit
Television broadcast – ACRcloud, Actv8, Alphonso, Cifrasoft, Digimarc, Hotstar, Inscape, Red Brick Lane, Silverpush, Soundpays, Zapr
Peer-to-Peer – Chirp, Cifrasoft, CopSonic, Dov-e, Infosonic (soniccode), Lisnr, Sonarax, ToneTag, Uber
Mic Jammer app
Experimental app designed to counter NUHF beacon audio.
The Passive Jammer requests use of the hardware microphone from the Android system and holds it. This technique locks up the microphone from any other apps attempting to gain access to it. The Android system should halt Mic Jammer from blocking the microphone whenever a phone call is received or made.
An Active Jammer emits audio tones with a carrier frequency and a drift limit with rate all constrained to NUHF of 18 kHz to 24 kHz depending on the device capabilities. For instance 20000 Hz carrier, drift limit 1000 Hz and rate slow – will output random frequency between 19 kHz and 21 kHz approximately every second.
Also includes a user app summary function that lists relevant capabilities: record audio, boot listen, services, receivers and any known NUHF/ACR SDKs installed. A user app scanner lists any receivers and services for a selected user app.
It requires RECORD_AUDIO permission so that it may access and lock up the microphone from use.
It does NOT record or listen to any audio.
It does NOT connect to the internet.
Research
– The Technology of Computer Music (Sound Processing) (6.7 mb pdf)
– A Study of Scripts Accessing Smartphone Sensors (0.8 mb pdf)
– Active Acoustic Side-Channel Attacks (3.4 mb pdf)
– Acoustic Indoor Smart Phone Tracking (3.0 mb pdf)
– Audio and Video Exfiltration from Android Applications (0.5 mb pdf)
– Localization using controlled ambient sounds (0.6 mb pdf)
– Global Study of the Mobile Tracking Ecosystem (2.0 mb pdf)
– Mobile Device Sensor Fingerprinting (1.6 mb pdf)
– Privacy and Security of the Ultrasound Ecosystem (1.6 mb pdf)
– Smartphone Audio Acquisition Using an Acoustic Beacon (3.7 mb pdf)
– Using Smartphones to Collect Behavioral Data in Psychological Science (0.5 mb pdf)
– Third Party Tracking in the Mobile Ecosystem (1.4 mb pdf)
– A 1-million-site Measurement and Analysis (2.5 mb pdf)
– Shazaam audio search algorithm (0.5 mb pdf)
Patents
– UTILIZING AUDIO BEACONING IN AUDIENCE MEASUREMENT (2009) (1.1 mb pdf)
– MATCHING TECHNIQUES FOR CROSS-PLATFORM MONITORING (2010) (1.3 mb pdf)
– BROADCAST CONTENT VIEW BASED ON AMBIENT AUDIO RECORDING (2016) (0.2 mb pdf)
– MODULATE A CODE AND PROVIDE CONTENT TO A USER (2011) (0.5 mb pdf)
– ACOUSTIC MODULATION PROTOCOL (2011) (2.3 mb pdf)
– DEMODULATE A MODULATED CODE PROVIDE CONTENT TO A USER (2013) (0.7 mb pdf)
– MODULATE A CODE AND PROVIDE CONTENT TO A USER (2012) (0.2 mb pdf)
– TRANSMITTING DATA OVER VOICE CHANNEL (2013) (1.5 mb pdf)
IoT Beacon materials
– Sonic Notify user manual (2.5 mb pdf)
ACR materials
– Zapr SDK Developer Guide (0.7 mb pdf)
