Breakthrough AI Researchers Achieve 93% Accuracy in Zoom Audio Keystroke Detection, Revealing Exciting Potential

Introduction

By recording keystrokes and training a deep learning model, three researchers claim to have achieved upwards of 90 percent accuracy in interpreting remote keystrokes, based on the sound profiles of individual keys.

Threat to Keyboards

In their paper A Practical Deep Learning-Based Acoustic Side Channel Attack on Keyboards, UK researchers Joshua Harrison, Ehsan Toreini, and Marhyam Mehrnezhad claim that the trio of ubiquitous machine learning, microphones, and video calls “present a greater threat to keyboards than ever.” Laptops, in particular, are more susceptible to having their keyboard recorded in quieter public areas, like coffee shops, libraries, or offices, the paper notes. And most laptops have uniform, non-modular keyboards, with similar acoustic profiles across models.

Past Attempts and Recent Innovations

Previous attempts at keylogging VoIP calls, without physical access to the subject, achieved 91.7 percent top-5 accuracy over Skype in 2017 and 74.3 percent accuracy in VoIP calls in 2018. This new research makes use of recent advancements in neural network technology, including self-attention layers, to propagate an audio side channel attack.

Testing and Results

The researchers used a 2021 MacBook Pro to test their concept, typing on 36 keys 25 times each to train their model on the waveforms associated with each key. They achieved higher than 93 percent accuracy in their tests, with the phone-recorded audio edging closer to 95-96 percent.

Mitigating Attacks

• Changing your typing style, with touch typing in particular being less accurately recognized
• Using randomized passwords with multiple cases
• Adding randomly generated false keystrokes to the transmitted audio of video calls
• Using biometric tools, like fingerprint or face scanning, rather than typed passwords

Conclusion

Sound-based side channel attacks on sensitive computer data are a real threat. It is important for individuals and organizations to be aware of the potential risks and take necessary precautions to mitigate them.