In an extraordinary way of recovering speech from vibrations of things like potted plant and potato chips bag, researchers at MIT, Microsoft and Adobe have created an algo that can decipher audio signal by analyzing the object’s infinitesimal vibrations. Upon performing experiment on detecting vibrations of a potato-chip bag, the researchers were able to reach a good precision with respect to speech. The bag was being photographed by a high quality camera with a distance of 15 feet installed in a soundproof glass-room.
While in similar other experiments, the team was able to extricate sound from the video samples of aluminum foil, leaves of a common houseplant and water contained in a glass.
When sound waves hit any object, it causes fluttering in the object. However, the vibrations so produced are very subtle that cannot be captured or observed by naked eye. As a matter of fact, people are unaware of this information being present there, said Abe Davis one of the lead researcher of the project.
The video below shows how the MIT researchers were able to extract audio from objects including vibrations of plant and a packet of chips:
Reconstruction hardware
It is important to have higher frequency video samples with respect to audio since only then the restructuring of audio from video can occur. Therefore, for capturing 2,000 to 6,000 frames per second, high-speed cameras were used. Although in one experiment, the team did make use of standard 60 frames per second camera. The output or the audio reconstruction with the hardware was not as clear as it was with the high-speed camera but it was good enough to identify the acoustic properties of speakers’ voices like gender and number of speakers.
Next generation imaging
It is anticipated that the technique would take forensics to the next level but Davis is more ambitious, he wants to take forward the application to a “new kind of imaging”. He added that the new technique is not only helping in recovering sounds from objects but also, equally giving light on the object itself. Different object would respond to sound in different ways since a lot is dependent on the material, mechanical and structural properties of objects, getting into this unexplored dimension of research is going to be exciting.
