Blog Date: 9/8/2016
Author: Ray Coulombe

The beginning of video is traced back to Edward Muybridge in 1878. He rigged a line of cameras at a horse farm to take photographs of a galloping horse in a rapid-fire sequence. These images proved that when horses gallop, all four legs come off the ground simultaneously—though it happens too fast for the naked eye to see. This illusion of motion still happens in video cameras today: Capture a sequence of still images and play them back rapidly to create one moving image.

To track the action properly (and with one camera, unlike Muybridge), you’ll need a high frame rate, or lots of pictures taken per second. But sometimes, even with a high frame rate, the action happens too quickly to be captured.

The trick here is that technology needs to be adapted to act more like our eyes. The idea of creating electronic signal-processing systems inspired by biology (our eyes) is called neuromorphic engineering.

Our eyes can sample different parts of what we see at different rates. The bits of what you see containing fast motions are sampled quickly, while slow or non-changing portions of the scene are sampled at lower rates. New research in camera technology is focusing on equipment that could “see” an image at different rates, like an eye.

Overall, this is tricky because you don’t know which parts of the scene change rapidly and which parts stay stagnant, but it would make fast-changing images (explosions, shattering glass, etc.) easier to analyze. It would also allow cameras on smartphones and other battery-operated devises to record ordinary motion using less battery.

While the video camera is similar to an eye, there are also many differences. The retina, for example, doesn’t just turn light into electric signals. Retinas process the eye’s photoreceptor cells, capture the content that is important, and sends that information to the brain instantly.

New research in camera technology attempts to replicate the biology of the eye, sending information to the brain: Each pixel sensor operates independently, increasing sampling and transmitting more information when the light on that pixel change. When the amount of light reaching a pixel is not changing (i.e. redundant imagery), the individual pixel goes idle.

The key to this is that the visual information isn’t based on frames per second anymore. Parts of the image that change may generate selectively higher rates of data. This allows the camera to acquire and transmit only the relevant information, and then sends that information along to GPU’s (parallel processors which apply machine-vision like algorithms). This results in blazing-fast data acquisition to resolve things that would otherwise be easily missed. Further, the processing is so powerful that next generation analytics become much more powerful and adaptive. And the relevance of video to "Big Data" increase exponentially.

We’ll also see this technology impact other areas. This equipment might allow for “always on” visual input on smartphones, currently impossible because of power requirements. There is currently a medical application in trial use that leverages neuromorphic vision sensors as electronic retinal implants to restore sight to humans who lost their vision to disease.

Eye-like vision sensors are just the beginning of a huge evolving set of applications in medical technology, robotics, security, and more. So keep an eye out.

Back to Blog List