Neuromorphic Image Sensors Inspired By The Human Eye

Blog Date:  9/8/2016
Author:  Ray Coulombe
Category(s):  Advanced Technology, Product Technology

The beginning of video is traced back to Edward Muybridge in 1878. He rigged a line of cameras at a horse farm to take photographs of a galloping horse in a rapid-fire sequence. These images proved that when horses gallop, all four legs come off the ground simultaneously—though it happens too fast for the naked eye to see. This illusion of motion still happens in video cameras today: Capture a sequence of still images and play them back rapidly to create one moving image.

To track the action properly (and with one camera, unlike Muybridge), you’ll need a high frame rate, or lots of pictures taken per second. But sometimes, even with a high frame rate, the action happens too quickly to be captured.

The trick here is that technology needs to be adapted to act more like our eyes. The idea of creating electronic signal-processing systems inspired by biology (our eyes) is called neuromorphic engineering.

Our eyes can sample different parts of what we see at different rates. The bits of what you see containing fast motions are sampled quickly, while slow or non-changing portions of the scene are sampled at lower rates. New research in camera technology is focusing on equipment that could “see” an image at different rates, like an eye.

Overall, this is tricky because you don’t know which parts of the scene change rapidly and which parts stay stagnant, but it would make fast-changing images (explosions, shattering glass, etc.) easier to analyze. It would also allow cameras on smartphones and other battery-operated devises to record ordinary motion using less battery.

While the video camera is similar to an eye, there are also many differences. The retina, for example, doesn’t just turn light into electric signals. Retinas process the eye’s photoreceptor cells, capture the content that is important, and sends that information to the brain instantly.

New research in camera technology attempts to replicate the biology of the eye, sending information to the brain: Each pixel sensor operates independently, increasing sampling and transmitting more information when the light on that pixel change. When the amount of light reaching a pixel is not changing (i.e. redundant imagery), the individual pixel goes idle.

The key to this is that the visual information isn’t based on frames per second anymore. Parts of the image that change may generate selectively higher rates of data. This allows the camera to acquire and transmit only the relevant information, and then sends that information along to GPU’s (parallel processors which apply machine-vision like algorithms). This results in blazing-fast data acquisition to resolve things that would otherwise be easily missed. Further, the processing is so powerful that next generation analytics become much more powerful and adaptive. And the relevance of video to "Big Data" increase exponentially.

We’ll also see this technology impact other areas. This equipment might allow for “always on” visual input on smartphones, currently impossible because of power requirements. There is currently a medical application in trial use that leverages neuromorphic vision sensors as electronic retinal implants to restore sight to humans who lost their vision to disease.

Eye-like vision sensors are just the beginning of a huge evolving set of applications in medical technology, robotics, security, and more. So keep an eye out.


Resource Blogs

Most Recent Blogs Listed for Ray Coulombe.


Stay Safe! While Traveling This Summer

It’s summer vacation time! The last thing you need to worry about it is getting your identity stolen while you’re sitting on a beach somewhere exotic. In 2016, more than 15 million Americans were victims of identity theft, up 16 percent from the previous year, according to Experian. Plus, about 33 percent of that fraud took place when people were traveling. Here’s a few tips to staying safe all summer while traveling...
read more -->


Rethinking Cabling

Cat 5e became an ANSI/TIA/EIA standard in 2001, Cat 6 in 2002, and Cat 6a in 2008. However, it may be extremely useful to consider taking advantage of other existing cabling infrastructure in lieu of running new. Read more to learn how to approach cabling.
read more -->


Off the Beaten Path at ISC West

This year at ISC (the International Security Conference and Exposition), I was determined to try to see the latest iStechnologies hiding in the nooks and crannies—literally! I visited booths in the back, the basement, small kiosks hidden inside larger vendor books, and throughout the Emerging Technology Zone.

In case you missed the show, I’ll round up some of the best new technologies and companies to keep an eye on. Read more.
read more -->


Cyber Crime Taking Down Cities

Earlier this year, in March, the City of Atlanta’s nearly 8,000 employees heard words they never thought they would hear: “It’s okay to turn your computers on.” Their computers were powered off for five days. In those five days Atlanta residents could not pay traffic tickets, water bills, or report city issues. Read how ransomware impacted this metropolitan area.
read more -->


A Few Thoughts on K-12 School Security

There is no one size fits all when it comes to K-12 school security. Schools vary in so many ways: size, age, local environment, affluence, culture, governance, and more. Read some helpful tips and resources that might just help your school be better prepared.
read more -->