News
Revolutionary Neuromorphic Visual Sensor Accurately Detects and Predicts Moving Objects with Hidden Information
A new bio-inspired sensor can recognise moving objects in a single frame from a video and successfully predict where they will move to.
A team of researchers at Aalto University has developed a new bio-inspired sensor that can detect moving objects in a single video frame and predict their path. This smart sensor is based on neuromorphic vision technology that integrates sensing, memory, and processing in a single device. It can be used in various fields, including automatic inspection, industrial process control, robotic guidance, and autonomous driving technology.
The sensor is made of an array of photomemristors.
« A neuromorphic visual sensor can recognise moving objects and predict their path
The sensor’s core technology is an array of photomemristors, electrical devices that produce electric current in response to light. This allows the sensor to “remember” whether it has been exposed to light recently. As a result, the sensor doesn’t just record instantaneous information about a scene but also includes a dynamic memory of the preceding instants. This is similar to how the human visual system works.
The sensor’s ability to integrate a series of optical images in one frame is unique. The information of each image is embedded in the following images as hidden information. The final frame in a video, therefore, has information about all the previous frames. This allows the system to detect motion earlier in the video by analyzing only the final frame with a simple artificial neural network.
To demonstrate the technology, the researchers used videos showing the letters of a word one at a time. The photomemristor array could use hidden information in the final frame to infer which letters had preceded it and predict what the word was with nearly 100% accuracy. In another test, the team showed the sensor videos of a simulated person moving at three different speeds. The system was able to recognize motion by analyzing a single frame and predict the next frames accurately.
The accuracy of detecting motion and predicting where an object will be are essential for self-driving technology and intelligent transport. Autonomous vehicles need accurate predictions of how cars, bikes, pedestrians, and other objects will move to make the right decisions. By adding a machine learning system to the photomemristor array, the researchers showed that their integrated system can predict future motion based on in-sensor processing of an all-informative frame.
The researchers believe that their integrated system provides new opportunities in autonomous robotics and human-machine interactions. The in-frame information obtained using photomemristors avoids redundant data flows, enabling energy-efficient decision-making in real-time.
https://q5i.09c.myftpupload.com/category/science/
