Taking a picture of a bird or a car in motion inevitably is tainted by “smearing” of the image using an ordinary still camera. Now, Tel Aviv University (TAU) researchers have developed a computational photography process based on an optical element that encodes motion information and a corresponding digital image processing algorithm. This makes possible clear, sharp photos of moving objects without motion blur. 

This integrated processing method was developed by doctoral student Shay Elmalem from TAU’s School of Electrical Engineering under the joint guidance of Prof. Emanuel Marom and Dr. Raja Giryes. The results of the study have been published in the prestigious Optica Journal under the title “Plenoptic imaging.”

“Finding the proper exposure setting is a well-known challenge in photography,” the team wrote. “In general, one has to balance aperture size, exposure time, and gain to achieve a good image (the trade-off between these factors is sometimes referred to as the “exposure triangle”). This balancing process involves many trade-offs and, therefore, requires complex skills and rich experience. In many cases, a large exposure is necessary to allow a sufficient amount of light to reach the sensor in order to achieve a good image with respect to the lighting condition, which usually is not controllable.”

The computational image technique they developed can enhance any camera and at minimum cost. “The potential is very broad, from basic uses like smartphone cameras to research, medical and industrial uses such as for production line controllers, microscopes and telescopes. They all suffer from the same smearing problem, and we offer a systemic solution to it,” the TAU researcher said. 

The term “long exposure” always refers to the velocity of the photographed object, explained Elmalem. “If you photograph a racing car, even an exposure of a tenth of a second could be too long, and if you’re photographing a person walking, long exposure could be a second or longer. According to the conventional camera-design approach, the lens is designed to produce the best possible image – the most similar to what the human eye sees. So digital image-processing algorithms are applied to remove the optical distortions. However, as anyone with a camera in his mobile phone knows, this isn’t always effective, so it is still very difficult to photograph moving objects.” 

Through integrated design of the optical components and image post-processing algorithms, Elmalem and colleagues have encoded motion information cues in the raw optical image; these cues are in turn decoded by the image processing algorithm which utilizes them for motion deblurring. The cues have been encoded using two optical components integrated in a conventional lens: a clear phase plate developed by the researchers, and a commercial electronic focusing lens. The phase plate contains a micro-optical structure designed to introduce a color-focus dependency, whereas the focusing lens is synchronized in order to make a gradual focus change during the image exposure. 

As a result, moving objects are colored with various colors as they move. Encoding the colors enables the algorithm to decode the direction and velocity of the object’s movement, which enables it to correct the motion blur and restore the image sharpness.

“In every split second of exposure, our lens generates a bit different image,” Elmalem continued. “As a result, the blur of a moving object will not be uniform, but it will instead change gradually with its movement. To understand where and how fast the object in the image is going, we use color. For example, a white ball suddenly thrown into the frame will be colored with different colors over the course of its movement, like passing light through a prism. According to these colors, our algorithm knows where the ball has been thrown from and at what velocity. It will thus know how to correct the blur. With a regular camera, we’d see a white wake that would compromise the sharpness of the whole picture, whereas with our camera the final image will be a clear focused white ball,” the doctoral student concluded. 

Ramot, TAU’s technology transfer company, has filed several patent applications covering this breakthrough technology, which is generating great interest among industry players.