By Avigayil Kadesh
An Israeli-innovated tool for downsizing photos and videos without sacrificing resolution or details will soon be available thanks to a deal between Adobe Systems and the Weizmann Institute of Science in Rehovot.
This significant improvement over cropping or clipping began as a purely academic inquiry in the computer vision and image-processing lab of
Prof. Michal Irani at the Weizmann Institute.
“Given a large image/video, we often want to display it in a different (often smaller) size – e.g., for generating image thumbnails, for obtaining short summaries of long videos, or for displaying images/videos on different screen sizes,” Irani’s team wrote in a paper describing their research. “This smaller representation (the visual summary) should faithfully represent the original visual appearance and dynamics as best as possible, and be visually pleasing.
“The simplest and most commonly used methods for generating smaller-sized visual displays are scaling and cropping. Image scaling maintains the entire global layout of the image, but compromises its visual resolution, and distorts appearance of objects when the aspect ratio changes. Cropping, on the other hand, preserves visual resolution and appearance within the cropped region, but loses all visual information outside that region.”
Irani and her PhD students Denis Simakov, Yaron Caspi and Eli Shechtman were curious to see if they could develop an algorithm to make smaller versions of original images while keeping the most relevant information intact. They discovered that the key lies in eliminating the many visual redundancies in the larger original.
Losing nothing in translation“Let’s say you have an image of a field of flowers,” Irani explains. “Instead of shrinking the image and producing a smaller image with the same number of flowers, our algorithm will recognize and remove the redundancies, and generate an image of a new field with a smaller number of flowers at the same size and resolution. It synthesizes a coherent-looking image with as much information as the original image.”
The new image has “bidirectional similarity” with respect to the original. “One direction guarantees that all elements in the large image are found in the small one -- it’s complete, meaning the original can be recovered from the small one. The other direction guarantees that you haven’t introduced any new visual element that was not in the original image -- it’s coherent.”
Before and after images showing summarization
with the bidirectional similarity measure
From Weizmann to Adobe
The Israeli scientists completed their invention and applied for a patent in 2007. “We thought it could have commercial applications,” says Irani. They presented their
research results and paper at the International IEEE Conference on Computer Vision and Pattern Recognition in 2008.
Weizmann’s technology transfer company,
Yeda Research and Development, recently announced a licensing agreement with Adobe that will enable the computer imaging giant to add the bidirectional similarity measure to its future products.
In addition to summarizing, the bidirectional similarity measure has many potential uses, such as completing missing parts in images or videos; creating montages out of separate images; photo reshuffling; automatic cropping; image synthesis (in which an image is expanded rather than summarized); and image morphing (generating a video sequence displaying a smooth transition from one image to another, possibly unrelated, image).