MIT researcher on computer vision study: 'You want to know that the true image is contained within that range so you are not missing anything critical'

Researchers at the Massachusetts Institute of Technology have developed a way to accurately measure and display uncertainty in computer vision algorithms that can be understood by the average person, according to a news release.

Computer vision involves training computers to obtain information from digital images. The researchers' focus was on images that are partially smudged or corrupted due to missing pixels, MIT said.

Led by Swami Sankaranarayanan, a postdoctoral student at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), the team also included researchers from the University of California at Berkeley and the Technion, the Israeli Institute of Technology. The research aims to uncover the part of the signal that is marred or concealed, using computer algorithms designed to take the blurred image as input and produce a clean image as output. The process typically occurs in two steps, with the first being the use of an encoder, which creates an abstract or latent representation of a clean image from a distorted image. The next step is a decoder, which takes the numbers from the encoded representation and constructs a complete, cleaned-up image.

The researchers' main objective was to establish the uncertainty in the accuracy of the output image and determine the best way to represent that uncertainty. The standard approach involves creating a "saliency map," which ascribes a probability value somewhere between 0 and 1 to indicate the model's confidence in the correctness of every pixel, taken one at a time.

However, the researchers' approach centers around the "semantic attributes" of an image, which groups pixels together that have meaning, making up a human face, dog, or other recognizable thing. Their aim is to estimate uncertainty in a way that relates to the groupings of pixels that humans can interpret readily. Rather than producing a single image, which is the "best guess" of what the true picture should be, the researchers developed a procedure for generating a range of images, each of which might be correct. They can also set precise bounds on the range or interval, providing a probabilistic guarantee that the true depiction lies somewhere within that range. The researchers believe their paper puts forth the first algorithm, designed for a generative model, that can establish uncertainty intervals that relate to semantically interpretable features of an image and come with a formal statistical guarantee.

The authors' approach provides a way to display uncertainty that holds meaning for people who are not experts in machine learning. A narrower range can be provided if the user is comfortable with, say, 90% certitude, and a narrower range still if more risk is acceptable.

Sankaranarayanan sees this work as a step toward the ultimate goal: "So far, we have been able to do this for simple things, like restoring images of human faces or animals, but we want to extend this approach into more critical domains, such as medical imaging, where our ‘statistical guarantee’ could be especially important,” the news release quotes him as saying. 

 "Suppose that the film, or radiograph, of a chest X-ray is blurred, and you want to reconstruct the image," he said. "If you are given a range of images, you want to know that the true image is contained within that range so you are not missing anything critical." 

Such information might reveal, for example, whether or not a patient has lung cancer. The researchers' work could have practical applications in several fields, including autonomous vehicles, facial recognition and medical imaging. The new approach could ensure that uncertainty is presented in a way that is easily understood, giving people greater confidence in the accuracy of computer vision algorithms.