Part of Google’s Alphabet soup, Google Brain, had a breakthrough in sharpening pixelated images. The company’s deep learning research center devised a new software that can create a detailed image from an extremely pixelated source.
[REWIND: GOOGLE RAISR | IMAGE RESOLUTION ENHANCEMENT STRAIGHT OUT OF CSI]
Daily followers of SLR Lounge might recognize that we reported on Google RAISR last month and this may seem a bit familiar. However, Google Brain’s software is much more powerful. Whereas RAISR can enhance and enlarge a recognizable photo, the software from Google Brain can take an 8×8 source (with the clever combination of two neural networks) can create something that resembles the original image.
It is impossible to create more detail than there is in the source image; so how does Google Brain do it? The image is first run through a conditioning network, where the software tries to map the 8×8 source against other high-resolution images that have been downsized to 8×8. Second, the prior network uses an implementation of the PixelCNN (a generative model of images with a tractable likelihood) to try and add realistic high-resolution details to the 8×8 source. In essence, it takes downsized references and uses them to upscale the 8×8 source.
An example provided by Ars Technica:
If there’s a brown pixel towards the top of the image, the prior network might identify that as an eyebrow: so, when the image is scaled up, it might fill in the gaps with an eyebrow-shaped collection of brown pixels.
In real world testing Google Brain’s super-resolution technique was reasonably successful fooling human observers 10 percent of the time when showed computer generated celebrity faces and 28 percent when shown images of bedrooms (50 percent would be a perfect score). As a reference, zero percent were fooled when using existing bicubic sampling techniques.
The common trope typical of criminal drama shows when the pixelated footage is “zoomed and enhanced” is a joke that the writers have since stopped, but it is becoming a reality. If you would like to know more about the technology behind the ‘Pixel Recursive Super Resolution’, Google Released a paper here.
Via: Resourcemagonline
