is a blog about design, technology and culture written by Khoi Vinh, and has been more or less continuously published since December 2000 in New York City. Khoi is currently Principal Designer at Adobe. Previously, Khoi was co-founder and CEO of Mixel (acquired in 2013), Design Director of The New York Times Online, and co-founder of the design studio Behavior, LLC. He is the author of “How They Got There: Interviews with Digital Designers About Their Careers”and “Ordering Disorder: Grid Principles for Web Design,” and was named one of Fast Company’s “fifty most influential designers in America.” Khoi lives in Crown Heights, Brooklyn with his wife and three children.
Dumb Movies May Turn Out to Be Right About Photo Enhancement
For years we’ve groaned every time a character in a movie commands a computer to “Enhance!” a low resolution image, and then watched as an implausibly clear, high resolution replacement appears before our eyes. For me, this has always been one of the worst kinds of lazy storytelling; it always suggests a fundamental lack of understanding of how digital imaging works on the part of the moviemakers.
Well as it turns out, the work of scientists over at the Max Planck Institute for Intelligent Systems in Germany may ultimately give technologically clueless film directors from the 1980s and 1990s the last laugh.
The picture is downsampled, reducing the data to this pixelated state:
That image is then processed with their “ENet-PAT” method and results in this:
I’m not sure this approach can resolve a grainy image of a face into something instantly recognizable, which is usually what one sees in films, but this example is stunningly effective nevertheless.
To give some context, the resampling techniques most of us are familiar with from Photoshop and other image editors generate new pixels and details strictly from what’s available in a given low-resolution image. That results in glaringly unconvincing results that are usually either overly smooth or pock-marked with unsightly image artifacts. By contrast ENet-PAT uses machine learning techniques to teach a neural network how to best guess what details should be added to an image. Once trained, the system can produce reliably believable results like these. Scary.