TOUCHING MONA LISA’S FACE (2017)

MOTIVATION
Every time I am in front of a portrait painting I feel this urge to touch it.
I can’t help to imagine how, according to the techniques the artist used, the skin of that specific face would feel like when touching it. Well, it seems we are getting closer to that. Jackson et al. (2017) have just published a paper titled “Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric Convolutional Neural Network (CNN) Regression“. I couldn’t help myself to play a bit with their results and here I present the outcome. I took some of the most famous portrait paintings in art history and I run them through their algorithm. I know, this is not like a real touch of a portrait painting’s face but at least now we can experience them with volume and texture. And come on, it looks terribly cool!

JACKSON ET AL.’s (2017) WORK
3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the availability of multiple facial images (sometimes from the same subject) as input, and must address a number of methodological challenges such as establishing dense correspondences across large facial poses, expressions, and non-uniform illumination. In general these methods require complex and inefficient pipelines for model building and fitting. In this work, they propose to address many of these limitations by training a Convolutional Neural Network (CNN) on an appropriate dataset consisting of 2D images and 3D facial models or scans. Their CNN works with just a single 2D facial image, does not require accurate alignment nor establishes dense correspondence between images, works for arbitrary facial poses and expressions, and can be used to reconstruct the whole 3D facial geometry (including the non-visible parts of the face) bypassing the construction (during training) and fitting (during testing) of a 3D Morphable Model. They achieve this via a simple CNN architecture that performs direct regression of a volumetric representation of the 3D facial geometry from a single 2D image. They also demonstrate how the related task of facial landmark localization can be incorporated into the proposed framework and help improve reconstruction quality, especially for the cases of large poses and facial expressions. Code and models are available at http://aaronsplace.co.uk

(BE PATIENT UNTIL ALL THE IMAGES LOAD)