Research

Before founding Volograms, I was a Postdoctoral Research Fellow at V-SENSE, Trinity College Dublin, where I was advised by Prof. Aljosa Smolic. I did my PhD at GTI, Universidad Politécnica de Madrid, under the supervision of Prof. Francisco Morán.

My research interests are computer vision, 3D reconstruction, volumetric video and image processing. I’m especially interested in the reconstruction of the shape an appearance of human models. Representative papers are highlighted.

Publications

RGB-D-Fusion: Image Conditioned Depth Diffusion of Humanoid Subjects
Sascha Kirch, Valeria Olyunina, Jan Ondrej, Rafael Pagés, Sergio Martín, Clara Pérez-Molina,
IEEE Access, Vol 11, 2023

We present RGB-D-Fusion, a multi-modal conditional denoising diffusion probabilistic model to generate high resolution depth maps from low-resolution monocular RGB images of humanoid subjects.

Volumetric Video Content Creation for Immersive XR Experiences
Aljosa Smolic, Konstantinos Amplianitis, Matthew Moynihan, Neill O'Dwyer, Jan Ondrej, Rafael Pagés, Gareth Young,
London Imaging Meeting, 2023

This paper highlights technology for VV content creation developed by the V-SENSE lab and the startup company Volograms. It further showcases a variety of creative experiments applying VV for immersive storytelling in XR.

VoloGAN: Adversarial Domain Adaptation for Synthetic Depth Data
Sascha Kirch, Sergio Arnaldo, Sergio Martín, Rafael Pagés
Arxiv, 2022

We present VoloGAN, an adversarial domain adaptation network that translates synthetic RGB-D images of a high-quality 3D model of a person, into RGB-D images that could be generated with a consumer depth sensor.

Volograms & V-SENSE Volumetric Video Dataset
Rafael Pagés, Emin Zerman, Konstantinos Amplianitis, Jan Ondrej, Aljosa Smolic
ISO/IEC JTC1/SC29/WG07 MPEG2021/m56767, 2021
project page

This paper describes the Volograms & V-SENSE Volumetric Video Dataset which is made publicly available to help said research and standardisation efforts.

Influence of Mirror Therapy (Specular Face Software) on Electromyographic Behavior of the Facial Muscles for Facial Palsy
Alfonso Gil-Martínez, Sergio Lerma-Lara, Alfredo Hernando-Jorge, Ana Campos-Vegas, Audrey Aceval, Rafael Pagés, Francisco Morán, Hector Beltran-Alacreu
Brain Sciences, 11(7), 930, 2021

Using facial detection and tracking to simulate “mirror therapy” to help patients with facial paralysis.

Autonomous Tracking For Volumetric Video Sequences
Matt Moynihan, Susana Ruano, Rafael Pagés, Aljosa Smolic
WACV, 2021
project page / video

A robust, autonomous method for tracking volumetric sequences which can detect missing geometry and propagate user edits.

A Self-regulating Spatio-Temporal Filter for Volumetric Video Point Clouds
Matt Moynihan, Rafael Pagés, Aljosa Smolic
Computer Vision, Imaging and Computer Graphics Theory and Applications, Springer, 2020
project page / book chapter

A self-regulating filter that is capable of performing accurate upsampling of dynamic point cloud data sequences captured using wide-baseline multi-view camera setups.

Augmenting Hand-Drawn Art with Global Illumination Effects through Surface Inflation
Matis Hudon, Sebastian Lutz, Rafael Pagés, Aljosa Smolic
European Conference on Visual Media Production (CVMP), 2019
project page

A method for augmenting hand-drawn characters and creatures with global illumination effects.

2DToonShade: A stroke based toon shading system
Matis Hudon, Mairead Grogan, Rafael Pagés, Jan Ondrej, Aljosa Smolic
Computers & Graphics: X Volume 1, 2019

A semi-automatic method for creating shades and self-shadows in cel animation.

Spatio-Temporal Upsampling for Free Viewpoint Video Point Clouds
Matt Moynihan, Rafael Pagés, Aljosa Smolic
Computer Vision, Imaging and Computer Graphics Theory and Applications, Springer, 2019
project page

An approach to upsampling point cloud sequences captured through a wide baseline camera setup in a spatio-temporally consistent manner.

Affordable Content Creation for Free-viewpoint Video and VR/AR Applications
Rafael Pagés, Konstantinos Amplianitis, David Monaghan, Jan Ondrej, Aljosa Smolic
Journal of Visual Communication and Image Representation, Volume 53, 2018
Best paper award 🎖
project page / video

A scalable pipeline for Free-Viewpoint Video (FVV) content creation, considering also visualisation in Augmented Reality and Virtual Reality.

Jonathan Swift: Augmented Reality Application for Trinity Library’s Long Room
Néill O'Dwyer, Jan Ondrej, Rafael Pagés, Konstantinos Amplianitis, Aljosa Smolic
International Conference on Interactive Digital Storytelling, 2018
project page

The interactive prototype visualises Jonathan Swift in AR, allowing the user to be immersed in world of cultural heritage.

Samuel Beckett in VR: Exploring narrative using free viewpoint video
Néill O'Dwyer, Nicholas Johnson, Rafael Pagés, Jan Ondrej, Konstantinos Amplianitis, Enda Bates, David Monaghan, Aljosa Smolic
ACM SIGGRAPH - Leonardo, MIT Press, 2018
project page / video

Virtual Play is a reinterpretation of Play, with a view to engaging a 21st Century viewership that is increasingly accessing content via VR.

Deep Normal Estimation for Automatic Shading of Hand-Drawn Characters
Matis Hudon, Mairead Grogan, Rafael Pagés, Aljosa Smolic
ECCV, 2018
project page / video / code

A new fully automatic pipeline for generating shading effects on hand-drawn characters.

2D Shading for Cel Animation
Matis Hudon, Rafael Pagés, Mairead Grogan, Jan Ondrej, Aljosa Smolic
Expressive Graphics, 2018
project page / video

A semi-automatic method for creating shades and self-shadows in cel animation.

Virtual Play in Free-viewpoint Video: Reinterpreting Samuel Beckett for Virtual Reality
Néill O'Dwyer, Nicholas Johnson, Enda Bates, Rafael Pagés, Jan Ondrej, Konstantinos Amplianitis, David Monaghan, Aljosa Smolic
IEEE ISMAR, 2017
project page / video

Virtual Play is a reinterpretation of Play, with a view to engaging a 21st Century viewership that is increasingly accessing content via VR.

Simulation framework for a 3D high-resolution imaging radar at 300 GHz with a scattering model based on rendering techniques
Guillermo Ortiz-Jiménez, Federico García-Rial, Luis A. Úbeda-Medina, Rafael Pagés, Narciso García, Jesús Grajal
IEEE Transactions on Terahertz Science and Technology, Volume 7, 2017

A simulation framework for a 3D high-resolution imaging radar at 300 GHz with mechanical scanning.

Fast Feature Matching for Detailed Point Cloud Generation
Daniel Berjón, Rafael Pagés, Francisco Morán
International Conference on Image Processing Theory, Tools and Applications (IPTA), 2016

We propose a technique based on epipolar geometry restrictions to significantly cut down on processing time and an efficient implementation thereof on a GPU.

Seamless, Static Multi‐Texturing of 3D Meshes
Rafael Pagés, Daniel Berjón, Francisco Morán, Narciso García
Computer Graphics Forum, Volume 34, 2015
code

We present a static multi‐texturing system yielding a seamless texture atlas calculated by combining the colour information from several photos from the same subject covering most of its surface.

SPLASH: a Hybrid 3D Modeling/Rendering Approach Mixing Splats and Meshes
Rafael Pagés, Sergio García, Daniel Berjón, Francisco Morán
Web3D, 2015

We propose a hybrid 3D modeling and rendering approach called SPLASH to combine the modeling flexibility and robustness of SPLAts together with the rendering simplicity and maturity of meSHes.

Textured splat-based point clouds for rendering in handheld devices
Sergio García, Rafael Pagés, Daniel Berjón, Francisco Morán
Web3D, 2015

We propose a novel technique for modeling and rendering a 3D point cloud obtained from a set of photographs of a real 3D scene as a set of textured elliptical splats.

Automatic system for virtual human reconstruction with 3D mesh multi-texturing and facial enhancement
Rafael Pagés, Daniel Berjón, Francisco Morán
Signal Processing: Image Communication, Volume 28 , 2013

We present a fully automatic low-cost system for generating animatable and statically multi-textured avatars of real people captured with several standard cameras.

3D facial merging for virtual human reconstruction
Rafael Pagés, Daniel Berjón, Francisco Morán
3DTV Conference, 2012

A technique to merge a VH-based 3D mesh of a reconstructed humanoid and the depth data of its face, captured by a single structured light projector.

Refined facial disparity maps for automatic creation of 3D avatars
Rafael Pagés, Francisco Morán, Luis Salgado, Daniel Berjón
IS&T/SPIE Electronic Imaging, 2012

We propose a new method to automatically refine a facial disparity map obtained with standard cameras and under conventional illumination conditions by using a smart combination of traditional computer vision and 3D graphics techniques.

Multi-resolution texture coding for multi-resolution 3D meshes
David Fuentes, Rafael Pagés, Francisco Morán
VCIP, 2011

We present an innovative system to encode and transmit textured multi-resolution 3D meshes in a progressive way, with no need to send several texture images, one for each mesh LOD.

Face lift surgery for reconstructed virtual humans
Rafael Pagés, Sergio Arnaldo, Francisco Morán
International Conference on Cyberworlds, 2011
code

We introduce an innovative, semi-automatic method to transform low resolution facial meshes into high definition ones, based on the tailoring of a generic, neutral human head model.

ITEM: inter-texture error measurement for 3D meshes
Rafael Pagés, David Fuentes, Francisco Morán
Web3D, 2011

We introduce a simple and innovative method to compare any two texture maps, regardless of their sizes, aspect ratios, or even masks, as long as they are both meant to be mapped onto the same 3D mesh.

Composition of Texture Atlases for 3D Mesh Multi-texturing
Rafael Pagés, Sergio Arnaldo, Francisco Morán
Eurographics, 2010

We introduce an automatic technique for mapping onto a 3D triangle mesh a high resolution texture synthesized from several pictures taken by standard cameras surrounding the object.

Patents

Volumetric Imaging
Rafael Pagés, Jan Ondrej, Konstantinos Amplianitis, Sergio Arnaldo, Valeria Olyunina
US12026823B2, EP Application EP21154369.9

A method for generating a moving volumetric image of a moving object from data recorded by a user-held device comprising: acquiring, from the user-held device, video and depth data of the moving object, and pose data; and communicating the acquired data to a computing module.

Method and apparatus for generating a three-dimensional model
Rafael Pagés, Konstantinos Amplianitis, David Monaghan, Jan Ondrej, Aljosa Smolic
US20200320727B2, EP Application PCT/EP2018/086331.

A method comprising providing a plurality of images of a scene captured by a plurality of image capturing devices.

A method and a system for generating a realistic 3d reconstruction model for an object or being
Rafael Pagés, Daniel Berjón, Sergio Arnaldo, Francisco Morán, Tomas Montserrat Mora, Julien Quelen, Oscar Divorra Escoda, Christian Ferran Bernstrom
US Application US20150178988A1, EP Application EP2852932A1, WO Application WO2013174671A1

A method for generating a realistic 3D reconstruction model for an object or being.


This page was "stolen" from Jon Barron's website. You can find the source code here.