Virtual Reality reconstruction of Piet Mondrian’s ‘Salon for Madame B’, presented at VSS 2018 Demo Night, 21/05/2018, St. Pete Beach, Florida, by Johannes Zanker & Jasmina Stevanov (RHUL), and Tim Holmes (Tobii Pro Insight UK)
Background
Piet Mondrian, the most prominent member of the Dutch ‘De Stijl’ arts movement who created some of the iconic paintings of abstract art - composed of horizontal and vertical lines and rectangles filled in white, grey, black and primary colours, in order to expressing pure shapes, colours, and balance - was commissioned by Ida Bienert in 1926 to design a room in his idiosyncratic style. The reasons why this room was never realised has been discussed by experts over many decades, and there were three attempts to reconstruct this room at full scale in international galleries – which however were met with criticism about their realisation, and were generally regarded of limited success. We argued previously (Stevanov & Zanker, Leonardo, 2017) that one reason why Mondrian did not realise his design could have been that Mondrian patterns in a 3D space diverged from the key principles of De Stijl aesthetics due to conflicts with the rules of perspective, and demonstrated this by using a computer graphic 3D modelling and animations. Most importantly, when looking at the corners of the room, the image projected onto the observer’s retina will show oblique lines and non-orthogonal intersections that are inconsistent with Mondrian’s dogmatic demand for pure aesthetics.
The Demonstration
At VSS 2018, the leading international conference on visual science, we presented the first reconstruction of Madam B’s Salon in a Virtual Reality environment, which allowed visitors to move freely in the room and contemplate their perceptual experience. An important aspect of the retrofitted version of the HTC Vive headset used in this demo is its integrated eye-tracker system (driven by Tobii Pro VR Analytics), which allows the experimenter to interrogate effects of body, head and eye movements during the interaction between the visitor and the 3D artwork. In addition, some details of the VR environment were corrected from Mondrian’s original design, in order to fit it into the physical space in Ida Bienert’s home.
We presented the first Virtual Reality realisation of Mondrian’s design during an evening Demo event at VSS 2018. Wearing a modified Vive VR headset with integrated eye tracking capability, visitors were moving freely in an empty space whilst viewing and exploring the interior of the virtual salon. Oblique lines and non-orthogonal intersections become immediately - and dynamically - visible in the scene views from the viewpoint of the visitors, together with their fixation point. Also, highlights in orientation that are not parallel to the rectangular colour grids on walls, and floor arise from simulated sunlight entering the room through the windows, and pieces of furniture that Mondrian had included in his design pop out of the background walls as three-dimensional bodies in the VR salon, rather than appearing flat and static as in Mondrian’s drawings.
Initial Outcomes
During the 4 hours evening session, approximately 100 conference attendees visited the Mondrian VR demo, and exploration data were collected from 16 participants for observation of their first person view with fixations on a large wall-mounted screen. The visualisation of data outputs provided by the system includes a replay function, temporal sequence of fixations and cumulative heat maps showing the density of fixation on all regions of the 3D room surface, which can be complements by tracks the observer movements and head positions and gaze directions. Altogether, a comprehensively description of participant’s behaviour during visiting the virtual room is generated. Data were presented for individual participant, as well as the superposition of several observers.
Beyond online and retrospective visualisation, the VR system proved quantitative gaze data on regions of interest (‘colliders’) that have been defined for the environment: for the Mondrian room, these are defined for walls, ceiling, floor, and particular sub-regions of particular interest - such as colour rectangles of prime aesthetic interest - and windows where regions outside of the room are fixated, or pieces of furniture. This access to quantitative data provides the key to analyse fixation patterns in relation to composition elements and 2D vs 3D structure, and commonalities and individual differences of body, head and eye movement patterns.
Outlook
The VSS 2018 demo delivers a proof of concept that with the use of VR in arts environment we can address with empirical methods exciting questions about the relationship between conceptual frameworks of aesthetics and the subjective but quantifiable experience of observers interacting freely with the virtual arts space.
Future work will initially focus on a controlled experiment in the virtual Mondrian room, in which we manipulate the environment (lighting, colour schemes, physical layout, inclusion and position of furniture), and establish a structured structure data set to quantity observer interaction, from verbal reports about their experience, through the analysis of movement patterns, to the description of spatiotemporal gaze behaviour in the room – looking for indicators of processing mechanisms that flag up aesthetic conflicts and their resolution. Beyond these specific questions it will be exciting to explore how this novel approach can develop its great potential for a wide range of academic enquiry and for tackling practical problems of arts practitioners, as well as adding a cutting-edge methodology to the toolbox of archivers and conservationists.
