More selected projects

The Narrative in Cinematic Virtual Reality - Camera angle simulator

produced by: Andi Wang

 

Introduction

Since the technology developed and commercial company such as Facebook started their investment, virtual reality seems to become the mainstream of future trend. More film festival has also set up VR film award which means VR film has been included and accepted in the film industry. However, the public seems not that welcomed and satisfied with the cinematic VR experience and the directors also met a lot of problems when creating a VR film. One of the biggest challenges is that the storytelling principle is totally different from the traditional method. As there is already a lot of existence theory among virtual reality, this article aims to explore the practical storytelling approach in Cinematic Virtual Reality(CVR) and an artifact called CVR simulator is created to full fill the gap between the theoretical approach and practical usage.

The purpose of implement Cinematic VR is to present a better effect and to bring a more sensory experience. Vision, as a dominant sense, is the key to shifting consciousness from the physical reality to the virtual world. Besides, it is also the key to enabling another sense, the sense of presence. According to the experiment of the emotional effect of CVR compare with traditional 2D film by Ni Ding, Wen Zhou, Anthony Y.H. Fung, they found that four emotions are closely correlated with CVR, those are, namely, excitement,  nervousness, hostility and jitteriness. Therefore, this article will sharpen down the discussion point to the camera/eye position.

In this research, the first-person point of view refers to the audience to experience the movie through the eye of the main character. The third person point of view “clears that narrator is an unspecified entity or an uninvolved person who conveys the story and is not a character of any kind within the story, or at least is not referred to as such.” (Ricoeur, 2012)

Since the existed VR equipment can provide poor experience, and a considerable amount of VR users report that VR brings their dizzy feeling when playing for a long time. The suggesting length of the CVR experience will not as long as the traditional 2D film. This article will discuss through an ideal technology context and consider the length of the experience as an additional condition.

Before starting the discussion, it is important to claim that design a Cinematic VR piece is not about to design the narration or storyline as traditional 2D film, but to design an immersive experience. Besides, the CVR experience is also not like the VR game experience, it allows limited or even no interaction within the whole experience. The interactive narrative will be mentioned in this article but will not be analyzed in the artifact.

Literature Review

To define Cinematic Virtual Reality, it is to immersive the audience in a synthetic world experience in 360-degree. (Mateer, 2017) If classifies CVR as a category of film, CVR is more than a 360 video, while if to put CVR as part of virtual reality, it also has its limitation in the interactivity.

Immersive theater - John Dewey is the first person who brings the idea of the theory and athletics of immersive and interactive theater. He mentioned that it is the experience that significant stay with the visitors but not the image of the object itself.(John, D, 1930)

Sleep no more is an immersive theater production presented by the British theater company Punchdrunk, Emursive and Rebecca Gold production. The story is based on Willian Shakespeare’s tragedy Macbeth, not like traditional film or play, this experience allows the audience to move freely in a 7- floor building, exploring their own journey and creating their own story.

Traditional narratives

Linear storytelling and non-linear storytelling

Linear storytelling narratives in a logical manner by telling what happens in a linear timeline.“Nonlinear storytelling occurs when narrative events are portrayed out of chronological order or in ways such that the events don’t follow a structure where effects are the direct result of causality.”(Bucher, n.d.)

First-person view versus third-person view

First person point of view - telling the story through the main character’s position.

Second person point of view is not suitable to use in the CVR so it will not be discussed in this article.

Third-person point of view - telling the story in which the narrator is not a character in the story and the characters are also not aware of their existence. It can be specifically divided in third-person limited, third-person multiple and third-person omniscient.

Third- person limited - the character is limited in one.

Third-person Multiple - follow multiple characters.

Third-person Omniscient - narrator knows everything and is not limited only to what one character knows. But this angle will also not be discussed in this article.

“First, self- identification does not depend on the experienced direction of the first-person perspective, whereas self-location does. Second, bodily self-consciousness strongly depends on visual gravitational signals. Third, individual differences in the experienced direction of the first-person perspective correlate with individual differences in visuo-vestibular integration. ”(Christian. P, 2013)

The three illusion in VR

The three illusion is the key to promote the level of immersiveness.

Place illusion is feeling the presence in another place but not physically presence in the virtual world.

Plausibility illusion is related to the event in the virtual space. To be more specific, this illusion will appear when the character in the virtual world didn’t realize the presence of the audience. The audience feels it is happening while consciously aware that is not really happening.

Embodiment illusion is feeling the virtual body is the real body of the audience. This illusion all called the Pinocchio illusion because of the Pinocchio experiment. (Gillies and Pan, 2018)

The philosophy concepts

Metaphysics: a division of philosophy that concerned with the fundamental nature of reality and being.

Ontology: metaphysics concerned with the nature and relations of being.

Epistemology: he studies or theory of the nature and grounds of knowledge especially with reference to its limits and validity.

 

 

Discussion

Observer or participator?

As a presenting consequence, the audience is remaining audience during the whole experience, although the point of view can be shifted to a characters eye and sharing the body of the character. But can the audience be participator? The answer also can be a “Yes” when the whole experience didn’t disobey the notion of the CVR. The Best musical of Tony award in 1985, the Mystery of Edwin Drood requires the audience to vote for who kill the key character and creating seven different endings. This format makes the audience become a participator and affecting the endings, and this format is technically available to achieve in CVR. Besides, if put the camera view to the first person view, it will assign the another character identity to the audience and make the audience a real participator in this project.

To see this question through a philosophy angle, as Danaher points out, this ‘apple’ clearly exists in some form. It is not a mirage or hallucination. It really exists within the virtual environment. But its existence has a distinctive metaphysical quality to it. It does not exist qua real apple. You cannot bite into it or taste its flesh. But it does exist qua representation or simulation. In this sense it is somewhat like a fictional character. In other words, when we put on a VR headset and just engage in a VR film, our perceptions of reality are switching in subject, our consciousness has shifted to a virtual world that may be indistinguishable from our own. VR headsets allow the users to experience the sight and the sound of the world around them, but the human brain still gives it spatial importance, the brain is aware that every time we put on the headset, we are intentionally switching realities, hence why it is called virtual reality.

First person angle verses Third person angle during the whole experience

It seems that Cinematic VR has an awkward position in Cinema and VR area. To be analyzed as a film, it is not using the same notion to create and evaluate as traditional film. And if see CVR as a brunch of VR, it also limited the audience interaction in the virtual space. I prefer to take CVR as an independent category. As the form of its present, it resembles the form of immersive theater, which allows audience to choose their storyline but remain the role of an audience -  observing the irrelevant story but not involving in the story. For example, Sleep no more is the most suitable production to be filmed in VR. If implement sleep no more in a VR version, the audience position is a typical third-person angle - the audience is remaining the role of audience and stay outside the storyline, although the characters will choose some “special audience” to perform the hiding storyline.

Differ from the immersive theater, CVR allows the audience to observe the story through other position that impossible in the real world. As it is mentioned above, you can set the camera position on an object, or on a character in the story. According to the embodiment illusion theory, once the audience is having the same view and even same body with a character, we expect that the audience is able to share their conciseness or identity with the virtual character. However, according to Christian Pfeiffer’s experiment, “self- identification does not depend on the experienced direction of the first-person perspective.” That is to say, the audience will never feel the same and react the same while they were put inside an uncontrollable character.

The workflow of designing a Cinematic VR experience

“Your input in VR is you.” (Lanier, 2018) To design the experience, the key is to consider the whole experience through all the sensations that can be enabled in VR.

- Location

When an audience putting on the VR headset, being transferred from the real world to the virtual world, they will normally be looking around and try to figure out where they are and what should they do. Therefore, it is necessary to put the essential elements which have key information or clue of the whole story at the beginning and take that viewpoint to keep the audience engaged.

- Key elements/events

In order to keep the audience attention engaged, trying to set up one key element or events in one scene. The human eye can catch up with the key object quickly and following the movement, then drawing out the plot of the story will be natural and fluent.

- Camera position

In order to keep the place and embodiment illusion in the virtual space, the usage of third person limited view is reasonable and understandable. That is to say, the role of the audience is decided at the beginning and is unchangeable through the whole experience. Once the role of the audience is changed without cushion, it might confuse the audience of their identity in the virtual space. It also expected to work well with the linear storytelling principles. On the other hand, it does not mean that the third person multiple is the worse choice. In fact, sometimes it is necessary to use third person multiple when implementing a non-linear narration. But the scene-switching frequency is not suggested to as high as the traditional 2D film.   

To connect them all together, the director needs to consider the reason of the audience presence in a certain environment. All the elements are designed to serve the audience experience and emotional feeling. The suggestion to design this part is to experiment and rehearsal in the real world as much as possible. This is the most reliable and intuitive method to design the whole experience. Therefore, the artifact, CVR simulator is created in order to visualize and fill the gap between the theory and practical usage. It is a simulation system that have the capacity to adjust the camera angle and position, simulating the experience and attain the goal of creating a CVR prototype. Besides, the system also allows the director to conduct user testing in a fast and practical approach and experienced the narrative directly.

 

 

Conclusion

In conclusion, the key of designing narrative of CVR is sharing the same logical process of designing an experience. The immersive theater has the similarity with the CVR experience as they both regard the audience as observer and participator at the same time. Camera, as an eye of the audience, is the essential media to shift the consciousness from the real world to the virtual world.  The camera position requires to be carefully selected because it can provide different emotional feelings during the whole experience. The camera position is closely related to the place illusion, plausibility illusion and also embodiment illusion. The three illusions, as the key to determine the level of immersiveness and the sensational experience, should be seen as evaluated standard when choosing the camera position.

The camera position could be divided into first-person view, third person limited and third person multiple. It is able to bring different experience and emotionally affect the audience. The chosen of this camera position is depends on the content of individual CVR experience. The artifact is a reflection of the camera position, illusions and the experience. It allows the director to simulate all the camera position mentioned in this article and achieve the goal of combining theory and practical usage.

 

 

References

Bucher, J.K., 2018. Storytelling for virtual reality : methods and principles for crafting immersive narratives,

The author, an award-winning writer and a narrative consultant, write this book in order to serve as a bridge between new media student and professional working in the emerging VR technology and the art form of classical storytelling. He concludes an all-around process of designing the VR experience. 

Heim, M., 1993. The metaphysics of virtual reality, New York ; Oxford: Oxford University Press.

The author, known as “the philosopher of cyberspace”, looks into this new technology through a philosophical way. He explores the logical and historical origins of our computer-generated world and speculates about the future direction of our computerized lives. 

Lanier, J., 2018. Dawn of the new everything : a journey through virtual reality

The author, a computer philosophy writer, founding father of the field of virtual reality, reflects his own lifelong experience with the technology. He explored and explained this technology through a poetic and romantic way, presents how virtual reality went from a dream to reality in the last three decades.

Henrikson, R. et al., 2016. Multi-Device Storyboards for Cinematic Narratives in VR. ACM Symposium on User Interface Software and Technology (UIST '16), pp.787–796.

Christian Pfeiffer et al., 2013. Multisensory origin of the subjective first-person perspective: visual, tactile, and vestibular mechanisms. PLoS ONE, 8(4), p.e61751.

Dewey, J., 2005. Art as experience, New York: Perigee.

Ricoeur, P. (2012). Time and Narrative. University of Chicago Press.

Mateer, J., 2017. Directing for cinematic virtual reality: how the traditional film director’s craft applies to immersive environments and notions of presence. J. Media

Gillies, M. and Pan, X. 2018. Introduction to the Three Illusions - The Psychology of VR: the Three Illusions | Coursera. [online] Coursera. Available at: https://zh.coursera.org/lecture/introduction-virtual-reality/introduction-to-the-three-illusions-AeyX9 [Accessed 12 May 2019].

Tricart, C., 2018. Virtual reality filmmaking : techniques & best practices for VR filmmakers,

Danaher, John, 2017‘The Reality of Virtual Reality: A Philosophical Analysis’.

Ding, Zhou & Fung, 2018. Emotional effect of cinematic VR compared with traditional 2D film. Telematics and Informatics, 35(6), pp.1572–1579.

Punchdrunk. (2011). Sleep No More — Punchdrunk. [online] Available at: https://www.punchdrunk.org.uk/sleep-no-more [Accessed 13 May 2019].