¿What Is Reality?
Senior Thesis Self Exploration Into XR
XR opens an entirely new world of possibility and breaks the bounds of what reality truly is.
Reality for most is a culmination of their sensory experiences. In XR, the three senses which are commonly affected are vision, touch, and sound.
Three code-based WebXR experiments each primarily built around one of the mentioned three sensory perceptions in VR.
Experiment 1 was split into two mini experiments revolving around vision and sound. I wanted to better understand through observation how accurately people were able to understand their sensory perceptions. For the visual portion I asked participants to estimate a distance based on a predetermined distance. For the sound portion, I dropped an object on a grid and asked participants to select the marker which the sound had come from.
From these experiments I found that without multiple objects to use as reference, we have difficulties identifying and analyzing what we are seeing. I also learned that although we are quite adept at hearing direction, we cannot hear distance or depth well.
Experiment 2 was also split into two mini experiments, but this time revolved around vision and touch. This experiment was a bit different as I wanted to further observe how different senses aid one another. Thus, I was observing how others make cross-sensory perceptions. The first portion was asking participants how they thought an object felt based off of how they visually perceived it. Second, participants described how an object looked based off of how it felt in their hands.
From experiment 2 I learned that sound/touch/vision all relate to one another similar to taste/smell. They are nearly inseparable for those who have experienced them together their whole lives and they are all pieces of one whole sensory experience.
Experiment 3 was about making simple but focused observations about one's surroundings/ sensory experience first with their eyes open and then with their eyes closed. Participants were asked to try and describe what they saw, felt, and heard in their environments without using pre-concieved definitions such as "desk" or "chair". This was to allow participants to truly think about what they were seeing and perceiving through their senses. They were then asked to make the same observations but with their eyes closed.
From this experiment I learned that each person's reality is different. Regardless of how well you describe something to someone, they will not perceive the same thing unless they are experiencing the exact same thing. Even then, it is unlikely. Additionally, we heavily rely on preconceived definitions and our perception of reality begins with what we are seeing.
.gif)
I decided to continue by creating three virtual experiments to experience the same three senses I had physically observed, but digitally. To do this I decided to use Three.JS, but as a novice coder I needed some practice first. Thus I learned some basics; how to animate, position, rotate, and group objects. I also learned how to resize the window and give the camera full movement controls.
.gif)
The first of my virtual experiments was the haptic cubes. These cubes primarily revolve around the sense of touch. To visually aid the perception of weight to the cubes, I first added darker color for the "heavier" cubes. Additionally, the heavier cubes move and rotate slower. The true sensory experience occurs when one touches the cubes. They provide varying levels of haptic feedback further indicated by their visual weight.
The second of these experiments is spatial sound. It primarily revolves around sound with some visual cubes/walls to orient the user. For this experiment, I separated a music experience into four of its instruments. Each instrument is placed at one corner of the space and you can alter the way you experience the music by moving around in the space. This give depth and space to the sound.
Still a work in progress, the third experiment focuses primarily on visual cues with some haptic and audio feedback. You can create your own sensory experience by touching the spheres. Each sphere will be a random color and size, with each color indicating a different sound and each size indicating the haptic feedback that will be given.
