I got some suggestions for changes in this week’s presentation. The first is the percussion sound at the beginning, it is not in sync with the picture and I need to make them consistent. Secondly, there needs to be some seagull sounds and human voices (ambient) in the background. Some wind sounds need to be added to the sky.
The hit sounds are consistent with the picture.
I added the sound of the wind and some wing stirring. This makes the overall sound richer.
Some of the VR films currently available are ‘ABE VR’ and ‘HELP’
I was looking up and VR related material and came across the content. As I am personally inclined to work in film scoring in the future, VR film is also a direction I may come across, as the name implies the use of panoramic filming to expand the space of the picture, the distinction between in-picture and out-picture space no longer comes from the director’s restriction of the viewer’s field of vision, but from the viewer’s own choice. Sound and picture must be perfectly matched in terms of spatial relationship, accurate positioning and a sense of space, including the positioning and presentation of sound horizontally, vertically and vertically. I believe that sound production needs to take into account every source of sound in the panoramic image, so that there is no longer a distinction between in- and out-of-frame, and that the sound from every direction may become the focus of the audience’s attention. It was a very interesting project, and it made me think about how the soundtrack can act as a propeller of the overall atmosphere when people are watching the film from a third perspective. However, when the viewer is in the first view of the film the soundtrack should be diminished (ambient sound over background sound) to give the viewer a greater sense of immersion.
But in any case, the VR mode of film viewing heightens the audience’s sensitivity to sound and places greater demands on sound production. Sound detail needs to be further enhanced, detailed sound effects are useful to enhance realism and immersion, and the accurate positioning of object-based sound keeps the audience from getting lost in the process of finding the sound source. Sound is one of the most important organs of perception, and if the sound and soundtrack are well crafted they can be immersive. The important technique is to give the sound a sense of space and accuracy to the scene the audience is in.
The use of head tracking allows the sound to interact with the viewer’s perspective, making the audience believe that the sound exists independently of the space they are seeing. This is the same sense of spatiality that I mentioned in my previous blog, where the spatiality of VR sound is based on a three-dimensional space, and the person in the sound field needs to feel the reflections from all directions, both horizontally and vertically, to get the illusion of being there. There is also the enhancement of detailed sounds and ambient sounds, and the exaggeration of small sounds to make them more impactful.
All in all, I have a lot to learn. I’ll also be working on more software in this area, including Head-Tracking and the like. This collaborative project has really given me a lot of food for thought about my future plans.
This project was a collaborative project with animation students. The difference with seagulls (VR) is that when I approached seagulls they were about 70% done and the animation team was just at the beginning. I can say that I was involved in every stage of the anime, from the first draft idea -> the first version of the music -> the second version of the music, there was a lot of communication. Jingya was in charge of the ambient sound and mixing for this project, and I was in charge of the music. This is my first draft of the music for the anime.
Music:
None or give a short melody in the first twelve seconds when entering the forest (it can be the main melody of the theme song, and the theme song can be placed at the end when the staff list comes out)
There are two places where the music is very much in contrast to the first version and I have made significant changes. The first is the opening music. Because the animation is so short, the opening music basically sets the general direction of the melody of the short. My first version used Tone Hammer so it sounded upbeat and had a light-hearted feel to it. However, when I saw that the final tone was dark and the story was thought-provoking, I changed the music. The second area of change was the monster. The drums sounded too monotonous and had no resonance, so I added some sound effects and removed the drums. After discussing the effect with them it was the second version of the draft that was created. There is a clear difference in comparison with the old version.
Finally the complete version came out after more careful adjustments
Final Version
I found some very interesting vr projects when I was looking for vr games. The first is Soundstage which is a virtual music production studio based on VR technology. With features such as virtual instrument playing, recording and mixing production, it can mimic the professional music studio that musicians need. For example, MIDI keyboards, Roland 808s and a range of synthesizers are at your fingertips. You can also choose to use various instruments in this app, such as drums and electric piano, while experiencing the virtual world of matching and connecting these instruments together with glowing virtual cables, an experience you won’t find in production software in the past. I think this is really interesting, as you can really experience the physical properties of the devices, instruments and cables in the virtual world without any worries, as well as the realism of hitting the instruments.
The second game is Block Rocking Beats, a VR music app released in early July 2017. In this app, users can create music using instruments such as drums and guitars, as well as post-production such as mixing, but what makes it most special is that it supports multiple people creating at the same time and allows those people to communicate in real time, effectively, and share their ideas in creating electronic music. The design of the finished music compositions in this app, which can be saved and downloaded to other platforms for further use, makes perfect sense. When it comes to music technology and creation, it also has several features to be improved, and the developers say that subsequent versions will also support sharing the music they create to other users.
In such a music studio, creators can select various virtual instruments and their timbres, and enter notes or electronic sound samples into virtual tracks and sequencers, or record virtual instruments being played in order to compose. The basic principles of electronic music composition do not change much here. On this basis, further code detailing is required by the developers for the different ways of playing the different instruments, a step which is also clearly possible. For example, to play a violin in a virtual world involves techniques such as holding and dribbling the bow and pressing the strings with your fingers, which is currently implemented in a bowling-themed game for the HTC Vive. In this game, you can use the controller with the joystick to play the game. In this game, picking up the bowling ball on the ground in the virtual world with the joystick controller is almost identical in code to picking up the bow with the joystick controller; the action of swinging the bowling ball, then swinging it forward and finally throwing it, is also very similar to the bowing action of a violin, except that in the case of the bowling ball, a parabolic design is used, whereas in the case of the violin, the angle of the bow is taken into account. The change.
I think that in the development of the vr program it would be possible to create a special playing mode for creators who do not play the instrument, and also play it with acceptable quality. For example, the user does not need to use the real playing string press to achieve the sound effect at that force. Engineers could even develop a function that would automatically process the technical aspects of the intensity of each note played by the player, possibly using some of the results of artificial intelligence. The technical aspects of music composition in VR still need to be improved, and boldly so, and the spectacle value of VR performance will require higher demands in terms of the quality of the sound of the virtual instruments, the sensitivity of the playing operations, the aesthetics of the playing environment, and so on. Having a full range of virtual music creation studios on top of the technical implementation would be a great convenience for the electronic music business. I would love to try it out.
MA, Communications Art & Design, Royal College of Art, London
BA Honours, Sonic Arts, Middlesex University, London
PhD in-progress, GSA, Glasgow
Lo-fi. Field Recordings. Collage. Sculpture. Installation. Expanded Cinema. Photography. Time Medias. Printed Material. Hacking Culture. Interactive Technologies. AI. VR. AR. Gesture Sound. Mesmerism. Post New Age. Post Net Clara Rockmore. Perspective. Immersion. Affect.
lesliedeere.com. 2022. Modern Conjuring for Amateurs | lesliedeere.com. [online] Available at: <https://lesliedeere.com/modern_conjuring.html> [Accessed 16 May 2022].
I found this project very interesting. The purpose of this study is to induce an altered state of consciousness by utilising Using music, frequency, light, and the colour spectrum, this piece tries to produce a peaceful or contemplative As the artist travels about the venue, dazzling ribbons of colour appear and are sent to audience headsets. Sounds and tones are produced in unison with the performer’s motions and are broadcast over a high-quality audio system. The vr experience is not just for one person. The vr experience is no longer for one person, but for a group of people. Everyone can experience it at the same time through this vr game. Similarly Leslie Deere has taken great care in the sound aspect of the vr game. There is a variety of instruments used including harps and cello’s, as well as some vocals added in. The overall feel of the music is ethereal and I personally feel that there is a bit of a religious choral element to it. These combined with the colourful vr graphics I felt went very well together. It gave me a very strong sense of immersion. Overall I thought it was amazing and fascinating.
What I can learn from this is the concept of group vr. This kind of interactive game is very appealing to me. I’ve seen some videos with group vr before but they weren’t as interactive as this one, more other people watching the vr performers on a TV screen rather than all wearing vr’s. I think Array Infinitive brings a little more immersion to the viewer. But I think Array Infinitive is more of a vr show than a vr game. The audience can immerse themselves in her performance, and every movement and every pose of the artist can bring the audience a different visual and auditory experience. Inspired by the video ‘Array Infinitive: VR performance by Leslie Deere’ I think the vocal padding and a little bit of reverb really makes a difference to the overall atmosphere of the music. Especially in a somewhat abstract visual art. So I tried adding some vocal melodies to some of my previous music (some half-finished pieces) and it made the whole thing sound a bit different. The whole thing sounded a lot more complete.
I got a lot of suggestions and things to change in this tutorial. There are two areas where I need to make the most changes, i.e. where I have the most problems. The first is that I need to clean up some of the ambient sounds in the audio. This is because some of the ambient sounds are overlaid on top of each other, making the whole audio sound confusing and the presentation not very clear. The second is the change in the background music. This is because there is a mini-game included in this vr game. The shift in sound between the two games needs to be a more pronounced change for the player. This would make the whole soundscape sound richer.
I also got some advice about the blog. I can put together a list of what I need to read each week and post my thoughts.
Rachel Simpson is an experienced composer and sound designer living in Dundee in Scotland with over 15 years in game development. She create music and sound effects for various media including video games, television, animations and any other media in need of unique, creative audio. I’m also a performing and recording musician.
She work draws from Jazz, Classical and Pop music with a strong harmonic basis and includes a long standing collaboration with Dundee recording artist Andrew Wasylyk, performing on his 3 most recent records (including the Scottish Album of the Year nominated The ‘Paralian’).
Rachel Simpson. 2022. Rachel Simpson. [online] Available at: <http://www.rachelsimpsonsound.com> [Accessed 19 May 2022].
I could feel during Rachel Simpson’s talk that she is a very good artist. She has created music for many games. I also learnt a lot of technical aspects of her experience in creating music during her talk. I also got to see some of her sharing and advice on sound effects. Rachel Simpson in Sweetgrass AR: Exploring Augmented Reality as a Resource for Indigenous people mentioned that “AR has the ability to reveal and criticise settler colonial images while evoking relational ethics and Indigenous ways of knowing in the context of Indigenous- settler interactions. However, it also has the potential to spread disinformation and commodify Indigenous Knowledge.”
This critical thinking is also a topic that is being researched throughout week24.
I read some of the articles given by week24 starting with 5 key considerations for ethical virtual reality storytelling – Caroline Scott. in this article there are five things to look out for in vr and they are
While I agree with some of the above points, I have also looked up some other sources on this topic and I believe that there are several factors that make games ethically problematic. The first is the subjectivity of the creator of the game; VR is primarily a simulation of reality. The extent to which virtual reality is used in the gaming industry, how realistic it is, and whether there is any subjectivity on the part of the creator. To analyse these questions we need to return to the level of VR’s rendering of reality. The size, shape, colour and perspective of objects in a virtual scene are designed by the creator. Is the realism in the design of the scene, different from the reality? How does this subtle variability affect people’s understanding of how real things are? Perhaps it is not a question that people care about. But it is incumbent on the game producer to explain to people what details have been left out and what details have been designed. Obviously, game creators cannot avoid their own subjectivity permeating the production process of virtual games, and both the motives and biases of the creator will be reflected in the VR gaming process. Maintaining objectivity in VR games is a code of ethics for game creation teams and is about a clear understanding of the facts.
The second is virtual reality technology and emotional sensitivity. the strength of VR games in their ability to shape stories is significant. the high perceived focus of the game’s backstory by VR technology can lead to emotional sensitivity. People are immersed in the narrative space constructed by VR games, concentrating all their attention to highly perceive everything in the game. If the content presented in a VR game is pleasurable in nature, then this can easily contribute to people’s emotional cohesion. The potential impact of VR on people includes the potential to have an emotional impact on them. When people focus all of their attention on the virtual space, it intensifies their emotional responses to the game story content. The speed of people’s emotional reactions in virtual spaces is much greater than in real spaces. To avoid the stimulation of emotional reactions in virtual reality technology, it is necessary to manage the content of virtual reality game and follow the appropriate balance to avoid the risk of emotional generalisation.
As a sound designer the most important ethical principle I have to respect is originality. I think it’s not just sound designers but all those who work in the arts that should respect this code of ethics.
Immersion is the process of full mental concentration, the development, change and transition from one state of mind to another. The reduction of the display distance from the object being displayed increases the emotional engagement with the current event. In the study of virtual reality technology, “immersion” stands for making the user feel as if he is in the episode displayed on the screen”, “the user feels as if he is a part of the virtual environment and not an observer. He feels surrounded by the virtual scene and can look to the right and left, move freely around the environment, and interact with the objects as if he were in the real world, which he has already experienced”. In the current field of research, physiological immersion refers to the use of virtual reality technology to provide the user with visual, auditory and haptic sensory perceptions in order to make the user feel present in the world constructed by the virtual reality technology, a sense of ‘presence’. The term ‘immersion’ in psychological immersion refers to the psychological satisfaction of the user during the interaction, i.e. “an experience that is suitably challenging and immerses a person so deeply that they forget the passage of time and are unaware of their own existence”.
In almost all VR scenarios, there is a virtual image of the user themselves, which initially exists as an environment in relation to the user’s real body. vr gamers perceive and recognise this through a series of activities that lead the user to a sense of self-identity. If what the player body perceives and what is inherent is unified and believable, then the user will naturally integrate into the VR. This sense of identity needs to be maintained in many ways to be achieved.
I believe that the first thing that needs to be taken into account when designing sound in VR is the proximity of the sound, or the sense of space. Hearing is one of the main ways in which people acquire information. Therefore, the study of sound is one of the more important issues to think about in terms of how to create a sense of immersion in the virtual reality process. Sound can be described using interventions on perception and through the organisation and construction of perceptions and cognition. A hierarchy of contrasting strengths and weaknesses of sound will provide the brain with a very complete spatial imagery. Therefore we should not only demand the presence of sound, but also higher requirements for sound, such as more realistic detailing; displaying sound that fits the current position of the observer and follows the line of sight; ambient sound that gets different levels of nuance, etc. Another thing that can enhance immersion is sound surround. Virtual surround sound technology uses two-channel stereo hardware to create a simulated three-dimensional sound field without increasing the number of channels or sound materials by circuitically processing the sound field signal to make the listener feel that the sound is coming from multiple directions. The sound field can be reconstructed with head rotation and movement.
I’ve played a lot of games on the switch and PC (steam), not a lot of vr games. I’ve only played less than 5 games since vr gaming technology came out, so I don’t really have much experience with them. I will however describe a few of the vr games that have impressed me the most and my most recent experience with vr games based on what I will write in the next few posts.
The first vr game that impressed me the most was ‘Beat Saber’. This was the first vr game I played and is by far my favourite vr game. It’s a musical beat game. It contains various genres of music such as hip hop and rock, and uses lightsaber swings to hit the rhythm cubes. Combined with the exquisite scene modelling and rhythm pattern changes within the game and scene changes such as bombs gave me an immediate immersive environment gaming experience. I received feedback for every move I made and the fixed point gameplay prevented me from getting motion sickness from the game. I was also pleased with the tactile aspect. When the lightsaber simulated with the joystick hit the blocks, the joystick emitted a subtle vibrating sensation and I got feedback on every move I made. I’ve actually played this game on steam (PC) without the vr version, so I was very excited when I found out it was coming to vr. As it turned out, Beat Saber’s vr game did not disappoint me, both in terms of feel and graphics and music. And because it’s a Chinese/Chinese collaboration game, ‘Beat Saber’ also has a lot of Chinese songs.
The second game I would like to talk about is ‘The Walking Dead: Saints & Sinners’. It’s a bit more technically challenging than ‘Beat Saber’. I played this game because I saw a lot of people recommending it and I was interested in the zombie genre so I bought it. Personally, I think there are a lot of things that make this game shine. For example, the killing of zombies, although I personally found it a bit disgusting when I played it, I have to admit that the brains and guts of the zombies are well rendered in detail. And most importantly, the tactile feel, with each weapon having just the right amount of weight and texture. Heavy weapons such as axes and rifles require two handed grips for added stability, while smaller weapons such as hunting knives can be operated with one hand, making it easier to make precise attacks. I’ve only played the game a few times though, firstly because it’s open-ended but still revolves around a closed loop of exploration, scavenging, killing and crafting, and the main quest to find the stockpile can seem thin. The second is because the immersion that vr games bring to me, the feeling of being surrounded by loss is scary and I’m afraid to play alone.
The third is the game I recently played, ‘HITMAN 3’. I didn’t bring my vr device with me but my friend brought it and bought the game so I borrowed my friend’s stuff to play it. I played ‘HITMAN 2’ on the computer and it was really fun and it was very playable. I had high hopes for the ‘HITMAN 3’ vr game. But when I finished it I was actually a bit disappointed. It was very difficult to manoeuvre and it felt like the characters couldn’t follow my perspective. My friend told me that when she watched me play it, I felt like I was in a weird position, like I had new limbs that hadn’t been trained yet. On top of that I’ve seen a lot of negative comments about the graphics quality. I didn’t really notice this when I was playing because I’m a bit nearsighted with astigmatism so I always thought it was my eyes that were the problem. I think the biggest problem with ‘HITMAN 3’ is that it doesn’t integrate into the vr game and the overall feeling while playing is fragmented. It was very strange to have my eyes and ears in the vr game but my hands still on the keyboard.
Finally, I think the most important thing about vr games is the sense of immersion. The synchronization of picture and sound, the synchronization of picture and touch. You have to imagine all the parts as a whole to have a better gaming experience.
When I first saw the title I thought that this project was going to be a 3d adaptation of an anime. But when I saw the Cartoon Adaptation Seagull I was completely out of ideas because it was a very short four-panel cartoon. I don’t know what the MA students are going to do to make this four-panel cartoon into a vr effect. But I think they would need some kind of sound effects like bird calls. I was a bit nervous about working with them at first, it was my first time working with a vr professional and I didn’t know if I could do what they wanted. Later on in the group meeting they showed me a more complete piece of work and I thought it was really incredible. They created a whole new world through that four-panel comic. In the video they showed us, you could see the urban environment, some of the buildings like cafes, overpasses and residential buildings. It’s a vr effect that I hadn’t envisioned at all. They also said they would create a mini-game within the vr game, which was interesting to me. I was also surprised at how well they were able to do it. I was very happy to work with them and the high level of completion of their project was a big positive for me. It also helped me to finish the sound work faster and not to give them any trouble.
They have a very clear goal regarding the sound. One of the things I found most enjoyable about working with them was that they knew what they wanted. I think that’s a very important thing in group work, is that you can be clear about what you need or want. Instead of not having a clear idea of what sounds you need and not having a clearer description of what you need. They made a list of the sounds they needed, which was very detailed and had clear categories of sounds. This made it much less difficult for us to produce, as our group was a bit heavier than the others because there were only two of us. This list of sounds really took the pressure off me. It also allowed me to focus more on the production process and to get to the point and fit in with the goal. Overall I feel very confident in meeting them for the first time and I hope I can make a sound that they will be happy with.