Iris.Fall—a 3D puzzle game with an extraordinary art style and a core concept of "light and shadow." To match its unique graphics and delicate mechanical devices, the NExT audio team hopes to use experimental music as the keynote and timbre to give the game a one-of-a-kind feast for the ears that combines unique instruments and devices, bringing the "dark tale" set in a dilapidated theater an acoustic texture and imagination. As it turns out, the result is satisfactory!
In this excerpt from the Music Design and Production section in Iris.Fall's developer diary. Mr. Zhang Xin, the project's audio designer, has summed up his thoughts and methods in the process of music design and production, and feels honored to share his experience. Mr. Zhang Xin has 10+ years of experience in game audio design, and he has participated in a wide range of projects covering most platforms and genres. At present, he holds the position of senior audio designer at NExT Studio.
In Iris.Fall, players travel back and forth as Iris through the labyrinth to solve puzzles by utilizing mechanics or placing objects. In most cases, she would simply stay in one locale and interact with it. Most of the puzzles are static, and the game maintains a slow, steady pace. Meanwhile, the game adopts a linear narrative with each chapter having its own fixed locale. How do you make music for a linear static puzzle game?
At midnight, Iris wakes up from a bad dream and follows a black cat into a dilapidated yet fantastic theater. As the black-and-white-toned story unfolds, she discovers and recalls some of the clues that are connected to her fate—she's a puppet who was granted life through magic.
Granny was the mage who granted Iris her soul. However, these souls were taken from other children by magic. Only in the last act will Granny reveal herself. Before that, she makes her presence felt in photos and memories.
There are other characters in the game—the Black Cat, Iris' Shadow, and the children whose souls were taken.
1) Gain adequate appreciation of the game and deconstruct it by level
Deconstruction allows you to get a clear view of what to do and where to do it, and make changes in the
process of development. For instance, two chapters have been removed from the original plan. For each level,
I usually sort them into one sheet by the order of cutscenes and interactions.
2) Search for clues—a single or multiple event(s), item(s) or character(s) that connects the whole
story.
This is an interesting part. Music can be designed or expanded based on these events or items. Every audio
designer has the freedom to define the clues in a way that suits their own tastes and habits. Iris.Fall has
a treasure trove of clues if you look for it—the cat that makes an appearance in every scene, Iris' shadow
in the later chapters of the story, the eye elements that frequently appear, etc.—all of them are
jumping-off points in design.
With these clues in hand, we can summarize their similarity or connection, and identify a theme. For instance, in this game I have chosen a float as the theme of music, and Home as the subtheme.
1) The Float
The float is one of the themes I chose. As a piece of décor, it can be found in the front, mid, and later part of the game.
At first, it appears in a dilapidated basement. Players are required to open the door at the middle of the float by solving the puzzle.
In the middle part of the game, the float is found in a 2D cutscene. Iris accidentally intrudes on a scene in a grimoire, where she finds a group of children gathered at a fascinating puppet show. The kids are thrilled at the sight of the candies on the float. Two puppets holding needles and threads join the show as it progresses, while buttons and papermen dance in the air. The buttons fall upon the children's eyes, sewing up their eyelids and taking away their souls. The idea of buttons draws its inspiration from the movie Caroline by changing the eyes into the buttons and children into the puppets. This is a crucial plot point when Iris realizes that Granny's grimoire might be the cause of everything—she might be the one who deprives the children of their souls. It's a big twist of the scenario while the float is sort of a witness. A puppet show always comes with music, but what music would suit it? It can be somehow related to the entire story—I came to this conclusion when I was sorting out the clues, which is one of the reasons why I defined the float as the game's music theme.
In a magnificent show in the last level of the game, the children with lost souls spin and dance under the command of the giant puppet, which got me thinking about adding a piece of variation to the float.
The theme was selected after rumination.。
A01 A preliminary concept of the float theme
A02 Music of the float device
A03 A hint of the float theme in the menu
A04 A variation of the float music in a 2D cutscene
A05 A variation of the float music in the last level
2) Home
This is another theme of Iris.Fall's music, which takes the following aspects into account.
Granny makes her first appearance in a small attic that is beautifully and warmly decorated as a girl's room. In this room, Iris finds a book that records her close relationship with Granny.
In the middle and later stages of the game, Iris enters a carton with a variety of toys hidden behind every door. The carton itself is in fact an origami toy. Between the toys and the cozy attic, there is a room for imagination.
After the carton falls into the abyss, Iris bears witness to a merry-go-round, which is a step closer to dream or illusion. The merry-go-round, origami, toys—all of these clues reflect the relationship between the girl and the theater.
In light of these reasons, it would be more natural to connect these scenes with the theme of "Home".
From the perspective of music, the theme of "Home" drops a hint that the girl once lived here and has sweet memories connected to the place. The tone for the whole game is set as desolate, dilapidated, and even depressing, while the "Home" theme brings warmth to narration so that the dynamic range of emotion is enlarged, and a sharper contrast is brought out.
B01 The theme music "Home"
B02 Application of the theme music in the carton scene
B03 Specialized processing of the theme music in the merry-go-round scene
These two themes are enough to support the story-telling. After that, we can make music according to different scenes and stories.
The following list shows the summary of all in-game chapters. The intensity and genre of the music in each chapter is distinguished by color so that the pacing of music for all levels is self-explanatory, and it will be easier for us to control the music intensity and density. We can also quickly build up a session in DAW or Wwise to preview the musical changes in levels to ensure the fluency and rhythm of music.
Throughout the process of music design, we keep faith with the principle of one strength for each level—either being interesting or stunning. From this chart, you'll have an overview of the key points and strong points in design, as well as their timings.
The following chart sets the basic tone for every level. At the early stage of design, the minimum requirement is to guarantee a change of tone at every level, so that players won't feel bored staying in an unchanging environment. The tones for every level are tailor-made for the stories and scenes. Meanwhile, I've also separated one tone from another. For instance, the first level is set in a dilapidated basement, so we paint the colors cooler; When she comes into rooms, the tone will become a little bit warmer. Though the room still feels ghastly, I hope the puzzles here give players a warmer sense. To set the mood of "Home," the two pieces of music at this level are created to feel lighter and warmer.
A chart like this makes it easier to assess the game's overall tone and story progression, and it sets the pace for post-production. Additionally, at the early stage of development for these games, it's not easy to get things in order among countless puzzles and story pieces. Reviewing your schedule from time to time helps to keep your mind sharp.
We've introduced some new approaches in terms of Iris' music design—by modifying some instruments to change their tones. Most of the regular instruments are pleasing to the ear, but the references we prefer are mostly based on samples or devices. Such pieces of music are unpolished in general, and lay stress on the atmosphere. The noise produced by musical instruments and devices is also a part of the music. Therefore, in terms of the principle of design, we maintain our previous techniques and styles while also overhauling the timbre of instruments.
1) On the guitar strings, install a small screw on one side of the strings to keep them in place.
2) Clamp the guitar strings, and use Ebow to produce the sound of Pad.
3) Clamp the piano strings. Use guitar strings as a bow to draw across the piano strings, creating a super-cool sound that is also frequently used during post-production.
Once the experiments are completed, we sample these refreshing sounds and save them for post-production.
Zither is an instrument similar to the dulcimer. A common approach online is to use steel cans as the diaphragm, connect guitar strings to the bottom of the cans, and resonate with the cans by hitting the guitar strings. Technically, the steel can is not a regular dulcimer or cavity. It produces a rough sound, with lighter notes in the upper register, and deeper notes in the lower register, yet it makes a superb instrument for Iris.Fall. To facilitate its performance, we've added a damper to the zither to make soundproofing easier.
The resonance violin is a very bold idea—belt-sand the back of a violin and install a metal membrane. In the upper register, it sounds similar to common violins, but in the middle or lower register, its timbre becomes quite interesting—hoarse, rough, and vibrant, in deep contrast with a common violin's gorgeous sound.
First of all, we arrive at a consensus with the project's core team over the principle of design. Next, we begin to look for the music team. Having some insight into their style and skill, we need to double-check with them regarding the direction we're heading in. We always gather as many materials as we can at the early stage and only make the next move when we're sure that they fully understand what we want. During the pre-project period, we shared with them our experience in modifying instruments. After a long series of careful experiments and repeated siftings, the final design was eventually determined.
When the composition phase kicks off, we would ask the composer to take the positioning of instruments into account by leaving spaces at the beginning or replacing them with similar instruments, such as harps or violins.
When the composing and recording are completed, we will receive all stems of each music piece. I will double-check the balance of different instruments, and put the experimental timbre or phrases backward in the mix. So that normal players won't find it overwhelming and hard to accept. In the process of composing, I asked the composer to make some music phrases out of tune slightly. I personally love it, but after putting myself in the players' shoes, I put all those challenging parts backward in the mix. Coming back to hear it again, I feel a little pity for some of the removed parts, but ensuring the safety of our product is always my top priority.
Creating game music is engaging and demanding, especially when you're trying to sort out clues—hide them throughout the game while telling a good story. When it comes to the tone color of instruments, we've introduced some special approaches to add a new flavor to the music, which are very exciting and satisfying.
-END-
The joint research paper on cloth simulation from NExT Studios and Professor Xiaogang Jin’s team of Zhejiang University State Key Lab of CAD & CG, “Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks”, will be presented in the SIGGRAPH 2022 conference, and the demonstration has also been selected by the Technical Paper Preview collection of the conference. Fast forward to 3:10 to have a look!
Clothing plays an important part in a digital character’s appearance. The dynamic deformation of loose clothing can help express a character’s emotions and show their personality, such as the floating, swirling, and falling of a dancing character's skirt. Cloth animation is an important subject in computer graphics. Tight clothes usually fit snugly to the surface muscles of the body and can be approximately driven based on the skeleton’s movement. However, loose clothes usually have a certain distance from the body, and the combined actions of external forces and various body movements will result in complex deformation and collision. Consequently, there have been no good real-time solutions.
This research improves upon the current real-time cloth simulation method of loose clothing based on deep learning, which helps to improve the real-time animation quality and artistic expression of digital human and in-game characters. In the upcoming SIGGRAPH 2022 online conference, which will be held in early August, we will introduce this cloth simulation technology in detail. See you then and look forward to its coming!
Zhejiang University State Key Lab of CAD & CG
Founded in 1992, it is a world-class computer graphics laboratory established as part of the National Seventh Five-Year Plan. This institution is mainly engaged in the basic theory, algorithms, and related application research of computer-aided design and computer graphics. Over the past two decades, relying on the disciplines of computer, mathematics, and machinery of Zhejiang University, the laboratory has undertaken and led a number of major national scientific research projects and international cooperation projects, and in doing so, it has made a number of important achievements in the basic research and system integration of computer-aided design and graphics.
Professor Xiaogang Jin
Professor and doctoral supervisor of Zhejiang University’s College of Computer Science and Technology. He is a chief scientist of the "13th Five-Year Plan" national key research and development program, the director of the ZJU-Tencent Game Joint Lab for Intelligent Graphics Innovation Technology, the chairman of the Zhejiang Virtual Reality Industry Alliance, the vice chairman of the Virtual Reality and Visualization Special Committee of the Chinese Computer Federation, and a distinguished expert from Qianjiang, Hangzhou. He has published more than 140 articles in ACM TOG (Proc. of Siggraph), IEEE TVCG, and other key international academic journals, and has been awarded many accolades both in China and abroad.