Musitation - A Social Experience in VR for Collaborative Music Making
Project Overview
The Innovation Award Program is a design thinking incubator and team competition hosted by Columbia University. In the intensive program, students are challenged to create innovative solutions in respond to educational problems with technologies.
The design challenge for INA2020 was "Life after COVID," which empowered my team to think creatively about new challenges and opportunities in post-COVID life.
As the product manager and designer, I led a team in the competition and designed a social music learning experience in VR. I pitched the project in the final round of the competition and won the People's Choice Award.
Context
Columbia University Innovation Award and EdTech Competition
My Role
Product Manager & Designer
Timeline
2020
Type
UX/UI Design,
VR Design
Project Brief
Musitation is a collaborative music-making experience in VR, where you can move, see, hear and touch music. It gives you the tangible music blocks, chord progressions, and most importantly an immersive creation playground that will make your shared music-making experience fun and intuitive.
Our team followed the 5-phases design thinking methodology during the Innovation Award program:
EMPATHIZE
Understand the problem and needs of the users, while brainstorming ideas given the topic of "Life after COVID."
DEFINE
Narrow down the topic and define the problem we are trying to solve in "life after COVID."
IDEATE
Brainstorm solutions and discuss solutions to best address the problem defined and needs of the target users.
DESIGN
Select the most suitable technology and start creating the design based on research findings.
ITERATE
Conduct user testing and iterate the design to make adjustments based on user feedback.
After creating the MVP, the last process was to "Communicate Ideas" by pitching the product to audiences and judges.
Background
Having a background in the arts and humanities, I resonate a lot with the unprecedented challenges that musicians and music lovers have faced since the COVID-19 pandemic.
I was daunted by the fact that my singing group could not sing in person anymore, and I can no longer jam with other musicians to make live music together.
Therefore, when I came to the first general meeting of the INA program knowing the design challenge was "Live after COVID", I had a clear proposal in mind, which was to support music lovers and learners in an engaging way.
I recruited four other teammates through interviews with interested participants and eventually formed a team that was passionate about music, education, and new technologies.
Empathize with the Users
To understand the problem and needs of the users, we conducted 9 user interviews via Zoom with music lovers and analyzed the results to create a user persona for our main target audience.
Our target user (13+) is someone who is interested in music creation but has minimal music knowledge and time to learn music extensively. They want an immersive and instantaneous way to connect with their friends socially and musically without traveling long distances or risking COVID-19 infection.
Ultimately, our user wants an online music environment where they are an active and social creative participant in a tangible, interactive world beyond the desktop.
Define a Problem within the Scope of "Life-after-COVID" Challenge
After a deep analysis of the users and the market, we defined the problem to solve as:
After COVID, music lovers who have an interest in music creation are concerned about continuing in-person social musical activities.
Set Design Goal
After understanding the users and their potential problem in life after COVID, we set our design goals as to:
-
Support meaningful social interaction through music
-
Inspire people to creatively express themselves through collaborative music experiences
Understand the Market
Compared to similar existing platforms for collaborative music experience, we want to build a new solution that could bridge the gap in the market to better provide a more immersive, and authentic social experience that also supports simple music-making in a fun way.
Competitive Analysis
Strengths
-
The VR medium brings more fun for beginners to continue their musical exploration and motivates the informal social learning to continue.
-
The tangible formats of music supports beginners to experiment with melody making without any prior music background.
Opportunities
-
With the popularity of VR and rising interest in learning music, the market for more high-quality VR music games is expanding. Musitation fills the gap for beginners who are interested in music creation and social interaction.
-
People are eager about learning music for leisure activities, especially since COVID as people have more alone time at home
Weaknesses
-
VR is still a niche market that limits the use of the product to a certain population
Threats
-
There are some existing platforms or games that support collaborative music experiences, including computer-based software sound as Soundtrap, Hookpad, and Chordify, VR music experiences such as Beat Saber and EXA: The Infinite Instrument, and sandbox video games such as Minecraft, Stardew Valley, and Animal Crossing.
Since COVID, we see huge potential in the growing market of virtual musical instruments, as people turn to music to fill their extra alone time.
Besides, the VR market is promising too as people seek innovative ways to learn, have fun, and connect.
Our extensive research into literature, user interviews, and market analysis led us to a solution around using VR in creating a unique experience for music lovers and learners.
Ideate Potential Solutions
After we selected VR as the technology for the solution, we started ideating possible ways to make the product come to live.
Here is an example of my paper sketch for the 3D design during the initial ideation process.
The blocks on the sketch represent the tangible music notes and the letters on the y-axis represent the pitch of the notes. The shapes of blocks on the x-axis represent the types of musical instruments.
Altogether, I sketched a 3D music creation interface as the basis of Musitation. I shared my sketch with the team, and we further brainstorm different solutions and decided to create a paper prototype to test the concept of tangible music notes with the target users first.
Rationale behind the Design
The tangible visual format of notes transforms the abstract concept of music into concrete materials that players can directly interact with. This design is backed by the theory of agent-based restructuration of Constructivism that makes learning music more accessible and engaging (Wilensky 2010).
Besides, research has shown that educational video games have benefits in learning music, which supports the use of VR in music-making experiences (Lesser 2020).
The collaborative experience is grounded in our user research, market analysis, and the following human behavior and learning theories:
Self-Determination Theory
Self-Determination Theory suggests that people are motivated by Autonomy, Relatedness, and Competency.
More recently, it has also been suggested by researchers that there is a fourth need, namely the need to satisfy drives for novelty. COVID has created massive changes to how we fulfill those drives to satisfy those 4 needs.
As WFH has increased HBR and Brookings has suggested that millions of people will continue to WFH post-COVID. WFH has demonstrated aspects of isolation, leading to a number of consequences, namely the desire to and comfort with connecting with others online through AR and VR and other novel communication channels to fulfill the need for relatedness and novelty.
Additionally, BBC has reported how people WFH pick up new hobbies out of a need to create, or autonomy restoration and competency drives. Studies on music in community have demonstrated how there is a therapeutic and healing effect to community based music activities. Mobile technologies, such as VR, have demonstrated to not only be effective in connecting others but also help others learn while connecting in both formal and informal ways.
Finally, studies on video games have demonstrated how primary needs in self-determination theory are met through video game engagement.
Constructivist Learning Theory
Constructivism's central idea is that human learning is an active engagement with the world.
Knowledge is constantly constructed upon previous ones, the experience with environments and other people. As a result, the same piece of music can be interpreted differently by different people.
Therefore, Musitation is designed to have playing engage directly with the melody and harmony of music. This engagement is made tangible by turning musical notes into manipulable blocks. There is no prerequisite of music theory needed prior to playing the game.
Design with Paper Prototypes
Backed by research findings, we first designed the paper prototype and low-fidelity wireframes to illustrate the concept of tangible music creation with blocks.
.png)
Iterate with Testing
Then, each of my team members conducted usability testing individually by having users play with the paper mockup. As a musician myself, I translated each tester's move into digital music production software to instantly playback the result generated from their move.
.png)
The testing with various music lovers has proved the feasibility of our idea with positive feedback, as the users commented how fun and helpful the experience is in learning to make music.
It also provided us with new insights into the user flow and led us to keep improving the design from paper to software in CoSpaces.
-
Improve design from low fidelity to high fidelity
-
From physical prototype to WebVR prototype
-
Use Co-Spaces to illustrate the concept in VR device
Feedback from Users
"The prototype was honestly so helpful because it made making music fun. Plugging notes into digital audio workstations (DAWs) like Ableton is really tedious. It's hard to be motivated to open up a computer and play something, especially because I'm not too great at playing the keyboard. It can be frustrating finding melodies. If I had to come up with my own chords first, it would have been a much longer process."
Gravity, A. (age 29)
Minimal Viable Product (MVP)
After the iterative design process, I took the lead in bringing the low-fidelity paper prototype to a high-fidelity prototype using CoSpaces.
Key features include:
-
Simple music-making with tangible music blocks
-
Community building through sharing and collaboration
The final process of designing for MVP tested my ability to handle emergent situations calmly and get things done quickly and with quality under pressure and unexpected challenges.
When our VR technical lead had a personal emergency and was no longer able to create the VR prototype, I didn’t give up on delivering the MVP just days before the end of the program. I stood up to solve the challenge right away by quickly learning to use the software and making our final VR prototype and presentation video in less than a week. The hard work paid off as our team was selected to the final round of the competition as 1 of the 3 finalists.
Communicating Ideas
Creating design is one piece, and communicating the messages clearly to get support is as important as the design itself, if not more important.
As the product manager and designer, I created promotional videos, a prototype concept video, a 3-minute pitch video, and a 10-minute live pitch for Musitation.
Here are some quotes from my final pitch in the competition, which I find crucial to the meaning of designing Musitation:
"Music is social glue.
2020 was a special year in many ways, It was the year when COVID broke out globally, and it was also my favorite composer, Beethoven's 250th birthday.
2020 was supposed to be the year of Beethoven's Celebration when we could enjoy his masterpiece together in concert halls until COVID separated us and turned off the music in public spaces.
As we were honoring how music kept Beethoven’s creative spirit and joy during his difficult life, we as well, could reflect on his persevering life attitude and stay strong together through virtual music-making during the challenging time."
Impact
After I pitched the project in front of 5 judges from the industries and hundreds of participants in the final competition, our team was voted to win the People's Choice Award during the 2020 Innovation Award and EdTech Competition.
I was also selected to present the project at the XR for Change Stage for the 2021 Games for Change Festival.
The popularity of Musitation showed how the design resonated with people when thinking about life after COVID. With art, music, and technology, we can support music lovers and learners to be well and connected innovatively against isolation.
People's Choice Award
2020 Innovation Award Program
In a world of uncertainty,
Let us stay creative, confident, and connected together in Musitation.
Conclusion & Takeaways
Taking the INA program was like taking an independent course but with more flexibility and hands-on tasks to complete each week. Over the course of the semester, we went through the six intensive stages of Empathy –> Define –> Ideate –> Prototype –> Test –> Communicating Ideas.
As the team leader, I have deepened my skills in project management, including finding the right balance among ideas from teammates, leading the team to be on the good track to seek user-centered solutions in responding to the identified problem, and making things happen.
The project has taught me that it can be easy to come up with ideas, but the key is to select the right ones and execute them successfully. In order to have a strong design rationale that makes the design meaningful, we spent lots of time clearly defining the problem, conducting user research, testing to understand their needs, and reviewing research on psychology and learning theories.
The experience has demonstrated my creativity, fast learning capability, and can-do attitude. It opened up my horizon to solving EdTech problems, especially in a time of uncertainty where technology has become a greater part of people’s lives. It also encouraged me to find more like-minded people to work on innovative projects for social impact.
In the future, I hope to use my creative problem-solving skills to build meaningful design solutions for people to have fun and achieve growth.