Speaking of the music industry and AI, technologies such as followings instantly come to mind: music applications utilizing AI, live experiences using VR and production side's technologies utilizing projection mapping and 3D hologram. Although, Shinichi Yamada, belongs to the digital creative CEO headquarters,wondered if it would be possible to utilize AI for marketing. Yamada acquired skills that led to system engineering through the web design work when he joined the company, and has supported startups of various services from the system side. As a result, he was assigned to the current department. Currently, he is in charge of data utilization of AI Audience Analysis.
Quantify human emotions rather than impressions.
Using analysis to get satisfaction level of entertainment.
"It wasn't anything like the live production team told me 'We have some problem with these kind of things'. It's just me, who cannot help observing the audiences' expressions, like how they react, what kind of people they are. Although, those kind of information are not to be taken as data. I use AI simply as a tool. "
For Yamada, who has worked on web design and launching of new services, AI was "his own seasoning" in the marketing area while learning traditional marketing theory and receiving training to improve his skills.
Effect measurement has just started. Beginning with explaining the concept to the management, his main job is to set up cameras and system on site (without rehearsals most of the time), and to report the measurement results. Obviously, the setup shouldn't hinder the production. It's not so rare to have troubles on site.
"When I brought an IP camera into the movie theater, I was told 'don't put anything white there'. So Next day I went to the site at 6:00 AM and removed all the cameras and LAN cables that I'd already installed, went to buy black cables and worked on the camera with black cloth that I bought in a short period of time. Sometimes I've survived with a tightrope walk like that."
"Although it shouldn't hinder the productions, it doesn't mean that I can let the installation work to onsite staffs because there are also PC and network setups. Besides, some venues have LAN and internet connections, and some have one but we are not allowed to use them. We usually use mobile routers, but sometimes radio waves don't reach. Each time we repeat trial and error while observing the situation on site. "
What matters is how the collected data highlight the tasks that haven't been recognized before. There are eight kinds of human emotions sensed by AI: "anger" "contempt" "disgust" fear" "joy" "neutral" "sadness" and "surprise". What did he discover in the field trials?
"The expressions by AI were not so laughing sometimes when standards tunes were playing, and it could mean the core fans were bored. So we can predict that they have listened to the same tunes at several venues, and they weren't so touched anymore. Using AI analysis, what was visually evaluated like 'waving in some vague way' or 'loud cheer' can be sensed like 'might there be different meaning?'."
By those information, the idea of changing the set list or adding some direction in real time could also be realized.
"Once there was a time that the smile and the sadness of audiences reversed at the end of the live. While smiling faces exceeded all the time during the live, sad faces suddenly exceeded at some point. What happened at the moment was, MC said "Next is the last tune". I think those are the quite interesting data that remarkably reveal this kind of fact in real time. The reactions also depend on what MC talks, so I bet artists would be also interested into it. Since we've taken the data only on the single-shot live so far, our next target is to do it on tour. How does the performance change on the tour? It might be better to review the MC, or could be better to reverse the timing. I want to challenge those kind of things."
Who are the real audiences?
Data collection applicable to production and merchandise
After taking data on relatively small single-shot lives for approximately 8 months since the launch, this approach is expanding to different categories, he says.
"When it's small scale or the first attempt, we start from the situation that we don't even know who the fans are. I guess that's also the why internal people are interested into our project."
Recently, he took all the data from 5 days / 6 performances of the "Theater Company 4 dollar 50 cents" which were total produced by Mr. Yasushi Akimoto and caused a buzz.
AI Audience Analysis is applicable to purposes other than live satisfaction.
"We also carry out so-called attribution determination for all audiences with setting up cameras at the entrance of event venues. This makes the actual audience segments clear. People don't usually buy single ticket at ticket agencies. Besides, people who bought tickets don't always show up, there are some cases like father buys ticket for his daughter etc.. In those cases, audience attribution comes to a black box. But if we can see the actual audiences, their attributions become clear. And if we can accumulate those data, we can present the fact that this kind of audience segments would come for this artist. It could be considered as targets for an artist, and externally, it could also used as a tool to find right sponsors. I think it has various potentialities."
When we found out the real audience segment, obviously we can reflect the data in merchandise.
"As for merchandise (hereinafter described MD), I think it's possible to see who are actually buying goods clearly. For example, the goods were supposed to made for women in their twenties, but actually those in their forties buy them more. Creativity of the goods might be slightly missing the point, or it may not be very well appealing to the original target group and only the rich people buying them. As people pay with cash most of the time at the venue, we don't know what they exactly bought from the sum of the daily sales. But if we can analyze the data properly, then direction of MD will be also be clarified, I think."
Possibility of AI Audience Analysis:
social contribution starts from Avex
Yamada sublimes AI Audience Analysis as his own essence of marketing methods. What does he think this business will bring to audiences, artists and the future of music industry?
"For example, I think that if the artist or the production side can evaluate what they do correctly and perceive current status from the evaluation properly, they will get the right direction to make an effort. The point is, if continuing this analysis makes things better or not. I believe that the artists can change the way of thinking about the performance if they have these information. As an entertainment company, I think we should seek wider variety of things. This is not the answer to everything. But as a way of approaching, I think there is an enough value to challenge. "
The most important thing is to be a hint to provide more satisfying new entertainment. This axis would be firm. Furthermore, the accumulation of the effect measurement information itself will lead to external sales and collaboration with other industries.
" Currently, we discuss about the filed sales as a platform with another in-house team. There are also third parties that are interested in this project itself, so we have some collaboration plan behind the scenes. As it started from entertainment point of view, I record scenes where people's emotions appear most prominently. But if we keep doing as it is, I believe that we will be able to provide data of the situations that don't lead to such a strong emotional values as well. Now I believe that I can lead this business by doing ahead of any other company, and I see plenty of possibilities. "
In Yamada's case, AI was inevitably an Audience Analysis utilizing his own experiences in marketing department. It was not the client work, an order from the boss, or solutions for the existing problems. It was born in the process of imagining the future entertainment and business. I believe that's the reason to excite everyone involved, including production team, artists, managements, MDs and even third parties.
Avex Inc.
Digital Creative CEO Headquarters
General manager Shinichi Yamada
photo by Hayato Ooishi