After receiving a favorable response from users to the new tool, Spotify has bigger aspirations for the technology underlying its new AI DJ feature. The AI DJ, which was introduced last week just in time for the company’s Stream On event in Los Angeles, curates a personalized playlist of music and provides spoken commentary in a natural-sounding, AI-generated voice. On top of Spotify’s existing efforts in personalization and machine learning, the functionality also makes use of huge language models, generative voice, and the most recent AI technology.
Spotify feels that these new tools don’t necessarily have to be restricted to a particular feature, which is why it’s currently exploring various uses for the technology.
Despite the mobile app redesign, which now emphasizes TikTok-like discovery streams for music, podcasts, and audiobooks, which was the highlight of Spotify’s Stream On event, the AI DJ is now a key aspect of the streaming service’s new experience. The DJ, which went live in late February for Spotify Premium subscribers in the United States and Canada, is intended to get to know users so well that it can play any song you want with the push of a button.
With the app’s redesign, the DJ will now be visible at the top of the screen for subscribers under the Music subfeed, acting as both a relaxed way to play beloved music and a tool to encourage free users to upgrade.
Spotify claims that it made use of the knowledge base and insights of its internal music experts to provide the commentary that goes along with the music the DJ streams. The DJ is then able to scale their commentary to the app’s end users using OpenAI’s Generative AI technology. Because Spotify has a smaller collection of musical knowledge than ChatGPT, which tries to generate responses by compiling data from other websites, the DJ’s comments are more likely to be correct and up-to-date.
The actual music choices made by the DJ are based on the DJ’s knowledge of the user’s preferences and interests, echoing what was previously put into customized playlists like Discover Weekly and others.
The AI DJ’s voice, meanwhile, was developed using the Somatic technology that Spotify purchased last year and is modeled on that of Xavier “X” Jernigan, head of Spotify’s Cultural Partnerships and host of the now-defunct morning program podcast “The GetUp.” Amazingly, the voice doesn’t seem at all robotic and sounds realistic. (Jernigan talked alongside his AI substitute during Spotify’s live event, and it was challenging to distinguish between the two. He made a joke about how he could hear his voice all day.
“The goal of the Somatic technology, the team that we bought, is the reason it sounds this amazing. After Stream On, Ziad Sultan, Spotify’s head of personalization, spoke with TechCrunch on the topic. “It is about the emotion in the voice,” he said. “You can tell where the breathing pauses are when you hear the AI DJ. You’ll notice the various intonations. You can detect enthusiasm for particular genres,” he claims.
Naturally, a human-sounding AI voice is nothing new; Google stunned the world with its creation years ago. But, Duplex’s usage of technology drew controversy because the AI originally called businesses on the end user’s behalf without making clear that it wasn’t a real person. Given that Spotify’s service is labeled an “AI DJ,” there shouldn’t be any similar worries.
Jernigan went into the recording studio to provide top-notch voice recordings while collaborating with speech technology specialists to make Spotify’s AI voice sound natural. He was given instructions to read different sentences while evoking various emotions, and the data was subsequently fed into the AI model. Spotify declined to provide any time estimates or specifications, referring to the technology as its “secret sauce” and emphasizing that it is always developing.
The created voice, according to Sultan, is entirely artificial intelligence (AI). “With that high-quality input that has a lot of various permutations, [Jernigan] then doesn’t need to say anything anymore,” he explains. To make sure he has ongoing involvement, Jernigan will occasionally drop by Spotify’s writers’ room to share his thoughts on how he would interpret a sentence.
While Sonantic and OpenAI technology was used to create the AI DJ, Spotify is also funding internal research to gain a deeper understanding of the most recent developments in AI and massive language models.
Sultan tells TechCrunch, “We have a research team that works on the latest language models. In reality, it has a few hundred people focused on machine learning and personalization. Sultan observes that the team is utilizing the OpenAI approach for the AI DJ. “But, generally speaking, we have a sizable research team that is grasping all the potentials across Big Language Models, across generative voice and personalization. This is moving quickly, he declares. We aim to be recognized for our expertise in AI.
Yet, Spotify might or might not power future improvements with its own internal AI technology. It might decide that working with a partner makes more sense, as it is doing right now with OpenAI. But it’s still too early to say.
Sultan claims, “We publish articles all the time. “We’ll be making investments in the newest technologies, and LLMs are one of those technologies in our business, as you could expect. As a result, we will build our competence.
Spotify can advance into other fields involving AI, LLMs, and generative AI technologies with the help of this fundamental technology. The corporation is still mum on what those potential consumer product sectors might be. (We have heard that one of the solutions being tested is a chatbot similar to ChatGPT. But, nothing has been decided regarding a launch because it is only one experiment among many.
“We haven’t made clear when we want to expand to additional markets or languages, for example. But it is a platform-based technology. We can accomplish it, and we intend to give more information as it develops, says Sultan.
Early consumer response to AI is encouraging, according to Spotify
Because the company wasn’t sure how consumers would respond to the DJ, it decided against creating a whole line of AI products. Do people desire AI DJs? They would use the feature, right? Everything was unclear. After all, the voice assistant for Spotify, “Hello Spotify,” had been discontinued due to poor user uptake.
Nonetheless, there were early indications that the DJ feature would succeed. Before going public, Spotify tested the product internally among its staff, and usage and re-engagement figures had been “very, very good.”
Sultan informs us that the public adoption thus far is consistent with internal Spotify findings. This indicates that it may be possible to build new goods on the same fundamental principles.
“People use this product for hours every day…
It guides them in making decisions and discoveries, explains why the next piece of music they should listen to is important and suggests it to them.
So the response is quite good and emotional, as you can see if you look at different social media platforms, says Sultan.
Additionally, Spotify revealed that on the days when users tuned in, they spent 25% of that time listening to the DJ and that more than 50% of first-time users came back the next day to use the service. The feature hasn’t yet been fully implemented in the United States and Canada, thus these metrics are preliminary. Yet, the company sees promise in them.
Sultan declares, “I think it is one incredible step in developing a link between highly useful items and users. The issue, though, would be to “identify the proper application and then to create it correctly,” he warns.
“In this instance, we claimed that the DJ was an AI. For it, we set up the writers’ room. Users are given complete control over it to carry out the task for which it was designed. It’s doing a great job. But daydreaming about what else we could accomplish and how quickly is certainly entertaining,” he continues.
