Stability AI releases an audio-generating model that can run on smartphones

AI startup Stability AI has released Stable Audio Open Small, a “stereo” audio-generating AI model that the company claims is the fastest on the market — and efficient enough to run on smartphones.

Stable Audio Open Small is the fruit of a collaboration between Stability AI and Arm, the chipmaker that produces many of the processors inside tablets, phones, and other mobile devices. While a number of AI-powered apps can generate audio, like Suno and Udio, most rely on cloud processing, meaning that they can’t be used offline.

Stability also claims that Stable Audio Open Small’s training set is made up entirely of songs from the royalty-free audio libraries Free Music Archive and Freesound. That’s as opposed to the training sets of the aforementioned Suno and Udio, which reportedly contain copyrighted content, posing an IP risk.

Stable Audio Open Small is 341 million parameters in size and optimized to run on Arm CPUs. (Parameters, sometimes referred to as weights, are the internal components of a model that guide its behavior.) Designed for quickly generating short audio samples and sound effects (e.g., drum and instrument riffs), Stable Audio Open Small can produce up to 11 seconds of audio on a smartphone in less than 8 seconds, claims Stability AI.

Here’s a sample generated by Stable Audio Open Small:

And here’s another one:

The model isn’t without its limitations. Stable Audio Open Small only supports prompts written in English, and Stability notes in its documentation that the model can’t generate realistic vocals or high-quality songs. The model also doesn’t perform equally well across musical styles, Stability warns — a consequence of its Western-biased training data.

In another potential wrinkle for devs, Stable Audio Open Small has somewhat restrictive usage terms. It’s free to use for researchers, hobbyists, and businesses with less than $1 million in annual revenue, but developers and organizations making over $1 million in revenue have to pay for Stability’s enterprise license.

Stability, the beleaguered firm behind the popular image generation model Stable Diffusion, raised new cash last year as investors, including Eric Schmidt and Napster founder Sean Parker, sought to turn the business around. Emad Mostaque, Stability’s co-founder and ex-CEO, reportedly mismanaged Stability into financial ruin, leading staff to resign, a partnership with Canva to fall through, and investors to grow concerned about the company’s prospects.

In the last few months, Stability has hired a new CEO, appointed Titanic director James Cameron to its board of directors, and released several new image generation models.