AI Creates Living Sound Archive for Artists

What happens to an artist's creative legacy when they die? For sound artists whose work depends on specific spaces and moments, traditional archiving often fails to capture the essence of their creations. A new collaboration between researchers and artist Evala demonstrates how artificial intelligence can preserve and extend artistic vision beyond an artist's lifetime, creating what the team calls a "new archive" that continues to generate original sounds even after the artist is gone.

The key finding from this three-month exhibition at Tokyo's NTT InterCommunication Center is that a specially designed AI system can maintain an artist's creative identity while introducing novel, unexpected sound elements. The system used Evala's past sound artworks as training data, ensuring the generated sounds remained faithful to his distinctive style while producing new compositions he had never heard before.

Researchers employed SpecMaskGIT, a lightweight sound generation model that processes audio as spectrograms rather than raw waveforms. This approach allowed for faster processing and real-time generation, crucial for the exhibition's continuous eight-channel sound environment. The model was trained exclusively on approximately 200 hours of Evala's previous sound artworks, internalizing his unique acoustic characteristics and spatialization preferences.

The technical implementation involved significant optimizations to handle the artist's requirement for 48kHz sampling frequency and real-time performance. The team reduced the Transformer blocks from 24 to 12, cutting parameter count from 172 million to 89.27 million while maintaining quality. They also replaced the standard HiFi-GAN vocoder with the faster Vocos system to eliminate processing bottlenecks. These modifications enabled the system to generate 10-second sound segments with 2-second overlaps, creating seamless, uninterrupted audio throughout the three-month exhibition.

Results showed the system successfully balanced preservation with innovation. Initially, the model produced sounds that resembled collages of Evala's previous works, but after incorporating textual prompts and CLAP (Contrastive Language-Audio Pre-training) conditioning, the outputs became more abstract and novel while still reflecting the artist's style. The eight-channel spatial arrangement—with speakers positioned around listeners at left, right, front, back, above, and below—created an immersive environment where visitors could walk through the space and experience varying sonic perspectives.

This work matters because it addresses a fundamental challenge in sound art preservation. Unlike visual art, sound installations are often tied to specific performative contexts and spatial arrangements that are difficult to capture through traditional archiving methods. The "new archive" concept offers a speculative solution where AI doesn't just document past work but actively continues the creative process. For the 20,000 visitors who experienced the exhibition, this represented a living artwork that learned from the artist's legacy while generating new elements.

The approach has limitations, particularly in its reliance on the artist's existing body of work. The system requires substantial high-quality source material—in this case, 200 hours of Evala's sound artworks—to effectively capture artistic identity. Additionally, while the model introduces novelty, the degree to which these new elements align with the artist's evolving intentions remains uncertain. The paper notes that ensuring the AI's surprises remain meaningful rather than merely random requires careful conditioning and iterative refinement through artist feedback.

This collaboration between Sony researchers and Evala demonstrates how AI can serve as more than just a tool—it can become an active participant in preserving and extending artistic legacy. By creating a system that maintains creative identity while introducing appropriate novelty, the project offers a model for how technology might help artists achieve a form of digital immortality, their work continuing to evolve and engage audiences long after they're gone.