Test: Six AI tools competed to make a music video

Can an AI program make an entire music video that holds together from start to finish? The film site Film Threat set out to find out by running the same three-minute song through six different AI tools: BeatViz, Freebeat, Neural Frames, Kaiber, Runway and Kling.

Most AI video tools are good at making short, good-looking clips of a few seconds. The challenge arises when you have to put many clips together into an entire song. The test therefore looked at five things: whether the people in the video look the same throughout (character consistency), whether the transitions between scenes flow naturally, whether the lips follow the song's words (lip sync), whether you can fix errors in individual scenes, and whether the tool can handle an entire song at all and not just short snippets.

According to the test, Runway and Kling made beautiful individual clips but could only generate up to ten seconds at a time. This means you have to stitch a great many clips together to make a three-minute video, which easily leads to awkward transitions and characters that suddenly look different. Kaiber was good at colorful, abstract images that follow the rhythm of the music, but lacked a coherent story.

Neural Frames and Freebeat performed best among the rest. Both start by analyzing the music and have features that hold on to how the characters look throughout the song.

The winner of the test was BeatViz, which according to the article treats the entire music video as one unified project: The tool analyzes the music, plans scenes and lets the user approve how each character and each setting looks before the rest of the video is made.

The conclusion of the test is that it is not enough for an AI tool to be able to make one good clip. The hard part is keeping track of dozens of scenes so the video feels like one coherent film and not just a series of random clips glued together. For regular users considering AI for music videos, the point is: Look for tools made for entire songs, not just short snippets.

What it means for you

If you want to try your hand at AI music videos yourself, you should choose a tool built for entire songs rather than short clips. If you plan characters, settings and mood from the start, you avoid awkward transitions and people who change appearance along the way. This makes it easier for both musicians and content creators to make a coherent video without having to cut many short pieces together themselves.