Madonna on her 2006 Confessions Tour. Madonna was a local success in New York dance clubs before a talent scout spotted her.If Brian Epstein hadn’t spotted the Beatles at an obscure Liverpool club in 1961, we would probably never even have heard of the group, and they would certainly never have reached international stardom.

The same is true for other pop superstars like Madonna and Britney Spears – without talent scouts, they might have remained only local successes in New York dance clubs (Madonna), and Louisiana kiddie dance revues (Britney).

Now an Israeli professor has developed software that he believes can accurately predict who will make it onto the next billboard charts, and who will be trapped on the local bar circuit.

Professor Yuval Shavitt, of Tel Aviv University (TAU), believes the ability to predict the next big music phenomenon could become a profitable tool for music producers and record labels – and a boon to young people who want to be in the know.

Using data collected from Gnutella, the most popular peer-to-peer file-sharing network in the United States, Shavitt has developed a computer algorithm that can spot an emerging artist several weeks or months before national success hits.

A new frontier in the record business

“Until now, talent scouts for record companies used instinct to predict the next rock personality. Our software has an astonishing success rate – about 30 percent, and in some cases up to 50%. We’ve crossed a new frontier in the record business,” he says.

Shavitt, from TAU’s School of Electrical Engineering has already used the technology with notable success. Soulja Boy (Crank That) and Sean Kingston (Temperature) were both flagged by his system in April last year, weeks before they emerged into the national spotlight – both songs became Billboard hits when they entered the charts in June 2007.

And the group Shop Boyz skyrocketed to popularity in their home city of Atlanta in just two weeks. Their Party Like a Rockstar became a hit single, and Shop Boyz was catapulted to national fame. But not before the band popped up on Tel Aviv University’s algorithm “radar” a few weeks before they signed with Universal.

Shavitt developed the algorithm with graduate students Tomer Tankel and Noam Koenigstein. They examined a large amount of data from Gnutella user queries for unknown artists over a nine-month period during 2007. By examining data from the first six months, and then using data from the remaining three months to track the increasing popularity of those artists, they developed a system to predict which artists would break out of their local markets.

Location is the key

“The key was understanding the role of geography in the rising popularity of these artists,” says Shavitt. As part of the largest study ever done on geographically tied searches, Tel Aviv University researchers examined the 30 to 40 million queries that are entered daily on Gnutella. They realized that the artists who eventually make it big nationally, first had a huge number of user queries in their local region, even when they had zero queries from elsewhere in the US.

The numbers for new artists started small, often with five, then 20, then 150 queries within the artist’s home city each week. Sometimes queries were even localized to a specific urban neighborhood. At first glance, these numbers seem insignificant, but Shavitt explains that exponential growth in search queries sent from one geographical region proved a reliable predictor of a future breakout artist.

For a record company, an algorithm like this is a powerful tool. The software can be applied to television programs, video clips, and other entertainment products, including home videos on sites like YouTube.

In an effort to continue collecting data for future study, Shavitt has now started his own collection network on Direct Connect, which gets about a million hits a day.

Koenigstein, his student, is hoping to expand the scope of the algorithm predictions to look at individual songs by well-established artists. “Will a Madonna song sell because it’s a hit, or just because it’s sung by Madonna?” he asks. “That’s what we’re looking at now.”