rhinoceros
Archon
Gender: 
Posts: 1318
Reputation: 8.04
Rate rhinoceros
My point is ...
  
|
 |
Word bursts and tracking memes
« on: 2003-02-19 15:57:57 » |
 |
http://www.newscientist.com/news/news.jsp?id=ns99993405
Word 'bursts' may reveal online trends
18:24 18 February 03
NewScientist.com news service
Searching for sudden "bursts" in the usage of particular words could be used to rapidly identify new trends and sort information more efficiently, says a US computer scientist.
Jon Kleinberg, at Cornell University in New York, has developed computer algorithms that identify bursts of word use in documents.
While other popular search techniques simply count the number of words or phrases in documents, Kleinberg's approach also takes into account the rate at which the word usage increases.
Kleinberg suggests that the method could be applied to weblogs to track new social trends. For example, identifying word bursts in the hundreds of thousands of personal diaries now on the web could help advertisers quickly spot an emerging craze.
<snip>
Researchers at Google, the world's most widely used internet search engine, have already shown that identifying spikes in search terms can be used to track the spread of news and rumours around the world. The algorithms that run Google's automated news aggregation service remain secret, but it is not difficult to imagine that word bursts could, or do, have a useful role.
<snip>
In a simple historical test of the technique, Kleinberg analysed all the annual State of the Union addresses given by US Presidents since 1790. He found that particular word "bursts" could indeed be linked to important events at the time the speeches were delivered.
In the years that immediately followed the American Revolution, for example, sudden bursts in the use of words such as "militia", "British" and "savages" are found.
From 1930 to 1937 a spike in the use of the word "depression" is seen. And from 1949 to 1959 "atomic" is the word with the greatest "burstiness". Later in the 20th century, words such as "Vietnam", "Soviet", "communist" and "Afghanistan" increase sharply in usage.
|
Author
Logged
