A game-with-a-purpose based on Twitter

I am happy to announce that my research group at TU Graz has launched Bulltweetbingo!, a game-with-a-purpose based on Twitter, today. The game is already live and available at http://bingo.tugraz.at. For an introduction to the idea of Buzzword Bingo, please see the following IBM commercial (Youtube video).

 

IBM Innovation Buzzword Bingo (Youtube)

 

Rather than playing buzzword bingo while listening to a talk, the idea of Bulltweetbingo! is to play Buzzword Bingo with the people you follow on Twitter. All people you follow on Twitter automatically participate in the game by tweeting. A Bulltweetbingo game terminates (i.e. hits “Bingo!”) if the people you follow on Twitter use a particular combination of the defined buzzwords in their tweets. We intend to use the data provided by each game in our research on analzying the semantics of short messages on systems such as Twitter or Facebook. Each game provides information about the relevance and topics of tweets for a particular person as well as some information on the topics of tweets that a person expects to receive in the future.

I’m copy’n pasting some more information about the game that we have made available on the game website  (about the project).

Bulltweetbingo!
Playing a game of bingo with people you follow on Twitter.

A team of researchers from Graz University of Technology, Austria has developed one of the first games-with-a-purpose that is exclusively based on Twitter.

The goal of this project is to annotate and to better understand the short messages posted to so-called social awareness streams such as Twitter or Facebook. Using this data, the researchers aim to improve the ability of computers to effectively organize and make sense out of the sea of short messages available today.

Dr. Markus Strohmaier, Assistant Professor at the Knowledge Management Institute at Graz University of Technology, Austria explains: “While social awareness streams such as Twitter or Facebook have experienced significant popularity over the last few years, we know little about how to best understand, search and organize the information that is contained in them.”

To tackle this problem, the researchers have developed a game of Buzzword Bingo that users can play with people they follow on Twitter.

“With each game users play on our website, we will collect data that helps us develop more effective algorithms for better understanding this new kind of data” Dr. Markus Strohmaier says, “and in addition to that, we simply hope users would enjoy playing a game of Bingo on Twitter. Each game is unique and exciting in a sense that users generally don’t know what tweets people will publish during the course of a bingo game”.

The researchers have launched the site bulltweetbingo! and ask users to sign up and to play a game of Bingo with the people they follow on Twitter. Twitter users can sign up at http://bingo.tugraz.at.

The game was implemented by one of my talented students, Simon Walk – Make sure to hire him if you need a complex web project to be realized quickly and effectively!

On taxonomies, folksonomies, and tweetonomies

Towards a Taxonomy of Meta-Desserts (by several_bees @flickr)

For centuries, taxonomies have been a tool for mankind to bring structure to the world. Taxonomies (wikipedia: “the practice and science of classification”) were developed in different fields of science, including – but not limited to – biology (e.g. taxonomies of animals) or library sciences (e.g. taxonomies of literature). Regardless of the particular domain of application, in most cases those taxonomies were developed by a selected few (e.g. librarians), and were used by many.

With the emergence of personal computers and file directories, the task of taxonomy development was brought to the masses. Suddenly everyone (i.e. every computer user) was in charge of developing, maintaining and transforming personal taxonomical structures in order to organize and (re-)find resources. While this development has led to a vast increase of personal taxonomies, it was only since del.icio.us has popularized tagging as a new form of resource organization that users’ personal taxonomies were exposed publicly. This has made it possible to aggregate a large number of personal taxonomies into collective taxonomic structures. The result of such aggregation has since then been refered to as folksonomies, i.e. an emergent structure collectively produced by a large number of users in a bottom-up manner.

In social awareness streams (pdf) such as Twitter of Facebook, users typically do not aim to classify or organize resources, but they engage in casual chatter and dialogue, ocassionally using syntax to coordinate communication (such as #hashtags or @replies). Taxonomic structures can be assumed to play a subordinate role for users of social awareness streams.

In a recent paper to be presented at the SemSearch Workshop at WWW2010 [1] however, we show that there exist latent conceptual structures – similar to taxonomies or folksonomies – in social awareness streams, and that we can acquire these structures through simple aggregation mechanisms.

Abstract: Although one might argue that little wisdom can be conveyed in messages of 140 characters or less, this paper sets out to explore whether the aggregation of messages in social awareness streams, such as Twitter, conveys meaningful information about a given domain. As a research community, we know little about the structural and semantic properties of such streams, and how they can be analyzed, characterized and used. This paper introduces a network-theoretic model of social awareness streams, a so-called “tweetonomy”, together with a set of stream-based measures that allow researchers to systematically define and compare different stream aggregations. We apply the model and measures to a dataset acquired from Twitter to study emerging semantics in selected streams. The network-theoretic model and the corresponding measures introduced in this paper are relevant for researchers interested in information retrieval and ontology learning from social  awareness streams. Our empirical findings demonstrate that different social awareness stream aggregations exhibit interesting differences, making them amenable for different applications [1].

In the paper, we introduce the notion of tweetonomies, and a corresponding tri-partite model of social awareness streams that extends the existing model of folksonomies by accomodating user-generated syntax (such as slashtags and other emerging syntax) and thereby integrating the communicative nature of such streams.

In the figure below, we have applied the network-theoretic model of tweetonomies to acquire a semantic network of hashtags that could be used for a range of different purposes, such as for navigating social awareness streams or for recommendation problems.

A tweetonomy of hashtags, aquired from Twitter (with the help of Jan Poeschko, click for full image 2.6 MB)

Our work shows that tweetonomies are a far more complex structure than – for example – taxonomies or folksonomies. One reason for that observation lies in the dynamic and user-generated nature of its syntax, but also in the fact that tweetonomies accomodate a much richer language than the language used in social tagging systems (tweets vs tags).

The results of our work suggest that tweetonomies are a novel and promising concept, different from taxonomies and folksonomies where people engage in conscious acts of classification. Whether tweetonomies have the potential to bring order and structure to social awareness streams similar to the way folksonomies brought order to social tagging systems remains a question to be answered.

Update (May 5 2010): An interesting question that was raised during the presentation of the paper at the WWW’2010 workshop was whether it would be justified to introduce Tweetonomies as a new concept. In other words, are the structures that we observe on twitter not just a different form of folksonomies? I’d argue for the necessity of a new concept for the following reasons: While taxonomies and folksonomies emerge when users structure resources, tweetonomies emerge when users structure conversation. Because conversations are inherently different than resources (e.g. they are dynamic, and involve multiple users) the structures that emerge from social awareness streams (tweetonomies) can be expected to be different from the structures that emerge from social bookmarking systems (folksonomies). Whether this is really the case however needs to be investigated in future work.

References:

[1] C. Wagner, M. Strohmaier, The Wisdom in Tweetonomies: Acquiring Latent Conceptual Structures from Social Awareness Streams, Semantic Search 2010 Workshop (SemSearch2010), in conjunction with the 19th International World Wide Web Conference (WWW2010), Raleigh, NC, USA, April 26-30, ACM, 2010. (pdf)

Call for Papers: International Workshop on Modeling Social Media 2010 (MSM’10)

I’d like to point you to a Call for Papers for a workshop I’m involved in organizing at Hypertext 2010 in Toronto this June. I’m really excited about the focus of this event, and I’m looking forward to lots of exciting discussions and presentations (check out the invited talks and panelists!).

International Workshop on
Modeling Social Media 2010 (MSM’10)

Website: http://kmi.tugraz.at/workshop/MSM10/

June 13, 2010, co-located with Hypertext 2010,
Toronto, Canada

Important Dates:

* Submission Deadline: April 9, 2010
* Notification of Acceptance: May 13, 2010
* Final Papers Due: May 20, 2010
* Workshop date: June 13, 2010, Toronto, Canada

Workshop Organizers:

• Alvin Chin, Nokia Research Center, Beijing, China, alvin.chin (at) nokia.com
• Andreas Hotho, University of Wuerzburg, Germany, hotho (at) informatik.uni-wuerzburg.de
• Markus Strohmaier, Graz University of Technology, Austria, markus.strohmaier (at) tugraz.at

Format:

The workshop will be opened by an invited talk given by Ed Chi (Palo Alto Research Center). The talk will be followed by a number of peer-reviewed research and position paper presentations and a discussion panel including Barry Wellman (University of Toronto), Marti Hearst (University of California, Berkeley) and Ed Chi (Palo Alto Research Center).

Workshop’s Objectives and Goals:

The goal of this workshop is to focus the attention of researchers on the increasingly important role of modeling social media. The workshop aims to attract and discuss a wide range of modeling perspectives (such as justificative, explanative, descriptive, formative, predictive, etc models) and approaches (statistical modeling, conceptual modeling, temporal modeling, etc). We want to bring together researchers and practitioners with diverse backgrounds interested in 1) exploring different perspectives and approaches to modeling complex social media phenomena and systems, 2) the different purposes and applications that models of social media can serve, 3) issues of integrating and validating social media models and 4) new modeling techniques for social media. The workshop aims to start a dialogue aiming to reflect upon and discuss these issues.

Topics:

Topics may include, but are not limited to:

+ new modeling techniques and approaches for social media
+ models of propagation and influence in twitter, blogs and social tagging systems
+ models of expertise and trust in twitter, wikis, newsgroups, question and answering systems
+ modeling of social phenomena and emergent social behavior
+ agent-based models of social media
+ models of emergent social media properties
+ models of user motivation, intent and goals in social media
+ cooperation and collaboration models
+ software-engineering and requirements models for social media
+ adapting and adaptive hypertext models for social media
+ modeling social media users and their motivations and goals
+ architectural and framework models
+ user modeling and behavioural models
+ modeling the evolution and dynamics of social media

Preliminary Program Committee (confirmed):

• Ansgar Scherp, Koblenz University, Germany
• Roelof van Zwol, Yahoo! Research Barcelona, Spain
• Marti Hearst, UC Berkeley, USA
• Ed Chi, PARC, USA
• Peter Pirolli, PARC, USA
• Steffen Staab, Koblenz University, Germany
• Barry Wellman, University of Toronto, Canada
• Daniel Gayo-Avello, University of Oviedo, Spain
• Jordi Cabot, INRIA, France
• Pranam Kolari, Yahoo! Research, USA
• Tad Hogg, Institute for Molecular Manufacturing, USA
• Wai-Tat Fu, University of Illinois at Urbana-Champaign, USA
• Thomas Kannampallil, University of Texas, USA
• Justin Zhan, Carnegie Mellon University, USA
• Marc Smith, ConnectedAction, USA
• Mark Chignell, University of Toronto, Canada

Website: http://kmi.tugraz.at/workshop/MSM10/

Measuring Earthquakes on Twitter: The Twicalli Scale

I got interested in the signal that Twitter received from the two last earthquakes happening in California and Haiti. It has been recently suggested that Twitter can play a role in assessing the magnitude of an earthquake, by studying the stream of tweets that contain a reference to the event, such as the stream of messages related to #earthquake, including messages like this. The term “Twichter Scale” has been used in this context to discuss the relation between Twitter and external events such as earthquakes.

Different people have expressed different ideas about a Twichter Scale, for example:

Twichter Scale (n): the fraction of Twitter traffic caused by an earthquake. Unused on the east coast. (@ian_soboroff)

While this definition does not necessarily imply that the Twichter scale indicates the magnitude of earthquakes, it is interesting to ask whether Twitter data can be used for that purpose.

Impact of two earthquakes on different Twitter hashtag streams: #earthquake, #earthquakes and #quake between Jan 9 and Jan 15

When we look at the data, we can clearly identify both earthquakes represented as spikes in the data. Both earthquakes were comparable in terms of Magnitude (6,5 vs. 7.0 on the Richter Scale). And in fact, both events produced a comparable amplitude for the #earthquake hashtag stream. On the surface, this might be a confirmation of the idea of a Twichter Scale, based on the Richter Scale, which is a scale measuring the magnitude of an earthquake. The Richter scale produces the same value for a given earthquake, no matter where you are.

However, there is another, less scientific measure to characterize earthquakes – the so-called Mercalli scale – which is a measure of an earthquake’s effect on people and structures.

Which yields to the interesting question, whether Twitter streams can better serve as an indicator of strength (Richter) or impact (Mercalli) of an earthquake?

As we can see in the figure, the amplitude produced on Twitter is approximately equal for both events (almost 400 messages per hour). My suspicion however is, that this is not because Twitter accurately captures the strengths of earthquakes, but because the Jan 9 earthquake was closer to California, where more people (more Twitter users) are willing to share their experiences. So it seems that this produced an amplitude of similar extent, although the impact of the Jan 9 earthquake in California on structures and people was much weaker than the impact of the Jan 12 earthquake in Haiti.

So how can we identify the difference of an earthquake in terms of its impact on people and structures?

When we look at the diagram above, we can see a clear difference after the initial spike: While the Californian earthquake did not cause many follow-up tweets, the aftermath of the Haiti earthquake is clearly visible.

What does that say about Twitter as a signal for earthquakes?

1. The amplitude of the signal on Twitter is very likely biased by the density of Twitter users in a given region, and thereby can neither give reliable information about the magnitude nor the impact of an earthquake. This suggests that Twitter can not act as a reliable sensor to detect the magnitude of an earthquake in a “Richter Scale” sense.
2. However, the “aftermath” of a spike on twitter (the integral) seems to be a good indication of an earthquake’s impact on people and structures – in a “Mercalli Scale” sense. Long after the initial spike, the Haiti earthquake is still topic of conversations on Twitter (those are likely related to  fundraising efforts and other related aid activities). Indepentent of the density of Twitter users in Haiti (which is probably low), the aftermath can clearly be identified.

The Twicalli Scale:

This suggests that Twitter as a sensor for the magnitude of earthquakes (in a Richter Scale sense) does not seem very useful. Twitter is more indicative of earthquakes in a “Twicalli scale” sense:

Using the aftermath (not the amplitude) of twitter stream data, the impact (not the magnitude) of earthquakes becomes visible on Twitter.

Update: Here are links to further resources and the datasets this analysis is based on:

• Wired article “Freaked-Out Tweets After Earthquakes Help Scientists“
• An effort to introduce disaster-specific Twitter Syntax (by the University of Colorado) to coordinate relieve activities
• An interesting paper at WWW2010: “Earthquake Shakes Twitter Users: Real-time Event Detection by Social Sensors (PDF)”
  
• A blog post by Gene Golovchinsky on Mechanical Turk, Twitter Syntax and disaster coordination
• Archives for hashtagstreams #earthquake, #earthquakes, #quake

Update II (Aug 27 2010): The Twicalli scale was mentioned in a recent paper on the importance of trust in social awareness streams such as Twitter (page 8, left column)

Marcelo Mendoza, Barbara Poblete and Carlos Castillo, Twitter Under Crisis: Can we trust what we RT?, Workshop on Social Media Analytics, In conjunction with the International Conference on Knowledge Discovery & Data Mining (KDD 2010), PDF download (see page 8, left column)

Update III (Oct 11 2011): Now there’s also a WWW2011 paper mentioning the Twicalli Scale (page 2, top of right column)

Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on twitter. In Proceedings of the 20th international conference on World wide web (WWW ’11). ACM, New York, NY, USA, 675-684.

Update IV (2017): The Twicalli Scale has been implemented in a decision support tool by the National Seismology Office in Chile, See the corresponding SIGIR poster below.

Poblete, Barbara. “Twicalli: An Earthquake Detection System Based on Citizen Sensors Used for Emergency Response in Chile.” Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM link, 2017.

http://twitter.com/ian_soboroff/status/7488111821

Why we can’t quit searching in Twitter

I’m still trying to get my head around this recent Slate magazine article on Seeking: How the brain hard-wires us to love Google, Twitter, and texting. And why that’s dangerous.

In this blog post, I’m basically trying to tie this article on “Seeking” together with two related topics: “Information Foraging” and “Twitter”.

Seeking

The Slate magazine article starts by observing that we (humans) are insatiably curious, and that we gather data even if it gets us into trouble. To give an example:

Nina Shen Rastogi confessed in Double X, “My boyfriend has threatened to break up with me if I keep whipping out my iPhone to look up random facts about celebrities when we’re out to dinner.“

The article goes on making several intertwined arguments which I’m trying to sort out here.

One of the arguments focuses on reporting how lab rats can be artificially put in “a constant state of sniffing and foraging”.  It has been observed in experiments that rats tend to endlessly push a button if the button would stimulate electrodes that are connected to the rat’s lateral hypothalamus. This gets rats locked into a state of endless repetitive behavior. Scientists have since concluded that the lateral hypothalamus represents the brain’s pleasure center.

Another point is made based on work by University of Michigan professor of psychology Kent Berridge (make sure to check out his website after reading this post) who argues that the mammalian brain has separate systems for wanting and liking. Think of it as the difference between wanting to buy a car, and liking to drive it.

Wanting [or seeking] and liking are complementary. The former catalyzes us to action; the latter brings us to a satisfied pause. Seeking needs to be turned off, if even for a little while, so that the system does not run in an endless loop.

Interestingly, our brain seems to have evolved into “being more stingy with mechanisms for pleasure than desire”: Mammals are rewarded by wanting (as opposed to liking), because creatures that lack motivation (but thrive on pleasure) are likely to lead short lives (due to negative selection).

There are lab experiments reporting on how the wanting system can take over the liking system. Washington State University neuroscientist Jaak Panksepp says that “a way to drive animals into a frenzy is to give them only tiny bits of food which sends their seeking system into hypereactivity.”

Information Foraging

This brings me to a book I’m currently reading on “Information Foraging Theory” by Peter Pirolli (I have not finished it yet!). At the beginning, the book argues that the way users search for information can be likened to the way animals forage for food. Information Foraging Theory makes a basic distinction between two types of states a forager is in: between-patch and within-patch states. In between-patch states, foragers are concerned with finding new patches to feed on, whereas in within-patch states, creatures are concerned with consuming a patch. Information Foraging Theory is in part concerned with modeling “optimal” strategies for foragers that would maximize some gain (e.g. information value) function, based on the Marginal Value Theorem, depicted in the illustration below.

In this illustration from wikipedia, Transit time refers to between patch (left side) and Time foraging in patch refers to within patch (right side of the diagram). The slope of the tangent corresponds to the optimal rate of gain. There is an interesting relationship between time spent within and between patches. If patches yield very little average gain (e.g. calories, or information value), patches are easily exhausted, quickly putting foragers into the between patch state again.

Twitter, Seeking and Information Foraging

Now I’m trying to tie these two topics together (there might even be a common basis for the relationship between these topics in the literature).

Seeking and Information Foraging: It seems that wanting and liking systems relate to within and between patch states in Information Foraging Theory. If lab rats push a button in an experiment, it seems that the rat’s electrodes modify their liking system in a way that prevents them from engaging within a patch, and puts them into a between patch state. When animals are sent into a frenzy by giving them tiny bits of food, within patch time is minimalized, sending them right back into a between patch state. In this scenario, animals spend relatively more time searching than actually consuming food, effectively reducing their overall gain in comparison to scenarios where they would be confronted with large bits of food (higher gain patches).

Finally, how this all might relate to Twitter: I’m arguing that Twitter’s message restriction to 140 characters (disregarding links that might be posted in Twitter messages) artificially reduces within-patch time. The gains of a patch (a tweet) might still vary, but gain is not dependent on within-patch time anymore. The average “within-patch/gain function” (right side of the above illustration) seems to be constant! It always takes the same approximate amount to read a tweet (assuming there are no URLs in a tweet), reading “longer” does not increase the gain.

In addition, Twitter’s particular user interface (chronological listing of tweets) seems to be weak in terms of information scent: Judging whether a tweet is relevant or not requires a forager to read the entire tweet, regardless whether the patch (the tweet) contains a gain (an informative value) or not.  This seems to yield to a situation where in systems such as Twitter, users quickly change between within patch (reading a tweet) and between patch (finding the next tweet to read) states. The reason to that might be the following: When a forager has exhausted a patch, he would switch back to a between patch state. However, due to a deprivation of information scent on the Twitter user interface, the user is largely helpless in the between patch state (he does not know where to search next, other than reading the next tweet). This leaves users with a desire to change back to within patch states as quickly as possible (only reading entire tweets can help to assess relevance), thereby potentially adapting chaotic and/or irrational strategies.

The above observation might also explain the frenzy that animals are sent into when being offered tiny bits of food while being deprived of “scent” to inform their between patch phases. The hypothesis would be that the frenzy would not occur if the animals were offered clues regarding where the next patch is to be expected, and what gain they could get from exhausting it (of course behavioural biologists might have already studied this question).

Returning to Twitter, it seems that the same effect that sends animals into a frenzy could be at place at Twitter, where users – due to a combination of small within-patch times and weak information scent – engage in uninformed foraging of artificially small information patches.

This of course is the provocative conclusion of the Slate Magazine article. What I found interesting is how these three topics – seeking, information foraging and Twitter – nicely tie into each other on a theoretical level.

I still have not figured out what a reduction of within patch times alone means from an Information Foraging Theoretic perspective – I’d like to figure that out at some point in the future.