Are Tag Clouds Useful for Navigating Social Media?

16 08 2010

This week, my colleague Denis Helic will present results from a recent collaboration investigating the usefulness of tag clouds at the IEEE SocialCom 2010 conference in Minneapolis, Minnesota, USA. In this paper (download pdf), we investigated if and to what extent tag clouds – a popular mechanism for interacting with social media – are useful for navigation.

An Exemplary Tag Cloud from Flickr.com

While tag clouds can potentially serve different purposes, there seems to be an implicit assumption among engineers of social tagging systems that tag clouds are specifically useful to support navigation. This is evident in the large-scale adoption of tag clouds for interlinking resources in numerous systems such as Flickr, Delicious, and BibSonomy. However, this Navigability Assumption has hardly been critically reflected (with some notable exceptions, for example [1]), and has largely remained untested in the past. In this paper, we demonstrate that the prevalent approach to tag cloud-based navigation in social tagging systems is highly problematic with regard to network-theoretic measures of navigability. In a series of experiments, we will show that the Navigability Assumption only holds in very specific settings, and for the most common scenarios, we can assert that it is wrong.

While recent research has studied navigation in social tagging systems from user interface [2], [3], [4] and network-theoretic [5] perspectives, the unique focus of this paper is the intersection of these issues. This paper provides answers to questions such as: How do user interface constraints of tag clouds affect the navigability of tagging systems? And how efficient is navigation via tag clouds from a network-theoretic perspective? Particularly, we first 1) investigate the intrinsic navigability of tagging datasets without considering user interface effects, and then 2) take pragmatic user interface constraints into account. We 3) will demonstrate that for many social tagging systems, the so-called Navigability Assumption does not hold and we will finally 4) use our findings to illuminate a path towards improving the navigability of tag clouds.

Here’s the abstract:

Abstract: It is a widely held belief among designers of social tagging systems that tag clouds represent a useful tool for navigation. This is evident in, for example, the increasing number of tagging systems offering tag clouds for navigational purposes, which hints towards an implicit assumption that tag clouds support efficient navigation. In this paper, we examine and test this assumption from a network-theoretic perspective, and show that in many cases it does not hold. We first model navigation in tagging systems as a bipartite graph of tags and resources and then simulate the navigation process in such a graph. We use network-theoretic properties to analyse the navigability of three tagging datasets with regard to different user interface restrictions imposed by tag clouds. Our results confirm that tag-resource networks have efficient navigation properties in theory, but they also show that popular user interface decisions (such as “pagination” combined with reverse-chronological listing of resources) significantly impair the potential of tag clouds as a useful tool for navigation. Based on our findings, we identify a number of avenues for further research and the design of novel tag cloud construction algorithms. Our work is relevant for researchers interested in navigability of emergent hypertext structures, and for engineers seeking to improve the navigability of social tagging systems.

The results presented in this paper make a theoretical and an empirical argument against existing approaches to tag cloud construction. Our work thereby both confirms and refutes the assumption that current tag cloud incarnations are a useful tool for navigating social tagging systems. While we confirm that tag-resource networks have efficient navigational properties in theory, we show that popular user interface decisions (such as “pagination” combined with reverse-chronological listing of resources) significantly impair navigability. Our experimental results demonstrate that popular approaches to using tag clouds for navigational purposes suffer from significant problems. We conclude that in order to make full use of the potential of tag clouds for navigating social tagging systems, new and more sophisticated ways of thinking about designing tag cloud algorithms are needed.

Here’s the full reference for the paper, and a link to the pdf as well as to preliminary slides:

Reference and PDF Download: D. Helic, C. Trattner, M. Strohmaier and K. Andrews, On the Navigability of Social Tagging Systems, The 2nd IEEE International Conference on Social Computing (SocialCom 2010), Minneapolis, Minnesota, USA, 2010. (download pdf) (related slides)

Further references:

[1] M. A. Hearst and D. Rosner, “Tag clouds: Data analysis tool or social signaller?” in HICSS ’08: Proceedings of the Proceedings of the 41st Annual Hawaii International Conference on System Sciences. Washington, DC, USA: IEEE Computer Society, 2008.
[2] C. S. Mesnage and M. J. Carman, “Tag navigation,” in SoSEA ’09: Proceedings of the 2nd international workshop on Social software engineering and applications. New York, NY, USA: ACM, 2009, pp. 29–32.
[3] A. W. Rivadeneira, D. M. Gruen, M. J. Muller, and D. R. Millen, “Getting our head in the clouds: toward evaluation studies of tagclouds,” in CHI ’07: Proceedings of the SIGCHI conference on Human factors in computing systems. New York, NY, USA: ACM, 2007, pp. 995–998.
[4] J. Sinclair and M. Cardew-Hall, “The folksonomy tag cloud: when is it useful?” Journal of Information Science, vol. 34, p. 15, 2008. [Online]. Available: http://jis.sagepub.com/cgi/content/abstract/34/1/15
[5] N. Neubauer and K. Obermayer, “Hyperincident connected components of tagging networks,” in HT ’09: Proceedings of the 20th ACM conference on Hypertext and hypermedia. New York, NY, USA: ACM, 2009, pp. 229–238.

Advertisements




On taxonomies, folksonomies, and tweetonomies

17 04 2010

Towards a Taxonomy of Meta-Desserts (by several_bees @flickr)

For centuries, taxonomies have been a tool for mankind to bring structure to the world. Taxonomies (wikipedia: “the practice and science of classification”) were developed in different fields of science, including – but not limited to – biology (e.g. taxonomies of animals) or library sciences (e.g. taxonomies of literature). Regardless of the particular domain of application, in most cases those taxonomies were developed by a selected few (e.g. librarians), and were used by many.

With the emergence of personal computers and file directories, the task of taxonomy development was brought to the masses. Suddenly everyone (i.e. every computer user) was in charge of developing, maintaining and transforming personal taxonomical structures in order to organize and (re-)find resources. While this development has led to a vast increase of personal taxonomies, it was only since del.icio.us has popularized tagging as a new form of resource organization that users’ personal taxonomies were exposed publicly. This has made it possible to aggregate a large number of personal taxonomies into collective taxonomic structures. The result of such aggregation has since then been refered to as folksonomies, i.e. an emergent structure collectively produced by a large number of users in a bottom-up manner.

In social awareness streams (pdf) such as Twitter of Facebook, users typically do not aim to classify or organize resources, but they engage in casual chatter and dialogue, ocassionally using syntax to coordinate communication (such as #hashtags or @replies). Taxonomic structures can be assumed to play a subordinate role for users of social awareness streams.

In a recent paper to be presented at the SemSearch Workshop at WWW2010 [1] however, we show that there exist latent conceptual structures – similar to taxonomies or folksonomies – in social awareness streams, and that we can acquire these structures through simple aggregation mechanisms.

Abstract: Although one might argue that little wisdom can be conveyed in messages of 140 characters or less, this paper sets out to explore whether the aggregation of messages in social awareness streams, such as Twitter, conveys meaningful information about a given domain. As a research community, we know little about the structural and semantic properties of such streams, and how they can be analyzed, characterized and used. This paper introduces a network-theoretic model of social awareness streams, a so-called “tweetonomy”, together with a set of stream-based measures that allow researchers to systematically de fine and compare di fferent stream aggregations. We apply the model and measures to a dataset acquired from Twitter to study emerging semantics in selected streams. The network-theoretic model and the corresponding measures introduced in this paper are relevant for researchers interested in information retrieval and ontology learning from social  awareness streams. Our empirical findings demonstrate that di fferent social awareness stream aggregations exhibit interesting di fferences, making them amenable for di fferent applications [1].

In the paper, we introduce the notion of tweetonomies, and a corresponding tri-partite model of social awareness streams that extends the existing model of folksonomies by accomodating user-generated syntax (such as slashtags and other emerging syntax) and thereby integrating the communicative nature of such streams.

In the figure below, we have applied the network-theoretic model of tweetonomies to acquire a semantic network of hashtags that could be used for a range of different purposes, such as for navigating social awareness streams or for recommendation problems.

A tweetonomy of hashtags, aquired from Twitter (with the help of Jan Poeschko, click for full image 2.6 MB)

Our work shows that tweetonomies are a far more complex structure than – for example – taxonomies or folksonomies. One reason for that observation lies in the dynamic and user-generated nature of its syntax, but also in the fact that tweetonomies accomodate a much richer language than the language used in social tagging systems (tweets vs tags).

The results of our work suggest that tweetonomies are a novel and promising concept, different from taxonomies and folksonomies where people engage in conscious acts of classification. Whether tweetonomies have the potential to bring order and structure to social awareness streams similar to the way folksonomies brought order to social tagging systems remains a question to be answered.

Update (May 5 2010): An interesting question that was raised during the presentation of the paper at the WWW’2010 workshop was whether it would be justified to introduce Tweetonomies as a new concept. In other words, are the structures that we observe on twitter not just a different form of folksonomies? I’d argue for the necessity of a new concept for the following reasons: While taxonomies and folksonomies emerge when users structure resources, tweetonomies emerge when users structure conversation. Because conversations are inherently different than resources (e.g. they are dynamic, and involve multiple users) the structures that emerge from social awareness streams (tweetonomies) can be expected to be different from the structures that emerge from social bookmarking systems (folksonomies). Whether this is really the case however needs to be investigated in future work.

References:

[1] C. Wagner, M. Strohmaier, The Wisdom in Tweetonomies: Acquiring Latent Conceptual Structures from Social Awareness Streams, Semantic Search 2010 Workshop (SemSearch2010), in conjunction with the 19th International World Wide Web Conference (WWW2010), Raleigh, NC, USA, April 26-30, ACM, 2010. (pdf)





Call for Papers: International Workshop on Modeling Social Media 2010 (MSM’10)

15 03 2010

I’d like to point you to a Call for Papers for a workshop I’m involved in organizing at Hypertext 2010 in Toronto this June. I’m really excited about the focus of this event, and I’m looking forward to lots of exciting discussions and presentations (check out the invited talks and panelists!).

International Workshop on
Modeling Social Media 2010 (MSM’10)

Website: http://kmi.tugraz.at/workshop/MSM10/

June 13, 2010, co-located with Hypertext 2010,
Toronto, Canada

Important Dates:

* Submission Deadline: April 9, 2010
* Notification of Acceptance: May 13, 2010
* Final Papers Due: May 20, 2010
* Workshop date: June 13, 2010, Toronto, Canada

Workshop Organizers:

  • Alvin Chin, Nokia Research Center, Beijing, China, alvin.chin (at) nokia.com
  • Andreas Hotho, University of Wuerzburg, Germany, hotho (at) informatik.uni-wuerzburg.de
  • Markus Strohmaier, Graz University of Technology, Austria, markus.strohmaier (at) tugraz.at

Format:

The workshop will be opened by an invited talk given by Ed Chi (Palo Alto Research Center). The talk will be followed by a number of peer-reviewed research and position paper presentations and a discussion panel including Barry Wellman (University of Toronto), Marti Hearst (University of California, Berkeley) and Ed Chi (Palo Alto Research Center).

Workshop’s Objectives and Goals:

The goal of this workshop is to focus the attention of researchers on the increasingly important role of modeling social media. The workshop aims to attract and discuss a wide range of modeling perspectives (such as justificative, explanative, descriptive, formative, predictive, etc models) and approaches (statistical modeling, conceptual modeling, temporal modeling, etc). We want to bring together researchers and practitioners with diverse backgrounds interested in 1) exploring different perspectives and approaches to modeling complex social media phenomena and systems, 2) the different purposes and applications that models of social media can serve, 3) issues of integrating and validating social media models and 4) new modeling techniques for social media. The workshop aims to start a dialogue aiming to reflect upon and discuss these issues.

Topics:

Topics may include, but are not limited to:

+ new modeling techniques and approaches for social media
+ models of propagation and influence in twitter, blogs and social tagging systems
+ models of expertise and trust in twitter, wikis, newsgroups, question and answering systems
+ modeling of social phenomena and emergent social behavior
+ agent-based models of social media
+ models of emergent social media properties
+ models of user motivation, intent and goals in social media
+ cooperation and collaboration models
+ software-engineering and requirements models for social media
+ adapting and adaptive hypertext models for social media
+ modeling social media users and their motivations and goals
+ architectural and framework models
+ user modeling and behavioural models
+ modeling the evolution and dynamics of social media

Preliminary Program Committee (confirmed):
  • Ansgar Scherp, Koblenz University, Germany
  • Roelof van Zwol, Yahoo! Research Barcelona, Spain
  • Marti Hearst, UC Berkeley, USA
  • Ed Chi, PARC, USA
  • Peter Pirolli, PARC, USA
  • Steffen Staab, Koblenz University, Germany
  • Barry Wellman, University of Toronto, Canada
  • Daniel Gayo-Avello, University of Oviedo, Spain
  • Jordi Cabot, INRIA, France
  • Pranam Kolari, Yahoo! Research, USA
  • Tad Hogg, Institute for Molecular Manufacturing, USA
  • Wai-Tat Fu, University of Illinois at Urbana-Champaign, USA
  • Thomas Kannampallil, University of Texas, USA
  • Justin Zhan, Carnegie Mellon University, USA
  • Marc Smith, ConnectedAction, USA
  • Mark Chignell, University of Toronto, Canada

Website: http://kmi.tugraz.at/workshop/MSM10/





Measuring Earthquakes on Twitter: The Twicalli Scale

15 01 2010

I got interested in the signal that Twitter received from the two last earthquakes happening in California and Haiti. It has been recently suggested that Twitter can play a role in assessing the magnitude of an earthquake, by studying the stream of tweets that contain a reference to the event, such as the stream of messages related to #earthquake, including messages like this. The term “Twichter Scale” has been used in this context to discuss the relation between Twitter and external events such as earthquakes.

Different people have expressed different ideas about a Twichter Scale, for example:

Twichter Scale (n): the fraction of Twitter traffic caused by an earthquake. Unused on the east coast. (@ian_soboroff)

While this definition does not necessarily imply that the Twichter scale indicates the magnitude of earthquakes, it is interesting to ask whether Twitter data can be used for that purpose.

Impact of two earthquakes on different Twitter hashtag streams: #earthquake, #earthquakes and #quake between Jan 9 and Jan 15

When we look at the data, we can clearly identify both earthquakes represented as spikes in the data. Both earthquakes were comparable in terms of Magnitude (6,5 vs. 7.0 on the Richter Scale). And in fact, both events produced a comparable amplitude for the #earthquake hashtag stream. On the surface, this might be a confirmation of the idea of a Twichter Scale, based on the Richter Scale, which is a scale measuring the magnitude of an earthquake. The Richter scale produces the same value for a given earthquake, no matter where you are.

However, there is another, less scientific measure to characterize earthquakes – the so-called Mercalli scale – which is a measure of an earthquake’s effect on people and structures.

Which yields to the interesting question, whether Twitter streams can better serve as an indicator of strength (Richter) or impact (Mercalli) of an earthquake?

As we can see in the figure, the amplitude produced on Twitter is approximately equal for both events (almost 400 messages per hour). My suspicion however is, that this is not because Twitter accurately captures the strengths of earthquakes, but because the Jan 9 earthquake was closer to California, where more people (more Twitter users) are willing to share their experiences. So it seems that this produced an amplitude of similar extent, although the impact of the Jan 9 earthquake in California on structures and people was much weaker than the impact of the Jan 12 earthquake in Haiti.

So how can we identify the difference of an earthquake in terms of its impact on people and structures?

When we look at the diagram above, we can see a clear difference after the initial spike: While the Californian earthquake did not cause many follow-up tweets, the aftermath of the Haiti earthquake is clearly visible.

What does that say about Twitter as a signal for earthquakes?

  1. The amplitude of the signal on Twitter is very likely biased by the density of Twitter users in a given region, and thereby can neither give reliable information about the magnitude nor the impact of an earthquake. This suggests that Twitter can not act as a reliable sensor to detect the magnitude of an earthquake in a “Richter Scale” sense.
  2. However, the “aftermath” of a spike on twitter (the integral) seems to be a good indication of an earthquake’s impact on people and structures – in a “Mercalli Scale” sense. Long after the initial spike, the Haiti earthquake is still topic of conversations on Twitter (those are likely related to  fundraising efforts and other related aid activities). Indepentent of the density of Twitter users in Haiti (which is probably low), the aftermath can clearly be identified.

The Twicalli Scale:

This suggests that Twitter as a sensor for the magnitude of earthquakes (in a Richter Scale sense) does not seem very useful. Twitter is more indicative of earthquakes in a “Twicalli scale” sense:

Using the aftermath (not the amplitude) of twitter stream data, the impact (not the magnitude) of earthquakes becomes visible on Twitter.

Update: Here are links to further resources and the datasets this analysis is based on:

Update II (Aug 27 2010): The Twicalli scale was mentioned in a recent paper on the importance of trust in social awareness streams such as Twitter (page 8, left column)

Marcelo Mendoza, Barbara Poblete and Carlos Castillo, Twitter Under Crisis: Can we trust what we RT?, Workshop on Social Media Analytics, In conjunction with the International Conference on Knowledge Discovery & Data Mining (KDD 2010), PDF download (see page 8, left column)

Update III (Oct 11 2011): Now there’s also a WWW2011 paper mentioning the Twicalli Scale (page 2, top of right column)

Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on twitter. In Proceedings of the 20th international conference on World wide web (WWW ’11). ACM, New York, NY, USA, 675-684.

Update IV (2017): The Twicalli Scale has been implemented in a decision support tool by the National Seismology Office in Chile, See the corresponding SIGIR poster below.

Poblete, Barbara. “Twicalli: An Earthquake Detection System Based on Citizen Sensors Used for Emergency Response in Chile.” Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM link, 2017.