Are Tag Clouds Useful for Navigating Social Media?

This week, my colleague Denis Helic will present results from a recent collaboration investigating the usefulness of tag clouds at the IEEE SocialCom 2010 conference in Minneapolis, Minnesota, USA. In this paper (download pdf), we investigated if and to what extent tag clouds – a popular mechanism for interacting with social media – are useful for navigation.

An Exemplary Tag Cloud from Flickr.com

While tag clouds can potentially serve different purposes, there seems to be an implicit assumption among engineers of social tagging systems that tag clouds are specifically useful to support navigation. This is evident in the large-scale adoption of tag clouds for interlinking resources in numerous systems such as Flickr, Delicious, and BibSonomy. However, this Navigability Assumption has hardly been critically reflected (with some notable exceptions, for example [1]), and has largely remained untested in the past. In this paper, we demonstrate that the prevalent approach to tag cloud-based navigation in social tagging systems is highly problematic with regard to network-theoretic measures of navigability. In a series of experiments, we will show that the Navigability Assumption only holds in very specific settings, and for the most common scenarios, we can assert that it is wrong.

While recent research has studied navigation in social tagging systems from user interface [2], [3], [4] and network-theoretic [5] perspectives, the unique focus of this paper is the intersection of these issues. This paper provides answers to questions such as: How do user interface constraints of tag clouds affect the navigability of tagging systems? And how efficient is navigation via tag clouds from a network-theoretic perspective? Particularly, we first 1) investigate the intrinsic navigability of tagging datasets without considering user interface effects, and then 2) take pragmatic user interface constraints into account. We 3) will demonstrate that for many social tagging systems, the so-called Navigability Assumption does not hold and we will finally 4) use our findings to illuminate a path towards improving the navigability of tag clouds.

Here’s the abstract:

Abstract: It is a widely held belief among designers of social tagging systems that tag clouds represent a useful tool for navigation. This is evident in, for example, the increasing number of tagging systems offering tag clouds for navigational purposes, which hints towards an implicit assumption that tag clouds support efficient navigation. In this paper, we examine and test this assumption from a network-theoretic perspective, and show that in many cases it does not hold. We first model navigation in tagging systems as a bipartite graph of tags and resources and then simulate the navigation process in such a graph. We use network-theoretic properties to analyse the navigability of three tagging datasets with regard to different user interface restrictions imposed by tag clouds. Our results confirm that tag-resource networks have efficient navigation properties in theory, but they also show that popular user interface decisions (such as “pagination” combined with reverse-chronological listing of resources) significantly impair the potential of tag clouds as a useful tool for navigation. Based on our findings, we identify a number of avenues for further research and the design of novel tag cloud construction algorithms. Our work is relevant for researchers interested in navigability of emergent hypertext structures, and for engineers seeking to improve the navigability of social tagging systems.

The results presented in this paper make a theoretical and an empirical argument against existing approaches to tag cloud construction. Our work thereby both confirms and refutes the assumption that current tag cloud incarnations are a useful tool for navigating social tagging systems. While we confirm that tag-resource networks have efficient navigational properties in theory, we show that popular user interface decisions (such as “pagination” combined with reverse-chronological listing of resources) significantly impair navigability. Our experimental results demonstrate that popular approaches to using tag clouds for navigational purposes suffer from significant problems. We conclude that in order to make full use of the potential of tag clouds for navigating social tagging systems, new and more sophisticated ways of thinking about designing tag cloud algorithms are needed.

Here’s the full reference for the paper, and a link to the pdf as well as to preliminary slides:

Reference and PDF Download: D. Helic, C. Trattner, M. Strohmaier and K. Andrews, On the Navigability of Social Tagging Systems, The 2nd IEEE International Conference on Social Computing (SocialCom 2010), Minneapolis, Minnesota, USA, 2010. (download pdf) (related slides)

Further references:

[1] M. A. Hearst and D. Rosner, “Tag clouds: Data analysis tool or social signaller?” in HICSS ’08: Proceedings of the Proceedings of the 41st Annual Hawaii International Conference on System Sciences. Washington, DC, USA: IEEE Computer Society, 2008.
[2] C. S. Mesnage and M. J. Carman, “Tag navigation,” in SoSEA ’09: Proceedings of the 2nd international workshop on Social software engineering and applications. New York, NY, USA: ACM, 2009, pp. 29–32.
[3] A. W. Rivadeneira, D. M. Gruen, M. J. Muller, and D. R. Millen, “Getting our head in the clouds: toward evaluation studies of tagclouds,” in CHI ’07: Proceedings of the SIGCHI conference on Human factors in computing systems. New York, NY, USA: ACM, 2007, pp. 995–998.
[4] J. Sinclair and M. Cardew-Hall, “The folksonomy tag cloud: when is it useful?” Journal of Information Science, vol. 34, p. 15, 2008. [Online]. Available: http://jis.sagepub.com/cgi/content/abstract/34/1/15
[5] N. Neubauer and K. Obermayer, “Hyperincident connected components of tagging networks,” in HT ’09: Proceedings of the 20th ACM conference on Hypertext and hypermedia. New York, NY, USA: ACM, 2009, pp. 229–238.

WWW’2010 – Stop Thinking, Start Tagging: Tag Semantics Emerge From Collaborative Verbosity

I want to share the abstract of our upcoming paper at WWW’2010 (here is a link to the full paper). In case you are interested in our research and going to WWW in Raleigh this year as well, I’d be happy if you’d get in touch.

C. Körner, D. Benz, A. Hotho, M. Strohmaier, G. Stumme, Stop Thinking, Start Tagging: Tag Semantics Emerge From Collaborative Verbosity, 19th International World Wide Web Conference (WWW2010), Raleigh, NC, USA, April 26-30, ACM, 2010.

Abstract: Recent research provides evidence for the presence of emergent semantics in collaborative tagging systems. While several methods have been proposed, little is known about the factors that influence the evolution of semantic structures in these systems. A natural hypothesis is that the quality of the emergent semantics depends on the pragmatics of tagging: Users with certain usage patterns might contribute more to the resulting semantics than others. In this work, we propose several measures which enable a pragmatic differentiation of taggers by their degree of contribution to emerging semantic structures. We distinguish between categorizers, who typically use a small set of tags as a replacement for hierarchical classification schemes, and describers, who are annotating resources with a wealth of freely associated, descriptive keywords. To study our hypothesis, we apply semantic similarity measures to 64 different partitions of a real-world and large-scale folksonomy containing different ratios of categorizers and describers. Our results not only show that ‘verbose’ taggers are most useful for the emergence  of tag semantics, but also that a subset containing only 40% of the most ‘verbose’ taggers can produce results that match and even outperform the semantic precision obtained from the whole dataset. Moreover, the results suggest that there exists a causal link between the pragmatics of tagging and resulting emergent semantics. This work is relevant for designers and analysts of tagging systems interested (i) in fostering the semantic development of their platforms, (ii) in identifying users introducing “semantic noise”, and (iii) in learning ontologies.

More details can be found in the full paper.

This work is funded in part by the Know-Center and the FWF Research Grant TransAgere. It is the result of a collaboration with the KDE group at University of Kassel and the  University of Würzburg. You might  also want to have a look at a related blog post on the bibsonomy blog.

Some background about the distinction between categorizers and describers can be found in a related paper:

M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users’ Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media (ICWSM2010), Washington, DC, USA, May 23-26, 2010. (Download pdf)

Notes on the Relationship between Search and Tagging

I had a number of exciting and very inspiring conversations this week with Marc, Rakesh, Fabian, Cathy, and Pranam, as well as with Ed and Rowan. It was great talking to everybody and I wanted to share some of the issues that were discussed. Most conversations focused on the role of tagging, and how it relates to searching the web. I do not claim that any of these interesting thoughts are mine or that my notes offer answers.  They merely aim to serve as pointers for what I consider important issues.

A minority of resources on the web is tagged:

A number of current research projects study the question how tagged resources can inform/improve search. However, a minority of resources on the web is tagged, and the gap between tagged and non-tagged resources is likely increasing (although this seems difficult to predict cf. Paul Heymann’s work). This would mean that a decreasing ratio of resources on the web have tagged information associated with it. The question then becomes: Why bother analyzing tagging systems in the first place when their (relative) importance is likely to decrease over time?

Tagged resources exhibit topical bias (that’s a bad thing!):

Tagging is often a geek activity. I am not aware of any studies of delicious’ user population, but it is likely that delicious’ users are more geeky than the rest of the population. This is a bad thing because it would bias any broad attempt leveraging tagging for search. The bias might depend on the particular tagging system though: Flickr seems to have a much broader, and thereby more representative, user base.

Bookmarks exhibit timely bias (that’s a good thing!):

Bookmarking typically represents an event in time triggered by some user. Most tagging systems therefore provide timestamp information, allowing to infer more information about the context in which a given resource is being tagged. This allows us to use tagging systems for studying how information on the web is organized, filtered, diffused and consumed.

Search supercedes any other form of information access/organisation:

I found this issue to be the most fundamental and controversial one. How do increasingly sophisticated search engines change the way we interact with information? What is the role that directories (such as Yahoo!) and personal ressource collections (such as “Favorite folders”) play in a world where search engines can (re)find much information we require with increasing precision? To give an example: Would an electronic record of all resources that a user has ever visited – and a corresponding search interface to them – replace the need for information organization ala delicious or Browser Favorites? (all privacy concerns set aside for a moment). How would such a development relate to the desire of users to share information with friends?

Search intent is poorly understood:

While there has been some work on search queries and query log analysis, the intent behind queries remains largely elusive. Existing distinctions (such as the one by Broder) need further elaboration and refinement. An example would be what Rakesh called pseudo-navigational queries – where the user has a certain expectation about the information, but this information can be found on several sites (e.g. wikipedia, an encyclopedia or other sites).

Conflict in tagging systems:

Tagging systems are largely tolerant of conflicts, for example, with regard to tagging semantics. This is different from systems such as wikipedia, where conflict is regarded to be an important aspect of the collaboration process. Twitter seems to lie in between those extremes, where conflict can emerge easily (e.g. around hashtags) , with some rudimentary support for resolution.

I truly enjoyed these conversations, and hope that they will continue at some point in the future.

Motivations for Tagging: Categorization vs. Description

UPDATE March 17 2010: More results can be found in the following publication: M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users’ Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media (ICWSM2010), Washington, DC, USA, May 23-26, 2010. (Download pdf)

In a past post, I talked about the role of tagging motivation in social tagging systems, and a distinction between users who use tags for Categorization and users who use tags for Description purposes.

One question that is interesting in this context is: “How do tag clouds of Categorizers respectively Describers actually look like – and what can we learn from them?“.

Categorizers vs. Describers: Our previous work suggests how tag clouds of Categorizers/Describers would look like theoretically: Categorizers would rather use general terms for tagging, terms that are useful labels for categories based on his model of the world.  On the other hand, Describers would use terms that are specific to a resource or concepts that can be found directly within a resource, based on characteristics of the resource. That’s the theory.

Christian Körner, one of my PhD students, looked into this question empirically based on his current work, where he applies previously discussed measures to detect tagging motivation (Conditional Tag Entropy and Orphaned Tags) to several tagging datasets. While in reality we expected that most tagging behaviour is the result of a combination of categorization and description motivation, Christian was particulary interested in “extreme” cases, i.e. cases of “extreme” Categorizers and “extreme” Describers. Here are selected results:

Example of an Extreme Categorizer: Among 445 delicious users, the following screenshot shows the tag cloud of the single user that scored highest on our “Categorization” measure (the most extreme Categorizer in our dataset).

An example tag cloud of an "Extreme Categorizer" (based on ~1900 bookmarks)

The results are quite intriguing: The above user clearly uses very general terms to annotate his resources, and introduces an elaborated taxonomy to categorize them. While some parts of his vocabulary are more elaborate and fine grained (e.g. “fashion” and corresponding sub-categories “fashion_blog” and “fashion_brand”) others are less elaborated  (e.g. “games, health, etc”). The user also produced a controlled vocabulary and sticked to it over the course of 1900 bookmarks, which I think can be seen as another indication for the inclination of this user to use tags for categorization purposes. The fact that a combination of our measures for tagging motivation (Conditional Tag Entropy and Orphaned Tags) has produced this interesting example of an extreme Categorizers provides some evidence for the plausibility of these measures. I think that’s great news.

Example of an Extreme Describer: The next screenshot shows an excerpt of a tag cloud of the user that scored highest on the “Description” measure (the most extreme Describer in our dataset).

An example tag cloud of an "Extreme Describer" (excerpt, based on ~1700 bookmarks)

It is interesting to note that this tag cloud represents an excerpt, the original tag cloud of this user is ~twice this size. The user clearly introduces a large set of tags, and uses many different variations of the same or similiar concepts, without much consideration with regard to terminological or conceptual differences (e.g. exce,  excel, Excel_Functions, Excel2007, Exceler, excelets, ExcelPoster, Excl, excxel). Again, the fact that our measures for tagging motivation produced this particular user as an extreme example of a Describer can be seen as an indicator for the principle plausibility of our measures.

However, what is also apparent from this example is that even in the case of this extreme Describer, some categories seem to be present in his tag vocabulary (e.g. “ebooks, fun, etc”). This suggests that a binary approach to understanding tagging motivation (a user is EITHER a Categorizer OR a Describer) is inplausible.

Open Questions: Overall, the examples of two users motivated by diametrically different motivations for tagging raises a number of interesting questions worth studying: What are characteristics, utilities and properties of tags produced by Categorizers and Describers? How do these different types of tagging motivation influence resulting folksonomies? And how do they influence quality attributes of algorithms (e.g. search, ranking) and applications (e.g. tag recommendation) that are processing folksonomical data? We are looking into some of these questions in our current research.

UPDATE March 17 2010: More results can be found in the following publication: M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users’ Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media (ICWSM2010), Washington, DC, USA, May 23-26, 2010. (Download pdf)

Why do users tag? Detecting user motivation in tagging systems

On the “social web” or “web2.0”, where user participation is entirely voluntarily, User Motivation has been identified as a key factor in the mechanisms contributing to the success of tagging systems. Web researchers are trying to identify the reasons why tagging systems work for a couple of years now, evident in, for example, the organization of a panel at CHI 2006 and a number of conferences and workshops on this topic.

Recent research on tagging motivation suggests that it is a rather complex construct. However, there seems to be emerging consensus that a distinction between at least two categories of tagging motivation appears useful: Categorization vs. Description. (Update May 30 2009: I was able to trace back the earliest mention of this distinction to a blog post by Tom Coates from 2005).

UPDATE March 15 2010 – More results can be found in: M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users’ Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media (ICWSM2010), Washington, DC, USA, May 23-26, 2010. (Download pdf)

UPDATE April 23 2010 – Even more results in: C. Körner, R. Kern, H.P. Grahsl, M. Strohmaier, Of Categorizers and Describers: An Evaluation of Quantitative Measures for Tagging Motivation, 21st ACM SIGWEB Conference on Hypertext and Hypermedia (HT2010), Toronto, Canada, June 13-16, ACM, 2010. (download pdf)

Categorization vs. Description

Categorization: Users who are motivated by Categorization engage in tagging because they want to construct and maintain a navigational aid to the resources (URLs, photos, etc) being tagged. This typically implies a limited set of tags (or categories) that is rather stable. Resources are assigned to tags whenever they share some common characteristic important to the mental model of the user (e.g. ‘family photos’, ‘trip to Vienna’ or ‘favorite list of URLs’). Because the tags assigned are very close to the mental models of users, they can act as suitable facilitators for navigation and browsing.

Description: On the other hand, users who are motivated by Description engage in tagging because they want to accurately and precisely describe the resources being tagged. This typically implies an open set of tag, with a rather dynamic and unlimited tag vocabulary. The goal of tagging is to identify those tags that match the resource best. Because the tags assigned are very close to the content of the resources, they can act as suitable facilitators for description and searching.

Related Research: This basic distinction can be identified in the work of a number of researchers who have made similar distinctions: Xu et al 2006 (“Context-based” vs. “Content-based”), Golder and Huberman 2006 (“Refining Categories” vs. “Identifying what it is/is about”), Marlow et al 2006 (“Future retrieval” – “Contribution and Sharing”),  Ames and Naaman 2007 (“Organization” vs. “Communication”) and Heckner et al 2008 (“Personal Information Management vs. Sharing”), just to give a few examples, all represent recent research aiming to demystify and conceptualize the reasons why users participate in tagging systems.

Why should we care?

“In the wild“, user behavior on social tagging systems is often a combination of both. So why is this distinction interesting? I believe that this distinction is interesting because it has a number of important implications, including but not limited to:

1. Tag Recommender Systems: Assuming that a user is a “Categorizer”, he will more likely reject tags that are recommended from a larger user population because he is primarily interested in constructing and maintaing “her” taxonomy, using “her” individual tag vocabulary.
2. Search: Tags produced by “Describers” are more likely to be helpful for search and retrieval because they focus on the content of resources, where tags produced by “Categorizers” focus on their mental model. Tags by categorizers thus are more subjective, whereas tags by describers are more objective.
3. Knowledge Acquisition: Folksonomies, i.e. the conceptual structures that can be inferred from the tripartite graph of tagging systems, are likely to be influenced by the “mixture” or dominance of categorizers and describers in their system. A tagging system primarily populated by categorizers is likely to give rise to a completely different set of possible folksonomies than tagging systems primarily populated  by describers. More importantly, it is plausible to assume that even within certain tagging systems, tagging motivation among users vary.

This brings me to a small research project I am currently working on: Assuming that a) this distinction in user motivation exists in real-world tagging systems and b) it has important implications, it would be interesting to measure and detect the degree to which users are Categorizers or Describers. Due to the latent nature of “tagging motivation”, past research has mostly focused on questionnaire or sample-based studies of motivation, asking users how they interpret their tagging behavior themselves. While this early work has provided fundamental insights into tagging motivation and contributed significantly to theory building, as a research community, we currently lack robust metrics and automatic methods to detect tagging motivation in tagging systems without direct user interaction.

Detecting Tagging Motivation

I think there are several approaches to detect whether users are Categorizers or Describers without the need to ask them directly. One approach would focus on analyzing the semantics of tags, using wordnet and other knowledge bases to determine the meaning of tags and infer user motivation. This would require parsing of text and performing linguistic analysis, which I believe is difficult in the presence of typos, named entities, combined tags (“toread”) and other issues. Another approach would focus on comparing the tag vocabulary of users to the tag vocabulary of “the crowd”. Users that share a greater set of common tag vocabulary might be describers, whereas users having a highly individual vocabulary might be categorizers. Again there are problems: Tagging systems that accomodate users with different language backgrounds might be prone to detecting user motivation based on false premises.

So what would be a more robust way of detecting user motivation? I am currently interested in developing a model that would be agnostic to language, semantics or social context, focusing solely on statistical properties of individual tagging histories. This way, a determination of user motivation could be made without linguistic analysis or acquiring complete folksonomies from tagging systems, based on a single users’ log of tagging. Let me explain what I mean. I hypothesize that the following statistical properties of users’ tagging history allows to conduct interesting analyses:

1. Tag Vocabulary size over time: Over time, an ideal Categorizer’s tag vocabulary would reach a plateau, because there is only limited categories that are of interest to him. An ideal Describer is not limited in terms of her tagging vocabulary. This should be easy to observe.
2. Tag Entropy over time: A Categorizer has an incentive to maintain high entropy (or “information value”) in his tag cloud. Tags would need to be as discriminative as possible in order for him to use them as a navigational aid, otherwise tags would be of little use in browsing. A describer would not have an interest to maintain high entropy.
3. Percentage of Tag Orphans over time: Categorizers have an interest in a low ratio of Tag Orphans (tags that are only used once) in their set of tags, because lots of orphans would inhibit the usage of their set of tags for browsing. Describers naturally produce lots of orphans when trying to find the most descriptive and complete set of tags for resources.
4. Tag Overlap: While a Describer would be perfectly fine assigning two or more synonymous tags to the same resource (he might not know which term to use when searching for this resource at a later point), a Categorizer would not have an interest in creating two categories that contain the exact same set of resources. This would again inhibit the usage of tags for browsing, a Categorizers’ main motivation for tagging.

Preliminary Investigations

I have done some preliminary investigations to explore whether these statistical properties of users’ tagging history can actually serve as indicators of tagging motivation. Here are my preliminary results:

Growth of tag vocabulary in different tagging systems

The diagram above shows the growth of tag vocabulary of different taggers. The upper most red line represents tagging behavior of an almost “ideal” Describer, in this case tags produced by the ESP game, that contain a set of tags that represent valid descriptions of the resources they are assigned to. The lower most green line represents tagging behavior of an almost “ideal” Categorizer, tags (in this case: a number of photo sets) produced by a flickr user that categorized photos into a limited set of categories (> 100 sets). All other lines represent tagging behavior of real users on different tagging platforms (bibsonomy, delicious, flickr tags). It is worth noting that all other data lies between the two identified extremes.

In the following, I will discuss the suitability of tag entropy of single users (as opposed to the work by Chi and T. Mytkowicz 2008 focusing on large sets of users) as an indicator for detecting tagging motivation:

Change of tag entropy over time

In this diagram, we can see that while our “ideal” Categorizer and our “ideal” Describer almost describe extremes, there are some users “outdoing” them (e.g. “u5 bibsonomy bookmarks” has even lower entropy than the tags acquired from the “ideal” Describer “ESP game”). Entropy thus seems to be -to some extent – a useful indicator for tagging motivation.

Next, I’ll discuss data comparing the rate of tag orphans in different datasets:

Rate of Tag Orphans over time

Like in the previous diagram, extreme behavior represents a good (but not optimal) upper and lower bound for real tagging behavior. While the “ideal” Categorizer (flickr sets, green line at the bottom) has a very small number of tag orphans, the “ideal” Describer (ESP game data, red line at the top) has a much higher tag orphan rate.

If we can identify the functions of extreme user motivation “(ideal” Categorizers and Describers), and position real user motivation between those extremes, we might be able to come up with scores indicative of user motivation in tagging systems – e.g. a user might be 80% Categorizer, and 20% Describer. Having such a model could help exploring the implications of different user motivations outlined above. Together with students (in particular Christian Körner, Hans-Peter Grahsl and Roman Kern), I am working on constructing and validating such a model, which we are aiming to submit to a conference this year.

UPDATE March 15 2010: More results can be found in the following publication: M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users’ Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media (ICWSM2010), Washington, DC, USA, May 23-26, 2010. (Download pdf)

UPDATE April 23 2010 – Even more results in: C. Körner, R. Kern, H.P. Grahsl, M. Strohmaier, Of Categorizers and Describers: An Evaluation of Quantitative Measures for Tagging Motivation, 21st ACM SIGWEB Conference on Hypertext and Hypermedia (HT2010), Toronto, Canada, June 13-16, ACM, 2010. (download pdf)

References:

Z. Xu and Y. Fu and J. Mao and D. Su. Towards the Semantic Web: Collaborative Tag Suggestions. Proceedings of the Collaborative Web Tagging Workshop at the WWW 2006, Edinburgh, Scotland, 2006.

Kerstin Bischoff and Claudiu S. Firan and Wolfgang Nejdl and Raluca Paiu. Can all tags be used for search?. CIKM ’08: Proceeding of the 17th ACM conference on Information and knowledge management, 193–202, ACM, New York, NY, USA, 2008.

C. Marlow and M. Naaman and D. Boyd and M. Davis. HT06, tagging paper, taxonomy, Flickr, academic article, to read. HYPERTEXT ’06: Proceedings of the seventeenth conference on Hypertext and hypermedia, 31–40, ACM, New York, NY, USA, 2006.

S.A. Golder and B.A. Huberman. Usage patterns of collaborative tagging systems. Journal of Information Science, (32)2:198, 2006.

Cameron Marlow and Mor Naaman and Danah Boyd and Marc Davis. HT06, tagging paper, taxonomy, Flickr, academic article, to read. HYPERTEXT ’06: Proceedings of the seventeenth conference on Hypertext and hypermedia, 31–40, ACM, New York, NY, USA, 2006.

M. Heckner and M. Heilemann and C. Wolff. Personal Information Management vs. Resource Sharing: Towards a Model of Information Behaviour in Social Tagging Systems. Int’l AAAI Conference on Weblogs and Social Media (ICWSM), San Jose, CA, USA, 2009.

M. Ames and M. Naaman. Why we tag: motivations for annotation in mobile and online media. CHI ’07: Proceedings of the SIGCHI conference on Human factors in computing systems, 971–980, ACM, New York, NY, USA, 2007.

E.H. Chi and T. Mytkowicz. Understanding the efficiency of social tagging systems using information theory. Proceedings of the nineteenth ACM conference on Hypertext and hypermedia, 81–88, 2008.