Notes on the Relationship between Search and Tagging

13 09 2009

I had a number of exciting and very inspiring conversations this week with Marc, Rakesh, Fabian, Cathy, and Pranam, as well as with Ed and Rowan. It was great talking to everybody and I wanted to share some of the issues that were discussed. Most conversations focused on the role of tagging, and how it relates to searching the web. I do not claim that any of these interesting thoughts are mine or that my notes offer answers.  They merely aim to serve as pointers for what I consider important issues.

A minority of resources on the web is tagged:

A number of current research projects study the question how tagged resources can inform/improve search. However, a minority of resources on the web is tagged, and the gap between tagged and non-tagged resources is likely increasing (although this seems difficult to predict cf. Paul Heymann’s work). This would mean that a decreasing ratio of resources on the web have tagged information associated with it. The question then becomes: Why bother analyzing tagging systems in the first place when their (relative) importance is likely to decrease over time?

Tagged resources exhibit topical bias (that’s a bad thing!):

Tagging is often a geek activity. I am not aware of any studies of delicious’ user population, but it is likely that delicious’ users are more geeky than the rest of the population. This is a bad thing because it would bias any broad attempt leveraging tagging for search. The bias might depend on the particular tagging system though: Flickr seems to have a much broader, and thereby more representative, user base.

Bookmarks exhibit timely bias (that’s a good thing!):

Bookmarking typically represents an event in time triggered by some user. Most tagging systems therefore provide timestamp information, allowing to infer more information about the context in which a given resource is being tagged. This allows us to use tagging systems for studying how information on the web is organized, filtered, diffused and consumed.

Search supercedes any other form of information access/organisation:

I found this issue to be the most fundamental and controversial one. How do increasingly sophisticated search engines change the way we interact with information? What is the role that directories (such as Yahoo!) and personal ressource collections (such as “Favorite folders”) play in a world where search engines can (re)find much information we require with increasing precision? To give an example: Would an electronic record of all resources that a user has ever visited – and a corresponding search interface to them – replace the need for information organization ala delicious or Browser Favorites? (all privacy concerns set aside for a moment). How would such a development relate to the desire of users to share information with friends?

Search intent is poorly understood:

While there has been some work on search queries and query log analysis, the intent behind queries remains largely elusive. Existing distinctions (such as the one by Broder) need further elaboration and refinement. An example would be what Rakesh called pseudo-navigational queries – where the user has a certain expectation about the information, but this information can be found on several sites (e.g. wikipedia, an encyclopedia or other sites).

Conflict in tagging systems:

Tagging systems are largely tolerant of conflicts, for example, with regard to tagging semantics. This is different from systems such as wikipedia, where conflict is regarded to be an important aspect of the collaboration process. Twitter seems to lie in between those extremes, where conflict can emerge easily (e.g. around hashtags) , with some rudimentary support for resolution.

I truly enjoyed these conversations, and hope that they will continue at some point in the future.

Advertisements




Motivations for Tagging: Categorization vs. Description

21 07 2009

UPDATE March 17 2010: More results can be found in the following publication: M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users’ Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media (ICWSM2010), Washington, DC, USA, May 23-26, 2010. (Download pdf)

In a past post, I talked about the role of tagging motivation in social tagging systems, and a distinction between users who use tags for Categorization and users who use tags for Description purposes.

One question that is interesting in this context is: “How do tag clouds of Categorizers respectively Describers actually look like – and what can we learn from them?“.

Categorizers vs. Describers: Our previous work suggests how tag clouds of Categorizers/Describers would look like theoretically: Categorizers would rather use general terms for tagging, terms that are useful labels for categories based on his model of the world.  On the other hand, Describers would use terms that are specific to a resource or concepts that can be found directly within a resource, based on characteristics of the resource. That’s the theory.

Christian Körner, one of my PhD students, looked into this question empirically based on his current work, where he applies previously discussed measures to detect tagging motivation (Conditional Tag Entropy and Orphaned Tags) to several tagging datasets. While in reality we expected that most tagging behaviour is the result of a combination of categorization and description motivation, Christian was particulary interested in “extreme” cases, i.e. cases of “extreme” Categorizers and “extreme” Describers. Here are selected results:

Example of an Extreme Categorizer: Among 445 delicious users, the following screenshot shows the tag cloud of the single user that scored highest on our “Categorization” measure (the most extreme Categorizer in our dataset).

delicious_categorizer

An example tag cloud of an "Extreme Categorizer" (based on ~1900 bookmarks)

The results are quite intriguing: The above user clearly uses very general terms to annotate his resources, and introduces an elaborated taxonomy to categorize them. While some parts of his vocabulary are more elaborate and fine grained (e.g. “fashion” and corresponding sub-categories “fashion_blog” and “fashion_brand”) others are less elaborated  (e.g. “games, health, etc”). The user also produced a controlled vocabulary and sticked to it over the course of 1900 bookmarks, which I think can be seen as another indication for the inclination of this user to use tags for categorization purposes. The fact that a combination of our measures for tagging motivation (Conditional Tag Entropy and Orphaned Tags) has produced this interesting example of an extreme Categorizers provides some evidence for the plausibility of these measures. I think that’s great news.

Example of an Extreme Describer: The next screenshot shows an excerpt of a tag cloud of the user that scored highest on the “Description” measure (the most extreme Describer in our dataset).

delicious_describer

An example tag cloud of an "Extreme Describer" (excerpt, based on ~1700 bookmarks)

It is interesting to note that this tag cloud represents an excerpt, the original tag cloud of this user is ~twice this size. The user clearly introduces a large set of tags, and uses many different variations of the same or similiar concepts, without much consideration with regard to terminological or conceptual differences (e.g. exce,  excel, Excel_Functions, Excel2007, Exceler, excelets, ExcelPoster, Excl, excxel). Again, the fact that our measures for tagging motivation produced this particular user as an extreme example of a Describer can be seen as an indicator for the principle plausibility of our measures.

However, what is also apparent from this example is that even in the case of this extreme Describer, some categories seem to be present in his tag vocabulary (e.g. “ebooks, fun, etc”). This suggests that a binary approach to understanding tagging motivation (a user is EITHER a Categorizer OR a Describer) is inplausible.

Open Questions: Overall, the examples of two users motivated by diametrically different motivations for tagging raises a number of interesting questions worth studying: What are characteristics, utilities and properties of tags produced by Categorizers and Describers? How do these different types of tagging motivation influence resulting folksonomies? And how do they influence quality attributes of algorithms (e.g. search, ranking) and applications (e.g. tag recommendation) that are processing folksonomical data? We are looking into some of these questions in our current research.

UPDATE March 17 2010: More results can be found in the following publication: M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users’ Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media (ICWSM2010), Washington, DC, USA, May 23-26, 2010. (Download pdf)Motivations for Tagging: Categorization vs. Description





ACM Hypertext’09 Student Competition

4 07 2009

I just came back from a road trip to ACM Hypertext’09 with my students, and I’m particulary happy that one of them, Christian Körner, won the 1st place in this year’s ACM Hypertext’09 Graduate Student Research Challenge. Bravo Christian!

The competition was strong, and Christian did a great job in presenting preliminary results from his PhD research on Tagging Motivation. Here are a few links to his competition material:

I would also like to congratulate all runner-ups at the competition. All participating students worked really hard to present their research to conference attendees in an engaging way. I really liked the enthusiasm of the students, and the student competition as well as the conference as a whole gave me a bunch of new ideas and perspectives.

You might also be interested in my liveblogging notes on Ricardo Baeza-Yates‘ and Lada Adamic‘s very interesting keynotes at the conference.

ACM Hypertext'09 Student Research Competition





Real Estate Intent on the Web

26 09 2008

The US housing crisis has been frequently reported as one major cause for the current financial turmoil. I was wondering whether indicators for the housing crisis can be found in Google’s Search Query Logs.

To look into this, I compared the search volume for the following explicit intentional queries: “buy a house”, “rent a house”, “sell my house” and “find a house”. The results are interesting, yet hardly surprising.

Tracking Real Estate Intent on Google Trends

Tracking Real Estate Intent on Google Trends

” Buy a house” undergoes seasonal fluctuations, with peaks at the end/beginning of every year. Overall though, there seems to be a downward trend. At the same time, “sell my house” and “rent a house” are on the rise. “Find a house” is relatively stable, but slowly declining as well. Although subtle, the housing crisis can be identified in the data.

Google Trends seems to provide some interesting data (such as the one above), yet I miss some features and several questions remain unanswered. I’d love to see a mashup with Google Maps, where I can not only plot the queries over time, but map them on different regions of the US. Questions that I would like to have answered include: 1) What is the absolute search volume of queries? Google does not give a way the absolute number of queries per of interest. 2) Does Google account for rising query volume? I assume that the total number of queries issued in 2004 is significantly lower than the total number of queries issued in 2007. So does a decline in, for example, the blue curve refer to an absolute decline in numbers, or to a relative decline that factors in overall query volume increase? 3) “Sell my house” does not have any data before 2005 – what does this mean exactly? It seems odd that people on the web did not (or hardly) use the term “sell my house” before 2005 on Google.

Yet, the kind of analysis provided by Google hints towards potential applications of taking an explicit goal-oriented stance on the web.

Update (Oct 7 2008): It seems that Google Insights for Search covers many of the issues identified above. See the example here (you must be logged into your Google Account to see the numbers).





Customer desires in disguise: My Starbucks Idea

11 06 2008

I recently stumbled upon “My Starbucks Idea“, which I think is a pretty smart move by Starbucks to 1) integrate their customers knowledge into the innovation process and 2) learn more about their customers needs and desires.

Customers can post their ideas, have them rated, discuss them, and receive updates on the status of their ideas (“Ideas in Action”). With “My Starbucks Idea”, Starbucks not only receives access to a large set of users expectations that can be analyzed and studied, but also some preliminary relevance ratings.

There are some great examples of things that customers want, but Starbucks does not yet adequately support, such as:

You could easily think of similar websites for other products, such as cars, fast food restaurants, holiday trips or others. I am sort of surprised to see that Starbucks focuses on ideas only, rather than the more general goals and desires of their customers (e.g. how many of Starbucks customers have an interest in meditation? and would they buy meditation products in a coffee shop?).

What is certainly interesting about the Starbucks site is that there is no monetary reward for customers to contribute (not even free coffee!).





CSKGOI Workshop Proceedings online

13 02 2008

Mathias just announced that the proceedings of our CSKGOI workshop at the IUI conference are now available at the CEUR workshop proceedings server (CSKGOI Workshop proceedings CEUR-WS Vol-323). Thanks Mathias’ for putting all this together and thanks to everyone who contributed to this nice little event, which was an excellent occasion to discuss current research on commonsense knowledge and goal-oriented interfaces with like-minded colleagues.

Mathias, Peter and I also presented some results of our own research that focused on studying the nature and structure of user goals in the AOL search query log – a log that contains more than 20 mio search queries.

Here’s the link to our paper in case you are interested:

Different Degrees of Explicitness in Intentional Artifacts: Studying User Goals in a Large Search Query Log, Markus Strohmaier, Peter Prettenhofer and Mathias Lux





Studying Goals on 43things.com

8 02 2008

In a recent effort, Thomas, a student in my research group, explored goals and goal relations on 43things.com. You can see an outcome of his effort in the Pajek screenshot below (click here for a larger version of the graph):

Goal Association Graph

We used 43things.com API to crawl a minor set of goals and relations between them. The relations were inferred via a rather naive approach using simple weights for user- and tag- co-occurrences as well as 43things.com’s similiar-goal API call. Surprisingly, many of the strongest associations inferred are rather intuitive. Utilizing 43things.com’s API, this small exercise illustrates the potential of large socially-constructed corpora for collecting common sense knowledge (such as “fall in love” helps to “be happy” – as illustrated in the above screenshot). Making such common sense knowledge more explicit on one hand might help to aid users in formulating, progressing towards and satisfying their goals on the web, and on the other hand might help systems to identify, understand, assess and reason about users’ requirements and goals.