3 Panciera, K. Priedhorsky, R. Erickson, T. Terveen, L. 2010 Lurking? Cyclopaths? A Quantitative Lifecyle Analysis of User Behavior in a Geowiki ACM Conference on Computer-Human Interaction (CHI) Atlanta, GA Association for Computing Machinery 04/2010 Wiki, geowiki, open content, geographic volunteer work, volunteered geographic information, lurking 3 Chen, J. Nairn, R. Nelson, L. Bernstein, M. Chi, E. 2010 Short and Tweet: Experiments on Recommending Content from Information Streams ACM Conference on Human Factors in Computing Atlanta, GA Association for Computing Machinery 04/10/2010 <p>More and more web users keep up with newest information through information streams such as the popular microblogging website Twitter. In this paper we studied content recommendation on Twitter to better direct user attention. In a modular approach, we explored three separate dimensions in designing such a recommender: content sources, topic interest models for users, and social voting. We implemented 12 recommendation engines in the design space we formulated, and deployed them to a recommender service on the web to gather feedback from real Twitter users. The best performing algorithm improved the percentage of interesting content to 72% from a baseline of 33%. We conclude this work by discussing the implications of our recommender design and how our design can generalize to other information streams.</p> 3 Chen, J. Ren, Y. Riedl, J. 2010 The Effects of Diversity on Group Productivity and Member Withdrawal in Online Volunteer Groups ACM Conference on Human Factors in Computing Atlanta, GA Association for Computing Machinery 04/10/2010 online volunteer group, diversity, performance, Wikipedia <p>The "wisdom of crowds" argument emphasizes the importance of diversity in online collaborations, such as open source projects and Wikipedia. However, decades of research on diversity in offline work groups have painted an inconclusive picture. On the one hand, the broader range of insights from a diverse group can lead to improved outcomes. On the other hand, individual differences can lead to conflict and diminished performance. In this paper, we examine the effects of group diversity on the amount of work accomplished and on member withdrawal behaviors in the context of WikiProjects. We find that increased diversity in experience with Wikipedia increases group productivity and decreases member withdrawal -- up to a point. Beyond that point, group productivity remains high, but members are more likely to withdraw. Strikingly, no such diminishing returns were observed for differences in member interest, which increases productivity and decreases member withdrawal in a linear fashion. Our results suggest that the low visibility of individual differences in online groups may allow them to harvest more of the benefits of diversity while bearing less of the cost. We discuss how our findings can inform further research of online collaboration.</p> 3 Priedhorsky, R. Masli, M. Terveen, L. 2010 Eliciting and Focusing Geographic Volunteer Work 2010 Conference on Computer Supported Cooperative Work Savannah, GA, USA Association for Computing Machinery 02/2010 wiki, geowiki, open content, geographic volunteer work, volunteered geographic information, collaborative computing, computer-supported cooperative work, web-based interaction 3 Sen, S. Vig, J. Riedl, J. 2009 Tagommenders: Connecting Users to Items through Tags International World Wide Web Conference Madrid, Spain ACM 4/20/2009 <p>Tagging has emerged as a powerful mechanism that enables users to ï¬］d, organize, and understand online entities. Recommender systems similarly enable users to efſtiently navigate vast collections of items. Algorithms combining tags with recommenders may deliver both the automation inherent in recommenders, and the ï¬Ｆxibility and conceptual comprehensibility inherent in tagging<br />systems. In this paper we explore tagommenders, recommender algorithms that predict usersâ preferences for items based on their inferred preferences for tags. We describe tag preference inference algorithms based on usersâ interactions with tags and movies, and evaluate these algorithms based on tag preference ratings collected from 995 MovieLens users. We design and evaluate algorithms that predict usersâ ratings for movies based on their inferred tag preferences. Our tag-based algorithms generate better recommendation rankings than state-of-the-art algorithms, and they may lead to ï¬Ｆxible recommender systems that leverage the characteristics of items users ï¬］d most important.</p> 3 Sen, S. Vig, J. Riedl, J. 2009 Learning to Recognize Valuable Tags International Conference on Intelligent User Interfaces Sanibel Island, FL ACM 2/8/2009 <p>Many websites use tags as a mechanism for improving item metadata through collective user eï¬ort. Users of tagging systems often apply far more tags to an item than a system can display. These tags can range in quality from tags that capture a key facet of an item, to those that are subjective, irrelevant, or misleading. In this paper we explore tag selection algorithms that choose the tags that sites display. Based on 225,000 ratings and survey responses, we conduct oï¬ine analyses of 21 tag selection algorithms. We select the three best performing algorithms from our oï¬ine analysis, and deploy them live on the MovieLens website to 5,695 users for three months. Based on our results, we oï¬er tagging system designers advice about tag selection algorithms.</p> 3 Ekstrand, M.D. Riedl, J.T. 2009 rv you're dumb: Identifying Discarded Work in Wiki Article History The Fifth International Symposium on Wiki's and Open Collaboration Orlando, FL 10/25/2009 <p>Wiki systems typically display article history as a linear sequence of revisions in chronological order. This presentation hides deeper relationships among the revisions, such as which earlier revision provided most of the content for a later revision, or when a revision effectively reverses the changes made by a prior revision. These relationships are valuable in understanding what happened between editors in conflict over article content. We present methods for detecting when a revision discards the work of one or more other revisions, a means of visualizing these relationships in-line with existing history views, and a computational method for detecting discarded work. We show through a series of examples that these tools can aid mediators of wiki content disputes by making salient the structure of the ongoing conflict. Further, the computational tools provide a means of determining whether or not a revision has been accepted by the community of editors surrounding the article.</p> 3 Halfaker, A. Kittur, N. Kraut, R. Riedl, J. 2009 A Jury of Your Peers: Quality, Experience and Ownership in Wikipedia The International Symposium on Wiki's and Open Collaboration Orlando, FL 10/2009 wikipedia, peer, peer review, wikiwork, experience, ownership, quality <p>Wikipedia is a highly successful example of what mass collaboration in an informal peer review system can accomplish. In this paper, we examine the role that the quality of the contributions, the experience of the contributors and the ownership of the content play in the decisions over which contributions become part of Wikipedia and which ones are rejected by the community. We introduce and justify a versatile metric for automatically measuring the quality of a contribution. We ï¬］d little evidence that experience helps contributors avoid rejection. In fact, as they gain experience, contributors are even more likely to have their work rejected. We also ï¬］d strong evidence of ownership behaviors in  practice despite the fact that ownership of content is discouraged within Wikipedia.</p> 3 Lam, S.K. Riedl, J. 2009 Is Wikipedia Growing a Longer Tail? ACM 2009 International Conference on Supporting Group Work Sanibel Island, FL Association for Computing Machinery 05/10/2009 <p>Wikipedia has millions of articles, many of which receive little attention. One group of Wikipedians believes these obscure entries should be removed because they are uninteresting and neglected; these are the deletionists. Other Wikipedians disagree, arguing that this long tail of articles is precisely Wikipediaâ冱 advantage over other encyclopedias; these are the inclusionists. This paper looks at two overarching questions on the debate between deletionists and inclusionists: (1) What are the implications to the long tail of the evolving standards for article birth and death? (2) How is viewership affected by the decreasing notability of articles in the long tail? The answers to five detailed research questions that are inspired by these overarching questions should help better frame this debate and provide insight into how Wikipedia is evolving.</p> 3 Reily, K. Finnerty, P.L. Terveen, L. 2009 Two peers are better than one: aggregating peer reviews for computing assignments is surprisingly accurate ACM 2009 International Conference on Supporting Group Work Sanibel Island, FL Association for Computing Machinery 115-124 05/10/2009 978-1-60558-500-0 <p>Scientific peer review, open source software development, wikis, and other domains use distributed review to improve quality of created content by providing feedback to the work's creator. Distributed review is used to assess or improve the quality of a work (e.g., an article). However, it can also provide learning benefits to the participants in the review process. We developed an online review system for beginning computer programming students; it gathers multiple anonymous peer reviews to give students feedback on their programming work. We deployed the system in an introductory programming class and evaluated it in a controlled study. We find that: peer reviews are accurate compared to an accepted evaluation standard, that students prefer reviews from other students with less experience than themselves, and that participating in a peer review process results in better learning outcomes.</p> <p></p> 3 Panciera, K. Halfaker, A. Terveen, L. 2009 Wikipedians are born, not made: a study of power editors on Wikipedia ACM 2009 International Conference on Group Work Sanibel Island, FL Association for Computing Machinery 51-60 05/10/2009 computer-supported cooperative work, web-based interaction <p>Open content web sites depend on users to produce information of value. Wikipedia is the largest and most well-known such site. Previous work has shown that a small fraction of editors â Wikipedians â do most of the work and produce most of the value. Other work has oï¬ered conjectures about how Wikipedians differ from other editors and how Wikipedians change over time. We quantify and test these conjectures. Our key findings include: Wikipedians' edits last longer; Wikipedians invoke community norms more often to justify their edits; on many dimensions of activity, Wikipedians start intensely, tail off a little, then maintain a relatively high level of activity over the course of their career. Finally, we show that the amount of work done by Wikipedians and non-Wikipedians differs significantly from their very first day. Our results suggest a design opportunity: customizing the initial user experience to improve retention and channel new users' intense energy.</p> <p></p> 3 Chen, J Nairn, R. Nelson, L. Bernstein, M. Chi, E. 2009 Short and Tweet: Experiments on Recommending Content from Information Proceedings of the 28th annual conference on Human Factors in Computing Systems Atlanta, GA ACM Press 04/10/10 <p>More and more web users keep up with newest information through information streams. One prominent example of an information stream is the popular micro-blogging website Twitter. In this paper we studied recommending content on Twitter to alleviate information overload and better direct user attention. In a modular approach, we explored three separate dimensions in designing such a recommender: selecting promising subsets of content for consideration, modeling user topic interest, and leveraging social process. We implemented 12 possible recommendation engines in the design space we formulated, and deployed them to a recommender service on the web to gather feedback from real Twitter users. The best performing algorithm improved the percentage of interesting content to 72% from a baseline of 33%. We conclude this work by discussing implications of our result and how our recommender design can generalize to other information streams.</p> 3 Vig, J. Sen, S. Riedl, J. 2009 Tagsplanations: Explaining Recommendations using Tags International Conference on Intelligent User Interfaces Sanibel Island, FL 02/08/2009 tagging, recommender systems <p>While recommender systems tell users what items they might like, explanations of recommendations reveal why they might like them. Explanations provide many beneï¬》s, from improving user satisfaction to helping users make better decisions. This paper introduces tagsplanations, which are explanations based on community tags. Tagsplanations have two key  components: tag relevance, the degree to which a tag describes an item, and tag preference, the userâ冱 sentiment toward a tag. We develop novel algorithms for estimating tag relevance and tag preference, and we conduct a user study exploring the roles of tag relevance and tag preference in promoting effective tagsplanations. We also examine which types of tags are most useful for tagsplanations.</p> 3 Harper, F. Maxwell Moy, D. Konstan, J.A. 2009 Facts or Friends? Distinguishing Informational and Conversational Questions in Social Q&A Sites ACM SIGCHI Conference on Human Factors in Computing Systems <p>Tens of thousands of questions are asked and answered<br />every day on social question and answer (Q&amp;A) Web sites<br />such as Yahoo Answers. While these sites generate an<br />enormous volume of searchable data, the problem of<br />determining which questions and answers are archival<br />quality has grown. One major component of this problem is<br />the prevalence of conversational questions, identified both<br />by Q&amp;A sites and academic literature as questions that are<br />intended simply to start discussion. For example, a<br />conversational question such as â彭o you believe in<br />evolution?â might successfully engage users in discussion,<br />but probably will not yield a useful web page for users<br />searching for information about evolution. Using data from<br />three popular Q&amp;A sites, we confirm that humans can<br />reliably distinguish between these conversational questions<br />and other  informational questions, and present evidence<br />that conversational questions typically have much lower<br />potential archival value than informational questions.<br />Further, we explore the use of machine learning techniques<br />to automatically classify questions as conversational or<br />informational, learning in the process about categorical,<br />linguistic, and social differences between different question<br />types. Our algorithms approach human performance,<br />attaining 89.7% classification accuracy in our experiments.</p> 3 Chen, J. Geyer, W. Dugan, C. Muller, M. Guy, I. 2009 Making New Friends, but Keep the Old - Recommending People on Social Networking Sites (forthcoming) ACM SIGCHI Conference on Human Factors in Computing Systems <p>This paper studies people recommendations designed to help users find known, offline contacts and discover new friends on social networking sites. We evaluated four  recommender  algorithms in an enterprise social networking site using a personalized survey of 500 users and a field study of 3,000 users. We found all algorithms effective in expanding usersâ friend lists. Algorithms based on social network information were able to produce better-received recommendations and find more known contacts for users, while algorithms using similarity of user-created content were stronger in discovering new friends. We also collected qualitative feedback from our survey users and draw several meaningful design implications.</p> 3 Ludwig, M. Priedhorsky, R. and Terveen, L. 2009 Path Selection: A Novel Interaction Technique for Mapping Applications ACM SIGCHI Conference on Human Factors in Computing Systems ACM 2309-2318 978-1-60558-246-7 bubble cursors, bubble targets, path selection, routing, selection techniques, human factors <p>Many online mapping applications let users define routes, <br />perhaps for sharing a favorite bicycle commuting route or rating several contiguous city blocks. At the UI level, defining a route amounts to selecting a fairly large number of objects â the individual segments of roads and trails that make up the route. We present a novel interaction technique for this task called  path selection. We implemented the technique and evaluated it experimentally, finding that adding path selection to a state-of-the-art technique for selecting individual objects reduced route definition time by about a factor of 2, reduced errors, and improved user satisfaction. Detailed analysis  of the results showed path selection is most advantageous (a) for routes with long straight segments and (b) when objects that are optimal click targets also are visually attractive.</p> 3 Riedl, J. 2008 Altruism, Selfishness, and Destructiveness on the Social Web Nejdl, W., Kay, J., Pu, P., Herder, E. 5th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems Hannover, Germany Springer-Verlag 9-11 7/29/2008 978-3-540-70984-8 <p>Many online communities are emerging that, like Wikipedia, bring people together to build community-maintained artifacts of lasting value (CALVs). What is the nature of people's participation in building these repositories? What are their motives? In what ways is their behavior destructive instead of constructive? Motivating people to contribute is a key problem because the quantity and quality of contributions ultimately determine a CALV's value. We pose three related research questions: 1) How does intelligent task routing--matching people with work--affect the quantity of contributions? 2) How does reviewing contributions before accepting them affect the quality of contributions? 3) How do recommender systems affect the evolution of a shared tagging vocabulary among the contributors? We will explore these questions in the context of existing CALVs, including Wikipedia, Facebook, and MovieLens.</p> 3 Agrahri, A.K., Manickam, D.T. and Riedl, J. 2008 Can people collaborate to improve the relevance of search results? ACM Conference on Recommender Systems Lausanne, Switzerland Association of Computing Machinery 283-286 10/23/2008 978-1-60558-093-7 <p>Search engines are among the most-used resources on the internet. However, even today's most successful search engines struggle to provide high quality search results. According to recent studies as many as 50 percent of web search sessions fail to find any relevant results for the searcher. Researchers have proposed social search techniques, in which early searchers provide feedback that is used to improve relevance for later searchers. In this paper we investigate foundational questions of social search. In particular, we directly assess the degree of agreement among users about the relevance ranking of search results. We developed a simulated search engine interface that systematically randomizes Google's normal relevance ordering of the items presented to users. Our results show that (a) people are biased toward items in the top of the search lists, even if the list is randomized; (b) people explicit feedback is not biased and (c) people's shared preferences do not always agree with Google's result order. These results suggest that social search techniques might improve the effectiveness of web search engines.</p> 3 Krishnan, V. Narayanashetty, P.K. Nathan, M. Davies, R.T. Konstan, J.A. 2008 Who predicts better? Results from an online study comparing humans and an online recommender system. ACM Conference on Recommender Systems Lausanne, Switzerland ACM 211-218 10/23/2008 978-1-60558-093-7 <p>Algorithmic recommender systems attempt to predict which items a target user will like based on information about the user's prior preferences and the preferences of a larger community. After more than a decade of widespread use, researchers and system users still debate whether such "impersonal" recommender systems actually perform as well as human recommenders. We compare the performance of MovieLens algorithmic predictions with the recommendations made, based on the same user profiles, by active MovieLens users. We found that algorithmic collaborative filtering outperformed humans on average, though some individuals outperformed the system substantially and humans on average outperformed the system on certain prediction tasks.</p> 3 Nathan, M. Harrison, C. Yarosh, S. Terveen, L. Stead, L. Amento, B. 2008 CollaboraTV: making television viewing social again 1st International Conference on Designing Interactive User Experiences for TV and Video Silicon Valley, CA ACM vol 291 85-94 10/22/2008 978-1-60558-100-2 <p>With the advent of video-on-demand services and digital video recorders, the way in which we consume media is undergoing a fundamental change. People today are less likely to watch shows at the same time, let alone the same place. As a result, television viewing, which was once a social activity, has been reduced to a passive and isolated experience. To study this issue, we developed a system called CollaboraTV and demonstrated its ability to support the communal viewing experience through a month-long field study. Our study shows that users understand and appreciate the utility of asynchronous interaction, are enthusiastic about CollaboraTV's engaging social communication primitives and value implicit show recommendations from friends. Our results both provide a compelling demonstration of a social television system and raise new challenges for social television communication modalities.</p> http://portal.acm.org/citation.cfm?doid=1453805.1453824 3 Reily, K. Ludford, P.J. Terveen, L. 2008 Sharescape: An interface for place annotation Nordic Conference on Human-Computered Interaction: Building Bridges Lund, Sweden ACM 358 326-333 10/20/2008 978-1-59593-704-9 <p>Many people use the Internet to search for geographically local information, with a growing number of websites dedicated to this task. However, it is not clear exactly how users integrate geographic search with content-based search, nor how to obtain reliable information about places in a geographic region. We created Sharescape, a map-based application in which information is contributed by community members. We conducted a user study to evaluate the utility of this means of obtaining information and to investigate how users integrate geographic and content-based search. Our results suggest that 1) maps create an implicit context in an interface that designers should honor, 2) community-maintained information about local geography has important benefits over information mined from web sites, and 3) users often are not aware of the privacy implications of their actions, and therefore designers should incorporate special privacy safeguards.</p> http://portal.acm.org/citation.cfm?doid=1463160.1463196 0 Jones, Q. Grandhi, S.A. Karam, S. Whittaker, S. Zhou, C. Terveen, L. 2008 Geographic 'Place' and 'Community Information' Preferences Computer Supported Cooperative Work 17 2-3 137-167 04/2008 P3-Systems, locomotive media, pervasive computing, place, social computing <p>People dynamically structure social interactions and activities at various locations in their environments in specialized types of places such as the office, home, coffee shop, museum and school. They also imbue various locations with personal meaning, creating group `hangouts' and personally meaningful `places'. Mobile location-aware community systems can potentially utilize the existence of such `places' to support the management of social information and interaction. However, acting effectively on this potential requires an understanding of how: (1) places and place-types relate to people's desire for place-related awareness of and communication with others; and (2) what information people are willing to provide about themselves to enable place-related communication and awareness. We present here the findings from two qualitative studies, a survey of 509 individuals in New York, and a study of how mobility traces can be used to find people's important places in an exploration of these questions. These studies highlight how people value and are willing to routinely provide information such as ratings, comments, event records relevant to a place, and when appropriate their location to enable services. They also suggest how place and place-type data could be used in conjunction with other information regarding people and places so that systems can be deployed that respect users' People-to-People-to-Places data sharing preferences. We conclude with a discussion on how `place' data can best be utilized to enable services when the systems in question are supported by a sophisticated computerized user-community social-geographical model.</p> 3 Drenner, S. Sen, S. Terveen, L. 2008 Crafting the initial user experience to achieve community goals ACM Conference on recommender Systems Lausanne, Switzerland ACM 978-1-60558-093-7 <p>Recommender systems try to address the "new user problem" by quickly and painlessly learning user preferences so that users can begin receiving recommendations as soon as possible. We take an expanded perspective on the new user experience, seeing it as an opportunity to elicit valuable contributions to the community and shape subsequent user behavior. We conducted a field experiment in MovieLens where we imposed additional work on new users: not only did they have to rate movies, they also had to enter varying numbers of tags. While requiring more work led to fewer users completing the entry process, the benefits were significant: the remaining users produced a large volume of tags initially, and continued to enter tags at a much higher rate than a control group. Further, their rating behavior was not depressed. Our results suggest that careful design of the initial user experience can lead to significant benefits for an online community.</p> http://portal.acm.org/citation.cfm?doid=1454008.1454039 3 F. Harper D. Raban S. Rafaeli J. Konstan 2008 Predictors of Answer Quality in Online Q&A Sites ACM SIGCHI Conference on Human Factors in Computing Systems <p>Question and answer (Q&amp;A) sites such as Yahoo! Answers are places where users ask questions and others answer them. In this paper, we investigate predictors of answer quality through a comparative, controlled field study of responses provided across several online Q&amp;A sites. Along with several quantitative results concerning the effects of factors such as question topic and rhetorical strategy, we present two high-level messages. First, you get what you pay for in Q&amp;A sites. Answer quality was typically higher in Google Answers (a fee-based site) than in the free sites we studied, and paying more money for an answer led to better outcomes. Second, we find that a Q&amp;A siteâ冱 community of users contributes to its success. Yahoo! Answers, a Q&amp;A site where anybody can answer questions, outperformed sites that depend on specific individuals to answer questions, such as library reference services.</p> http://doi.acm.org/10.1145/1357054.1357191 3 McCarthy, J.F. Congleton, B. Harper, F. Maxwell 2008 The Context, Content & Community Collage: Sharing Personal Digital Media in the Physical Workplace Computer Supported Cooperative Work San Diego, CA 97-106 978-1-60558-007-4 <p>Online social media services enable people to share many aspects of their personal interests and passions with friends, acquaintances and strangers. We are investigating how the display of social media in a workplace context can improve relationships among collocated colleagues. We have designed, developed and deployed the Context, Content and Community Collage, which runs on large LCD touchscreen computers installed in eight locations throughout a research laboratory. This proactive display application senses nearby people via Bluetooth phones, and responds by incrementally adding photos associated with those people to an ambient collage shown on the screen. This paper describes the motivations, goals, design and impact of the system, highlighting the ways the system has increased interactions and improved personal relationships among coworkers at the deployment site. We also look at how the creation of a shared physical window into online media has affected the use of that media.</p> http://portal.acm.org/citation.cfm?id=1460563.1460580&coll=GUIDE&dl=GUIDE&CFID=83003187&CFTOKEN=88055765 3 Priedhorsky, R. Terveen, L. 2008 The Computational Geowiki: What, Why, and How Computer Supported Cooperative Work <p>Google Maps and its spin-offs are highly successful, but they have a major limitation: users see only pictures of geographic data. These data are inaccessible except by limited vendor-deï¬］ed APIs, and associated user data are weakly linked to them. But some applications require access, specifically geowikis and computational geowikis. We present the design and implementation of a computational geowiki. We also show empirically that both geowiki and computational geowiki features are necessary for a representative domain, bicycling, because (a) cyclists have useful knowledge unavailable except from cyclists and (b) cyclist-oriented automatic route-ï¬］ding is enhanced by user input. Finally, we derive design implications: for example, user contributions presented within a route description are useful, and wikis should support contribution of opinion as well as fact.</p> 3 Pamela Ludford Reid Priedhorsky Ken Reily Loren Terveen 2007 Capturing, Sharing, and Using Local Information ACM SIGCHI Conference on Human Factors in Computing Systems San Jose, CA ACM 1235-1244 28/04/2007 978-1-59593-593-9 <p>With new technology, people can share information about everyday places they go; the resulting data helps others find and evaluate places. Recent applications like Dodgeball and Sharescape repurpose everyday place information: users create local place data for personal use, and the systems display it for public use. We explore both the opportunities -- new local knowledge, and concerns -- privacy risks, raised by this implicit information sharing. We conduct two empirical studies: subjects create place data when using PlaceMail, a location-based reminder system, and elect whether to share it on Sharescape, a community map-building system. We contribute by: (1) showing location-based reminders yield new local knowledge about a variety of places, (2) identifying heuristics people use when deciding what place-related information to share (and their prevalence), (3) detailing how these decision heuristics can inform local knowledge sharing system design, and (4) identifying new uses of shared place information, notably opportunistic errand planning.</p> http://doi.acm.org/10.1145/1240624.1240811 3 N. Good J. Grossklags D. Mulligan J.A. Konstan 2007 Noticing Notice: A large-scale experiment on the timing of software license agreements. ACM SIGCHI Conference on Human Factors in Computing Systems San Jose, CA ACM 607-616 28/04/2007 978-1-59593-593-9 <p>Spyware is an increasing problem. Interestingly, many programs carrying spyware honestly disclose the activities of the software, but users install the software anyway. We report on a study of software installation to assess the effectiveness of different notices for helping people make better decisions on which software to install. Our study of 222 users showed that providing a short summary notice, in addition to the End User License Agreement (EULA), before the installation reduced the number of software installations significantly. We also found that providing the short summary notice after installation led to a significant number of uninstalls. However, even with the short notices, many users installed the program and later expressed regret for doing so. These results, along with a detailed analysis of installation, regret, and survey data about user behaviors informs our recommendations to policymakers and designers for assessing the "adequacy" of consent in the context of software that exhibits behaviors associated with spyware.</p> http://doi.acm.org/10.1145/1240624.1240720 3 Justin D. Weisz Sara Kiesler Hui Zhang Yuqing Ren Robert E. Kraut Joseph A. Konstan 2007 Watching Together: Integrating Text Chat with Video ACM ACM SIGCHI Conference on Human Factors in Computing Systems San Jose, CA ACM 877-886 28/04/2007 978-1-59593-593-9 <p>Watching video online is becoming increasingly popular, and new video streaming technologies have the potential to transform video watching from a passive, isolating experience into an active, socially engaging experience. However, the viability of an active social experience is unclear: both chatting and watching video require attention, and may interfere with one another and detract from the experience. In this paper, we empirically examine the activity of chatting while watching video online. We examine how groups of friends and strangers interact, and find that chat has a positive influence on social relationships, and people chat despite being distracted. We discuss the benefits and opportunities provided by mixing chat and video, uncover some of the attentional and social challenges inherent in this combination of media, and provide guidance for structuring the viewing experience.</p> http://doi.acm.org/10.1145/1240624.1240756 3 D. Cosley D. Frankowski L. Terveen J. Riedl 2007 SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia International Conference on Intelligent User Interfaces Honolulu, Hawaii, USA ACM 32 - 41 Proceedings of the 12th international conference on Intelligent user interfaces 28/01/2007 1-59593-481-2 <p>Member-maintained communities ask their users to perform tasks the community needs. From Slashdot, to IMDb, to Wikipedia, groups with diverse interests create community-maintained artifacts of lasting value (CALV) that support the group's main purpose and provide value to others. Said communities don't help members find work to do, or do so without regard to individual preferences, such as Slashdot assigning meta-moderation randomly. Yet social science theory suggests that reducing the cost and increasing the personal value of contribution would motivate members to participate more.We present SuggestBot, software that performs intelligent task routing (matching people with tasks) in Wikipedia. SuggestBot uses broadly applicable strategies of text analysis, collaborative filtering, and hyperlink following to recommend tasks. SuggestBot's intelligent task routing increases the number of edits by roughly four times compared to suggesting random articles. Our contributions are: 1) demonstrating the value of intelligent task routing in a real deployment; 2) showing how to do intelligent task routing; and 3) sharing our experience of deploying a tool in Wikipedia, which offered both challenges and opportunities for research.</p> http://portal.acm.org/citation.cfm?doid=1216295.1216309 3 F. Harper D. Frankowski S. Drenner Y. Ren S. Kiesler L. Terveen R. Kraut J. Riedl 2007 Talk Amongst Yourselves: Inviting Users to Participate in Online Conversations. International Conference on Intelligent User Interfaces Honolulu, Hawaii ACM 62-71 Proceedings of the 12th international conference on Intelligent user interfaces 28/01/2007 1-59593-481-2 <p>Many small online communities would benefit from increased diversity or activity in their membership. Some communities run the risk of dying out due to lack of participation. Others struggle to achieve the critical mass necessary for diverse and engaging conversation. But what tools are available to these communities to increase participation? Our goal in this research was to spark contributions to the movielens.org discussion forum, where only 2% of the members write posts. We developed personalized invitations, messages designed to entice users to visit or contribute to the forum. In two field experiments, we ask (1) if personalized invitations increase activity in a discussion forum, (2) how the choice of algorithm for intelligently choosing content to emphasize in the invitation affects participation, and (3) how the suggestion made to the user affects their willingness to act. We find that invitations lead to increased participation, as measured by levels of reading and posting. More surprisingly, we find that invitations emphasizing the social nature of the discussion forum increase user activity, while invitations emphasizing other details of the discussion are less successful.</p> http://portal.acm.org/citation.cfm?doid=1216295.1216313 3 F. Harper X. Li Y. Chen J. Konstan 2007 Social Comparisons to Motivate Contributions to an Online Community. Persuasive Technology Palo Alto, CA 26/04/2007 It is increasingly common for online communities to rely on members rather than editors to contribute and moderate content. To motivate members to perform these tasks, some sites display social comparisons, information designed to show members how they compare to others in the system. For example, Amazon, an online book store, shows a list of top reviewers. In this study, we investigate the effect of email newsletters that tell members of an online community that their contributions are above, below, or about average. We find that these comparisons focus membersâ energy on the system features we highlight, but do not increase overall interest in the site. We also find that men and women perceive the comparisons very differently. http://www-users.cs.umn.edu/~harper/publications/harper_persuasive2007.pdf 3 F. Maxwell Harper Shilad Sen Dan Frankowski 2007 Supporting social recommendations with activity-balanced clustering ACM Conference On Recommender Systems Minneapolis, MN, USA ACM 19/10/2007 978-1-59593-730--8 In support of social interaction and information sharing, online communities commonly provide interfaces for users to form or interact with groups. For example, a user of the social music recommendation site last.fm might join the "First Wave Punk" group to discuss his or her favorite band (The Clash) and listen to playlists generated by fellow fans. Clustering techniques provide the potential to automatically discover groups of users who appear to share interests. We explore this idea by describing algorithms for clustering users of an online community and automatically describing the resulting user groups. We designed these techniques for use in an online recommendation system with no pre-existing group functionality, which led us to develop an "activity-balanced clustering" algorithm that considers both user activity and user interests in forming clusters. http://doi.acm.org/10.1145/1297231.1297262 3 Kapoor, N., Chen, J., Butler, J.T., Fouty, G.C., Stemper, J.A., Riedl, J., Konstan, J.A. 2007 Techlens: a researcher's desktop ACM Conference on Recommender Systems Minneapolis, MN Association for Computing Machinery 183-184 10/19/2007 978-1-59593-730--8 <p>Rapid and continuous growth of digital libraries, coupled with brisk advancements in technology, has driven users to seek tools and services that are not only customized to their specific needs, but are also helpful in keeping them stay abreast with the latest developments in their field. TechLens is a recommender system that learns about its users through implicit feedback, builds correlations among them, and uses that information to generate recommendations that match the user's profile. It gives users control over which parts of their profile of known citations are used in forming recommendations for new articles. This demonstration is a prototype that showcases some of the tools and services that TechLens offers to the users of digital libraries.</p> 3 Riedl, J., Jameson, A. 2007 Advanced topics in recommendation 12th International Conference on Intelligent User Interfaces Honolulu, Hawaii, USA Association for Computing Machinery 11 1/28/2007 1-59593-481-2 <p>This full-day tutorial is designed to convey an up-to-date, active understanding of a representative set of current developments in recommender systems that will help the participants to conduct cutting-edge research and/or to work more effectively with the currently widespread recommendation technology.</p> 3 B.R.S. Rosser M.H. Miner W.O. Bockting M.W. Ross J.A. Konstan L. Gurak J. Stanton A. Carballo-Dieguez R. Mazin E. Coleman 2007 HIV Risk: Results of the Men's INTernet Study (MINTS). AIDS & Behavior Springer-Verlag 13 746-756 4 08/2009 <p>This study assessed the feasibility of online recruitment of high-risk Latino men who have sex with men (MSM) for HIV prevention survey research and investigated the relationship between Internet use and unsafe sex. Participants (N = 1,026) were Internet-using Latino MSM living in the U.S. recruited using online banner advertisements. Respondents completed a cross-sectional, online survey in English or Spanish. Sample characteristics reflected national statistics within 5%. Nearly all (99%) reported having used the Internet to seek sex with another man. Two-thirds of respondents reported having unprotected anal sex with â¥1 man in the last year, 57% of these with multiple partners. Participants reported engaging in anal sex and unprotected anal sex with nearly twice as many men first met online versus offline, but risk proportions did not differ. Internet-based HIV prevention research is possible even with geographically-dispersed minority populations. Efficiency appears the primary risk associated with meeting partners online.</p> http://www.springerlink.com/content/ft42733730q32n70/?p=53928559f44640fe953c93a3d316eafc&pi=0 0 Zhou, Changqing Bhatnagar, N. Shekhar, S. Terveen, L. 2007 Mining personally Important Places from GPS Tracks International Conference on Data Engineering 517-526 04/2007 978-1-4244-0832-0 9792677 Global Positioning System, data mining, geophysics computing <p>The discovery of a person's personally important places involves obtaining the physical locations for a person's places that matter to his daily life and routines. This problem is driven by the requirements from emerging location-aware applications, which allow a user to pose queries awl obtain, information in reference, to places, e.g., ''home", ''work" or ''Northwest Health Club". It is a challenge to map from physical locations to jxtrsonally meaningful places because GPS tracks are continuous data both spatially and temporally, while most existing data mining techniques expect discrete data. Previous work has explored algorithms to discover personal places from location data. However, they all have limitations. Our work proposes a two-step approach that discretized continuous GPS data into places and learns important places from the place features. Our approach was validated using real user data and shown to have good accuracy when applied in predicting not only important and frequent places, but also important and not so frequent places.</p> 3 Nishikant Kapoor John T Butler Gary C Fouty James A Stemper Joseph A Konstan 2007 A Study of Citations in Users' Online Personal Collections ACM 2007 European Conference on Digital Libraries <p>Usersâ personal citation collections reflect usersâ interests and thus offer great potential for personalized digital services. We studied 18,120 citations in the personal collections of 96 users of RefWorks citation management system to understand these in terms of their resolvability i.e. how well these citations can be resolved to a unique identifier and to their online sources. While fewer than 4% of citations to articles in Journals and Conferences included a DOI, we were able to increase this resolvability to 50% by using a citation resolver. A much greater percentage of book citations included an ISBN (53%), but using an online resolver found ISBNs for an additional 20% of the book citations. Considering all citation types, we were able to resolve approximately 47% of all citations to either an online source or a unique identifier.</p> http://www-users.cs.umn.edu/~nkapoor/pubs/nkapoor_ecdl07.pdf 3 Reid Priedhorsky Jilin Chen Shyong K. Lam Katherine Panciera Loren Terveen John Riedl 2007 Creating, Destroying, and Restoring Value in Wikipedia Conference on Supporting Group Work <p>Wikipediaâ冱 brilliance and curse is that any user can edit any of the encyclopedia entries. We introduce the notion of the impact of an edit, measured by the number of times the edited version is viewed. Using several datasets, including recent logs of all article views, we show that frequent editors dominate what people see when they visit Wikipedia, and that this domination is increasing. Similarly, using the same impact measure, we show that the probability of a typical article view being damaged is small but increasing, and we present empirically grounded classes of damage. Finally, we make policy recommendations for Wikipedia and other wikis in light of these findings.</p> 3 Reid Priedhorsky Benjamin Jordan Loren Terveen 2007 How a Personalized Geowiki Can Help Bicyclists Share Information More Effectively. WikiSym 2007 Montreal Quebec <p>The bicycling community is focused around a real-world activity â navigating a bicycle â which requires planning within a complex and ever-changing space. While all the knowledge needed to find good routes exists, it is highly distributed. We show, using the results of surveys and interviews, that cyclists need a comprehensive, up-to-date, and personalized information resource.We introduce the personalized geowiki, a new type of wiki which meets these requirements, and we formalize the notion of geowiki. Finally, we state some general prerequisites for wiki contribution and show that they are met by cyclists.</p> 3 Nishikant Kapoor John T Butler Gary C Fouty James A Stemper Joseph A Konstan 2007 Resolvability of References in Users窶 Personal Collections Conceptions of Library and Information Sciences (CoLIS 2007) Introduction. Digital library users collect, enhance and manage their online reference collections to facilitate their research tasks. These personal collections, therefore, are likely to reflect users' interests, and are representative of their profile. Understanding these collections offers great opportunities for developing personalized digital library services, such as reference recommender systems. <br/><br/> Method. We recruited subjects by individual e-mails to the users of RefWorks - a web-based personal reference management tool installed for use at the University of Minnesota. To participate, subjects needed to give their consent and share their references with us. 96 subjects participated, majority (65) of who were graduate students, resulting into 30,336 references. Based on the type of the reference, these were stratified into one of the three valid identifying IDs - DOI, ISBN, or URL. Multiple reference resolvers (CrossRef, WorldCat) were used to enhance the overall resolvability of these collections. <br/><br/> Analysis. Descriptive statistics and simple graphics analysis were used to describe the dataset. <br/><br/> Results. Over 90% of the total references in users' personal collections could possibly have a valid ID (DOI, ISBN, URL), and therefore, are potentially resolvable. However, only about 17% of the references in these collections had a valid ID, and fewer than 11% actually resolved successfully. Using a combination of reference resolvers, the total resolvability of the references in these collections was enhanced from under 11% to over 41%. <br/><br/> Conclusions. Users' personal reference collections have a tremendous potential of building, supporting, and enhancing personalized digital library services, such as reference recommender systems. http://informationr.net/ir/12-4/colis/colis13.html 3 Dan Frankowski Shyong K. Lam Shilad Sen F. Maxwell Harper Scott Yilek Michael Cassano John Riedl 2007 Recommenders Everywhere: The WikiLens Community-Maintained Recommender System Wikisym 2007 Montreal, Quebec, Canada 978-1-59593-861-9 <p>Suppose you have a passion for items of a certain type, and you wish to start a recommender system around those items. You want a system like Amazon or Epinions, but for cookie recipes, local theater, or microbrew beer. How can you set up your recommender system without assembling complicated algorithms, large software infrastructure, a large community of contributors, or even a full catalog of items?</p> <p>WikiLens is open source software that enables anyone, anywhere to start a <em>community-maintained recommender</em> around any type of item. We introduce five principles for community-maintained recommenders that address the two key issues: (1) community contribution of items and associated information; and (2) finding items of interest. Since all recommender communities start small, we look at feasibility and utility in the <em>small world</em>, one with few users, few items, few ratings. We describe the features of WikiLens, which are based on our principles, and give lessons learned from two years of experience running wikilens.org.</p> http://www-users.cs.umn.edu/~dfrankow/files/wiki06f-frankowski.pdf 3 Shilad Sen F. Maxwell Harper Adam LaPitz John Riedl 2007 The Quest for Quality Tags Conference on Supporting Group Work <p>Many online communities use tags â community selected words or phrases â to help people find what they desire. The quality of tags varies widely, from tags that capture a key dimension of an entity to those that are profane, useless, or unintelligible. Tagging systems must often select a subset of available tags to display to users due to limited screen space. Because users often spread tags they have seen, selecting good tags not only improves an individualâ冱 view of tags, it also encourages them to create better tags in the future. We explore implicit (behavioral) and explicit (rating) mechanisms for determining tag quality. Based on 102,056 tag ratings and survey responses collected from 1,039 users over 100 days, we offer simple suggestions to designers of online communities to improve the quality of tags seen by their users.</p> http://www.grouplens.org/system/files/group07-sen.pdf 3 S.M. McNee J. Riedl J.A. Konstan 2006 Making Recommendations Better: An Analytic Model for Human-Recommender Interaction ACM SIGCHI Conference on Human Factors in Computing Systems Montréal, Québec, Canada ACM 1103-1108 28/04/2007 1-59593-298-4 <p>Recommender systems do not always generate good recommendations for users. In order to improve recommender quality, we argue that recommenders need a deeper understanding of users and their information seeking tasks. Human-Recommender Interaction (HRI) provides a framework and a methodology for understanding users, their tasks, and recommender algorithms using a common language. Further, by using an analytic process model, HRI becomes not only descriptive, but also constructive. It can help with the design and structure of a recommender system, and it can act as a bridge between user information seeking tasks and recommender algorithms.</p> http://www.grouplens.org/papers/pdf/mcnee-chi06-hri.pdf 3 Ludford, P. Frankowski, D. Reily, K. Wilms, K. Terveen, L. 2006 Because I Carry My Cell Phone Anyway: Functional Location-Based Reminder Applications ACM SIGCHI Conference on Human Factors in Computing Systems Montréal, Québec, Canada ACM 889-898 22/04/2006 1-59593-372-7 <p>Although they have potential, to date location-based information systems have not radically improved the way we interact with our surroundings. To study related issues, we developed a location-based reminder system, PlaceMail, and demonstrate its utility in supporting everyday tasks through a month-long field study. We identify current tools and practices people use to manage distributed tasks and note problems with current methods, including the common "to-do list". Our field study shows that PlaceMail supports useful location-based reminders and functional place-based lists. The study also sheds rich and surprising light on a new issue: when and where to deliver location-based information. The traditional 'geofence' radius around a place proves insufficient. Instead, effective delivery depends on people's movement patterns through an area and the geographic layout of the space. Our results both provide a compelling demonstration of the utility of location-based information and raise significant new challenges for location-based information distribution.</p> http://www.grouplens.org/papers/pdf/ludford-chi2006.pdf 3 S.M. McNee J. Riedl J. A. Konstan 2006 Being Accurate is Not Enough: How Accuracy Metrics have hurt Recommender Systems ACM SIGCHI Conference on Human Factors in Computing Systems Montréal, Québec, Canada ACM 1097-1101 22/04/2006 1-59593-298-4 <p>Recommender systems have shown great potential to help users find interesting and relevant items from within a large information space. Most research up to this point has focused on improving the accuracy of recommender systems. We believe that not only has this narrow focus been misguided, but has even been detrimental to the field. The recommendations that are most accurate according to the standard metrics are sometimes not the recommendations that are most useful to users. In this paper, we propose informal arguments that the recommender community should move beyond the conventional accuracy metrics and their associated experimental methodologies. We propose new user-centric directions for evaluating recommender systems.</p> http://www.grouplens.org/papers/pdf/mcnee-chi06-acc.pdf 3 Sara Drenner Max Harper Dan Frankowski John Riedl Loren Terveen 2006 Insert Movie Reference Here: A System to Bridge Conversation and Item-Oriented Web Sites ACM SIGCHI Conference on Human Factors in Computing Systems ACM 951-954 22/04/2006 1-59593-372-7 <p>Item-oriented Web sites maintain repositories of information about things such as books, games, or products. Many of these Web sites offer discussion forums. However, these forums are often disconnected from the rich data available in the item repositories. We describe a system, movie linking, that bridges a movie recommendation Web site and a movie-oriented discussion forum. Through automatic detection and an interactive component, the system recognizes references to movies in the forum and adds recommendation data to the forums and conversation threads to movie pages. An eight week observational study shows that the system was able to identify movie references with precision of .93 and recall of .78. Though users reported that the feature was useful, their behavior indicates that the feature was more successful at enriching the interface than at integrating the system.</p> 3 Al Mamunur Rashid Kimberly Ling Regina D Tassone Paul Resnick Robert Kraut John Riedl 2006 Motivating Participation by Displaying the Value of Contribution ACM SIGCHI Conference on Human Factors in Computing Systems Montréal, Québec, Canada ACM 955-958 22/04/2006 1-59593-372-7 <p>One of the important challenges faced by designers of online communities is eliciting sufficent contributions from community members. Users in online communities may have difficulty either in finding opportunities to add value, or in understanding the value of their contributions to the community. Various social science theories suggest that showing users different perspectives on the value they add to the community will lead to differing amounts of contribution. The present study investigates a design augmentation for an existing community Web site that could benefit from additional contribution. The augmented interface includes individualized opportunities for contribution and an estimate of the value of each contribution to the community. The value is computed in one of four different ways: (1) value to self; (2) value to a small group the user has affinity with; (3) value to a small group the user does not have affinity with; and (4) value to the entire user community. The study compares the effectiveness of the different notions of value to 160 community members.</p> http://www.grouplens.org/papers/pdf/rashidAl_chi06.pdf 3 Ludford, P 2006 Sharing everyday places I go while preserving privacy ACM SIGCHI Conference on Human Factors in Computing Systems ACM 1771-1774 22/04/2006 1-59593-298-4 <p>Several new location-based information applications reveal sets of places that an individual frequently visits. This practice gives rise to related privacy questions and new interface needs. For example, while electronic system users want to be in control of private data and know how those who have it will employ it [10], there are no design guidelines for garnering informed consent for using place-based information. In addition, the set of places a person frequents may reveal information such as: 1) when they are likely to go to a place, or 2) within close proximity, where they live. If a user considers this information private, they may still inadvertently disclose it: humans have difficulty comprehending aggregate effects of their actions [1]. A system could therefore deliver benefit by identifying notable risks and informing the user. This research plan will address these key issues and will ultimately inform privacy interface design.</p> http://www.grouplens.org/papers/pdf/ludford-dc-chi2006.pdf 3 Cosley, D. Frankowski, D. Terveen, L. Riedl, J 2006 Using Intelligent Task Routing and Contribution Review to Help Communities Build Artifacts of Lasting Value ACM SIGCHI Conference on Human Factors in Computing Systems Montréal, Québec, Canada ACM 1037-1046 22/04/2006 1-59593-372-7 <p>Many online communities are emerging that, like Wikipedia, bring people together to build community-maintained artifacts of lasting value (CALVs). Motivating people to contribute is a key problem because the quantity and quality of contributions ultimately determine a CALVâ冱 value. We pose two related research questions: 1) How does intelligent task routingâ芭atching people with workâ蚤ffect the quantity of contributions? 2) How does reviewing contributions before accepting them affect the quality of contributions? A field experiment with 197 contributors shows that simple, intelligent task routing algorithms have large effects. We also model the effect of reviewing contributions on the value of CALVs. The model predicts, and experimental data shows, that value grows more slowly with review before acceptance. It also predicts, surprisingly, that a CALV will reach the same final value whether contributions are reviewed before or after they are made available to the community.</p> http://www.grouplens.org/papers/pdf/itr-chi2006.pdf 3 Al Mamunur Rashid Shyong K. Lam George Karypis John Riedl 2006 ClustKNN: A Highly Scalable Hybrid Model- & Memory-Based CF Algorithm Conference on Knowledge Discovery and Data Mining Philadelphia Pennsylvania 20/08/2006 <p>Collaborative Filtering (CF)-based recommender systems are indispensable tools to find items of interest from the unmanageable number of available items. Moreover, companies who deploy a CF-based recommender system may be able to increase revenue by drawing customersâ attention to items that they are likely to buy. However, the sheer number of customers and items typical in e-commerce systems demand specially designed CF algorithms that can gracefully cope with the vast size of the data. Many algorithms proposed thus far, where the principal concern is recommendation quality, may be too expensive to operate in a large-scale system. We propose CLUSTKNN, a simple and intuitive algorithm that is well suited for large data sets. The method first compresses data tremendously by building a straightforward but efficient clustering model. Recommendations are then generated quickly by using a simple Nearest Neighbor-based approach. We demonstrate the feasibility of CLUSTKNN both analytically and empirically. We also show, by comparing with a number of other popular CF algorithms that, apart from being highly scalable and intuitive, CLUSTKNN provides very good recommendation accuracy as well.</p> http://www.grouplens.org/papers/pdf/clustKNN.pdf 0 W. Pequegnat B.R.S. Rosser A. Bowen S.S. Bull R.J. DiClemente W.O. Bockting J. Elford M. Fishbein L. Gurak K. Horvath J. Konstan S. Noar M.W. Ross L. Sherr D. Spiegel R. Zimmerman 2006 Internet-based HIV/STD Prevention Survey Lessons: Practical Challenges and Opportunities. AIDS & Behavior 10/2006 0 B.P. Bailey J.A. Konstan 2006 On the Need for Attention-Aware Systems: Measuring Effects of Interruption on Task Performance, Error Rate and Affective State Journal of Computers in Human Behavior 22 4 685-708 07/2006 <p>This paper reports results from a controlled experiment (N = 50) measuring effects of interruption on task completion time, error rate, annoyance, and anxiety. The experiment used a sample of primary and peripheral tasks representative of those often performed by users. Our experiment differs from prior interruption experiments because it measures effects of interrupting a userâ冱 tasks along both performance and affective dimensions and controls for task workload by manipulating only the time at which peripheral tasks were displayed â between vs. during the execution of primary tasks. Results show that when peripheral tasks interrupt the execution of primary tasks, users require from 3% to 27% more time to complete the tasks, commit twice the number of errors across tasks, experience from 31% to 106% more annoyance, and experience twice the increase in anxiety than when those same peripheral tasks are presented at the boundary between primary tasks. An important implication of our work is that attention-aware systems could mitigate effects of interruption by deferring presentation of peripheral information until coarse boundaries are reached during task execution. As our results show, deferring presentation for a short time, i.e. just a few seconds, can lead to a large mitigation of disruption.</p> 3 Dan Frankowski Dan Cosley Shilad Sen Loren Terveen John Riedl 2006 You Are What You Say: Privacy Risks of Public Mentions Annual ACM Conference on Research and Development in Information Retrieval Seattle Washington ACM 565-572 06/08/2006 1-59593-369-7 <p>In today's data-rich networked world, people express many aspects of their lives online. It is common to segregate different aspects in different places: you might write opinionated rants about movies in your blog under a pseudonym while participating in a forum or web site for scholarly discussion of medical ethics under your real name. However, it may be possible to link these separate identities, because the movies, journal articles, or authors you mention are from a sparse relation space whose properties (e.g., many items related to by only a few users) allow re-identification. This re-identification violates people's intentions to separate aspects of their life and can have negative consequences; it also may allow other privacy violations, such as obtaining a stronger identifier like name and address.This paper examines this general problem in a specific setting: re-identification of users from a public web movie forum in a private movie ratings dataset. We present three major results. First, we develop algorithms that can re-identify a large proportion of public users in a sparse relation space. Second, we evaluate whether private dataset owners can protect user privacy by hiding data; we show that this requires extensive and undesirable changes to the dataset, making it impractical. Third, we evaluate two methods for users in a public forum to protect their own privacy, suppression and misdirection. Suppression doesn't work here either. However, we show that a simple misdirection strategy works well: mention a few popular items that you haven't rated.</p>

[Video]

 3 McNee, S.M. Kapoor, N. Konstan, J.A. 2006 Don't look stupid: avoiding pitfalls when recommending research papers 20th Anniversary Conference on Computer Supported Cooperative Work Banff, Alberta, Canada Association for Computing Machinery 171-180 04/2007 1-59593-249-6 <p>If recommenders are to help people be more productive, they need to support a wide variety of real-world information seeking tasks, such as those found when seeking research papers in a digital library. There are many potential pitfalls, including not knowing what tasks to support, generating recommendations for the wrong task, or even failing to generate any meaningful recommendations whatsoever. We posit that different recommender algorithms are better suited to certain information seeking tasks. In this work, we perform a detailed user study with over 130 users to understand these differences between recommender algorithms through an online survey of paper recommendations from the ACM Digital Library. We found that pitfalls are hard to avoid. Two of our algorithms generated 'atypical' recommendations recommendations that were unrelated to their input baskets. Users reacted accordingly, providing strong negative results for these algorithms. Results from our 'typical' algorithms show some qualitative differences, but since users were exposed to two algorithms, the results may be biased. We present a wide variety of results, teasing out differences between algorithms. Finally, we succinctly summarize our most striking results as "Don't Look Stupid" in front of users.</p> 3 Shilad Sen Shyong K. Lam Dan Cosley Al Mamunur Rashid Dan Frankowski Franklin Harper Jeremy Osterhouse John Riedl 2006 tagging, community, vocabulary, evolution Computer Supported Cooperative Work Banff, Alberta, Canada ACM 181-190 04/11/2006 1-59593-249-6 <p>A tagging community's vocabulary of tags forms the basis for social navigation and shared expression.We present a user-centric model of vocabulary evolution in tagging communities based on community influence and personal tendency. We evaluate our model in an emergent tagging system by introducing tagging features into the MovieLens recommender system.We explore four tag selection algorithms for displaying tags applied by other community members. We analyze the algorithms 'effect on vocabulary evolution, tag utility, tag adoption, and user satisfaction.</p> http://www.grouplens.org/papers/pdf/sen-cscw2006.pdf 0 D. Johnson D. Lilja J. Riedl 2006 Circulating shared-registers for multiprocessor systems Journal of Systems Architecture 52 3 152-168 03/2006 0 N. Good J. Grossklags J.A. Konstan D. Mulligan A. Perzanowski D. Thaw 2006 User Choices and Regret: Understanding Users' Decision Process about Consensually Acquired Spyware I/S: A Journal of Law and Policy for the Information Society 2 2 01/2006 3 S. K. Lam D. Frankowski J. Riedl 2006 Do You Trust Your Recommendations? An Exploration Of Security and Privacy Issues in Recommender Systems International Conference on Emerging Trends in Information and Communication Security (ETRICS) Freiburg, Germany <p>Recommender systems are widely used to help deal with the problem of information overload. However, recommenders raise serious privacy and security issues. The personal information collected by recommenders raises the risk of unwanted exposure of that information. Also, malicious users can bias or sabotage the recommendations that are provided to other users. This paper raises important research questions in three topics relating to exposure and bias in recommender systems: the value and risks of the preference information shared with a recommender, the effectiveness of shilling attacks designed to bias a recommender, and the issues involved in distributed or peer-to-peer recommenders. The goal of the paper is to bring these questions to the attention of the information and communication security community, to invite their expertise in addressing them.</p> http://www.grouplens.org/papers/pdf/lam-etrics2006-security.pdf 3 Riedl, J. 2005 Recommenders for Commerce, Content, and Community 7th IEEE International Conference on E-Commerce Technology Washington, D.C. IEEE Computer Society 5 7/19/2005 0-7695-2277-7 <p>Recommender systems are ubiquitous on the Internet for helping sell productsâ覇verything from automobiles to zebras (stuffed, anyway). Novel applications are emerging that use recommenders for non-Internet applications and that apply them to the problems of distributing content on the Internet and to developing online communities. Community-building is proving one of the most successful ways to create â徭tickinessâ among customers. A vibrant community<br />of practice around a companyâ冱 products creates a powerful barrier to competition and enables consumers to help sell and support your products. <br />Recommender systems are ubiquitous on the Internet for helping sell productsâ覇verything from automobiles to zebras (stuffed, anyway). Novel applications are emerging that use recommenders for non-Internet applications and that apply them to the problems of distributing content on the Internet and to developing online communities. Community-building is proving one of the most successful ways to create â徭tickinessâ among customers. A vibrant community</p> 3 Resnick, P., Hansen, D., Riedl, J., Terveen, L., and Ackerman, M. 2005 Beyond threaded conversation CHI '05 Extended Abstracts on Human Factors in Computing Systems Portland, OR Association for Computing Machinery 2138-2139 4/2/2005 1-59593-002-7 asynchronous communication, online discussion, information systems, system design, computer-mediated communications <p>Online asynchronous communication is an important mechanism for sharing information, building relationships, and collaborating. Most asynchronous communication systems are dominated by a design theme that we'll refer to as "threaded discussion".</p> 0 Riedl, J., Dourish, P. 2005 Introduction to the Special Section on Recommender Systems ACM Transactions on Computer-Human Interaction 12 3 371-373 09/2005 0 Terveen, L. McDonald, D.W. 2005 Social matching: A framework and research agenda ACM Transactions on Computer-Human Interaction 12 3 401-434 09/2005 Human-computer interaction, collaborative filtering, information visualization, recommender systems, social networks <p>Social matching systems bring people together in both physical and online spaces. They have the potential to increase social interaction and foster collaboration. However, social matching systems lack a clear intellectual foundation: the nature of the design space, the key research challenges, and the roster of appropriate methods are all ill-defined. This article begins to remedy the situation. It clarifies the scope of social matching systems by distinguishing them from other recommender systems and related systems and techniques. It identifies a set of issues that characterize the design space of social matching systems and shows how existing systems explore different points within the design space. It also reviews selected social science results that can provide input into system design. Most important, the article presents a research agenda organized around a set of claims. The claims embody our understanding of what issues are most important to investigate, our beliefs about what is most likely to be true, and our suggestions of specific research directions to pursue.</p> 3 Good, N. Dhamija, R. Grossklags, J. Thaw, D. Aronowitz, S. Mulligan, D. Konstan, J. 2005 Stopping spyware at the gate: a user study of privacy, notice and spyware 2005 Symposium on Usable Privacy and Security Pittsburgh, PA Association for Computing Machinery 43-52 07/2005 1-59593-178-3 EULA, ToS, end user license agreement, notice, privacy, security and usability, spyware, terms of service, Design, Experimentation, Human Factors, Legal Aspects, Security <p>Spyware is a significant problem for most computer users. The term "spyware" loosely describes a new class of computer software. This type of software may track user activities online and offline, provide targeted advertising and/or engage in other types of activities that users describe as invasive or undesirable.While the magnitude of the spyware problem is well documented, recent studies have had only limited success in explaining the broad range of user behaviors that contribute to the proliferation of spyware. As opposed to viruses and other malicious code, users themselves often have a choice whether they want to install these programs.In this paper, we discuss an ecological study of users installing five real world applications. In particular, we seek to understand the influence of the form and content of notices (e.g., EULAs) on user's installation decisions.Our study indicates that while notice is important, notice alone may not be enough to affect users' decisions to install an application. We found that users have limited understanding of EULA content and little desire to read lengthy notices. Users found short, concise notices more useful, and noticed them more often, yet they did not have a significant effect on installation for our population. When users were informed of the actual contents of the EULAs to which they agreed, we found that users often regret their installation decisions.We discovered that regardless of the bundled content, users will often install an application if they believe the utility is high enough. However, we discovered that privacy and security become important factors when choosing between two applications with similar functionality. Given two similar programs (e.g. KaZaA and Edonkey), consumers will choose the one they believe to be less invasive and more stable. We also found that providing vague information in EULAs and short notices can create an unwarranted impression of increased security. In these cases, it may be helpful to have a standardized format for assessing the possible options and trade-offs between applications.</p> 3 Zhou, C. Ludford, P. Frankowski, D. Terveen, L. 2005 An experiment in discovering personally meaningful places from location data ACM SIGCHI Conference on Human Factors in Computing Systems Portland, OR Association for Computing Machinery 2029-2032 04/2005 1-59593-002-7 clustering algorithms, field studies, location-aware applications, place discovery, ubiquitous computing <p>As mobile devices become location-aware, they offer the promise of powerful new applications. While computers work with physical locations like latitude and longitude, people think and speak in terms of places, like "my office" or ``Sue's house''. Therefore, location-aware applications must incorporate the notion of places to achieve their full potential. This requires systems to acquire the places that are meaningful for each user. Previous work has explored algorithms to discover personal places from location data. However, we know of no empirical, quantitative evaluations of these algorithms, so the question of how well they work currently is unanswered. We report here on an experiment that begins to provide an answer; we show that a place discovery algorithm can do a good job of discovering places that are meaningful to users. The results have important implications for system design and open up interesting avenues for future research.</p> 3 F.M. Harper X. Li Y. Chen J.A. Konstan 2005 An Economic Model of User Rating in an Online Recommender System 10th International Conference on User Modeling Edinburgh, UK <p>Economic modeling provides a formal mechanism to understand user incentives and behavior in online systems. In this paper we describe the process of building a parameterized economic model of user-contributed ratings in an online movie recommender system. We constructed a theoretical model to formalize our initial understanding of the system, and collected survey and behavioral data to calibrate an empirical model. This model explains 34% of the variation in user rating behavior. We found that while economic modeling in this domain requires an initial understanding of user behavior and access to an uncommonly broad set of user survey and behavioral data, it returns significant formal understanding of the activity being modeled.</p> http://www.grouplens.org/papers/pdf/harper-um05.pdf 31 van Setten, M. McNee, S.M. Konstan, J.A. 2005 Beyond personalization: the next stage of recommender research 10th international Conference on intelligent user interfaces San Diego, CA Association for Computing Machinery 1-58113-894-6 <p>This workshop intends to bring recommender systems researchers and practitioners together in order to discuss the current state of recommender systems research, both on existing and emerging research topics, and to determine how research in this area should proceed. We are at a pivotal point in recommender systems research where researchers are both looking inward at what recommender systems are and looking outward at where recommender systems can be applied, and the implications of applying them out 'in the wild.' This creates a unique opportunity to both reassess the current state of research and directions research is taking in the near and long term.</p> 3 Kapoor, N. Konstan, J. A. Terveen, L. G. 2005 How Peer Photos Influence Member Participation in Online Communities ACM SIGCHI Conference on Human Factors in Computing Systems Portland, OR ACM http://www.grouplens.org/papers/pdf/lbr-590-kapoor.pdf 3 D. Cosley D. Frankowski S. Kiesler L. Terveen J. Riedl 2005 How Oversight Improves Member-Maintained Communities ACM SIGCHI Conference on Human Factors in Computing Systems Portland OR ACM <p>Online communities (OLCs) are gatherings of like -minded people, brought together in cyberspace by shared interests. Creating such communities is not a big challenge; sustaining members' participation is. In this paper, we describe a technique for presenting members' photos and evaluate how it affects member participation in the community. We compare three different policies for presenting peer photos on the home page of the web site. Our results show that explicit requests in the form of simple and short messages on the home page of a community can induce participation. We show that we were able to motivate members to (a) log into the system to see photos of fellow members, and (b) upload their personal photos.</p> 3 C-N Ziegler S.M. McNee J.A. Konstan G. Lausen 2005 Improving Recommendation Lists Through Topic Diversification Fourteenth International World Wide Web Conference (WWW2005) Chiba, Japan http://www.grouplens.org/papers/pdf/ziegler-www05.pdf 3 Rashid, A.M. Karypis, G. Riedl, J. 2005 Influence in Ratings-Based Recommender Systems: An Algorithm-Independent Approach SIAM International Conference on Data Mining Newport Beach, CA 31 Cosley, D. 2005 Mining Social Theory to Build Member Maintained Communities Proceedings of KCVC http://www.grouplens.org/papers/pdf/SS505CosleyD.pdf 0 J. A. Konstan B.R.S. Rosser M.W. Ross J. Stanton W.M. Edwards 2005 The Story of Subject Naught: A Cautionary but Optimistic Tale of Internet Survey Research Journal of Computer-Mediated Communication 10 2 <p>In a web-based, sexual behavior risk study using a rigorous response validation protocol, we identified 124 invalid responses out of 1,150 total (11% rejection). Nearly all of these (119) were due to repeat survey submissions from the same participants, and 65 of them came from a single participant. This brief describes how we were able to detect these repeat submissions using the validation protocol, and highlights the importance of using both automated and manual validation techniques</p> http://jcmc.indiana.edu/vol10/issue2/konstan.html 31 J.A. Konstan N. Kapoor S.M. McNee J.T. Butler 2005 TechLens: Exploring the Use of Recommenders to Support Users of Digital Libraries Coalition for Networked Information Fall 2005 Task Force Meeting Phoenix, AZ <p>The immense collection of valuable information in digital libraries is changing the way students and scholars access information. Indeed, just as library users are accessing libraries and research librarians remotely, the potential is increasing for value-added services that promise to help patrons use digital libraries in ever more powerful ways, moving beyond basic search to a new collection of awareness, field summary, and people-finding services. For the past four years we've been exploring means by which recommender systems technologyâ杯he technology used today by e-commerce vendors to help customers find productsâ把an be adapted to serve the needs of students and scholars exploring scientific literature. The model shown here illustrates the types of data that can be used to fulfill a user's need. We have already demonstrated the success of some of the basic approachesâ盃sing citations, keywords, and abstracts to find works a researcher is unfamiliar with. Much work remains, however, as we explore ways to meet a more diverse set of information needs.</p> http://www.grouplens.org/papers/pdf/CNI-TechLens-Final.pdf 0 Ling, K. Beenen, G. Ludford, P. Wang, X. Chang, K. Li, X. Cosley, D. Frankowski, D. Terveen, L. Rashid, A. M. Resnick, P. Kraut, R. 2005 Using Social Psychology to Motivate Contributions to Online Communities Journal of Computer-Mediated Communication 10 4 http://jcmc.indiana.edu/vol10/issue4/ling.html 3 Ludford, P.J. Cosley, D. Frankowski, D. Terveen, L 2004 Think Different: Increasing Online Community Participation Using Uniqueness and Group Dissimilarity ACM SIGCHI Conference on Human Factors in Computing Systems Vienna Austria ACM 631-368 24/04/2004 1-58113-702-8 <p>Online communities can help people form productive relationships. Unfortunately, this potential is not always fulfilled: many communities fail, and designers don't have a solid understanding of why. We know community activity begets activity. The trick, however, is to inspire participation in the first place. Social theories suggest methods to spark positive community participation. We carried out a field experiment that tested two such theories. We formed discussion communities around an existing movie recommendation web site, manipulating two factors: (1) similarity-we controlled how similar group members' movie ratings were; and (2) uniqueness-we told members how their movie ratings (with respect to a discussion topic) were unique within the group. Both factors positively influenced participation. The results offer a practical success story in applying social science theory to the design of online communities.</p> http://www.grouplens.org/papers/pdf/thinkdifferent-chi2004.pdf 3 Lam, S.K. Riedl, J 2004 Shilling recommender systems for fun and profit 13th international conference on World Wide Web (WWW2004) New York NY ACM 393-402 17/05/2004 1-58113-844-X <p>Recommender systems have emerged in the past several years as an effective way to help people cope with the problem of information overload. One application in which they have become particularly common is in e-commerce, where recommendation of items can often help a customer find what she is interested in and, therefore can help drive sales. Unscrupulous producers in the never-ending quest for market penetration may find it profitable to shill recommender systems by lying to the systems in order to have their products recommended more often than those of their competitors. This paper explores four open questions that may affect the effectiveness of such shilling attacks: which recommender algorithm is being used, whether the application is producing recommendations or predictions, how detectable the attacks are by the operator of the system, and what the properties are of the items being attacked. The questions are explored experimentally on a large data set of movie ratings. Taken together, the results of the paper suggest that new ways must be used to evaluate and detect shilling attacks on recommender systems.</p> http://www.grouplens.org/papers/pdf/p333-lam.pdf 0 M. Ross B.R.S. Rosser J. Stanton J. Konstan 2004 Characteristics of Latino men who have sex with men on the Internet who complete and drop out of an Internet-based sexual behavior survey. AIDS Education and Prevention 16 6 526-537 12/2004 <p>To identify biases and threats to validity of Internet survey data collection on HIV-related risk behaviors, we studied 1,546 Latino men who have sex with men on the Internet recruited through banner impressions on a leading national gay Internet site. The study could be completed in English or Spanish. Of those commencing, 33.6% dropped out before completing the 450-field questionnaire. None of the linguistic variables (level of use of Spanish or English) predicted dropout. However, dropouts were more likely to identify as Puerto Rican or Black, to reject the $20 compensation or offer it to a charity, to not have met men for sex on the Internet, to identify as bisexual or heterosexual, and to use Web sites or personal ads for contact and to use the Internet less at home than those who completed the study. Men in seroconcordant monogamous relationships and those who had not met a man for sex on the Internet were also more likely to drop out. These data suggest that there are no linguistic and few demographic and Internet use variables that are associated with dropout. Issues of compensation and respondent characteristics that make it likely that there will be a large number of inapplicable data fields in the questionnaire appear to be significant predictors of dropout. Although there were many data missing, the dropouts did not appear to be at greater HIV-associated risk than the completers. The fact that there appear to be few systematic demographic or Internet use biases in dropouts suggests that the completers do not represent a seriously skewed sample of those Latinos who commence the Internet survey.</p> 0 Whittaker, S. Jones, Q. Nardi, B. Creech, M. Terveen, L. Isaacs, E. Hainsworth, J. 2004 ContactMap: Organizing communication in a social desktop ACM Transactions on Human-Computer Interaction 11 4 445-471 12/2004 Email, human-computer interaction, instant messaging, interpersonal communication, iterative user-centered design, personal information management, personal social desktop, social data mining, social reminding, visualization <p>Modern work is a highly social process, offering many cues for people to organize communication and access information. Shared physical workplaces provide natural support for tasks such as (a) &lt;i&gt;social reminding&lt;/i&gt; about communication commitments and keeping track of collaborators and friends, and (b) &lt;i&gt;social data mining&lt;/i&gt; of local expertise for advice and information. However, many people now collaborate remotely using tools such as email and voicemail. Our field studies show that these tools do not provide the social cues needed for group work processes. In part, this is because the tools are organized around &lt;i&gt;messages&lt;/i&gt;, rather than &lt;i&gt;people&lt;/i&gt;. In response to this problem, we created ContactMap, a system that makes &lt;i&gt;people&lt;/i&gt; the primary unit of interaction. ContactMap provides a structured social desktop representation of users' important contacts that directly supports social reminding and social data mining. We conducted an empirical evaluation of ContactMap, comparing it with traditional email systems, on tasks suggested by our fieldwork. Users performed better with ContactMap and preferred ContactMap for the majority of these tasks. We discuss future enhancements of our system and the implications of these results for future communication interfaces and for theories of mediated communication.</p> 0 Schafer, J.B. Konstan, J.A. Riedl, J. 2004 View through MetaLens: usage patterns for a meta-recommendation system IEEE Proceedings Software 151 6 267-279 12/2004 8236046 Internet, information filtering, information filters, meta data <p>In a world where a person's number of choices can be overwhelming, recommender systems help users find and evaluate items of interest. They do so by connecting users with information regarding the content of recommended items or the opinions of other individuals. Such systems have become powerful tools in domains such as electronic commerce, digital libraries and knowledge management. The authors address such systems, as well as a relatively new class of recommender system called meta-recommenders. Meta-recommenders provide users with personalised control over the generation of a single recommendation list formed from a combination of rich data using multiple information sources and recommendation techniques. They discuss observations made from the public trial of a meta-recommender system in the domain of movies and lessons learned from the incorporation of features that allow persistent personalisation of the system. Finally, they consider the challenges of building real-world, usable meta-recommenders across a variety of domains.</p> 3 C. Zhou D. Frankowski P. Ludford S. Shekhar L. Terveen 2004 Discovering Personal Gazetteers: An Interactive Clustering Approach. ACM international workshop on Geographic information systems Washington D.C. ACM 266-273 12/11/2004 1-58113-979-9 <i>Personal gazetteers</i> record individuals' most important <i>places</i>, such as home, work, grocery store, etc. Using personal gazetteers in location-aware applications offers additional functionality and improves the user experience. However, systems then need some way to acquire them. This paper explores the use of novel semi-automatic techniques to discover gazetteers from users' travel patterns (time-stamped location data). There has been previous work on this problem, e.g., using ad hoc algorithms [13]or K-Means clustering[4]; however, both approaches have shortcomings. This paper explores a deterministic, density-based clustering algorithm that also uses temporal techniques to reduce the number of uninteresting places that are discovered. We introduce a general framework for evaluating personal gazetteer discovery algorithms and use it to demonstrate the advantages of our algorithm over previous approaches. http://www.grouplens.org/papers/pdf/zhou-acmgis04.pdf 3 Jones, Q. Grandhi, S.A. Whittaker, S. Chivakula, K. Terveen, L. 2004 Putting systems into place: a qualitative study of design requirements for location-aware community systems 2004 Computer Supported Cooperative Work Chicago, IL Association for Computing Machinery 202-211 11/2004 1-58113-810-5 Design, Experimentation, Human Factors <p>We present a conceptual framework for location-aware community systems and results from two studies of how socially-defined places influence people's information sharing and communication needs. The first study identified a relationship between people's familiarity with a place and their desire for either stable or dynamic place-related information. The second study explored the utility of various system features highlighted by our conceptual framework. It clarified the role of place information in informal social interaction; it also showed that people valued, and were willing to provide information such as ratings, comments, and event records relevant to a place. These preliminary findings have important implications for the design of location-aware community systems. In particular, they suggest that such systems must integrate information about places with data about users' personal routines and social relationships.</p> 0 Jones, Q. Grandhi, S.A. Terveen, L. Whittaker, S. 2004 People-to-People-to-Geographical-Places: The P3 Framework for Location-Based Community Systems Computer Supported Cooperative Work 13 3-4 249-282 08/2004 Design, Experimentation, Human Factors <p>In this paper we examine an emerging class of systems that link People-to-People-to-Geographical-Places; we call these P3-Systems. Through analyzing the literature, we have identified four major P3-System design techniques: People-Centered systems that use either absolute user location (e.g. Active Badge) or user proximity (e.g. Hocman) and Place-Centered systems based on either a representation of people's use of physical spaces (e.g. ActiveMap) or on a matching virtual space that enables online interaction linked to physical location (e.g. Geonotes). In addition, each feature can be instantiated synchronously or asynchronously. The P3-System framework organizes existing systems into meaningful categories and structures the design space for an interesting new class of potentially context-aware systems. Our discussion of the framework suggests new ways of understanding and addressing the privacy concerns associated with location aware community system and outlines additional socio-technical challenges and opportunities.</p> 0 B. Miller J. Konstan L. Terveen J. Riedl 2004 PocketLens: Towards a Personal Recommender System ACM Transactions on Information Systems 22 3 437-476 07/2004 Recommender systems using collaborative filtering are a popular technique for reducing information overload and finding products to purchase. One limitation of current recommenders is that they are not portable. They can only run on large computers connected to the Internet. A second limitation is that they require the user to trust the owner of the recommender with personal preference data. Personal recommenders hold the promise of delivering high quality recommendations on palmtop computers, even when disconnected from the Internet. Further, they can protect the user's privacy by storing personal information locally, or by sharing it in encrypted form. In this article we present the new PocketLens collaborative filtering algorithm along with five peer-to-peer architectures for finding neighbors. We evaluate the architectures and algorithms in a series of offline experiments. These experiments show that Pocketlens can run on connected servers, on usually connected workstations, or on occasionally connected portable devices, and produce recommendations that are as good as the best published algorithms to date. http://portal.acm.org/citation.cfm?doid=1010614.1010618 3 R. Torres S.M. McNee M. Abel J.A. Konstan J. Riedl 2004 Enhancing Digital Libraries with TechLens+ The Fourth ACM/IEEE Joint Conference on Digital Libraries (JCDL 2004) Tuscon AZ ACM 228-236 07/06/2004 1-58113-832-6 <p>The number of research papers available is growing at a staggering rate. Researchers need tools to help them find the papers they should read among all the papers published each year. In this paper, we present and experiment with hybrid recommender algorithms that combine Collaborative Filtering and Content-based. Filtering to recommend research papers to users. Our hybrid algorithms combine the strengths of each filtering approach to address their individual weaknesses. We evaluated our algorithms through offline experiments on a database of 102, 000 research papers, and through an online experiment with 110 users. For both experiments we used a dataset created from the CiteSeer repository of computer science research papers. We developed separate English and Portuguese versions of the interface and specifically recruited American and Brazilian users to test for cross-cultural effects. Our results show that users value paper recommendations, that the hybrid algorithms can be successfully combined, that different algorithms are more suitable for recommending different kinds of papers, and that users with different levels of experience perceive recommendations differently These results can be applied to develop recommender systems for other types of digital libraries.</p> http://www-users.cs.umn.edu/%7Emcnee/torres-jcdl2004.pdf 0 Riedl, J. and Wellman, M.P. 2004 Introduction to the Special Issue: Matching Buyers and Sellers for e-Commerce International Journal of Electronic Commerce 8 4 7-8 06/2004 0 J. Herlocker J. Konstan L. Terveen J. Riedl 2004 Evaluating Collaborative Filtering Recommender Systems. ACM Transactions on Information Systems (TOIS) 22 1 5-53 01/2004 Recommender systems have been evaluated in many, often incomparable, ways. In this article, we review the key decisions in evaluating collaborative filtering recommender systems: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole. In addition to reviewing the evaluation strategies used by prior researchers, we present empirical results from the analysis of various accuracy metrics on one content domain where all the tested metrics collapsed roughly into three equivalence classes. Metrics within each equivalency class were strongly correlated, while metrics from different equivalency classes were uncorrelated. http://doi.acm.org/10.1145/963770.963772 31 Riedl, J. and Hill, R. 2004 Guest editors' introduction in AI Magazine, volume 25, number 3 3 Preece, J. Lazar, J. Churchill, E. de Graaff, H. Friedman, B. Konstan, J. 2003 Spam, spam, spam, spam: how can we stop it ACM SIGCHI Conference on Human Factors in Computing Systems Ft. Lauderdale, FL 706-707 04/2003 1-58113-637-4 <p>How do we keep our channels of electronic communication, both individual and group, open, while keeping out inappropriate and unrelated materials, such as spam? Does someone other than the intended recipient have the right to control what electronic mail users see? Might this lead to censorship? If others DO have the right to control what e-mail users see, how should this filtering or censorship occur? Are users aware of this filtering? If others are NOT controlling what users receive, what can users themselves do to control their environments to limit the amount of incoming spam? These are some of the topics that this CHI panel will address.</p> 0 Amento, B. Terveen, L. Hill, W. Hix, D. Schulman, R. 2003 Experiments in Social Data Mining: The TopicShop System CM Transactions on Computer-Human Interaction 10 1 54-85 03/2003 Social data mining systems enable people to share opinions and benefit from each other's experience. They do this by mining and redistributing information from computational records of social activity such as Usenet messages, system usage history, citations, or hyperlinks. Some general questions for evaluating such systems are: (1) is the extracted information valuable? and (2) do interfaces based on the information improve user task performance? We report here on TopicShop, a system that mines information from the structure and content of Web pages and provides an exploratory information workspace interface. We carried out experiments that yielded positive answers to both evaluation questions. First, a number of automatically computable features about Web sites do a good job of predicting expert quality judgments about sites. Second, compared to popular Web search interfaces, the TopicShop interface to this information lets users select significantly more high-quality sites, in less time and with less effort, and to organize the sites they select into personally meaningful collections more quickly and easily. We conclude by discussing how our results may be applied and considering how they touch on general issues concerning quality, expertise, and consensus. http://www.grouplens.org/papers/pdf/tochi-amento-2003.pdf 3 Lam, S.K. Pennock, D.M. Cosley, D. Lawrence, S 2003 1 Billion Pages = 1 Million Dollars? Mining the Web to Play "Who Wants to be a Millionaire?" Uncertainty in Artificial Intelligence (UAI2003) Acapulco, Mexico 337-345 We exploit the redundancy and volume of information on the web to build a computerized player for the ABC TV game show â弩ho Wants To Be A Millionaire?â. The player consists of a questionanswering module and a decision-making module. The question-answering module utilizes question transformation techniques, natural language parsing, multiple information retrieval algorithms, and multiple search engines; results are combined in the spirit of ensemble learning using an adaptive weighting scheme. Empirically, the system correctly answers about 75% of questions from the Millionaire CD-ROM, 3rd editionâ波eneral-interest trivia questions often about popular culture and common knowledge. The decision-making module chooses from allowable actions in the game in order to maximize expected risk-adjusted winnings, where the estimated probability of answering correctly is a function of past performance and confidence in correctly answering the current question. When given a six question head start (i.e., when starting from the $2,000 level), we find that the system performs about as well on average as humans starting at the beginning. Our system demonstrates the potential of simple but well-chosen techniques for mining answers from unstructured information such as the web. http://www.grouplens.org/papers/pdf/1m-uai2003.pdf 3 B. Bailey J. Konstan 2003 Are Informal Tools Better? Comparing DEMAIS, Pencil and Paper, and Authorware for Early Multimedia Design ACM SIGCHI Conference on Human Factors in Computing Systems Ft. Lauderdale FL 313-320 1-58113-630-7 <p>DEMAIS is an informal design tool that we claim helps a multimedia designer explore and communicate temporal and interactive (behavioral) design ideas better than existing tools. This paper seeks to empirically validate our claim. We report on an evaluation comparing DEMAIS to pencil and paper and Authorware for the exploration and communication of behavior in early multimedia design. The main results are that (i) DEMAIS was better than Authorware for both exploring and communicating behavior, (ii) DEMAIS was better than pencil and paper for communicating behavior, and (iii) DEMAIS was able to capture most of a designer's behavioral design ideas. Our results show that DEMAIS bridges the early investment/communication gap that exists among current multimedia design tools.</p> http://doi.acm.org/10.1145/642611.642666 7 Konstan, J.A., Riedl, J. 2003 Collaborative filtering: supporting social navigation in large, crowded infospaces. In: Designing information Spaces: the Social Navigation Approach Hook, K., Benyon, D., Munro, A.J., Diaper, D. and Sanger, C. Designing information Spaces: the Social Navigation Approach London Springer-Verlag 43-82 3 S.M. McNee S.K. Lam C. Guetzlaff J.A. Konstan J. Riedl 2003 Confidence Displays and Training in Recommender Systems INTERACT '03 IFIP TC13 International Conference on Human-Computer Interaction 176-183 Recommender systems help users sort through vast quantities of information. Sometimes, however, users do not know if they can trust the recommendations they receive. Adding a confidence metric has the potential to improve user satisfaction and alter user behavior in a recommender system. We performed an experiment to measure the effects of a confidence display as a component of an existing collaborative filtering-based recommender system. Minimal training improved use of the confidence display compared to no training. Novice users were less likely to notice, understand, and use the confidence display than experienced users of the system. Providing training about a confidence display to experienced users greatly reduced user satisfaction in the recommender system. These results raise interesting issues and demonstrate subtle effects about how and when to train users when adding features to a system. http://www.grouplens.org/papers/pdf/mcnee-interact2003.pdf 3 S.M. McNee S.K. Lam J.A. Konstan J. Riedl 2003 Interfaces for Eliciting New User Preferences in Recommender Systems The 9th International Conference on User Modeling (UM'2003) 178-188 Recommender systems build user models to help users find the items they will find most interesting from among many available items. One way to build such a model is to ask the user to rate a selection of items. The choice of items selected affects the quality of the user model generated. In this paper, we explore the effects of letting the user participate in choosing the items that are used to develop the model. We compared three interfaces to elicit information from new users: having the system choose items for users to rate, asking the users to choose items themselves, and a mixed-initiative interface that combines the other two methods. We found that the two pure interfaces both produced accurate user models, but that directly asking users for items to rate increases user loyalty in the system. Ironically, this increased loyalty comes despite a lengthier signup process. The mixed-initiative interface is not a reasonable compromise as it created less accurate user models with no increase in loyalty. http://www.grouplens.org/papers/pdf/mcnee-um2003.pdf 3 Cosley, D. Lam, S.K. Albert, I. Konstan, J. Riedl, J. 2003 Is Seeing Believing? How Recommender Systems Influence Users' Opinions ACM SIGCHI Conference on Human Factors in Computing Systems 585-592 <p>Recommender systems use peopleâ冱 opinions about items in an information domain to help people choose other items. These systems have succeeded in domains as diverse as movies, news articles, Web pages, and wines. The psychological literature on conformity suggests that in the course of helping people make choices, these systems probably affect usersâ opinions of the items. If opinions are influenced by recommendations, they might be less valuable for making recommendations for other users. Further, manipulators who seek to make the system generate artificially high or low recommendations might benefit if their efforts influence users to change the opinions they contribute to the recommender. We study two aspects of recommender system interfaces that may affect usersâ opinions: the rating scale and the display of predictions at the time users rate items. We find that users rate fairly consistently across rating scales. Users can be manipulated, though, tending to rate toward the prediction the system shows, whether the prediction is accurate or not. However, users can detect systems that manipulate predictions. We discuss how designers of recommender systems might react to these findings.</p> http://www.grouplens.org/papers/pdf/conform-chi03.pdf 3 B. Miller I. Albert S.K. Lam J. Konstan J. Riedl 2003 MovieLens Unplugged: Experiences with a Recommender System on Four Mobile Devices ACM SIGCHI Conference on Human Factors in Computing Systems <p>Recommender systems have changed the way people shop online. Recommender systems on wireless mobile devices may have the same impact on the way people shop in stores. We present our experience with implementing a recommender system on a PDA that is occasionally connected to the network. This interface helps users of the MovieLens movie recommendation service select movies to rent, buy, or see while away from their computer. The results of a nine month field study show that although there are several challenges to overcome, mobile recommender systems have the potential to provide value to their users today.</p> 3 Cosley, D. Ludford, P. Terveen, L. 2003 Studying the Effect of Similarity in Online Task-Focused Interactions Group 2003 Sanibel Island, FL 321-329 <p>Although the Internet provides powerful tools for social interactions, many tasksâ杷or example, information-seekingâ蚤re undertaken as solitary activities. Information seekers are unaware of the invisible crowd traveling in parallel to their course through the information landscape. Social navigation systems attempt to make the invisible crowd visible, while social recommender systems try to introduce people directly. However, it is not clear whether users desire or will respond to social cues indicating the presence of other people when they are focused on a task. To investigate this issue, we created an online game-playing task and paired subjects to perform the task based on their responses to a short survey about demographics and interests. We studied how these factors influence task outcomes, the interaction process, and attitudes towards oneâ冱 partner. We found that demographic similarity affected how people interact with each other, even though this information was not explicit, while similarities or differences in task-relevant interests did not. Our findings suggest guidelines for developing social recommender systems and show the need for further research into conditions that will help such systems succeed.</p> http://www.grouplens.org/papers/pdf/simex-group2003.pdf 0 Herlocker, J. Konstan, J. Riedl, J. 2002 Empirical Analysis of Design Choices in Neighborhood-based Collaborative Filtering Algorithms Informational Retrieval 5 287-310 2002 <p>Collaborative filtering systems predict a user's interest in new items based on the recommendations of other people with similar interests. Instead of performing content indexing or content analysis, collaborative filtering systems rely entirely on interest ratings from members of a participating community. Since predictions are based on human ratings, collaborative filtering systems have the potential to provide filtering based on complex attributes, such as quality, taste, or aesthetics. Many implementations of collaborative filtering apply some variation of the neighborhood-based prediction algorithm. Many variations of similarity metrics, weighting approaches, combination measures, and rating normalization have appeared in each implementation. For these parameters and others, there is no consensus as to which choice of technique is most appropriate for what situations, nor how significant an effect on accuracy each parameter has. Consequently, every person implementing a collaborative filtering system must make hard design choices with little guidance. This article provides a set of recommendations to guide design of neighborhood-based prediction systems, based on the results of an empirical study. We apply an analysis framework that divides the neighborhood-based prediction approach into three components and then examines variants of the key parameters in each component. The three components identified are similarity computation, neighbor selection, and rating combination.</p> 3 McNee, S. Albert, I. Cosley, D. Gopalkrishnan, P. Lam, S.K. Rashid, A.M. Konstan, J.A. Riedl, J. 2002 On the Recommending of Citations for Research Papers ACM 2002 Conference on Computer Supported Cooperative Work (CSCW2002) New Orleans, LA ACM 116-125 16/11/2002 1-58113-560-2 <p>Collaborative filtering has proven to be valuable for recommending items in many different domains. In this paper, we explore the use of collaborative filtering to recommend research papers, using the citation web between papers to create the ratings matrix. Specifically, we tested the ability of collaborative filtering to recommend citations that would be suitable additional references for a target research paper. We investigated six algorithms for selecting citations, evaluating them through offline experiments against a database of over 186,000 research papers contained in ResearchIndex. We also performed an online experiment with over 120 users to gauge user opinion of the effectiveness of the algorithms and of the utility of such recommendations for common research tasks. We found large differences in the accuracy of the algorithms in the offline experiment, especially when balanced for coverage. In the online experiment, users felt they received quality recommendations, and were enthusiastic about the idea of receiving recommendations in this domain.</p> http://www.grouplens.org/papers/pdf/techlens-cscw2002.pdf 3 Rashid, A.M. Albert, I. Cosley, D. Lam, S.K. McNee, S. Konstan, J.A. Riedl, J. 2002 Getting to Know You: Learning New User Preferences in Recommender Systems 2002 International Conference on Intelligent User Interfaces (IUI2002) San Francisco CA ACM 127-134 13/01/2002 1-58113-459-2 <p>Recommender systems have become valuable resources for users seeking intelligent ways to search through the enormous volume of information available to them. One crucial unsolved problem for recommender systems is how best to learn about a new user. In this paper we study six techniques that collaborative filtering recommender systems can use to learn about new users. These techniques select a sequence of items for the collaborative filtering system to present to each new user for rating. The techniques include the use of information theory to select the items that will give the most value to the recommender system, aggregate statistics to select the items the user is most likely to have an opinion about, balanced techniques that seek to maximize the expected number of bits learned per presented item, and personalized techniques that predict which items a user will have an opinion about. We study the techniques thru offline experiments with a large pre-existing user data set, and thru a live experiment with over 300 users. We show that the choice of learning technique significantly affects the user experience, in both the user effort and the accuracy of the resulting predictions.</p> http://www.grouplens.org/papers/pdf/voi-final.pdf 3 Whittaker, S. Jones, Q. Terveen, L. 2002 Contact management: identifying contacts to support long-term communication Computer Supported Cooperative Work (CSCW) New Orleans, LA Association for Computing Machinery 216-225 11/2002 1-58113-560-2 PDAs, address books, asynchronous communication, communication history, contact management <p>Much of our daily communication activity involves managing interpersonal communications and relationships. Despite its importance, this activity of contact management is poorly understood. We report on field and lab studies that begin to illuminate it.A field study of business professionals confirmed the importance of contact management and revealed a major difficulty: selecting important contacts from the large set of people with whom one communicates. These interviews also showed that communication history is a key resource for this task. Informants identified several history-based criteria that they considered useful.We conducted a lab study to test how well these criteria predict contact importance. Subjects identified important contacts from their email archives. We then analyzed their email to extract features for all contacts. Reciprocity, recency and longevity of email interaction proved to be strong predictors of contact importance. The experiment also identified another contact management problem: removing 'stale' contacts from long term archives. We discuss the design and theoretical implications of these results.</p> 3 Schafer, J.B. Konstan, J.A. Riedl, J. 2002 Meta-recommendation Systems: User-controlled Integration of Diverse Recommendations The 11th International Conference on Information and Knowledge Management (CIKM 2002) McLean, VA ACM 43-51 11/04/2002 1-58113-492-4 In a world where the number of choices can be overwhelming, recommender systems help users find and evaluate items of interest. They do so by connecting users with information regarding the content of recommended items or the opinions of other individuals. Such systems have become powerful tools in domains such as electronic commerce, digital libraries, and knowledge management. In this paper, we address such systems and introduce a new class of recommender system called meta-recommenders. Meta-recommenders provide users with personalized control over the generation of a single recommendation list formed from a combination of rich data using multiple information sources and recommendation techniques. We discuss experiments conducted to aid in the design of interfaces for a meta-recommender in the domain of movies. We demonstrate that meta-recommendations fill a gap in the current design of recommender systems. Finally, we consider the challenges of building real-world, usable meta-recommenders across a variety of domains. http://www.grouplens.org/papers/pdf/schafer-cikm.pdf 3 Amento, B. Hill, W. Terveen, L. 2002 The sound of one hand: a wrist-bounded bio-acoustic fingertip gesture interface Computer-Supported Cooperative Work Minneapolis, MN Association for Computing Machinery 724-725 04/2002 1-58113-454-1 acoustics, fingers, gestural interfaces, gestures, human-computer interaction, mobile devices, wrist <p>Two hundred and fifty years ago the Japanese Zen master Hakuin asked the question, "What is the Sound of the Single Hand?" This koan has long served as an aid to meditation but it also describes our new interaction techinique. We discovered that gentle fingertip gestures such as tapping, rubbing, and flicking make quiet sounds that travel by bone conduction throughout the hand. A small wristband-mounted contact microphone can reliably and inexpensively sense these sounds. We harnessed this "sound in the hand" phenomenon to build a wristband-mounted bio-acoustic fingertip gesture interface. The bio-acoustic interface recognizes some common gestures that state-of-the-art glove and image-processing techniques capture but in a smaller, mobile package.</p> 3 Terveen, L.G. McMackin, J. Amento, B. Hill, W. 2002 Specifying Preferences Based On User History ACM SIGCHI Conference on Human Factors in Computing Systems Minneapolis, MN ACM 315-322 04/20/2002 1-58113-453-3 <p>Many applications require users to specify preferences. We support users in this task by letting them define preferences relative to their personal history or that of other users. We implement this idea using a graphical technique called control shadows, which we have implemented on both a desktop computer and on a cell phone with a small, grayscale display. An empirical study compared user performance on the graphical interface and a text table interface with identical functionality. On the desktop, users completed their tasks more quickly and effectively and strongly preferred the graphical interface. On the cell phone, there was no significant difference between the graphical and table interfaces. Finally, personal history proved useful in specifying preferences, but history of other users was not helpful</p> http://www.grouplens.org/papers/pdf/userhistory-chi2002.pdf 7 Miller, B. Riedl, J. Konstan, J. 2002 GroupLens for Usenet: Experiences in Collaborative Filtering to a Social Information System Leug, C. Fisher, D. From Usenet to CoWebs: Interacting with Social Information Spaces Springer-Verlag 3 Sarwar, B.M. Karypis, G. Konstan, J. Riedl, J. 2002 Incremental SVD-Based Algorithms for Highly Scaleable Recommender Systems Fifth International Conference on Computer and Information Technology ICCIT <p>We investigate the use of dimensionality reduction to improve the performance for a new class of data analysis software called â徨ecommender systemsâ. Recommender systems apply knowledge discovery techniques to the problem of making personalized product recommendations during a live customer interaction. The tremendous growth of customers and products in recent years poses some key challenges for recommender systems. These are:pr oducing high quality recommendations and performing many recommendations per second for millions of customers and products. Singular Value Decomposition(SVD)-based recommendation algorithms can quickly produce high quality recommendations, but has to undergo very expensive matrix factorization steps. In this paper, we propose and experimentally validate a technique that has the potential to incrementally build SVD-based models and promises to make the recommender systems highly scalable.</p> http://www.grouplens.org/papers/pdf/sarwar_SVD.pdf 3 Whittaker, S. Jones, Q. Terveen, L. 2002 Managing long term communications: conversation and contact management Hawaii International Conference on System Sciences Hawaii IEEE 1070-1079 0-7695-1435-9 7198084 human factors, personal information systems, time management, user interface management systems, contact management, conversations, long term communications, memory load, support tool <p><br />We investigate the use of dimensionality reduction to improve the performance for a new class of data analysis software called â徨ecommender systemsâ. Recommender systems apply knowledge discovery techniques to the problem of making personalized product recommendations during a live customer interaction. The tremendous growth of customers and products in recent years poses some key challenges for recommender systems. These are:pr oducing high quality recommendations and performing many recommendations per second for millions of customers and products. Singular Contact management is an important part of everyday work. People exchange business cards to try to enter each other's contact lists. Local businesses provide refrigerator magnets and calendars so they will be called on when a particular need arises. People who use the telephone extensively are selective about who they add to their speed dial lists. Contact management and conversation management are linked. Many busy professionals discourage voice calls and messages, because E-mail enables them to better manage their time, conversations, and contacts. People also spend large amounts of time transcribing voice mail, browsing E-mail archives and writing todo lists - all of these activities are intended to help track the content and status of outstanding conversations. Together, these practices reveal some of the complexities of contact and conversation management. We investigated contact and conversation management by carrying out twenty semi-structured interviews with professionals in assorted fields. Key properties of technologically-mediated conversations identified were: (1) they are extended in time, which means (2) people typically engage in multiple concurrent conversations, and (3) conversations often involve multiple participants. These properties led to a significant memory load for our informants: they spoke of the difficulty of keeping tracking of conversational content and status, as well as the identity, contact information, and expertise of their conversational partners. People respond to these problems by trying to make key aspects of their conversations persistent; however, with current support tools, this strategy meets with mixed success. Building on the findings of our study, we present a new support tool that aids in managing contacts and conversation status.</p> 3 Goecks, J. Cosley, D. 2002 NuggetMine: Intelligent Groupware for Opportunistically Sharing Information Nuggets 2002 International Conference on Intelligent UserFaces Assocation of Computing Machinery Proceedings of the 2002 International Conference on Intelligent User Interfaces <p>We investigate the use of dimensionality reduction to improve the performance for a new class of data analysis software called â徨ecommender systemsâ. Recommender systems apply knowledge discovery techniques to the problem of making personalized product recommendations during a live customer interaction. The tremendous growth of customers and products in recent years poses some key challenges for recommender systems. These are:pr oducing high quality recommendations and performing many recommendations per second for millions of customers and products. Singular NuggetMine is an intelligent groupware application that collaborates with a workgroup to increase information nugget sharing among the group. Information nuggets are small amounts of self-contained information, such as the URL of an interesting news article, a book title, or the time and location of a local art event. NuggetMine and the workgroup work together to build, maintain, and utilize a repositoryâ俳r â徇ineââ俳f information nuggets. Group members submit nuggets to NuggetMine, which organizes and augments the submitted nuggets and provides a desktop interface to each group member. This interface makes it easy for group members to submit nuggets, view nuggets, and explore the mine. NuggetMine distributes the tasks necessary to share nuggets between it and the workgroup so as to best utilize the skills of each collaborator. In this paper, we describe the NuggetMine application and interface and present a pilot study of the application.</p> http://doi.acm.org/10.1145/502716.502732 7 Konstan, J. Riedl, J. 2002 Recommender Systems for the Semantic Web Geroimenko, V. Chen, C. Visualizing the Semantic Web Springer Verlag 3 Cosley, D., Lawrence, S., Pennock, D.M. 2002 REFEREE: an open framework for practical testing of recommender systems using ResearchIndex. 28th International Conference on Very Large Data Bases (VLDB 2002) Hong Kong, China 35-46 <p>Automated recommendation (e.g., personalized product recommendation on an ecommerce web site) is an increasingly valuable service associated with many databases--typically online retail catalogs and web logs. Currently, a major obstacle for evaluating recommendation algorithms is the lack of any standard, public, real-world testbed appropriate for the task. In an attempt to fill this gap, we have created REFEREE, a framework for building recommender systems using ResearchIndex--a huge online digital library of computer science research papers--so that anyone in the research community can develop, deploy, and evaluate recommender systems relatively easily and quickly. Research Index is in many ways ideal for evaluating recommender systems, especially so-called hybrid recommenders that combine information filtering and collaborative filtering techniques. The documents in the database are associated with a wealth of content information (author, title, abstract, full text) and collaborative information (user behaviors), as well as linkage information via the citation structure. Our framework supports more realistic evaluation metrics that assess user buy-in directly, rather than resorting to offline metrics like prediction accuracy that may have little to do with end user utility. The sheer scale of ResearchIndex (over 500,000 documents with thousands of user accesses per hour) will force algorithm designers to make real-world trade-offs that consider performance, not just accuracy. We present our own tradeoff decisions in building an example hybrid recommender called PD-Live. The algorithm uses content-based similarity information to select a set of documents from which to recommend, and collaborative information to rank the documents. PD-Live performs reasonably well compared to other recommenders in ResearchIndex.</p> http://www.grouplens.org/papers/pdf/referee.pdf 3 Sarwar, B.M. Karypis, G. Konstan, J.A. Riedl, J. 2002 Recommender Systems for Large-Scale E-Commerce: Scalable Neighborhood Formation Using Clustering The Fifth International Conference on Computer and Information Technology (ICCIT 2002) <p>Recommender systems apply knowledge discovery techniques to the problem of making personalized product recommendations during a live customer interaction. These systems, especially the k-nearest neighbor collaborative filtering based ones, are achieving widespread success in E-commerce nowaday s. The tremendous growth of customers and products in recent ears poses some key challenges for recommender systems. These are:producing high quality recommendations and performing many recommendations per second for millions of customers and products. New recommender system technologies are needed that can quickly produce high quality recommendations, even for very large-scale problems. We address the performance issues by scaling up the neighborhood formation process through the use of clustering techniques.</p> http://www.grouplens.org/papers/pdf/sarwar_cluster.pdf 7 Riedl, J. Konstan, J. 2002 Word of Mouse: The Marketing Power of Collaborative Filtering Word of Mouse: The Marketing Power of Collaborative Filtering New York Warner Books 3 Riedl, J. 2001 Recommender systems in commerce and community 7th ACM SIGKDD international conference on Knowledge Discovery and Data Mining San Francisco, CA Association for Computing Machinery 15 8/26/2001 1-58113-391-X 0 Herlocker, J. Konstan, J. 2001 Content-Independent, Task-Focused Recommendations IEEE Internet Computing 5 Nov-Dec 40-47 2001 0 Riedl, J. 2001 Personalization and privacy IEEE Journal of Internet Computing 5 6 29-31 11/2001 7116293 data privacy, electronic commerce, information resources, marketing data processing <p>Personalization has been a hot topic or nearly a decade now, and many new products and advanced algorithms have emerged in that time. Several companies now sell tools such as recommender systems, which take input about users and products and generate recommendations about which products the users will like best. At their best, recommenders can be wonderful tools for users, helping them sort through myriad items they could read, buy, or watch to select those few that are most valuable to them. The algorithms that power these systems have evolved dramatically, and the best can produce rapid recommendations over data sets of millions of users and hundreds of thousands of products. The other edge of the sword is that recommender systems provide perfect tools for marketers and others to invade users' privacy. After all, recommenders; seek to learn everything about our preferences, including what we like to read, what we like to buy, how much money we spend, and what influences us to spend it. How a recommender deals with privacy decides whether its-users view it,as a boon or a bane. If the recommender only uses this information to help us find items to purchase on a Web site, we will probably value the feature - it might even bring us back to shop there again. On the other hand, if the Web site sells our information to other companies, so they can more effectively bother us. with phone calls at dinner time, we'll probably feel our privacy has been invaded. Privacy is a critical issue for recommender systems. In the end, personalization is an important factor in developing effective Web sites because it creates a user experience that is both compelling and sticky. The experience is compelling because it helps users find exactly the information, products, and services they need. It is sticky because a personalized Web site trains itself over time to serve its users better, which makes those users less likely to go to a new site that they would have to train all over again</p> 0 Konstan, J.A. 2001 Turn off the cameras--I'll take a traditional classroom eLearn 10 2 10/2001 human factors 3 Bailey, B.P. Konstan, J.A. Carlis, J.V. 2001 DEMAIS: designing multimedia applications with interactive storyboards Ninth ACM international Conference on Multimedia Ottawa, Canada Association for Computing Machinery 9 241-250 Multimedia '01 09/2001 1-58113-394-4 authoring, gestures, multimedia design, storyboards, design, experimentation, performance <p>To create an innovative interactive multimedia application, a multimedia designer needs to rapidly explore numerous behavioral design ideas early in the design process, as creating innovative behavior is the cornerstone of creating innovative multimedia. Current tools and techniques do not support a designer's need for early behavior exploration, limiting her ability to rapidly explore and effectively communicate behavioral design ideas. To address this need, we have developed a sketch-based, interactive multimedia storyboard tool that uses a designer's ink strokes and textual annotations as an input design vocabulary. By operationalizing this vocabulary, the tool transforms an otherwise static sketch into a working example. The behavioral sketch can be quickly edited using gestures and an expressive visual language. By enabling a designer to explore and communicate behavioral design ideas using working examples early in the design process, our tool facilitates the creation of a more effective, compelling, and entertaining multimedia application.</p> 3 Konstan, J.A. 2001 Heavyweight Applications of Lightweght User Models: A Look at Collaborative Filtering M. Bauer, P.J. Gmytrasiewicz, J. Vassileva 8th international Conference on User Modeling Springer-Verlag 2109 314 Lectures in Computer Science 07/2001 3-540-42325-7 0 Schafer, J., Konstan, J.A. and Riedl, J. 2001 E-Commerce Recommendation Applications Data Mining Knowledge Discovery 5 1-2 115-153 01/2001 7 Sarwar, B. M. Konstan, J. A. Riedl, J. 2001 Distributed Recommender Systems: New Opportunities for Internet Commerce Rahman, S. Bignall, R. Internet Commerce and Software Agents: Cases, Technologies and Opportunities 31 Schafer, J.B. Konstan, J. Riedl, J. 2001 Data Mining and Knowledge Discovery

In "Application of Data Mining to Electronic Commerce", Kluwer Academic Publishers and edited by Kohavi, R. and Provost, F.

 31 Terveen, L.G. Hill, W. 2001 Human-Computer Collaboration in Recommender Systems <p>The Internet and World Wide Web have brought us into a world of endless possibilities: interactive Web sites to experience, music to listen to, conversations to participate in, and every conceivable consumer item to order. But this world also is one of endless choice: how can we select from a huge universe of items of widely varying quality?<br />Computational recommender systems have emerged to address this issue. They enable people to share their opinions and benefit from each otherâ冱 experience. We present a framework for understanding recommender systems and survey a number of distinct approaches in terms of this framework. We also suggest two main research challenges:<br />(1) helping people form communities of interest while respecting personal privacy, and<br />(2) developing algorithms that combine multiple types of information to compute<br />recommendations.</p>

in "HCI In the New Millenium" by Carroll, J. Addison-Wesley

 http://www.grouplens.org/papers/pdf/rec-sys-overview.pdf 31 Dewan, P. Mashayekhi, V. Riedl, J. 2001 Infrastructure and tools for collaborative software engineering

In "Coordination Theory and Collaboration Technology", by Tom Malone, Gary Olson, and John Smith, Ed.ﾂ Published by Lawrence Erlbaum Associations

 3 Sarwar, B.M. Karypis, G. Konstan, J.A. Riedl, J. 2001 Item-based collaborative filtering recommendation algorithms 10th International World Wide Web Conference Association of Computing Machinery 2001 <p>Recommender systems apply knowledge discovery techniques to the problem of making personalized recommendations for information, products or services during a live interaction. These systems, especially the k-nearest neighbor collaborative filtering based ones, are achieving widespread success on the Web. The tremendous growth in the amount of available information and the number of visitors to Web sites in recent years poses some key challenges for recommender systems. These are: producing high quality recommendations, performing many recommendations per second for millions of users and items and achieving high coverage in the face of data sparsity. In traditional collaborative filtering systems the amount of work increases with the number of participants in the system. New recommender system technologies are needed that can quickly produce high quality recommendations, even for very large-scale problems. To address these issues we have explored item-based colllaborative filtering techniques. Item-based techniques first analyze the user-item matrix to identify relationships between different items, and then use these relationships to indirectly compute recommendations for users.</p> <p>In this paper we analyze different item-based recommendation generation algorithms. We look into different techniques for computing item-item similarities (e.g., item-item correlation vs. cosine similarities between item vectors) and different techniques for obtaining recommendations from them (e.g., weighted sum vs. regression model). Finally, we experimentally evaluate our results and compare them to the basic k-nearest neighbor approach. Our experiments suggest that item-based algorithms provide dramatically better performance than user-based algorithms, while at the same time providing better quality then the best available user-based algorithms.</p>

Published in Hong Kong.

 http://www.grouplens.org/papers/pdf/www10_sarwar.pdf 7 Bailey, B. Konstan, J.A. Cooley, R. Dejong, M. 2001 Nsync: a toolkit for building interactive multimedia presentations K. Jaffay and H. Zhang Readings in Multimedia Computing and Networking San Francisco, CA Morgan Kaufmann Publishers 761-770 1-55860-651-3 design 3 O'Connor, M. Cosley, D. Konstan, J. A. Riedl, J. 2001 PolyLens: A Recommender System for Groups of Users Europeon Conference on Computer Supported Co-Operative Work Bonn, Germany 199-218 <p>We present PolyLens, a new collaborative filtering recommender system designed to recommend items for groups of users, rather than for individuals. A group recommender is more appropriate and useful for domains in which several people participate in a single activity, as is often the case with movies and restaurants. We present an analysis of the primary design issues for group recommenders, including questions about the nature of groups, the rights of group members, social value functions for groups, and interfaces for displaying group recommendations. We then report on our PolyLens prototype and the lessons we learned from usage logs and surveys from a nine-month trial that included 819 users. We found that users not only valued group recommendations, but were willing to yield some privacy to get the benefits of group recommendations. Users valued an extension to the group recommender system that enabled them to invite non-members to participate, via email.</p>

in "Proceedings of ECSCW 2001".

 http://www.grouplens.org/papers/pdf/poly-camera-final.pdf 3 Wexelblat, A. Jones, Q. Abrams, H. Konstan, J.A. Vernick, M. 2001 Ph.D. vs. Startup ACM SIGCHI Conference on Human Factors in Computing Systems Seattle, Washington 229-230 CHI Extended Abstracts 1-58113-340-5 Ph.D., career choice, employment, startup, economics, human factors <p>How should we decide between the conflicting demands and resource drains of startups and Ph.D. programs? This panel discusses some of the current issues facing students, professors and employers, from the panelists' varied perspectives. We do not advocate any one single solution but rather seek to illuminate important issues that affect the CHI community now and will likely continue to do so in the future, both in the US and in other countries promoting high-tech startups as major parts of their economies.</p> 3 Herlocker, J. Konstan, J. Riedl, J. 2000 Explaining Collaborative Filtering Recommendations ACM 2000 Conference on Computer Supported Cooperative Work Association of Computing Machinery 241-250 12/2000 <p>Automated collaborative filtering (ACF) systems predict a personâ冱 affinity for items or information by connecting that personâ冱 recorded interests with the recorded interests of a community of people and sharing ratings between likeminded persons. However, current recommender systems are black boxes, providing no transparency into the working of the recommendation. Explanations provide that transparency, exposing the reasoning and data behind a recommendation. In this paper, we address explanation interfaces for ACF systems â how they should be implemented and why they should be implemented. To explore how, we present a model for explanations based on the userâ冱 conceptual model of the recommendation process. We then present experimental results demonstrating what components of an explanation are the most compelling. To address why, we present experimental evidence that shows that providing explanations can improve the acceptance of ACF systems. We also describe some initial explorations into measuring how explanations can improve the filtering performance of users.</p>

In proceedings

 http://www.grouplens.org/papers/pdf/explain-CSCW.pdf 3 Amento, B. Terveen, L. Hill, W. Hix, D. 2000 TopicShop: enhanced support for evaluating and organizing collections of Web sites 13th Annual Symposium on User interface Software adn Technology San Diego, CA Associate for Computing Machinery 201-209 11/2000 1-58113-212-3 Design, Human Factors, Management, Measurement, Performance, Theory <p><em>TopicShop</em> is an interface that helps users evaluate and organize collections of web sites. The main interface components are <em>site profiles</em>, which contain information that helps users select high-quality items, and a <em>work area,</em> which offers thumbnail images, annotation, and lightweight grouping techniques to help users organize selected sites. The two components are linked to allow task integration.</p> <p></p> <p>P</p> 0 Chi, E. H. Riedl, J. Shoop, E. Barry, P. 2000 A Novel Visualization Method for Biological Sequence Similarity Reports Journal of Electronic Imaging: Special Issue on Visualization and Data Analysis 10/2000 3 Sarwar, B. M. Karvpis, G. Konstan, J. A. Riedl, J. 2000 Analysis of Recommender Algorithms for E-Commerce ACM E-Commerce 2000 Conference 158-167 10/2000 <p class="MsoNormal">Recommender systems apply statistical and knowledge discovery techniques to the problem of making product recommendations during a live customer interaction and they are achieving widespread success in E-Commerce nowadays. In this paper, we investigate several techniques for analyzing large-scale purchase and preference data for the purpose of producing useful recommendations to customers. In particular, we apply a collection of algorithms such as traditional data mining, nearest-neighbor collaborative filtering, and dimensionality reduction on two different data sets. The first data set was derived from the web-purchasing transaction of a large E-commerce company whereas the second data set was collected from MovieLens movie recommendation site. For the experimental purpose, we divide the recommendation generation process into three sub processes-representation of input data, neighborhood formation, and recommendation generation. We devise different techniques for different sub processes and apply their combinations on our data sets to compare for recommendation quality and performance.</p>

In Proceedings

 http://www.grouplens.org/papers/pdf/ec00.pdf 3 Bailey, B.P. Konstan, J.A. Carlis, J.V. 2000 Measuring the effects of interruptions on task performance in the user interface IEEE International Conference on Systems, Man, and Cybernetics Nashville, TN 2 757-762 10/2000 0-7803-6583-6 6771340 human factors, information resources, interactive systems, user interfaces <p>As users continue offloading more control and responsibility to the computer, coordinating the asynchronous interactions between the user and computer is becoming increasingly important. Without proper coordination, an application attempting to gain the user's attention risks interrupting the user in the midst of performing another task. To justify why an application should avoid interrupting the user whenever possible, we designed an experiment measuring the disruptive effect of an interruption on a user's task performance. The experiment utilized six Web-based task categories and two categories of interruption tasks. The results of the experiment demonstrate that: (i) a user performs slower on an interrupted task than a non-interrupted task, (ii) the disruptive effect of an interruption differs as a function of the task category, and (iii) different interruption tasks cause similar disruptive effects on task performance. These results empirically validate the need to better coordinate user interactions among applications that are competing for the user's attention.</p> 0 Whittaker, S. Terveen, L. Nardi, B.A. 2000 Let's stop pushing the envelope and start addressing it: a reference task agenda for HCI Conference on Human Factors in Computing Systems 15 2 75-106 09/2000 Design, Human Factors, Management, Theory <p>We identify a problem with the process of research in the human-computer interaction (HCI) community-an overemphasis on "radical invention" at the price of achieving a common research focus. Without such a focus, it is difficult to build on previous work, to compare different interaction techniques objectively, and to make progress in developing theory. These problems at the research level have implications for practice, too; as researchers we often are unable to give principled design advice to builders of new systems. We propose that the HCI community try to achieve a common focus around the notion of reference tasks. We offer arguments for the advantages of this approach as well as consider potential difficulties. We explain how reference tasks have been highly effective in focusing research into information retrieval and speech recognition. We discuss what factors have to be considered in selecting HCI reference tasks and present an example reference task (for searching speech archives). This example illustrates the nature of reference tasks and points to the issues and problems involved in constructing and using them. We conclude with recommendations about what steps need to be taken to execute the reference task research agenda. This involves recommendations about both the technical research that needs to be done and changes in the way that the HCI research community operates. The technical research involves identification of important user tasks by systematic requirements gathering, definition and operationalization of reference tasks and evaluation metrics, and execution of task-based evaluation, along with judicious use of field trials. Perhaps more important, we have also suggested changes in community practice that HCI must adopt to make the reference tasks idea work. We must create forums for discussion of common tasks and methods by which people can compare systems and techniques. Only by doing this can the notion of reference tasks be integrated into the process of research and development, enabling the field to achieve the focus it desperately needs.</p> 3 Hagen, P., Kavahi, R., Lowell, B.J. and Riedl, J. 2000 Personalization and data mining: exploring the synergies (panel session) 6th ACM SIGKDD international conference on Knowledge Discovery and Data Mining Boston, MA Association for Computing Machinery 535 08/2000 1-58113-233-6 3 Amento, B. Terveen, L. Hill, W. 2000 Does "authority" mean quality? Predicting expert quality ratings of Web documents 23rd Annual international ACM SIGIR Conference on Research and Development in information retrieval Athens, Greece Association for Computing Machinery 296-303 07/2000 1-58113-226-3 Algorithms, Human Factors, Performance, Reliability, exploring hyperlink structure <p>For many topics, the World Wide Web contains hundreds or thousands of relevant documents of widely varying quality. Users face a daunting challenge in identifying a small subset of documents worthy of their attention. Link analysis algorithms have received much interest recently, in large part for their potential to identify high quality items. We report here on an experimental evaluation of this potential. We evaluated a number of link and content-based algorithms using a dataset of web documents rated for quality by human topic experts. Link-based metrics did a good job of picking out high-quality items. Precision at 5 is about 0.75, and precision at 10 is about 0.55; this is in a dataset where 0.32 of all documents were of high quality. Surprisingly, a simple content-based metric performed nearly as well; ranking documents by the total number of pages on their containing site.</p> 0 Kazi, I.H. Jose, D.P. Ben-Hamida, B. Hescott, C.J. Kwok, C. Konstan, J.A. Lilja, D.J. Yew, P. 2000 JaViz: a client/server Java profiling tool IBM Systems Journal 39 1 96-117 01/2000 Measurement, Performance, Standardization <p>For many topics, the World Wide Web contains hundreds or thousands of relevant documents of widely varying quality. Users face a daunting challenge in identifying a small subset of documents worthy of their attention. Link analysis algorithms have received much interest recently, in large part for their potential to identify high quality items. We report here on an experimental evaluation of this potential. The JavaTM programming language, with its portability, object-oriented model, support for multithreading and distributed programming, and garbage collection features, is becoming the language of choice for the development of large-scale distributed applications. Without a suitable performance analysis tool for Java programs, however, it is often difficult to analyze the programs for performance-tuning problems. The profiler included in Sun's Java Development Kit (JDKTM) 1.1 does not provide sufficiently detailed trace information to address performance issues in large applications. Also, it does not support the tracing of client/server applications, which are very important for analyzing distributed systems. The JaViz performance analysis tool generates execution traces with sufficient detail to determine program hot spots, including remote method calls, in a distributed Java application program. JaViz provides a graphical display of the program execution tree for the entire distributed application in the form of a call graph for ease of visualization. A number of features allow users to analyze the execution tree for performance-tuning problems more easily than other Java performance monitoring tools. The usability and functionality of the JaViz tool set is demonstrated by applying it to an example distributed Java application program.</p> 31 Sarwar, B. M. Karypis, G. Konstan, J. A. Riedl, J. 2000 Applications of Dimensionality Reduction in Recommender Systems - - A Case Study ACM WebKDD 2000 Web Mining for E-Commerce Workshop Association for Computing Machinery <p>We investigate the use of dimensionality reduction to improve performance for a new class of data analysis software called â徨ecommender systemsâ. Recommender systems apply knowledge discovery techniques to the problem of making product recommendations during a live customer interaction. These systems are achieving widespread success in E-commerce nowadays, especially with the advent of the Internet. The tremendous growth of customers and products poses three key challenges for recommender systems in the E-commerce domain. These are: producing high quality recommendations, performing many recommendations per second for millions of customers and products, and achieving high coverage in the face of data sparsity. One successful recommender system technology is collaborative filtering, which works by matching customer preferences to other customers in making recommendations. Collaborative filtering has been shown to produce high quality recommendations, but the performance degrades with the number of customers and products. New recommender system technologies are needed that can quickly produce high quality recommendations, even for very largescale problems. This paper presents two different experiments where we have explored one technology called Singular Value Decomposition (SVD) to reduce the dimensionality of recommender system databases. Each experiment compares the quality of a recommender system using SVD with the quality of a recommender system using collaborative filtering. The first experiment compares the effectiveness of the two recommender systems at predicting consumer preferences based on a database of explicit ratings of products. The second experiment compares the effectiveness of the two recommender systems at producing Top-N lists based on a real-life customer  purchase database from an E-Commerce site. Our experience suggests that SVD has the potential to meet many of the challenges of recommender systems, under certain conditions.</p> http://www.grouplens.org/papers/pdf/webKDD00.pdf 3 Chi, E.H. Riedl, J. 2000 Case Study: Resource Steering in a Visualization System VisSym00

In proceedings.

 0 Schafer, J.B. Konstan, J. Riedl, J. 2000 Electronic Commerce Recommender Applications Journal of Data Mining and Knowledge Discovery 5 1/2 115-152 <p>Recommender systems are being used by an ever-increasing number of E-commerce sites to help consumers find products to purchase. What started as a novelty has turned into a serious business tool. Recommender systems use product knowledge â either hand-coded knowledge provided by experts or â徇inedâ knowledge learned from the behavior of consumers â to guide consumers through the often-overwhelming task of locating products they will like. In this article we present an explanation of how recommender systems are related to some traditional database analysis techniques. We examine how recommender systems help E-commerce sites increase sales and analyze the recommender systems at six market-leading sites. Based on these examples, we create a taxonomy of recommender systems, including the inputs required from the consumers, the additional knowledge required from the database, the ways the recommendations are presented to consumers, the technologies used to create the recommendations, and the level of personalization of the recommendations. We identify five commonly used E-commerce recommender application models, describe several open research problems in the field of recommender systems, and examine privacy implications of recommender systems technology.</p> http://www.grouplens.org/papers/pdf/ECRA.pdf 3 Herlocker, J. Konstan, J. Borchers, A Riedl, J. 1999 An Algorithmic Framework for Performing Collaborative Filtering Research and Development in Information Retrieval American Association of Computing Machinery 8/1999 <p class="MsoNormal">Automated collaborative filtering is quickly becoming a popular technique for reducing information overload, often as a technique to complement content-based information filtering systems. In this paper we present an algorithmic framework for performing collaborative filtering and new algorithmic elements that increase the accuracy of collaborative prediction algorithms. We then present a set of recommendations on selection of the right collaborative filtering algorithmic components.</p> http://www.grouplens.org/papers/pdf/algs.pdf 3 Schafer, J.B. Konstan, J. Riedl, J. 1999 Recommender Systems in E-Commerce ACM Conference on Electronic Commerce Association of Computing Machinery 11/1999 <p>Recommender systems are changing from novelties used by a few E-commerce sites, to serious business tools that are re-shaping the world of E-commerce. Many of the largest commerce Web sites are already using recommender systems to help their customers find products to purchase. A recommender system learns from a customer and recommends products that she will find most valuable from among the available products. In this paper we present an explanation of how recommender systems help E-commerce sites increase sales, and analyze six sites that use recommender systems including several sites that use more than one recommender system. Based on the examples, we create a taxonomy of recommender systems, including the interfaces they present to customers, the technologies used to create the recommendations, and the inputs they need from customers. We conclude with ideas for new applications of<br />recommender systems to E-commerce.</p>

In Proceedings of the ACM Conference on Electronic Commerce

 http://www.grouplens.org/papers/pdf/ec-99.pdf 3 Stein, M.V., Heimdahl, M.P. and Riedl, J. 1999 Enhancing Annotation Visibilty for Software Inspection Enhancing Annotation Visibility for Software Inspection Washington, D.C. IEEE Computer Society 243 10/12/1999 <p>Annotation of software artifacts is common in software development, and vital for software inspection. People viewing annotated artifacts encounter delocalization: they must understand various parts of an artifact (and their annotations) to understand the part they are viewing. We taxonomize delocalization within software systems into lateral delocalization (different items of the artifact within the same development phase), longitudinal delocalization (related items in different phases), and historical delocalization (successive versions of the same item). We report on a pilot study of code inspection with AnnoSpec, an inspection tool supporting visibility of laterally-delocalized annotations. Our results suggest that addressing delocalization may help people perform inspections more effectively</p> 3 Amento, B. Hill, W. Terveen, L. Hix, D. Ju, P. 1999 An empirical evaluation of user interfaces for topic management of Web sites SIGCHI Conference on Human Factors in Computing Systems: the CHI is the limit Pittsburgh, PA Association for Computing Machinery 552-559 05/1999 0-201-48559-1 Design, Management, Verification, computer supported cooperative work, human-computer interaction, information access, information retrieval, information visualization, social filtering <p>Topic management is the task of gathering, evaluating, organizing, and sharing a set of web sites for a specific topic. Current web tools do not provide adequate support for this task. We created the TopicShop system to address this need. TopicShop includes (1) a webcrawler that discovers relevant web sites and builds site profiles, and (2) user interfaces for exploring and organizing sites. We conducted an empirical study comparing user performance with TopicShop vs. YahooTM. TopicShop subjects found over 80% more high-quality sites (where quality was determined by independent expert judgements) while browsing only 8 1% as many sites and completing their task in 89% of the time. The site profile data that TopicShop provides - in particular, the number of pages on a site and the number of other sites that link to it - was the key to these results, as users exploited it to identify the most promising sites quickly and easily.</p> 3 Baudisch, P. Terveen, L. 1999 Interacting with recommender systems ACM SIGCHI Conference on Human Factors in Computing Systems Pittsburgh, Pennsylvania Association for Computing Machinery 164 05/1999 1-58113-158-5 collaborative filtering, implicit feedback, recommender system, social filtering, user interface <p>Many people today live in information-rich worlds, constantly facing the question: what should I do next? Which papers should I read to learn about a new area I am interested in? Which movie should I go to? Which restaurant would I like? The experience of friends and colleagues is a valuable resource for making such decisions, especially friends who are familiar with the subject area and have similar tastes.The field of recommender systems (or collaborative filtering) attempts to automate this process, e.g., by supporting people in making recommendations, finding a set of people who are likely to provide good recommendations for a given person, or deriving recommendations from implicit behavior such as browsing activity, buying patterns, and time on task.</p> 3 Claypool, M. Riedl, J. 1999 The effects of high-speed networks on multimedia jitter EUROMEDIA Conference Munich, Germany 04/1999 0 Terveen, L.G. Hill, W.C. Amento, B. 1999 Constructing, Organizing, and Visualizing Collections of Topically Related Web Resources ACM Transactions on Computer-Human Interaction 6 1 67-94 03/1999 <p>For many purposes, the Web page is too small a unit of interaction and analysis. Web sites are structured multimedia documents consisting of many pages, and users often are interested in obtaining and evaluating entire collections of topically related sites. Once such a collection is obtained, users face the challenge of exploring, comprehending, and organizing the items. We report four innovations that address these user needs.</p> <p>· We replaced the web page with the web site as the basic unit of interaction and analysis.<br />· We defined a new information structure, the clan graph, that groups together sets of related sites.<br />· We augment the representation of a site with a site profile, information about site structure and<br />content that helps inform user evaluation of a site.<br />· We invented a new graph visualization, the auditorium visualization, that reveals important structural and content properties of sites within a clan graph.</p> <p>Detailed analysis and user studies document the utility of this approach. The clan graph construction algorithm tends to filter out irrelevant sites and discover additional relevant items. The auditorium visualization, augmented with drill down capabilities to explore site profile data, helps users to find high-quality sites as well as sites that serve a particular function.</p> http://www.grouplens.org/papers/pdf/collections-tochi99.pdf 3 Good, N. Schafer, J.B. Konstan, J. Borchers, A. Sarwar, B. Herlocker, J. Riedl, J. 1999 Combining Collaborative Filtering with Personal Agents for Better Recommendations 1999 Conference of the American Association of Artificial Intelligence American Association of Artificial Intelligence 439-446 <p>Information filtering agents and collaborative filtering both attempt to alleviate information overload by identifying which items a user will find worthwhile. Information filtering (IF) focuses on the analysis of item content and the development of a personal user interest profile. Collaborative filtering (CF) focuses on identification of other users with similar tastes and the use of their opinions to recommend items. Each technique has advantages and limitations that suggest that the two could be beneficially combined. This paper shows that a CF framework can be used to combine personal IF agents and the opinions of a community of users to produce better recommendations than either agents or users can produce alone. It also shows that using CF to create a personal combination of a set of agents produces better results than either individual agents or other combination mechanisms. One key implication of these results is that users can avoid having to select among agents; they can use them all and let the CF framework select the best ones for them.</p>

In Proceedings

 http://www.grouplens.org/papers/pdf/aaai-99.pdf 0 Terveen, L. Hill, W. Amento, B. 1998 Collaborative filtering to locate, comprehend, and organize collections of Web sites SIGART Bulletin 9 3-4 12/1998 Algorithms, Design, Management, Performance, Theory <p>Finding information is a common and fundamental task for Internet users. Search engines such as AltaVista and indices such as Yahoo! are the tools commonly used for this task. Users type in queries using keywords to indicate their interest or navigate through hierarchical directories of topics, eventually obtaining (usually quite large) collections of Web pages.</p> 3 Terveen, L. Hill, W. 1998 Evaluating emergent collaboration on the web 1998 Computer Supported Cooperative Work Seattle, WA Association for Computing Machinery 355-362 11/1998 1-58113-009-0 Design, Management, Measurement, Performance, Theory, collaborative filtering, computer supported cooperative work, human computer interaction, information access, information retrieval, social filtering <p>Links between web sites can be seen as evidence of a type of <em>emergent collaboration</em> among web site authors. We report here on an empirical investigation into emergent collaboration. We developed a webcrawling algorithm and tested its performance on topics volunteered by 30 subjects.</p> <p></p> 3 Carlis, J.V. Konstan, J.A. 1998 Interactve visualizaton of serial periodic data 11th Annual ACM Symposium on User interface Software and Technology San Francisco, CA Association for Computing Machinery 29-38 11/1998 1-58113-034-1 data visualization, information visualization, interactive data exploration, serial periodic data, spiral, algorithms 3 Whittaker, S. Terveen, L. Hill, W. Cherny, L. 1998 The dynamics of mass interaction 1998 ACM Conference on Computer Supported Cooperative Work Seattle, WA Association for Computing Machinery 257-264 11/1998 1-58113-009-0 Design, Management, Measurement, Performance, Theory, FAQS, Usenet, common ground, conversation, empirical, mass interaction, moderation, netiquitte, newsgroups <p>usenet may be regarded as the world's largest conversational application, with over 17,000 newsgroups and 3 million users. Despite its ubiquity and popularity, however, we know little about the nature of the interactions it supports. This empirical paper investigates <em>mass interaction</em> in Usenet. We analyse over 2.15 million messages from 659,450 posters, collected from 500 newsgroups over 6 months. We first characterise mass interaction, presenting basic data about demographics, conversational strategies and interactivity. Using predictions from the common ground model of interaction, we next conduct causal modelling to determine relations between <em>demographics, conversational strategies </em>and <em>interactivity.</em> We find evidence for moderate conversational threading, but large participation inequalities in Usenet, with a small minority of participants posting a large proportion of messages. Contrary to the common ground model and "Netiquette" guidelines, we also find that "cross-posting" to external newsgroups is highly frequent. Our predictions about the effects of demographics on conversational strategy were largely confirmed, but we found disconfirming evidence about teh relations between conversational strategy and interactivity. Contrary to our expectations, both cross-posting and short messages promote interactivity. We conclude that in order to explain <em>mass interaction</em>, the common ground model must be modified to incorporate notions of weak ties and communication overload.</p> 3 Sarwar, B. Konstan, J. Borchers, A. Herlocker, J. Miller, B. Riedl, J. 1998 Using Filtering Agents to Improve Prediction Quality in the GroupLens Research Collaborative Filtering System Conference on Computer Supported Cooperative Work Association of Computing Machinery 11/1998 <p>Collaborative filtering systems help address information overload by using the opinions of users in a community to make personal recommendations for documents to each user. Many collaborative filtering systems have few user opinions relative to the large number of documents available. This sparsity problem can reduce the utility of the filtering system by reducing the number of documents for which the system can make recommendations and adversely affecting the quality of recommendations. This paper defines and implements a model for integrating content-based ratings into a collaborative filtering system. The filterbot model allows collaborative filtering systems to address sparsity by tapping the strength of content filtering techniques. We identify and evaluate metrics for assessing the effectiveness of filterbots specifically, and filtering system enhancements in general. Finally, we experimentally validate the filterbot approach by showing that even simple filterbots such as spell checking can increase the utility for users of sparsely populated collaborative filtering systems.</p>

In Proceedings.

 http://www.grouplens.org/papers/pdf/filterbot-CSCW98.pdf 0 Konstan, J.A. Siegel, J. 1998 Unifying HCI: the impossible possibility SIGCHI Bulletin 30 4 30-32 10/1998 design, human factors, management 3 Chi, E.H. and Riedl, J. 1998 An Operator Interaction Framework for Visualization Systems 1998 IEEE Symposium on Information Visualization North Carolina IEEE Computer Society 63-70 10/19/1998 3 DeJong, M. Bailey, B. Konstan, J.A. 1998 Creating a multimedia extension for Tcl using the java media framework 6th Conference on Annual Tcl/Tk Workshop San Diego, CA USENIX Association 6 16 09/1998 Jacl, Java, Java media framework, Tcl extension, TclBlend, multimedia, synchronization, design, languages <p>As multimedia capable computers become cheaper and more pervasive in the consumer and corporate markets, and as the availability of digital information increases, the need for low-cost, cross-platform multimedia applications will steadily rise. However, because Tcl lacks native support for continuous media streams, such as audio, video, and animation, it is not well suited for this emerging application domain. At the same time, Java now provides a set of class libraries, called the Java Media Framework (JMF), which provides the multimedia support that Tcl lacks. With the recently introduced integration of Tcl and Java, Java can now be used to provide the cross-platform multimedia support required by Tcl; whereas Tcl can be used to provide the easy-touse programming environment required for building multimedia applications. In this paper, we introduce a Tcl extension that provides a high-level scripting interface to the Java Media Framework. In addition, we will highlight some interesting problems in the current Tcl/Java package as well as suggest some potential solutions. This paper will benefit Tcl programmers who would like to learn more about using Tcl to build multimedia applications, integrating Tcl and Java, or the multimedia support provided by the JMF.</p> 3 Stein, M., Heimdahl, M.P., Riedl, J. 1998 A General Framework for Interconnecting Annotations of Software Systems 22nd International Computer Software and Applications Conference Washington, D.C. IEEE Computer Society 421-429 08/1998 annotation, information filtering, object-oriented software development <p>Computer-supported annotation of software systems and their documentation, including design documentation and source code, is a common and important software engineering activity. Annotated documentation is used in both formal software inspection and informal software maintenance. Viewers of annotated systems may understand the software more easily if annotations are visible not just from the annotated item itself but from other, related items. We propose a general framework for interconnecting annotatable items in software systems to achieve this visibility. We describe filtering and broadening rules that viewers can use to select the annotations they desire to see. We illustrate this framework in the context of object-oriented software system development</p> 0 Chi, E.H., Riedl, J., Barry, P., Konstan, J. 1998 Principles for Information Visualization Spreadsheets IEEE Computer Graphics and Applications 18 4 30-38 07/1998 <p>The visualization spreadsheet provides a framework for exploring large and complex data sets. Structuring user interactions using a spreadsheet paradigm creates a powerful tool for information visualization</p> 3 Terveen, L. Hill, W. 1998 Finding and visualizing inter-site clan graphs ACM SIGCHI Conference on Human Factors in Computing Systems Los Angeles, CA Association for Computing Machinery/Addison-Wesley Publishing Company 448-455 04/1998 0-201-30987-4 Algorithms, Design, Human Factors, Management, Performance, Theory, co-citation analysis, collaborative filtering, computer supported cooperative work, human-computer interaction, information access, information retrieval, information visualization <p>For many purposes, the Web page is too small a unit of interaction. Users often want to interact with larger-scale entities, particularly collections of topically related items. We report three innovations that address this user need. 1. We replaced the web page with the web <em>site</em> as the basic unit of interaction and analysis 2. We defined a new information structure, the <em>clan graph</em>, that groups together sets of related sites and 3. We invented a new graph visualization, the <em>auditorium visualization,</em> that reveals important structural and content properties of sites within a clan graph. <br /><br />We have discovered interesting information that can be extracted from the structure of a clan graph. We can identify structurally important sites with many incoming or outgoing links. Links between sites serve important functions: they often identify "front door" pages of sites, sometimes identify especially significant pages within a site, and occasionally contain informative anchor text.</p> 0 Borchers, A. Herlocker, J. Konstan, J. Riedl, J. 1998 Ganging up on Information Overload Computer 31 4 106-108 04/1998 information filtering, collaborative computing 3 Gustafson, T. Schafer, J.B. Konstan, J.A. 1998 Agents in their midst: evaluating user adaptation to agent-assisted interfaces 3rd international Conference on intelligent user interfaces San Francisco Association for Computing Machinery 163-170 01/1998 0-89791-955-6 agent-assisted interface, electronic publication, learning agents, online newspaper production, user studies, user-centered interface design, Algorithms, Design, Human Factors, Languages, Measurement, Performance 7 Rafaeli, S. Sudweeks, F. Konstan, J. Mabry, E. 1998 ProjectH: a collaborative quantitative study of computer-mediated communication F. Sudweeks, M. McLaughlin, Rafaeli Network and Netplay: Virtual Groups on the internet Cambridge, MA MIT Press 265-281 0-262-69206-6 human factors 3 Stein, M., Riedl, J., Harner, S.J., Mashayekhi, V. 1997 A case study of distributed, asynchronous software inspection 19th international Conference on Software Engineering Boston, MA Assocation for Computing Machinery 107-117 5/1997 ISBN:0-89791-914-9 concurrent software engineering, software inspection, cscw, collaboration, groupware, world wide web <p>Traditional software inspection requires participants to meet together at the same time in the same place. Distributed, asynchronous inspection allows partcipants to conduct meetings independently of time and space, making inspection more convenient. We report on an industrial study that we have performed using a tool designed for distributed, asynchronous software inspection. Our experience suggests that distributed, asynchronous software inspection is feasible, and is a cost-effective means of collaboration for geographically distributed work groups.</p> 0 Konstan, J. Miller, B. Maltz, D. Herlocker, J. Gordon, L. Riedl, J. 1997 GroupLens: Applying Collaborative Filtering to Usenet News Communications of the ACM 40 3 77-87 1997 http://delivery.acm.org/10.1145/250000/245126/p77-konstan.pdf?key1=245126&key2=4383306021&coll=GUIDE&dl=GUIDE&CFID=60190076&CFTOKEN=76645379 0 Konstan, J.A. Sturm, P. McLeod, J. Lichtblau, L. 1997 Internet self-assessment in pharmacology: a model for Internet medical education Computer Education 29 2-3 63-71 11/1997 Design, Human Factors, Management, Measurement, Performance, Theory 3 Chi, E.H., Barry, P., Riedl, J. and Konstan, J. 1997 A spreadsheet approach to information visualization 1997 IEEE Symposium on information visualization Infovis '97 IEEE 17-24 10/1997 0-8186-8189-6 <p>In information visualization, as the volume and complexity of the data increases, researchers require more powerful visualization tools that enable them to more effectively explore multidimensional datasets. We discuss the general utility of a novel visualization spreadsheet framework. Just as a numerical spreadsheet enables exploration of numbers, a visualization spreadsheet enables exploration of visual forms of information. We show that the spreadsheet approach facilitates certain information visualization tasks that are more difficult using other approaches. Unlike traditional spreadsheets, which store only simple data elements and formulas in each cell, a visualization spreadsheet cell can hold an entire complex data set, selection criteria, viewing specifications, and other information needed for a full-fledged information visualization. Similarly, inter-cell operations are far more complex, stretching beyond simple arithmetic and string operations to encompass a range of domain-specific operators. We have built two prototype systems that illustrate some of these research issues. The underlying approach in our work allows domain experts to define new data types and data operations, and enables visualization experts to incorporate new visualizations, viewing parameters, and view operations.</p> 3 Miller, B. Riedl, J. Konstan, J. 1997 Experiences with GroupLens: Making Usenet Useful Again Usenix Winter Technical Conference 1/1997 <p>Collaborative filtering attempts to alleviate information overload by offering recommendations on whether information is valuable based on the opinions of those who have already evaluated it. Usenet news is an information source whose value is being severely diminished by the volume of low-quality and uninteresting information posted in its newsgroups. The GroupLens system applies collaborative filtering to Usenet news to demonstrate how we can restore the value of Usenet news by sharing our judgments of articles, with our identities protected by pseudonyms. This paper extends the original GroupLens work by reporting a significantly enhanced system and the results of a seven week trial with 250 users and over 20,000 news articles. GroupLens has an open and flexible architecture that allows easy integration of new newsreader clients and ratings bureaus. We show ratings and prediction profiles for three newsgroups, and assess the accuracy of the predictions.</p> http://www.grouplens.org/papers/pdf/usenix97.pdf 3 Hill, W. and Terveen, L. 1997 Involving remote users in continuous design of web content S. Coles 2nd Conference on Designing interactive Systems: Processes, Practices, Methods, and Techniques Amsterdam, The Netherlands Association for Computing Machinery 137-145 08/1997 0-89791-863-0 Usenet, World Wide Web, collaborative filtering, computer-supported cooperative work, end user modification, human interface, human-compter interaction, organizational computing, participatory design, remote evaluation, resource discovery <p>PHOAKS is a system that automatically recognizes URLs recommended in  Usenet Messages and continuously updates a large web site that that summarizes the recommendation data. We view the automatically generated pages as "rough drafts" that users help to refine. We report here on the mechanisms that allow users to do this, our rationale for these mechanisms, and the issues raised by involving thousands of remote anonymous users in the continuous design of web content.</p> 3 Safonov, A. Konstan, J.A. Carlis, J.V. Bailey, B. 1997 Extending traces with OAT: an object attribute trace for Tcl/Tk 5th Conference on Annual Tcl/Tk Workshop Boston, MA USENIX Association 14 07/1997 Design, Languages, Verification <p>Tcl supports variable traces, which associate arbitrary scripts with variable reads, writes and unsets. We developed OAT (Object Attribute Traces), a protocol for extending traces to attributes of arbitrary Tcl "objects." We wrote several OAT-based extensions including TkOAT, which provides traces on attributes of Tk widgets and canvas items. The OAT protocol and derived extensions bring the benefits of more expressive constraints to Tcl/Tk applications by providing extended traces. OAT requires no changes to the Tcl core and is implemented as a loadable library; OAT-based extended trace packages introduce minimal changes to the code of existing extensions (Tk, CMT, etc.). The new version of our formula manager, TclProp, takes advantage of extended traces provided by OAT.</p> 3 Terveen, L. Hill, W. Amento, B. McDonald, D. Creter, J. 1997 Building task-specific interfaces to high volume conversational data S. Pemberton ACM SIGCHI Conference on Human Factors in Computing Systems Atlanta, GA Association for Computing Machinery 226-233 03/1997 0-89791-802-9 Netnews, Usenet, World Wide Web, collaborative filtering, computer-supported cooperative work, data mining, human interface, human-computer interaction, organizatinal computing, resource discovery, social filtering 0 Terveen, L. Hill, W. Amento, B. McDonald, D. Creter, J. 1997 PHAOKS: a system for sharing recommendations Communications of the ACM 40 3 59-62 03/1997 documentation 3 Carlis, J.V. Safonov, A. Perrin, D. Konstan, J.A. 1997 The neighborhood viewer: a paradigm for exploring image databases ACM SIGCHI Conference on Human Factors in Computing Systems Atlanta, Georgia Association for Computing Machinery 299-300 03/1997 0-89791-926-2 brain neighborhood viewer, browsing, image databases, multi-resolution images, scientific visualization <p>The Brain Neighborhood Viewer is a tool developed to help neuroscientists explore massive databases of brain images. The viewer implements an interface paradigm based on stacks of 2D images that are "yoked together" to provide a common coordinate system. When a user navigates in an image stack, all yoked stacks are updated to display the same location, which we call a brain neighborhood. Experience with the neighborhood suggests that this interface is useful for neuroscience research.</p> 0 Johnson, D, Lilja, D., Riedl, J. and Anderson, J. 1997 Low-cost, high-performance barrier synchronization on networks of workstations Journal of Parallel Distributed Computing 40 1 131-137 01/1997 <p>Circulating active barrier (CAB) is a new low-cost, high-performance hardware mechanism for synchronizing multiple processing elements (PEs) in networks of workstations at fine-grained programmed barriers. CAB is significantly less complex than other hardware barrier synchronization mechanisms with equivalent performance, using only a single conductor, such as a wire or copper run on a printed-circuit board, to circulate barrier packets between PEs. When a PE checks in at a barrier, the CAB hardware will decrement the count associated with that barrier in a bit-serial fashion as a barrier packet passes through, and then will monitor the packets until all PEs have checked in at the barrier. The ring has no clocked sequential logic in the serial loop. A cluster controller (CC) generates packets for active barriers, removes packets when no longer needed, and resets counters when all PEs have seen the zero-count. A hierarchy of PEs can be achieved by connecting the CCs in intercluster rings. When using conservative timing assumptions, the expected synchronization times with optimal clustering are shown to be under 1 μs for as many as 4096 PEs in multiprocessor workstations or 1024 single-processor workstations. The ideal number of clusters for a two-dimensional hierarchy ofNPEs is shown to be [N(D+G)/(I+G)]1/2, whereGis the gate propagation delay,Dis the inter-PE delay, andIis the intercluster transmission time. CAB allows rapid, contention-free check-in and proceed-from- barrier and is applicable to a wide variety of system architectures and topologies.</p> 3 Stein, M. Redl, J. Harner, S. Mashayekhi, V. 1997 A case study with distributed, asynchronous software inspection Proceedings of the International Conference on Software Engineering Boston, MA, USA 3 Claypool, M. and Riedl, J. 1996 A quality planning model for distributed multimedia in the virtual cockpit 4th ACM International Conference on Multimedia Boston, MA Association for Computing Machinery 253-264 11/1996 0-89791-871-1 communications, networking, vod applications <p>Tomorrow's multimedia applications will stress all parts of a computer system. To determine the computer resources needed to meet application demands we have developed a new capacity planning model that is based on application quality as perceived by the user. We have applied our model to a Distributed Interactive Simulation flight simulator called the Virtual Cockpit. We investigate the quality of the Virtual Cockpit on existing networks and processors and predict the effects of high-speed networks and high-performance processors on Virtual Cockpit quality. We find processor performance is the current bottleneck in application quality for the Virtual Cockpit, but that higher-speed networks, such as ATM, will be needed to meet network requirements after two to three generations of processor improvement.</p> 3 Hill, W.C. Terveen, L.G. 1996 Using Frequency-of-Mention in Public Conversations for Social Filtering CSCW 1996 Boston, MA ACM Press 106-112 11/1996 <p>We report on an investigation of using Usenet newsgroups for social filtering of Web resources. Our main empirical results are: (1) for the period of May â96 to Jul â96, about<br />23% of Usenet news messages mention Web resources, (2) 19% of resource mentions are recommendations (as opposed, e.g., to home pages), (3) we canâ兮utomatically recognize recommendations with at least 90% accuracy, and (4) in some newsgroups, certain resources are mentioned significantly more frequently than others and thus appear to play a central role for that community. We have created a Web site that summarizes the most frequently and recently mentioned Web resources for 1400 newsgroups. Keywords: Human-computer interaction, human interface, computer-supported cooperative work, organizational computing, social filtering, collaborative filtering, browsing, resource discovery, World Wide Web, Usenet, netnews.</p> http://www-users.cs.umn.edu/%7Eterveen/papers/cscw96.pdf 3 Chi, E.H., Riedl, J., Shoop, E., Carlis, J.V., Retzel, E., Barry, P. 1996 Flexible information visualization of multivariate data from biological sequence similarity searches Yagel, R., Nielson, G.M. 7th Conference on Visualization San Francisco, CA IEEE Computer Society Press 133-140 10/1996 0-89791-864-9 information visualization, biomedical visualization, multimodal and multidimensional visualization, applications of visualization <p>Information visualization faces challenges presented by the need to represent abstract data and the relationships within the data. Previously, we presented a system for visualizing similarities between a single DNA sequence and a large database of other DNA sequences (E.H. Chi et al., 1995). Similarity algorithms generate similarity information in textual reports that can be hundreds or thousands of pages long. Our original system visualized the most important variables from these reports. However, the biologists we work with found this system so useful they requested visual representations of other variables. We present an enhanced system for interactive exploration of this multivariate data. We identify a larger set of useful variables in the information space. The new system involves more variables, so it focuses on exploring subsets of the data. We present an interactive system allowing mapping of different variables to different axes, incorporating animation using a time axis, and providing tools for viewing subsets of the data. Detail-on-demand is preserved by hyperlinks to the analysis reports. We present three case studies illustrating the use of these techniques. The combined technique of applying a time axis with a 3D scatter plot and query filters to visualization of biological sequence similarity data is both powerful and novel.</p> 3 Safonov, A., Perrin, D., Konstan, J., Carlis, J., Riedl, J. and Elde, R. 1996 Lessons from the neighborhood viewer: building innovative collaborative applications in Tcl and Tk 4th Conference on USENIX Tcl/Tk Workshop Monterey, CA USENIX Association 4 22 07/1996 <p>This paper discusses the development in Tk of a collaborative browser for scientific image databases. The browser, known as the "neighborhood viewer," allows groups of neuroscientists to explore systematically a large collection of brain images. The paper discusses the application, its development, and a set of lessons learned during development. In particular, it shows how constraints and distributed constraints simplified development, discusses the implementation of a wavelet-based image format, and draws lessons about engineering experimental, evolving systems.</p> 3 Terveen, L. Murray, L. 1996 Helping users program their personal agents M.J. Tauber SGICHI Conference on Human Factors in Computing Systems: Common Ground Vancouver, British Columbia, Canada Association for Computing Machinery 355-361 04/1996 0-89791-777-4 agents, end-user programming, intelligent systems <p>Links between web sites can be seen as evidence of a type of <em>emergent collaboration</em> among web site authors. We report here on an empirical investigation into emergent collaboration. We developed a webcrawling algorithm and tested its performance on topics volunteered by 30 subjects. our findings include that a) some topics exhibit emergent collaboration, some do not. The presence of commercial sites reduces collaboration. b) when sites are linked with other sites, they tend to group into one large, tightly connected component. c) connectivity can serve as the basis for collaborative filtering. Human experts rate connected sites as significantly more relevant and of higher quality.</p> 0 Schnepf, J. Konstan, J.A. Hung-Chang, Du D. 1996 Doing FLIPS: flexible interactive presentation synchronization IEEE Journal on Selected Areas in Communications 14 1 114-125 01/1996 5185331 computer animation, digital simulation, education, multimedia systems, simulation, synchronisation <p>Multimedia presentation technology has enormous potential for a myriad of applications including academic classrooms, industrial training, and business presentations. As presentation technology advances, it is possible to incorporate a wider range of media including variable duration media such as simulations and animations. At the same time, users are able to take more control over presentations by controlling the rate and selection of media being played. To make full use of these advances, multimedia systems must support flexible presentations that incorporate many variations in the way they are played. This paper identifies three requirements for flexible presentations and derives four requirements for synchronization of flexible presentations. The paper presents flexible interactive presentation synchronization (FLIPS), a model for specifying coarse synchronization for flexible presentations. FLIPS supports a wide range of temporal synchronization specifications. It also provides algorithms for attaining a consistent and coherent presentation state in response to user interaction (e.g. skipping to a different slide or selection) and other state-changing events. Applications of the FLIPS model are discussed</p> 3 Miller, B. and Riedl, J. 1996 A hands-on introduction to collaborative filtering (tutorial session) 1996 ACM Conference on Computer Supported Cooperative Work Boston, MA <p>Goals and content: The moring session will introduce the concepts of information filtering, develop a taxonomy of the techniques used, and take a detailed look at present and historical applications of collaborative filtering technology. The afternoon session will investigate design issues including algorithms for making recommendations, obtaining user ratings, privacy, communications, and data storage.</p> 7 Bickhard, M.H. Terveen, L. 1996 Foundational Issues in Artificial Intelligence and Cognitive Science: Impasse and Solutions Elsevier Science, Inc. 1 3 Chi, E.H. Riedl, J. Shoop, E. Carlis, J. Retzel, E. Barry, P. 1996 Flexible information visualization of multivariate data from biological sequence similarity searches IEEE Conference on Visualization San Francisco, CA IEEE CS 133-140 31 Konstan, J.A. Bharat, K. 1996 Integrating personal and community recommendations in collaborative filtering M.S. Ackerman Boston, MA Association for Computing Machinery 1996 ACM Conference on Computer Supported Cooperative Work (CSCW) 0-89791-765-0 human factors <p>This full-day workshop will bring together researchers and practitioners to explore techniques for integrating personal and community recommendations into CSCW systems. Personal recommendations are tailored to an individual user, while community recommendations reflect the values or tastes of a broader community of users.</p> 3 Smith, B.C. Rowe, L.A. Konstan, J.A. Patel, K.D. 1996 The Berkeley continuous media toolkit Fourth ACM International Conference on Multimedia Boston, MA Association for Computing Machinery 451-452 0-89791-871-1 continuous media, distributed multimedia, multimedia toolkits and frameworks, design, security 3 Herlocker, J.L. Konstan, J.A. 1995 Commands as media: design and implementation of a command stream Third ACM international Conference on Multimedia San Francisco, CA Association for Computing Machinery 155-165 11/1995 0-89791-751-0 TclStream, command stream, commands, multimedia presentations, reversibilty, design, management, performance, theory 0 Terveen, L. Stolze, M. Hill, W. 1995 From "model world" to "magic world": making graphical objects the medium for intelligent design assistance - a CHI '95 workshop From "model world" to "magic world": making graphical objects the medium for intelligent design assistance - a CHI '95 workshopFrom "model world" to "magic world": making graphical objects the medium for intelligent design assistance - a CHI '95 workshop 27 4 31-34 10/1995 Design, Human Factors, Management 3 Chi, E.H., Barry, P., Shoop, E., Carlis, J.V., Retzel, E. and Riedl, J. 1995 Visualization of Biological Sequence Similarity Search Results 6th Conference on Visualization Washington, D.C. IEEE Computer Society 44-51 10/1995 0-8186-7187-4 <p>Biological sequence similarity analysis presents visualization challenges, primarily because of the massive amounts of discrete, multi dimensional data. Genomic data generated by molecular biologists is analyzed by algorithms that search for similarity to known sequences in large genomic databases. The output from these algorithms can be several thousand pages of text, and is difficult to analyze because of its length and complexity. We developed and implemented a novel graphical representation for sequence similarity search results, which visually reveals features that are difficult to find in textual reports. The method opens new possibilities in the interpretation of this discrete, multidimensional data by enabling interactive investigation of the graphical representation.</p> 3 Shoop, E., Chi, E., Carlis, J., Bieganski, P., Riedl, J., Dalton, N., Newman, T. and Retzel, E. 1995 Implementation and testing of an automated EST processing and similarity analysis system 28th Hawaii international Conference on System Sciences Wailea, HI IEEE Computer Society 5 52-61 1/1995 0-8186-6921-7 4875342 <p>Expressed sequence tag (EST) sequencing projects are being undertaken in an effort to identify the function of as many genes as possible from entire genomes. Putative function can be determined by analyzing the similarity of the ESTs to sequences in the public databases. We are involved in a long-term project to research and develop database technology to store and analyze ESTs for Arabidopsis thaliana. The massive amounts of ESTs being produced through automated sequencing technologies necessitates the automated processing and similarity analysis of the ESTs. This paper describes a complete software system that takes ESTs from a sequencing machine, analyzes them for quality, and searches in public databases of previously known sequences. Automating the processing and analysis of the several thousand ESTs produced to date by the Michigan State University, Arabidopsis cDNA Sequencing Project has improved the quality of the EST data and the speed at which ESTs can be entered in the public databases</p> 3 Johnson, D. Lilja, D. Riedl, J. 1995 A circulating active barrier synchronization mechanism International Conference on Parallel Processing Oconomowoc, WI 08/1995 3 Iyengar, S. Konstan, J.A. 1995 TclProp: a data-propogation formula manager for Tcl and Tk 3rd Annual USENIX Workshop on Tcl/Tk Toronto, CN USENIX Association 3 3 07/1995 design, languages <p>TclProp is a data propagation formula manager for Tcl and Tk. It supports and enforces one-way declarative relationships among variables. If, for example, we enter the formula A = B + C, whenever B or C changes, A is also updated to reflect the new sum. TclProp also supports triggers -- code to be executed when one of a set of variables changes. And, TclProp includes a mechanism for linking variables to object attributes (e.g., the enabled/disabled status of a button) so these attributes can be used in formulas and triggers. This paper presents an example of how data propagation formulas can simplify programming and presents the design and implementation of TclProp 1.0, an implementation built in Tcl.</p> 3 Herlocker, J.L. Konstan, J.A. 1995 Tcl commands as media in a distributed multimedia toolkit 3rd Annual USENIX Workshop on Tcl/Tk Toronto, Canada USENIX Association 3 22 07/1995 design, languages <p>This paper discusses the design and implementation of a command stream based on Tcl. A command stream is a series of arbitrary commands that can be tightly synchronized with other media in a distributed multimedia presentation. In TclStream, we represent an arbitrary command as a collection of fragments of Tcl code. The command stream medium supports the standard manipulation functions of multimedia environments: reverse, fast-forward, random access, and variable speed. The ability to specify arbitrary actions, combined with fine playback control, make TclStream an extremely flexible and powerful presentation medium.</p> 0 Claypool, M. Riedl, J. Carlis, J. Wilcox, G. Eide, R. Retzel, E. Georgopoulos, A. Pardo, J. Ugurbil, K. Miller, B. Honda, C. 1995 Network requirements for 3-D flying in a zoomable brain database IEEE Journal on Selected Areas in Communications 13 5 816-827 06/1995 4988401 biomedical NMR, brain, computer networks, data compression, data visualisation, image coding, medical image processing, neurophysiology, visual databases <p>In laboratories around the world, neuroscientists from diverse disciplines are exploring various aspects of brain structure. Because of the size of the domain, neuroscientists must specialize, making it difficult to fit results together, causing some research efforts to be duplicated because of lack of sharing of information. The authors have begun a long-term project to build a neuroscience research database for brain structure. One aspect of the database is the ability to visualize high-quality, high-resolution micrographs montaged together into 3-D structures as they were in the living brain. As demonstrated in this paper's analysis, realistic presentation of these visualizations across computer networks will stress current and proposed gigabit networks. Image compression can reduce network loads, but wide-spread use of the visualizations will still require networks capable of sustaining terabits per second of throughput</p> 3 Terveen, L. Papavero, E. Tuomenoksa, M. 1995 DynaDesigner: a tool for rapid design and deployment of device-independent interactive services Computer-Human Interaction Conference Denver, CO Association for Computing Machinery 29-30 05/1995 0-89791-755-3 consumer systems, device-independent design, end user programming, service creation tools, design, human factors <p><em>DynaDesigner</em> is a tool for creating, testing, and deploying interactive services to be delivered on devices such as telephones, TVs and PCs. A key feature is that it supports <em>device-independent</em> service design - a service is designed once, independent of any particular device. This eases the design and maintenance task for service providers and makes services easier for consumers to use, since they are consistent across devices. DynaDesigner has been used to design and deploy many services. With DynaDesigner, services can be designed and deployed in hours.</p> 3 Mashayekhi, V. Maley, M. Riedl, J. 1995 User recovery of audio operations International Conference on Multimedia Computing and Systems Washington, D.C. 05/1995 0-8186-7105-X 4974221 audio systems, multimedia computing, system recovery, user interfaces <p>Computer interfaces that support user recovery can radically alter a user's interaction style. Users can explore alternatives freely, secure in the knowledge that they can undo actions and restore previous states if necessary. A text-editor, like EMACS, where users can restore the state of an editing session to a correct previous state, is an example of such a system. Editors for textual, graphical, and many other media types commonly support user recovery. Support for and understanding of recovery in applications that use audio is not as widespread. Audio is characterized by its large volume, lack of easy indexing, and difficulty in defining inverse operations. We present a theoretical model of recovery for audio operations to help user interface designers and implementers. Our model maps an audio operation to a recovery policy and then the recovery policy to a recovery mechanism. The model uses a classification of audio operations that aids in choosing applicable recovery policies</p> 0 Terveen, L.G. Selfridge, P.G. Long, M.D. 1995 Living design memory: framework, implementation, lessons learned Human-Computer Interaction 10 1 1-37 03/1995 design, human factors, management <p>We identify an important type of software design knowledge that we call community-specific folklore and discuss problems with current approaches to managing it. We developed a general framework for a living design memory, built a design memory tool, and deployed the tool in a large software development organization. The tool effectively disseminates knowledge relevant to local software design practice. It is embedded in the organizational process to help ensure that its knowledge evolves as necessary. This work illustrates important lessons in building knowledge management systems, integrating novel technology into organizational practice, and carrying out research-development partnerships.</p> 3 Johnson, D., Lilja, D. and Riedl, J. 1994 A Distributed Hardware Mechanism for Process Synchronization on Shared-Bus Multiprocessors 1994 international Conference on Parallel Processing North Carolina State University 268-275 8/1994 0-8493-2493-9 <p>Several techniques have been used to reduce the performance impact of process synchronization in fine-grained multiprocessor systems. These existing techniques tend to have long synchronization times or high shared-bus use, or they require complex and expensive hardware. A new technique is presented that uses distributed hardware locking queues to reduce both contention and latency to the minimum values that can be obtained using a shared-bus. This technique is shown to require at most two shared-bus transactions, with one transaction being typical. The latency for process continuation after obtaining a lock is reduced to near zero. Barrier synchronization using this distributed mechanism requires only one shared-bus transaction per processor involved in the barrier. This new technique is scalable and applicable to both new architectures and to existing systems, and is less complex than other hardware solutions.</p> 0 Schnepf, J. Mashayekhi, V. Riedl, J. Du, D. 1994 Closing the gap in distance learning: Computer supported, participative, media-rich education Educational Technology Review 1994 3 Mashayekhi, V., Fuelner, C. and Riedl, J. 1994 CAIS: collaborative asynchronous inspection of software 2nd ACM SIGSOFT Symposium on Foundations of Software Engineering New Orleans, LA Association for Computing Machinery 21-34 12/1994 0-89791-691-3 concurrent software engineering, asynchony, software inspection, computer-supported cooperative work (CSCW), collaboration, notification <p>Many software engineering tasks have a synchronous component that requires the participants to assemble together at the same time and place. This approach is expensive in terms of traveling, scheduling and human resources. Existing computer tools mitigate these constraints by adding structure to the meeting, providing on-line document support, and distributing the participants over geographic boundaries. The constraint remains, however, that all participants participate at the same timeWe propose relaxing the time constraint in software engineering tasks to resolve issues non-concurrently, in effect reducing (and in some cases eliminating) the need for the synchronous meeting. We hypothesize that support for asynchrony will enable software engineering teams to work together as effectively in different times as in same time.We have chosen software inspection as our candidate software engineering task because it is well-understood, highly-structured, and widely-practiced. We have designed and developed a Collaborative Asynchronous Inspection of Software (CAIS) meeting prototype that supports the meeting part of inspection. CAIS allows participants to effectively "meet" even when separated by time zones and working schedules. We have conducted a pilot study comparing the manual and CAIS meetings and present our results and lessons learned.</p> 0 Prakash, A. and Riedl, J. 1994 Distributed systems, multimedia and infrastructure support in CSCW SIGOIS Bull 15 2 18-58 12/1994 0 Konstan, J.A. 1994 State problems in programming human-controlled devices IEEE Transactions on Consumer Electronics 40 4 812-820 11/1994 4867022 control program, control sequences, design strategies, device interfaces, home audio, home electronics devices, home video, human-controlled devices, interactive human control, personal digital assistants, programmable remote controls, programmed control <p>Many consumer goods are complicated enough to benefit from programmed control. Today's home electronics devices support a wide range of options and controls. At the same time, personal digital assistants and programmable remote controls are now capable of learning and generating control sequences to control a wide range of devices. Unfortunately, most device interfaces are designed for interactive human control rather than programmed control. The paper analyzes state-based obstacles to programming devices designed for interactive human control. It develops a theory of statelock, a condition in which a control program is unable to synchronize with the state machine underlying the controlled device. The paper also presents design strategies to avoid statelock and applies these strategies to the home audio/video and telephone autodialer domains</p> 3 Resnick, P. Iacovou, N. Sushak, M. Bergstrom, P. Riedl, J. 1994 GroupLens: An open architecture for collaborative filtering of netnews 1994 ACM Conference on Computer Supported Collaborative Work Conference Chapel Hill, NC Association of Computing Machinery 175-186 10/1994 <p class="abstract">Collaborative filters help people make choices based on the opinions of other people. GroupLens is a system for collaborative filtering of netnews, to help people find articles they will like in the huge stream of available articles. News reader clients display predicted scores and make it easy for users to rate articles after they read them. Rating servers, called Better Bit Bureaus, gather and disseminate the ratings. The rating servers predict scores based on the heuristic that people who agreed in the past will probably agree again. Users can protect their privacy by entering ratings under pseudonyms, without reducing the effectiveness of the score prediction. The entire architecture is open: alternative software for news clients and Better Bit Bureaus can be developed independently and can interoperate with the components we have developed.</p> 3 Terveen, L.G. Selfridge, P.G. 1994 Intelligent assistance for software construction: a case study Knowledge-Based Software Engineering Conference Monterey, CA IEEE 14-21 09/1994 0-8186-6380-4 4785232 computer graphics, knowledge based systems, service industries, software reusability, telecommunication services, visual programming <p>An important type of software design task involves constructing software artifacts from existing components. Major user tasks are locating relevant components, reusing existing artifacts, and ensuring that the artifact is complete and consistent. We developed a prototype knowledge-based graphical system that supports these tasks. It delivers information to users by adding and deleting graphical objects and changing their appearance. The system was developed in cooperation with an AT&amp;T product organization. They created a production version of the tool for assembling sets of telecommunications features for customers. The tool lets users work at a high level and eases problems with locating relevant features, reusing feature lists, and maintaining consistency</p> 3 Carlis, J. Riedl, J. Georgopoulos, A. Wilcox, G., Elde, R. Pardo, J.H. Ugurbil, K. Retzel, E. Maguire, J. Miller, B. Claypool, M. Brelje, T. Honda, C. 1994 A zoomable DBMS for brain structure, function and behavior International Conference on Applications of Databases 06/1994 3 Bieganski, P. Riedl, J. Carlis, J.V. Retzel, E.F. 1994 Generalized suffix trees for biological sequence data: applications and implementation 27th International Conference on System Sciences Wailea, HI 5 9-18 05/1994 0-8186-5090-7 4687942 <p>This paper addresses applications of suffix trees and generalized suffix trees (GSTs) to biological sequence data analysis. We define a basic set of suffix trees and GST operations needed to support sequence data analysis. While those definitions are straightforward, the construction and manipulation of disk-based GST structures for large volumes of sequence data requires intricate design. GST processing is fast because the structure is content addressable, supporting efficient searches for all sequences that contain particular subsequences. Instead of laboriously searching sequences stored as arrays, we search by walking down the tree. We present a new GST-based sequence alignment algorithm, called GESTALT. GESTALT finds all exact matches in parallel, and uses best-first search to extend them to produce alignments. Our implementation experiences with applications using GST structures for sequence analysis lead us to conclude that GSTs are valuable tools for analyzing biological sequence data</p> 3 Claypool, M. Riedl, J. 1994 Silence is golden? the effects of silence deletion on the CPU load of an audio conference International Conference on Multimedia Computing and Systems Boston, MA 05/1994 0-8186-5530-5 4711442 multimedia systems, teleconferencing <p>This paper seeks to identify, improvements that reduce audioconference CPU load. A major contribution is the comparison of the performance benefits of five potential audioconference improvements: faster CPU, faster communication, better compression, digital signal processing (DSP) hardware, and silence deletion. To compare audioconference CPU load, we develop a model that identifies components of a typical audioconference. We hypothesize that silence deletion will improve the scalability of audio more than any of the above four improvements. We parameterize our model with measurements of the actual component performance. Overall, we find audioconference CPU loads with silence deletion scale better than audioconference CPU loads with any of the other four improvements. Techniques based on DSP hardware alone do not scale as well as silence deletion alone. However, DSP based silence deletion and compression together scale better than any other technique. These results hold even when using compression and even for ten times faster processors, networks and DSP hardware</p> 3 Hill, W. Terveen, L. 1994 New uses and abuses of interaction history: help form the research agenda C. Plaisant ACM SIGCHI Conference on Human Factors in Computing Systems Boston, MA Association for Computing Machinery 472 04/1994 0-89791-651-4 Interaction history, design capture, interface agents, digital audio, user modeling, active badges, usability, privacy, ethics, law <p>Recording human-computer interaction for the purposes of reusing commands, undoing actions, recovering from crashes, constructing keyboard macros, and observing users has been with us since the earliest command shells and text editors. For much of that time it remained a sleepy "back water" technology area except for a continuing increase in work-monitoring and associated incidents breeching user privacy. However, with the drastic fall of costs for digital storage, processing and telecommunications, all that is now rapidly changing. Digital records of activity are common at work, market-place and home. While new interaction history techniques such as design capture, automatic change bars, readwear, interface agents, digital audio recording, hot lists, version management, viewer histories, automatic biography, usability studies, active badges, wireless personal communicators, position-sensing and caller-id are enriching the experience of interfaces, the same techniques are enabling new and more invasive abuses. This one-day interdisciplinary workshop will gather 20 practitioners and researchers from the fields of human-computer interaction design, research, ethics and law to produce their "Top Ten" list of research questions concerning uses and abuses of interaction history for the CHI community to address in the coming years. There will be no presentations, but homework will be collected and redistributed via email prior to the workshop. The day will blend open discussions with directed small-group works.</p> 0 Bhargava, B. Mafla, E. Riedl, J. 1993 Experimental facility for kernal extensions International Journal of System Integration 3 5-21 1993 3 Terveen, L.G. 1993 Interface support for data archeology B. Bhargava, T. Finn, Y. Yesha 2nd international Conference on Information and Knowledge Management (CIKM '93) Washington, D.C. Association for Computing Machinery 356-363 11/1993 0-89791-626-3 knowledge discovery, interactive data exploration, marketing, knowledge representation, reuse <p>I describe the <em>IMACS</em> interface, which supports a type of interactive data exploration task called data archaeology. The interface facilitates users in performing this task using three key design principles: (1) combine power and ease of use, (2) provide direct support for integrated, iterative data exploration, and (3) assist users in managing their work over time. I show how these principles are relevant in the data archeology task, describe how knowledge representation technology provides a foundation for an adequate support system, and illustrate in detail how the interface offers powerful support for data archaeology.</p> 0 Mashayekhi, V., Drake, J.M., Tsai, W. T. and Riedl, J. 1993 Distributed collaborative software inspection Software, IEEE 10 5 66-75 09/1993 4536440 software metrics, software tools <p>The Collaborative Software Inspection (CSI) tool, which provides a distributed, structured environment for performing inspections on all software-development products, including specifications, designs, code, and test cases, is described. The inspection environment lets geographically distributed inspection participants meet with people in other cities through workstations at their desks. The current version of all material is accessible online. Inspection products are created online, so secondary data entry to permanent records is not necessary. The inspection information is also available for review and metrics collection. To assess the effectiveness of inspection in the distributive collaborative environment and compare it with face-to-face meetings, a case-study approach with replication logic is presented</p> 0 Riedl, J., Mashayekhi, V., Schnepf, J., Claypool, M. and Frankowski, D. 1993 SuiteSound: A System for Distributed Collaborative Multimedia IEEE Transactions on Knowledge and Data Engineering 5 4 600-610 08/1993 4551263 collaborative systems, continuous media, digital audio, distributed systems, Ethernet, multimedia, silence deletion, silence detection, teleconferencing <p>SuiteSound, a programming environment with integrated support for multimedia, is discussed. SuiteSound is built in the Suite object-based system on a conventional UNIX operating system. SuiteSound objects incorporate multimedia by creating flows and filters. Flows are streams of multimedia data moving through a sequence of objects. They bridge the gap between objects representing the state of an entity at a discrete point in time and space and continuous media such as live audio or video. Filters are intermediate objects between the source and destination of a flow. They take flow as input, perform one of several operations such as multiplex-in, multiplex-out, gain control, or silence deletion on it, and send the resulting flow to its destination. In effect, they provide a virtual device interface for the application programmer that is uniform and independent of any physical device. The design and implementation of SuiteSound on the Sun SparcStation are described. Experiments performed to determine the network and CPU load of the sound tool are reviewed</p> 3 Terveen, L.G. Selfridge, P.G. Long, M.D. 1993 From "folklore" to "living design memory" ACM SIGCHI Conference on Human Factors in Computing Systems Amsterdam, The Netherlands Association for Computing Machinery 04/1993 knowledge representation, organizational design, organizational interfaces, software productivity, design, document, management <p>We identify an important type of software design knowledge that we call community specific folklore and show problems with current approaches to managing it. We built a tool that serves as a living design memory for a large software development organization. The tool delivers knowledge to developers effectively and is embedded in organizational practice to ensure that the knowledge it contains evolves as necessary. This work illustrates important lessons in building knowledge management systems, integrating novel technology into organizational practice, and managing research-development partnerships.</p> 3 Shoop, E., Srivastava, J., Bieganski, P., Riedl, J. and Retzel, E. 1993 An object-oriented genetics information system Deaton, E., George, K.M., Berghel, H and Hedrick, G. 1993 ACM/SIGAPP Symposium on Applied Computng: States of the Art and Practice Indianapolis, IN Association for Computing Machinery 641-651 02/1993 0-89791-567-4 computational, molecular biology, genome sequencing, object-oriented database, graphical user interface, suffix tree <p>Sequence data is being produced by genomic sequencing laboratories at ever-increasing rates, making it impossible for individual researchers to keep track of all the new data that might affect their research. Computer systems are needed so that researchers can access this data. The systems must support high-level interfaces that communicate in the language of the researchers, database systems that guarantee availability and consistency of the data, and powerful search systems that rapidly scan for similarities between sequences. We have developed a prototype system that includes a graphical user interface, an object-oriented database management system, and high-performance similarity search algorithms. The prototype has the potential to increase researchersâ productivity by automating ermy of amotated sequence fragments as they are produced by sequencing machines, storing the fragment in the database, and automatically producing and displaying similarity search results of new sequences against the large public sequence databases GenBank and PIR. This paper describes the prototype, discusses the its of object oriented databases for complex and changing sequence data and presents an object-oriented schema for genetic information. Graphical tools for annotating sequences, storing them in the database, automating similarity searches, and viewing similarity search results are presented. A new suffix tree- based data structure that supports rapid similarity searches on sequence data is introduced. Finally, future plans for the system are discussed.</p> 0 Dewan, P. and Riedl, J. 1993 Toward Computer-Supported Concurrent Software Engineering Computer 26 1 17-27 01/1993 4391917 concurrent engineering, programming environments, software engineering <p>An experimental software engineering environment called the flexible environment for collaborative software engineering (Flecse), which supports concurrent software engineering, is discussed. Flecse features tools designed to surmount collaboration problems that software engineers are increasingly encountering. The implementation of five important themes of concurrent software engineering in Flecse tools, concepts, life cycles, integration, and sharing, is examined</p> 2 Konstan, J.A. 1993 An Event-Based Architecture for Graphical User Interface Toolkits University of California, Berkeley 3 Selfridge, P.G. Terveen, L.G. Long, M.D. 1992 Managing design knowledge to provide assistance to large-scale software development Knowledge-Based Software Engineering Conference 0-8186-2880-4 4523900 knowledge based systems, knowledge representation, project management, software engineering <p>The problem is examined of managing design knowledge as a crucial component in a large-scale software development project. The authors explore this design knowledge problem in more detail, describe both technical and nontechnical challenges, discuss the maintenance of such knowledge, and briefly explore the issue of acquisition. A framework is described for providing knowledge-based assistance to software developers. This framework is integrated with and extends an existing design process and exploits that process to address the problem of knowledge maintenance. Then, an implemented design knowledge tool is presented instantiating the framework that gives software developers access to knowledge about a particular error handling mechanism. The organization of the knowledge, the design of the interface, and the status of the implementation are discussed</p> 3 Rowe, L.A. Konstan, J.A. Smith, B.C. Seitz, S. Liu, C. 1991 The PICASSO applications framework 4th Annual ACM Symposium on User interface Software and Technology Hilton Head, South Carolina 95-105 11/1991 0-89791-451-1 Algorithms, Design, Languages 3 Konstan, J.A. Rowe, L.A. 1991 Developing a GUIDE using object-oriented programming Object-Oriented Programming Systems, Languages, and Applications Phoenix, AZ Association for Computing Machinery 75-88 10/1991 ISBN:0-201-55417-8 Algorithms, Design, Languages 0 Bhargava, B., Mafla, E. and Riedl, J. 1991 Communication in the raid distributed database system Computer Networking ISDN Systems 21 2 81-92 03/1991 0-8186-2030-7 3799710 distributed databases, input-output programs <p>The basic functions required from a communication subsystem in order to support a distributed, reliable, reconfigurable, and replicated database environment are identified. These functions include reliable multicast, remote procedure calls (RPCs), inexpensive datagram services, and efficient local interprocess communication (IPC). Data obtained via a series of experiments that measure the performance of several local interprocess communication methods are reported, and a kernel-level multicasting facility, the Raid system running on different network configurations, and a Push multicast program are discussed. Push is a tool that allows measurements to be conducted by supplementing and/or modifying the communication facilities in the operating system kernel while it is running</p> 2 Terveen, L.G. 1991 Person-Computer Cooperation Through Collaborative Manipulation Austin, TX 3 Bhargava, B. Friesen, K. Helal, A. Riedl, J. 1990 Adaptability experiments in the RAID distributed database system Ninth Symposium on Reliable Distributed Systems Huntsville, AL 76-85 10/1990 0-8186-2081-1 3908472 distributed databases, performance evaluation, protocols <p>A series of experiments is being conducted on the RAID distributed database system to study the performance and reliability implications of providing static and dynamic adaptability. The authors' studies of the cost of their adaptable implementation were conducted in the context of the concurrency controller and the replication controller. It is shown that adaptable implementations can be provided at costs comparable to those of special-purpose implementations. The experimentation with dynamic adaptability focuses on concurrency control. It is shown that dynamic adaptability can result in performance benefits and that system reconfiguration can be accomplished dynamically with less cost than stopping the system, performing reconfiguration, and then restarting the system. The authors' examination of the costs of providing greater data availability includes studying the replication control and atomicity control subsystems of RAID. The cost associated with increasing availability in an adaptable scheme of replication control and commit protocols is demonstrated</p> 2 Riedl, J.T. 1990 Adaptable Distributed Transaction Systems Purdue University 0 Bhargava, B. and Riedl, J. 1989 A Model for Adaptable Systems for Transaction Processing IEEE Transactions on Knowledge and Data Engineering 1 4 433-449 12/1989 3598179 concurrency control, distributed processing, fault tolerant computing, transaction processing, adaptability, reconfiguration <p>Adaptability is an essential tool for managing escalating software costs and to build high-reliability, high-performance systems. Algorithmic adaptability, which supports techniques for switching between classes of schedulers in distributed transaction systems, is modeled. RAID, an experimental system implemented to support experimentation in adaptability, is discussed. Adaptability features in RAID, including algorithmic adaptability, fault tolerance, and implementation techniques for an adaptable server-based design, are modeled</p> 0 Bhargava, B. and Riedl, J. 1989 The Raid Distributed Database System IEEE Transactions on Software Engineering 15 6 726-736 06/1989 3421312 computer communications software, concurrency control, distributed databases, software packages, transaction processing <p>Raid, a robust and adaptable distributed database system for transaction processing, is described. Raid is a message-passing system, with server processes on each site. The servers manage concurrent processing, consistent replicated copies during site failures and atomic distributed commitment. A high-level, layered communications package provides a clean, location-independent interface between servers. The latest design of the communications package delivers messages via shared memory in a high-performance configuration in which several servers are linked into a single process. Raid provides the infrastructure to experimentally investigate various methods for supporting reliable distributed transaction processing. Measurements on transaction processing time and server CPU time are presented. Data and conclusions of experiments in three categories are also presented: communications software, consistent replicated copy control during site failures, and concurrent distributed checkpointing. A software tool for the evaluation of transaction processing algorithms in an operating system kernel is proposed.</p> 3 Bhargava, B.K., Mafla, E., Riedl, J. and Sauder, B. 1989 Implementation and Measurements of Efficient Communication Facilities for Distributed Database Systems Fifth international Conference on Data Engneering IEEE Computer Society 200-207 0-8186-1915-5 3406298 distributed databases, network operating systems, protocols <p>Experimentation with several methods of providing efficient communication facilities for distributed database systems is described. These studies give insight into the delays incurred by applications running on distributed systems. Five different mechanisms for local interprocess communications (two variations with message queues, named pipes, shared memory, and UDP sockets) have been implemented, compared, and analyzed. The most efficient of these is three times as fast as UDP for 1000-byte messages. Kernel-level software multicast and hardware multicast have also been implemented and their performance analyzed. The results show the significant advantage of using these techniques instead of using multiple sends and receives at the user level. The design of a facility that allows the dynamic addition of user-level protocols such as two-phase commit, clock synchronization, etc. to an operating system kernel is presented. The facility is based on a simple stack-based language that provides the functionality and security required</p> 3 Bhargava, B. Riedl, J. 1988 Implementation of RAID Seventh Symposium on Reliable Distributed Systems Columbus, OH 157-166 10/1998 0-8186-0875-7 3328561 concurrency control, distributed processing <p>RAID is a robust and adaptable distributed system for transaction processing. It is a message-passing system, with server processes on each site. A high-level, layered communications package provides a clean, location independent interface between servers. RAID processes concurrent updates and retrievals on multiple sites. The servers manage concurrent processing, consistent replicated copies during site failures or network partitionings, and atomic distributed commitment. The latest version of the communications package is able to deliver messages in a high-performance configuration in which several servers are linked into a single process. RAID provides the infrastructure to investigate experimentally various methods for supporting reliable distributed-transaction processing. Experiments on handling site failure with partial replication, checkpointing, and alternative communications methods have been performed. Measurements on various aspects of RAID transaction processing performance are presented</p> 3 Bhargava, B. Riedl, J. 1988 A model for adaptable systems for transaction processing Fourth International Conference on Data Engineering Los Angeles, CA 40-50 02/1988 0-8186-0827-7 3148558 database theory, distributed databases, protocols <p>A model is presented for an adaptable system that allows online switching of classes of algorithms for database transaction processing. The basic idea is to identify conditions on the state of processing that will maintain consistency during the switch from one class to another. The classes of concurrency control algorithms and the formalism of history for transaction processing and serializability have been used to develop this research. In addition to the formalism, the precise conditions for switching digraph-serializable (DSR) algorithms have been given. This research is being applied to switching network partition protocols (conservative to optimistic), commit protocols, recovery block software, and has led towards the design of an adaptable and reconfigurable distributed database system. An experimental system called RAID has been implemented to test these ideas; it has been noted that adaptability provides for varying performance requirements and deals with failures of sites, transactions, and other components of the system</p> 3 Bhargava, B., Meller, T. and Riedl, J. 1987 Experimental analysis of layered Ethernet software 1987 Fall Joint Computer Conference on Exploring Technology: Today and Tomorrow Dallas, Texas IEEE Computer Society Press 559-568 31 Bhargava, B., Dilley, J. and Riedl, J. 1986 RAID: a robust and adaptable distributed system 2nd Workshop on Making Distributed Systems Work New York, NY Associaton for Computing Machinery <p>There is a need to design distributed systems that are not rigid in their choice of algorithms and that are responsive to faults/failures and performance degradation. To meet this challenge, we formalize and experiment with design principles that allow the implementation of an adaptable distributed system. The strategies for dynamic reconfiguration of the subsystems and determining their impact are being studied via experiments on a prototype system called RAID under development at Purdue University. RAID provides system level support for transaction management in a reliable manner. Other transaction based systems are TABS [SBD*85], ARGUS [LS83], and System R* [LHM*84].The key contribution of RAID is the system level support provided for building transaction based applications. RAID provides support for atomic objects and atomic commitment across a set of sites. It also includes concurrency control mechanisms based on time-stamps that provide a variety of choices of methods spanning from two-phase locking to optimistic methods utilizing the semantics of transactions and the objects accessed by them. In addition RAID has site failure and network partition control algorithms integrated with the rest of concurrent transaction processing and a replicated copy control subsystem.</p>