College guidance platform PossibilityU forms research partnership with GroupLens

White House, Department of Education, and Chronicle of Higher Education highlight our college recommendation software

PossibilityU logo
PossibilityU logo

Two organizations have joined forces to answer important questions related to how search, discovery and recommendation online are affecting student college choices. PossibilityU, an innovative college guidance platform, and GroupLens, a research lab dedicated to recommender systems and online communities in the Department of Computer Science and Engineering at the University of Minnesota, Twin Cities, announced a research partnership to explore how college recommendations online can effect a broad range of student outcomes.

For the past 3 years, PossibilityU’s unique college guidance program has been matching high school students across the country with colleges that match their talents, aspirations and family budgets. The program features guided inquiry, data visualizations and personalized, Netflix-like recommendations—all designed to give students confidence in their choices. PossibilityU also features a blended-learning video curriculum created to help students and parents become better consumers of higher education.

“College recommendations are different from any other project we’ve taken on at GroupLens. This is the kind of project we love—one where research potential is large, multifaceted and one where the outcomes work toward a social good” said Dr. Joseph Konstan, co-director of the GroupLens lab.

At the White House and Department of Education’s Education Datapalooza in 2012 and 2014, PossibilityU was recognized for innovative use of open government education data and highlighted by College Summit students alongside Secretary of Education Arne Duncan. The Chronicle of Higher Education recently featured PossibilityU’s recommender system on its Data blog, and Brock Tibert has a good follow-up article from the perspective of enrollment management. In 2012, the Software and Information Industry Association’s Innovation Incubator awarded PossibilityU national top-10 honors for innovative ideas within small software companies. The educational technology industry has been most active around the college search and recommendation space, and in 2013 PossibilityU won a College Knowledge Challenge grant funded by the Bill & Melinda Gates Foundation in partnership with College Summit and Facebook. PossibilityU was also featured on Boston’s New England Cable News.

PossibilityU uses content-based item similarity metrics calculated using a version of K-nearest-neighbor. While traditional college search engines ask users to enter desired values (e.g., a college with 5,000-9,999 students in California), PossibilityU doesn’t require students to know relevant item features. The system asks for one or several college names, then displays a list of features which are both common to the set of input colleges and are unlikely to arise by chance, given the entire population of American colleges. It then displays other colleges that have properties similar to the calculated set of features. Under the hood, PossibilityU relies on an extensive collection of data from the US Department of Education (the IPEDS system).

Users control recommendations in several ways:

  1. adding and removing colleges from the input set;

  2. removing a feature (e.g., “higher number of students studying engineering”) from the automatically-calculated list of important features;

  3. adding and removing academic fields (e.g., “business, management, and marketing”);

  4. stating a preference to maximize the size and proportion of chosen academic departments or simultaneously maximize the diversity of academic options

“We are thrilled to have Dr. Konstan and the team at the GroupLens deepen the research base that informs our product”, said Betsy Peters, CEO of PossibilityU. “By closely studying how user experience, recommendations, and collaboration between students and their mentors work together online, we’ll be in better position to help scale the work that great college counselors do.”

PossibilityU is actively seeking schools and other student serving organizations interested in participating in this research beginning in Fall 2014. For more information contact betsy.peters@possibilityU.com or call 855-480-9126.

About Me

I’m a first-year Ph.D. student in GroupLens, and I’m working on exploration and matching in domains of high-cost sparse items with strategic user behavior. The college search displays significantly different properties than recommendation in book, movie, or music domains. There are relatively few colleges (about 2000 four-year colleges and 2000 two-year colleges in the United States) which are unevenly geographically distributed; item cost is quite variable and very high; traditional consumption- and memory-based ratings are not possible because students can’t try out and rate many colleges; and a number of other unusual or unique properties make college recommendation an exciting area of research.

As I continue work on college recommendation research at GroupLens, I’d love to hear from you. Drop me a line at @danjarratt or jarratt@cs.umn.edu.

About PossibilityU

PossibilityU is an education company dedicated to helping students become smart consumers of higher education. Our award-winning college guidance platform has been featured at the White House twice and was a winner of the Gates Foundation funded College Knowledge Challenge. Most importantly, it has been helping high school students across the country find the colleges that fit – academically, socially and financially – since 2010. PossibilityU features a Netflix-like recommendation engine, a 10-hour blended learning curriculum, and project management dashboards to help students and their mentors make great choices about choosing and financing college. To learn more about PossibilityU, visit www.PossibilityU.com.

Taking Recommendations Improves Consumption Diversity — A Surprise Result from Exploring the Filter Bubble and MovieLens

This post describes work being presented at WWW 2014, by Tien Nguyen

Those of you following recommender systems have almost certainly heard the debate about filter bubbles.  This concept, perhaps best articulated by Eli Pariser, argues that recommenders have the potential to trap users into increasingly similar content, isolating them from the diversity of content that makes people rich learners.

We decided to test this concept empirically, using longitudinal data from MovieLens.  Specifically, we wanted to answer two questions:

  • Do recommended movies get narrower as users continue to rate movies?

  • Do users consume narrower movies — and if so, is this a consequence of taking recommendations?

What we found surprised us.

(more…)

Tag Genome Dataset Released

Want to know how quirky a particular movie is? Or how to find the most visually appealing movies of all time? Or how to find a movie that is similar to another movie you’ve seen but less big budget and more cerebral?

The tag genome is a data structure that enables you to answer queries such as these. As described in this article, the tag genome encodes how strongly movies exhibit particular properties represented by tags (atmospheric, thought-provoking, realistic, etc.). The tag genome was computed using a machine learning algorithm on user-contributed content including tags, ratings, and textual reviews.

We’re announcing the release of a tag genome dataset, containing the relevance values for 1,128 tags and 9,734 movies. We hope you will explore this dataset and come up with new and creative ways to use it! You can find more details here.

Evaluating MOOC Learning — Experiences from our Recommender Systems Course

Last fall Michael Ekstrand and I co-taught An Introduction to Recommender Systems on Coursera (if you search for the course, you can find the lectures open as part of the course preview).  In offering the course we had three goals:
  • to make a high-quality introductory recommender systems course available to the world
  • to actually experience the MOOC-teaching process, including exploring how elements of the MOOC could be useful in on-campus teaching
  • to study the effectiveness of the MOOC in student learning
To accomplish these goals, we had extensive support from not only videographers and course support staff, but also from learning technology and evaluation experts.  Our first published result of this work is the paper:
Teaching recommender systems at large scale:  Evaluation and lessons learned from a hybrid MOOC (Proceedings of the first ACM Conference on Learning @ Scalehttp://doi.acm.org/10.1145/2556325.2566244.
 
I wanted to share a few key findings and experiences from the paper, but first I should probably say a few things about the MOOC itself. Teaching the course was a wonderful experience, but it was also an incredible amount of work.  We decided to offer a full course — simultaneously offered as a 3-credit graduate course and a free Coursera course.  The course had 14 weeks of content covering recommender systems design, algorithms, and evaluation.  We divided the course into two “tracks,” a comprehensive “programming” track, and a “concepts” track that included everything except the six programming assignments.  Most programming assignments were designed to be completed using our open-source LensKit toolkit.
 
The course attracted a large number of students (over 28,000 of them), but like most large-scale free courses, there were many students who registered and never returned, or who visited for a while and then left.  At the end of the course there were about 2200 students still active.  
We measured everything we could, including knowledge gain (tested using a knowledge test administered before and after the course).  We wanted to answer four key questions:
  • What factors predict student retention?
  • Do students really learn from these courses?
  • What factors predict student learning?
  • How are results different for on-campus enrolled students vs. online students?
Here’s what we found:
  • It is very hard to predict student retention.  Students who intend to complete the course, know more about the topic at the start, and have taken MOOCs before are more likely to finish.  Students taking lots of other MOOCs at the same time are les likely to finish. But all of these factors only explain 6% of the variation in retention. More interestingly, age, sex, language proficiency, and country are not significant predictors of student retention.
  • Student learning is much easier to measure, though only for the subset of students who completed the course and finished the pre-course and post-course knowledge tests.  Among those 262 students, average scores on our assessment increased from 25% to 70%.  Gains were consistent across high-knowledge and low-knowledge students, and between the concepts and programming tracks.  The only positive predictor of learning was effort, measured by the number of written assignments submitted.  Again, age, sex, country, and English proficiency were not significant predictors.
  • We didn’t have enough on-campus students to get statistically significant results, but we did find much higher knowledge gains for on-campus students (67% vs. 58%, normalized knowledge gain).  We also found from student feedback that on-campus students strongly preferred the online course format (they only came to the classroom for help and discussion).  Some students just liked having the freedom to take the course at home on their own schedule, but many cited the benefits of being able to control lecture speed — speeding up when they understood, or slowing down when concepts or vocabulary required them to take time to understand what was said.
We don’t know what will happen with this course next, but stay tuned!

Social Curation in Pinterest: Specialization, Homophily, and Gender

As the third-largest English-language social network behind Facebook and Twitter, Pinterest has surpassed Reddit, Digg, and others to become the world’s most popular social curation site. Despite the popularity of Pinterest, there has been little scientific work examining the strategies of successful Pinterest users. We have been studying Pinterest for the past year and a half, with one paper appearing in CHI 2013 and another one to be presented at CSCW 2014. This post summarizes the highlights of the CSCW paper.

In this work we studied the types of content and behavior that attract attention — namely repins and follows — in Pinterest. We looked at a number of factors, including the diversity of pinned content, homophily (the tendency for similar people to have more social connections), and gender.

Using our dataset of thousands of Pinterest users and millions of pins, we identified a number of factors that correlated with the number of followers a user had: the most powerful correlates were obvious factors like the amount of content the user pinned and the number of other users the user followed. However, topical diversity also played a role:  the more topically diverse one’s set of pins, the more followers one tends to have, but only up to a certain point. So, in other words, the Pinterest user who pins content in many categories – e.g. food/drink, DIY, home decor, travel, etc. – tends to have more followers than the Pinterest user who sticks to a single or small number of categories. However, when the diversity of categories gets too great, the number of followers tends to go down. The figure below shows this relationship in more detail.

(more…)