Blog

I just finished reading "Bounding Rationality to the World" by Todd and Gigerenzer, published in the Journal of Economic Psychology in 2003.  It’s an interesting article with the core theme that the reason we see cognitive limitations in human decision making is that people may be optimized to make decisions in a particular environment.  In particular, this argument suggests that the apparent deficits in human decision making discovered in various economic experiments may be due to those experiments creating environments that do not correspond well to the sorts of environments humans regularly have to make decisions about.

Several of the issues seem to have interesting implications for recommenders, including:

• people are often making decisions in an environment in which information is not free.  collecting more information to make a better decision might lead to an overall decision that is worse, ironically.  (This reminds me of Colin Powell’s talk in which he explained that he was trained to make decisions with 2/3 of the information; waiting for more information would mean that the decision would not be available in time to help during a battle.)  Can a recommender be set to help a person satisfice appropriately?
• how do known stable cognitive illusions like the overconfidence bias and the hard-easy effect relate to recommenders?  Can and should recommenders be tuned to avoid these illusions?
• knowing more sometimes decreases decision performance.  Can this sort of problem be seen in recommenders? 
• one common decision approach is sequential choice, in which a decider must make a decision without having the opportunity to choose a prior choice once it has been passed on.  What would a recommender be like for a sequential choice problem?  Could it improve performance?
• the paper mentions research that suggests that humans benefit during language acquisition from having limited cognitive ability, so higher abilities don’t get in the way while learning the basics.  This feels related to concepts in machine learning, like those that limit the complexity of the function search space.  However, humans over time increase the complexity of their cognitive function.  What does this correspond to in machine learning?

Overall, recommender research that explores the ways recommenders interact with human decision processes would be fascinating.

Paul Graham is one of my favorite tech essayists.  His essays on startup companies, Lisp, and — especially — American high schools are smart, funny, and insightful.  On these topics, even when I think he's wrong, I learn something.

However, he also writes periodically about economics, about which he seems to know a lot less.  His most recent foray is an article on unions.  Boiled down, the argument is that unions were successful a few decades ago because the manufacturing industries they were organizing were growth industries, and because growth industries don't mind overpaying for stuff, since time to market is more important that expense.  I know enough to know I don't know the economics behind why unions are not doing well — but this hypothesis is both naive and lacks predictive power.  It attempts to explain why unions are not doing well by explaining why *manufacturing* unions are not doing well.  What about all the other types of unions that are not doing well?  Why aren't growth industries unionizing so the workers can get some of the wealth?  I don't know the answers … but at least I know I don't know :).  

An interesting article discusses the problem in the reddit community that the community has wide agreement on a variety of issues, and that therefore only articles with the "correct" viewpoint on those issues get many votes.  (Similar problems apply to the other social news sites like slashdot and digg.)

The off-the-cuff response from the recommender community might be "let's solve the problem by creating personalized reddit".  In this world, everyone would read the articles he or she was most interested in, creating many overlapping communities of interest. A concern with this approach is that social psychology suggests that by making it easy for people to only talk with others with whom they agree, we would be creating a world that would emphasize our differences, amplifying them over time, balkanizing the community of news readers.  For instance, all of the atheists would only read articles that support their views, and would become increasingly resistant to theist views — and eventually unable to even find common ground for discussion with theists.  

An alternative would be to find a way to create a community news reader that would simultaneously support personalization and encourage the sharing of opinions. What would such a news reader look like? How would it use recommenders in a novel way?