The other night I was speaking with Courtney outside of the 12th Street Pub in Lincoln and the subject of information filtering came up, specifically with regards to Twitter and Google Reader. Since “this is what Twitter should do” posts are all the rage right now, I figured I would share some thoughts on the topic.

Courtney brought up the insightful and for some reason not obvious (to me at least) point that all filtering in 1 to many services like this is done on the receiving end, the “pull” end of the process. We talked about why this is a problem and how it could be addressed.

Generally, in a world of “information overload” there needs to be a better way for producers to selectively target and filter the information they send out. Not all of this burden should lie on the information consumer. 

The Problem

The Problem

Let’s say that our friend Alfred the producer has something to share. He pushes out a message that gets propagated to every one of his “subscribers.” These subscribers might have software that allows them to filter messages they receive based on keywords, where it comes from, etc. However all of this filtering takes place on the receiving end. Alfred has little to no control over where his messages get sent beyond possibly the ability to block individual subscribers.

For a concrete example take twitter. Everything you tweet gets shared with all of your followers. This is problematic when you have a decent number of followers across a variety of aspects of your life. If you are tweeting about Nebraska Football, for example, then your co-workers or family in Arizona probably aren’t going to care and this content becomes noise to them. This may seem like a trivial example, but when multiplied 10s or 100s of times it can quickly become overwhelming.

The Solution

The Solution

Client-side filters and groups can only do so much to mitigate this signal to noise problem. It is likely that in most cases the producer understands who the message is actually relevant to and is more than able to filter out uninterested subscribers before sending the message. This reduces the amount of irrelevant content that subscribers have to deal with. Unfortunately this capability is basically non-existent on a message by message basis in most applications.

The ideal solution allows a producer to categorize her subscribers into custom groups, and then target each message at particular individuals or groups of subscribers. In this way she will be able to push out messages that are only relevant to a subset of subscribers to those subscribers only.

Current Implementations

The different applications where this is an issue have differing degrees of support for either push and/or group filtering.

• RSS — The parent service for pretty much all of the following, RSS has basically no push filtering. The only way I have seen this partially accomplished is by allowing separate feeds for different categories on a website.
• Twitter – As mentioned before many Twitter clients will allow you to categorize people you follow into different groups and display tweets only from a particular group or groups. This is still client-side filtering but the concept is there and just needs to be implemented for followers along with followees. Twitter also allows push filtering of a sort through hashtags. Hashtags are useful for certain things but like any kind of tag they have very limited uses. They are most useful for creating temporary groups or categorizing tweets of a similar nature and are not the best tool for targeting individual tweets at a subset of followers. Another way to accomplish push filtering in twitter is to simply have a separate twitter account for each category of message you think you might send (similar to RSS category feeds).
• Google Reader — The new Google Reader has “groups” that you can specify commenting permissions on, but you are unable to share individual articles/notes/etc with a particular group. You can set bulk permissions to only share your items with certain groups, but this setting applies to all of your items and cannot be set on an item by item basis.
• Yammer — A Twitter clone targeted at organizations for internal use, Yammer has groups that you can set up and join similar in a way to internal mailing lists (but for tweets). This is closer to the ideal, but these groups are set up for anybody to join, post and receive messages from. They have a use (as mailing lists have a use), but are not a good tool for targeting individual messages at just a subset of your followers. GroupTweet is an application that adds a similar functionality to Twitter
• FriendFeed — FriendFeed has groups in mailing list style similar to Yammer.

Every single one of these applications could benefit from a simple push-filter implementation.

There is one application that I haven’t mentioned yet that integrates both push and pull filtering quite effectively: e-mail. Each e-mail message allows you to specify recipients, and often times groups of recipients. Many e-mail readers allow you to filter messages based on sender/keyword/etc and create groups based on this data. E-mail is a great example of how push and pull filtering can be combined to manage information flow.

Why Twitter, et al, chose to imitate RSS with their paradigms instead of e-mail is beyond me. My only guess is that this choice was at some level based on the assumption that a follower will want to know everything that a producer might shares. In reality this assumption is flawed, partly  because there are so many producers worth following across many different domains.

There are some other implementation specific concerns with the details of how a push filter system would work in certain applications, but I do not intend to explore those now. I just hope that social sharing applications will begin to see the need to add some sort of push filter system into their models.