Aggregation is the New Black

by Matt Stratton on June 9, 2009

It’s been a while since I’ve blogged about technology, and that’s really my strong suit. I’m going to make a marked effort to post more tech-related content, and not just “Tech Tips”, but also my reactions to technology and where I think things are going.

It’s no great revelation that we are all awash in a sea of data. This data can come from various sources – for example, one thing I have long wrestled with is the disparate nature of my Contacts – I have contact information for people in Outlook, my BlackBerry, Gmail, and Facebook, just to name a few. For a long time, I have struggled with the concept of “synchronizing” this data – making sure that every one of those sources has an identical copy of the same information. I’m coming to the conclusion that this is perhaps not the right solution – we think that we want sync, but maybe we want more than that.

In addition to information that I create/control (i.e., contact information), there is a glut of data coming in to me every day for outside sources – emails from friends, blog posts, tweets, Facebook status updates, pictures on Flickr, and forum postings, just to name a few. This is all information that is created by other people, and either published to me from the creator, or possibly shared with me from a third-party (i.e., a friend who forwards me a link to a blog post). Keeping “on top” of all of this information requires me to manage multiple views into this information – the great example would be multiple tabs in a browser window, open to the various services that are specific to each channel of information, like so:

tabs1

Both of these scenarios, which seem to be different types of problems, have one common solution: aggregation. At first, it seems like the contact issue is solved with synchronization (and there are times that is helpful, where I want to have that information available to every service), but really, what I want is to have one common “view” into the aggregated contacts of ALL of my contact services.

The same is true of the inbound data – why should I have to read Tweets separately from emails, separate from Facebook status updates, separate from blog posts? Ultimately, it’s all incoming feed data, and I should be able to view it abstracted away from the source platform. I honestly believe that the platform of delivery is not relevant to the message.

This is what “Web 3.0″ is going to be all about, mark my words. Web 1.0 was about publishing – creating content and delivering it in a simple form, to be consumed in a one-way fashion. Web 2.0 brought us the “social” side of content – feedback and engagement, but really, it’s all about sharing. Well, guess what – we’ve started “sharing” so much that there’s a glut. We are overloaded with shared content – but we’re looking at it in lots of different places. I really think that the next phase of technology is going to be about getting data consolidated. Web 3.0 is going to be the aggregation story.

There are already signs of this. Look at all the buzz about the new Palm Pre smartphone – one of the major selling points is that it consolidates all of your various address books into a single one, with no duplicates. This is not a syncro solution – it’s aggregation. Consider Google Wave – it’s based upon the idea of abstracting all communication outside of the “email” and “IM” siloes into one location. FriendFeed is also built upon the concept of creating one timeline to track all content coming from people you are interested in. All of this points to the fact that aggregation is where it is at.

So what do I think the ultimate aggregator would be like? I want a single dashboard where I can stream in all of my incoming information. Actually, I think “dashboard” is the wrong term – that implies multiple widgets or areas dedicated to each stream/data source. I want a single timeline that streams all of the information and updates together.

However, I should be able to filter this timeline dynamically and on-demand. I might not want to see everything all at once – that would be overload. This filtering should be based upon content and people – not on the information platform. I want to be able to, for example, see all blog posts, twitter updates, Facebook statuses, Flickr pictures, etc, from a group of people, for example – or filter it based upon content. But again, content is abstracted away from the platform of delivery. I see no value in breaking things up by whether they are from Twitter or Facebook – but rather, based upon WHO they are from, or WHAT they are about. The HOW is the least important thing, and of limited to no value to filter upon.

FriendFeed tries REALLY hard to be this timeline. But there’s a problem – I can only stream in updates and information from people who publish to FriendFeed. I can’t add tweets from users who don’t use FriendFeed, or include blog posts from authors who don’t subscribe to that service. It’s going in the wrong direction – it’s aggregation for the publisher, not the subscriber.

One final note – this service should be cloud-based – that is, I should be able to consume and access this information and timeline from any device that is internet connected. Most aggregators already are smart in this way – Google Reader (which only aggregates RSS feeds) keeps my configuration and state accross any device that I use to access it. FriendFeed operates similarly. However, applications like TweetDeck (which can be used to aggregate Twitter and Facebook updates) use local configurations, which means that not only are my filterings and groupings not available on any computer I use, but my state also will change. This creates new problems of data/information management instead of solving them.

I realize this started to come across as a “wish list” for a service that may not exist yet (although I’m sure someone will tell me of one if it does). My intent isn’t to find this service, or even to create it…yet. My point is to identify the value of aggregation, and why I think that we are going to see it being the most important concept in the next “wave” (no pun intended) of user technology. It may not be identified as “aggregation” (perhaps it will have some sexy name along the lines of “social networking”, like “consolidated media”), but mark my words – at the base, it will be all about aggregation.

  • suhail ali

    The future has arrived Matt and it is called Microsoft Sync Services.

    • Matt Stratton

      But that’s still a sync story, right? I am pulling that data into a new source which I then view – I suppose even though it has “sync” in its name, that framework does provide the underlying architecture for the “magic app”, as Ken calls it.

      Azure will save us all :)

  • KennyNegs

    I agree with you for the most part. It would be nice. However, I don’t see how it’s gonna be feasible.

    It would require some sort of framework that every player agrees to…and every emerging service would have to get along with as well.

    If studying social media has taught me anything it’s that the “next big thing” becomes the “next old thing.” Then, new over-hyped channels emerge.

    I suppose the framework could be expandable (hint: open-sourced) so that it could potentially handle all newcomers.

    I also suppose that if you continually abstract content away from the delivery platform (which you so elegantly put), what’s the point of having the delivery platform? Why not just have one magical super app that contains all the functionality? No more Twitter as a stand-alone site (please god, make this happen) and no more Facebook…just a code provider, much like Google will be doing with Wave, and the functionality is just added to the framework of the magic app.

    I’d imagine it’d be quite a beast, pretty unwieldy and someone would probably totally screw up the interface… but anything’s possible.

    • Matt Stratton

      Ken,

      I’m not sure why each player would have to “subscribe” to the framework – all of these services are already consumable in some way, right? The trick would be for the central framework to know how to consume them (API, feed, etc). But you are exactly right in that it would need to be extendable, since each publisher would offer its data in a different way.

      If we do abstract away from source, you ask “why would we have the delivery platform?” For that matter, if we have RSS feeds, why actually have websites for blogs? I think that you would still be able to go back to the source if you wanted to – let’s say that my stream has a tweet from you – while I may be abstracted away from it being a tweet, I should still be able to access the original tweet (say by clicking on it) and the stream would have to “know” it was a tweet so that I could interact with it (replying means a different thing for a tweet than it does for an email, etc).

      • http://poleleaningdouchebag.com KennyNegs

        “I think that you would still be able to go back to the source if you wanted to – let’s say that my stream has a tweet from you – while I may be abstracted away from it being a tweet, I should still be able to access the original tweet (say by clicking on it) and the stream would have to “know” it was a tweet so that I could interact with it (replying means a different thing for a tweet than it does for an email, etc).

        Wouldn’t that sort of be the point of something like Wave? The source would always be the same, regardless if it resembled a Tweet, blog post update or status update. Just have one thing that does all of that. Just because it wouldn’t necessarily be “from” twitter, doesn’t mean you couldn’t interact with it in the same way.

        I’m not suggesting doing away with news sites in lieu of RSS aggregators. I’m suggesting doing away with the onslaught of social networking sites and providing one “thing” we can hang all of their functionality on and add more as it becomes available… like “Super Facebook” or the “super-mega-tweetdeck”. Don’t build the twitter site… just give me the functionality to add to my core application/website.

        KennyNegs’s last blog post..El Gigante vs. Humanity!

        • Matt Stratton

          I think that is EXACTLY the point of Wave, which is why I’m pretty excited about it. And again, I think we’re on the same page when it comes to the “don’t build the twitter site” thing – we’re going to see more and more of a demarcaction between sites/companies that specialize in content and those that provide the aggregation. Right now, places like Facebook and Twitter are kind of doing both (although they don’t create their own content, their users do) – you’re either providing the framework for the publishing, or for the consumption. I shouldn’t have to be a Twitter user in order to consume Tweets, for example.

          (on an unrelated note…if when you reply to a comment you click on the “reply” link, it will make it all threaded and junk)

  • http://poleleaningdouchebag.com KennyNegs

    Dang it. I forgot to include my website above.

    KennyNegs’s last blog post..El Gigante vs. Humanity!

  • http://curiousillusion.com/ Ari

    Maybe I just don’t get it, or maybe it’s just too much for me to process… but it’s like overload for me. I started a friendfeed a while back and never figured it out and then just gave up. Just now I went back and tried to add some friends to it but it’s just TOO much to sort through. Maybe I’m doing it wrong.

    I do like Google Reader though, because, like you said… it has all my blogs in one place. But I think I like Google Reader better because I decide which blogs I want to read right then, instead of having it all lumped together for me to have to filter and decide what updates I actually care about.

    Stuff like this makes me feel old, lol.

    Ari’s last blog post..Wherein I sneeze all over my book…

  • http://www.developerHash.com Shane

    Matt I totally agree with this post. I think this will also lead into more people becoming micro entrepreneurs. As the cost of aggregation goes down then larger corporations have less scale opportunities versus a single person. However I do think you will see more consolidation under “indie blog banners” so a group of people can scale their marketing together. This will be interesting to see. Content is KING not how you get content.
    [rq=407,0,blog][/rq]Using GitHub.com from Windows for .NET samples

  • http://freeandflawed.com Jenn

    I’m with Ari on this one. I’m completely overwhelmed. I signed up for Friendfeed a while ago, but didn’t use it much. I’ve gone back recently, but it’s almost too much. And you’re right. I can only follow those who use Friendfeed. So I can only imagine how much more overwhelmed I’d feel using it if everyone were on it.

    I’d like to see companies play with aggregation and see what they can come up with. I like the idea of Friendfeed, but I need more control I think. Until then, I’ll just have to suffer through the 5-8 open tabs in Firefox.

    (I had a much longer comment, but your blog ate it.)

  • Carol

    What can I say?! I’m a dinosaur and DESPISE the lack of privacy from this type of technology-blogging-twittering. Who the hell really needs to tweet? What purpose does it serve? And how damn insecure is one if they *need* a legion of relative strangers knowing about every last move they make. You want to know if I went to the can this morning? Seriously…once you give up your privacy you CAN’T GET IT BACK! Ever read Orwell??

    And on a more humorous note… I think it was last night Keith Olbermann was talking about following CNN on Twitter – so they can publish “news” that’s not important enough for their regular 24/7 broadcasting?!? Who needs to know that kind of crap?

    • http://poleleaningdouchebag.com KennyNegs

      All you’re missing here, Carol, is a “GET OFF MY LAWN!”
      [rq=3605,0,blog][/rq]Paging Dr. Inconsiderate…

  • Carol

    GET OFF MY LAWN!

    :oD

  • http://www.gorillasushi.com Jason – GorillaSushi

    I have some FriendFeed info that may excite you. You can set up an “Imaginary Friend” ( http://friendfeed.com/settings/imaginary ) to aggregate feeds from non-FriendFeed users.

    • Matt Stratton

      Wow, that DOES excite me. Seriously, pretty cool. Thanks for the tip! I was talking to Suhail on the train last night about the feasibility of developing the “universal timeline” aggregator I wishlist in my post. I think it’s a lot easier for me to design and develop my own for my own use, rather than one that other people could use. It might be a fun project as if I decide to crack down and actually teach myself Ruby this summer.

      • http://www.gorillasushi.com Jason – GorillaSushi

        I swear FriendFeed keeps making that feature harder and harder to find. You develop the “universal timeline” aggregator, I’ll come up with an awesome logo/site and we’ll make jillions.

Previous post:

Next post: