Trying out ThinkUp

I've been following the tool "ThinkUp" off an on for a while now. Gina Trapani, founder of LifeHacker long ago, created it as a hobby project and it was described as lots of fuzzy things like "social media insights engine". I was never very sure what it was, but every now and then I would hear neat things about it.

I noticed last week that apparently they've decided to create a commercial entity now around it and launch a whole website where you could just access it, instead of the current model where you have to download and install it on your own LAMP stack after some configuration.

So I figured if they saw it worth that, then maybe I should experiment with it. So I downloaded it (version 2.0-beta8, available on GitHub) and set it up on a system of mine. After a failed start or two I got it setup and was rather unimpressed. I kept tinkering with it and just couldn't see why I should care. But I left it running...

In my head I kept thinking about it and figured that since it's already pulling in all my FourSquare, Twitter, and Facebook history, it could create a nice Timeline view. Similar to TimeHop, but rather than showing me today in the past, just showing me today. Then I could take the results in a nice interleaved temporal format and toss them in Evernote for archive. ThinkUp has no internal capability for this, but has a decent plugin architecture and rather simple MySQL format that should make building this easy.

Then, I got to thinking about what else I could add. I use FitBit, so I wondered if I could get my FitBit steps & sleep stats pulled in and merged into the timeline as well. And hey, how about MyFitnessPal to show what/when I ate. And then RunKeeper blended in as well to see distance/speed. Oh, and I could access EverNote to pull in when, what, and where notes were created. And if I could get access to my PlaceMe stream I could merge that in too.

In short, I was fantasizing about building a single tool that could build a daily "Day in my Life", showing me everything I did and everywhere I went on a minute-to-minute basis. It's kinda scary to think that I leave that much "digital dust" around the internet, but it's reality. I've already begun on the FitBit integration and have a basically functioning Crawler and graph generator that shows my daily steps down at the 5-minute increment. It needs a bit of integration work to clean it up, and then a smarty template and such to get it viewable inside ThinkUp in a reasonable way, but I intend to release it on my GitHub fork once it's ready.

Oh, and while I was tinkering with all this and doing other stuff, ThinkUp kept churning away with it's hourly updates. And when I logged in the other day I finally started to catch a glimpse of what it really does. It took a few days to gather enough data, but now it shows me some interesting statistics like:

Most of my Tweets are questions

Most of my status updates are personal (containing "i", "me", "my", etc)

Some neat graphs about topic vs reply/like count

Interesting graphs about which of my friends most commonly respond

So I'm going to let it keep going. I figure it can't hurt to get some information like this while I integrate the rest of the functions I want.

Also, I might need to find a better place to host it for reliability. Right now it's just running on a system internal to my own network, so to get it I have to establish an SSH tunnel and such. It works, and it's safe so that's a plus, but it's then at the (somewhat flakey) reliability of my own network and hardware. I saw a post where you can theoretically now run ThinkUp on Google App Engine's new PHP support, but I can't find any information on how it actually performs or where anyone's ever tried it. GAE has a 60-second limit on scripts which, in my experience, would cripple crawling, so I'm curious to know if anyone's tried it.

However, I am a bit concerned about the future of ThinkUp. The latest version available is 2.0-beta8, with no commit's on the "main" branch in 2 months. I see lots of activity on forks, including a few important bugfixes that are critical to get it running with recent changes in Facebook, but none of them have been merged back to the master. Surely if they're preparing to roll out a company they've fixed them internally. I fear that they may internally fork the project into an "Awesome for-pay version" and "Less functional but open-source version that's always a bit out of date".I guess only time will tell.......