Wednesday, February 18, 2009

TunkRank Scoring Improvement

Recently Daniel Tunkelang blogged about an influence rank for Twitter. Jason Adams took up the implementation challenge and coined it TunkRank.

After some poking at it, I'm suggesting a scoring improvement. At the moment the primary rank is percetile in the UI, however the raw score is given as well. I checked a few users and put together the table below, and it feels wrong. It saturates @ 100 too quickly and there is not enough differentiation between people with healthy versus massive influence.

Why 'feel'? Human interpretable numbers need a tactile sense to them in my opinion. One critique of the metric system is that the English system just feels more human compatible, an inch is not too small, a foot is reasonable, a mile is a long way and 100 miles per hour is darn fast.

I'm proposing two new scoring possibilities. Both are based upon logarithms and span from 1-100. The slight difference between them is how 'linear' the resulting rank feels across the accounts I compared.
  1. LEAST(100,ROUND(POWER(LN( tunkrank-raw-score +1);1.82)))

  1. LEAST(100,ROUND(LN( tunkrank-raw-score+1)/LN(3.5) * 10))

What are the constants? They are magic numbers to map Barak Obama to a TunkRank of 100 as well as provide an interesting spread between the test accounts below. Comments welcome! Which is my choice? I can't decide.. #1 is smells more accurate, #2 tastes more natural.

Yes this is an inexact science.

Possible Tunkrank Bug? Check out dewitt's rank.. looks off given his number of followers and that he's an influential guy from Google.

BarackObama 100 277770 100 100
wilw 100 79118 82 90
guykawasaki 100 62543 79 88
JasonCalaca 100 59075 78 88
THErealDVOR 100 43207 74 85
anamariecox 100 38177 73 84
WilliamShat 100 13932 61 76
fredwilson 100 13340 60 76
abdur 100 1351 36 58
johnhcook 99 407 26 48
johndcook 94 61 13 33
gutelius 84 20 8 24
nealrichter 81 16 7 23
ealdent 80 16 7 23
dtunkelang 79 15 6 22
dewitt 1 2 1 9


Jason M. Adams said...

Thanks for the great feedback on the scoring mechanism! I agree that saturation happens too quickly, but I wasn't sure what to do with it.

Also, some of my numbers still have bugs in them that haven't been cleared up yet. So many top player numbers will come down an order of magnitude.. Also, I'm still trying to get the social graph expanded. I know of over 2 million users, but know the friend relationships for only about 500k.

Anyhow, I'm happy to play around with displaying different measures.

Anthony said...

My raw score is a whopping 2 (pathetic, I know), but that puts me in the 27th percentile. Could TunkRank suffer from an inverse bell curve in its raw scores?