Thinking About NiceRank

Optimism is a wonderful thing. With it we can find the silver lining in every cloud and turn a frown upside-down. Optimism is what makes it possible to interact with a complete stranger without fear of being ignored, insulted, or injured. Optimism is something that I, a born pessimist, have worked very hard to integrate into my psyche since the late 90s and have achieved a level of confidence that seemed impossible just fifteen years ago. What does this have to do with App.net, you ask? Everything. I want my App.net experience to be positive and optimistic. The best way to do this is to find and interact with prolific conversationalists … but how do we find them? I have a solution; NiceRank.

This is the third in a series of posts outlining a possible future vision of App.net as an infrastructure, and the various tools built on top of this infrastructure.

Part I: Alpha Needs an Owner

Part I.I: Clarification on Alpha Ownership

Part II: Thinking About NiceRank

When I joined App.net in March of 2013, I was surprised by the fact that the Global timeline was not only readable, but a great way to find new people to interact with. For the first few days I saw the network as another Twitter, but with much less link-spam. But after interacting with people, it quickly became obvious that ADN was not another Twitter. It wasn’t like Facebook, Google+, or many of the other social networking sites, either. ADN was a place where people came for conversation, debate, and exchange of ideas.

I was hooked.

Six months later, though, the Global Timeline was a wasteland — a cesspool of English and Spanish-language spam that nobody wanted to see. Many people became very vocal about the loss of Global to the plethora of bots that came onto the platform after the floodgates were opened and truly free accounts were possible1. I was one of the more vocal denizens of the network and opted to do something about the state of the Global Timeline.

Several projects were born to trawl through the incredible amount of information being posted to the network. Accounts were identified. Methods were extrapolated. Patterns were found. In no time at all I had devised the mechanisms that could be employed by people all across the network to keep their timelines free of spam. It was such an easy method, and it would send a clear message to the people running App.net that spam was not welcome!

It worked like this:

  • spammers were identified algorithmically by looking at their post histories and determining whether they interact with people on the network, or merely shout into the void

  • huge lists of accounts for muting were created

  • people could apply the custom mute list to their accounts to eliminate all of the spam from Global

    It was really quite simple, and the project was more or less “done” in a weekend. There was a big problem, though. Every day there would be between 200 and 700 new accounts that would need to be added to the list. This meant creating a tool that could auto-block accounts identified as being from a spammer without any human oversight.

    I didn’t like this idea. There would undoubtedly be many people coming over to App.net from other places who, through no fault of their own, got added to this unforgiving list. Such a tool could ruin a community faster than it could “protect” one from spam. No … something more intelligent was needed. A nicer, more logical way to determine if a person was a spam account or a real account was needed. This is how NiceRank was born.

    NiceRank is something that I’ve worked on over the last six months with varying degrees of success. When the algorithm was the most accurate, I couldn’t return results fast enough. When the algorithm was the loosest, I couldn’t filter spam accounts well enough. NiceRank needs to have a combination of incredible speed and accuracy. Anything else would make this service completely untenable. The project was set aside until I could work out the best ways to get around the limitations of not having several hundred thousand yen to throw at dedicated server hardware. And, as is quite common in my life, the answer came to me in a dream2.

    The technical solution isn’t the star of the idea, though. The algorithm is what’s important. This is how I see NiceRank working:

  • an App.net client sends a request to the service containing up to 200 User IDs

  • the NiceRank for these 200 users is returned in under 0.5 seconds

  • the App.net client filters the timeline results based on the minimum setting attribute configured by the person using App.net

    The worst NiceRank would be 0, meaning that an account has no avatar, no bio, no followers, follows few, has taken part in 0 conversations, uses a little-known application to post, and 80% or more of their posts contain short-links.

    The best NiceRank would be 5, meaning that an account has an avatar and bio, a verified domain, a healthy number of followers, follows a good number of people, is part of many conversations, uses known (a.k.a. “trusted”) clients, and very few short-links.

    Accounts under 14 days of age with fewer than 100 posts will have a minimum NiceRank of 1.95 regardless of the other variables, which will make it slightly easier for new people to be identified in the Global Timeline if they’re not jumping into conversations to get their voice out there. Naturally, the rank can also increase quite nicely if they are actively participating with the community3.

    The calculation for NiceRank is a little more complicated than this, of course, but one can’t reveal everything. Spammers are very resourceful and will already have solutions in their head to get around some of the items I’ve listed above.

    Using this metric, what sorts of numbers can people expect to see? Well, as of this post:

  • @mps - 4.872

  • - 4.865

  • @annatarkov - 4.445

  • @benjamindorsey - 3.451

  • @daringfireball - 0.873

    My , @discoveradn, and @myfirstpodcast accounts don’t fare too well, hovering between 2.155 and 2.893, but they’re not in a terrible place. I am assuming that most people would have a minimum threshold of 2 in their clients.

    There’s still a bit of work to be done on this, and I would like to make it much faster before approaching app developers to trial it in their applications, but I feel this could be a good step towards adding value to the network. If people can join App.net knowing that they will not have to navigate automated accounts to find humans, that can make all the difference in a world where online companies are actively looking to extract as much revenue as possible from every pixel on our screens.