More and more when I talk with application developers and people who run startups, they throw around numbers like I’ve gotten 400,000 people who’ve created an account with my application or 100,000 people who’ve added my applications on Facebook.
These numbers are really decieving. While it may sound like getting 100,000 people to add your application on Facebook is really good, if you dig deeper, you’ll find out that they only have 3,000 active uers.
Is a 3% return rate good enough? It wouldn’t be for me… I’d start questioning the long term success of my product.
You don’t judge a new brick and mortar store by how many people show up on grand opening day. You go three months later and see if people still care. Is it still serving a need when all the buzz has warn off?
Almost 10 million people have been like, hey Second Life sounds cool (which is AWESOME). They’re going to give that a try. Only around a million have decided to stick around and login in the last 30 days. Thats a 10% return rate. I’m not going to run around and just tell people the approx. 10 million signups number. I’m going to also tell them about the number of active users.
I dunno… it just seems like we need to become more knowledgeable about the metrics that we throw around and hold accountable the people who use them irresponsibly.
Oatmeal Stout - Justin Thorp's blog 
That’s why they taught us to discriminate and analyze our sources in middle school. I see a number like that and I automatically skim directly over it. If I really want to know, I dig up my own data.
Anon, this makes you smarter than some people I know.
To me, this seems like this is the result of sites like Techcrunch and Digg. Sites will get all this hype and all the tech savvy early adapters will check it out and create an account and then abandon it 5 minutes later if it’s not their thing.
You’re right that it says something about an app or site, that it’s not keeping those users. Active users are the more accurate numbers, but a lot of times the higher (inaccurate) number is used because it sounds more impressive initially without being analyzed. So deceptive.
I can’t remember where I heard this, but the quote is as true here as it was wherever it was first used:
“There are lies, damn lies, and then there are statistics.”
The problem is that at some level these companies and people are looking to establish a “nice” number that they can use to sell their product. Often they pick the nicest number that happened somewhere along their time line and then continue using that number.
The type of thing you’re bringing up with this article is part of an inherent problem with information in the information age. These pieces of data are taken at face value without any scrutiny given as to whether or not it actually indicates what it purports to.
Good call Justin.