Wednesday, December 12, 2012

The First Anonymous Statistics Summary

For context, see the previous post Anonymous Statistic Gathering.  As of release 1.5 almost four weeks ago we added opt-in statistics gathering.  The goal is to let us know the sorts of things that real users are doing, to help inform future development decisions.  This is the first post where I start to look at the data that's been gathered, in aggregate.

Let me stress that point for a moment: in aggregate.  It is my intent to never publish or reveal specific users' data in any way.  When I do it will be in aggregate, like the rest of the post.  I actually never plan on even reading the specific rows carefully, unless there is a very specific issue to address.  That's part of the reason that I'll have no links or sources for any of the data below, like I would do for other things.  You've just got to trust me that it's accurate, I'm not sharing specifics on purpose.

That said, the first interesting piece of data is the opt-in rate.  According to Mozilla's statistics, Greasemonkey 1.5 has 2,092,151 users (basically everyone is using 1.5 besides some users still on 0.9, presumably also on older Firefox versions).  There are 23,790 distinct users sending data, which is just 1.14%.  But still hopefully enough

The distribution of submissions is interesting.
Submissions peak on Wednesdays, as version 1.5 was first made available on November 19th (the 20th, the next full day, being a Wednesday) with a weekly reporting period.

Among the users reporting data, by far the most common number of scripts installed is one; nearly half of all reporting users have only one script installed.  The next most common configuration is zero scripts installed!  While four users are reporting one hundred scripts or more installed, 95% of all users have ten or fewer.
Zero scripts installed by over twenty percent of users makes me worry that there might be a recording/reporting bug.  So this is worth revisiting.  Or if it's correct, it might warrant some sort of messaging to such users to coach them into getting use out of Greasemonkey.

The next interesting thing we can do is look at metadata imperative usage, where we are counting scripts by the number of times they are installed by real users, not once each for all that exist like we've done before.
This looks pretty familiar, if you followed the link above.  The top eight come in the same order, with similar percentages, following a similar power law curve.  Grant is used in 9.09% of scripts.

Finally one more number worth watching.  Today, just a bit under two thirds of installed scripts depend on implicit @grant detection.  Almost all of the remaining third need no grants at all, but it will be some time before we can remove that backwards-compatibility feature.  On the other hand, I'm currently detecting exactly zero scripts with explicit grants; I know that's wrong because I have some installed, and because usage of grant was at 9% in the graph above.  I'll need figure whether I'm reading the data wrong, or if it's being reported wrong.

6 comments:

Cicero said...

Interesting, so only 1% of users trust enough to report, and these are clearly not power users, since the vast majority have less than 7 scripts installed. (And ~55% have zero or 1 script!)

The power users I know (8 in my work group alone) all have dozens of scripts, and they all use @require. If they are still on pure GM, they now use @grant on every script too. But most of us have migrated to Scriptish or to internal forks of GM.

arantius said...

> Interesting, so only 1% of users trust enough to report

Not really. The feature was _very_ intentionally built to do nothing unless you opted in. You actually don't even get a lot of chance to: one info bar that doesn't stay open very long. More nags to opt in may be added at some point, but still over 20k users is a decent sample size.

> and these are clearly not power users, since the vast majority have less than 7 scripts installed. (And ~55% have zero or 1 script!)

That's probably good. "Power users" are probably well represented on the mailing lists; it's regular users that we had less insight into.

jerone said...

Great data. While 20k of users is just 1%, that's still a lot.

It would be interesting if this 1% are Firefox power users; detect Firefox online time.

Besides a post about this, it would be handy to make a special page for this. You won't post any specific data, only statics that are up-to-date.

SoboLAN said...

Well, I think one important factor is missing from here: When a user installs GreaseMonkey, what is the default state of that checkbox ? Is it checked or unchecked ?

On one hand, most users rarely or never look inside the options page of a program. On the other hand, GreaseMonkey users are a bit more advanced (I doubt my mom will ever use GreaseMonkey), so who knows...

arantius said...

> When a user installs GreaseMonkey, what is the default state of that checkbox ?

As I already said, it is "opt in". Users must opt in before anything happens. It defaults to off, it's intentionally difficult to turn on.

SoboLAN said...

@arantius Well in that case that 1.14 % shouldn't surprise you. (I fit inside that number :D ).