Make more money with Mechanical Turk by writing your own scripts

Hello Turkers!

This blog is intended for mTurk power users who want to optimize the time that they spend working on HITs. If you are familiar with performing crowd-sourced labor via mTurk, you are most certainly aware that completing HITs quickly can make the difference between an average day and standout success. Some Requesters provide exemplary interfaces that make it easy to complete their tasks in the most efficient way possible. One example would be CrowdSurf's transcription HITs.

Fig 1: Helpful keyboard shortcuts allow for fast work

CrowdSurf provides an interface that allows the Turker to rapidly scan the audio clip with a variety of buttons and keyboard shortcuts, amongst other features that facilitate quick and effective work.

Fig 2: Great interface from CrowdSurf

Unfortunately, most Requesters do not invest very much in their interfaces. In fact, I would say that most interfaces on mTurk are clunky and difficult to use, which prevents you from getting the most money for your time.

Enter the userscript. A userscript is essentially the same thing as a web browser extension. Typically written in Javascript, a userscript can enhance the existing functionality of a web page or add features to that page. Userscripts can also automate repetitive or redundant activities that are performed as a consequence of poor interface design.

Here's an example: There are many HITs which require a Worker to moderate a collection of images for adult content. The Worker is presented with a grid of images. Under each image there is a list of un-ticked bullet selection fields, one bullet for each image classification (e.g: Hardcore, Explicit, Suggestive, Non-Adult, etc).

Fig 3: A poor interface for image moderation

The most common selection for these image moderation tasks, in my experience, is Non-Adult. However, when the HIT loads in the Worker's browser, there is no default value for the bullet selection fields (that is, they are un-ticked). This means that the Worker must select a value for each image, even if we can reliably guess what most of the selections will be ahead of time. If the Worker were able to write a script that ticks "Non-Adult" by default, they could shave valuable seconds off of the time it takes them to complete the HIT, thus earning more money for their time.

This is a small example, but it has big implications when you apply the few seconds that you save to hundreds of HITs. This kind of automation can be even more successful when multiple aspects of the interface and task workflow are scripted in tandem. I would like to suggest that if you have a rudimentary understanding of programming concepts and a sharp eye for areas where your performance could be optimized, you can develop your own HIT-specific userscripts to maximize the gains from all of your hard work. 

I aim to discuss the following topics to aid you in the development of your own mTurk userscripts:

• Browser considerations
• Javascript fundamentals
• Framework for a typical userscript
• How to identify sluggish areas of task workflow
• Examples of successful userscripts

Stay tuned!

Best,

RCJ