MTurk HIT Database Goals Edition

Adds Today's Goal and Longterm Goal, as well as bonus tracking, to the venerable HIT Database script, which originally included: Extended ability to search HITs you have worked on and other useful tools (CSV export/import, requester notes, requester block, pending/projected earnings)

Daily installs
Total installs
5 1 4
Applies to

Author's Description

***NOTE***: This is a modified version of the HIT DataBase script. This script can be installed alongside the original, since it has a different name. Once run, you cannot go back to the regular HIT DataBase without making a slight change to that script (change var v = 4; to var v = 6;). You should disable HIT DataBase before installing this version (Goals Edition). Before proceeding with installation, I strongly recommend that you backup your database using the CSV Export option.

As of v1.10.0, this script is compatible with Firefox!

Here are the major changes in v1.10.0:
  • New: Firefox compatibility! HIT Database Goals Edition now works in Firefox. There are some caveats to note:
    • Firefox is generally slower at loading HIT Database in general, including the dashboard and /mturk/status.
    • When searching, try to think of queries that will return smaller result sets. Depending on your computer's capability, searches that return more than about 1000 results, or that have very broad criteria (such as searching for HITs completed within the last six months) will load very slowly on Firefox, if at all.
    • Requester Overview in particular is very slow if you have a lot of requesters. I have tried to structure the loading so you can see progress as it loads, but it will still be slow.
  • New: Added import/export of bonuses and settings (goal amounts, longterm goal start date, show/hide various sections on the dashboard, and the settings listed under the Settings menu on the dashboard). Added dedicated Export Database button.
  • New: A tutorial for new users will automatically appear on first-time installation. Anybody can revisit this at any time by clicking the Tutorial link on the dashboard (towards the bottom, next to the HIT Database Goals Edition header).
  • New: Earnings breakdowns have been added to the Daily Overview. To accommodate this, the first time you run HIT Database Goals Edition after updating, the database will be updated. This may take some time. Please do not refresh or close the window while this is happening.
  • New: Added basic calculator (plus/minus) to Bonus entry window. No more mental math.
  • Changed: Moved Import Database button into new Import/Export pane.
  • Changed: Separate Search and Overview buttons into two separate panes on the dashboard. Overviews can have date filters put on them, and exported to CSV by ticking the appropriate box. Search retains all previous options such as status filter, text search, date filter and CSV export.
  • Changed: Completely revamped Overviews and Search Results. You can now sort and use filters in real-time on any results from the Overviews or Search panes on the dashboard. Once sorted/filtered, you can export any Overview or Search Results window to a CSV file. (These cannot be imported back into HIT Database, but they may be useful for analytical purposes outside of the browser.)
  • Changed: Ensure all dialog boxes, buttons and messages are converted to the 'Flick' style introduced in v1.9.7.
  • Changed: Move "Transfer" link into header for the Pending/Available Earnings table when it is collapsed.
  • Changed: Changed loading time allocation for both dashboard and /mturk/status, to try to prevent timeouts, especially in Firefox.
  • Fixed: Standardize name to HIT Database instead of HIT DataBase. This has bugged me as long as I have known of this script and my version will be corrected! :aetsch:
  • Fixed: Loading dialog box should no longer jump around the page or appear at the bottom.
  • Fixed: Should no longer be able to set a longterm goal start date without also setting a goal amount.
  • Other: Lots of little changes here and there that are too numerous to mention (or remember).
Note that much of the below information in the v1.9.7 section still pertains to v1.10.0 and beyond. Here are some screenshots of changed/new features in v1.10.0.

I have revamped much of the code I added in the first version of Goals Edition, and goal tracking is now basically automatic. There's no more "clocking out", or having to calculate earnings for a particular day to add it to a goal. Goals Edition keeps track of all earnings categories - pending, approved/paid, rejected, and submitted - and displays them on the Dashboard and Status pages, and uses this information to show progress against a daily goal, and a longterm goal.

There's also no more need to add/subtract bonuses from your longterm goal and your daily goal separately, since the other major new feature is bonus tracking. Goals Edition is not magic - you still have to enter bonuses by hand, because MTurk does not provide any information about bonuses in the interface except total amount earned - but you can now enter a bonus for any day that can be viewed on MTurk, and that includes days listed on the Status page. It will automatically be included in the appropriate day's earnings for purposes of goal tracking.

(It is not currently possible to add bonuses to days that are not listed on the even though HIT DataBase tracks your earning history all the way back to when you started using it, right now bonuses can only be entered for up to the last 45 days. This will likely be addressed in a future update.)

Goals Edition has been somewhat "prettified" with the addition of a loading progress bar when the page is loading, enhanced Settings and Actions menus, and the revamp of the bottom "HIT DataBase" area on the Dashboard to show the various buttons within logical groups and in an "accordion"-style tab interface.

When you initially load Goals Edition with an existing HIT DataBase database (Goals Edition or not), it will take a moment to upgrade your database and switch some of stored setting names. Once this is done, you should see the new interface, which will look something like this. (If you load it for the first time without an existing database, you will need to click on Maintenance at the bottom and then Update database, so it can pull in the data it needs to operate.)

The first thing to notice is the minus sign (-) next to some of the headers. This collapses those sections. (The one next to Date actually collapses the "HITs/Dollars" row above that section, since after a bit you can hopefully figure out which columns are referring to HIT numbers and which are referring to dollars...) For Earnings Pending & Available and Projected Earnings and Goals, pertinent information will be transferred into the header that remains after the section is collapsed. See the example below. It shows that the amounts for Pending Earnings and Earnings Available for Transfer are now in the header, and the two goal earned and target amounts are listed.

Whether a section is collapsed or not is saved, so when you reload the page they will stay that way.

In the Projected Earnings and Goals header, you will see a Settings button. You currently have two options that you can set.
  • Cumulative goal tracking: If enabled, when you don't hit your goal for a day, the unearned amount will be added to the next day's goal. Note that this only affects the display, and does not actually change the goal in the database. (This idea came from rosesword975 at MTG.)
  • Show amount remaining: If set to Earned, your goal display will show the amount that you have earned so far, as a positive (black) number. If set to Remaining, it will show the amount remaining toward your goal, as a negative (red) number. Once you reach your goal, having a setting of Remaining will show just the amount above your goal, while the Earned setting will show the total amount earned.
The Actions buttons across from Today's Projected Earnings and Goal Progress and Longterm (LT) Goal Progress allow you to set your daily or longterm goal, and for Today, adjust today's bonus earnings, or for Longterm, set your Longterm goal start date.

You can also adjust bonus earnings for any day by clicking in the Bonus cell across from any date. (Note this also works on the Status page, for all days listed, up to the last 45 days.) In the screenshot, if I clicked on the amount $40.73 across from Aug 14, 2015, I would be editing the bonus amount for yesterday. The bonus cell will always show $0.00 unless a bonus amount has been entered.

This is also explained in the dialog that appears when you go to edit a bonus amount, but bonus amounts are cumulative, meaning you should always enter the *total* bonus amount you want to have displayed. If you previously had $23.16, and just received $5.00, the amount you enter should be $28.16 (not $5.00). If you just enter $5.00, you will change your bonus amount for that day to $5.00.

At the bottom of the screen, the "accordion"-style layout groups all previously-available options into three logical groups. Overviews contains the buttons that launch the Pending, Requester, and Daily overviews. Search lets your search your database for a given timeframe and/or phrase, and export to CSV. Maintenance has Update, Delete and and Import functions. Note that one of these sections will always be visible - it is not possible to collapse all three at the same time, nor is it possible to open more than one at the same time. Clicking on an open accordion header will not do anything. This is the intended functionality.

With the exception of Update, no change has been made to how any of this functionality works. This may be changed in future updates. The only change made to Update is to display a progress bar on the screen the entire time the database is being updated (similar to the loading progress bar when the page first loads), and it should be relatively correct - if you have 1 day to update with 5 pages to scrape, then the bar should jump by about 17% each time. (The beginning steps take about 7% of the progress bar.) Goals Edition does the math so the progress bar advances logically as each page is read, and each day is finished.

Finally, I have not tested Goals Edition at length with other scripts. I do know that MTurk Dashboard Change Notifier (with 12-value mod) breaks because of it, but I have a simple fix for that. Same thing with MTurk Dashboard HIT Status Links. See this post at MTurkGrind for more information. mmmturkeybacon Add Hidden Stats to Dashboard works fine with no changes.