[RED/OPS] Upload Assistant

Accurate filling of new upload/request and group/request edit forms based on foobar2000's playlist selection or web link, offline and online release integrity check, tracklist format customization, featured artists extraction, classical works formatting, cover art fetching from store, checking for previous upload, form enhancements and more

As of 2021-01-25. See the latest version.

Author
Anakunda
Ratings
0 0 0
Version
1.346
Created
2019-08-30
Updated
2021-01-25
License
GPL-3.0-or-later
Applies to

Script using GreaseMonkey extensions, requiring GM, TamperMonkey or other GM clone, tested on Firefox, Chrome, Opera on Windows, Linux and Tampermonkey as script manager. On other platforms, browsers or script managers may not work.
Additional setup - to read audio player's playlist, custom copy format must be set:

foobar2000's native copy command: File > Preferences > Display > Classic User Interface > Title Formatting > Copy command, or Text Tools plugin quick copy command
$replace($replace([%album artist%]$char(30)[%album%]$char(30)[$if3(%date%,%ORIGINAL RELEASE DATE%,%year%)]$char(30)[$if3(%releasedate%,%retail date%,%date%,%year%)]$char(30)[$if2(%label%,%publisher%)]$char(30)[$if3(%catalog%,%CATALOGNUMBER%,%CATALOG NUMBER%,%labelno%,%catalog #%,%SKU%)]$char(30)[%country%]$char(30)%__encoding%$char(30)%__codec%$char(30)[%__codec_profile%]$char(30)[%__bitrate%]$char(30)[%__bitspersample%]$char(30)[%__samplerate%]$char(30)[%__channels%]$char(30)[%__channel_mode%]$char(30)[$if3(%media%,%format%,%source%,%MEDIATYPE%,%SOURCEMEDIA%,%discogs_format%)]$char(30)[%genre%[|%style%]]$char(30)[%discnumber%]$char(30)[$if2(%totaldiscs%,%disctotal%)]$char(30)[%discsubtitle%]$char(30)[%track number%]$char(30)[$if2(%totaltracks%,%TRACKTOTAL%)]$char(30)[%title%]$char(30)[%track artist%]$char(30)[$if($strcmp(%performer%,%artist%),,%performer%)]$char(30)[$if3(%composer%,%writer%,%SONGWRITER%,%author%,%LYRICIST%)]$char(30)[%conductor%]$char(30)[%remixer%]$char(30)[$if2(%compiler%,%mixer%)]$char(30)[$if2(%producer%,%producedby%)]$char(30)[%length_seconds_fp%]$char(30)[%length_samples%]$char(30)[%filesize%]$char(30)[%replaygain_album_gain%]$char(30)[%replaygain_album_peak%]$char(30)[%replaygain_track_gain%]$char(30)[%replaygain_track_peak%]$char(30)[%album dynamic range%]$char(30)[%dynamic range%]$char(30)[%__tool%][ | $if2(%MQAENCODER%,%ENCODER%)][ | %ENCODER_OPTIONS%]$char(30)[$if2(%url%,%www%)]$char(30)[$directory_path(%path%)]$char(30)[$if2(%comment%,%description%)]$char(30)$trim([BARCODE=$trim($replace($if3(%barcode%,%UPC%,%EAN%,%MCN%), ,)) ][DISCID=$trim(%DISCID%) ][ASIN=$trim(%ASIN%) ][ISRC=$trim(%ISRC%) ][ISWC=$trim(%ISWC%) ][DISCOGS_ID=$trim(%discogs_release_id%) ][MBID=$trim(%MUSICBRAINZ_ALBUMID%) ][ACCURATERIPCRC=$trim(%ACCURATERIPCRC%) ][ACCURATERIPDISCID=$trim(%ACCURATERIPDISCID%) ][ACCURATERIPID=$trim(%ACCURATERIPID%) ][SOURCEID=$trim($replace(%SOURCEID%, ,_)) ][CT_TOC=$trim(%CDTOC%) ][ITUNES_TOC=$trim(%ITUNES_CDDB_1%) ][RELEASETYPE=$replace($if2(%RELEASETYPE%,%RELEASE TYPE%), ,_) ][COMPILATION=$trim(%compilation%) ][EXPLICIT=$trim(%EXPLICIT%) ]SCENE=$if($and(%ENCODER%,%LANGUAGE%,%MEDIA%,%PUBLISHER%,%RELEASE TYPE%,%RETAIL DATE%,%RIP DATE%,%RIPPING TOOL%),1,0) [LANGUAGE=$trim($replace(%LANGUAGE%, ,_)) ][ORIGINALFORMAT=$trim($replace(%ORIGINALFORMAT%, ,_)) ][BPM=$trim(%BPM%) ][MD5=$info(md5)])$char(30)[%lyrics%],$char(13),$char(29)),$char(10),$char(28))

Note: Text Tools plugin substitutes linebreaks by underscores, making it ambiguous to restore original comment (remap_texttools_newlines=true for backward substitution), that's why using standard copy command is more reliable.

[Optional] More setup for proper transfer of secondary metadata:
In %UserProfile%\AppData\Roaming\foobar2000\LargeFieldsConfig.txt (requires restart)
- increase basicMetaMax to 16384 or more
- add fieldBasic=comment
- remove fieldSpam=ITUNES_CDDB_1

Script already fills form with minimal set of required tags (album, artist, title, tracknumber), but requires additional edits. Accuracy and completeness of description improves with increasing amount of expectably filled relevant tags, especially RELEASEDATE/RETAIL DATE, LABEL/PUBLISHER, CATALOG/CATALOGNUMBER, GENRE/STYLE, COUNTRY, URL (for lineage, online integrity check & cover art), COMMENT/DESCRIPTION (for additional album description), MEDIA/SOURCE/FORMAT, RELEASETYPE, COMPILATION (otherwise guessed from various parameters, not reliable), ALBUM DYNAMIC RANGE, DISCNUMBER, DISCSUBTITLE (for multivolume albums). Beyond naming volumes script uses DISCSUBTITLE also for grouping of classical works and logical units within playlist (usually on extensive anthologies).

Supported sites for online capturing of release details:
- Qobuz (store)
- HighResAudio (store)
- BandCamp (store)
- Presto Music (store)
- Discogs (community)
- Supraphonline (store, CZ)
- Bontonland (store, CZ, closing soon)
- NativeDSD (store)
- Juno Download (store)
- HDtracks (store)
- Deezer (store)
- Spotify (store)
- ProStudioMasters (store)
- 7digital (store)
- e-onkyo music (store, JP)
- Acoustic Sounds (store)
- Indies Scope (label, CZ)
- Beatport (store)
- Traxsource (store)
- Apple Music (store)
- MusicBrainz (community)
- VGMdb (community, JP, score related)
- Tidal (store, requires account - see remarks below for details about setup)
- OTOTOY (store, JP)
- Yandex Music (store, RU)
- Mora (store, JP)
- AllMusic (community)
- Bleep (store)
- Boomkat (store)
- ECM Records (label)
- ACT Music (label)
- JPC.de (store)
- [PIAS] Recordings (label)
- Domino Recording (label)
- Kompakt (store, DE)
- eClassical.com (store)
- QQmusic (store, CN)
- Muziekweb (store, NL)
- Beatsource (store)
- Netease - music.163.com (store, CN)
- Extreme Music (store)
- RecoChoku (store, JP)
- YouTube Music (store)
- Amazon Music (store)

For e-books and audiobooks
- Martinus (store, CZ, SK)
- Goodreads (community)
- Databáze knih (community, CZ)
- Boomkat (store)
- Open Library (community)
- Google Books (library)
- Google Play Books (store)

For applications:
- SoftArchive (SAnet - dl blog)

Fetching from online sources in most cases works out of the box, except these services:
Tidal requires user account, script's user prefs to set: tidal_userid and tidal_userpassword.

User customization is accessible via GM local storage, no own UI for now (use userscripts control panel to access the complete list of preferences).
How to access user configuration for Tampermonky: Run the script at least once by letting it fill form, then open the script in editor and switch to storage tab (quick way - open upload page, click TM toolbar icon, right click UA)

Enhancements to web form:

  • custom drag & drop handler for all description fields; accepted content: local text files (foo_dr.txt), local images (uploaded), html sourced formatted text, web links and web images (rehosted)
    PTPimg account is required for any image related content (API key is autoconfigured on first use, be logged in to image host with same browser)
  • custom paste handler for all description fields (handles html formatted text and local images)
  • custom drag & drop handler for cover image input (upload local image / rehost web sourced image)
  • Pre-submit description cleanup (remove excessive linebreaks and empty tag pairs), configurable via cleanup_descriptions preference
  • (Orpheus/new upload only) altering upload form to stick with editions standard from other trackers (don't have original release but always use edition); this only affects the form, auto-written data always set to edition; enabled by default, for disabling: set user preference ops_always_edition to false

Release integrity checking online is performed only for inputs from local files metadata. What is compared: title, main artist, playlist (track names, track artists and durations). Playlist size and overall duration is compared as well. This background check helps to prevent uploading incomplete/damaged albums, or releases that have messed up tracklist. Can be disabled by user pref check_integrity_online.
Requires valid URL tag to supported website (if missing, script tries to look up album in stores, but may fail case by case, eg. commute album and single of the same name)

Available online resources for metadata lookup (in particular order): Spotify, iTunes/Apple Music, MusicBrainz, Deezer, Qobuz, Discogs, Tidal, Supraphonline, AllMusic, Beatsource, Beatport, TraxSource, Netease, BandCamp, OTOTOY, Last.fm
Available online resources for cover lookup (in particular order): Deezer, Qobuz, iTunes/Apple Music, Tidal, MusicBrainz, Netease, Discogs, Beatsource, BandCamp, AllMusic, OTOTOY, YouTube Music, Last.fm. Services used for lookup and order of querying can be customized via cover_lookup_providers storage entry (defaults to all).

Existing torrents and open requests matching upload are looked up for new upload and request, and should workaround the known Gazelle unicode search bug. Is on by default, to suppress it set find_relations to false.

Site log lookup is performed for new upload to advise of possible previous fails of the same release and is on by default, to suppress it set check_logs to false.

Bulky images auto downsizing is on by default. Related preferences:
image_size_warning states size threshold in KiB (defaults to 1MB) which exceeded invokes a reminder,
image_size_reduce_threshold states size threshold in KiB (defaults to 2MB), which exceeded invokes image auto-reduction (downsizing and recompressing to JPEG90). Active only for cover image input, report proof input, user avatar and with working PTPimg account, not in long descriptions. More refinements: image_reduce_maxheight(1500), image_reduce_jpegquality(90).

All generated tracklists for stylesheets 1 (default) and 3 are coloured (syntax highlighting) by default. Default colour scheme is tested to be well visible on as light as dark stylesheets. If there's a problem with specific stylesheet, particular colours can be changed via tracklist_#####_color entries, or stripped off completely (colorless_tracklist). Built-in layouts (tracklist_style) are: left-aligned with colours highlighting (1), block-aligned colourless (2), and center-aligned with colours highlighting (3). Singles with one track only use own centered layout, standard numbered layout is forced by singles_conventional_format entry.

#### [EXPERIMENTAL] Auto fill upload form for e-books
This feature tries to deduce name from inserted torrent file, and lookup the information online. As soon as satisfactory result is found, resource url is used to fill the form. Supported category is only E-Books atm. Activates only if the upload form is empty.
Turn it off by setting `auto_fill_by_torrent_name` to false

Known issues:
- Progress meter for locally sourced uploads doesn't work on current browsers, which is kind of Tampermonkey limitation or bug.
- Violentmonkey (+GM) problems with names hoisting possibly resolved, report fails if any remaining