[RED/OPS] Upload Assistant

Accurate filling of new upload/request and group/request edit forms based on foobar2000's playlist selection or web link, offline and online release integrity check, tracklist format customization, featured artists extraction, classical works formatting, online cover art lookup, reporting open requests, form enhancements and more

La data de 20-02-2024. Vezi ultima versiune.

Autor
Anakunda
Rating-uri
0 0 0
Versiune
1.415
Creat
30-08-2019
Actualizat
20-02-2024
Licență
GPL-3.0-or-later
Se aplică pe

Make new uploads quicker and easier, main features in brief -

  • fills in upload/request form using release link or copied album from foobar2000 playlist
  • adds ways of inserting content to form (particularly images, inlined files and formatted text)
  • numerous additional checks, validations, rearrangements and lesser adjustments to improve user experience

Script using Greasemonkey extensions. Tested on Firefox, Chrome, Opera on Windows, Linux and Tampermonkey as script manager. On other platforms, browsers or script managers may not work, functionality on Violentmonkey may be limited (see known issues at bottom for more details).

A short notice: though Upload Assistant is designed to simplify and speedup making new uploads, it's purpose is not to encourage for effortlessly uploading massive batches of kinda valueless stuff, so avoid doing so.

New upload/request form filling

Supported sites for online capturing of release details:
- Qobuz (store)
- HighResAudio (store)
- Bandcamp (store)
- Presto Music (store)
- Discogs (community)
- Supraphonline (store, CZ)
- Bontonland (store, CZ, closing soon)
- NativeDSD (store)
- Juno Download (store)
- HDtracks (store)
- Deezer (store)
- Spotify (store)
- ProStudioMasters (store)
- 7digital (store)
- e-onkyo music (store, JP)
- Acoustic Sounds (store)
- Indies Scope (label, CZ)
- Beatport (store)
- Traxsource (store)
- Apple Music (store)
- MusicBrainz (community)
- VGMdb (community, JP, score related)
- Tidal (store, requires account - see remarks below for details about setup)
- OTOTOY (store, JP)
- Yandex Music (store, RU)
- Mora (store, JP)
- AllMusic (community)
- Bleep (store)
- Boomkat (store)
- ECM Records (label)
- ACT Music (label)
- JPC.de (store)
- [PIAS] Recordings (label)
- Domino Recording (label)
- Kompakt (store, DE)
- eClassical.com (store)
- QQmusic (store, CN)
- Muziekweb (store, NL)
- Beatsource (store)
- Netease - music.163.com (store, CN)
- Extreme Music (store)
- RecoChoku (store, JP)
- YouTube Music (store)
- Amazon Music (store)
- Kuwo (store, CN)
- FLO (store, KR)
- JOOX (store, HK)
- SoundCloud
- Bugs (store, KR)
- Melon (store, KR)

For e-books and audiobooks
- Martinus (store, CZ, SK)
- Goodreads (community)
- Databáze knih (community, CZ)
- Boomkat (store)
- Open Library (community)
- Google Books (library)
- Google Play Books (store)

For applications:
- SoftArchive (SAnet - dl blog)

Fetching from online sources may stop to work during time, I'm not checking most of them on regular basis, then in case site changes it's format I mostly rely on users feedback to update the script.
Fetching from online sources in most cases works out of the box, except these services:
- Tidal requires user account, script's user prefs to set: tidal_userid and tidal_userpassword
- Qobuz gets the metadata from HTML, if no account is linked; to fetch metadata using API, set these storage entries: qobuz_userid (registered e-mail), qobuz_userpassword; API brings in some extra metadata, especially UPC codes as cat№; API meta are available only if the release is available for country where account was registered (falls back to HTML parser). In brief Qobuz account credentials are better but not required to use with Qobuz.

User customization is accessible via script's local storage, no own UI for now.
How to access user configuration for Tampermonky: Run the script at least once by letting it fill form, then open the script in editor and switch to storage tab.

Additional setup for using foobar2000 as input source - to read audio player's playlist, custom copy format must be set:

foobar2000's native copy command: File > Preferences > Display > Classic User Interface > Title Formatting > Copy command, or Text Tools plugin quick copy command
$replace($replace([%album artist%]$char(30)[%album%]$char(30)[$if3(%date%,%ORIGINAL RELEASE DATE%,%year%)]$char(30)[$if3(%releasedate%,%retail date%,%date%,%year%)]$char(30)[$if2(%label%,%publisher%)]$char(30)[$if3(%catalog%,%CATALOGNUMBER%,%CATALOG NUMBER%,%labelno%,%catalog #%,%SKU%)]$char(30)[$if2(%country%,%RELEASECOUNTRY%)]$char(30)%__encoding%$char(30)%__codec%$char(30)[%__codec_profile%]$char(30)[%__bitrate%]$char(30)[%__bitspersample%]$char(30)[%__samplerate%]$char(30)[%__channels%]$char(30)[%__channel_mode%]$char(30)[$if3(%media%,%format%,%source%,%MEDIATYPE%,%SOURCEMEDIA%,%discogs_format%)]$char(30)[%genre%[|%style%]]$char(30)[%discnumber%]$char(30)[$if2(%totaldiscs%,%disctotal%)]$char(30)[%discsubtitle%]$char(30)[%track number%]$char(30)[$if2(%totaltracks%,%TRACKTOTAL%)]$char(30)[%title%]$char(30)[%track artist%]$char(30)[$if($strcmp(%performer%,%artist%),,%performer%)]$char(30)[$if3(%composer%,%writer%,%SONGWRITER%,%author%,%LYRICIST%)]$char(30)[%conductor%]$char(30)[%remixer%]$char(30)[$if2(%compiler%,%mixer%)]$char(30)[$if2(%producer%,%producedby%)]$char(30)[%length_seconds_fp%]$char(30)[%length_samples%]$char(30)[%filesize%]$char(30)[%replaygain_album_gain%]$char(30)[%replaygain_album_peak%]$char(30)[%replaygain_track_gain%]$char(30)[%replaygain_track_peak%]$char(30)[%album dynamic range%]$char(30)[%dynamic range%]$char(30)[%__tool%][ | $if2(%MQAENCODER%,%ENCODER%)][ | %ENCODER_OPTIONS%]$char(30)[$if2(%url%,%www%)]$char(30)[$directory_path(%path%)]$char(30)[$filename(%path%).$ext(%path%)]$char(30)[$if2(%comment%,%description%)]$char(30)$trim([BARCODE=$trim($replace($if3(%barcode%,%MCN%,%ICPN%), ,)) ][UPC=$trim($replace($if3(%UPC%,%UPC-A%,%UPCA%), ,)) ][UPC-E=$trim($replace($if2(%UPC-E%,%UPCE%), ,)) ][EAN=$trim($replace($if3(%EAN%,%EAN-13%,%EAN13%), ,)) ][IAN=$trim($replace($if3(%IAN%,%GTIN%,%GTIN-13%), ,)) ][EAN-8=$trim($replace($if2(%EAN-8%,%EAN8%), ,)) ][DISCID=$trim(%DISCID%) ][ASIN=$trim(%ASIN%) ][ISRC=$trim(%ISRC%) ][ISWC=$trim(%ISWC%) ][DISCOGS_ID=$trim(%discogs_release_id%) ][MBID=$trim(%MUSICBRAINZ_ALBUMID%) ][ACCURATERIPCRC=$trim(%ACCURATERIPCRC%) ][ACCURATERIPDISCID=$trim(%ACCURATERIPDISCID%) ][ACCURATERIPID=$trim(%ACCURATERIPID%) ][SOURCEID=$trim($replace(%SOURCEID%, ,_)) ][CT_TOC=$trim(%CDTOC%) ][ITUNES_TOC=$trim(%ITUNES_CDDB_1%) ][RELEASETYPE=$replace($if2(%RELEASETYPE%,%RELEASE TYPE%), ,_) ][COMPILATION=$trim(%compilation%) ][EXPLICIT=$trim($if2(%ITUNESADVISORY%,%EXPLICIT%)) ]SCENE=$if($and(%ENCODER%,%LANGUAGE%,%MEDIA%,%PUBLISHER%,%RELEASE TYPE%,%RETAIL DATE%,%RIP DATE%,%RIPPING TOOL%),1,0) [LANGUAGE=$trim($replace(%LANGUAGE%, ,_)) ][ORIGINALFORMAT=$trim($replace(%ORIGINALFORMAT%, ,_)) ][BPM=$trim(%BPM%) ][MD5=$info(md5)])$char(30)[$if2(%lyrics%,%unsynced lyrics%)],$char(13),$char(29)),$char(10),$char(28))

Note: Text Tools plugin substitutes linebreaks by underscores, making it ambiguous to restore original comment (remap_texttools_newlines=true for backward substitution), that's why using standard copy command is more reliable.

[Optional] More setup for proper transfer of secondary metadata
In %UserProfile%\AppData\Roaming\foobar2000\LargeFieldsConfig.txt (requires restart)
- increase basicMetaMax to 16384 or more
- remove fieldSpam=comment
- remove fieldSpam=lyrics
- remove fieldSpam=ITUNES_CDDB_1

Script already fills form with minimal set of required tags (album, artist, title, tracknumber), but requires additional edits. Accuracy and completeness of description improves with increasing amount of expectably filled relevant tags, especially RELEASEDATE/RETAIL DATE, LABEL/PUBLISHER, CATALOG/CATALOGNUMBER, GENRE/STYLE, COUNTRY, URL (for lineage, online integrity check & cover art), COMMENT/DESCRIPTION (for additional album description), MEDIA/SOURCE/FORMAT, RELEASETYPE, COMPILATION (otherwise guessed from various parameters, not reliable), ALBUM DYNAMIC RANGE, DISCNUMBER, DISCSUBTITLE (for multivolume albums). Beyond naming volumes script uses DISCSUBTITLE also for grouping of classical works and logical units within playlist (usually on extensive anthologies).

All generated tracklists are not coloured by default to avoid controversy. Colouring is still available by setting colorless_tracklist preference to false. Default colour scheme is tested to be well visible on as light as dark internal stylesheets. If there's a problem with specific stylesheet, particular colours can be changed via tracklist_#####_color entries. Built-in layouts (tracklist_style) are: left-aligned with optional colours highlighting (1), block-aligned colourless (2), and center-aligned with optional colours highlighting (3). Singles with one track only use own centered layout, standard numbered layout can be enforced by activating singles_conventional_format entry.

To suppress warnings about missing LOG/CUE files for CD uploads, set care_cd_extras storage value to false (ON by default).

[EXPERIMENTAL] Auto fill upload form for e-books

This feature tries to deduce name from attached torrent file, and lookup the information online. As soon as satisfactory result is found, resource url is used to fill the form. Supported category is only E-Books atm. Activates only if the upload form is empty.
Turn it off by setting auto_fill_by_torrent_name to false

Auto fill via url params

/upload.php?url=<URL>
/requests.php?action=new&url=<URL>
(URL parameter needs to be properly URI-encoded)

Enhancements to web form

  • custom drag & drop handler for all description fields; accepted content: local text files (foo_dr.txt), local images (uploaded), html sourced formatted text, web links and web images (rehosted)
    PTPimg account is required for any image related content (API key is autoconfigured on first use, be logged in to image host with same browser)
  • custom paste handler for all description fields (handles html formatted text and local images)
  • custom drag & drop handler for cover image input (upload local image / rehost web sourced image)
  • Pre-submit description cleanup (remove excessive linebreaks and empty tag pairs), configurable via cleanup_descriptions preference
  • (Orpheus/new upload only) altering upload form to stick with editions standard from other trackers (don't have original release but always use edition); this only affects the form, auto-written data always set to edition; enabled by default, for disabling: set user preference ops_always_edition to false

Some other features explained

Release integrity checking online is performed only for inputs from local files metadata. What is compared: title, main artist, playlist (track names, track artists and durations). Playlist size and overall duration is compared as well. This background check helps to prevent uploading incomplete/damaged albums, or releases that have messed up tracklist. Can be disabled by user pref check_integrity_online.
Requires valid URL tag to supported website (if missing, script tries to look up album in stores, but may fail case by case, eg. commute album and single of the same name)
Available online resources for metadata lookup (in particular order): Spotify, iTunes/Apple Music, MusicBrainz, Deezer, Qobuz, Discogs, Tidal, Supraphonline, AllMusic, Beatsource, Beatport, TraxSource, Netease, BandCamp, OTOTOY, Last.fm

Available online resources for cover lookup (in particular order): Deezer, Qobuz, iTunes/Apple Music, Tidal, MusicBrainz, Netease, Discogs, Beatsource, BandCamp, AllMusic, OTOTOY, YouTube Music, Last.fm. Services used for lookup and order of querying can be customized via cover_lookup_providers storage entry (defaults to all).

Existing torrents and open requests matching upload are looked up for new upload and request. Is on by default, to suppress it set find_relations to false.

Bulky images auto downsizing is on by default. Related preferences:
image_size_warning states size threshold in KiB (defaults to 2MiB) which exceeded invokes a reminder,
image_size_reduce_threshold states size threshold in KiB (defaults to 4MiB), which exceeded invokes image auto-reduction (downsizing and recompressing to JPEG90). Active only for cover image input, report proof input, user avatar and with working PTPimg account, not in long descriptions. More refinements: image_reduce_maxheight(1500), image_reduce_jpegquality(90).

Known issues

- script may encounter case by case incompatibility executing in Violentmonkey on certain browsers/platforms, which is bug or unwanted side effect, if observed switch to Tampermonkey or use different script till it's fixed
- image upload progress meter for locally sourced files doesn't work on current browsers, which is kind of Tampermonkey limitation or bug.
- it's reported to raise runtime errors when auto fill at Orpheus; this issue can't be fixed presently due to no access to the site