Greasy Fork

The code below works perfectly, but because it's an interval it doesn't return any promise

var totalanimes;
var totalanimestwo;
var TotalCompletedEntries = 0;
var nextpagenum = 0;
var increaseby = 1;
var chkReadyState = setInterval(async function GetVariables() {
nextpagenum += increaseby;
const response2 = await fetch('https://api.jikan.moe/v3/user/deg/animelist/completed/'+nextpagenum); //Fetch
const html2 = await response2.text(); //Gets the fetch response
totalanimestwo = JSON.parse(html2).anime.length;
TotalCompletedEntries += totalanimestwo; //Creates a variable to hold the actual TotalCompletedAnimes value
if (html2.match('mal_id') === null) {
clearInterval(chkReadyState);
}
}, 500); //Finishes the async function

I want to call this function and await for a promise, like as if I was calling a normal async function using
await GetVariables();

How can I make this code be called like this await GetVariables(); while looping every 500 ms, and having the code stop running when the condition html2.match('mal_id') === null is met?

async function GetVariables() {
  return new Promise(resolve => {
    const timer = setInterval(async () => {
      nextpagenum += increaseby;
      const url = 'https://api.jikan.moe/v3/user/deg/animelist/completed/' + nextpagenum;
      const html2 = await (await fetch(url)).text();
      totalanimestwo = JSON.parse(html2).anime.length;
      TotalCompletedEntries += totalanimestwo;
      if (!html2.includes('mal_id')) {
        clearInterval(timer);
        resolve();
      }
    }, 500);
  });
}

Example:

(async () => {
  await GetVariables();
  console.log(TotalCompletedEntries);
})();

Thanks, your code works great, but another problem I have is that the Jikan API has some rate limits, so that I can't have more than 1 fetch request on pending and already make another one, when I have something like 5 fetch requests on pending and they are all finished at once the API blocks the next requests, so sometimes I get some failed fetch requests, in this case I would need to somehow retry doing the fetch request for the same page.

Also for some reason, even though the interval is on 500 ms,it seems that the requests are being requested a bit faster than that, I even changed to 700 ms but I still got some error fetch requests for this url https://api.jikan.moe/v3/user/abystoma2/animelist/completed/

Also when the code
console.log(TotalCompletedEntries);
was displayed It didn't show the correct amount on TotalCompletedEntries, because I just pasted TotalCompletedEntries on the browser console after everything as done and it showed a different number, so it means that the await GetVariables(); isn't really awaiting for all the fetch requests to be successfully finished.

I also tried to use your if condition
if (!html2.includes('mal_id'))
and mine if condition a few times if (html2.match('mal_id') === null)
But the script still fetching a lot 5/10 urls even after the condition was already met, this is probably because the script isn't checking when the fetch request is done right

I also would like to know how I can fetch this
var nextpagenum = 0;
var increaseby = 300;
const url = 'https://myanimelist.net/animelist/abystoma2/load.json?status=2&offset=' + nextpagenum;

The problem is that the first nextpagenum value should be 0,not 300. How can I do this?

async function IIFE like (async () => { ... })() doesn't block the code that follows this. The asynchronous code runs differently. It's similar to registering a click event and then in the future when the click happens the function will run. In the meantime, until the event happened, your other code can run, it's not blocked by waiting for the event.

If you want to use the results of an async call like this you should do it inside the async function after await:

(async () => {
  await GetVariables();
  console.log(TotalCompletedEntries);
  // do something else here
})();

The fact that you're seeing a different number in console means that your code modifies the variable elsewhere.

Thanks

Is it possible to make the code wait for the fetch request to be finished before making the next request?
The problem is that the first nextpagenum value should be 0,not 300. How can I do this?

The variable TotalCompletedEntries shows a different number, because even after this is run
(async () => {
await GetVariables();
console.log(TotalCompletedEntries); //THIS
// do something else here
})();

The variable is summed with the var totalanimestwo, because console.log(TotalCompletedEntries); was run before the fetch request was completed, this is why TotalCompletedEntries is changed even after the code console.log(TotalCompletedEntries); was run.
That's why I asked that first question here now.

Yes, this is because of setInterval - it's the wrong tool here since a network request takes some time, but each setInterval cycle runs at the same pace regardless of how much time has actually passed since the last request.

A more reliable version should use setTimeout:

async function GetVariables() {
  while (true) {
    nextpagenum += increaseby;
    const url = 'https://api.jikan.moe/v3/user/deg/animelist/completed/' + nextpagenum;
    const html = await (await fetch(url)).text();
    totalanimestwo = JSON.parse(html).anime.length;
    TotalCompletedEntries += totalanimestwo;
    if (!html.includes('mal_id')) {
      return;
    }
    await new Promise(resolve => setTimeout(resolve, 600));
  }
}

nextpagenum

Just move the += line after fetch line:

    const url = 'https://api.jikan.moe/v3/user/deg/animelist/completed/' + nextpagenum;
    nextpagenum += increaseby;

Thanks, this is working perfectly now.

Now that I've implemented your script on my real script, your script is making much more requests than needed (the double or a bit more). I think that the script isn't waiting, or isn't correctly checking the if condition before returning the false value.

https://0bin.net/paste/a8GFEa3T#0Un2LhrSDCqkpEWl8az+J-rwLfvWmot9LksvmueMZPm
https://greasyfork.org/en/scripts/407957
https://myanimelist.net/animelist/hacker09?status=2

Do you have any idea on how to fix this?

loadingscreen() is called from different click events so it will run in parallel. You can chain the calls to run sequentially like this:

let loadingSequence = Promise.resolve();

function startLoading() {
  loadingSequence = loadingSequence.then(loadingscreen);
  return loadingSequence;
}

////////// call it
await startLoading();

Thanks for your reply and for sharing your time.
Actually that wasn't the problem, because the rewatched.onclick was only appended once,not twice, so I'm pretty sure that the loadingscreen function is only being called once.

I've just tested again my code and I've noticed that I forgot something very important, this time I was looking over the initiators on the fetch requests, and your and mine script is working perfectly. This line interval = setInterval(function() { is the one duplicating the fetch requests, even though the code isn't making any fetch request, this is because even if I didn't use this script, every time that I scroll the page down, the page makes a fetch request, this interval just automates it, so it appears that the fetch requests are being duplicated, but they are not.

Thanks for your help

@wOxxOm

https://myanimelist.net/forum/?topicid=1894779 page to run the script

Can you please help me understand why this isn't working?

var ChiakiIDSArray = [] //Creates a new blank array
async function GetChiakiIDS() { //Creates a new function
while (true) { //While the if condition returns true
var matches = document.querySelector("div.clearfix.word-break").firstChild.innerText.match(/(?:\|\b\d+)/gi); //Get all the anime ids on the MAL page
for(match in matches) //For all anime ids on the MAL page
{ //Starts the for condition
var FetchChiaki = matches[match].replace(/(?:\|)/gi, ' https://api.allorigins.win/raw?url=https://chiaki.site/?/tools/watch_order/id/'); //Creates a variable to fetch chiaki.site
const html = await (await fetch(FetchChiaki)).text(); //Gets the fetch response
const ChiakiDocument = new DOMParser().parseFromString(html, 'text/html'); //Parses the fetch response

var TextElement = ChiakiDocument.querySelectorAll("span.uk-text-muted.uk-text-small"); //Creates a variable to loop though the elements after
for (var i = 0; i < TextElement.length; i++) { //Starts the for condition
var FinalChiakiIDS = ChiakiDocument.querySelectorAll("span.uk-text-muted.uk-text-small > a")[i].href.match(/\d+/)[0]; //Get the final numbers on the chiaki.site MAL links
ChiakiIDSArray.push(FinalChiakiIDS); //Push all anime ids that chiaki.site has
} //Finishes the for condition

} //Finishes the for condition
if (ChiakiDocument.body.innerText.search('Watch') > -1) { //If the text Watch was found on the chiaki.site document
return; //Return true
} //Finishes the if condition
await new Promise(resolve => setTimeout(resolve, 1000)); //Wait 1 sec before fetching chiaki.site again
} //Finishes the while condition
} //Finishes the async GetChiakiIDS function
GetChiakiIDS(); //Starts the async GetChiakiIDS function


The ChiakiIDSArray.length is ending with 62 numbers, but it should be 11

Your inner loop is incorrectly using querySelectorAll. It should be something like this:

      for (const a of ChiakiDocument.querySelectorAll('span.uk-text-muted.uk-text-small > a')) {
        ChiakiIDSArray.push(a.href.match(/\d+/)[0]);
      }

@wOxxOm

That is still not working. Now it shows and error message saying that "ChiakiDocument is not defined"
So I tried to make the variable "global" by doing

var ChiakiDocument; //Makes the variable global
async function GetChiakiIDS() { //Creates a new function
} //Finishes the async function

And this fixes that error, but the loop still doesn't wait for 1 sec, and the ChiakiIDSArray.length is still ending with 62 numbers, but it should be 11
The 11 numbers contained on the array should be
38864 35180 38154 34647 34611 31646 28789 34777 36756 33964 32998

My comment meant you need to replace this:

var TextElement = ChiakiDocument.querySelectorAll("span.uk-text-muted.uk-text-small"); //Creates a variable to loop though the elements after
        for (var i = 0; i < TextElement.length; i++) { //Starts the for condition
var FinalChiakiIDS = ChiakiDocument.querySelectorAll("span.uk-text-muted.uk-text-small > a")[i].href.match(/\d+/)[0]; //Get the final numbers on the chiaki.site MAL links
ChiakiIDSArray.push(FinalChiakiIDS); //Push all anime ids that chiaki.site has
        } //Finishes the for condition

with my code.

I've managed a way to make it work. Thank you so much again for your help!

Yes, I know that you meant to replace that part with your codes. I did it, and I got that error message.

But this
var ChiakiDocument; //Makes the variable global
async function GetChiakiIDS() { //Creates a new function
} //Finishes the async function

Fixes that error.

I'm getting 62 numbers on the ChiakiIDSArray, because the script fetches chiaki.site 11 times, and it gets all the anime ids there, that are something like 14 anime ids, so 14 anime ids * 11 times it's kind of equal to 62, it's actually equal 152, so I'm not sure what it wrong here, but anyways...

Then I do this to get the 11 numbers that are on the https://myanimelist.net/forum/?topicid=1894779 page to run the script

var GuideIndexIDSmatches = document.querySelector("div.clearfix.word-break").firstChild.innerText.match(/(?:\|\b\d+)/gi); //Creates a new variable to hold the whole text of the first post on the topic
var GuideIndexIDS = []; //Creates a new blank array
var match; //Creates a new blank variable
for (match in GuideIndexIDSmatches) //For every anime id existent on the GuideIndexIDSmatches text content
{ //Starts the for condition
GuideIndexIDS.push(GuideIndexIDSmatches[match].replace(/(?:\|)/gi, '')); //Remove the first | symbol in front of the anime id numbers
} //Finishes the for condition

Then when I filter the 62 anime id numbers of the array ChiakiIDSArray, comparing this array against the GuideIndexIDS array

ChiakiIDSArray.filter(d => !GuideIndexIDS.includes(d));

Then I'm able to get the ids that chiaki.site has and the Guide is missing. This is what I wanted. So the final total length number of the array actually isn't 11, it's 3.
["16", "644", "1142"]

@wOxxOm

Now I'm having another problem (with step 3) were I can't filter arrays
ChiakiIDSArray.filter(d => !GuideIndexIDS.includes(d)); //Get the ids that chiaki.site has and the Guide is missing

alert(FinalArray.length) //Should be equal 3 ["16", "644", "1142"] not equal 900

https://pastebin.com/dUbxXXVj script codes

If I manually run on the browser console step by step 1,2,3 I can get the correct final result were the FinalArray.length is equal 3 ["16", "644", "1142"]

* test page https://myanimelist.net/forum/?topicid=1894779#msg61909608

One of the actual problems is that some numbers there (anime ids) arent like |ANIMEID|, they have some spaces | ANIMEID |, this isn't matching the regex .match(/(?:\|\b\d+)/gi) and this is giving some errors. I guess that this is the problem, I will remove the spaces, and if this doesn't work I will come here again...

@wOxxOm

Yeah the arrays aren't being filtered if I run the function as "buttons", but if I manually run the 3 functions I can get the array containing only
alert(FinalArray.length) //Should be equal 3 ["16", "644", "1142"]

Here are the codes to run manually, I've added multiple line breaks so you can know when each of them must be executed manually
https://pastebin.com/E7XJDUiD

test page https://myanimelist.net/forum/?topicid=1894779#msg61909608

Use devtools to debug your code. This is the primary programmer's tool.

@wOxxOm

Yes,I know that. The problem is that nothing is shown on the de tools,so I don't know what's the error.

I tried to add multiple alerts and console.logs and everything works,besides the FinalArray variable

By debugging I mean actually using the debugger in devtools, not console.log or alert. Set a breakpoint, then use the step function to execute the code line by line and inspect the variables, objects, DOM.

Oh, I didn't know that. I will do some checks using that now.

Thank you!

If I can't understand the problem I will comment here again

Thanks, I've managed to make it work!

The problem was that the array var GuideIndexIDS = []; was being set twice. Once outside anything to make it global, then it was being set again inside another button, this was stopping the array from being global

@wOxxOm

How can I do a "fetch" loop using GM.xmlHttpRequest?

https://pastebin.com/7eKCrqeB

Promisify it:

var entries = await new Promise((resolve, reject) => GM.xmlHttpRequest({
  // ............
  onload: r => resolve(r.response.data),
  onerror: reject,
}));

@wOxxOm

If I wrap xmlHttpRequest inside a while true loop, then after your code I add
console.log(entries)
var TotalEntries = entries.length; //Save the Total Entries Number
I get an undefined result for console.log(entries) and a error saying that "Cannot read properties of undefined (reading 'length')"

Show your new code.

@wOxxOm

https://pastebin.com/wtah2AbE

It means the value of entries is undefined i.e. r.response.data is undefined i.e. the response doesn't have data property or there was an error. Use devtools to set breakpoints and debug the code.

@wOxxOm

The problem was that the variable nextpagenum was 0

If I try to fetch more than 2 times one after the other the API does not return the data object key.

How do I add a delay between each request?

Promisify setTimeout: await new Promise(r => setTimeout(r, 1000))

Thanks!

@wOxxOm

GM.xmlHttpRequest doesn't run anything inside onerror even when the status is = 403.
Is is a normal GM.xmlHttpRequest issue or is it a loop issue?

It replicates the underlying behavior of XMLHttpRequest so apparently an error means a network error, whereas a different status still means a successfully completed request.