Discussions » Development

How/(is it possible) to fetch and also follow the href dofollow parameter?

§
Posted: 2021-04-02
Edited: 2021-04-02
EXAMPLE: For this site https://filmesviatorrents.gratis/2021/04/dificil-de-matar-torrent-2021-dual-audio-5-1-dublado-bluray-1080p-download.html

There are 5 redirects till we finally get to the final download page, so I did this code that fetches all next pages that we will end up being redirected to

Code to fetch till the final page like is fetched

var newDocument;
var URL = 'https://www.adssuper.com/instagram/campanha.php?idd===QZj5Wdv5mbhZmMlkjN5YTYzUCdl5mLh5WaiJ3bj5CehJHdmJTJmJTJhNTJwRHdo1jc0tDctFmJlNmb19mbuFmZyUyNzMTMhNTJnJ3buI3ajFmc05WZw9mLyV2ajFmc0ZmMlYmMlE2MlAHZ11jc0tDctFmJlNmb19mbuFmZyUCM4E2Ml02bj5CduVmcy9Gd0lmYuVGcv5iclt2YhJHdmJTJmJTJhNTJwRWd9IHd7AXbhZidr1mLvRWYsJWdE5CcpJVTBNkLwBjM34SMyAjMucmbvtEMyUiLzZHMyUSYsxWa6R2bH1jbktDctFmJBZzQDdTRxczMDVUOERDR0U0Q5QjMxUDOCVTRxkTOwgjQ5EERzEUR3oDapRnY64mc11Dd49jO0VmbnFWb&ref=viatorrents&titulo=R29kemlsbGEgdnMuIEtvbmcgVG9ycmVudCAoMjAyMSkgRHVibGFkbyDigJMgRG93bmxvYWQ=';
(async () => { //Creates a function to get the unprotected magnet link and Starts the function
while (true) { //While the if condition returns true
const response = await (await fetch('https://api.allorigins.win/raw?url='+URL)).text(); //Fetch
newDocument = new DOMParser().parseFromString(response, 'text/html'); //Parses the fetch response
URL = newDocument.querySelector('a').href; //Get the new url to be fetched on the next loop
if (newDocument.querySelectorAll("a[href*='magnet']").length === 1) { //If the magnet link exists on the final fetched document
window.open(newDocument.querySelector("a[href*='magnet']").href) //Open the magnet link
return; //Return true
} //Finishes the if condition
await new Promise(resolve => setTimeout(resolve, 2000)); //Wait 1 sec before fetching the new site again
} //Finishes the while condition
})(); //Finishes the async function


The only problem is that this site uses the href attribute dofollow to modify the whole final website, so if I fetch all sites I end up fetching the final real website without the modifications I want and Without the magnet link...
§
Posted: 2021-04-03

The code is stucking on the 3rd step, I guess

§
Posted: 2021-04-03
Edited: 2021-04-03
@Konf

I'm pretty sure that my code is 100% without bugs, the problem is what I said...
The website uses the href parameter "dofollow", so the previous page/link can "follow" the next link and modify it's HTML. That's why the final download page is a black download page that has a magnet link when manually opened.
But if I fetch all the links/redirects the fetch doesn't have something like "dofollow", so even though I'm fetching the correct links, the fetched final download page isn't black (background) and doesn't have the magnet link, that's why my if condition never becomes true and the "return" isn't executed, so the loop never ends.

Try clicking on the download button and you will see the black background arnolds download page with the magnet link.
Try fetching all the links (running my code for example) and analyze the FIRST fetched arnolds page HTML, you will see that the fetched arnolds page is just a random page of a random news post... (There's no magnet link using fetch, but the magnet link is there if the download link is opened in a new tab)

Arnolds.com.br is a news website, but because https://filmesviatorrents.gratis uses the dofollow parameter the filmesviatorrents website is able to modify 100% of the final arnolds.com.br HTML.

I'm pretty curious to know why this happen (how the filmesviatorrents website was able to modify the arnolds website HTML), and the only explanation possible is that all the redirect links/pages use the "dollow" parameter...

Can I somehow mimic the "dofollow" parameter using fetch?
Or is it possible to somehow find other way to get the final magnet link?
I just care about getting the magnet link in the shortest and fastest way possible, it doesn't necessarily needs to be a method using fetch, though I think that this is the only possible way to get the magnet link...

Post reply

Sign in to post a reply.