javascript - NodeJS + Request - Access denied when requesting website

I'm trying to request the html of a website using request but I keep getting an access denied error. How do I get past this? Here is the code for the function below:

const request = require('request');
function firstShoe() {
        request('/', function (error, response, body) {
            console.log('body:', body); 
        });
}

Error:

</BODY>
</HTML>

body: <HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>

You don't have permission to access "http&#58;&#47;&#47;www&#46;jdsports&#46;co&#46;uk&#47;product&#47;green&#45;nike&#45;vapormax&#47;281735&#47;" on this server.<P>
Reference&#32;&#35;18&#46;609d3e17&#46;1500116386&#46;15f0cb85
</BODY>
</HTML>

Found a solution by passing the user-agent into the headers.

function firstShoe() {
        var options = {
            headers: {'user-agent': 'node.js'}
        }
        request('/', options, function (error, response, body) {
            console.log(body);
            message.channel.send(body);
        });
    }

I'm trying to request the html of a website using request but I keep getting an access denied error. How do I get past this? Here is the code for the function below:

const request = require('request');
function firstShoe() {
        request('https://www.jdsports.co.uk/product/green-nike-vapormax/281735/', function (error, response, body) {
            console.log('body:', body); 
        });
}

Error:

</BODY>
</HTML>

body: <HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>

You don't have permission to access "http&#58;&#47;&#47;www&#46;jdsports&#46;co&#46;uk&#47;product&#47;green&#45;nike&#45;vapormax&#47;281735&#47;" on this server.<P>
Reference&#32;&#35;18&#46;609d3e17&#46;1500116386&#46;15f0cb85
</BODY>
</HTML>

Found a solution by passing the user-agent into the headers.

function firstShoe() {
        var options = {
            headers: {'user-agent': 'node.js'}
        }
        request('https://www.jdsports.co.uk/product/green-nike-vapormax/281735/', options, function (error, response, body) {
            console.log(body);
            message.channel.send(body);
        });
    }

Share Improve this question edited Jul 15, 2017 at 23:16 asked Jul 15, 2017 at 11:07 hsel 1631 silver badge7 bronze badges

doesn't fs.readFile work for this kind of things? – yBrodsky Commented Jul 15, 2017 at 13:00
Where are you running this code? It sounds like there could be a firewall or web filter in place. – gregnr Commented Jul 15, 2017 at 13:11
You might have some CORS issues in the server. Install that package. Install this package github./expressjs/cors in your server. – Millenial2020 Commented Jul 15, 2017 at 14:32

Add a ment |

1 Answer 1

Sorted by: Reset to default 5

You are getting a 403 Forbidden because that website is blocking all requests sent using non mon user agents (basically they check User-Agent header). It is a very simple protection to avoid scrappers.

For example, if you send the following cURL using its standard User-Agent, the response is received perfectly:

curl -v 'https://www.jdsports.co.uk/product/green-nike-vapormax/281735/'

Nevertheless, if you repeat that request specifying a non existing User-Agent, the request is blocked:

curl -v 'https://www.jdsports.co.uk/product/green-nike-vapormax/281735/' -H 'User-Agent: StackOverflow'

发布者：admin，转转请注明出处：http://www.yc00.com/questions/1744699750a4588709.html

javascript - NodeJS + Request - Access denied when requesting website - Stack Overflow

1 Answer 1

发表回复

评论列表（0条）

联系我们

400-800-8888

javascript - NodeJS + Request - Access denied when requesting website - Stack Overflow

1 Answer 1

相关推荐