江阴网站建设,江阴网站制作,江阴网站设计,江阴SEO优化,江阴小程序开发-江阴雨辰互联

  1. JustNews 首页
  2. questions

node.js - porting curl command to nodejs program: fetch and http2 behaviour is different than curl program - Stack Overflow

admin••questions•阅读0

I tried to port this curl command into nodejs program:curl -H 'upgrade-insecure-requests: 1'

I tried to port this curl command into nodejs program:

curl -H 'upgrade-insecure-requests: 1' -H 'user-agent: Mozilla/5.0 (Linux; Android 13; Generic Android-x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.165 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' -H 'x-requested-with: .eu.droid_ng.jellyfish' -H 'sec-fetch-site: none' -H 'sec-fetch-mode: navigate' -H 'sec-fetch-user: ?1' -H 'sec-fetch-dest: document' -H 'sec-ch-ua: "Android WebView";v="125", "Chromium";v="125", "Not.A/Brand";v="24"' -H 'sec-ch-ua-mobile: ?0' -H 'sec-ch-ua-platform: "Android"' --compressed -H 'accept-language: en-US,en;q=0.9' -H 'priority: u=0, i' /

first attempt I use nodejs built-in fetch:

const response = await fetch("/", {
  headers: {
    Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    Priority: "u=0, i",
    "Sec-Ch-Ua": "\"Android WebView\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"",
    "Sec-Ch-Ua-Mobile": "?0",
    "Sec-Ch-Ua-Platform": "\"Android\"",
    "Sec-Fetch-Dest": "document",
    "Sec-Fetch-Mode": "navigate",
    "Sec-Fetch-Site": "none",
    "Sec-Fetch-User": "?1",
    "Upgrade-Insecure-Requests": "1",
    "X-Requested-With": ".eu.droid_ng.jellyfish"
  }
}).then(value => value.text())
console.log(response);

and later, I tried using nodejs http2 package:

const client = http2.connect(";);

const req = client.request({
    ":method": "GET",
    ":path": "/",

    Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    Priority: "u=0, i",
    "Sec-Ch-Ua": "\"Android WebView\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"",
    "Sec-Ch-Ua-Mobile": "?0",
    "Sec-Ch-Ua-Platform": "\"Android\"",
    "Sec-Fetch-Dest": "document",
    "Sec-Fetch-Mode": "navigate",
    "Sec-Fetch-Site": "none",
    "Sec-Fetch-User": "?1",
    "Upgrade-Insecure-Requests": "1",
    "X-Requested-With": ".eu.droid_ng.jellyfish"
});

let data = "";

req.on("response", (headers, flags) => {
    for (const name in headers) {
        console.log(`${name}: ${headers[name]}`);
    }

});

req.on("data", chunk => {
    data += chunk;
});
req.on("end", () => {
    console.log(data);
    client.close();
});
req.end();

on fetch and http2, I receive the following http response:

:status: 302
location: 
accept-ch-lifetime: 4838400
accept-ch: viewport-width,dpr,Sec-CH-Prefers-Color-Scheme,Sec-CH-UA-Full-Version-List,Sec-CH-UA-Platform-Version,Sec-CH-UA-Model
content-type: text/html; charset=utf-8
strict-transport-security: max-age=15552000; preload; includeSubDomains
x-fb-debug: +IvwboBtCg4h+RQcSxIqKQnT+fbZ4UIIA5kD6rhw3hAaTYWithPwt1Ga7USn6zWQ6R5O1rlNoroqcNywh+cdkg==
content-length: 0
date: Mon, 24 Mar 2025 05:17:09 GMT
x-fb-connection-quality: EXCELLENT; q=0.9, rtt=7, rtx=0, c=10, mss=1380, tbw=3534, tp=-1, tpl=-1, uplat=176, ullat=0
alt-svc: h3=":443"; ma=86400

while on curl, it's not redirected and work fine:

> GET / HTTP/2
> Host: m.facebook
> Accept-Encoding: deflate, gzip, br, zstd
> upgrade-insecure-requests: 1
> user-agent: Mozilla/5.0 (Linux; Android 13; Generic Android-x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.165 Safari/537.36
> accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
> x-requested-with: .eu.droid_ng.jellyfish
> sec-fetch-site: none
> sec-fetch-mode: navigate
> sec-fetch-user: ?1
> sec-fetch-dest: document
> sec-ch-ua: "Android WebView";v="125", "Chromium";v="125", "Not.A/Brand";v="24"
> sec-ch-ua-mobile: ?0
> sec-ch-ua-platform: "Android"
> accept-language: en-US,en;q=0.9
> priority: u=0, i
> 
* Request completely sent off
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [149 bytes data]
< HTTP/2 200 
< vary: Accept-Encoding
< content-encoding: zstd
< set-cookie: datr=h-jgZ5B3F055hnGkuJoTTc3T; expires=Tue, 28-Apr-2026 05:07:19 GMT; Max-Age=34560000; path=/; domain=.facebook; secure; httponly; SameSite=None
< set-cookie: fr=0i9xjAUxT5znP0c08..Bn4OiH..AAA.0.0.Bn4OiH.AWVNjQUxOKo; expires=Sun, 22-Jun-2025 05:07:19 GMT; Max-Age=7776000; path=/; domain=.facebook; secure; httponly; SameSite=None
< set-cookie: sb=h-jgZ35AmwVm0XVwmSy5H4Kg; expires=Tue, 28-Apr-2026 05:07:19 GMT; Max-Age=34560000; path=/; domain=.facebook; secure; httponly; SameSite=None
< reporting-endpoints: coop_report="/?minimize=0", default="/?cpp=C3&cv=1021179074&st=1742792839156", permissions_policy="/"
< report-to: {"max_age":2592000,"endpoints":[{"url":"https:\/\/www.facebook\/browser_reporting\/coop\/?minimize=0"}],"group":"coop_report","include_subdomains":true}, {"max_age":259200,"endpoints":[{"url":"https:\/\/m.facebook\/ajax\/weblite_error_reports\/?cpp=C3&cv=1021179074&st=1742792839156"}]}, {"max_age":21600,"endpoints":[{"url":"https:\/\/www.facebook\/ajax\/browser_error_reports\/"}],"group":"permissions_policy"}
< content-security-policy: default-src blob: 'self' https://*.fbsbx *.facebook *.fbcdn;script-src *.facebook *.fbcdn *.facebook 127.0.0.1:* 'nonce-qzk0BZXO' blob: data: 'self' connect.facebook 'unsafe-eval' https://*.google-analytics *.google;style-src *.fbcdn data: *.facebook 'unsafe-inline' ;connect-src *.facebook facebook *.fbcdn *.facebook wss://*.facebook:* wss://*.whatsapp:* wss://*.fbcdn attachment.fbsbx ws://localhost:* blob: *.cdninstagram 'self' http://localhost:3103 wss://gateway.facebook wss://edge-chat.facebook wss://snaptu-d.facebook wss://kaios-d.facebook/ v.whatsapp *.fbsbx *.fb https://*.google-analytics;font-src data: *.facebook *.fbcdn *.fbsbx ;img-src *.fbcdn *.facebook data: https://*.fbsbx facebook *.cdninstagram fbsbx fbcdn connect.facebook *.carriersignal.info blob: android-webview-video-poster: *.whatsapp *.fb *.oculuscdn *.tenor.co *.tenor *.giphy / https://*.paywithmybank/   https://*.google-analytics;media-src *.cdninstagram blob: *.fbcdn *.fbsbx www.facebook *.facebook data: *.tenor.co *.tenor https://*.giphy;child-src data: blob: 'self' https://*.fbsbx *.facebook *.fbcdn;frame-src *.facebook *.fbsbx fbsbx data: www.instagram *.fbcdn / https://*.paywithmybank/     *.google *.doubleclick;manifest-src data: blob: 'self' https://*.fbsbx *.facebook *.fbcdn;object-src data: blob: 'self' https://*.fbsbx *.facebook *.fbcdn;worker-src blob: *.facebook data:;
< document-policy: force-load-at-top
< permissions-policy: accelerometer=(self), attribution-reporting=(), autoplay=(), bluetooth=(), camera=(self), ch-device-memory=(), ch-downlink=(), ch-dpr=(), ch-ect=(), ch-rtt=(), ch-save-data=(), ch-ua-arch=(), ch-ua-bitness=(), ch-viewport-height=(), ch-viewport-width=(), ch-width=(), clipboard-read=(), clipboard-write=(self), compute-pressure=(), display-capture=(), encrypted-media=(), fullscreen=(self), gamepad=(), geolocation=(self), gyroscope=(self), hid=(), idle-detection=(), interest-cohort=(), keyboard-map=(), local-fonts=(), magnetometer=(), microphone=(self), midi=(), otp-credentials=(self), payment=(), picture-in-picture=(), private-state-token-issuance=(), publickey-credentials-get=(), screen-wake-lock=(), serial=(), shared-storage=(), shared-storage-select-url=(), private-state-token-redemption=(), usb=(), unload=(self), window-management=(), xr-spatial-tracking=();report-to="permissions_policy"
< cross-origin-resource-policy: same-origin
< cross-origin-embedder-policy: require-corp
< cross-origin-opener-policy: same-origin-allow-popups
< pragma: no-cache
< cache-control: private, no-cache, no-store, must-revalidate
< expires: Sat, 01 Jan 2000 00:00:00 GMT
< x-content-type-options: nosniff
< x-xss-protection: 0
< x-frame-options: DENY
< origin-agent-cluster: ?1
< accept-ch-lifetime: 4838400
< accept-ch: viewport-width,dpr,Sec-CH-Prefers-Color-Scheme,Sec-CH-UA-Full-Version-List,Sec-CH-UA-Platform-Version,Sec-CH-UA-Model
< content-type: text/html; charset=utf-8
< strict-transport-security: max-age=15552000; preload; includeSubDomains
< x-fb-debug: x8cBnHPLrM9aYVf5XN7r6KSqBedYcG5lRku6ZXJI8rmgTjuVoi015lYIN87nap1FPfr6OCDA4iA8Iq8ThX+jwA==
< date: Mon, 24 Mar 2025 05:07:19 GMT
< x-fb-connection-quality: EXCELLENT; q=0.9, rtt=7, rtx=0, c=10, mss=1380, tbw=3577, tp=-1, tpl=-1, uplat=344, ullat=0
< alt-svc: h3=":443"; ma=86400
< 
{ [5 bytes data]
* Connection #0 to host m.facebook left intact

I tried with http2 because the cURL verbose output shows that it's communicating with HTTP/2 protocol, but after tried with http2 module and the behavior is still same, I'm convinced it's something else, but at this point I have no idea what configuration to adjust on the fetch/http2 side.

I tried to port this curl command into nodejs program:

curl -H 'upgrade-insecure-requests: 1' -H 'user-agent: Mozilla/5.0 (Linux; Android 13; Generic Android-x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.165 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' -H 'x-requested-with: .eu.droid_ng.jellyfish' -H 'sec-fetch-site: none' -H 'sec-fetch-mode: navigate' -H 'sec-fetch-user: ?1' -H 'sec-fetch-dest: document' -H 'sec-ch-ua: "Android WebView";v="125", "Chromium";v="125", "Not.A/Brand";v="24"' -H 'sec-ch-ua-mobile: ?0' -H 'sec-ch-ua-platform: "Android"' --compressed -H 'accept-language: en-US,en;q=0.9' -H 'priority: u=0, i' https://m.facebook/

first attempt I use nodejs built-in fetch:

const response = await fetch("https://m.facebook/", {
  headers: {
    Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    Priority: "u=0, i",
    "Sec-Ch-Ua": "\"Android WebView\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"",
    "Sec-Ch-Ua-Mobile": "?0",
    "Sec-Ch-Ua-Platform": "\"Android\"",
    "Sec-Fetch-Dest": "document",
    "Sec-Fetch-Mode": "navigate",
    "Sec-Fetch-Site": "none",
    "Sec-Fetch-User": "?1",
    "Upgrade-Insecure-Requests": "1",
    "X-Requested-With": ".eu.droid_ng.jellyfish"
  }
}).then(value => value.text())
console.log(response);

and later, I tried using nodejs http2 package:

const client = http2.connect("https://m.facebook");

const req = client.request({
    ":method": "GET",
    ":path": "/",

    Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    Priority: "u=0, i",
    "Sec-Ch-Ua": "\"Android WebView\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"",
    "Sec-Ch-Ua-Mobile": "?0",
    "Sec-Ch-Ua-Platform": "\"Android\"",
    "Sec-Fetch-Dest": "document",
    "Sec-Fetch-Mode": "navigate",
    "Sec-Fetch-Site": "none",
    "Sec-Fetch-User": "?1",
    "Upgrade-Insecure-Requests": "1",
    "X-Requested-With": ".eu.droid_ng.jellyfish"
});

let data = "";

req.on("response", (headers, flags) => {
    for (const name in headers) {
        console.log(`${name}: ${headers[name]}`);
    }

});

req.on("data", chunk => {
    data += chunk;
});
req.on("end", () => {
    console.log(data);
    client.close();
});
req.end();

on fetch and http2, I receive the following http response:

:status: 302
location: https://www.facebook/unsupportedbrowser?_rdr
accept-ch-lifetime: 4838400
accept-ch: viewport-width,dpr,Sec-CH-Prefers-Color-Scheme,Sec-CH-UA-Full-Version-List,Sec-CH-UA-Platform-Version,Sec-CH-UA-Model
content-type: text/html; charset=utf-8
strict-transport-security: max-age=15552000; preload; includeSubDomains
x-fb-debug: +IvwboBtCg4h+RQcSxIqKQnT+fbZ4UIIA5kD6rhw3hAaTYWithPwt1Ga7USn6zWQ6R5O1rlNoroqcNywh+cdkg==
content-length: 0
date: Mon, 24 Mar 2025 05:17:09 GMT
x-fb-connection-quality: EXCELLENT; q=0.9, rtt=7, rtx=0, c=10, mss=1380, tbw=3534, tp=-1, tpl=-1, uplat=176, ullat=0
alt-svc: h3=":443"; ma=86400

while on curl, it's not redirected and work fine:

> GET / HTTP/2
> Host: m.facebook
> Accept-Encoding: deflate, gzip, br, zstd
> upgrade-insecure-requests: 1
> user-agent: Mozilla/5.0 (Linux; Android 13; Generic Android-x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.165 Safari/537.36
> accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
> x-requested-with: .eu.droid_ng.jellyfish
> sec-fetch-site: none
> sec-fetch-mode: navigate
> sec-fetch-user: ?1
> sec-fetch-dest: document
> sec-ch-ua: "Android WebView";v="125", "Chromium";v="125", "Not.A/Brand";v="24"
> sec-ch-ua-mobile: ?0
> sec-ch-ua-platform: "Android"
> accept-language: en-US,en;q=0.9
> priority: u=0, i
> 
* Request completely sent off
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [149 bytes data]
< HTTP/2 200 
< vary: Accept-Encoding
< content-encoding: zstd
< set-cookie: datr=h-jgZ5B3F055hnGkuJoTTc3T; expires=Tue, 28-Apr-2026 05:07:19 GMT; Max-Age=34560000; path=/; domain=.facebook; secure; httponly; SameSite=None
< set-cookie: fr=0i9xjAUxT5znP0c08..Bn4OiH..AAA.0.0.Bn4OiH.AWVNjQUxOKo; expires=Sun, 22-Jun-2025 05:07:19 GMT; Max-Age=7776000; path=/; domain=.facebook; secure; httponly; SameSite=None
< set-cookie: sb=h-jgZ35AmwVm0XVwmSy5H4Kg; expires=Tue, 28-Apr-2026 05:07:19 GMT; Max-Age=34560000; path=/; domain=.facebook; secure; httponly; SameSite=None
< reporting-endpoints: coop_report="https://www.facebook/browser_reporting/coop/?minimize=0", default="https://m.facebook/ajax/weblite_error_reports/?cpp=C3&cv=1021179074&st=1742792839156", permissions_policy="https://www.facebook/ajax/browser_error_reports/"
< report-to: {"max_age":2592000,"endpoints":[{"url":"https:\/\/www.facebook\/browser_reporting\/coop\/?minimize=0"}],"group":"coop_report","include_subdomains":true}, {"max_age":259200,"endpoints":[{"url":"https:\/\/m.facebook\/ajax\/weblite_error_reports\/?cpp=C3&cv=1021179074&st=1742792839156"}]}, {"max_age":21600,"endpoints":[{"url":"https:\/\/www.facebook\/ajax\/browser_error_reports\/"}],"group":"permissions_policy"}
< content-security-policy: default-src blob: 'self' https://*.fbsbx *.facebook *.fbcdn;script-src *.facebook *.fbcdn *.facebook 127.0.0.1:* 'nonce-qzk0BZXO' blob: data: 'self' connect.facebook 'unsafe-eval' https://*.google-analytics *.google;style-src *.fbcdn data: *.facebook 'unsafe-inline' https://fonts.googleapis;connect-src *.facebook facebook *.fbcdn *.facebook wss://*.facebook:* wss://*.whatsapp:* wss://*.fbcdn attachment.fbsbx ws://localhost:* blob: *.cdninstagram 'self' http://localhost:3103 wss://gateway.facebook wss://edge-chat.facebook wss://snaptu-d.facebook wss://kaios-d.facebook/ v.whatsapp *.fbsbx *.fb https://*.google-analytics;font-src data: *.facebook *.fbcdn *.fbsbx https://fonts.gstatic;img-src *.fbcdn *.facebook data: https://*.fbsbx facebook *.cdninstagram fbsbx fbcdn connect.facebook *.carriersignal.info blob: android-webview-video-poster: *.whatsapp *.fb *.oculuscdn *.tenor.co *.tenor *.giphy https://paywithmybank/ https://*.paywithmybank/ https://www.googleadservices https://googleads.g.doubleclick https://*.google-analytics;media-src *.cdninstagram blob: *.fbcdn *.fbsbx www.facebook *.facebook data: *.tenor.co *.tenor https://*.giphy;child-src data: blob: 'self' https://*.fbsbx *.facebook *.fbcdn;frame-src *.facebook *.fbsbx fbsbx data: www.instagram *.fbcdn https://paywithmybank/ https://*.paywithmybank/ https://www.googleadservices https://googleads.g.doubleclick https://www.google https://td.doubleclick *.google *.doubleclick;manifest-src data: blob: 'self' https://*.fbsbx *.facebook *.fbcdn;object-src data: blob: 'self' https://*.fbsbx *.facebook *.fbcdn;worker-src blob: *.facebook data:;
< document-policy: force-load-at-top
< permissions-policy: accelerometer=(self), attribution-reporting=(), autoplay=(), bluetooth=(), camera=(self), ch-device-memory=(), ch-downlink=(), ch-dpr=(), ch-ect=(), ch-rtt=(), ch-save-data=(), ch-ua-arch=(), ch-ua-bitness=(), ch-viewport-height=(), ch-viewport-width=(), ch-width=(), clipboard-read=(), clipboard-write=(self), compute-pressure=(), display-capture=(), encrypted-media=(), fullscreen=(self), gamepad=(), geolocation=(self), gyroscope=(self), hid=(), idle-detection=(), interest-cohort=(), keyboard-map=(), local-fonts=(), magnetometer=(), microphone=(self), midi=(), otp-credentials=(self), payment=(), picture-in-picture=(), private-state-token-issuance=(), publickey-credentials-get=(), screen-wake-lock=(), serial=(), shared-storage=(), shared-storage-select-url=(), private-state-token-redemption=(), usb=(), unload=(self), window-management=(), xr-spatial-tracking=();report-to="permissions_policy"
< cross-origin-resource-policy: same-origin
< cross-origin-embedder-policy: require-corp
< cross-origin-opener-policy: same-origin-allow-popups
< pragma: no-cache
< cache-control: private, no-cache, no-store, must-revalidate
< expires: Sat, 01 Jan 2000 00:00:00 GMT
< x-content-type-options: nosniff
< x-xss-protection: 0
< x-frame-options: DENY
< origin-agent-cluster: ?1
< accept-ch-lifetime: 4838400
< accept-ch: viewport-width,dpr,Sec-CH-Prefers-Color-Scheme,Sec-CH-UA-Full-Version-List,Sec-CH-UA-Platform-Version,Sec-CH-UA-Model
< content-type: text/html; charset=utf-8
< strict-transport-security: max-age=15552000; preload; includeSubDomains
< x-fb-debug: x8cBnHPLrM9aYVf5XN7r6KSqBedYcG5lRku6ZXJI8rmgTjuVoi015lYIN87nap1FPfr6OCDA4iA8Iq8ThX+jwA==
< date: Mon, 24 Mar 2025 05:07:19 GMT
< x-fb-connection-quality: EXCELLENT; q=0.9, rtt=7, rtx=0, c=10, mss=1380, tbw=3577, tp=-1, tpl=-1, uplat=344, ullat=0
< alt-svc: h3=":443"; ma=86400
< 
{ [5 bytes data]
* Connection #0 to host m.facebook left intact

I tried with http2 because the cURL verbose output shows that it's communicating with HTTP/2 protocol, but after tried with http2 module and the behavior is still same, I'm convinced it's something else, but at this point I have no idea what configuration to adjust on the fetch/http2 side.

  • node.js
  • http
  • curl
Share Improve this question edited Mar 24 at 5:40 ReYuki asked Mar 24 at 5:35 ReYukiReYuki 851 silver badge8 bronze badges 1
  • 2 well your code clearly missing "User-Agent" header, its is a must. – bogdanoff Commented Mar 24 at 6:17
Add a comment  | 

1 Answer 1

Reset to default 0

well... It turns out the problem was with me not being careful in reading the http header as @bogdanoff pointed out (thank you very much for the pointer! ^^)

I'm not aware as I use this site to convert from curl to fetch, I should disable the clean headers checkbox to make all headers imported

发布者：admin，转转请注明出处：http://www.yc00.com/questions/1744259814a4565574.html

adminadmin
0

相关推荐

发表回复

评论列表（0条）

  • 暂无评论

联系我们

400-800-8888

在线咨询： QQ交谈

邮件：admin@example.com

工作时间：周一至周五，9:30-18:30，节假日休息

关注微信