Race conditions when requesting shortened URLs #2390
Description
Describe the bug
- Node.js version: 20.10.0
- OS & version: Ubuntu 22.04.5 LTS
When making requests to various types of URLs, we see some failures for shortened URLs. The failures are not consistent, so I assume there is some race conditions going on.
The error logs look like this:
Nov 20 23:41:08 info: got-scraping called; parameters: (url:'https://fcld.ly/5hdnik1', ...)
Nov 20 23:41:08 file:///app/node_modules/got/dist/source/core/index.js:446
Nov 20 23:41:08 if (this._request._writableState?.errored) {
Nov 20 23:41:08 ^
Nov 20 23:41:08 TypeError: Cannot read properties of undefined (reading '_writableState')
Nov 20 23:41:08 at ClientRequest.<anonymous> (file:///app/node_modules/got/dist/source/core/index.js:446:35)
Nov 20 23:41:08 at Object.onceWrapper (node:events:628:28)
Nov 20 23:41:08 at ClientRequest.emit (node:events:526:35)
Nov 20 23:41:08 at request.emit (file:///app/node_modules/got-scraping/dist/index.js:434:14)
Nov 20 23:41:08 at onFinish (node:_http_outgoing:1005:10)
Nov 20 23:41:08 at afterWrite (node:internal/streams/writable:693:5)
Nov 20 23:41:08 at onwrite (node:internal/streams/writable:671:7)
Nov 20 23:41:08 at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:106:10)
Nov 20 23:41:08 at WriteWrap.callbackTrampoline (node:internal/async_hooks:130:17)
Actual behavior
The call fails inconsistently (sometimes successful, sometimes fails)
Expected behavior
The call should be successful
Code to reproduce
The code is calling got-scraping (https://github.com/apify/got-scraping), but I think the problem lies in got.
const response = await gotScraping({
url: 'https://fcld.ly/5hdnik1',
throwHttpErrors: true,
timeout: {
request: 20000
}
});Checklist
- I have read the documentation.
- I have tried my code with the latest version of Node.js and Got.