Skip to content

Reopening: M_Dwarfs_ASPCAPandBossNet.ipynb Download fits.gz file issue #9 - curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2) #10

Description

@JosephKarpinski

I've tried everything. No issues with other larger FIT files.

https://dr20.sdss.org/sas/dr20/spectro/astra/0.6.0/summary/astraAllStarASPCAP-0.6.0.fits.gz times out.

Across multiple days, times and browsers.

Image

curl -L -O --retry 10 --retry-delay 5 --continue-at -
https://dr20.sdss.org/sas/dr20/spectro/astra/0.6.0/summary/astraAllStarASPCAP-0.6.0.fits.gz

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
92 1116M 92 1036M 0 0 4019k 0 0:04:44 0:04:23 0:00:21 4437k
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)

curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR
This is not a client‑side failure.
This is the SDSS DR20 SAS server aborting the HTTP/2 stream mid‑transfer.
And the fact that it always dies around ~1.03–1.08 GB (your browser screenshot showed the same plateau) means the file is sitting behind a misconfigured or unhealthy HTTP/2 reverse proxy.
This is exactly what happens when:
the backend file server supports only HTTP/1.1
the front‑end proxy tries to serve it over HTTP/2
the proxy resets the stream when its internal buffer overflows
This is why other large FITS files download fine: they’re served by a different node.

curl --http1.1 -L -O --retry 10 --retry-delay 5 --continue-at -
https://dr20.sdss.org/sas/dr20/spectro/astra/0.6.0/summary/astraAllStarASPCAP-0.6.0.fits.gz

** Resuming transfer from byte position 1086668800
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 80.5M 100 80.5M 0 0 3982k 0 0:00:20 0:00:20 --:--:-- 4425k

That final 80.5 MB chunk completing cleanly tells us something very important:
Your download is now fully complete.
The file is 1,116,000,000 bytes, and your resumed transfer picked up at 1,086,668,800 bytes and pulled the remaining ~80 MB without error.
In other words:
The workaround (forcing HTTP/1.1 + resume) successfully bypassed the broken HTTP/2 proxy on the SDSS DR20 SAS node.

What happened technically
Your earlier failures were caused by:
The DR20 SAS HTTPS endpoint serving this file over HTTP/2
The proxy resetting the stream at ~1.03–1.08 GB
Browsers and default curl using HTTP/2 automatically
The server returning:
INTERNAL_ERROR — HTTP/2 stream not closed cleanly
By switching to:
curl --http1.1 ...

you forced the connection to use HTTP/1.1, which avoids the buggy HTTP/2 layer entirely.
The fact that the remaining chunk downloaded at ~4 MB/s with no resets confirms the backend file server itself is healthy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions