Unstable results from the API

[gnodet]

I've been playing with the Disco API since a few days as part of apache/maven-wrapper#337.

I did setup some IT that do use the Disco API to actually perform a few requests and tests the real behaviours. However, the results are really unstable and the same requests may work or return empty results, no idea why.

For example, in this workflow, most ITs failed:
https://github.com/apache/maven-wrapper/actions/runs/16498393081/job/46650180851?pr=337

There are mainly two different errors:
the first one is about LTS versions:

Warning:  DEBUG: Known LTS version 17 detected as non-LTS!
Warning:  DEBUG: This indicates a Disco API issue. Re-fetching with debug logging...
Warning:  DEBUG: Fetching LTS versions from Disco API due to incorrect LTS detection
Warning:  DEBUG: Disco API request details:
Warning:    URL: https://api.foojay.io/disco/v3.0/major_versions?ea=false&ga=false&maintained=true&include_build=false&include_versions=false
Warning:    Max attempts: 3
Warning:    Base delay: 2000ms
Warning:    Attempt 1/3: HTTP 200
Warning:    Response length: 32 characters
Warning:    Response body: {  "result":[  ],  "message":""}
Warning:  DEBUG: Parsed LTS versions from API response: []
Warning:  DEBUG: Fallback LTS versions: [17, 33, 21, 8, 25, 11, 29]
Warning:  DEBUG: LTS version 17 still not found after debug re-fetch!
Warning:  DEBUG: This is a confirmed Disco API issue. Using fallback LTS detection.

the second one is about a JDK 17 temurin which is not found

Querying Disco API for JDK versions: https://api.foojay.io/disco/v3.0/packages?distro=temurin&package_type=jdk&jdk_version=17&operating_system=macos&architecture=aarch64&archive_type=tar.gz&latest=available&release_status=ga&directly_downloadable=true&include_versions=false
No matching JDK version found for 17 with distribution temurin
DEBUG: JDK 17 with Temurin should be available but was not found!
DEBUG: This indicates a Disco API issue. Logging request details:
DEBUG: API URL: https://api.foojay.io/disco/v3.0/packages?distro=temurin&package_type=jdk&jdk_version=17&operating_system=macos&architecture=aarch64&archive_type=tar.gz&latest=available&release_status=ga&directly_downloadable=true&include_versions=false
DEBUG: HTTP Status: 
DEBUG: Response length: 45 characters
DEBUG: Full API response: {"result":[],"message":"No package(s) found"}
DEBUG: Expected pattern: "java_version":"17.x.x"
DEBUG: Actual matches: 

I often had 503 errors in the past days, so I implemented a retry strategy.
We also had the failure where most packages were not returned anymore.

Is there anything we can do to help stabilise the infrastructure ? What could cause such instability ?

If there's no way to improve it, I'll have to find another solution for Maven.

[gnodet]

Is that caused by updates to the database somehow?
If so, it would be much more convenient to fail requests rather than returning wrong results, it would allow clients to retry at a later time, rather than not being able to trust the results.

[gnodet]

As of today (2025/09/29), the Temurin packages are not available anymore:

> curl 'https://api.foojay.io/disco/v3.0/packages?distribution=temurin' 
{"result":[],"message":"No package(s) found"}

[HanSolo]

Thanks for reporting, will take a look...

[joffrey-bion]

For the record, requesting this URL gives me either 1 or 2 packages randomly:
https://api.foojay.io/disco/v3.0/packages?distribution=temurin&latest=available&jdk_version=23&architecture=x64&archive_type=zip,tar.gz,tgz&package_type=jdk&javafx_bundled=false&directly_downloadable=true&free_to_use_in_production=true&operating_system=macos

I guess the correct result should be only 1 package, but sometimes I get an older patch as well.

Response with 1 package (expected, only 23.0.2+7):

{
  "result": [
    {
      "id": "9c6285812d65d1cd3bc24546a886d321",
      "archive_type": "tar.gz",
      "distribution": "temurin",
      "major_version": 23,
      "java_version": "23.0.2+7",
      "distribution_version": "23.0.2",
      "jdk_version": 23,
      "latest_build_available": false,
      "release_status": "ga",
      "term_of_support": "sts",
      "operating_system": "macos",
      "lib_c_type": "libc",
      "architecture": "x64",
      "fpu": "unknown",
      "package_type": "jdk",
      "javafx_bundled": false,
      "directly_downloadable": true,
      "filename": "OpenJDK23U-jdk_x64_mac_hotspot_23.0.2_7.tar.gz",
      "links": {
        "pkg_info_uri": "https://api.foojay.io/disco/v3.0/ids/9c6285812d65d1cd3bc24546a886d321",
        "pkg_download_redirect": "https://api.foojay.io/disco/v3.0/ids/9c6285812d65d1cd3bc24546a886d321/redirect"
      },
      "free_use_in_production": true,
      "tck_tested": "unknown",
      "tck_cert_uri": "",
      "aqavit_certified": "unknown",
      "aqavit_cert_uri": "",
      "size": 201048651,
      "feature": []
    }
  ],
  "message": "1 package(s) found"
}

Response with 2 packages (unexpected 23.0.1+11 result in addition to 23.0.2+7):

{
  "result": [
    {
      "id": "9c6285812d65d1cd3bc24546a886d321",
      "archive_type": "tar.gz",
      "distribution": "temurin",
      "major_version": 23,
      "java_version": "23.0.2+7",
      "distribution_version": "23.0.2",
      "jdk_version": 23,
      "latest_build_available": false,
      "release_status": "ga",
      "term_of_support": "sts",
      "operating_system": "macos",
      "lib_c_type": "libc",
      "architecture": "x64",
      "fpu": "unknown",
      "package_type": "jdk",
      "javafx_bundled": false,
      "directly_downloadable": true,
      "filename": "OpenJDK23U-jdk_x64_mac_hotspot_23.0.2_7.tar.gz",
      "links": {
        "pkg_info_uri": "https://api.foojay.io/disco/v3.0/ids/9c6285812d65d1cd3bc24546a886d321",
        "pkg_download_redirect": "https://api.foojay.io/disco/v3.0/ids/9c6285812d65d1cd3bc24546a886d321/redirect"
      },
      "free_use_in_production": true,
      "tck_tested": "unknown",
      "tck_cert_uri": "",
      "aqavit_certified": "unknown",
      "aqavit_cert_uri": "",
      "size": 201048651,
      "feature": []
    },
    {
      "id": "8ff87bfdfa7aeca89217a3e6fcd3fc16",
      "archive_type": "tar.gz",
      "distribution": "temurin",
      "major_version": 23,
      "java_version": "23.0.1+11",
      "distribution_version": "23.0.1",
      "jdk_version": 23,
      "latest_build_available": false,
      "release_status": "ga",
      "term_of_support": "sts",
      "operating_system": "macos",
      "lib_c_type": "libc",
      "architecture": "x64",
      "fpu": "unknown",
      "package_type": "jdk",
      "javafx_bundled": false,
      "directly_downloadable": true,
      "filename": "OpenJDK23U-jdk_x64_mac_hotspot_23.0.1_11.tar.gz",
      "links": {
        "pkg_info_uri": "https://api.foojay.io/disco/v3.0/ids/8ff87bfdfa7aeca89217a3e6fcd3fc16",
        "pkg_download_redirect": "https://api.foojay.io/disco/v3.0/ids/8ff87bfdfa7aeca89217a3e6fcd3fc16/redirect"
      },
      "free_use_in_production": true,
      "tck_tested": "unknown",
      "tck_cert_uri": "",
      "aqavit_certified": "unknown",
      "aqavit_cert_uri": "",
      "size": 201014412,
      "feature": []
    }
  ],
  "message": "2 package(s) found"
}

[HanSolo]

Can you check again? This should be fixed.

[joffrey-bion]

I do get consistently one package now, thanks!