-
Notifications
You must be signed in to change notification settings - Fork 470
fix(openai): remove openai API key parsing entirely #15626
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 250 ± 3 ms. The average import time from base is: 253 ± 4 ms. The import time difference between this PR and base is: -2.6 ± 0.2 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate yunkim/openai-remove-api-key (acf7397) with baseline main (53cf907) 📈 Performance Regressions (4 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 0.379µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -0.8% Memory: ✅ 38.572MB (SLO: <41.500MB -7.1%) vs baseline: +4.6% ✅ add_inplace_aspectTime: ✅ 0.383µs (SLO: <10.000µs 📉 -96.2%) vs baseline: +0.2% Memory: ✅ 38.618MB (SLO: <41.500MB -6.9%) vs baseline: +5.2% ✅ add_inplace_noaspectTime: ✅ 0.285µs (SLO: <10.000µs 📉 -97.1%) vs baseline: -0.3% Memory: ✅ 38.737MB (SLO: <41.500MB -6.7%) vs baseline: +5.6% ✅ add_noaspectTime: ✅ 0.357µs (SLO: <10.000µs 📉 -96.4%) vs baseline: ~same Memory: ✅ 38.606MB (SLO: <41.500MB -7.0%) vs baseline: +5.4% ✅ bytearray_aspectTime: ✅ 1.291µs (SLO: <10.000µs 📉 -87.1%) vs baseline: -2.8% Memory: ✅ 38.678MB (SLO: <41.500MB -6.8%) vs baseline: +4.4% ✅ bytearray_extend_aspectTime: ✅ 1.488µs (SLO: <10.000µs 📉 -85.1%) vs baseline: -2.8% Memory: ✅ 38.570MB (SLO: <41.500MB -7.1%) vs baseline: +5.0% ✅ bytearray_extend_noaspectTime: ✅ 0.614µs (SLO: <10.000µs 📉 -93.9%) vs baseline: -1.0% Memory: ✅ 38.673MB (SLO: <41.500MB -6.8%) vs baseline: +5.1% ✅ bytearray_noaspectTime: ✅ 0.483µs (SLO: <10.000µs 📉 -95.2%) vs baseline: +0.4% Memory: ✅ 38.434MB (SLO: <41.500MB -7.4%) vs baseline: +4.6% ✅ bytes_aspectTime: ✅ 1.274µs (SLO: <10.000µs 📉 -87.3%) vs baseline: ~same Memory: ✅ 38.467MB (SLO: <41.500MB -7.3%) vs baseline: +4.4% ✅ bytes_noaspectTime: ✅ 0.499µs (SLO: <10.000µs 📉 -95.0%) vs baseline: +0.9% Memory: ✅ 38.811MB (SLO: <41.500MB -6.5%) vs baseline: +5.7% ✅ bytesio_aspectTime: ✅ 1.300µs (SLO: <10.000µs 📉 -87.0%) vs baseline: -0.7% Memory: ✅ 38.700MB (SLO: <41.500MB -6.7%) vs baseline: +4.8% ✅ bytesio_noaspectTime: ✅ 0.497µs (SLO: <10.000µs 📉 -95.0%) vs baseline: -0.4% Memory: ✅ 38.599MB (SLO: <41.500MB -7.0%) vs baseline: +5.3% ✅ capitalize_aspectTime: ✅ 0.742µs (SLO: <10.000µs 📉 -92.6%) vs baseline: ~same Memory: ✅ 38.472MB (SLO: <41.500MB -7.3%) vs baseline: +4.7% ✅ capitalize_noaspectTime: ✅ 0.439µs (SLO: <10.000µs 📉 -95.6%) vs baseline: -0.1% Memory: ✅ 38.532MB (SLO: <41.500MB -7.2%) vs baseline: +4.8% ✅ casefold_aspectTime: ✅ 0.740µs (SLO: <10.000µs 📉 -92.6%) vs baseline: +0.5% Memory: ✅ 38.572MB (SLO: <41.500MB -7.1%) vs baseline: +4.6% ✅ casefold_noaspectTime: ✅ 0.373µs (SLO: <10.000µs 📉 -96.3%) vs baseline: -0.5% Memory: ✅ 38.564MB (SLO: <41.500MB -7.1%) vs baseline: +4.7% ✅ decode_aspectTime: ✅ 0.725µs (SLO: <10.000µs 📉 -92.8%) vs baseline: -1.1% Memory: ✅ 38.485MB (SLO: <41.500MB -7.3%) vs baseline: +4.7% ✅ decode_noaspectTime: ✅ 0.421µs (SLO: <10.000µs 📉 -95.8%) vs baseline: +0.9% Memory: ✅ 38.596MB (SLO: <41.500MB -7.0%) vs baseline: +5.0% ✅ encode_aspectTime: ✅ 0.713µs (SLO: <10.000µs 📉 -92.9%) vs baseline: -1.5% Memory: ✅ 38.662MB (SLO: <41.500MB -6.8%) vs baseline: +4.9% ✅ encode_noaspectTime: ✅ 0.402µs (SLO: <10.000µs 📉 -96.0%) vs baseline: -0.7% Memory: ✅ 38.764MB (SLO: <41.500MB -6.6%) vs baseline: +5.1% ✅ format_aspectTime: ✅ 3.420µs (SLO: <10.000µs 📉 -65.8%) vs baseline: ~same Memory: ✅ 38.403MB (SLO: <41.500MB -7.5%) vs baseline: +4.1% ✅ format_map_aspectTime: ✅ 3.624µs (SLO: <10.000µs 📉 -63.8%) vs baseline: +0.9% Memory: ✅ 38.482MB (SLO: <41.500MB -7.3%) vs baseline: +4.5% ✅ format_map_noaspectTime: ✅ 0.819µs (SLO: <10.000µs 📉 -91.8%) vs baseline: +0.5% Memory: ✅ 38.831MB (SLO: <41.500MB -6.4%) vs baseline: +5.7% ✅ format_noaspectTime: ✅ 0.602µs (SLO: <10.000µs 📉 -94.0%) vs baseline: +1.2% Memory: ✅ 38.578MB (SLO: <41.500MB -7.0%) vs baseline: +5.2% ✅ index_aspectTime: ✅ 0.347µs (SLO: <10.000µs 📉 -96.5%) vs baseline: +1.6% Memory: ✅ 38.576MB (SLO: <41.500MB -7.0%) vs baseline: +4.6% ✅ index_noaspectTime: ✅ 0.313µs (SLO: <10.000µs 📉 -96.9%) vs baseline: -0.9% Memory: ✅ 38.547MB (SLO: <41.500MB -7.1%) vs baseline: +4.5% ✅ join_aspectTime: ✅ 1.276µs (SLO: <10.000µs 📉 -87.2%) vs baseline: -0.6% Memory: ✅ 38.714MB (SLO: <41.500MB -6.7%) vs baseline: +5.2% ✅ join_noaspectTime: ✅ 0.535µs (SLO: <10.000µs 📉 -94.6%) vs baseline: +0.6% Memory: ✅ 38.515MB (SLO: <41.500MB -7.2%) vs baseline: +5.0% ✅ ljust_aspectTime: ✅ 2.856µs (SLO: <20.000µs 📉 -85.7%) vs baseline: 📈 +10.4% Memory: ✅ 38.722MB (SLO: <41.500MB -6.7%) vs baseline: +5.5% ✅ ljust_noaspectTime: ✅ 0.408µs (SLO: <10.000µs 📉 -95.9%) vs baseline: +0.4% Memory: ✅ 38.471MB (SLO: <41.500MB -7.3%) vs baseline: +4.6% ✅ lower_aspectTime: ✅ 2.224µs (SLO: <10.000µs 📉 -77.8%) vs baseline: +0.2% Memory: ✅ 38.633MB (SLO: <41.500MB -6.9%) vs baseline: +4.5% ✅ lower_noaspectTime: ✅ 0.376µs (SLO: <10.000µs 📉 -96.2%) vs baseline: +1.5% Memory: ✅ 38.508MB (SLO: <41.500MB -7.2%) vs baseline: +4.4% ✅ lstrip_aspectTime: ✅ 2.161µs (SLO: <20.000µs 📉 -89.2%) vs baseline: -1.8% Memory: ✅ 38.613MB (SLO: <41.500MB -7.0%) vs baseline: +4.9% ✅ lstrip_noaspectTime: ✅ 0.382µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -0.9% Memory: ✅ 38.543MB (SLO: <41.500MB -7.1%) vs baseline: +4.8% ✅ modulo_aspectTime: ✅ 0.975µs (SLO: <10.000µs 📉 -90.3%) vs baseline: ~same Memory: ✅ 38.539MB (SLO: <41.500MB -7.1%) vs baseline: +4.7% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 1.470µs (SLO: <10.000µs 📉 -85.3%) vs baseline: -0.6% Memory: ✅ 38.758MB (SLO: <41.500MB -6.6%) vs baseline: +4.9% ✅ modulo_aspect_for_bytesTime: ✅ 0.960µs (SLO: <10.000µs 📉 -90.4%) vs baseline: +1.7% Memory: ✅ 38.694MB (SLO: <41.500MB -6.8%) vs baseline: +5.3% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 1.175µs (SLO: <10.000µs 📉 -88.3%) vs baseline: ~same Memory: ✅ 38.667MB (SLO: <41.500MB -6.8%) vs baseline: +5.3% ✅ modulo_noaspectTime: ✅ 0.666µs (SLO: <10.000µs 📉 -93.3%) vs baseline: -1.0% Memory: ✅ 38.607MB (SLO: <41.500MB -7.0%) vs baseline: +5.3% ✅ replace_aspectTime: ✅ 4.988µs (SLO: <10.000µs 📉 -50.1%) vs baseline: +1.0% Memory: ✅ 38.507MB (SLO: <41.500MB -7.2%) vs baseline: +4.5% ✅ replace_noaspectTime: ✅ 0.463µs (SLO: <10.000µs 📉 -95.4%) vs baseline: -1.1% Memory: ✅ 38.528MB (SLO: <41.500MB -7.2%) vs baseline: +5.0% ✅ repr_aspectTime: ✅ 0.954µs (SLO: <10.000µs 📉 -90.5%) vs baseline: +0.8% Memory: ✅ 38.604MB (SLO: <41.500MB -7.0%) vs baseline: +4.9% ✅ repr_noaspectTime: ✅ 0.457µs (SLO: <10.000µs 📉 -95.4%) vs baseline: -0.7% Memory: ✅ 38.681MB (SLO: <41.500MB -6.8%) vs baseline: +5.7% ✅ rstrip_aspectTime: ✅ 1.845µs (SLO: <20.000µs 📉 -90.8%) vs baseline: +1.5% Memory: ✅ 38.717MB (SLO: <41.500MB -6.7%) vs baseline: +5.4% ✅ rstrip_noaspectTime: ✅ 0.380µs (SLO: <10.000µs 📉 -96.2%) vs baseline: ~same Memory: ✅ 38.430MB (SLO: <41.500MB -7.4%) vs baseline: +4.9% ✅ slice_aspectTime: ✅ 0.486µs (SLO: <10.000µs 📉 -95.1%) vs baseline: +0.7% Memory: ✅ 38.657MB (SLO: <41.500MB -6.9%) vs baseline: +5.2% ✅ slice_noaspectTime: ✅ 0.452µs (SLO: <10.000µs 📉 -95.5%) vs baseline: +1.0% Memory: ✅ 38.571MB (SLO: <41.500MB -7.1%) vs baseline: +5.2% ✅ stringio_aspectTime: ✅ 1.670µs (SLO: <10.000µs 📉 -83.3%) vs baseline: -0.2% Memory: ✅ 38.553MB (SLO: <41.500MB -7.1%) vs baseline: +4.7% ✅ stringio_noaspectTime: ✅ 0.910µs (SLO: <10.000µs 📉 -90.9%) vs baseline: -1.0% Memory: ✅ 38.655MB (SLO: <41.500MB -6.9%) vs baseline: +5.1% ✅ strip_aspectTime: ✅ 2.159µs (SLO: <20.000µs 📉 -89.2%) vs baseline: -1.5% Memory: ✅ 38.467MB (SLO: <41.500MB -7.3%) vs baseline: +4.4% ✅ strip_noaspectTime: ✅ 0.394µs (SLO: <10.000µs 📉 -96.1%) vs baseline: +1.4% Memory: ✅ 38.645MB (SLO: <41.500MB -6.9%) vs baseline: +5.1% ✅ swapcase_aspectTime: ✅ 2.445µs (SLO: <10.000µs 📉 -75.5%) vs baseline: +0.2% Memory: ✅ 38.500MB (SLO: <41.500MB -7.2%) vs baseline: +4.7% ✅ swapcase_noaspectTime: ✅ 0.542µs (SLO: <10.000µs 📉 -94.6%) vs baseline: -0.2% Memory: ✅ 38.657MB (SLO: <41.500MB -6.9%) vs baseline: +5.2% ✅ title_aspectTime: ✅ 2.474µs (SLO: <10.000µs 📉 -75.3%) vs baseline: +3.7% Memory: ✅ 38.607MB (SLO: <41.500MB -7.0%) vs baseline: +5.1% ✅ title_noaspectTime: ✅ 0.500µs (SLO: <10.000µs 📉 -95.0%) vs baseline: -0.7% Memory: ✅ 38.686MB (SLO: <41.500MB -6.8%) vs baseline: +4.7% ✅ translate_aspectTime: ✅ 3.207µs (SLO: <10.000µs 📉 -67.9%) vs baseline: -0.6% Memory: ✅ 38.572MB (SLO: <41.500MB -7.1%) vs baseline: +5.3% ✅ translate_noaspectTime: ✅ 1.042µs (SLO: <10.000µs 📉 -89.6%) vs baseline: -0.2% Memory: ✅ 38.677MB (SLO: <41.500MB -6.8%) vs baseline: +5.1% ✅ upper_aspectTime: ✅ 2.237µs (SLO: <10.000µs 📉 -77.6%) vs baseline: +1.0% Memory: ✅ 38.803MB (SLO: <41.500MB -6.5%) vs baseline: +5.6% ✅ upper_noaspectTime: ✅ 0.377µs (SLO: <10.000µs 📉 -96.2%) vs baseline: +1.6% Memory: ✅ 38.518MB (SLO: <41.500MB -7.2%) vs baseline: +5.0% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.125µs (SLO: <10.000µs 📉 -48.8%) vs baseline: 📈 +21.6% Memory: ✅ 38.339MB (SLO: <41.000MB -6.5%) vs baseline: +4.8% ✅ ospathbasename_noaspectTime: ✅ 1.084µs (SLO: <10.000µs 📉 -89.2%) vs baseline: ~same Memory: ✅ 38.555MB (SLO: <41.000MB -6.0%) vs baseline: +5.0% ✅ ospathjoin_aspectTime: ✅ 6.032µs (SLO: <10.000µs 📉 -39.7%) vs baseline: +0.2% Memory: ✅ 38.437MB (SLO: <41.000MB -6.3%) vs baseline: +4.4% ✅ ospathjoin_noaspectTime: ✅ 2.294µs (SLO: <10.000µs 📉 -77.1%) vs baseline: +0.4% Memory: ✅ 38.457MB (SLO: <41.000MB -6.2%) vs baseline: +4.6% ✅ ospathnormcase_aspectTime: ✅ 3.523µs (SLO: <10.000µs 📉 -64.8%) vs baseline: ~same Memory: ✅ 38.378MB (SLO: <41.000MB -6.4%) vs baseline: +4.1% ✅ ospathnormcase_noaspectTime: ✅ 0.581µs (SLO: <10.000µs 📉 -94.2%) vs baseline: +1.9% Memory: ✅ 38.673MB (SLO: <41.000MB -5.7%) vs baseline: +5.4% ✅ ospathsplit_aspectTime: ✅ 4.845µs (SLO: <10.000µs 📉 -51.6%) vs baseline: -0.2% Memory: ✅ 38.437MB (SLO: <41.000MB -6.3%) vs baseline: +4.5% ✅ ospathsplit_noaspectTime: ✅ 1.592µs (SLO: <10.000µs 📉 -84.1%) vs baseline: +0.6% Memory: ✅ 38.594MB (SLO: <41.000MB -5.9%) vs baseline: +4.8% ✅ ospathsplitdrive_aspectTime: ✅ 3.702µs (SLO: <10.000µs 📉 -63.0%) vs baseline: -0.5% Memory: ✅ 38.574MB (SLO: <41.000MB -5.9%) vs baseline: +5.0% ✅ ospathsplitdrive_noaspectTime: ✅ 0.703µs (SLO: <10.000µs 📉 -93.0%) vs baseline: +1.4% Memory: ✅ 38.614MB (SLO: <41.000MB -5.8%) vs baseline: +5.0% ✅ ospathsplitext_aspectTime: ✅ 4.551µs (SLO: <10.000µs 📉 -54.5%) vs baseline: +0.2% Memory: ✅ 38.417MB (SLO: <41.000MB -6.3%) vs baseline: +4.5% ✅ ospathsplitext_noaspectTime: ✅ 1.393µs (SLO: <10.000µs 📉 -86.1%) vs baseline: +1.0% Memory: ✅ 38.633MB (SLO: <41.000MB -5.8%) vs baseline: +5.3% 📈 iastaspectssplit - 12/12✅ rsplit_aspectTime: ✅ 1.530µs (SLO: <10.000µs 📉 -84.7%) vs baseline: 📈 +11.0% Memory: ✅ 38.476MB (SLO: <41.000MB -6.2%) vs baseline: +4.8% ✅ rsplit_noaspectTime: ✅ 0.580µs (SLO: <10.000µs 📉 -94.2%) vs baseline: +0.6% Memory: ✅ 38.594MB (SLO: <41.000MB -5.9%) vs baseline: +5.0% ✅ split_aspectTime: ✅ 1.426µs (SLO: <10.000µs 📉 -85.7%) vs baseline: +4.8% Memory: ✅ 38.319MB (SLO: <41.000MB -6.5%) vs baseline: +4.1% ✅ split_noaspectTime: ✅ 0.572µs (SLO: <10.000µs 📉 -94.3%) vs baseline: +0.6% Memory: ✅ 38.555MB (SLO: <41.000MB -6.0%) vs baseline: +4.9% ✅ splitlines_aspectTime: ✅ 1.422µs (SLO: <10.000µs 📉 -85.8%) vs baseline: +4.8% Memory: ✅ 38.633MB (SLO: <41.000MB -5.8%) vs baseline: +4.9% ✅ splitlines_noaspectTime: ✅ 0.582µs (SLO: <10.000µs 📉 -94.2%) vs baseline: -0.2% Memory: ✅ 38.398MB (SLO: <41.000MB -6.3%) vs baseline: +4.4% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.349µs (SLO: <20.000µs 📉 -83.3%) vs baseline: 📈 +11.9% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ 1-count-metrics-100-timesTime: ✅ 201.678µs (SLO: <220.000µs -8.3%) vs baseline: -0.1% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.7% ✅ 1-distribution-metric-1-timesTime: ✅ 3.318µs (SLO: <20.000µs 📉 -83.4%) vs baseline: -0.6% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +5.0% ✅ 1-distribution-metrics-100-timesTime: ✅ 216.813µs (SLO: <230.000µs -5.7%) vs baseline: -0.9% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.6% ✅ 1-gauge-metric-1-timesTime: ✅ 2.169µs (SLO: <20.000µs 📉 -89.2%) vs baseline: -1.5% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.492µs (SLO: <150.000µs -9.0%) vs baseline: +0.1% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.7% ✅ 1-rate-metric-1-timesTime: ✅ 3.130µs (SLO: <20.000µs 📉 -84.3%) vs baseline: ~same Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ 1-rate-metrics-100-timesTime: ✅ 213.593µs (SLO: <250.000µs 📉 -14.6%) vs baseline: -0.8% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +5.2% ✅ 100-count-metrics-100-timesTime: ✅ 20.116ms (SLO: <22.000ms -8.6%) vs baseline: -0.4% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.288ms (SLO: <2.550ms 📉 -10.3%) vs baseline: +1.2% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.413ms (SLO: <1.550ms -8.8%) vs baseline: +0.4% Memory: ✅ 34.760MB (SLO: <35.500MB -2.1%) vs baseline: +4.6% ✅ 100-rate-metrics-100-timesTime: ✅ 2.214ms (SLO: <2.550ms 📉 -13.2%) vs baseline: -0.4% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.686µs (SLO: <20.000µs 📉 -76.6%) vs baseline: ~same Memory: ✅ 35.173MB (SLO: <35.500MB 🟡 -0.9%) vs baseline: +5.0% ✅ flush-100-metricsTime: ✅ 175.189µs (SLO: <250.000µs 📉 -29.9%) vs baseline: -0.2% Memory: ✅ 35.212MB (SLO: <35.500MB 🟡 -0.8%) vs baseline: +4.9% ✅ flush-1000-metricsTime: ✅ 2.180ms (SLO: <2.500ms 📉 -12.8%) vs baseline: ~same Memory: ✅ 35.960MB (SLO: <36.500MB 🟡 -1.5%) vs baseline: +4.6% 🟡 Near SLO Breach (16 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
## Description Closes #15585. This PR removes API key parsing from the openai integration for two reasons: 1. We were not using this anywhere (we used to tag the last 4 digits of the API key similar to how OpenAI formats theirs in UI, but tagging this was removed quite some time ago) 2. This caused errors in use cases involving `azure-identity` that involved API keys as functions (i.e. not directly storing API keys in app/memory), because we always assumed this was a string. <!-- Provide an overview of the change and motivation for the change --> ## Testing <!-- Describe your testing strategy or note what tests are included --> ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes <!-- Any other information that would be helpful for reviewers --> (cherry picked from commit 1714b7e)
Description
Closes #15585.
This PR removes API key parsing from the openai integration for two reasons:
azure-identitythat involved API keys as functions (i.e. not directly storing API keys in app/memory), because we always assumed this was a string.Testing
Risks
Additional Notes