-
Notifications
You must be signed in to change notification settings - Fork 470
fix(profiling): enable sub-classing wrapped lock types by custom lock implementations #15604
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
|
09e4d31 to
f9fd5ea
Compare
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 262 ± 3 ms. The average import time from base is: 269 ± 4 ms. The import time difference between this PR and base is: -6.7 ± 0.2 ms. Import time breakdownThe following import paths have shrunk:
|
f5c3837 to
50733c4
Compare
c2116a2 to
dc95707
Compare
Performance SLOsComparing candidate vlad/lockprof-fix-subclassing-wrapped-locks (dc95707) with baseline main (f23eb40) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 0.386µs (SLO: <10.000µs 📉 -96.1%) vs baseline: +0.9% Memory: ✅ 38.613MB (SLO: <41.500MB -7.0%) vs baseline: +4.5% ✅ add_inplace_aspectTime: ✅ 0.381µs (SLO: <10.000µs 📉 -96.2%) vs baseline: +0.4% Memory: ✅ 38.516MB (SLO: <41.500MB -7.2%) vs baseline: +4.6% ✅ add_inplace_noaspectTime: ✅ 0.283µs (SLO: <10.000µs 📉 -97.2%) vs baseline: -2.3% Memory: ✅ 38.482MB (SLO: <41.500MB -7.3%) vs baseline: +4.7% ✅ add_noaspectTime: ✅ 0.355µs (SLO: <10.000µs 📉 -96.5%) vs baseline: -0.4% Memory: ✅ 38.603MB (SLO: <41.500MB -7.0%) vs baseline: +4.8% ✅ bytearray_aspectTime: ✅ 1.267µs (SLO: <10.000µs 📉 -87.3%) vs baseline: -2.5% Memory: ✅ 38.664MB (SLO: <41.500MB -6.8%) vs baseline: +5.0% ✅ bytearray_extend_aspectTime: ✅ 1.537µs (SLO: <10.000µs 📉 -84.6%) vs baseline: +3.7% Memory: ✅ 38.480MB (SLO: <41.500MB -7.3%) vs baseline: +4.7% ✅ bytearray_extend_noaspectTime: ✅ 0.622µs (SLO: <10.000µs 📉 -93.8%) vs baseline: +1.4% Memory: ✅ 38.655MB (SLO: <41.500MB -6.9%) vs baseline: +4.9% ✅ bytearray_noaspectTime: ✅ 0.484µs (SLO: <10.000µs 📉 -95.2%) vs baseline: -0.4% Memory: ✅ 38.612MB (SLO: <41.500MB -7.0%) vs baseline: +5.1% ✅ bytes_aspectTime: ✅ 1.268µs (SLO: <10.000µs 📉 -87.3%) vs baseline: +0.6% Memory: ✅ 38.515MB (SLO: <41.500MB -7.2%) vs baseline: +5.0% ✅ bytes_noaspectTime: ✅ 0.494µs (SLO: <10.000µs 📉 -95.1%) vs baseline: -0.4% Memory: ✅ 38.579MB (SLO: <41.500MB -7.0%) vs baseline: +4.7% ✅ bytesio_aspectTime: ✅ 1.315µs (SLO: <10.000µs 📉 -86.8%) vs baseline: +1.3% Memory: ✅ 38.603MB (SLO: <41.500MB -7.0%) vs baseline: +4.9% ✅ bytesio_noaspectTime: ✅ 0.502µs (SLO: <10.000µs 📉 -95.0%) vs baseline: ~same Memory: ✅ 38.644MB (SLO: <41.500MB -6.9%) vs baseline: +4.8% ✅ capitalize_aspectTime: ✅ 0.742µs (SLO: <10.000µs 📉 -92.6%) vs baseline: +0.7% Memory: ✅ 38.479MB (SLO: <41.500MB -7.3%) vs baseline: +4.6% ✅ capitalize_noaspectTime: ✅ 0.435µs (SLO: <10.000µs 📉 -95.6%) vs baseline: -1.5% Memory: ✅ 38.519MB (SLO: <41.500MB -7.2%) vs baseline: +4.8% ✅ casefold_aspectTime: ✅ 0.739µs (SLO: <10.000µs 📉 -92.6%) vs baseline: ~same Memory: ✅ 38.557MB (SLO: <41.500MB -7.1%) vs baseline: +4.8% ✅ casefold_noaspectTime: ✅ 0.370µs (SLO: <10.000µs 📉 -96.3%) vs baseline: +0.3% Memory: ✅ 38.571MB (SLO: <41.500MB -7.1%) vs baseline: +4.7% ✅ decode_aspectTime: ✅ 0.724µs (SLO: <10.000µs 📉 -92.8%) vs baseline: +0.5% Memory: ✅ 38.711MB (SLO: <41.500MB -6.7%) vs baseline: +4.9% ✅ decode_noaspectTime: ✅ 0.422µs (SLO: <10.000µs 📉 -95.8%) vs baseline: +0.1% Memory: ✅ 38.608MB (SLO: <41.500MB -7.0%) vs baseline: +4.9% ✅ encode_aspectTime: ✅ 0.710µs (SLO: <10.000µs 📉 -92.9%) vs baseline: -0.6% Memory: ✅ 38.530MB (SLO: <41.500MB -7.2%) vs baseline: +4.6% ✅ encode_noaspectTime: ✅ 0.407µs (SLO: <10.000µs 📉 -95.9%) vs baseline: +0.6% Memory: ✅ 38.521MB (SLO: <41.500MB -7.2%) vs baseline: +4.8% ✅ format_aspectTime: ✅ 3.390µs (SLO: <10.000µs 📉 -66.1%) vs baseline: +0.4% Memory: ✅ 38.673MB (SLO: <41.500MB -6.8%) vs baseline: +5.1% ✅ format_map_aspectTime: ✅ 3.664µs (SLO: <10.000µs 📉 -63.4%) vs baseline: +0.8% Memory: ✅ 38.567MB (SLO: <41.500MB -7.1%) vs baseline: +4.6% ✅ format_map_noaspectTime: ✅ 0.815µs (SLO: <10.000µs 📉 -91.9%) vs baseline: -0.7% Memory: ✅ 38.456MB (SLO: <41.500MB -7.3%) vs baseline: +4.2% ✅ format_noaspectTime: ✅ 0.596µs (SLO: <10.000µs 📉 -94.0%) vs baseline: -0.6% Memory: ✅ 38.608MB (SLO: <41.500MB -7.0%) vs baseline: +5.0% ✅ index_aspectTime: ✅ 0.345µs (SLO: <10.000µs 📉 -96.5%) vs baseline: +1.6% Memory: ✅ 38.619MB (SLO: <41.500MB -6.9%) vs baseline: +5.1% ✅ index_noaspectTime: ✅ 0.316µs (SLO: <10.000µs 📉 -96.8%) vs baseline: -0.3% Memory: ✅ 38.548MB (SLO: <41.500MB -7.1%) vs baseline: +5.0% ✅ join_aspectTime: ✅ 1.296µs (SLO: <10.000µs 📉 -87.0%) vs baseline: -0.3% Memory: ✅ 38.628MB (SLO: <41.500MB -6.9%) vs baseline: +4.8% ✅ join_noaspectTime: ✅ 0.531µs (SLO: <10.000µs 📉 -94.7%) vs baseline: ~same Memory: ✅ 38.653MB (SLO: <41.500MB -6.9%) vs baseline: +4.9% ✅ ljust_aspectTime: ✅ 2.588µs (SLO: <20.000µs 📉 -87.1%) vs baseline: +2.0% Memory: ✅ 38.524MB (SLO: <41.500MB -7.2%) vs baseline: +4.9% ✅ ljust_noaspectTime: ✅ 0.411µs (SLO: <10.000µs 📉 -95.9%) vs baseline: +1.9% Memory: ✅ 38.690MB (SLO: <41.500MB -6.8%) vs baseline: +5.2% ✅ lower_aspectTime: ✅ 2.219µs (SLO: <10.000µs 📉 -77.8%) vs baseline: -0.6% Memory: ✅ 38.572MB (SLO: <41.500MB -7.1%) vs baseline: +5.1% ✅ lower_noaspectTime: ✅ 0.371µs (SLO: <10.000µs 📉 -96.3%) vs baseline: ~same Memory: ✅ 38.565MB (SLO: <41.500MB -7.1%) vs baseline: +4.6% ✅ lstrip_aspectTime: ✅ 2.180µs (SLO: <20.000µs 📉 -89.1%) vs baseline: +0.4% Memory: ✅ 38.679MB (SLO: <41.500MB -6.8%) vs baseline: +4.9% ✅ lstrip_noaspectTime: ✅ 0.388µs (SLO: <10.000µs 📉 -96.1%) vs baseline: +0.5% Memory: ✅ 38.437MB (SLO: <41.500MB -7.4%) vs baseline: +4.4% ✅ modulo_aspectTime: ✅ 0.973µs (SLO: <10.000µs 📉 -90.3%) vs baseline: +0.3% Memory: ✅ 38.640MB (SLO: <41.500MB -6.9%) vs baseline: +5.1% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 1.491µs (SLO: <10.000µs 📉 -85.1%) vs baseline: -0.1% Memory: ✅ 38.622MB (SLO: <41.500MB -6.9%) vs baseline: +4.5% ✅ modulo_aspect_for_bytesTime: ✅ 0.947µs (SLO: <10.000µs 📉 -90.5%) vs baseline: -0.9% Memory: ✅ 38.590MB (SLO: <41.500MB -7.0%) vs baseline: +4.8% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 1.161µs (SLO: <10.000µs 📉 -88.4%) vs baseline: -1.2% Memory: ✅ 38.571MB (SLO: <41.500MB -7.1%) vs baseline: +4.9% ✅ modulo_noaspectTime: ✅ 0.675µs (SLO: <10.000µs 📉 -93.2%) vs baseline: +1.1% Memory: ✅ 38.583MB (SLO: <41.500MB -7.0%) vs baseline: +4.8% ✅ replace_aspectTime: ✅ 5.057µs (SLO: <10.000µs 📉 -49.4%) vs baseline: +2.9% Memory: ✅ 38.500MB (SLO: <41.500MB -7.2%) vs baseline: +4.7% ✅ replace_noaspectTime: ✅ 0.465µs (SLO: <10.000µs 📉 -95.4%) vs baseline: +0.6% Memory: ✅ 38.647MB (SLO: <41.500MB -6.9%) vs baseline: +4.9% ✅ repr_aspectTime: ✅ 0.952µs (SLO: <10.000µs 📉 -90.5%) vs baseline: +0.4% Memory: ✅ 38.674MB (SLO: <41.500MB -6.8%) vs baseline: +4.7% ✅ repr_noaspectTime: ✅ 0.455µs (SLO: <10.000µs 📉 -95.5%) vs baseline: +0.7% Memory: ✅ 38.597MB (SLO: <41.500MB -7.0%) vs baseline: +4.8% ✅ rstrip_aspectTime: ✅ 1.832µs (SLO: <20.000µs 📉 -90.8%) vs baseline: -0.3% Memory: ✅ 38.530MB (SLO: <41.500MB -7.2%) vs baseline: +5.1% ✅ rstrip_noaspectTime: ✅ 0.384µs (SLO: <10.000µs 📉 -96.2%) vs baseline: ~same Memory: ✅ 38.479MB (SLO: <41.500MB -7.3%) vs baseline: +4.7% ✅ slice_aspectTime: ✅ 0.490µs (SLO: <10.000µs 📉 -95.1%) vs baseline: +0.9% Memory: ✅ 38.583MB (SLO: <41.500MB -7.0%) vs baseline: +4.9% ✅ slice_noaspectTime: ✅ 0.449µs (SLO: <10.000µs 📉 -95.5%) vs baseline: -0.1% Memory: ✅ 38.511MB (SLO: <41.500MB -7.2%) vs baseline: +4.7% ✅ stringio_aspectTime: ✅ 1.675µs (SLO: <10.000µs 📉 -83.2%) vs baseline: -0.9% Memory: ✅ 38.615MB (SLO: <41.500MB -7.0%) vs baseline: +4.4% ✅ stringio_noaspectTime: ✅ 0.919µs (SLO: <10.000µs 📉 -90.8%) vs baseline: +0.6% Memory: ✅ 38.566MB (SLO: <41.500MB -7.1%) vs baseline: +4.6% ✅ strip_aspectTime: ✅ 2.395µs (SLO: <20.000µs 📉 -88.0%) vs baseline: 📈 +10.2% Memory: ✅ 38.424MB (SLO: <41.500MB -7.4%) vs baseline: +4.8% ✅ strip_noaspectTime: ✅ 0.386µs (SLO: <10.000µs 📉 -96.1%) vs baseline: +1.1% Memory: ✅ 38.640MB (SLO: <41.500MB -6.9%) vs baseline: +4.5% ✅ swapcase_aspectTime: ✅ 2.416µs (SLO: <10.000µs 📉 -75.8%) vs baseline: -1.2% Memory: ✅ 38.553MB (SLO: <41.500MB -7.1%) vs baseline: +4.6% ✅ swapcase_noaspectTime: ✅ 0.539µs (SLO: <10.000µs 📉 -94.6%) vs baseline: +0.4% Memory: ✅ 38.606MB (SLO: <41.500MB -7.0%) vs baseline: +4.8% ✅ title_aspectTime: ✅ 2.377µs (SLO: <10.000µs 📉 -76.2%) vs baseline: +0.3% Memory: ✅ 38.385MB (SLO: <41.500MB -7.5%) vs baseline: +4.2% ✅ title_noaspectTime: ✅ 0.509µs (SLO: <10.000µs 📉 -94.9%) vs baseline: +0.3% Memory: ✅ 38.456MB (SLO: <41.500MB -7.3%) vs baseline: +4.8% ✅ translate_aspectTime: ✅ 3.254µs (SLO: <10.000µs 📉 -67.5%) vs baseline: -0.3% Memory: ✅ 38.511MB (SLO: <41.500MB -7.2%) vs baseline: +4.8% ✅ translate_noaspectTime: ✅ 1.052µs (SLO: <10.000µs 📉 -89.5%) vs baseline: +0.9% Memory: ✅ 38.656MB (SLO: <41.500MB -6.9%) vs baseline: +5.2% ✅ upper_aspectTime: ✅ 2.426µs (SLO: <10.000µs 📉 -75.7%) vs baseline: +6.1% Memory: ✅ 38.397MB (SLO: <41.500MB -7.5%) vs baseline: +4.0% ✅ upper_noaspectTime: ✅ 0.374µs (SLO: <10.000µs 📉 -96.3%) vs baseline: +1.0% Memory: ✅ 38.592MB (SLO: <41.500MB -7.0%) vs baseline: +4.9% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.209µs (SLO: <10.000µs 📉 -47.9%) vs baseline: 📈 +23.3% Memory: ✅ 38.633MB (SLO: <41.000MB -5.8%) vs baseline: +4.7% ✅ ospathbasename_noaspectTime: ✅ 1.080µs (SLO: <10.000µs 📉 -89.2%) vs baseline: -1.0% Memory: ✅ 38.633MB (SLO: <41.000MB -5.8%) vs baseline: +5.4% ✅ ospathjoin_aspectTime: ✅ 6.104µs (SLO: <10.000µs 📉 -39.0%) vs baseline: +1.3% Memory: ✅ 38.457MB (SLO: <41.000MB -6.2%) vs baseline: +4.4% ✅ ospathjoin_noaspectTime: ✅ 2.286µs (SLO: <10.000µs 📉 -77.1%) vs baseline: +0.2% Memory: ✅ 38.594MB (SLO: <41.000MB -5.9%) vs baseline: +4.8% ✅ ospathnormcase_aspectTime: ✅ 3.459µs (SLO: <10.000µs 📉 -65.4%) vs baseline: -1.6% Memory: ✅ 38.633MB (SLO: <41.000MB -5.8%) vs baseline: +5.0% ✅ ospathnormcase_noaspectTime: ✅ 0.571µs (SLO: <10.000µs 📉 -94.3%) vs baseline: -1.1% Memory: ✅ 38.633MB (SLO: <41.000MB -5.8%) vs baseline: +5.3% ✅ ospathsplit_aspectTime: ✅ 4.868µs (SLO: <10.000µs 📉 -51.3%) vs baseline: +0.7% Memory: ✅ 38.555MB (SLO: <41.000MB -6.0%) vs baseline: +4.5% ✅ ospathsplit_noaspectTime: ✅ 1.594µs (SLO: <10.000µs 📉 -84.1%) vs baseline: ~same Memory: ✅ 38.633MB (SLO: <41.000MB -5.8%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 3.691µs (SLO: <10.000µs 📉 -63.1%) vs baseline: -1.0% Memory: ✅ 38.653MB (SLO: <41.000MB -5.7%) vs baseline: +4.8% ✅ ospathsplitdrive_noaspectTime: ✅ 0.695µs (SLO: <10.000µs 📉 -93.1%) vs baseline: -0.4% Memory: ✅ 38.594MB (SLO: <41.000MB -5.9%) vs baseline: +4.6% ✅ ospathsplitext_aspectTime: ✅ 4.586µs (SLO: <10.000µs 📉 -54.1%) vs baseline: -0.6% Memory: ✅ 38.555MB (SLO: <41.000MB -6.0%) vs baseline: +4.8% ✅ ospathsplitext_noaspectTime: ✅ 1.374µs (SLO: <10.000µs 📉 -86.3%) vs baseline: -0.8% Memory: ✅ 38.614MB (SLO: <41.000MB -5.8%) vs baseline: +5.4% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.499µs (SLO: <20.000µs 📉 -82.5%) vs baseline: 📈 +15.7% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +5.0% ✅ 1-count-metrics-100-timesTime: ✅ 206.480µs (SLO: <220.000µs -6.1%) vs baseline: +0.7% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.6% ✅ 1-distribution-metric-1-timesTime: ✅ 3.386µs (SLO: <20.000µs 📉 -83.1%) vs baseline: +0.2% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ 1-distribution-metrics-100-timesTime: ✅ 219.608µs (SLO: <230.000µs -4.5%) vs baseline: +0.2% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.9% ✅ 1-gauge-metric-1-timesTime: ✅ 2.201µs (SLO: <20.000µs 📉 -89.0%) vs baseline: +0.7% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +5.0% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.144µs (SLO: <150.000µs -9.2%) vs baseline: -0.5% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +5.3% ✅ 1-rate-metric-1-timesTime: ✅ 3.189µs (SLO: <20.000µs 📉 -84.1%) vs baseline: -0.7% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.7% ✅ 1-rate-metrics-100-timesTime: ✅ 218.613µs (SLO: <250.000µs 📉 -12.6%) vs baseline: +0.2% Memory: ✅ 34.780MB (SLO: <35.500MB -2.0%) vs baseline: +4.6% ✅ 100-count-metrics-100-timesTime: ✅ 20.654ms (SLO: <22.000ms -6.1%) vs baseline: -0.7% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.3% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.292ms (SLO: <2.550ms 📉 -10.1%) vs baseline: -0.7% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.405ms (SLO: <1.550ms -9.4%) vs baseline: +0.2% Memory: ✅ 34.957MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.2% ✅ 100-rate-metrics-100-timesTime: ✅ 2.255ms (SLO: <2.550ms 📉 -11.6%) vs baseline: -0.7% Memory: ✅ 34.741MB (SLO: <35.500MB -2.1%) vs baseline: +4.8% ✅ flush-1-metricTime: ✅ 4.637µs (SLO: <20.000µs 📉 -76.8%) vs baseline: -0.5% Memory: ✅ 35.212MB (SLO: <35.500MB 🟡 -0.8%) vs baseline: +4.9% ✅ flush-100-metricsTime: ✅ 173.350µs (SLO: <250.000µs 📉 -30.7%) vs baseline: -0.6% Memory: ✅ 35.212MB (SLO: <35.500MB 🟡 -0.8%) vs baseline: +5.2% ✅ flush-1000-metricsTime: ✅ 2.193ms (SLO: <2.500ms 📉 -12.3%) vs baseline: +0.7% Memory: ✅ 35.979MB (SLO: <36.500MB 🟡 -1.4%) vs baseline: +5.1% 🟡 Near SLO Breach (17 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
Description
Fixes a bug where custom locks crash when inheriting from a wrapped lock type, when the profiler is enabled. In this case, user imported a third-party libraries
neo4jwhich defines a classAsyncRLockinheritingasyncio.Lock(which we wrap).A customer using neo4j v6.0.3 with ddtrace v4.0.0 reported their application failing to start:
The error occurred because neo4j's
AsyncRLockclass subclassesasyncio.Lock:Root Cause
When the profiler is enabled, it replaces lock classes (e.g.,
asyncio.Lock,threading.Lock) with_LockAllocatorWrapperinstances to track lock usage. However,_LockAllocatorWrapperdidn't implement__mro_entries__(PEP 560), which Python requires when a non-type object appears in a class's bases.When Python evaluates
class AsyncRLock(asyncio.Lock):andasyncio.Lockis a_LockAllocatorWrapperinstance (not a type), it tries to use the wrapper directly as a metaclass, leading to theTypeError.Interestingly that using
wraptwould not have prevented this bug, as it doesn't implement the__mro_entries__either; so, removing this dependency didn't cause the issue.Fix
Implement
__mro_entries__on_LockAllocatorWrapperto support PEP 560. This tells Python to use the original lock class as the actual base when subclassing.Testing
test_subclassing_wrapped_lockMAIN branch - FAIL
TypeError: _LockAllocatorWrapper.__init__() takes from 2 to 3 positional arguments but 4 were givenPR branch - PASS
References