Skip to content

Commit f23eb40

Browse files
taegyunkimtaegyunkim
authored
chore(claude/skills): cpython skills (#15588)
## Description For [feat(profiling): support Python 3.14](#15546), I first gathered how we use CPython internal headers/structs, and then compared how they changed from 3.13.0 to 3.14.0. Then, generated an action plan based off on these. This PR is to have those two skills available so that we can follow similar steps when adding support for newer versions of Python, 3.15+, and free-threading Python. <!-- Provide an overview of the change and motivation for the change --> ## Testing <!-- Describe your testing strategy or note what tests are included --> ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes <!-- Any other information that would be helpful for reviewers -->
1 parent bff4abd commit f23eb40

File tree

4 files changed

+364
-2
lines changed

4 files changed

+364
-2
lines changed

‎.claude/settings.local.json‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,10 @@
3535
"mcp__github__get_pull_request_review_comments",
3636
"mcp__github__pull_request_read",
3737
"Skill(run-tests)",
38-
"Skill(lint)"
38+
"Skill(lint)",
39+
"Skill(find-cpython-usage)",
40+
"Skill(compare-cpython-versions)"
3941
],
4042
"deny": []
4143
}
42-
}
44+
}

‎.claude/skills/compare-cpython-versions/SKILL.md‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
---
2+
name: compare-cpython-versions
3+
description: >
4+
Compare CPython source code between two Python versions to identify changes in
5+
headers and structs. Use this when adding support for a new Python version to
6+
understand what changed between versions.
7+
allowed-tools:
8+
- Bash
9+
- Read
10+
- Grep
11+
- WebSearch
12+
- TodoWrite
13+
---
14+
15+
# Compare CPython Versions Skill
16+
17+
This skill helps compare CPython source code between two Python versions to identify
18+
changes in headers, structs, and APIs that affect our codebase.
19+
20+
## When to Use This Skill
21+
22+
Use this skill when:
23+
- Adding support for a new Python version
24+
- Need to understand what changed between versions
25+
- Investigating compatibility issues
26+
- Have a list of headers/structs from `find-cpython-usage` skill
27+
28+
## Key Principles
29+
30+
1. **Compare systematically** - Focus on headers and structs identified in Step 1
31+
2. **Use multiple methods** - Git diff, manual diff, or AI-assisted comparison
32+
3. **Document changes** - Note all breaking changes and API modifications
33+
4. **Check context** - Understand why changes were made (PEPs, GitHub issues)
34+
35+
## How This Skill Works
36+
37+
### Step 1: Prepare CPython Repository
38+
39+
```bash
40+
# Create ~/dd directory if it doesn't exist
41+
mkdir -p ~/dd
42+
43+
# Clone CPython repository if needed (to ~/dd/cpython)
44+
if [ ! -d ~/dd/cpython ]; then
45+
git clone https://github.com/python/cpython.git ~/dd/cpython
46+
cd ~/dd/cpython
47+
git fetch --tags
48+
else
49+
cd ~/dd/cpython
50+
# Update existing repository
51+
git fetch --tags
52+
git fetch origin
53+
fi
54+
```
55+
56+
### Step 2: Compare Specific Headers
57+
58+
Using the list of headers from `find-cpython-usage`, compare each header between
59+
the old version and new version. Replace `OLD_VERSION` and `NEW_VERSION` with the
60+
actual version tags (e.g., `v3.13.0`, `v3.14.0`):
61+
62+
```bash
63+
# Compare specific headers between versions
64+
git diff OLD_VERSION NEW_VERSION -- Include/internal/pycore_frame.h
65+
git diff OLD_VERSION NEW_VERSION -- Include/frameobject.h
66+
67+
# Compare all internal headers
68+
git diff OLD_VERSION NEW_VERSION -- 'Include/internal/pycore*.h'
69+
70+
# Compare specific struct definitions
71+
git diff OLD_VERSION NEW_VERSION -- Include/internal/pycore_frame.h | grep -A 20 "struct _PyInterpreterFrame"
72+
```
73+
74+
75+
### Step 3: Identify Changes
76+
77+
For each header/struct, look for:
78+
79+
**Struct Changes:**
80+
- Field additions
81+
- Field removals
82+
- Field type changes
83+
- Field reordering
84+
- Struct moves to different headers
85+
86+
**API Changes:**
87+
- Removed functions/structures
88+
- New functions/structures
89+
- Changed function signatures
90+
- Deprecated APIs
91+
92+
**Header Changes:**
93+
- Headers moved to different locations
94+
- Headers split or merged
95+
- New headers introduced
96+
97+
### Step 4: Analyze Impact
98+
99+
For each change identified:
100+
101+
1. **Understand the change:**
102+
- Why was it changed? (Check Python's What's New, PEPs, or GitHub issues)
103+
- Is it a breaking change or backward compatible?
104+
- What's the replacement API?
105+
- **Find the specific commit(s) that introduced the change:**
106+
107+
```bash
108+
# Find commits that modified a specific file between versions
109+
git log OLD_VERSION..NEW_VERSION -- Include/internal/pycore_frame.h
110+
111+
# Find commits that mention a specific struct or function
112+
git log OLD_VERSION..NEW_VERSION --all --grep="_PyInterpreterFrame" -- Include/
113+
114+
# Show the commit that introduced a specific change
115+
git log -p OLD_VERSION..NEW_VERSION -S "struct _PyInterpreterFrame" -- Include/
116+
```
117+
118+
- **Find related GitHub issues:**
119+
- Check commit messages for issue references (e.g., `gh-123923`, `#123923`)
120+
- Search CPython GitHub issues: `https://github.com/python/cpython/issues`
121+
- Look for "What's New" documentation: `https://docs.python.org/3/whatsnew/`
122+
- Check PEPs if the change is part of a larger feature
123+
124+
2. **Assess impact:**
125+
- Which files in our codebase are affected?
126+
- What functionality might break?
127+
- Are there alternative approaches?
128+
129+
3. **Document findings:**
130+
- Create a summary document of key changes
131+
- Note any breaking changes
132+
- List files that need updates
133+
134+
### Step 5: Use AI Tools (Optional)
135+
136+
You can use AI coding assistants to help analyze differences by:
137+
- Providing header file contents from both versions
138+
- Asking about specific struct changes
139+
- Understanding migration paths
140+
141+
## Common Change Patterns
142+
143+
When comparing versions, look for these types of changes (examples):
144+
145+
**Struct Field Changes:**
146+
- Field type changes (e.g., pointer types → tagged pointer types)
147+
- Field renamed
148+
- Field removed and replaced with different mechanism
149+
- Field reordering
150+
151+
**Header Moves:**
152+
- Internal headers moved to new locations
153+
- Structs moved between headers
154+
- Headers split or merged
155+
156+
**API Deprecations:**
157+
- Internal functions removed
158+
- Public API replacements available
159+
- Function signature changes
160+
161+
## Output Format
162+
163+
After running this skill, you should have:
164+
1. A list of all changed headers
165+
2. A list of all changed structs with details
166+
3. Impact assessment for each change
167+
4. Files in our codebase that need updates
168+
169+
## Related
170+
171+
- **find-cpython-usage skill**: Use to identify what to compare

‎.claude/skills/find-cpython-usage/SKILL.md‎

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
---
2+
name: find-cpython-usage
3+
description: >
4+
Find all CPython internal headers and structs used in the codebase, particularly
5+
for profiling functionality. Use this when adding support for a new Python version
6+
to identify what CPython internals we depend on.
7+
allowed-tools:
8+
- Bash
9+
- Read
10+
- Grep
11+
- Glob
12+
- TodoWrite
13+
---
14+
15+
# Find CPython Internal Usage Skill
16+
17+
This skill helps identify all CPython internal headers and structures used in the
18+
codebase, which is essential when adding support for new Python versions.
19+
20+
## When to Use This Skill
21+
22+
Use this skill when:
23+
- Adding support for a new Python version
24+
- Investigating CPython API dependencies
25+
- Understanding what internal APIs the profiler uses
26+
- Preparing to compare CPython versions
27+
28+
## Key Principles
29+
30+
1. **Focus on internal headers** - These are most likely to change between versions
31+
2. **Check all native extensions** - CPython internals are used in profiling, AppSec, and internal modules
32+
3. **Look for struct field access** - Direct field access is version-sensitive
33+
4. **Document findings** - Keep track of what you find for comparison
34+
35+
## How This Skill Works
36+
37+
### Step 1: Find CPython Header Includes
38+
39+
Search for CPython header includes across all C/C++/Cython files:
40+
41+
```bash
42+
# Find all CPython internal header includes
43+
grep -r "include.*internal/pycore" ddtrace/ --include="*.c" --include="*.cpp" --include="*.h" --include="*.hpp" --include="*.pyx"
44+
45+
# Find all CPython cpython header includes
46+
grep -r "include.*cpython" ddtrace/ --include="*.c" --include="*.cpp" --include="*.h" --include="*.hpp" --include="*.pyx"
47+
48+
# Find all Python.h includes (indicates CPython API usage)
49+
grep -r "#include.*Python\.h" ddtrace/ --include="*.c" --include="*.cpp" --include="*.h" --include="*.hpp" --include="*.pyx"
50+
51+
# Find frameobject.h includes (common CPython API)
52+
grep -r "#include.*frameobject\.h" ddtrace/ --include="*.c" --include="*.cpp" --include="*.h" --include="*.hpp" --include="*.pyx"
53+
54+
# Find PyO3 FFI usage in Rust (Rust extensions may use CPython internals)
55+
grep -r "pyo3_ffi::\|pyo3::ffi::" src/native/ --include="*.rs" || true
56+
```
57+
58+
**Note:** These patterns search across all native extension files (`.c`, `.cpp`, `.h`, `.hpp`, `.pyx`, `.rs`)
59+
regardless of their location in the codebase. Check `setup.py` to see which extensions are built.
60+
Rust extensions use PyO3 which may access CPython internals through the `pyo3_ffi` module.
61+
62+
### Step 2: Find Struct Field Access
63+
64+
Search for direct struct field access and struct definitions across all native files:
65+
66+
```bash
67+
# Find struct field accesses (arrow operator)
68+
grep -r "->f_\|->[a-z_]*\." ddtrace/ --include="*.c" --include="*.cpp" --include="*.h" --include="*.hpp"
69+
70+
# Find struct definitions
71+
grep -r "struct.*Py" ddtrace/ --include="*.c" --include="*.cpp" --include="*.h" --include="*.hpp"
72+
73+
# Find common CPython struct usage
74+
grep -r "PyFrameObject\|PyThreadState\|_PyInterpreterFrame" ddtrace/ --include="*.c" --include="*.cpp" --include="*.h" --include="*.hpp" --include="*.pyx"
75+
grep -r "PyFrameObject\|PyThreadState\|PyInterpreterState" src/native/ --include="*.rs" || true
76+
77+
# Find PyCodeObject usage
78+
grep -r "PyCodeObject" ddtrace/ --include="*.c" --include="*.cpp" --include="*.h" --include="*.hpp" --include="*.pyx"
79+
grep -r "PyCodeObject" src/native/ --include="*.rs" || true
80+
```
81+
82+
**Look for patterns like:**
83+
- Frame structure access (`PyFrameObject`, `_PyInterpreterFrame`)
84+
- Thread state access (`PyThreadState`)
85+
- Code object access (`PyCodeObject`)
86+
- Generator/coroutine structures
87+
- Asyncio task structures
88+
89+
### Step 3: Identify Common Structures
90+
91+
Common CPython structures we typically access:
92+
93+
**Frame structures:**
94+
- `PyFrameObject` / `struct _frame`
95+
- `_PyInterpreterFrame`
96+
97+
**State structures:**
98+
- `PyThreadState`
99+
- `PyInterpreterState`
100+
- `_PyRuntimeState`
101+
102+
**Code structures:**
103+
- `PyCodeObject`
104+
105+
**Generator structures:**
106+
- `PyGenObject`
107+
- `PyAsyncGenASend`
108+
109+
**Asyncio structures:**
110+
- `FutureObj`
111+
- `TaskObj`
112+
113+
### Step 4: Document Findings
114+
115+
Create a list of:
116+
- All headers that are included
117+
- All structs that are accessed
118+
- All struct fields that are used directly
119+
120+
This will be used in the next step to compare against the new Python version.
121+
122+
## Native Extensions Using CPython APIs
123+
124+
To find all native extensions that may use CPython APIs, check `setup.py`:
125+
126+
```bash
127+
# View all native extensions defined in setup.py
128+
grep -A 5 "Extension\|CMakeExtension\|Cython.Distutils.Extension\|RustExtension" setup.py
129+
```
130+
131+
The `setup.py` file defines all native extensions (C, C++, CMake, Cython, and Rust) that
132+
are built for the project. Not all extensions use CPython internals - focus on those
133+
that access frame objects, thread state, or internal structures when searching for
134+
CPython API usage.
135+
136+
**Note:** Rust extensions use PyO3 bindings which may access CPython internals through
137+
`pyo3_ffi` module. Search Rust source files (`.rs`) for CPython API usage as well.
138+
139+
## Common Headers to Look For
140+
141+
The grep commands above will identify which CPython headers are actually used in the codebase.
142+
Common patterns include:
143+
144+
**Public Headers:**
145+
- Headers matching `*.h` in `Include/` directory (e.g., `frameobject.h`, `unicodeobject.h`)
146+
- Headers in `Include/cpython/` directory (e.g., `cpython/genobject.h`)
147+
148+
**Internal Headers (require `Py_BUILD_CORE`):**
149+
- Headers matching `internal/pycore*.h` pattern
150+
- These are most likely to change between Python versions
151+
152+
Focus on headers that are actually found by the grep commands rather than maintaining
153+
a hardcoded list, as the headers used may change over time.
154+
155+
## Output Format
156+
157+
After running this skill, you should have:
158+
1. A list of all CPython headers included in the codebase
159+
2. A list of all CPython structs accessed
160+
3. A list of struct fields accessed directly
161+
4. Files that use each header/struct
162+
163+
This information can then be used with the `compare-cpython-versions` skill to identify what changed.
164+
165+
## Related
166+
167+
- **compare-cpython-versions skill**: Use findings from this skill to compare versions

‎AGENTS.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,28 @@ This project has custom skills that provide specialized workflows. **Always chec
7474

7575
**Usage:** Use the Skill tool with command "lint"
7676

77+
### find-cpython-usage
78+
79+
**Use whenever:** Adding support for a new Python version or investigating CPython API dependencies.
80+
81+
**Purpose:** Identifies all CPython internal headers and structures used in the codebase:
82+
- Searches for CPython header includes
83+
- Finds struct field accesses
84+
- Documents CPython internals we depend on
85+
86+
**Usage:** Use the Skill tool with command "find-cpython-usage"
87+
88+
### compare-cpython-versions
89+
90+
**Use whenever:** Comparing CPython source code between two Python versions to identify changes.
91+
92+
**Purpose:** Compares CPython headers and structs between versions:
93+
- Uses git diff or manual diff to compare versions
94+
- Identifies breaking changes and API modifications
95+
- Assesses impact on our codebase
96+
97+
**Usage:** Use the Skill tool with command "compare-cpython-versions"
98+
7799
---
78100

79101
<!-- Add more skills below as they are created -->

0 commit comments

Comments
 (0)