[tools/msbuild] Add InlineDlfcnMethodsStep as an opt-in custom linker step#24888
[tools/msbuild] Add InlineDlfcnMethodsStep as an opt-in custom linker step#24888rolfbjarne wants to merge 2 commits intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an opt-in build+linker pipeline to inline ObjCRuntime.Dlfcn symbol lookups into direct __Internal P/Invokes, with post-trimming symbol discovery (ILTrim + NativeAOT) and native C code generation to keep only surviving symbol references.
Changes:
- Introduces
InlineDlfcnMethodsStep+GenerateInlinedDlfcnNativeCodeStepand correspondingInlineDlfcnMethodsMSBuild property wiring. - Adds post-trimming MSBuild tasks/targets to detect surviving inlined symbols (ILTrim scanning + NativeAOT unresolved-symbol parsing) and generate/compile native glue code.
- Extends Mach-O parsing to read unresolved symbols and adds/adjusts tests and test variations for the new modes.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/mtouch/Errors.resx | Adds new MX2254–MX2257 warning strings for the inlining step. |
| tools/mtouch/Errors.designer.cs | Regenerated resource accessors for new MX225x entries. |
| tools/linker/MonoTouch.Tuner/ProcessExportedFields.cs | Collects [Field] symbol names for compatibility mode. |
| tools/linker/MonoTouch.Tuner/ListExportedSymbols.cs | Skips adding required field symbols when dlfcn inlining is enabled. |
| tools/dotnet-linker/Steps/InlineDlfcnMethodsStep.cs | New custom linker step that rewrites Dlfcn calls into generated helpers + P/Invokes. |
| tools/dotnet-linker/Steps/GenerateInlinedDlfcnNativeCodeStep.cs | Emits a symbol list file for later post-trimming MSBuild steps. |
| tools/dotnet-linker/LinkerConfiguration.cs | Adds InlineDlfcnMethods configuration and symbol tracking collections. |
| tools/dotnet-linker/AppBundleRewriter.cs | Adds type/method references used by the new inlining step. |
| tools/common/MachO.cs | Adds LC_SYMTAB parsing and unresolved-symbol extraction for Mach-O/static libraries. |
| tests/xharness/Jenkins/TestVariationsFactory.cs | Adds new CI variations to exercise inline-dlfcn modes (incl. NativeAOT). |
| tests/monotouch-test/ObjCRuntime/DlfcnTest.cs | Adds tests for [Field]-backed constants and adjusts behavior under strict mode. |
| tests/monotouch-test/Foundation/AttributedStringTest.cs | Removes legacy field-absence test that would conflict with new behavior. |
| tests/monotouch-test/CoreFoundation/ProxyTest.cs | Removes legacy field-absence test that would conflict with new behavior. |
| tests/common/test-variations.csproj | Adds inline-dlfcn-methods-compat and inline-dlfcn-methods-strict test variations. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/PostTrimmingProcessing.cs | Generates native C glue for surviving symbols and outputs compile/link items. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/ComputeNativeAOTSurvivingNativeSymbols.cs | Filters NativeAOT unresolved symbols to determine surviving inlined-dlfcn symbols. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/CompileNativeCode.cs | Ensures output directory exists before compiling native files. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/CollectUnresolvedNativeSymbols.cs | New task to extract unresolved symbols from a NativeAOT static library. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/CollectPostILTrimInformation.cs | New task to scan trimmed assemblies and cache surviving inlined-dlfcn P/Invokes. |
| dotnet/targets/Xamarin.Shared.Sdk.targets | Wires property defaults, custom linker steps, and post-trimming targets into the SDK. |
| docs/code/native-symbols.md | New doc explaining the native symbol pipeline and InlineDlfcnMethods modes. |
| docs/building-apps/build-properties.md | Documents the new InlineDlfcnMethods MSBuild property and defaults. |
| .github/copilot-instructions.md | Adds MSBuild guidance (DeviceSpecificIntermediateOutputPath) to repo copilot instructions. |
Files not reviewed (1)
- tools/mtouch/Errors.designer.cs: Language not supported
You can also share your feedback on Copilot code review. Take the survey.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
1 similar comment
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Add a new custom linker step that replaces runtime `Dlfcn.dlsym` calls with direct native symbol lookups via P/Invokes to `__Internal`. This improves startup performance and app size, and ensures the native linker can see symbol references that would otherwise only be resolved at runtime. Two modes are supported, controlled by the `InlineDlfcnMethods` MSBuild property: * `strict`: inlines all calls to `ObjCRuntime.Dlfcn` APIs. * `compatibility`: only inlines calls that reference symbols from `[Field]` attributes. Post-trimming MSBuild targets ensure native code is only generated for symbols that survive trimming (ILTrim or NativeAOT): * `_CollectPostILTrimInformation`: scans trimmed assemblies for surviving inlined dlfcn P/Invokes, with per-assembly caching for incremental builds. * `_CollectPostNativeAOTTrimInformation`: reads unresolved symbols from the NativeAOT object file or static library and filters for inlined dlfcn symbols. * `_PostTrimmingProcessing`: generates native C code for surviving symbols and adds it to the native link inputs. Extends `MachO.cs` with LC_SYMTAB parsing to read unresolved symbols from both Mach-O object files and static libraries. Contributes towards #17693 (even though this adds a *new* custom trimmer step, it's a pre-marking step that's easy to move out of the linker later on, and it's a partial replacement for the ListExportedSymbols step, which can't be moved out of the trimmer).
1e92a98 to
4d6b334
Compare
|
/review |
|
✅ .NET for Apple Platforms PR Reviewer completed successfully! |
There was a problem hiding this comment.
Code Review Summary
This PR adds a sophisticated optimization to inline Dlfcn calls at build time, replacing runtime dlsym lookups with direct P/Invokes to native symbols. The implementation is well-structured with comprehensive documentation and test coverage.
Issues Found
❌ 1 Critical Bug:
PostTrimmingProcessing.csline 54: Wrong variable in switch statement (symbolModeinstead ofsymbolType) causing incorrect symbol filtering logic.
- Missing input validation in
CollectPostILTrimInformation.csandComputeNativeAOTSurvivingNativeSymbols.csfor symbol name extraction that could throw exceptions.
💡 1 Optimization:
- Unnecessary
.Where (s => s.Length > 0)filter inPostTrimmingProcessing.cs.
Positive Highlights
✅ Excellent documentation in docs/code/native-symbols.md explaining the architecture
✅ Proper incremental build support with per-assembly caching
✅ Comprehensive test coverage including edge cases
✅ Well-structured MSBuild integration with proper Inputs/Outputs attributes
✅ Clean separation of concerns between IL transformation and post-trimming processing
✅ Proper nullable reference type usage throughout
Architecture Review
The two-phase approach (IL transformation during linking + post-trimming native code generation) is sound and handles both ILTrim and NativeAOT scenarios elegantly. The MachO.cs symbol table parsing implementation looks correct and properly handles both 32-bit and 64-bit Mach-O files.
The critical bug in PostTrimmingProcessing.cs must be fixed before merge, as it breaks the ReferenceNativeSymbol ignore functionality.
Generated by .NET for Apple Platforms PR Reviewer for issue #24888 · ● 1.1M
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
- Fix switch(symbolMode) -> switch(symbolType) in PostTrimmingProcessing.cs - Fix instr.Previous -> ins.Previous in 3-arg Dlfcn pattern matcher - Fix MachO.cs ar padding: % 1 -> % 2 - Fix typo 'qas' -> 'was' in Frameworks.cs - Fix #endif comment to match #if BUNDLER - Replace null! with TaskItem default in CollectUnresolvedNativeSymbols - Add Substring length validation in CollectPostILTrimInformation and ComputeNativeAOTSurvivingNativeSymbols - Use dlsym(RTLD_DEFAULT, ...) instead of extern void* for generated C code, which works for both function and data symbols - Remove unnecessary .Where(s => s.Length > 0) filter - Add IsValidCIdentifier check to skip symbols with invalid chars Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
✅ [PR Build #703de17] Build passed (Build packages) ✅Pipeline on Agent |
✅ [PR Build #703de17] Build passed (Detect API changes) ✅Pipeline on Agent |
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
✅ [CI Build #703de17] Build passed (Build macOS tests) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
🚀 [CI Build #703de17] Test results 🚀Test results✅ All tests passed on VSTS: test results. 🎉 All 162 tests passed 🎉 Tests counts✅ cecil: All 1 tests passed. Html Report (VSDrops) Download macOS tests✅ Tests on macOS Monterey (12): All 5 tests passed. Html Report (VSDrops) Download Linux Build VerificationPipeline on Agent |
|
/review |
|
|
||
| The valid options are: | ||
|
|
||
| * `compatibility`: Inlines dlfcn method calls but only creates native references for symbols used in `[Field]` attributes. This is more conservative and avoids link errors for symbols that don't exist at build time. |
There was a problem hiding this comment.
The compatibility bullet says it “inlines dlfcn method calls but only creates native references for symbols used in [Field] attributes”, but the implementation in InlineDlfcnMethodsStep actually skips inlining entirely for symbols not coming from [Field] attributes (in compatibility mode). Please adjust this description to match the actual behavior (only [Field]-backed symbol usages are inlined in compatibility mode).
| * `compatibility`: Inlines dlfcn method calls but only creates native references for symbols used in `[Field]` attributes. This is more conservative and avoids link errors for symbols that don't exist at build time. | |
| * `compatibility`: Only inlines symbol usages backed by `[Field]` attributes. This is more conservative and avoids link errors for symbols that don't exist at build time. |
| <PostTrimmingProcessing | ||
| Architecture="$(TargetArchitectures)" | ||
| OutputDirectory="$(DeviceSpecificIntermediateOutputPath)inlined-dlfcn" | ||
| ReferenceNativeSymbol="@(ReferenceNativeSymbol)" | ||
| SurvivingNativeSymbolsFiles="@(_SurvivingNativeSymbolsFile)" |
There was a problem hiding this comment.
Architecture="$(TargetArchitectures)" may be a comma-separated list (often with spaces, e.g. "ARM64, x86_64"), but PostTrimmingProcessing/CompileNativeCode expect a single Arch value per compilation. As-is this can result in invalid -arch arguments. Consider splitting $(TargetArchitectures) into per-arch items and invoking PostTrimmingProcessing per arch (or having the task emit one output item per arch).
| <ItemGroup> | ||
| <FileWrites Include="@(_PostTrimmingSourceFiles)" /> | ||
| <_PostTrimmingSourceFiles> | ||
| <OutputFile>$(DeviceSpecificIntermediateOutputPath)posttrim-info-compiled/%(Filename).o</OutputFile> |
There was a problem hiding this comment.
OutputFile for the post-trimming generated source is set to a single %(Filename).o. If you compile the same inlined-dlfcn.c for multiple architectures, this will cause object-file collisions/overwrites. Ensure the output path is unique per architecture (for example include %(Arch) in the path/name) if multi-arch compilation is supported here.
| <OutputFile>$(DeviceSpecificIntermediateOutputPath)posttrim-info-compiled/%(Filename).o</OutputFile> | |
| <OutputFile>$(DeviceSpecificIntermediateOutputPath)posttrim-info-compiled/%(Arch)/%(Filename).o</OutputFile> |
| for (var i = 0; i < name.Length; i++) { | ||
| var c = name [i]; | ||
| if (c == '_' || char.IsLetterOrDigit (c)) | ||
| continue; | ||
| return false; | ||
| } | ||
| return name.Length > 0; |
There was a problem hiding this comment.
IsValidCIdentifier allows digits in any position, including as the first character. Since the symbol name becomes part of generated native identifiers (and corresponding P/Invokes), a leading digit would produce invalid generated C identifiers. Update the validation to require the first character to be '_' or a letter, and only allow digits after the first character.
| for (var i = 0; i < name.Length; i++) { | |
| var c = name [i]; | |
| if (c == '_' || char.IsLetterOrDigit (c)) | |
| continue; | |
| return false; | |
| } | |
| return name.Length > 0; | |
| if (name.Length == 0) | |
| return false; | |
| var first = name [0]; | |
| if (first != '_' && !char.IsLetter (first)) | |
| return false; | |
| for (var i = 1; i < name.Length; i++) { | |
| var c = name [i]; | |
| if (c == '_' || char.IsLetterOrDigit (c)) | |
| continue; | |
| return false; | |
| } | |
| return true; |
| var item = new Microsoft.Build.Utilities.TaskItem (outputPath); | ||
| item.SetMetadata ("Arch", Architecture.ToLowerInvariant ()); |
There was a problem hiding this comment.
Architecture is passed through to the generated item metadata Arch as Architecture.ToLowerInvariant(). $(TargetArchitectures) can contain multiple comma-separated architectures (often with spaces, e.g. "ARM64, x86_64"), which would result in clang being invoked with -arch "arm64, x86_64" (invalid) and/or missing per-arch compilation. This task should either (1) emit one NativeSourceFiles item per architecture (each with a single Arch value), or (2) validate that Architecture is a single architecture and fail with a clear error if not.
| var item = new Microsoft.Build.Utilities.TaskItem (outputPath); | |
| item.SetMetadata ("Arch", Architecture.ToLowerInvariant ()); | |
| var architectures = Architecture.Split (',').Select (v => v.Trim ()).Where (v => v.Length > 0).ToArray (); | |
| if (architectures.Length != 1) { | |
| Log.LogError ("The '{0}' property must contain exactly one architecture for post-trimming processing, but was '{1}'.", nameof (Architecture), Architecture); | |
| return; | |
| } | |
| var item = new Microsoft.Build.Utilities.TaskItem (outputPath); | |
| item.SetMetadata ("Arch", architectures [0].ToLowerInvariant ()); |
| if (Enum.TryParse<InlineDlfcnMethodsMode> (value, true, out var inlineDlfcnMode)) | ||
| InlineDlfcnMethods = inlineDlfcnMode; | ||
| else if (string.Equals (value, "compatibility", StringComparison.OrdinalIgnoreCase)) | ||
| InlineDlfcnMethods = InlineDlfcnMethodsMode.Compat; | ||
| else if (string.IsNullOrEmpty (value)) | ||
| InlineDlfcnMethods = InlineDlfcnMethodsMode.Disabled; | ||
| else |
There was a problem hiding this comment.
InlineDlfcnMethods parsing uses Enum.TryParse, which will accept values like Disabled. However the MSBuild condition that wires in InlineDlfcnMethodsStep is based on the property being non-empty, so InlineDlfcnMethods=Disabled would still run the step and effectively behave like compatibility mode. Consider rejecting Disabled explicitly (throw) or treating it as empty/disabled so that it can’t accidentally enable the step.
| if (Enum.TryParse<InlineDlfcnMethodsMode> (value, true, out var inlineDlfcnMode)) | |
| InlineDlfcnMethods = inlineDlfcnMode; | |
| else if (string.Equals (value, "compatibility", StringComparison.OrdinalIgnoreCase)) | |
| InlineDlfcnMethods = InlineDlfcnMethodsMode.Compat; | |
| else if (string.IsNullOrEmpty (value)) | |
| InlineDlfcnMethods = InlineDlfcnMethodsMode.Disabled; | |
| else | |
| if (string.IsNullOrEmpty (value)) | |
| InlineDlfcnMethods = InlineDlfcnMethodsMode.Disabled; | |
| else if (string.Equals (value, "compatibility", StringComparison.OrdinalIgnoreCase)) | |
| InlineDlfcnMethods = InlineDlfcnMethodsMode.Compat; | |
| else if (Enum.TryParse<InlineDlfcnMethodsMode> (value, true, out var inlineDlfcnMode)) { | |
| if (inlineDlfcnMode == InlineDlfcnMethodsMode.Disabled) | |
| throw new InvalidOperationException ($"Unknown InlineDlfcnMethods value: {value}"); | |
| InlineDlfcnMethods = inlineDlfcnMode; | |
| } else |
Add a new custom linker step that replaces runtime
Dlfcn.dlsymcalls withdirect native symbol lookups via P/Invokes to
__Internal. This improvesstartup performance and app size, and ensures the native linker can see symbol
references that would otherwise only be resolved at runtime.
Two modes are supported, controlled by the
InlineDlfcnMethodsMSBuildproperty:
strict: inlines all calls toObjCRuntime.DlfcnAPIs.compatibility: only inlines calls that reference symbols from[Field]attributes.
Post-trimming MSBuild targets ensure native code is only generated for symbols
that survive trimming (ILTrim or NativeAOT):
_CollectPostILTrimInformation: scans trimmed assemblies for survivinginlined dlfcn P/Invokes, with per-assembly caching for incremental builds.
_CollectPostNativeAOTTrimInformation: reads unresolved symbols from theNativeAOT object file or static library and filters for inlined dlfcn symbols.
_PostTrimmingProcessing: generates native C code for surviving symbols andadds it to the native link inputs.
Extends
MachO.cswith LC_SYMTAB parsing to read unresolved symbols from bothMach-O object files and static libraries.
Contributes towards #17693.