[pull] master from JuliaLang:master #2115
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Also fix a bug in linearization of `K"isdefined"`
Co-authored-by: spaette <[email protected]>
…#511) Julia's ecosystem (including Base.Docs and flisp lowering) assumes that strings within `struct` definitions are per-field docstrings, but the flisp parser doesn't handle these - they are only recognized when the struct itself has a docstring and are processed by the `@doc` macro recursing into the struct's internals. For example, the following doesn't result in any docs attached to `A`. ```julia struct A "x_docs" x "y_docs" y end ``` This change adds `K"doc"` node parsing to the insides of a struct, making the semantics clearer in the parser tree and making it possible to address this problems in the future within JuliaLowering. Also ensure that the `Expr` form is unaffected by this change.
…ng/JuliaSyntax.jl#506) * Don't assume that `SubString` has `pointer` and copy instead * Still assume `Substring{String}` has `pointer` * Test with `Test.GenericString`
…liaLang/JuliaSyntax.jl#500) * Remove the method `convert(::Type{String}, ::Kind)` This patch removes the method `convert(::Type{String}, ::Kind)` used for converting kinds to strings and replaces it with the already existing method of `Base.string`. There are two reason for this: i) the method causes invalidations when loading the package and ii) `convert` is called implicitly in e.g. constructors and should therefore typically only be defined between similar enough types. * Remove the method `Base.convert(::Type{Kind}, ::String)` This patch removes the method `Base.convert(::Type{Kind}, ::AbstractString)` and replaces it with a `Kind(::AbstractString)` constructor. The reason for this is that `convert` is called implicitly in e.g. constructors and should therefore typically only be defined between similar enough types.
Also introduce `K"code_info"` to distinguish the `CodeInfo`-like form with indexed statements from the more symbolic cross references that are used internally by lowering within `K"lambda"` prior to statement+SSA renumbering.
Still todo: * inner constructors * outer constructors * doc binding Also included here is `K"alias_binding"` - a more general replacement for the `outerref` used in flisp lowering. `alias_binding` allows one to allocate a binding early during desugaring and make this binding an alias for a given name. Bindings don't participate in scope resolution, so this allows us to bypass the usual scoping rules. For example, to refer to a global struct_name from an outer scope, but within an inner scope where the identifier struct_name is bound to a local variable. (We could also replace outerref by generating a new scope_layer and perhaps that would be simpler?)
This form where `K"lambda"` has four children [args, static_parameters, body, ret_var] feels more natural as it keeps AST pieces within the AST rather than as auxiliary attributes. These pieces do still need special treatment in scope resolution, but lambdas are already special there.
Avoid creating `::` expressions - just add these directly to the function argument name and type lists instead.
Perhaps this was used historically but it's now only used for method tables in method overlays.
As much as alias_binding is a neat idea, it seems like using a scope layer to distinguish the global vs local bindings might be good enough and allow us to remove the alias_binding concept. As a side effect, this may allow us to avoid needing support arbitrary bindings in some early lowering code.
Detangling this ball of string ... felt quite epic 😬😅 Here we take a different approach from the flisp code - we don't try to reproduce the function signature matching logic of `expand_function_def` to rewrite constructor signatures within the struct expansion code. Instead, we harness that existing logic by calling expand_function_def with custom rewrite functions for the inner part of the signature expression and the function body where `new()` occurs.
* Remove outterref - this has been removed upstream * Make expand_unionall_def its own function - this will be required shortly to match some changes upstream. * JuliaSyntax has removed the `convert` overload for `Kind` in the latest dev version
…args an error Here we introduce a `meta` attribute rather than - or perhaps in addition to - the `K"meta"` kind and use it to tag local variables which derived from function argument destructuring. We use this to make it an error to have duplicate destructured argument names. This is technically breaking, but probably only a good thing - without this users will silently have the intial duplicate argument names overwritten with the result of the last destructuring assignment. Also add tests for the various variable scope conflict errors: argument/local, static-parameter/local, local/global etc.
…wering These macros are a part of the language itself because they emit special syntax trees which is known to lowering. This is regardless of the fact that they don't have a surface syntax form. Where a `Base` form of these exists we add a method to that macro so it can be used as usual without needing to import from JuliaLowering.
I've chosen to attach the nospecialize metadata here as an attribute on the function argument names. This ensures it travels with the function arguments, without otherwise disturbing the AST. TODO: `@ nospecialise` within the function body is not done yet - it's not very natural in this scheme but I guess we should still be able to recognize it during scope analysis and turn it into a tombstone, moving the metadata onto the lambda's argument bindings.
* Logically arrange algorithm description into lowering passes. Move existing descriptions there as necessary and write at least a little about each pass where it doesn't exist already. * Describe high level outline of closure conversion pass. * Remove some older descriptions of existing lowering mechanisms which are not really needed anymore.
* Generalize attribute lambda_locals -> lambda_bindings * New structs LambdaBindings + CaptureInfo; rearrange existing code to use the bindings struct (capture info currently unused)
…JuliaLowering.jl#100) The removed test attempted to pass a runtime-computed function name to `ccall` via `ccallable_sptest_name(T)`, but `ccall` now requires its function name argument to be a compile-time constant. This pattern only works with `@generated` functions from Julia 1.13 onwards, where the function name can be evaluated at code generation time. Currently JL cannot handle `@generated` functions, so the commenting out the test case updated in the last commit. --------- Co-authored-by: Em Chu <[email protected]>
…uliaLang/JuliaLowering.jl#91) Adds support for nested splat expressions like `tuple((xs...)...)` by restructuring the splat expansion to match the native lowerer's recursive algorithm. The native lowerer unwraps only one layer of `...` per pass and relies on recursive expansion to handle nested cases. This approach naturally builds the nested `_apply_iterate` structure through multiple expansion passes, avoiding the need for explicit depth tracking and normalization. Changes: - Refactor `_wrap_unsplatted_args` to unwrap only one layer of `...` - Refactor `expand_splat` to construct unevaluated `_apply_iterate` call then recursively expand it - Add test cases for nested splats including triple-nested and mixed-depth
…tree (JuliaLang/JuliaSyntax.jl#603) * Fix whitespace using Julia contrib/check-whitespace.jl * Fix typos as detected by the rust "typos" tool
…tree (JuliaLang/JuliaLowering.jl#104) * Fix whitespace using Julia contrib/check-whitespace.jl * Fix typos as detected by the rust "typos" tool
…liaLowering.jl#89) * Fix quoted property access syntax (e.g., `Core.:(!==)`) Handle `K"quote"` nodes in property access during macro expansion by unwrapping them before processing. This allows syntax like `Core.:(!==)` to lower correctly, matching the behavior of `Meta.lower`. Co-Authored-By: Claude <[email protected]>
…#106) When desugaring introduces a `K"Identifer"` it should always decorate it with an associates scope layer - either adopted from the users code, or an internal layer created on the fly. This ensures desugaring treats hygiene consistently with macro expansion (thus ensuring that desugaring itself is hygienic).
Just an off-by-one in desugaring's argument counting. Test case from Mmap.jl: ``` ccall(:fcntl, Cint, (RawFD, Cint, Cint...), s, F_GETFL) ```
…g.jl#105) * Interpolation and type-stability improvements Should be a quick fix for JuliaLang/JuliaLowering.jl#94. Also improve the interpolation algorithm: instead of starting with a copy of the AST and re-scanning the tree for interpolations with each call to `_interpolate_ast`, do one full unconditional pass over the initial tree that copies and interpolates. Also fixes interpolation into QuoteNode in expr compat mode (e.g. `@eval Base.$x`) --------- Co-authored-by: Claire Foster <[email protected]>
…uliaLang/JuliaLowering.jl#109) Macros may pull apart an expression (eg, a module expression or the right hand side of a `.` expression) or quote that expression, and we should keep track of the scope where this originated. A particular example is the `@eval` macro. Consider ``` let name = :x @eval A.$name end ``` In this case the right hand side of `.` would normally be quoted (as a plain symbol) but in the case of `@eval` an extra `quote` is added around the expression to make the `name` variable valid unquoted code after quote expansion. In general, macros may pull apart or rearrange what's passed to them, so we can't make the assumption that normally-inert syntax passed to them should go without a scope layer. To fix this, this change adds a scope layer to all ASTs passed to macros. After macro expansion is done, we can then remove the layer from any AST we know is definitely inert to prevent it from interfering with future lowering passes over that quoted code. This helps but isn't a full solution - see JuliaLang/JuliaLowering.jl#111 for further work.
For vendoring into Base we need to avoid absolute import paths as in `using JuliaLowering` and `using JuliaSyntax` in the test files as neither of these packages will be top level modules. Thus, replace all occurrences of these with relative import paths except for one central location (currently in util.jl) which can be easily adjusted.
…uliaLang/JuliaLowering.jl#117) Co-authored-by: Claire Foster <[email protected]>
* Support curly outer constructor * Fixes and tests Co-authored-by: Claire Foster <[email protected]> --------- Co-authored-by: Claire Foster <[email protected]>
Co-authored-by: Em Chu <[email protected]>
These are the simplest possible adaptions to create the vendored Base.JuliaSyntax from an in-tree version of JuliaSyntax (JuliaLowering to be hooked up later). Also remove JuliaLowering / JuliaSyntax GHA actions and update JuliaLowering `[sources]` to use local `JuliaSyntax`.
The environment when running under Distributed is slightly different than running via `Pkg.test()` so this required some tweaks to error printing, etc.
There's been some interest in having the new Julia compiler frontend (JuliaSyntax + JuliaLowering) in the main Julia tree so that these are easier to work on together and so the new lowering code can co-evolve with changes to Core more easily.
…the name used by the equivalent C struct (#60115) See PR title. I plan to clean up the GC metrics code a bit more in subsequent PRs.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
18 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )