feat: relative directory usage type

[andrius-puksta-sensmetry]

This PR adds support for referencing local directory paths as project dependencies, alongside the existing IRI/URL-based usages.

New Directory usage type

A new usage variant references a project by a directory path (relative to the declaring project's root) plus its publisher and name. It serialises as {"dir": ..., "publisher": ..., "name": ...}. The declared publisher/name must match what's actually found at the path, and there is no version constraint since a directory contains a single version. Only local_src projects are supported; KPARs will use another type, to keep resolution unambiguous.

Unlike legacy resource usages, where the IRI is both the project's identity and (possibly) its location, typed usages separate the two: identity is always publisher+name, and the source is explicit. This allows avoiding resolution altogether for typed usages. Currently resolution is still done due to large changes required to avoid it; typed usages always resolve to their type.

Resolution and solving

Resolvers now receive the full usage (plus an optional base path for relative filesystem paths) instead of a bare IRI. Resolution outcomes distinguish "not found" from "unresolvable", letting a file-resolver miss fall through to other resolvers.

Every usage derives an identifier (a PURL where possible, an URN fallback otherwise; legacy usages keep their IRI). The solver identifies packages by this identifier rather than by version constraint or source, so a package referenced with different constraints/paths by different dependents appears only once in the solution.

experimental CLI subcommand

Adds sysand experimental add --dir <path> and sysand experimental remove <publisher> <name>:

• add computes a path relative to the current project, reads publisher/name from the dependency, and adds a directory usage. By default it then locks and syncs (--no-lock/--no-sync opt out), rolling back the project file if resolution fails. No source overrides — typed usages will support everything overrides do, so they can be added directly instead.
• remove removes a usage by publisher and name, which will work for all typed usage types.

These are separate from regular add/remove to avoid breaking changes, and because the UX for other usage types is still unclear. experimental commands will likely replace regular commands on next major release, but some accommodation to minimize breaking changes will be required.

Other

• Java and Python bindings are adapted to the new usage model, including the new directory usage.

Breaking changes

• In the Java and Python bindings, info no longer takes a relative_file_root argument (it had no effect previously). Java code matching on usage types must also handle the new directory usage class.
• Lockfile interpretation is a bit stricter, but lockfiles generated by sysand remain valid (previously some fields could be missing and the lockfile would still be usable, now it's not).

[vytautas-astrauskas-sensmetry]

It serialises as {"dir": "relative/path", "publisher": "...", "name": "..."}.

Why are publisher and name fields needed? In Cargo, as far as I understand, only path is used to specify the dependency and name is provided to be able to rename the dependency from the user's side.

[andrius-puksta-sensmetry]

Publisher and name are needed for a few reasons:

• consistent UX for different usage types: some usage types (git, index) require publisher+name for obtaining the correct project; others (src path, kpar path, URL) in principle don't, but the UX will be inconsistent if publisher/name are unknown
• not requiring publisher and name for all typed usages will have bad UX for:
  • dependency overrides
  • manual inspection of usage list (also info output): ../../some-dir/aaa/tmp/something is less informative than publisher = <pub>, name = <name>, dir = <dir>
  • error reporting, whenever it relies on usage specification
  • other tools, like IDEs: e.g. if imports are of the form publisher::name or something, not having publisher/name explicitly specified in .project.json would result in confusing (and unactionable) errors about unknown names if the usage is renamed/moved
• consistency with cargo: cargo does require name to match actual name; if a user wants to rename the dep, package = "original-name" must be specified. Note that Cargo has good reasons to do it this way; not all of them apply to us:
  • each use can be checked from just Cargo.toml (any identifier is fine for this, so the real name is not needed)
  • explicit intent to have that dependency: allows ensuring that the expected project is at the destination if renames/moves happen
  • simpler resolution implementation by having all the info upfront
  • better error reporting if duplicates of the same project from different sources are resolved
• simpler implementation, especially for error reporting
• push users to have publisher set to have typed usages
• simpler project sources deduplication/merging

[vytautas-astrauskas-sensmetry]

Seeing this and other edge cases, I am kind of starting to wonder whether we should just require that all transitive dependencies are registry only.

[andrius-puksta-sensmetry]

Requiring transitive deps to be only from registry would be too strict in my opinion, and would be very annoying for local development of a few related projects.

Everything works fine (AFAIK) as long as the environment is not involved. Dependencies of projects in the environment would be cleanly resolvable if env.toml records the original source of the project (so that relative path usages can be resolved correctly), which it currently doesn't. Will make a PR that adds this info to env.toml. Also, env.toml does not record which versions of dependencies of the installed projects (just their identifiers, so it would break with two versions of the same PURL present), this also should be fixed eventually...