[Feature] Check that LICENSE/NOTICE files exist alongside published packages

[abnobdoss]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

skywalking-eyes has been great for catching license header and dependency issues, but there's a gap I ran into recently that I think could be a natural fit for the tool.

In Apache Iceberg Rust, the 0.10.0 RC3 release vote failed because three crates were missing LICENSE and NOTICE files. The repo root had them, but the individual crate directories didn't, and Cargo just silently leaves them out of the published package. Nobody caught it until a reviewer flagged it mid-vote.

This is especially painful in Rust because workspace repos tend to grow new crates over time, and each one needs its own LICENSE/NOTICE. Apache License 2.0 §4 requires including the license with any redistribution, and each cargo publish is a redistribution. It's easy to forget when adding a crate, and nothing in the toolchain warns you. Other ecosystems like npm and Python could hit the same issue in monorepo setups, but Rust workspaces seem to run into it the most in practice.

Since eyes already covers license headers and dependency licenses, I was curious whether validating license file presence would be something that fits the project's scope. I'm not familiar enough with the codebase to have an opinion on how it'd work - just wanted to raise the problem and see if it's something worth exploring together. Happy to help with a PR if so.

Use case

As of now the core use case would be flagging in CI when a Rust workspace crate is missing LICENSE/NOTICE files that Cargo needs in order to include them in published packages.

Related issues

No response

Are you willing to submit a pull request to implement this on your own?

• Yes I am willing to submit a pull request on my own!

Code of Conduct

• I agree to follow this project's Code of Conduct

[tisonkun]

I'm not familiar with the SkyWalking Eyes tool. Generally, you can specify the license file and include it in the Cargo.toml file. The basic approach is to add a symlink under the manifest path of each crate. This way, when it's packaged, the license/notice file will be directly added to the final crate.

I'm not sure what needs to be verified here. If you're missing these files in the Iceberg Rust release as you mentioned, you can just add a symlink from each crate to the LICNESE/NOTICE files in the root directory. That should work.

[wu-sheng]

@abnobdoss Could you provide a demo repo to help us to reproduce the case?

[abnobdoss]

@tisonkun Agreed, symlinking LICENSE/NOTICE into each crate directory is the fix, and that's what iceberg-rust and arrow-rs do. The issue is that as Rust workspaces grow new crates, the discipline tends to slip (see the table below). iceberg-rust is considering hand-rolling their own CI check for this, which got me wondering whether it'd make sense as part of skywalking-eyes instead.

Not trying to overstate this. Just exploring whether it's a reasonable addition to skywalking-eyes so Apache-licensed Rust projects can get this out of the box. If it doesn't feel like the right fit or is too big of an addition, totally respect that. This is purely an inquiry, and I'm happy to contribute if the community thinks it makes sense and is open to guiding me on how you'd prefer it added.

@wu-sheng Here's a live example. apache/paimon-rust uses skywalking-eyes/header@v0.8.0 in CI and passes clean, but none of its published crates have LICENSE or NOTICE files in their directories. For contrast, apache/datafusion has symlinked LICENSE.txt/NOTICE.txt in most crates so they package correctly, but datafusion-cli/ is missing both and header CI still passes.

I checked 9 Apache Rust workspace repos:

Repo	Published crates with LICENSE/NOTICE	Header tool	File-presence check
iceberg-rust	15/15	skywalking-eyes	none
avro-rs	4/4	HawkEye	none
datafusion	35/38	HawkEye	none
arrow-rs	30/35	Apache RAT	none
iggy	0/6	HawkEye	none
hudi-rs	0/6	skywalking-eyes	none
paimon-rust	0/5	skywalking-eyes	none
horaedb	0/12	HawkEye	none
ballista	0/7	Apache RAT	none

[wu-sheng]

I am on vacation, @kezhenxu94 do we have dependency resolve support for Rust?

[wu-sheng]

@abnobdoss Currently, we don't have sub-packages concepts for the whole projects. As eyes is covering many other languages, we must consider who dependencies' licence generated, not just the existing LICENSE and NOTICE of source tars, right?

What are your projects putting into LICENSE and NOTICE? Source LICENSE and NOTICE are usually manually maintained. Dependencies' of binary are generated.

This thing is a little complex considering this and covering all supported languages.

[abnobdoss]

@wu-sheng For Rust specifically, these are the manually maintained source files: the Apache 2.0 license text and the short ASF NOTICE, copied or symlinked from the repo root into each crate directory. A published crate contains only the project's own source code and no dependency code, so its LICENSE and NOTICE never need generated dependency licenses.

On the dependency side, I have definitely seen that case in the iceberg ecosystem as well. iceberg (Java) has had both issues: missing licenses for bundled dependencies, and dependencies with incompatible licenses. Automating those checks would clearly be valuable, but it is a much bigger feature than what I am raising here. If useful, I can open a separate issue for that conversation so it is documented even if it ends up out of scope, and if the project is open to building it I would be glad to help contribute.

The issue listed here is much narrower, so maybe the real question is whether there is appetite for a slim Rust-only check: validate that each crate in a workspace ships a LICENSE and NOTICE, without going anywhere near artifact scanning. Cargo can already print the exact file list a crate would publish, so the check is just confirming those two files are in that list for each crate.

[kezhenxu94]

@abnobdoss please take a look at the --summary and --output flags (dependency command), https://github.com/apache/skywalking-eyes#summary-template, it allows you to generate a LICENSE (or NOTICE) based on a template file, for your use case, I would do this:

• create a LICENSE.tpl and NOTICE.tpl in the root folder, shared by all crates;
• build a matrix from the crates, like listing the folders under iceberg-rust/crates or any other ways that you can list the crates including future new crates;
• for each crate, while drafting a release, generate the LICENSE/NOTICE with --summary LICENSE.tpl and --output <crate-root-folder>, similar for the NOTICE file, and package them into the release tarball;

you can integrate this into your Makefile or add it to your automation workflow if you do release in an automated workflow;

[abnobdoss]

@kezhenxu94 Thanks, this is helpful! I hadn't looked at the --summary/--output flags before, and I can see cases where generating the files at release time would remove the drift problem at the source.

Unfortunately I'm not sure whether this would be accepted for Rust projects under ASF governance. cargo publish packages each crate straight from the working tree (and refuses a dirty tree without --allow-dirty), so anything shipping to crates.io needs the per-crate LICENSE/NOTICE committed in-tree, otherwise the published crates wouldn't match the voted source. This would also be true for the source tarball when it's cut with git archive from the release tag (as iceberg-rust does): the artifact has to match the audited tree.

Do you know of any Rust projects under ASF or similar governance that publish to crates.io and generate LICENSE/NOTICE at package time? If this is the direction you'd recommend for eyes users, a precedent like that would make it easier to point to as the answer whenever this kind of drift comes up.

[wu-sheng]

Do you know of any Rust projects under ASF or similar governance that publish to crates.io and generate LICENSE/NOTICE at package time?

From an ASF policy perspective, it doesn't have hard-requirements about how to manage source codes on GitHub/Git. The key is the final source tar. So, once you have the correct LICENSE/NOTICE in the source tar on the SVN, you are all good.

[kezhenxu94]

Unfortunately I'm not sure whether this would be accepted for Rust projects under ASF governance. cargo publish packages each crate straight from the working tree (and refuses a dirty tree without --allow-dirty), so anything shipping to crates.io needs the per-crate LICENSE/NOTICE committed in-tree, otherwise the published crates wouldn't match the voted source.

Artifacts published to convenient platforms like crates.io, pypi.org, npmjs.org, etc., are not source releases in ASF, so all you need is to have a source tar with required LICENSE/NOTICE files included, and then publish to crates.io using that source tar after voted passed.

This would also be true for the source tarball when it's cut with git archive from the release tag (as iceberg-rust does): the artifact has to match the audited tree.

From my understanding, the ASF doesn’t require you to release a source tar from git archive, you can generate/gather as many files as you wish to build a tar ball that contains necessary "source files", and vote against that tar ball, even if not all "source files" appear in the git repository;

If I were you, I’d create a make target like make release-src and generate required source files, compress them to a source tar ball, vote against it, and then do cargo publish --allow-dirty using that released tar ball;