Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@ This action runs several checks on the software repository submitted for review
The action also looks for a `paper.md` file in the specified repository and post back information on:

- **Wordcount**: This will count the number of words in the paper file.
- **Detect statement of need**: This check will look for an `Statement of need` section in the paper content.
- **Detect statement of need**: This check will look for a `Statement of need` section in the paper content.
- **Detect software design section**: For submissions without the `pre-2026-submission` label, this check will look for a `Software Design` section in the paper.
- **Detect research impact statement**: For submissions without the `pre-2026-submission` label, this check will look for a `Research Impact Statement` section in the paper.
- **Detect AI usage disclosure**: For submissions without the `pre-2026-submission` label, this check will look for an `AI usage disclosure` section in the paper.


## Usage
Expand Down
55 changes: 49 additions & 6 deletions checks.rb
Original file line number Diff line number Diff line change
Expand Up @@ -409,24 +409,67 @@ def build_repository_history_section(repo, repo_url)

# Count paper file length
word_count = Open3.capture3("cat #{paper_path} | wc -w")[0].to_i
word_count_icon = word_count > 1999 ? "🚨" : (word_count > 1200 ? "⚠️" : "📄")
word_count_msg = "#{word_count_icon} Wordcount for `#{File.basename(paper_path)}` is **#{word_count}**"

# Detect a "Statement of need" section
# Read paper content once
paper_file_text = File.open(paper_path).read

# Check if issue has pre-2026-submission label
labels_output = Open3.capture3("gh issue view #{issue_id} --json labels --jq '.labels[].name'")[0]
is_pre_2026 = labels_output.include?("pre-2026-submission")

# Apply different word count thresholds based on submission type
if is_pre_2026
# Pre-2026 papers: original thresholds (target ~1000-1200 words)
word_count_icon = word_count > 1999 ? "🚨" : (word_count > 1200 ? "⚠️" : "📄")
else
# 2026+ papers: expanded thresholds (target ~1500-1800 words)
word_count_icon = word_count > 1999 ? "🚨" : (word_count > 1800 ? "⚠️" : "📄")
end
word_count_msg = "#{word_count_icon} Wordcount for `#{File.basename(paper_path)}` is **#{word_count}**"

# Check for required sections

# 1. Statement of need (always required)
if paper_file_text =~ /# Statement of Need/i
statemend_of_need_msg = "✅ The paper includes a `Statement of need` section"
statement_of_need_msg = "✅ The paper includes a `Statement of need` section"
else
statemend_of_need_msg = "🔴 Failed to discover a `Statement of need` section in paper"
statement_of_need_msg = "🔴 Failed to discover a `Statement of need` section in paper"
end

# New sections (only checked if NOT pre-2026)
section_checks = []
unless is_pre_2026
# 2. Software Design
if paper_file_text =~ /# Software Design/i
section_checks << "✅ The paper includes a `Software Design` section"
else
section_checks << "🔴 Failed to discover a `Software Design` section in paper"
end

# 3. Research Impact Statement
if paper_file_text =~ /# Research Impact( Statement)?/i
section_checks << "✅ The paper includes a `Research Impact Statement` section"
else
section_checks << "🔴 Failed to discover a `Research Impact Statement` section in paper"
end

# 4. AI usage disclosure
if paper_file_text =~ /# AI (usage|use) disclosure/i
section_checks << "✅ The paper includes an `AI usage disclosure` section"
else
section_checks << "🔴 Failed to discover an `AI usage disclosure` section in paper"
end
end

# Build message results
section_messages = [statement_of_need_msg, *section_checks].join("\n\n ")

paper_info = <<~PAPERFILEINFO
**Paper file info**:

#{word_count_msg}

#{statemend_of_need_msg}
#{section_messages}

PAPERFILEINFO

Expand Down