Skip to content

Add experimental tag filtering for flaky tests#45

Draft
Copilot wants to merge 4 commits intomainfrom
copilot/add-experimental-tag-to-tests
Draft

Add experimental tag filtering for flaky tests#45
Copilot wants to merge 4 commits intomainfrom
copilot/add-experimental-tag-to-tests

Conversation

Copy link
Copy Markdown

Copilot AI commented Jan 16, 2026

The TimeConsistencyConstraint with free time optimization test fails intermittently in CI with tolerance violations (0.066 < 1e-6). This PR implements a tagging mechanism to exclude flaky tests from CI while allowing manual execution.

Changes

  • Tag flaky test: Added tags=[:experimental] to the unstable test in time_consistency_constraint.jl
  • Filter logic: Modified test/runtests.jl to exclude experimental tests by default using TestItemRunner's filter parameter
  • Environment override: Set INCLUDE_EXPERIMENTAL=1 to run experimental tests when needed
  • Documentation: Added Testing section to README explaining the mechanism

Usage

# CI default - stable tests only
julia --project=. test/runtests.jl

# Include experimental tests
INCLUDE_EXPERIMENTAL=1 julia --project=. test/runtests.jl

The filter uses get(ti, :tags, []) for safe handling of tests without tags.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/EnzymeAD/Enzyme.jl/tarball/990991b8aa76d17693a98e3a915ac7aa49f08d1a
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/FluxML/MacroTools.jl/tarball/1e0228a030642014fe5cfe68c2c0a818f9e3f522
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaArrays/ArrayInterface.jl/tarball/d81ae5489e13bc03567d4fbbb06c546a5e53c857
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaArrays/FillArrays.jl/tarball/2f979084d1e13948a3352cf64a25df6bd3b4dca3
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaArrays/LazyArrays.jl/tarball/70ebe3bcf87d6a1e7435ef5182c13a91161ba9b8
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaArrays/StaticArrayInterface.jl/tarball/96381d50f1ce85f2663584c8e886a6ca97e60554
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaArrays/StaticArrays.jl/tarball/eee1b9ad8b29ef0d936e3ec9838c7ec089620308
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaArrays/StaticArraysCore.jl/tarball/6ab403037779dae8c514bad259f32a447262455a
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/ASL_jll.jl/tarball/6252039f98492252f9e47c312c8ffda0e3b9e78d
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Bzip2_jll.jl/tarball/1b96ea4a01afe0ea4090c5c8039690672dd13f2e
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Hwloc_jll.jl/tarball/3d468106a05408f9f7b6f161d9e7715159af247b
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Ipopt_jll.jl/tarball/8e9d217c63a8c8af96949300180ba0558f7f88b5
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Libiconv_jll.jl/tarball/be484f5c92fad0bd8acfef35fe017900b0b73809
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/METIS_jll.jl/tarball/2eefa8baa858871ae7770c98c3c2a7e46daba5b4
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/MUMPS_seq_jll.jl/tarball/afbaaa0fa2f001ad8091e27885d69973f8eae3d7
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/OpenBLAS32_jll.jl/tarball/ece4587683695fe4c5f20e990da0ed7e83c351e7
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/OpenSpecFun_jll.jl/tarball/1346c9208249809840c91b26703912dff463d335
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/SPRAL_jll.jl/tarball/139fa63f03a16b3d859d925ee9149dfc15f21ece
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/XML2_jll.jl/tarball/80d3930c6347cfce7ccf96bd3bafdf079d9c0390
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Xorg_libpciaccess_jll.jl/tarball/4909eb8f1cbf6bd4b1c30dd18b2ead9019ef2fad
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaBinaryWrappers/Zstd_jll.jl/tarball/446b23e73536f84e8037f5dce465e92275f6a308
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaCI/BenchmarkTools.jl/tarball/7fecfb1123b8d0232218e2da0c213004ff15358d
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaCollections/DataStructures.jl/tarball/e357641bb3e0638d353c4b29ea0e40ea644066a6
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaCollections/OrderedCollections.jl/tarball/05868e21324cede2207c6f0f466b4bfef6d5e7ee
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaData/Parsers.jl/tarball/7d2f8f21da5db6a806faf7b9b292296da42b2810
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaData/StructTypes.jl/tarball/159331b30e94d7b11379037feeb9b690950cace8
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaDiff/AbstractDifferentiation.jl/tarball/c8dcaacefdd7cb6bc94f50f0e998853e3b5557d1
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaDiff/ChainRulesCore.jl/tarball/e4c6a16e77171a5f5e25e9646617ab1c276c5607
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaDiff/DiffResults.jl/tarball/782dd5f4561f5d267313f23853baaaa4c52ea621
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaDiff/DiffRules.jl/tarball/23163d55f885173722d1e4cf0f6110cdbaf7e272
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaDiff/DifferentiationInterface.jl/tarball/44d9321761ed99e1d444b5081b3166d3259adcf0
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaDiff/FiniteDiff.jl/tarball/9340ca07ca27093ff68418b7558ca37b05f8aeb1
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaDiff/ForwardDiff.jl/tarball/b2977f86ed76484de6f29d5b36f2fa686f085487
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaDocs/DocStringExtensions.jl/tarball/7442a5dfe1ebb773c29cc2962a8980f47221d76c
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaFunctional/CompositionsBase.jl/tarball/802bb88cd69dfd1509f6670416bd4434015693ad
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaGPU/Adapt.jl/tarball/7e35fca2bdfba44d797c53dfe63a51fabf39bfc0
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaGPU/GPUArrays.jl/tarball/83cf05ab16a73219e5f6bd1bdfa9848fa24ac627
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaGizmos/Observables.jl/tarball/7438a59546cf62428fc9d1bc94729146d37a7225
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaIO/ChunkCodecs.jl/tarball/1a3ad7e16a321667698a19e77362b35a1e94c544
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaIO/ChunkCodecs.jl/tarball/34d9873079e4cb3d0c62926a225136824677073f
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaIO/ChunkCodecs.jl/tarball/cee8104904c53d39eb94fd06cbe60cb5acde7177
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaIO/CodecBzip2.jl/tarball/84990fa864b7f2b4901901ca12736e45ee79068c
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaIO/CodecZlib.jl/tarball/962834c22b66e32aa10f7611c08c8ca4e20749a9
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaIO/FileIO.jl/tarball/d60eb76f37d7e5a40cc2e7c36974d864b82dc802
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/JuliaIO/JLD2.jl/tarball/8f8ff711442d1f4cfc0d86133e7ee03d62ec9b98
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaIO/JSON.jl/tarball/b3ad4a0255688dcb895a52fafbaae3023b588a90
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaIO/TranscodingStreams.jl/tarball/0c45878dcfdcfa8480052b6ab162cdd138781742
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaLang/Compat.jl/tarball/9d8a54ce4b17aa5bdce0ea5c34bc5e7c340d16ad
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaLang/FunctionWrappers.jl/tarball/d62485945ce5ae9c0c48f124a84998d755bae00e
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/JuliaLang/PrecompileTools.jl/tarball/07a921781cab75691315adc645096ed5e370cb77
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaLang/ScopedValues.jl/tarball/c3b2323466378a2ba15bea4b2f73b081e022f473
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/JuliaLinearAlgebra/ArrayLayouts.jl/tarball/e0b47732a192dd59b9d079a06d04235e2f833963
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaLogging/LoggingExtras.jl/tarball/f00544d95982ea270145636c181ceda21c4e2575
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaMath/InverseFunctions.jl/tarball/a779299d77cd080bf77b97535acecd73e1c5e5cb
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaMath/IrrationalConstants.jl/tarball/b2d91fe939cae05960e760110b328288867b5758
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaMath/NaNMath.jl/tarball/9b8215b1ee9e78a293f99797cd31375471b2bcae
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaMath/SpecialFunctions.jl/tarball/f2685b435df2613e25fc10ad8c26dddb8640f547
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaObjects/Accessors.jl/tarball/856ecd7cebb68e5fc87abecd2326ad59f0f911f3
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/JuliaObjects/ConstructionBase.jl/tarball/b4b092499347b18a015186eae3042f72267106cb
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaPackaging/JLLWrappers.jl/tarball/0533e564aae234aff59ab625543145446d8b6ec2
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaPackaging/Preferences.jl/tarball/522f093a29b31a93e34eaea17ba055d850edea28
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaPackaging/Requires.jl/tarball/62389eeff14780bfe55195b7204c0d8738436d64
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaPlots/Plots.jl/tarball/5c3d09cc4f31f5fc6af001c250bf1278733100ff
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaSIMD/BitTwiddlingConvenienceFunctions.jl/tarball/f21cfd4950cb9f0587d5067e69405ad2acd27b87
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaSIMD/CPUSummary.jl/tarball/f3a21d7fc84ba618a779d1ed2fcca2e682865bab
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaSIMD/CloseOpenIntervals.jl/tarball/05ba0d07cd4fd8b7a39541e31a7b0254704ea581
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaSIMD/LayoutPointers.jl/tarball/a9eaadb366f5493a5654e843864c13d8b107548c
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaSIMD/ManualMemory.jl/tarball/bcaef4fc7a0cfe2cba636d84cda54b5e4e4ca3cd
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaSIMD/Polyester.jl/tarball/6f7cd22a802094d239824c57d94c8e2d0f7cfc7d
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/JuliaSIMD/PolyesterWeave.jl/tarball/645bed98cd47f72f67316fd42fc47dee771aefcd
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaSIMD/SIMDTypes.jl/tarball/330289636fb8107c5f32088d2741e9fd7a061a5c
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaSIMD/StrideArraysCore.jl/tarball/83151ba8065a73f53ca2ae98bc7274d817aa30f2
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaSIMD/ThreadingUtilities.jl/tarball/d969183d3d244b6c33796b5ed01ab97328f2db85
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaServices/StructUtils.jl/tarball/9297459be9e338e546f5c4bedb59b3b5674da7f1
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaStats/LogExpFunctions.jl/tarball/13ca9e2586b89836fd20cccf56e57e2b9ae7f38f
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/JuliaStats/Statistics.jl/tarball/ae3bb1eb3bba077cd276bc5cfc337cc65c3075c0
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/JuliaTesting/ExprTools.jl/tarball/27415f162e6028e81c72b82ef756bf321213b6ec
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/KristofferC/TimerOutputs.jl/tarball/3748bd928e68c7c346b52125cf41fff0de6937d0
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/MakieOrg/Makie.jl/tarball/76dab592fa553e378f9dd8adea16fe2591aa3daa
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/Roger-luo/ExproniconLite.jl/tarball/c13f0b150373771b0fdc1713c97860f8df12e6c2
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/Roger-luo/Jieko.jl/tarball/2f05ed29618da60c06a87e9c033982d4f71d0b6c
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/Roger-luo/Moshi.jl/tarball/53f817d3e84537d84545e0ad749e483412dd6b2a
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/ADTypes.jl/tarball/f7304359109c768cf32dc5fa2d371565bb63b68a
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/CommonSolve.jl/tarball/78ea4ddbcf9c241827e7035c3a03e2e456711470
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/CommonWorldInvalidations.jl/tarball/ae52d1c52048455e85a387fbee9be553ec2b68d0
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/DiffEqBase.jl/tarball/0514bf55835444420ce81f8e32c5e2369d4af456
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/FastPower.jl/tarball/1dd291358e0e3a31f77dbbb76ce7abbcca38d410
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/IfElse.jl/tarball/debdd00ffef04665ccbb3e150747a77560e8fad1
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/MaybeInplace.jl/tarball/54e2fdc38130c05b42be423e90da3bade29b74bd
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/SciML/MuladdMacro.jl/tarball/cac9cc5499c25554cba55cd3c30543cff5ca4fab
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/NonlinearSolve.jl/tarball/750f782fcc7e09283be7d8a7aa687a95e4911b60
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/NonlinearSolve.jl/tarball/db10be07416f36da7928e2a934047948cb6430ad
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/NonlinearSolve.jl/tarball/e96d5e96debf7f80a50d0b976a13dea556ccfd3a
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/SciML/OrdinaryDiffEq.jl/tarball/8be4cba85586cd2efa6c76d1792c548758610901
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/OrdinaryDiffEq.jl/tarball/8d8e8fd5c80b38c0cc2de5a8fcca8db1a2e77a06
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/SciML/PreallocationTools.jl/tarball/6c6925c42710b794dbb8fe7ab0cbc393c5b59fbe
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/RecursiveArrayTools.jl/tarball/31b3d7b7e14faad39583475b89aadbd9c3e7ef8b
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/SciML/RuntimeGeneratedFunctions.jl/tarball/2f609ec2295c452685d3142bc4df202686e555d2
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/SciMLBase.jl/tarball/9923adc7defeba6a52a99eb493ec317ac6c65942
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/SciMLLogging.jl/tarball/7eebb9985e35b123e12025a3a2ad020cd6059f71
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/SciML/SciMLOperators.jl/tarball/d1d14b15bbebf48dc80e8a7cfe640e2d835e22ea
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/SciMLPublic.jl/tarball/0ba076dbdce87ba230fff48ca9bca62e1f345c9b
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/SciMLStructures.jl/tarball/607f6867d0b0553e98fc7f725c9f9f13b4d01a32
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/Static.jl/tarball/49440414711eddc7227724ae6e570c7d5559a086
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/SymbolicIndexingInterface.jl/tarball/94c58884e013efff548002e8dc2fdd1cb74dfce5
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/SciML/TruncatedStacktraces.jl/tarball/ea3e54c2bdde39062abf5a9758a23735558705e1
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/YingboMa/FastBroadcast.jl/tarball/ab1b34570bcdf272899062e1a56285a53ecaae08
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/c42f/FastClosures.jl/tarball/acebe244d53ee1b461970f8910c235b259e772ef
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/chriselrod/FunctionWrappersWrappers.jl/tarball/b104d487b34566608f8b4e1c39fb0b10aa279ff8
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/fredrikekre/EnumX.jl/tarball/bddad79635af6aec424f53ed8aad5d7555dc6f00
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
  • https://api.github.com/repos/harmoniqs/NamedTrajectories.jl/tarball/7528bb0d0facbc35c0046e9a15cfe870f91de071
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/harmoniqs/TrajectoryIndexingUtils.jl/tarball/108da5f045005c9b57e6713ea84aca49d77fccaa
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/jonniedie/ConcreteStructs.jl/tarball/f749037478283d372048690eb3b5f92a79432b34
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/julia-vscode/TestItemRunner.jl/tarball/76f275a3f3d83ece88ec69d73058048de5acb1dc
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/julia-vscode/TestItems.jl/tarball/42fd9023fef18b9b78c8343a4e2f3813ffbcefcb
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/jump-dev/Ipopt.jl/tarball/30feb540d41fbc54ca46cc9d811380bf7c8877d9
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/jump-dev/MathOptInterface.jl/tarball/181c2611c7aa6a362fdf937b1e2af55e6691181f
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/jump-dev/MutableArithmetics.jl/tarball/22df8573f8e7c593ac205455ca088989d0a2c7a0
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/jw3126/Setfield.jl/tarball/c5391c6ace3bc430ca630251d02ea9687169ca68
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/m-j-w/CpuId.jl/tarball/fcbb72b032692610bfbdb15018ac16a36cf2e406
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/queryverse/IteratorInterfaceExtensions.jl/tarball/a3f24677c21f5bbe9d2a714f95dcd58337fb2856
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/quinnj/JSON3.jl/tarball/411eccfe8aba0814ffa0fdf4860913ed09c34975
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/rdeits/CommonSubexpressions.jl/tarball/cda2cfaebb4be89c9084adaca7dd7333369715c5
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/sethaxen/ExponentialAction.jl/tarball/c8e96f985aa517c5b8413661c8c06e1021895639
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/simonster/Reexport.jl/tarball/45e428421666073eab6f2da5c9d310d99bb12f9b
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • https://api.github.com/repos/vchuravy/HashArrayMappedTries.jl/tarball/2eaa69a7cab70a52b9687c8bf950a5a93ec895ae
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (http block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (http block)
  • pkg.julialang.org
    • Triggering command: /usr/bin/julia julia -e using Pkg; Pkg.status(&#34;TestItemRunner&#34;) (dns block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate(); using TestItemRunner; println(names(TestItemRunner)) (dns block)
    • Triggering command: /usr/bin/julia julia --project=. -e using Pkg; Pkg.instantiate() (dns block)
  • www.julia-vscode.org
    • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

The GitHub workflows for the DirectTrajOpt.jl repository currently include tests that have been failing intermittently or are unstable (flaky). The workflow titled CI running on main shows examples of test failures in the run linked here.

For example, the test for "TimeConsistencyConstraint with free time optimization" frequently fails with an assertion mismatch:

Expression: abs((t[k + 1] - t[k]) - Δt[k]) < 1.0e-6
Evaluated: 0.06645296983145835 < 1.0e-6

Objective:

  1. Add an experimental tag to tests that are identified as flaky or unstable. This can be implemented using @testset naming or metadata addition in Julia.
  2. Modify the test runner (run_all_package_tests or equivalent logic in runtests.jl) to enable filtering out tests tagged as experimental during routine CI runs.
  3. Include a mechanism (e.g., an environment variable or specific flag) to include the experimental tests during targeted runs when desired.

Files likely to be changed:

  • test/runtests.jl or equivalent files where tests are defined.
  • GitHub workflow file .github/workflows/CI.yml if environment variable changes influence test runs.

Additional Details:

Ensure all changes are documented and avoid impacting existing stable tests. All logic for tagging and filtering flaky tests should be localized and not disrupt the broader test structure.

This pull request was created from Copilot chat.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 3 commits January 16, 2026 20:27
…s.jl

Co-authored-by: jack-champagne <43344745+jack-champagne@users.noreply.github.com>
Co-authored-by: jack-champagne <43344745+jack-champagne@users.noreply.github.com>
Co-authored-by: jack-champagne <43344745+jack-champagne@users.noreply.github.com>
Copilot AI changed the title [WIP] Add experimental tag to flaky tests in CI workflow Add experimental tag filtering for flaky tests Jan 16, 2026
Copilot AI requested a review from jack-champagne January 16, 2026 20:37
@gennadiryan
Copy link
Copy Markdown
Member

gennadiryan commented Apr 29, 2026

@jack-champagne I have some things mocked up that will help us take advantage of this, but it comes with some additional potential overhauls to our CI. The idea is to run TestItemRunner's test item discovery routine up front to not only generate the list of test item locations, but also grab the associated code objects and cache them alongside the package itself. This would reduce CI latency spent rebuilding tests but would also allow for much more fine-grained control of which tests "apply" to a given CI run, since the tests would be getting filtered ahead of time. At present, the most obvious use case is filtering out GPU tests on certain runs (see https://github.com/harmoniqs/Piccolissimo.jl/commit/ade50e4a29225534163afe888ee7225a33d52d34), but there are many others. Another potential future use case is supporting separate development streams (e.g. in Piccolissimo, a main branch and a well-maintained altissimo_integration branch, where the former is not yet ready to merge into the latter, and where both need to be kept up to date with downstream upgrades in the meantime). More to follow (likely in a separate PR initially landing on Piccolo.jl, then propagated back down to here).

Yet another use case: temporarily gracefully skipping CI tests for particular extension packages due to that particular extension relying on a dep (QuantumToolbox.jl is the one I've got in mind today) that is behind on compat.

@jack-champagne
Copy link
Copy Markdown
Member

🤖 Nightly Rebase Bot: Conflicts detected during rebase. An agent-attempted resolution is available on rebase/app/copilot-swe-agent/copilot/add-experimental-tag-to-tests. Please review before merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants