Our recent efforts to upgrade the rust toolchain to nightly-2023-01-23 #2149 is currently blocked due to a significant performance regression.
2 out of 3 tests added to measure the performance of Kani with Vectors seems to be highly affected. Here the analysis we added to the PR for tests/perf/vec/box_dyn.
Running the failing test in my local machine with cargo kani, I got the following:
| Configuration |
Time |
Memory |
| main (CBMC 5.75) |
5.74s |
165MB |
| this PR (CBMC 5.75) |
206s |
16GB |
| main (CBMC 5.70) |
200.s |
16GB |
Note that the time in this PR is very similar to the one I obtained when running main with an older CBMC version (before diffblue/cbmc#7230).
I wonder if whatever has changed neutralized the optimization that @tautschnig introduced. 😕
Originally posted by @celinval in #2149 (comment)
Our recent efforts to upgrade the rust toolchain to
nightly-2023-01-23#2149 is currently blocked due to a significant performance regression.2 out of 3 tests added to measure the performance of Kani with Vectors seems to be highly affected. Here the analysis we added to the PR for
tests/perf/vec/box_dyn.Running the failing test in my local machine with
cargo kani, I got the following:Note that the time in this PR is very similar to the one I obtained when running
mainwith an older CBMC version (before diffblue/cbmc#7230).I wonder if whatever has changed neutralized the optimization that @tautschnig introduced. 😕
Originally posted by @celinval in #2149 (comment)