405 ux make config minimal

docs/modules/tutorial/nav.adoc

@@ -1,9 +1,10 @@
 
 * xref:tutorial:index.adoc[Documentation]
-** xref:tutorial:gettingstarted.adoc[Getting Started]
+** xref:tutorial:index.adoc#_getting_started[Getting Started]
 ** xref:tutorial:configuration.adoc[Configuration Guide]
-*** xref:tutorial:configurationfiles/system.adoc[System Configuration]
-*** xref:tutorial:configurationfiles/magicstrings.adoc[Magic Strings]
-*** xref:tutorial:configurationfiles/machine.adoc[Machine Configuration]
-*** xref:tutorial:configurationfiles/benchmark.adoc[Benchmark Configuration]
-*** xref:tutorial:configurationfiles/plots.adoc[Plots Configuration]
+*** xref:tutorial:configuration.adoc#_system_configuration[System Configuration]
+*** xref:tutorial:configuration.adoc#_magic_strings[Magic Strings]
+*** xref:tutorial:configuration.adoc#_machine_configuration[Machine Configuration]
+*** xref:tutorial:configuration.adoc#_benchmark_configuration[Benchmark Configuration]
+*** xref:tutorial:configuration.adoc#_figures[Plots Configuration]
+** xref:tutorial:examples.adoc[Examples]

docs/modules/tutorial/pages/configurationfiles/machine.adoc

@@ -63,7 +63,7 @@ input_user_dir [*str*] (Optional)::
     Base directory where input data can be found before running tests. If provided, `input_dataset_base_dir` should be present too. It is used to copy `input_file_dependencies` from this directory to the `input_dataset_base_dir`.
     Refer to xref:tutorial:advancedConfiguration.adoc[Advanced Configuration] for more information.
 
-output_app_dir [*str*]::
+output_app_dir [*str*] (Optional)::
 
     The base directory where the benchmarked application should write its outputs to.
 
@@ -171,4 +171,4 @@ Concerning containers:
     For example, when using apptainer, this will overwrite the `APPTAINER_TMPDIR` environment variable. link:https://apptainer.org/docs/user/main/build_env.html[Apptainer Build Environment]
 
 `"executable":"singularity"`::
-    Tells the framework to use the `singularity pull ...` command for pulling images instead of `apptainer pull ...` .
+    Tells the framework to use the `singularity pull ...` command for pulling images instead of `apptainer pull ...` .

docs/modules/tutorial/pages/gettingstarted.adoc

@@ -72,11 +72,37 @@ The previous script will do the following:
 
 == Quickstart
 
-For detailed documentation, refer to our link:https://bench.feelpp.org/benchmarking/tutorial/index.html[docs].
+For detailed documentation, refer to our link:https://bench.feelpp.org/benchmarking/tutorial/configuration.html[docs].
+
+[IMPORTANT]
+Make sure to set up the xref:tutorial:gettingstarted.adoc#_prerequisites[Antora Environment].
+
+Create a `bench.json` file :
+
+[source, json]
+----
+{
+    "use_case_name":"sleep_bench",
+    "executable":"sleep",
+    "options":["1"]
+}
+----
+
+And run the benchmark with
+
+[source,bash]
+----
+feelpp-benchmarking-exec -bc ./bench.json -w
+----
+
+This will launch a server to view the benchmark results.
+
+== Example
+
 
 === The benchmarked application
 
-The framwork includes a sample C++/MPI application that can be used to get familiar with the framework's core concepts. It can be found under _examples/parallelsum/parallelSum.cpp_, or you can download it here : link:https://github.com/feelpp/benchmarking/blob/master/examples/parallelsum/parallelSum.cpp[parallelSum.cpp]
+The framework includes a sample C++/MPI application that can be used to get familiar with the framework's core concepts. It can be found under _examples/parallelsum/parallelSum.cpp_, or you can download it here : link:https://github.com/feelpp/benchmarking/blob/master/examples/parallelsum/parallelSum.cpp[parallelSum.cpp]
 
 This Feel++ Benchmarking "Hello World" application will compute the sum of an array distributed across multiple MPI processes. Each process will compute a partial sum, and then it will be summed to get the total sum.
 
@@ -284,4 +310,4 @@ The script takes the following arguments:
     `--plot-configs` (`-pc`): Path the a plot configuration to use for a given benchmark. To be used along with --patch-reports
     `--patch-reports` (`-pr`) : Ids of the reports to path, the syntax of the id is machine:application:usecase:date e.g. gaya:feelpp_app:my_use_case:2024_11_05T01_05_32. It is possible to affect all reports in a component by replacing the machine, application, use_case or date by 'all'. Also, one can indicate to patch the latest report by replacing the date by 'latest'. If this option is not provided but plot-configs is, then the latest report will be patched (most recent report date)
     `--save-patches` (`-sp`) : If this flag is active, existing plot configurations will be replaced with the ones provided in patch-reports.
-    `--website` (`-w`) : [Optional] Automatically compite the website and start an http server.
+    `--website` (`-w`) : [Optional] Automatically compile the website and start an http server.

docs/modules/tutorial/pages/index.adoc

@@ -18,3 +18,8 @@ With support for Apptainer (or Singularity), the framework ensures that benchmar
 The integration with CI/CD pipelines enables ongoing performance tracking. This is especially useful in projects where monitoring application scaling and performance over time is critical.
 
 The feelpp.benchmarking tool serves as a centralized platform for aggregating benchmarking results into a clear dashboard, making it an essential resource for those looking to monitor and enhance HPC application performance.
+
+
+include::gettingstarted.adoc[leveloffset=+1]
+
+include::configuration.adoc[leveloffset=+1]

examples/sorting/plots.json

@@ -81,7 +81,8 @@
                                 "transformation": "performance",
                                 "xaxis": { "parameter": "elements", "label": "N" },
                                 "yaxis": { "label": "Execution time (s)" },
-                                "color_axis": { "parameter": "algorithm", "label": "Algorithm" }
+                                "color_axis": { "parameter": "algorithm", "label": "Algorithm" },
+                                "aggregations":[{"column":"perfvalue","agg":"filter:elapsed"}]
                             }
                         },
                         {

pyproject.toml

@@ -14,6 +14,7 @@ where = ["src"]
   'report/templates/**',
   'scripts/data/*',
   'scripts/data/website_images/*',
+  'scripts/data/supplemental-ui/**/*',
   'reframe/templates/**'
 ]
 

src/feelpp/benchmarking/dashboardRenderer/component/leaf.py

@@ -1,3 +1,4 @@
+import tempfile
 from feelpp.benchmarking.dashboardRenderer.component.base import GraphNode
 from feelpp.benchmarking.dashboardRenderer.repository.base import Repository
 from feelpp.benchmarking.dashboardRenderer.views.base import View
@@ -87,23 +88,34 @@ def patchTemplateInfo( self, patch : Union[dict,TemplateDataFile], prefix:str, s
             save (bool): If True, the patch will be written back to the associated data file
                          (e.g., a JSON file) on the filesystem.
         """
-        if save:
-            template_data_files = [d for d in self.view.template_info.data if isinstance(d,TemplateDataFile) and d.prefix and d.prefix == prefix ]
+        template_data_files = [d for d in self.view.template_info.data if isinstance(d,TemplateDataFile) and d.prefix and d.prefix == prefix ]
 
-            if len( template_data_files ) > 1:
-                warnings.warn(f"More than one file having prefix {prefix} found. First occurence will be overwritten")
+        if len( template_data_files ) > 1:
+            warnings.warn(f"More than one file having prefix {prefix} found. First occurence will be overwritten")
 
-            filepath = None
-            if len( template_data_files ) == 0:
-                warnings.warn(f"No data files with {prefix} found in {self.id}. Saving this patch will not be possible.")
+        patch_data = patch.model_dump() if hasattr(patch, "model_dump") else patch
+
+        if not template_data_files:
+            warnings.warn(f"No data files with {prefix} found in {self.id}. Saving/patching will not be possible.")
+        else:
+            target_file = template_data_files[0]
+            if save:
+                write_path = os.path.join(self.view.template_data_dir, target_file.filepath) if hasattr(self.view,"template_data_dir") and self.view.template_data_dir else target_file.filepath
             else:
-                filepath = template_data_files[0].filepath
-                format = template_data_files[0].format
-
-            if filepath:
-                with open( os.path.join( self.view.template_data_dir, filepath ), "w" ) as f:
-                    if format == "json":
-                        json.dump( patch.model_dump(), f )
-                    else:
-                        f.write( patch )
-        self.view.updateTemplateData( {prefix:patch} )
+                base_dir = self.view.template_data_dir if hasattr(self.view,"template_data_dir") and self.view.template_data_dir else (os.path.dirname(target_file.filepath) or ".")
+
+                tmp_fd, write_path = tempfile.mkstemp(dir=base_dir, suffix=f".{target_file.format}")
+                os.close(tmp_fd)
+            target_file.filepath = write_path
+
+            with open(write_path, "w") as f:
+                if target_file.format == "json":
+                    json.dump(patch_data, f)
+                else:
+                    f.write(patch_data)
+            self.view.updateTemplateData(target_file)
+
+            if not save: #Cleanup temp file
+                os.remove(write_path)
+
+        self.view.updateTemplateData( {prefix:patch_data} )

src/feelpp/benchmarking/dashboardRenderer/tests/test_node.py

@@ -1,13 +1,14 @@
 from feelpp.benchmarking.dashboardRenderer.component.base import GraphNode
 from feelpp.benchmarking.dashboardRenderer.component.leaf import LeafComponent
-from feelpp.benchmarking.dashboardRenderer.schemas.dashboardSchema import TemplateDataFile
+from feelpp.benchmarking.dashboardRenderer.schemas.dashboardSchema import TemplateDataFile, TemplateInfo
 
 import pytest
 
 class MockView:
     def __init__(self,name = ""):
         self.name = name
         self.template_data = {"test":"template_data"}
+        self.template_info = TemplateInfo(data=[TemplateDataFile(format="json",prefix="meta",filepath="")])
     def clone(self):
         return MockView(f"Cloned {self.name}")
     def updateTemplateData(self,data):
@@ -310,4 +311,4 @@ def test_leafcomponent_patchTemplateInfo_no_save():
     leaf.patchTemplateInfo(patch_data, prefix, save=False)
 
     assert view.template_data["meta"] == patch_data
-    # Ensure no file operations occurred (implicitly, as we aren't mocking open/json.dump)
+    # Ensure no file operations occurred (implicitly, as we aren't mocking open/json.dump)

src/feelpp/benchmarking/json_report/figures/plotly/plotlyFigures.py

@@ -1,5 +1,6 @@
 import plotly.graph_objects as go
 import plotly.express as px
+import plotly.colors as plotly_colors
 
 from feelpp.benchmarking.json_report.figures.base import Figure
 from numpy import float64 as float64
@@ -114,12 +115,13 @@ def createTraces(self,df):
             - df (pd.DataFrame): The dataframe containing the figure data.
         Returns: (list[go.Trace]) The Scatter traces to display in the scatter figure.
         """
+        pcolors = plotly_colors.qualitative.Plotly
         return [
             go.Scatter( x = df.index, y = df.loc[:,col], name = col, fill='tonexty' if i > 0 else None, line=dict(color="black",dash="dash") ,mode="lines")
             for i,col in enumerate(self.fill_lines)
         ] + [
-            go.Scatter( x = df.index, y = df.loc[:,col], name = col )
-            for col in [c for c in df.columns if c not in self.fill_lines]
+            go.Scatter( x = df.index, y = df.loc[:,col], name = col, line=dict(color=pcolors[i % len(pcolors)]))
+            for i,col in enumerate([c for c in df.columns if c not in self.fill_lines])
         ]
 
 class PlotlyMarkedScatter(PlotlyFigure):
@@ -516,4 +518,4 @@ def createTraces(self, df):
                 opacity=0.5, name=col
             )
             for col in df.columns
-        ]
+        ]

src/feelpp/benchmarking/reframe/__main__.py

@@ -10,7 +10,12 @@
 def main_cli():
     parser = Parser()
 
-    machine_reader = ConfigReader(parser.args.machine_config,MachineConfig,"machine",dry_run=parser.args.dry_run)
+    if parser.args.machine_config:
+        machine_reader = ConfigReader(parser.args.machine_config,MachineConfig,"machine",dry_run=parser.args.dry_run)
+    else:
+        machine_reader = ConfigReader(None,MachineConfig,"machine",dry_run=parser.args.dry_run)
+        machine_reader.config = MachineConfig(machine="default")
+
 
     #Sets the cachedir and tmpdir directories for containers
     for platform, dirs in machine_reader.config.containers.items():
@@ -24,7 +29,8 @@ def main_cli():
 
     cmd_builder = CommandBuilder(machine_reader.config,parser)
 
-    os.environ["MACHINE_CONFIG_FILEPATH"] = parser.args.machine_config
+    if parser.args.machine_config:
+        os.environ["MACHINE_CONFIG_FILEPATH"] = parser.args.machine_config
 
     website_config = WebsiteConfigCreator(machine_reader.config.reports_base_dir)
 
@@ -38,7 +44,10 @@ def main_cli():
         app_reader = ConfigReader(configs,ConfigFile,"app",dry_run=parser.args.dry_run,additional_readers=[machine_reader])
 
         executable_name = os.path.basename(app_reader.config.executable).split(".")[0]
-        report_folder_path = cmd_builder.createReportFolder(executable_name,app_reader.config.use_case_name)
+        if parser.args.dry_run:
+            report_folder_path = None
+        else:
+            report_folder_path = cmd_builder.createReportFolder(executable_name,app_reader.config.use_case_name)
 
         #===============PULL IMAGES==================#
         if not parser.args.dry_run:
@@ -73,41 +82,43 @@ def main_cli():
         #================================================#
 
         #============== UPDATE WEBSITE CONFIG FILE ==============#
-        common_itempath = (parser.args.move_results or report_folder_path).split("/")
-        common_itempath = "/".join(common_itempath[:-1 - (common_itempath[-1] == "")])
+        if not parser.args.dry_run:
+            common_itempath = (parser.args.move_results or report_folder_path).split("/")
+            common_itempath = "/".join(common_itempath[:-1 - (common_itempath[-1] == "")])
 
-        website_config.updateExecutionMapping(
-            executable_name, machine_reader.config.machine, app_reader.config.use_case_name,
-            report_itempath = common_itempath
-        )
+            website_config.updateExecutionMapping(
+                executable_name, machine_reader.config.machine, app_reader.config.use_case_name,
+                report_itempath = common_itempath
+            )
 
-        website_config.updateMachine(machine_reader.config.machine)
-        website_config.updateUseCase(app_reader.config.use_case_name)
-        website_config.updateApplication(executable_name)
+            website_config.updateMachine(machine_reader.config.machine)
+            website_config.updateUseCase(app_reader.config.use_case_name)
+            website_config.updateApplication(executable_name)
 
-        website_config.save()
+            website_config.save()
         #======================================================#
 
 
         #============ CREATING RESULT ITEM ================#
-        with open(os.path.join(report_folder_path,"report.json"),"w") as f:
-            f.write(json.dumps(app_reader.config.json_report.model_dump()))
-
-        #Copy use case description if existant
-        FileHandler.copyResource(
-            app_reader.config.additional_files.description_filepath,
-            os.path.join(report_folder_path,"partials"),
-            "description"
-        )
+        if not parser.args.dry_run:
+            with open(os.path.join(report_folder_path,"report.json"),"w") as f:
+                f.write(json.dumps(app_reader.config.json_report.model_dump()))
+
+            #Copy use case description if existant
+            FileHandler.copyResource(
+                app_reader.config.additional_files.description_filepath,
+                os.path.join(report_folder_path,"partials"),
+                "description"
+            )
         #===============================================#
 
         try:
             # ============== LAUNCH REFRAME =======================#
-            reframe_cmd = cmd_builder.buildCommand( app_reader.config.timeout)
+            reframe_cmd = cmd_builder.buildCommand( app_reader.config.timeout )
             exit_code = subprocess.run(reframe_cmd, shell=True)
             #======================================================#
         finally:
-            if not os.path.exists(os.path.join(report_folder_path,"reframe_report.json")):
+            if report_folder_path and not os.path.exists(os.path.join(report_folder_path,"reframe_report.json")):
                 if os.path.exists(os.path.join(report_folder_path,"report.json")):
                     os.remove(os.path.join(report_folder_path,"report.json"))
                 os.rmdir(report_folder_path)
@@ -116,8 +127,9 @@ def main_cli():
         if parser.args.move_results:
             if not os.path.exists(parser.args.move_results):
                 os.makedirs(parser.args.move_results)
-            os.rename(os.path.join(report_folder_path,"reframe_report.json"),os.path.join(parser.args.move_results,"reframe_report.json"))
-            os.rename(os.path.join(report_folder_path,"report.json"),os.path.join(parser.args.move_results,"report.json"))
+            if report_folder_path:
+                os.rename(os.path.join(report_folder_path,"reframe_report.json"),os.path.join(parser.args.move_results,"reframe_report.json"))
+                os.rename(os.path.join(report_folder_path,"report.json"),os.path.join(parser.args.move_results,"report.json"))
         #======================================================#
 
     if parser.args.website:
@@ -126,4 +138,4 @@ def main_cli():
         subprocess.run(["npm","run","start"])
 
     # return exit_code.returncode
-    return 0
+    return 0

src/feelpp/benchmarking/reframe/commandBuilder.py

@@ -48,7 +48,6 @@ def buildJobOptions(self,timeout):
 
 
     def buildCommand(self,timeout):
-        assert self.report_folder_path is not None, "Report folder path not set"
         cmd = [
             'reframe',
             f'-C {self.buildConfigFilePath()}',
@@ -57,7 +56,7 @@ def buildCommand(self,timeout):
             f'--system={self.machine_config.machine}',
             f'--exec-policy={self.machine_config.execution_policy}',
             f'--prefix={self.machine_config.reframe_base_dir}',
-            f'--report-file={str(os.path.join(self.report_folder_path,"reframe_report.json"))}',
+            f'--report-file={str(os.path.join(self.report_folder_path,"reframe_report.json"))}' if self.report_folder_path else "",
             f"{self.buildJobOptions(timeout)}",
             f'--perflogdir=logs',
             f'{self.buildExecutionMode()}'