Dear all,
based on the feedback of people and my own experience,
I have now drafted a simplified IO-500 run-suite:
https://github.com/VI4IO/io-500-script
I believe it is much easier to use, to debug and to optimize and quite minimal.
The design depends on phases:
1) A generation phase able to check for errors creates a hard-coded
run-script without branches. An example generated execution script is
enclosed.
=> during the generation phase you may use
https://www.vi4io.org/io500-info-creator/ to create the info tags.
It also creates a parallel minimal cleanup that purges the generated
files => again minimal.
2) The io-500.sh run-script can then be inspected / further modified
to include allowed tuning settings and the batch-submission commands
(which unfortunately were often missing in the last submissions).
3) An analysis phase which parses the output of the execution, checks
the validity of the result and computes the scores
Therewith, it minimizes the potential errors that may occur during the
*costly* execution and outsources them into the trivial preprocessing
and post-processing steps.
Outlook:
* I have not yet implemented step (3) or the build scripts, but will
do so depending on the feedback of the community.
* I reckon, the automatic detection of system information can be done
in the phases (1) as much as possible while some would need to go to
(2), but these calls can be added flexibly, too...
Best,
Julian
--
Dr. Julian Kunkel
Lecturer, Department of Computer Science
+44 (0) 118 378 8218
http://www.cs.reading.ac.uk/
https://hps.vi4io.org/