Some of you may have noticed a bug report last month on the mdtest GitHub repo:
This bug affected IO500 submissions for SC18 which used stonewalling.
Fortunately, the bug was only in the score calculation and we have
been able to parse the mdtest results output and calculate the correct
scores. Only the SC18 lists were affected and will remain unchanged
on the website. The corrected lists are now available on our webpage
as new lists dated 01/19 (the io500.org URL points to it now too).
As you may note, the corrected lists in one case do show a different
ranking than what was reported at SC18 for the 10 Node Challenge.
Nevertheless, the committee has decided to not retroactively change
the SC18 10 Node Challenge official rankings.
Enclosed is a script that shows how we recomputed the mdtest values
and checked for inconsistencies. You may find it useful to check your
late executions’ of the IO-500 if you ran with the prior version of
mdtest; however, we certainly urge everyone to run with the current
Github repo of mdtest which will of course be a requirement for future
We hope this is clear and believe it to be in the best interests of
fairness and transparency for the IO500 community.
The IO500 Committee