I will have a look at those papers. I can get system load information to some degree, and
that's probably an avenue I should investigate more fully, considering I'm sharing
this test load with people doing actual work on the HPC cluster, and in certain cases on
the storage as well, although I look for down times on both.
I have to simultaneously agree and disagree on your last statement. Considering that the
end goal here is to advise on the strengths and weaknesses of various storage systems, if
we throw our hands in the air and say that typical performance varies wildly, the point of
the exercise becomes somewhat moot. If you can't trust your results to stand up to an
average (or other mathematical leveling), are they really valid?
--Ken
On Jan 10, 2019, at 10:20 AM, Benson Muite via IO-500
<io-500(a)vi4io.org> wrote:
There are a number of relevant papers. Ones that come to mind are:
Kevin A. Brown, Nikhil Jain, Satoshi Matsuoka, Martin Schulz , Abhinav
Bhatele "Interference between I/O and MPI Traffic on Fat-tree Networks"
http://dx.doi.org/10.1145/3225058.3225144
T. Hoefler, R. Belli "Scientific Benchmarking of Parallel Computing
Systems"
https://htor.inf.ethz.ch/publications/index.php?pub=222
Are you able to get system load information? Figuring out conditions to
get best and worst performance may be helpful. Typical performance will
likely depend on system design and system load. This may be difficult to
quantify with just an average.
Regards,
Benson
On 1/10/19 5:10 PM, Carlile, Ken via IO-500 wrote:
> As I've been playing with the IO500 benchmark on various systems, I'm seeing
a fair amount of variability between runs. This may ultimately be to my detriment, but
have there been any discussions of perhaps requiring the mean of 3 or 5 runs for the
official numbers? I think the most striking examples are when I've managed to hit some
kind of storage side caching just right and produced numbers, particularly on
mdtest_hard_stat that are an order of magnitude greater on one run vs other runs. Now, I
really LIKE having a higher score, but it doesn't seem exactly fair or
representative...
>
> --Ken Carlile
> _______________________________________________
> IO-500 mailing list
> IO-500(a)vi4io.org
>
https://www.vi4io.org/mailman/listinfo/io-500
_______________________________________________
IO-500 mailing list
IO-500(a)vi4io.org
https://www.vi4io.org/mailman/listinfo/io-500