On Mar 10, 2020, at 4:43 PM, Harms, Kevin via IO-500 <io-500(a)vi4io.org> wrote:
Mark,
currently there is no requirement for replication = 2, you can run with replication = 1.
That is true, but it depends what you want to show to users. Some systems might
reasonably run without redundancy (e.g. short-term scratch space where NVMe MTTF is much
longer than the file lifetime), but that is not necessarily desirable of most storage. So
I think that should be _possible_, but using an unrealistic configuration for vendor
submissions may be setting users up for disappointment if that isn't how systems are
normally configured in the field.
As for somewhat apples-to-oranges comparisons (at least two fruits, not
apples-to-asteroids), in some cases it is possible to get _some_ information about the
underlying storage, but I don't think that is *easy* with the current results. If you
go to the IO-500 lists, and add all of the "information.*" fields at the bottom
of the page, this will give you some details about the systems that the tests were run
on.
That said, the information is recorded inconsistently in some places (e.g. in some cases
it looks like the number of devices is per server, and in others it is the total number of
devices), but at least it gives you some idea of what the other storage systems are
using.
One of the goals for the next IO-500 list is to automate some of the information capture
so that this is recorded more accurately and consistently. For example, having a script
for Lustre, Ceph, GPFS, BeeGFS, etc. to scrape information from the client and/or server
about RAM, CPU, network, filesystem size, version, devices, OS versions, tunable
parameters, etc. to include with the test results would be very useful, even if not all of
it fits into the database schema at this point.
Cheers, Andreas
________________________________________
From: IO-500 <io-500-bounces(a)vi4io.org> on behalf of Mark Nelson via IO-500
<io-500(a)vi4io.org>
Sent: Tuesday, March 10, 2020 4:30 PM
To: io-500(a)vi4io.org
Subject: [IO-500] How to judge scoring vs storage HW
Hi Folks,
I'm one of the Ceph developers but used to work in the HPC world in a
previous life. Recently I saw that we were listed on the SC19 IO-500 10
node challenge list but had ranked pretty low. I figured that it might
be fun to play around for a couple of days and see if I could get our
score up a bit.
Let me first say that it's great having mdtest and ior packaged up like
this. Already the hard test cases have identified a couple of
performance issues we should take care of with unaligned reads/writes
and cephfs dynamic subtree partitioning (which are also dragging our
score down). Very useful! I was so happy with the effort that I ended
up writing a new libcephfs aiori backend for ior/mdtest. The PR just
merged but is here for anyone interested:
https://github.com/hpc/ior/pull/217
Our test cluster has 10 nodes with 8 NVMe drives each, and we are
co-locating the metadata servers and client processes on the same nodes
during testing. So far with 2x replication we've managed to hit scores
in the 55-60 range which looks like it would have put us in 10th place
on the SC19 list (note that for that result we are pre-creating the
mdtest easy directories for static round-robin MDS pinning, though we
have a feature coming soon for ephemeral pinning via a single
parent-directory xattr). Anyway, I have really no idea how that score
actually compares to the other systems listed. I was wondering if
there's any way to easily compare what kind of hardware and software
configuration is being used for the storage clusters for each entry?
IE in our case we're using 2x replication and 10 nodes total with pretty
beefy Xeon CPUs, 8xP4610 NVMe drives, and 4x25GbE. Total storage
capacity before replication is ~640TB.
Thanks,
Mark
_______________________________________________
IO-500 mailing list
IO-500(a)vi4io.org
https://www.vi4io.org/mailman/listinfo/io-500
_______________________________________________
IO-500 mailing list
IO-500(a)vi4io.org
https://www.vi4io.org/mailman/listinfo/io-500
Cheers, Andreas