I agree with all of this. The longer run up of possible official code would be very
welcome. Unlike most, I’ve been working with the C program for months, but the
announcement that the requirement is a double run took me very much by surprise.
As a production site, I have to catch as catch can to run the benchmarks, so the more time
I have to do that with frozen code, the better.
Sent from my <advertising redacted>
On May 29, 2020, at 6:54 PM, Chaarawi, Mohamad via IO-500 <io-500(a)vi4io.org>
The deadline extension / BOF postponement is definitely welcome on our end 😊
As has been discussed here and on the slack channel, we were struggling after the official
announcement to get the new benchmark running.
We did make it work finally earlier this morning with a bunch of patches to the different
layers (io500-app, ior/mdtest, and the respective drivers); however given the nature of
production systems, resetting things up there is a different story vs. trying it on a
local setup where things are not available on demand.
For the concrete action, I think the timelines proposed make a lot of sense, but IMO the 6
week fixed period could be a little longer since production systems don’t have a lot of
maintenance windows where the cluster can be reserved fully for a full run. I would prefer
to see 2+ months for that period, but I do understand the challenge on the committee
members and the community to do the testing that is required to work out all the quirks
with the candidate benchmark.
Anyway, having known and fixed dates for the benchmark release, testing phase, and lock
phase regardless of the time span in between is a good measure to avoid surprises.
From: IO-500 <io-500-bounces(a)vi4io.org> on behalf of committee--- via IO-500
Reply-To: "committee(a)io500.org" <committee(a)io500.org>
Date: Friday, May 29, 2020 at 5:32 PM
To: "io-500(a)vi4io.org" <io-500(a)vi4io.org>
Subject: [IO-500] Proposal of possible postponement of ISC20 IO500 BOF
Dear IO500 Community,
We have heard your concerns about the late announcement of changes and have already taken
one concrete action and are considering a second but we want your feedback about it
We are considering postponing our BoF, and our deadline, by one month. Since we are not
tied to ISC this year, we have this latitude. Are there any objections from the community
to this postponement?
For our concrete action, we will shift future schedules so that three months from an event
we release a candidate benchmark and the likely rules set for public testing. Then commit
to 6 weeks before the event to finalize the benchmark and rules. Then things are locked
until after the event.
Please let us know how you feel about the possible postponement and please do know how
deeply we value you all in our community that makes everything we do possible.
Based on the feedback received, we will make and announce our final decision by EOD Sunday
The IO500 Committee
IO-500 mailing list