The new "shift" parameter did influence the stat result
by Ruibo Wang
Dear All,
I've built a cluster to test the IO500 benchmark.
The preliminary results showed that the new "shift" parameter did
influence the stat result badly.
If I use "-N" parameter, the mdtest result is:
mdtest_easy_stat 1429.930 kiops : time 253.86 seconds
mdtest_hard_stat 962.500 kiops : time 191.10 seconds
If I don't use "-N" parameter, the mdtest result is:
mdtest_easy_stat 200250.000 kiops : time 1.84 seconds
mdtest_hard_stat 51038.200 kiops : time 4.47 seconds
It is clear that the "shift" parameter change will impact the stat
result with the same hardware and software configuration. Now I'm still
trying to optimize the test. Hope I'll deliver a better result later.
Ruibo Wang
4 years, 5 months
IO500 at SC19 Update
by John Bent
Dear all,
This email is being sent to the IO500 community mailing list with bcc’s to
all historical submitters. This email pertains to changes made to the
benchmark for SC’19 as well as possible implications from those changes.
Much, if not all, of this information has already been discussed in our
Slack channel but, at the risk of being redundant, we are sending it again
to ensure maximum visibility.
We have introduced one major change for SC’19: we have added a “shift”
between the mdtest phases so that the set of processes doing creates, the
set of processes doing stats, the set of processes doing reads, and the set
of processes doing deletes are disjoint sets. Additionally, but perhaps
less materially to this email, bugs in IOR/mdtest source code were fixed to
ensure correctly reported results and we have insisted on using
stonewalling to more accurately and homogeneously measure the “straggler”
effect.
The mdtest “shift” is designed to guarantee that there are no client-side
caching effects artificially inflating any of the mdtest results. We had
hoped that client-side caching effects were already avoided due to the
interleaving of the phases and due to the 300 second required minimum
runtime. However, we were starting to see a few outlier submissions that
were achieving results that were obviously benefiting from client-side
cache since they were physically higher than would be possible without
caching.
It has been our hope that this change will not impact the historical
results because the only submissions that were benefiting from client-side
caching were the obvious outliers which we have already removed. However,
if our hope is unfounded and the new rules cause discernible differences,
then we may decide a new list is warranted and we may accordingly archive
the old results and start anew with SC’19.
After we get all of the SC19 submissions, we will analyze them and make
this decision. We do not yet have firm quantitative criteria for making
this judgment; we are considering simply looking at scatter-plots and using
our best judgement. Certainly, if anyone in the community can suggest
better statistical analysis methods, we would like to hear about them.
Given all this, we strongly encourage all historical submitters to submit
new results for SC’19. This will better enable us to make our decision and
this will ensure full inclusion if a new list is warranted. To provide
further motivation, and in the interests of full disclosure, some early
results that we have seen do suggest that the rule change might be more
significant than we hoped.
Ultimately, we are happy about this change which frankly was overdue even
if it may unfortunately result in a new list. As is evidenced by the fact
that IOR has always had the comparable “shift” and by the interleaving of
the phases, it has always been our intention to avoid client-side caching.
Unfortunately, the mdtest “shift” was never properly implemented until just
a few months ago. Change is never easy and we appreciate the patience of
the community as we continue to strive to ensure that IO500 is fair,
representative, and useful to all.
Thanks,
The IO500 Committee
4 years, 5 months
NEW mdtest parameter influence the stat result?
by Ruibo Wang
Dear All,
I've noticed that there is a newly added "-N 1" parameter to the mdtest
in the lastest SC19 version. I guess it would influence the stat result
dramatically.
If we use the new parameter, how could we compete the old results in
the IO500 list?
Ruibo
4 years, 5 months
Announce New VM Rule
by John Bent
To IO500 Community,
The committee has received some queries about the rules concerning virtual
machines for the 10 Node Challenge. As such, the committee has added the
following rule:
13. For the 10 Node Challenge, there must be exactly 10 physical nodes for
client processes and at least one benchmark process must run on each
1.
Virtual machines can be used but the above rule must be followed. More
than one virtual machine can be run on each physical node.
Although we recognize that this may disadvantage cloud architectures, we do
want to stress that this rule only applies to the 10 Node Challenge. The
committee did feel it was important to add this rule to ensure that the 10
Node Challenge sublist offers the maximum potential for fair comparisons by
ensuring equivalent client hardware quantities. Submissions with any
number/combination of virtual and physical machines can of course always be
submitted to the full list.
Thank you,
The IO500 Committee
4 years, 6 months
Trending Technology Specific Accounts 2019
by Kristi Jensen
Hi,
Hope all is well!
We are a database organization. We provide business executives' contact information.
Below, I've included a few examples:
Industry-Specific Lists: Agriculture, Business Services, Chambers of Commerce, Cities, Towns & Municipalities, Construction, Consumer Services, Cultural, Education, Energy, Utilities & Waste Treatment, Finance, Government, Healthcare, Hospitality, Insurance, Law Firms & Legal Services, Manufacturing, Media & Internet, Metals & Mining, Organizations, Real Estate, Retail, Software, Telecommunications, Transportation, and more!
Technology-Specific Lists: SAP users, PeopleSoft users, SIEBEL customers, Oracle Application customers, Symantec Users, Microsoft Dynamic users, Sales force users, Microsoft Exchange users, QuickBooks, Lawson users, Act users, JD Edward users, ASP users, Microsoft GP Applications users, Net Suite users, IBM DBMS Application users, McAfee users, MS Dynamics GP (Great Plains), and many more.
Title-Specific Lists: C-level executives: CEO, CFO, CIO, CTO, CMO, CISO, CSO, COO Key decision-makers: All C-level, VP-level, and Director-level executives HR Executives: VP of HR, HR Director & HR Manager, etc.
Marketing Executives: CMO, VP of Marketing, Director of Marketing,
Marketing Managers IT Executives: CIO, CTO, CISO, IT-VP, IT-Director, IT Manager, MIS Manager, etc.
Please keep me informed for any additional details. I look forward to hearing from you.
Thanks,
Kristi Jensen
Marketing Executive
if you don't want to include yourself in our mailing list, please reply back "Leave Out" in a subject line
4 years, 6 months
