On Feb 3, 2017, at 8:04 AM, Julian Kunkel <juliankunkel(a)googlemail.com&gt; wrote: Dear all, firstly, I want to thank you that you joined this mailing list and your interest in this topic. I think this is an important effort that we aim to achieve. I recognize that this will only work, if there will be community that supports it. Personally, my goal is to increase comparability for storage the sake of everyone not just me or DKRZ. Moreover, I hope you have seen that I'm willing to invest time for the discussion, testing and documentation. But I appreciate and try to honor every input -- not ignoring a single voice because I do not like this in such discussions either. Let us have a look at the current status: * We had quite some discussion about the features and flaws and problems with current/future architectures. * I tried to summarize the current status of the discussion of the list here: https://docs.google.com/document/d/1zIl6XHjAJxjX3NMjNR49wkFoe42Lt9ti-Qsfi... * We aimed to have until the end of January: "Open discussion to define goals, identify concerns, access patterns" => We did not meet that goal, but based on the initial enthusiasm, that seemed possible two month ago. * I also moved along in respect to the metadata / small file benchmarking proposing something but I'm not fixated to this, it is just a suggestion based on my experience that I had for acceptance testing for DKRZ while searching for acceptable metadata benchmarks. Basically, I'm willing to run any benchmark on our system for comparison reason, if you propose sth. Also I would support to improve e.g., IOR, once the patterns are clear and accepted by the community. * I realized that there have not been much responses to pending emails lately. Any technical input is important. Also: if you have particular reasons how to improve the progress or any personal reason why you don't like the current approach/status or e.g. me :-) -- they are welcome. For me, the next important step is still to ensure that access patterns useful for current / future architectures and any middleware can be identified. I'm naive: I think we can be a wonderful community that can contribute even to unfunded activities for the sake of everyone. Regards, Julian -- http://wr.informatik.uni-hamburg.de/people/julian_kunkel _______________________________________________ IO-500 mailing list IO-500(a)vi4io.org https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500

On Feb 3, 2017, at 8:41 AM, Julian Kunkel <juliankunkel(a)googlemail.com&gt; wrote: Dear John, great, thanks. A link with some local results is here: https://www.vi4io.org/tools/benchmarks/md-real-io i have results for DKRZ and the Earth Simulator, too. This follows up on github with the code. Julian 2017-02-03 16:32 GMT+01:00 John Bent <John.Bent(a)seagategov.com&gt;: > Hi Julian, > > Thanks for being proactive on this. I’ve had meetings at two US national labs recently where people came up to me unprompted and said that they are subscribed to the list and that they like the basic idea (you know who you are!). So I don’t think we should interpret the lack of activity as a lack of interest! > > On my part, I’ve been meaning to look at your proposed mdtest alternative but haven’t gotten around to it yet. Can you send a link please? > > Thanks, > > John > >> On Feb 3, 2017, at 8:04 AM, Julian Kunkel <juliankunkel(a)googlemail.com&gt; wrote: >> >> Dear all, >> firstly, I want to thank you that you joined this mailing list and >> your interest in this topic. >> >> I think this is an important effort that we aim to achieve. >> I recognize that this will only work, if there will be community that >> supports it. >> Personally, my goal is to increase comparability for storage the sake >> of everyone not just me or DKRZ. Moreover, I hope you have seen that >> I'm willing to invest time for the discussion, testing and >> documentation. >> But I appreciate and try to honor every input -- not ignoring a single >> voice because I do not like this in such discussions either. >> >> Let us have a look at the current status: >> * We had quite some discussion about the features and flaws and >> problems with current/future architectures. >> * I tried to summarize the current status of the discussion of the list here: >> https://docs.google.com/document/d/1zIl6XHjAJxjX3NMjNR49wkFoe42Lt9ti-Qsfi... >> * We aimed to have until the end of January: "Open discussion to >> define goals, identify concerns, access patterns" => We did not meet >> that goal, but based on the initial enthusiasm, that seemed possible >> two month ago. >> * I also moved along in respect to the metadata / small file >> benchmarking proposing something but I'm not fixated to this, it is >> just a suggestion based on my experience that I had for acceptance >> testing for DKRZ while searching for acceptable metadata benchmarks. >> Basically, I'm willing to run any benchmark on our system for >> comparison reason, if you propose sth. >> Also I would support to improve e.g., IOR, once the patterns are clear >> and accepted by the community. >> >> * I realized that there have not been much responses to pending emails lately. >> Any technical input is important. >> Also: if you have particular reasons how to improve the progress or >> any personal reason why you don't like the current approach/status or >> e.g. me :-) -- they are welcome. >> For me, the next important step is still to ensure that access >> patterns useful for current / future architectures and any middleware >> can be identified. >> >> I'm naive: I think we can be a wonderful community that can contribute >> even to unfunded activities for the sake of everyone. >> >> Regards, >> Julian >> >> -- >> http://wr.informatik.uni-hamburg.de/people/julian_kunkel >> _______________________________________________ >> IO-500 mailing list >> IO-500(a)vi4io.org >> https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 > > > STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain > confidential and proprietary material for the sole use of > the intended recipient. Any review or distribution by others > is strictly prohibited. If you are not the intended recipient > please contact the sender and delete all copies. > _______________________________________________ > IO-500 mailing list > IO-500(a)vi4io.org > https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 -- http://wr.informatik.uni-hamburg.de/people/julian_kunkel

Just curious where we are in terms of coming to consensus. For those of us who agree that IO-500 is a worthwhile pursuit, are we generally agreed to the following: 1. The value of the benchmark is that users can have a reasonably accurate expectation of their workload¹s performance 2. The way to provide this expectation is to measure across the range of possible workloads 3. Therefore we should test both data and metadata with both the easiest and the hardest workloads 4. The data test tool should be IOR And we are still trying to figure out: 1. The metadata test tool 2. Whether and how to collapse the benchmark results into a single number 3. How to curate the test results Thanks, John > On Feb 3, 2017, at 8:41 AM, Julian Kunkel <juliankunkel(a)googlemail.com&gt; >wrote: > > Dear John, > great, thanks. > > A link with some local results is here: > https://www.vi4io.org/tools/benchmarks/md-real-io > > i have results for DKRZ and the Earth Simulator, too. > > This follows up on github with the code. > Julian > > 2017-02-03 16:32 GMT+01:00 John Bent <John.Bent(a)seagategov.com&gt;: >> Hi Julian, >> >> Thanks for being proactive on this. I¹ve had meetings at two US >>national labs recently where people came up to me unprompted and said >>that they are subscribed to the list and that they like the basic idea >>(you know who you are!). So I don¹t think we should interpret the lack >>of activity as a lack of interest! >> >> On my part, I¹ve been meaning to look at your proposed mdtest >>alternative but haven¹t gotten around to it yet. Can you send a link >>please? >> >> Thanks, >> >> John >> >>> On Feb 3, 2017, at 8:04 AM, Julian Kunkel &gt;&gt;&gt;&lt;juliankunkel(a)googlemail.com&gt; wrote: >>> >>> Dear all, >>> firstly, I want to thank you that you joined this mailing list and >>> your interest in this topic. >>> >>> I think this is an important effort that we aim to achieve. >>> I recognize that this will only work, if there will be community that >>> supports it. >>> Personally, my goal is to increase comparability for storage the sake >>> of everyone not just me or DKRZ. Moreover, I hope you have seen that >>> I'm willing to invest time for the discussion, testing and >>> documentation. >>> But I appreciate and try to honor every input -- not ignoring a single >>> voice because I do not like this in such discussions either. >>> >>> Let us have a look at the current status: >>> * We had quite some discussion about the features and flaws and >>> problems with current/future architectures. >>> * I tried to summarize the current status of the discussion of the >>>list here: >>> >>>https://docs.google.com/document/d/1zIl6XHjAJxjX3NMjNR49wkFoe42Lt9ti-Qs >>>fipWJV8c/edit >>> * We aimed to have until the end of January: "Open discussion to >>> define goals, identify concerns, access patterns" => We did not meet >>> that goal, but based on the initial enthusiasm, that seemed possible >>> two month ago. >>> * I also moved along in respect to the metadata / small file >>> benchmarking proposing something but I'm not fixated to this, it is >>> just a suggestion based on my experience that I had for acceptance >>> testing for DKRZ while searching for acceptable metadata benchmarks. >>> Basically, I'm willing to run any benchmark on our system for >>> comparison reason, if you propose sth. >>> Also I would support to improve e.g., IOR, once the patterns are clear >>> and accepted by the community. >>> >>> * I realized that there have not been much responses to pending >>>emails lately. >>> Any technical input is important. >>> Also: if you have particular reasons how to improve the progress or >>> any personal reason why you don't like the current approach/status or >>> e.g. me :-) -- they are welcome. >>> For me, the next important step is still to ensure that access >>> patterns useful for current / future architectures and any middleware >>> can be identified. >>> >>> I'm naive: I think we can be a wonderful community that can contribute >>> even to unfunded activities for the sake of everyone. >>> >>> Regards, >>> Julian >>> >>> -- >>> http://wr.informatik.uni-hamburg.de/people/julian_kunkel >>> _______________________________________________ >>> IO-500 mailing list >>> IO-500(a)vi4io.org >>> https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 >> >> >> STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain >> confidential and proprietary material for the sole use of >> the intended recipient. Any review or distribution by others >> is strictly prohibited. If you are not the intended recipient >> please contact the sender and delete all copies. >> _______________________________________________ >> IO-500 mailing list >> IO-500(a)vi4io.org >> https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 > > > > -- > http://wr.informatik.uni-hamburg.de/people/julian_kunkel STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain confidential and proprietary material for the sole use of the intended recipient. Any review or distribution by others is strictly prohibited. If you are not the intended recipient please contact the sender and delete all copies. _______________________________________________ IO-500 mailing list IO-500(a)vi4io.org https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500