Dear all, I pushed some cleanups to the repository and a README.md file (top level). Hope that provides certain answers, we know the description are not yet perfect, but they will be improved over time given that additional questions arise. Thanks for those of you that send feedback, do not hesitate to give feedback. Do not hesitate to join the slack, here is the invite link that shall never expire: https://join.slack.com/t/vi4io/shared_invite/MjMyOTgxMDg0OTQ1LTE1MDQwNDE4... Usually, Georgios and me are there frequently and can help you with any question that arises. Regards Julian -- http://wr.informatik.uni-hamburg.de/people/julian_kunkel _______________________________________________ IO-500 mailing list IO-500(a)vi4io.org https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500

Hi George and all, Thanks for the information. BTW, ior_hard read benchmark has the option -R, which means readCheck. Why only this benchmark has the check option? Other write and read benchmarks do not have it. Regards, Osamu From: Georgios Markomanolis <georgios.markomanolis(a)kaust.edu.sa&gt; Subject: Re: [IO-500] Running of the benchmark Date: Fri, 1 Sep 2017 08:31:17 +0000 > Dear Osamu, > > The auto-detect is useful to tune the parameters for the five minutes minimum execution of the benchmark but this does not apply for the find command as it is too difficult and too extreme probably (Julian correct me if you have different opinion). Thus, the subtree is not important during auto-detect. > > Check in https://github.com/VI4IO/io-500-dev/tree/master/site- configs/kaust-georgios instructions in the 4th bullet about how to use the parallel find and how to define the number of the processes which should participate during the parallel find. > > As we are not the ones who developed some of these benchmarks (we just use them) we have not documented, but it is good idea to describe them, thanks for your feedback. > > Best regards, > George > ________________________________________ > George Markomanolis, PhD > Computational Scientist > KAUST Supercomputing Laboratory (KSL) > King Abdullah University of Science & Technology > Al Khawarizmi Bldg. (1) Room 0123 > Thuwal > Kingdom of Saudi Arabia > Mob: +966 56 325 9012 > Office: +966 12 808 0393 <tel:%2B966%2012%20808%200683> > > On 01/09/2017, 9:14 AM, "IO-500 on behalf of Osamu Tatebe" < io-500-bounces(a)vi4io.org on behalf of tatebe(a)cs.tsukuba.ac.jp&gt; wrote: > > Hi Julian and Georgios, > > Thanks for the information about scripts. > > Regarding subtree.cfg, it seems not to be determined by > auto-detect.sh. This may effect the find and the mdtest > benchmarks. How do we think about it? > > Also, if there is a document or an information about > the benchmark itself, i.e. ior-easy, md-easy, ior-hard, > md-hard and find, please let us know. > > Thanks, > Osamu > > From: Julian Kunkel <juliankunkel(a)googlemail.com&gt; > Subject: [IO-500] Running of the benchmark > Date: Tue, 29 Aug 2017 23:34:27 +0200 > > > Dear all, > > I pushed some cleanups to the repository and a README.md file (top level). > > Hope that provides certain answers, we know the description are not > > yet perfect, but they will be improved over time given that additional > > questions arise. > > > > Thanks for those of you that send feedback, do not hesitate to give feedback. > > > > Do not hesitate to join the slack, here is the invite link that shall > > never expire: > > https://join.slack.com/t/vi4io/shared_invite/ MjMyOTgxMDg0OTQ1LTE1MDQwNDE4MzctOTFmMmJiNmM4OQ > > > > Usually, Georgios and me are there frequently and can help you with > > any question that arises. > > > > Regards > > Julian > > > > -- > > http://wr.informatik.uni-hamburg.de/people/julian_kunkel > > _______________________________________________ > > IO-500 mailing list > > IO-500(a)vi4io.org > > https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 > _______________________________________________ > IO-500 mailing list > IO-500(a)vi4io.org > https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 > > > > ________________________________ > This message and its contents including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.

On Sep 26, 2017, at 7:53 PM, Osamu Tatebe <tatebe(a)cs.tsukuba.ac.jp&gt; wrote: Hi Julian, This checkRead re-reads data and check for errors between reads. I am afraid this re-read check may not be your intention. Regards, Osamu From: Julian Kunkel <juliankunkel(a)googlemail.com&gt; Subject: Re: [IO-500] Running of the benchmark Date: Tue, 26 Sep 2017 20:07:24 +0200 > Dear Osamu, > Thanks for carefully looking > The write benchmark does not have to do it since read does it. So then we > spot the problem. > We could do it for the sequential pattern too, that is right. I do have > much more confidence here that the result is bit identical than to the > random one. Would wonder if random works but sequential doesn't. The > amounts of data to check for random is far less so this is literally no > overhead while for the sequential it may add performance issues when e.g. > reading from bust buffer. Truly we could have done it but i hope u > understand this rationales here. > > Regards > Julian > > Am 26.09.2017 1:34 nachm. schrieb "Osamu Tatebe" <tatebe(a)cs.tsukuba.ac.jp&gt;: > >> Hi George and all, >> >> Thanks for the information. >> >> BTW, ior_hard read benchmark has the option -R, which >> means readCheck. Why only this benchmark has the check >> option? Other write and read benchmarks do not have it. >> >> Regards, >> Osamu >> >> From: Georgios Markomanolis <georgios.markomanolis(a)kaust.edu.sa&gt; >> Subject: Re: [IO-500] Running of the benchmark >> Date: Fri, 1 Sep 2017 08:31:17 +0000 >> >>> Dear Osamu, >>> >>> The auto-detect is useful to tune the parameters for the five minutes >> minimum execution of the benchmark but this does not apply for the find >> command as it is too difficult and too extreme probably (Julian correct me >> if you have different opinion). Thus, the subtree is not important during >> auto-detect. >>> >>> Check in https://github.com/VI4IO/io-500-dev/tree/master/site- >> configs/kaust-georgios instructions in the 4th bullet about how to use >> the parallel find and how to define the number of the processes which >> should participate during the parallel find. >>> >>> As we are not the ones who developed some of these benchmarks (we just >> use them) we have not documented, but it is good idea to describe them, >> thanks for your feedback. >>> >>> Best regards, >>> George >>> ________________________________________ >>> George Markomanolis, PhD >>> Computational Scientist >>> KAUST Supercomputing Laboratory (KSL) >>> King Abdullah University of Science & Technology >>> Al Khawarizmi Bldg. (1) Room 0123 >>> Thuwal >>> Kingdom of Saudi Arabia >>> Mob: +966 56 325 9012 >>> Office: +966 12 808 0393 <tel:%2B966%2012%20808%200683> >>> >>> On 01/09/2017, 9:14 AM, "IO-500 on behalf of Osamu Tatebe" < >> io-500-bounces(a)vi4io.org on behalf of tatebe(a)cs.tsukuba.ac.jp&gt; wrote: >>> >>> Hi Julian and Georgios, >>> >>> Thanks for the information about scripts. >>> >>> Regarding subtree.cfg, it seems not to be determined by >>> auto-detect.sh. This may effect the find and the mdtest >>> benchmarks. How do we think about it? >>> >>> Also, if there is a document or an information about >>> the benchmark itself, i.e. ior-easy, md-easy, ior-hard, >>> md-hard and find, please let us know. >>> >>> Thanks, >>> Osamu >>> >>> From: Julian Kunkel <juliankunkel(a)googlemail.com&gt; >>> Subject: [IO-500] Running of the benchmark >>> Date: Tue, 29 Aug 2017 23:34:27 +0200 >>> >>>> Dear all, >>>> I pushed some cleanups to the repository and a README.md file (top >> level). >>>> Hope that provides certain answers, we know the description are not >>>> yet perfect, but they will be improved over time given that >> additional >>>> questions arise. >>>> >>>> Thanks for those of you that send feedback, do not hesitate to >> give feedback. >>>> >>>> Do not hesitate to join the slack, here is the invite link that >> shall >>>> never expire: >>>> https://join.slack.com/t/vi4io/shared_invite/ >> MjMyOTgxMDg0OTQ1LTE1MDQwNDE4MzctOTFmMmJiNmM4OQ >>>> >>>> Usually, Georgios and me are there frequently and can help you with >>>> any question that arises. >>>> >>>> Regards >>>> Julian >>>> >>>> -- >>>> http://wr.informatik.uni-hamburg.de/people/julian_kunkel >>>> _______________________________________________ >>>> IO-500 mailing list >>>> IO-500(a)vi4io.org >>>> https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 >>> _______________________________________________ >>> IO-500 mailing list >>> IO-500(a)vi4io.org >>> https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 >>> >>> >>> >>> ________________________________ >>> This message and its contents including attachments are intended solely >> for the original recipient. If you are not the intended recipient or have >> received this message in error, please notify me immediately and delete >> this message from your computer system. Any unauthorized use or >> distribution is prohibited. Please consider the environment before printing >> this email. >> _______________________________________________ IO-500 mailing list IO-500(a)vi4io.org https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500

Dear Osamu, Thanks for catching this. You are right. Remove the option when running as it is not useful but does not harm either (except for the runtime). Indeed I see now that something is missing on IOR. Since the -R option of IOR does make sense only when someone runs with one iteration and write/read phase, I believe that IOR misses the option that checks upon each read that the data is correct. This is useful for the described scenario when splitting write and read phase and also when one runs multiple iterations (that actually each iteration the offset should be moved by the rank offset again for each iteration and not just one time to actually hit multiple ranks, just another minor suboptimality I find, which spoils IOR results if someone uses multiple iterations and read only).

I will prepare a patch for IOR to provide that functionality.

Julian Am 27.09.2017 03:53 schrieb "Osamu Tatebe" <tatebe(a)cs.tsukuba.ac.jp&gt;: Hi Julian, This checkRead re-reads data and check for errors between reads. I am afraid this re-read check may not be your intention. Regards, Osamu From: Julian Kunkel <juliankunkel(a)googlemail.com&gt; Subject: Re: [IO-500] Running of the benchmark Date: Tue, 26 Sep 2017 20:07:24 +0200 > Dear Osamu, > Thanks for carefully looking > The write benchmark does not have to do it since read does it. So then we > spot the problem. > We could do it for the sequential pattern too, that is right. I do have > much more confidence here that the result is bit identical than to the > random one. Would wonder if random works but sequential doesn't. The > amounts of data to check for random is far less so this is literally no > overhead while for the sequential it may add performance issues when e.g. > reading from bust buffer. Truly we could have done it but i hope u > understand this rationales here. > > Regards > Julian > > Am 26.09.2017 1:34 nachm. schrieb "Osamu Tatebe" <tatebe(a)cs.tsukuba.ac.jp&gt;: > >> Hi George and all, >> >> Thanks for the information. >> >> BTW, ior_hard read benchmark has the option -R, which >> means readCheck. Why only this benchmark has the check >> option? Other write and read benchmarks do not have it. >> >> Regards, >> Osamu >> >> From: Georgios Markomanolis <georgios.markomanolis(a)kaust.edu.sa&gt; >> Subject: Re: [IO-500] Running of the benchmark >> Date: Fri, 1 Sep 2017 08:31:17 +0000 >> >> > Dear Osamu, >> > >> > The auto-detect is useful to tune the parameters for the five minutes >> minimum execution of the benchmark but this does not apply for the find >> command as it is too difficult and too extreme probably (Julian correct me >> if you have different opinion). Thus, the subtree is not important during >> auto-detect. >> > >> > Check in https://github.com/VI4IO/io-500-dev/tree/master/site- >> configs/kaust-georgios instructions in the 4th bullet about how to use >> the parallel find and how to define the number of the processes which >> should participate during the parallel find. >> > >> > As we are not the ones who developed some of these benchmarks (we just >> use them) we have not documented, but it is good idea to describe them, >> thanks for your feedback. >> > >> > Best regards, >> > George >> > ________________________________________ >> > George Markomanolis, PhD >> > Computational Scientist >> > KAUST Supercomputing Laboratory (KSL) >> > King Abdullah University of Science & Technology >> > Al Khawarizmi Bldg. (1) Room 0123 >> > Thuwal >> > Kingdom of Saudi Arabia >> > Mob: +966 56 325 9012 >> > Office: +966 12 808 0393 <tel:%2B966%2012%20808%200683> >> > >> > On 01/09/2017, 9:14 AM, "IO-500 on behalf of Osamu Tatebe" < >> io-500-bounces(a)vi4io.org on behalf of tatebe(a)cs.tsukuba.ac.jp&gt; wrote: >> > >> > Hi Julian and Georgios, >> > >> > Thanks for the information about scripts. >> > >> > Regarding subtree.cfg, it seems not to be determined by >> > auto-detect.sh. This may effect the find and the mdtest >> > benchmarks. How do we think about it? >> > >> > Also, if there is a document or an information about >> > the benchmark itself, i.e. ior-easy, md-easy, ior-hard, >> > md-hard and find, please let us know. >> > >> > Thanks, >> > Osamu >> > >> > From: Julian Kunkel <juliankunkel(a)googlemail.com&gt; >> > Subject: [IO-500] Running of the benchmark >> > Date: Tue, 29 Aug 2017 23:34:27 +0200 >> > >> > > Dear all, >> > > I pushed some cleanups to the repository and a README.md file (top >> level). >> > > Hope that provides certain answers, we know the description are not >> > > yet perfect, but they will be improved over time given that >> additional >> > > questions arise. >> > > >> > > Thanks for those of you that send feedback, do not hesitate to >> give feedback. >> > > >> > > Do not hesitate to join the slack, here is the invite link that >> shall >> > > never expire: >> > > https://join.slack.com/t/vi4io/shared_invite/ >> MjMyOTgxMDg0OTQ1LTE1MDQwNDE4MzctOTFmMmJiNmM4OQ >> > > >> > > Usually, Georgios and me are there frequently and can help you with >> > > any question that arises. >> > > >> > > Regards >> > > Julian >> > > >> > > -- >> > > http://wr.informatik.uni-hamburg.de/people/julian_kunkel >> > > _______________________________________________ >> > > IO-500 mailing list >> > > IO-500(a)vi4io.org >> > > https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 >> > _______________________________________________ >> > IO-500 mailing list >> > IO-500(a)vi4io.org >> > https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 >> > >> > >> > >> > ________________________________ >> > This message and its contents including attachments are intended solely >> for the original recipient. If you are not the intended recipient or have >> received this message in error, please notify me immediately and delete >> this message from your computer system. Any unauthorized use or >> distribution is prohibited. Please consider the environment before printing >> this email. >> _______________________________________________ IO-500 mailing list IO-500(a)vi4io.org https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500

I thought -C was for invalidating the local caches by rearranging MPI ranks at the read phase. Read verify is just a reread for verifying the written pattern. I need to check the code again. Sent from my iPad > On Sep 27, 2017, at 12:43 AM, Andreas Dilger <adilger(a)dilger.ca&gt; wrote: > >> On Sep 26, 2017, at 8:35 PM, Julian Kunkel <juliankunkel(a)googlemail.com&gt; wrote: >> >> Dear Osamu, >> Thanks for catching this. >> You are right. Remove the option when running as it is not useful but does not harm either (except for the runtime). >> >> Indeed I see now that something is missing on IOR. >> Since the -R option of IOR does make sense only when someone runs with one iteration and write/read phase, I believe that IOR misses the option that checks upon each read that the data is correct. This is useful for the described scenario when splitting write and read phase and also when one runs multiple iterations (that actually each iteration the offset should be moved by the rank offset again for each iteration and not just one time to actually hit multiple ranks, just another minor suboptimality I find, which spoils IOR results if someone uses multiple iterations and read only). > > I believe the read-verify option to IOR does two things: > - it verifies that the data written during the write phase is valid > - it offsets the MPI ranks that are reading the file, so that they don't just > read from the local client cache > > While the read verification is of some interest to the IO-500 (we don't want > results of a filesystem that equates to "write > /dev/null; read < /dev/zero"), > I think the main reason for using read-verify was to avoid reading from the > local client cache. > >> I will prepare a patch for IOR to provide that functionality. > > I think the current option already does this. It may be that if the write and > read are not in the same IOR run that we need to pass a seed/timestamp as an > argument, so that the IOR can verify the data is correct ("-G N" it looks like). > > Cheers, Andreas > >> Julian >> >> Am 27.09.2017 03:53 schrieb "Osamu Tatebe" <tatebe(a)cs.tsukuba.ac.jp&gt;: >> Hi Julian, >> >> This checkRead re-reads data and check for errors between >> reads. I am afraid this re-read check may not be your >> intention. >> >> Regards, >> Osamu >> >> From: Julian Kunkel <juliankunkel(a)googlemail.com&gt; >> Subject: Re: [IO-500] Running of the benchmark >> Date: Tue, 26 Sep 2017 20:07:24 +0200 >> >>> Dear Osamu, >>> Thanks for carefully looking >>> The write benchmark does not have to do it since read does it. So then we >>> spot the problem. >>> We could do it for the sequential pattern too, that is right. I do have >>> much more confidence here that the result is bit identical than to the >>> random one. Would wonder if random works but sequential doesn't. The >>> amounts of data to check for random is far less so this is literally no >>> overhead while for the sequential it may add performance issues when e.g. >>> reading from bust buffer. Truly we could have done it but i hope u >>> understand this rationales here. >>> >>> Regards >>> Julian >>> >>> Am 26.09.2017 1:34 nachm. schrieb "Osamu Tatebe" <tatebe(a)cs.tsukuba.ac.jp&gt;: >>> >>>> Hi George and all, >>>> >>>> Thanks for the information. >>>> >>>> BTW, ior_hard read benchmark has the option -R, which >>>> means readCheck. Why only this benchmark has the check >>>> option? Other write and read benchmarks do not have it. >>>> >>>> Regards, >>>> Osamu >>>> >>>> From: Georgios Markomanolis <georgios.markomanolis(a)kaust.edu.sa&gt; >>>> Subject: Re: [IO-500] Running of the benchmark >>>> Date: Fri, 1 Sep 2017 08:31:17 +0000 >>>> >>>>> Dear Osamu, >>>>> >>>>> The auto-detect is useful to tune the parameters for the five minutes >>>> minimum execution of the benchmark but this does not apply for the find >>>> command as it is too difficult and too extreme probably (Julian correct me >>>> if you have different opinion). Thus, the subtree is not important during >>>> auto-detect. >>>>> >>>>> Check in https://github.com/VI4IO/io-500-dev/tree/master/site- >>>> configs/kaust-georgios instructions in the 4th bullet about how to use >>>> the parallel find and how to define the number of the processes which >>>> should participate during the parallel find. >>>>> >>>>> As we are not the ones who developed some of these benchmarks (we just >>>> use them) we have not documented, but it is good idea to describe them, >>>> thanks for your feedback. >>>>> >>>>> Best regards, >>>>> George >>>>> ________________________________________ >>>>> George Markomanolis, PhD >>>>> Computational Scientist >>>>> KAUST Supercomputing Laboratory (KSL) >>>>> King Abdullah University of Science & Technology >>>>> Al Khawarizmi Bldg. (1) Room 0123 >>>>> Thuwal >>>>> Kingdom of Saudi Arabia >>>>> Mob: +966 56 325 9012 >>>>> Office: +966 12 808 0393 <tel:%2B966%2012%20808%200683> >>>>> >>>>>> On 01/09/2017, 9:14 AM, "IO-500 on behalf of Osamu Tatebe" < >>>>> io-500-bounces(a)vi4io.org on behalf of tatebe(a)cs.tsukuba.ac.jp&gt; wrote: >>>>> >>>>> Hi Julian and Georgios, >>>>> >>>>> Thanks for the information about scripts. >>>>> >>>>> Regarding subtree.cfg, it seems not to be determined by >>>>> auto-detect.sh. This may effect the find and the mdtest >>>>> benchmarks. How do we think about it? >>>>> >>>>> Also, if there is a document or an information about >>>>> the benchmark itself, i.e. ior-easy, md-easy, ior-hard, >>>>> md-hard and find, please let us know. >>>>> >>>>> Thanks, >>>>> Osamu >>>>> >>>>> From: Julian Kunkel <juliankunkel(a)googlemail.com&gt; >>>>> Subject: [IO-500] Running of the benchmark >>>>> Date: Tue, 29 Aug 2017 23:34:27 +0200 >>>>> >>>>>> Dear all, >>>>>> I pushed some cleanups to the repository and a README.md file (top >>>> level). >>>>>> Hope that provides certain answers, we know the description are not >>>>>> yet perfect, but they will be improved over time given that >>>> additional >>>>>> questions arise. >>>>>> >>>>>> Thanks for those of you that send feedback, do not hesitate to >>>> give feedback. >>>>>> >>>>>> Do not hesitate to join the slack, here is the invite link that >>>> shall >>>>>> never expire: >>>>>> https://join.slack.com/t/vi4io/shared_invite/ >>>> MjMyOTgxMDg0OTQ1LTE1MDQwNDE4MzctOTFmMmJiNmM4OQ >>>>>> >>>>>> Usually, Georgios and me are there frequently and can help you with >>>>>> any question that arises. >>>>>> >>>>>> Regards >>>>>> Julian >>>>>> >>>>>> -- >>>>>> http://wr.informatik.uni-hamburg.de/people/julian_kunkel >>>>>> _______________________________________________ >>>>>> IO-500 mailing list >>>>>> IO-500(a)vi4io.org >>>>>> https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 >>>>> _______________________________________________ >>>>> IO-500 mailing list >>>>> IO-500(a)vi4io.org >>>>> https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 >>>>> >>>>> >>>>> >>>>> ________________________________ >>>>> This message and its contents including attachments are intended solely >>>> for the original recipient. If you are not the intended recipient or have >>>> received this message in error, please notify me immediately and delete >>>> this message from your computer system. Any unauthorized use or >>>> distribution is prohibited. Please consider the environment before printing >>>> this email. >> _______________________________________________ >> IO-500 mailing list >> IO-500(a)vi4io.org >> https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500 > > > Cheers, Andreas > > > > > > _______________________________________________ > IO-500 mailing list > IO-500(a)vi4io.org > https://www.vi4io.org/cgi-bin/mailman/listinfo/io-500

> On Sep 27, 2017, at 8:51 AM, Julian Kunkel <juliankunkel(a)googlemail.com&gt; wrote: > > Indeed I believe the current -R option should be replaced by comparing > the value with the expected (written) value. I suggest that we check with the IOR maintainers before changing the semantics of an argument. I know them and I suspect they won't like this. Instead, perhaps just leave -R alone and add a new flag? John