Thanks Patrick.  Peng, here's something else from Cray showing that Lustre was the file system on 77% of the top 100 systems from the June 2018 top500.org list:

image.png

https://www.cray.com/blog/business-cards-change-passion-open-source-lustre-doesnt/

Which is not nearly the same ratio that we have seen thus far in IO500:

image.png

Which I just created using https://www.vi4io.org/assets/io500/2019-06/data.csv ("The list shows the best result for a given combination of system/institution/filesystem").  Note that I did combine a few to reduce the number of slices (e.g. s/GPFS/Spectrum Scale/).

Thanks,

John



On Tue, Sep 17, 2019 at 10:39 AM Patrick Farrell via IO-500 <io-500@vi4io.org> wrote:
Peng,

You might find the graph on slide 7 here interesting:
https://hps.vi4io.org/_media/events/2019/hpc-iodc-lustre_next_20_years-dilger.pdf

This gives a very rough overview, from public sources, of the scaling of large supercomputer file systems over time.  It is not very detailed, by necessity, but it is interesting as an overview.

Regards,
Patrick
From: IO-500 <io-500-bounces@vi4io.org> on behalf of Patrick Farrell via IO-500 <io-500@vi4io.org>
Sent: Tuesday, September 17, 2019 11:22 AM
To: Julian Kunkel <juliankunkel@googlemail.com>; Peng Yu <pengyu.ut@gmail.com>
Cc: io-500@vi4io.org <io-500@vi4io.org>
Subject: Re: [IO-500] A list file systems used in top I/O system
 
It's also worth noting that the TOP500 list itself is far from a comprehensive compendium of the fastest machines.  Many large government and industry owned machines do not participate in the list, for various reasons.

Regards,
Patrick
From: Julian Kunkel <juliankunkel@googlemail.com>
Sent: Tuesday, September 17, 2019 11:15 AM
To: Peng Yu <pengyu.ut@gmail.com>
Cc: Patrick Farrell <pfarrell@whamcloud.com>; io-500@vi4io.org <io-500@vi4io.org>
Subject: Re: [IO-500] A list file systems used in top I/O system
 
Hi Peng,
welcome.
The list is considerably new, that is why we hadn't have 500
entries... Look at the beginning of the Green500, for instance.

Everyone that submits is there, not every user/data center in the world submits.

Please consider to join the Slack:
https://join.slack.com/t/vi4io/shared_invite/enQtMjMyOTgxMDg0OTQ1LTcyYWJkYzJiMDUzMDU2YjE1NjFjMGNjZWEwYTM2NzQxNzcxMDExYmFmMjJjMDY3NjBiYTRjYTM1M2I3ZGE3NmM
That makes answering easy questions easier.

Best,
Julian

On Tue, Sep 17, 2019 at 5:03 PM Peng Yu via IO-500 <io-500@vi4io.org> wrote:
>
> Also, why there is only 100 entries? Should there be 500 entries? Thanks.
>
> On Tue, Sep 17, 2019 at 10:50 AM Peng Yu <pengyu.ut@gmail.com> wrote:
> >
> > Thanks. In this list, does it have all the top-performing I/O system
> > in the whole world? (Just like top500.org for supercomputers).
> >
> > Or this list is just partial? Meaning, some top I/O systems in the
> > world may not be listed on it?
> >
> > > Since you're on the IO-500 mailing list, I would think you're aware of the IO500...?  But, the IO500 is still probably the best answer to your question.  It specifies the file system used as part of each entry.
> > >
> > > https://www.vi4io.org/io500/start?fields=information__system,information__institution,information__storage_vendor,information__filesystem_type,information__client_nodes,information__client_total_procs,io500__score,io500__bw,io500__md,information__data&equation=&sort_asc=false&sort_by=io500__score&radarmax=6&query=
> >
> > --
> > Regards,
> > Peng
>
>
>
> --
> Regards,
> Peng
> _______________________________________________
> IO-500 mailing list
> IO-500@vi4io.org
> https://www.vi4io.org/mailman/listinfo/io-500



--
Dr. Julian Kunkel
Lecturer, Department of Computer Science
+44 (0) 118 378 8218
http://www.cs.reading.ac.uk/
https://hps.vi4io.org/
PGP Fingerprint: 1468 1A86 A908 D77E B40F 45D6 2B15 73A5 9D39 A28E
_______________________________________________
IO-500 mailing list
IO-500@vi4io.org
https://www.vi4io.org/mailman/listinfo/io-500