Re: [Ifeffit] request for comments on XDI and NeXuS data formatting

Hi John,

The improved NeXuS format for XAS data looks fine to me. Good idea. But it makes me wonder whether a similar formatting for theoretical XAS data would be useful, especially for analysis purposes.

Thanks. That is an interesting question. I don't have an immediate answer. I think it is not too difficult to read the plain text output files from FEFF and FDMNES. I don't have much experience withther codes. That is, I expect that plain text files with a few labeled columns is probably good enough for "energy" and "calculated mu". I am not at all sure if there could be common ways to label the other parts of the calculation like broadening terms or partial DOS, etc. And, it would be great to be able to extend any conventions to resonant emission/absorption data like RIXS plane or q-dependent X-ray Raman. But it would be worth discussing this, maybe with other theorists, people doing multiple XANES+DFT++ calculations, and maybe people from the Materials Project. The situation for experimental data is sort of worse, even for beamlines that mostly collect XAFS and at well-run facilities. Beamline "raw data files" might have 100 columns, and will often vary beamline-by-beamline and sometimes month-to-month. For transmission data, it is often easy to guess how to build mu(E), but for fluorescence data, it is often very hard to know what "the data" is. It's OK if the people using the data know what to do and then publish the reduced data. But as we move (some faster than others) toward being required to make all data available, this is becoming a problem. But, yes, we want to keep in mind that "experimental plain XAFS data" should not be the only goal. On Mon, Aug 7, 2023 at 12:44 PM John J Rehr wrote:

Hi Matt et al.

The improved NeXuS format for XAS data looks fine to me. Good idea. But it makes me wonder whether a similar formatting for theoretical XAS data would be useful, especially for analysis purposes.

Cheers, John

Cheers, John

On Sun, Aug 6, 2023 at 11:45 PM Matt Newville wrote:

...

Hi Folks,

As some of you are aware, there is a Q2XAFS meeting in a few weeks. I will be presenting some work on using NeXuS and HDF5 to store and distribute XAS data. This builds on and closely follows the XDI format from Bruce Ravel.

While XDI is a great way to represent a single XAS spectrum, it is not able to contain multiple spectra. It also is not very specific about how to deal with data files with a large number (say, > 20) columns as might be found coming from many beamlines with multi-element detectors. I think that many of us are starting to see the need to regularly provide "raw" or "nearly raw" data as supplementary data for published articles or as part of datasets made available under FAIR data principles. While XDI could be one way to do that, I think it is also reasonable to reconsider formats other than plaintext files holding one spectrum.

As I am preparing to present this, I've put together something like a blog post on a proof-of-concept for a NeXuS file format based on XDI at https://millenia.cars.aps.anl.gov/nxxas/ https://urldefense.com/v3/__https://millenia.cars.aps.anl.gov/nxxas/__;!!K-H.... There are a few example data files using the proposed layout at https://millenia.cars.aps.anl.gov/nxxas/nexus_xas.html https://urldefense.com/v3/__https://millenia.cars.aps.anl.gov/nxxas/nexus_xa.... There is also a Pull Request in with the NeXuS developers for this.

If you are interested and have time over the next week or two, please read the pages above and let me know your thoughts on this. I know that not everyone is interested in this, but I also know that some who might be interested are not able to attend Q2XAFS. I think the discussion on this topic should be as wide as possible, which is why I am posting this here and before the meeting.

Thanks,

--Matt Newville https://urldefense.com/v3/__http://cars.uchicago.edu__;!!K-Hz7m0Vt54!kMFKt35...

...

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov

https://urldefense.com/v3/__http://millenia.cars.aps.anl.gov/mailman/listinf... Unsubscribe: https://urldefense.com/v3/__http://millenia.cars.aps.anl.gov/mailman/options...

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

-- --Matt Newville <newville at cars.uchicago.edu> 630-327-7411