[Ifeffit] request for comments on XDI and NeXuS data formatting

[Matt Newville]

Hi John,


>  The improved NeXuS format for XAS data looks fine to me. Good idea.  But
it makes me wonder whether
> a similar formatting for theoretical XAS data would be useful, especially
for analysis purposes.

Thanks.  That is an interesting question. I don't have an immediate answer.

I think it is not too difficult to read the plain text output files from
FEFF and FDMNES.  I don't have much experience withther codes.  That is, I
expect that plain text files with a few labeled columns is probably good
enough for "energy" and "calculated mu".  I am not at all sure if there
could be common ways to label the other parts of the calculation like
broadening terms or partial DOS, etc.   And, it would be great to be able
to extend any conventions to resonant emission/absorption data like RIXS
plane or q-dependent X-ray Raman.

But it would be worth discussing this, maybe with other theorists, people
doing multiple XANES+DFT++ calculations, and maybe people from the
Materials Project.

The situation for experimental data is sort of worse, even for beamlines
that mostly collect XAFS and at well-run facilities. Beamline "raw data
files" might have 100 columns, and will often vary beamline-by-beamline and
sometimes month-to-month.  For transmission data, it is often easy to guess
how to build mu(E), but for fluorescence data, it is often very hard to
know what "the data" is.   It's OK if the people using the data know what
to do and then publish the reduced data.   But as we move (some faster than
others) toward being required to make all data available, this is becoming
a problem.

But, yes, we want to keep in mind that "experimental plain XAFS data"
should not be the only goal.


On Mon, Aug 7, 2023 at 12:44 PM John J Rehr <jjr at uw.edu> wrote:

> Hi Matt et al.
>
>   The improved NeXuS format for XAS data looks fine  to me. Good idea.
> But it makes me wonder whether
> a similar formatting  for theoretical XAS data would be useful, especially
> for analysis purposes.
>
>   Cheers,
>   John
>
>   Cheers,
>   John
>
> On Sun, Aug 6, 2023 at 11:45 PM Matt Newville <newville at cars.uchicago.edu>
> wrote:
>
>> Hi Folks,
>>
>> As some of you are aware, there is a Q2XAFS meeting in a few weeks. I
>> will be presenting some work on using NeXuS and HDF5 to store and
>> distribute XAS data.  This builds on and closely follows the XDI format
>> from Bruce Ravel.
>>
>> While XDI is a great way to represent a single XAS spectrum, it is not
>> able to contain multiple spectra.  It also is not very specific about how
>> to deal with data files with a large number (say, > 20) columns as might be
>> found coming from many beamlines with multi-element detectors.   I think
>> that many of us are starting to see the need to regularly provide "raw" or
>> "nearly raw" data as supplementary data for published articles or as part
>> of datasets made available under FAIR data principles.   While XDI could be
>> one way to do that, I think it is also reasonable to reconsider formats
>> other than plaintext files holding one spectrum.
>>
>> As I am preparing to present this, I've put together something like a
>> blog post on a proof-of-concept for a NeXuS file format based on XDI at
>> https://millenia.cars.aps.anl.gov/nxxas/
>> <https://urldefense.com/v3/__https://millenia.cars.aps.anl.gov/nxxas/__;!!K-Hz7m0Vt54!kMFKt35rA0N6RTvxlblky6bO9JXBO8OrG0-1fASPtZRZgMioM0kN4pXZ8yciGaDW8EP7Ap43KgPKx1WyX9Np-g$>.
>> There are a few example data files using the proposed layout at
>> https://millenia.cars.aps.anl.gov/nxxas/nexus_xas.html
>> <https://urldefense.com/v3/__https://millenia.cars.aps.anl.gov/nxxas/nexus_xas.html__;!!K-Hz7m0Vt54!kMFKt35rA0N6RTvxlblky6bO9JXBO8OrG0-1fASPtZRZgMioM0kN4pXZ8yciGaDW8EP7Ap43KgPKx1V12lXMVw$>.
>> There is also a Pull Request in with the NeXuS developers for this.
>>
>> If you are interested and have time over the next week or two, please
>> read the pages above and let me know your thoughts on this.  I know that
>> not everyone is interested in this, but I also know that some who might be
>> interested are not able to attend Q2XAFS.  I think the discussion on this
>> topic should be as wide as possible, which is why I am posting this here
>> and before the meeting.
>>
>> Thanks,
>>
>> --Matt Newville <newville at cars.uchicago.edu
>> <https://urldefense.com/v3/__http://cars.uchicago.edu__;!!K-Hz7m0Vt54!kMFKt35rA0N6RTvxlblky6bO9JXBO8OrG0-1fASPtZRZgMioM0kN4pXZ8yciGaDW8EP7Ap43KgPKx1WDPX86OA$>
>> >
>>
>> _______________________________________________
>> Ifeffit mailing list
>> Ifeffit at millenia.cars.aps.anl.gov
>>
>> https://urldefense.com/v3/__http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit__;!!K-Hz7m0Vt54!kMFKt35rA0N6RTvxlblky6bO9JXBO8OrG0-1fASPtZRZgMioM0kN4pXZ8yciGaDW8EP7Ap43KgPKx1UZvXteSQ$
>> Unsubscribe:
>> https://urldefense.com/v3/__http://millenia.cars.aps.anl.gov/mailman/options/ifeffit__;!!K-Hz7m0Vt54!kMFKt35rA0N6RTvxlblky6bO9JXBO8OrG0-1fASPtZRZgMioM0kN4pXZ8yciGaDW8EP7Ap43KgPKx1VT4Ccjqg$
>>
> _______________________________________________
> Ifeffit mailing list
> Ifeffit at millenia.cars.aps.anl.gov
> http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
> Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit
>


-- 
--Matt Newville <newville at cars.uchicago.edu> 630-327-7411
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20230808/d1b8f98d/attachment-0001.htm>