Hi Matt,
sorry, for the delay with the reply.
I prepared a short example script with the data that I sent before. It
illustrates the high noise in the default Larch chi(k) data, the lack of
help from rebin_xafs function and the improvement obtained by rebinning
the data in k. Please find it attached (together with the data and
resulting figure).
Please let me know if you find a mistake in my approach. I will be
grateful for any suggestion.
Thank you!
Best regards,
Kirill
On 26/11/2024 21:42, Matt Newville wrote:
> Hi Kiril,
>
> Thanks, but I don't know how you got those different curves. Can you
> supply the code of how you got those?
> Larch's rebin_xafs() does rebin mu(E) data to an even k-grid, averaging
> data within a k-region. It can do the average either as the centroid
> (<mu*E>/<E>) or a simple boxcar average (<mu>) over any region.
>
> At k~15, it should be using 15 to 20 values if the energy spacing is
> 0.33 eV, so a noise reduction of a factor of 4 would seem believable.
>
> --Matt
>
>
>
> On Tue, Nov 26, 2024 at 6:38 AM Kirill LOMACHENKO via Ifeffit
> <ifeffit@millenia.cars.aps.anl.gov
> <mailto:ifeffit@millenia.cars.aps.anl.gov>> wrote:
>
> __
> Dear Matt, thank you for your reply. As I wrote, I did try
> rebin_xafs function, but it did not seem to improve the noise. It is
> not completely clear to me why, that was one of the questions.
> Regarding the suggestions, I found that rebinning
> ZjQcmQRYFpfptBannerStart
> __
> This Message Is From an External Sender
> This message came from outside your organization.
> __
> ZjQcmQRYFpfptBannerEnd
>
> Dear Matt,
> thank you for your reply.
>
> As I wrote, I did try rebin_xafs function, but it did not seem to
> improve the noise. It is not completely clear to me why, that was one of
> the questions.
>
> Regarding the suggestions, I found that rebinning of chi (chi(E) or
> chi(k)) yields much lower noise than rebinning of mu(E). With this
> approach the resulting chi(k) is noise-wise very similar to Athena.
>
> My findings were summarized in a figure. I am not sure whether you got
> it through the mailing list, so just in case I attach it here and add
> your e-mail in cc.
>
> All the best,
> Kirill
>
>
> On 23/11/2024 04:09, Matthew Newville wrote:
> > Hi Kiril,
> >
> > Yes, this is a common issue, especially for data from some beamlines
> > that do a continuous or fly-scan data collection (I'm 100% in favor of
> > this). There is a function for that. See |larch.xafs.rebin_xafs:|
> > https://urldefense.us/v3/__https://github.com/xraypy/xraylarch/blob/
> master/larch/xafs/__;!!G_uCfscf7eWS!
> Z6DCYyVN0uG5s3uCOcy03cAkLD6A6Fo0sZSDRwq_5OPp2mzDTlFp6NlwnNAIc2HqnSeCAjPQsPNkyAUmNHgINlwRXdqQSCWJkg$ <https://urldefense.us/v3/__https://github.com/xraypy/xraylarch/blob/master/larch/xafs/__;!!G_uCfscf7eWS!Z6DCYyVN0uG5s3uCOcy03cAkLD6A6Fo0sZSDRwq_5OPp2mzDTlFp6NlwnNAIc2HqnSeCAjPQsPNkyAUmNHgINlwRXdqQSCWJkg$>
> > rebin_xafs.py#L55 <https://urldefense.us/v3/__https://github.com/xraypy/xraylarch/blob/
> master/__;!!G_uCfscf7eWS!
> Z6DCYyVN0uG5s3uCOcy03cAkLD6A6Fo0sZSDRwq_5OPp2mzDTlFp6NlwnNAIc2HqnSeCAjPQsPNkyAUmNHgINlwRXdoZkMNpOQ$ <https://urldefense.us/v3/__https://github.com/xraypy/xraylarch/blob/master/__;!!G_uCfscf7eWS!Z6DCYyVN0uG5s3uCOcy03cAkLD6A6Fo0sZSDRwq_5OPp2mzDTlFp6NlwnNAIc2HqnSeCAjPQsPNkyAUmNHgINlwRXdoZkMNpOQ$>
> > larch/xafs/rebin_xafs.py#L55>
> >
> > For the data re-binning to each point in the output, this can do either
> > a boxcar average (which might reduce resolution) or use a centroid value
> > (which might do less resolution reduction). Either of these should be
> > better (in the sense of "use more of the input data") than a simple
> > interpolation.
> >
> >
> > The Larix GUI will prompt users to use this to re-bin XAFS data that
> > looks like it has too many points (more than 1200 points) or too fine an
> > energy spacing (< 0.75eV) in the high-energy portion of the spectra.
> > So, I think your data would be noticed as "needs rebinning".
> >
> > --Matt
> >
> > PS: suggestions for better approaches welcome!
> >
> >
> > ------------------------------------------------------------------------
> > *From:* Kirill LOMACHENKO via Ifeffit <ifeffit@millenia.cars.aps.anl.gov <mailto:ifeffit@millenia.cars.aps.anl.gov>>
> > *Sent:* Friday, November 22, 2024 12:10 PM
> > *To:* ifeffit@millenia.cars.aps.anl.gov <mailto:ifeffit@millenia.cars.aps.anl.gov> <ifeffit@millenia.cars.aps.anl.gov <mailto:ifeffit@millenia.cars.aps.anl.gov>>
> > *Cc:* Kirill LOMACHENKO <lomachenko@esrf.fr <mailto:lomachenko@esrf.fr>>
> > *Subject:* [Ifeffit] Larch: noise in chi(k) - another try
> > This Message Is From an External Sender
> > This message came from outside your organization.
> >
> > Dear Matt and dear all, seems that my original message has been cut by
> > the mailing server. I try again, this time without attachments. I have
> > encountered an issue processing oversampled EXAFS data with Larch. In my
> > example the energy step is 0.33 eV in all parts of the scan, including
> > the EXAFS. When I use the default procedure in Larch (pre_edge and then
> > autobk), I get the noise level in chi(k) that is higher than I would
> > expect. Strangely, rebinning of mu(E) using rebin_xafs function does
> > not help, and even increases the noise. The lowest level of noise can be
> > obtained when rebinning the data in k (starting from the "raw"
> > oversampled chi(E) provided by Larch). In this last case, the result is
> > noise-wise very similar to Athena's default. I think that the high level
> > of noise after the default Larch processing can be due to the fact that
> > Larch does not seem to do averaging when converting the data from E to
> > k. Instead, it does interpolation with UnivariateSpline which by design
> > is not supposed to decrease the noise when applied to oversampled data.
> > Is this correct? Would it be worth changing? The reason why the
> > rebinning in E does not decrease the noise is not clear to me. Do you
> > have any idea? Has anyone run into similar issues? Any comments and
> > suggestions are welcome. Thank you Best regards, Kirill
> >
>
> ifeffit mailing list: https://urldefense.us/v3/__https://millenia.cars.aps.anl.gov/mailman3/__;!!G_uCfscf7eWS!c3A5XNciXKQ7VhgLpD9inPitKQOSK_z2wZ7Y4Fs2Zz06RA12ilfR5OcM5izglOolk9mXjemAN7NEb0HhHmFYoO52ROKloy6CJw$
> lists/ifeffit.millenia.cars.aps.anl.gov/ <https://
> millenia.cars.aps.anl.gov/mailman3/lists/
> ifeffit.millenia.cars.aps.anl.gov/>
> to unsubscribe, send mail to ifeffit-leave@millenia.cars.aps.anl.gov
> <mailto:ifeffit-leave@millenia.cars.aps.anl.gov>
>
>
>
> --
> --Matt Newville <newville at cars.uchicago.edu <http://
> cars.uchicago.edu>> 630-327-7411