Larch: noise in chi(k) - another try - Ifeffit

Larch: noise in chi(k) - another try

older
Larch: noise in chi(k)

Kirill LOMACHENKO

22 Nov 2024 22 Nov '24

6:10 p.m.

Attachments:

attachment.html (text/html — 5.6 KB)

Show replies by date

Hi Kiril, Yes, this is a common issue, especially for data from some beamlines that do a continuous or fly-scan data collection (I'm 100% in favor of this). There is a function for that. See larch.xafs.rebin_xafs: https://urldefense.us/v3/__https://github.com/xraypy/xraylarch/blob/master/l... For the data re-binning to each point in the output, this can do either a boxcar average (which might reduce resolution) or use a centroid value (which might do less resolution reduction). Either of these should be better (in the sense of "use more of the input data") than a simple interpolation. The Larix GUI will prompt users to use this to re-bin XAFS data that looks like it has too many points (more than 1200 points) or too fine an energy spacing (< 0.75eV) in the high-energy portion of the spectra. So, I think your data would be noticed as "needs rebinning". --Matt PS: suggestions for better approaches welcome! ________________________________ From: Kirill LOMACHENKO via Ifeffit Sent: Friday, November 22, 2024 12:10 PM To: ifeffit@millenia.cars.aps.anl.gov Cc: Kirill LOMACHENKO Subject: [Ifeffit] Larch: noise in chi(k) - another try This Message Is From an External Sender This message came from outside your organization. Dear Matt and dear all, seems that my original message has been cut by the mailing server. I try again, this time without attachments. I have encountered an issue processing oversampled EXAFS data with Larch. In my example the energy step is 0.33 eV in all parts of the scan, including the EXAFS. When I use the default procedure in Larch (pre_edge and then autobk), I get the noise level in chi(k) that is higher than I would expect. Strangely, rebinning of mu(E) using rebin_xafs function does not help, and even increases the noise. The lowest level of noise can be obtained when rebinning the data in k (starting from the "raw" oversampled chi(E) provided by Larch). In this last case, the result is noise-wise very similar to Athena's default. I think that the high level of noise after the default Larch processing can be due to the fact that Larch does not seem to do averaging when converting the data from E to k. Instead, it does interpolation with UnivariateSpline which by design is not supposed to decrease the noise when applied to oversampled data. Is this correct? Would it be worth changing? The reason why the rebinning in E does not decrease the noise is not clear to me. Do you have any idea? Has anyone run into similar issues? Any comments and suggestions are welcome. Thank you Best regards, Kirill

Kirill LOMACHENKO

26 Nov 26 Nov

12:38 p.m.

New subject: Larch: noise in chi(k)

Matt Newville

8:42 p.m.

New subject: Larch: noise in chi(k)

Hi Kiril, Thanks, but I don't know how you got those different curves. Can you supply the code of how you got those? Larch's rebin_xafs() does rebin mu(E) data to an even k-grid, averaging data within a k-region. It can do the average either as the centroid (/<E>) or a simple boxcar average (<mu>) over any region. At k~15, it should be using 15 to 20 values if the energy spacing is 0.33 eV, so a noise reduction of a factor of 4 would seem believable. --Matt On Tue, Nov 26, 2024 at 6:38 AM Kirill LOMACHENKO via Ifeffit < ifeffit@millenia.cars.aps.anl.gov> wrote:

...

Dear Matt, thank you for your reply. As I wrote, I did try rebin_xafs function, but it did not seem to improve the noise. It is not completely clear to me why, that was one of the questions. Regarding the suggestions, I found that rebinning ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

Dear Matt, thank you for your reply.

As I wrote, I did try rebin_xafs function, but it did not seem to improve the noise. It is not completely clear to me why, that was one of the questions.

Regarding the suggestions, I found that rebinning of chi (chi(E) or chi(k)) yields much lower noise than rebinning of mu(E). With this approach the resulting chi(k) is noise-wise very similar to Athena.

My findings were summarized in a figure. I am not sure whether you got it through the mailing list, so just in case I attach it here and add your e-mail in cc.

All the best, Kirill

On 23/11/2024 04:09, Matthew Newville wrote:

...
Hi Kiril,

Yes, this is a common issue, especially for data from some beamlines that do a continuous or fly-scan data collection (I'm 100% in favor of this). There is a function for that. See |larch.xafs.rebin_xafs:| https://urldefense.us/v3/__https://github.com/xraypy/xraylarch/blob/master/l... rebin_xafs.py#L55 <https://urldefense.us/v3/__https://github.com/xraypy/xraylarch/blob/master/_... larch/xafs/rebin_xafs.py#L55>

For the data re-binning to each point in the output, this can do either a boxcar average (which might reduce resolution) or use a centroid value (which might do less resolution reduction). Either of these should be better (in the sense of "use more of the input data") than a simple interpolation.

The Larix GUI will prompt users to use this to re-bin XAFS data that looks like it has too many points (more than 1200 points) or too fine an energy spacing (< 0.75eV) in the high-energy portion of the spectra. So, I think your data would be noticed as "needs rebinning".

--Matt

PS: suggestions for better approaches welcome!

------------------------------------------------------------------------ *From:* Kirill LOMACHENKO via Ifeffit *Sent:* Friday, November 22, 2024 12:10 PM *To:* ifeffit@millenia.cars.aps.anl.gov *Cc:* Kirill LOMACHENKO *Subject:* [Ifeffit] Larch: noise in chi(k) - another try This Message Is From an External Sender This message came from outside your organization.

Dear Matt and dear all, seems that my original message has been cut by the mailing server. I try again, this time without attachments. I have encountered an issue processing oversampled EXAFS data with Larch. In my example the energy step is 0.33 eV in all parts of the scan, including the EXAFS. When I use the default procedure in Larch (pre_edge and then autobk), I get the noise level in chi(k) that is higher than I would expect. Strangely, rebinning of mu(E) using rebin_xafs function does not help, and even increases the noise. The lowest level of noise can be obtained when rebinning the data in k (starting from the "raw" oversampled chi(E) provided by Larch). In this last case, the result is noise-wise very similar to Athena's default. I think that the high level of noise after the default Larch processing can be due to the fact that Larch does not seem to do averaging when converting the data from E to k. Instead, it does interpolation with UnivariateSpline which by design is not supposed to decrease the noise when applied to oversampled data. Is this correct? Would it be worth changing? The reason why the rebinning in E does not decrease the noise is not clear to me. Do you have any idea? Has anyone run into similar issues? Any comments and suggestions are welcome. Thank you Best regards, Kirill

ifeffit mailing list: https://urldefense.us/v3/__https://millenia.cars.aps.anl.gov/mailman3/lists/... to unsubscribe, send mail to ifeffit-leave@millenia.cars.aps.anl.gov

-- --Matt Newville <newville at cars.uchicago.edu> 630-327-7411

Age (days ago)

Last active (days ago)

List overview

Download

3 comments

3 participants

participants (3)

Kirill LOMACHENKO
Matt Newville
Matthew Newville