Dear Matt Thank you very much for your quick reply. I attached three files: a raw data obtained from SPring-8 in Japan (PdO.q), .chik file obtained using Athena based on PdO.q (PdO.q.chik), and a python script I actually analyzed with Larch (EXAFSanalaysis.py). (When opening PdO.q with Athena we use the plugin named "PFBL12C".) If you put three files in the same directory and run “EXAFSanalysis.py”, you can see the previous PdO.png. I am looking forward to hearing from you. Best, Yuji From: Ifeffit [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of Matt Newville Sent: Tuesday, April 18, 2017 12:45 AM To: XAFS Analysis using Ifeffit Subject: Re: [Ifeffit] The difference of EXAFS oscillation when using Athena and when using Larch Hi Yuji, On Mon, Apr 17, 2017 at 9:49 AM, Yuji Mahara mailto:mahara.yuji@e.mbox.nagoya-u.ac.jp > wrote: Dear All, I am Yuji Mahara at Nagoya University, Japan. I am using Athena in Demeter 0.9.24 package and Larch 0.9.33 on windows 10. At that time, I am worrying about the difference of EXAFS oscillation when using Athena and when using Larch. k3-weighted Pd K-edge EXAFS and Fourier transform EXAFS spectrum for PdO bulk are shown in attached file(PdO.png). When XAFS raw data is read using Athena and EXAFS oscillation is obtained, EXAFS spectrum is smooth and noise is small. On the other hand, using Larch, noise is noticeable in k space. This phenomenon occurred when analyzing other XAFS data as well. I tried to analyze XAFS data by Athena and Larch with the almost same conditions. Does this difference of EXAFS results arise from the difference in autobk's algorithm between Athena and Larch? The analysis conditions are shown below. If someone knows this reason, please let me know. There may be subtle differences in the results from autobk() from Ifeffit and autobk() from Larch, but they should not be very big. If I recall correctly, the meaning of the values for the "clamps" is different between the two, but I don't think that accounts for the changes you're seeing. Like you say, the differences seem to be more about the noise in the data which is much more noticeable in the data processed with Larch than with Athena, as if the data is being smoothed or interpolated differently. if the data is very finely spaced or binned in some unexpected way from a QXAFS collection, that might help explain the difference, though I'm not sure of that or what behavior would be preferred. Is it possible to post one of the data files? --Matt Best regards, Yuji Mahara ****************************************** ■ Yuji Mahara Nagoya University(Japan) D3 Graduate School of Engineering Furo-cho, Chikusa-ku, Nagoya-shi, Aichi, 466-8603, Japan ・Phone : +81-52-789-3192 tel:%2B81-52-789-3192 ・E-mail : mailto:mahara.yuji@e.mbox.nagoya-u.ac.jp mahara.yuji@e.mbox.nagoya-u.ac.jp ****************************************** #####Analysis condition of Athena ##### e0 -->[ set e0 to a fraction of the edge step] --> 24355.9037323484 tel:%28903%29%20732-3484 Normalization order: 3 Pre-edge range: from -150 to -60 eV Normalization range: from 150 to 1100 eV Rbkg: 1 k-weight used by spline: 2 Spline clamp: low=None, high=Strong Spline range in k: from 0 to 17 Fourier transform k-range: from 3 to 15, dk=0.5, window=’hanning’ #####Analysis condition of Larch ##### dat.e0 = 24355.9037323484 tel:%28903%29%20732-3484 <-- e0 was estimated by Athena pre_edge(dat.e, dat.mu http://dat.mu , group=dat, e0=dat.e0, pre1=-150, pre2=-60, norm1=150, norm2=1100, nnorm=2, _larch=mylarch) autobk(dat.e, dat.mu http://dat.mu , group=dat, e0=dat.e0, rbkg=1, nclamp=5, clamp_lo=0, clamp_hi=24, kstep=0.05, kmin=0, kmax=17, kweight=2, win='hanning', _larch=mylarch) xftf(dat.k, dat.chi, group=dat, kmin=3, kmax=15, dk=0.5, window='hanning', kweight=3, _larch=mylarch) ######################### _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov mailto:Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

Dear Matt Thank you for detailed explanation about autobk() function. I could understand very well. It might be caused by data points being very dense and large as you say. When I analyzed with Larch using the data which tried to interpolate the energy, the noise certainly decreased (attached as “PdO_q_intp.ex3 and EXAFSanalysis_v3.py”). I had thought that autobk() function in Ifeffit(Athena) does "quatric interpolation". But, I didn’t know that the boxcar average smoothing was applied in Athena. Anyway, smoothing may be done using boxcar average if necessary. Sadly, I am not sure how to use Larch's algorithm of boxcar averaging convolution or Savitzky-Golay method. Although I confirmed Larch HP (http://cars.uchicago.edu/xraylarch/), I could not find out. please tell us how to use the algorithm if you can. Larch is a great library for me. I hope to further develop XAFS analysis using Larch. Best wishes, Yuji Mahara From: Ifeffit [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of Matt Newville Sent: Tuesday, April 18, 2017 11:52 AM To: XAFS Analysis using Ifeffit Subject: Re: [Ifeffit] The difference of EXAFS oscillation when using Athena and when using Larch Hi Yuji, On Mon, Apr 17, 2017 at 12:22 PM, Yuji Mahara mailto:mahara.yuji@e.mbox.nagoya-u.ac.jp > wrote: Dear Matt Thank you very much for your quick reply. I attached three files: a raw data obtained from SPring-8 in Japan (PdO.q), .chik file obtained using Athena based on PdO.q (PdO.q.chik), and a python script I actually analyzed with Larch (EXAFSanalaysis.py). (When opening PdO.q with Athena we use the plugin named "PFBL12C".) If you put three files in the same directory and run “EXAFSanalysis.py”, you can see the previous PdO.png. I am looking forward to hearing from you. Thank you very much for the nice report. I can reproduce what you see, and looked into this a bit more. The energy data you have is very finely spaced (the mu and energy arrays have 3991 energy points), leading to some question about how to convert that finely spaced energy data to the chi(k) grid of 0.05 Ang^-1. In both Ifeffit and Larch, the autobk() function determine an array for mu0(E) and chi(E) at all the energy data points provided. But to use later, chi(k) needs to be put onto evenly spaced values of k, with a step size of 0.05 Ang^-1 being both common and appropriate for EXAFS. For your data (see a modified version of your script, attached), there are 30 energy points between 15.0 and 15.10 Ang^-1. It's not obvious how to best use those data to give the value chi(k) at k=15.05. The autobk() function in Ifeffit essentially does a boxcar average, taking those ~15 data points closest to k=15.05 and averaging them with equal weighting. The autobk function in Larch takes the spline function of mu0(E) and evaluates that at the k-values of the grid, essentially doing a cubic-splice interpolation of the surrounding points to give the value at a particular k. I think this difference in interpolation explains the difference in chi(k) and chi(R) you see -- the Ifeffit data is smoother and so lower amplitude at high R. The high frequency "noise" has been filtered out by the boxcar average. The data processed by Larch does not impose this filter, so the data retains the higher-frequency components. The attached plot (from the the attached modified version of your EXAFSAnalysis.py) includes the data processed with your script, with Athena using the Larch backend, and with Athena using the Ifeffit backend. There are slight differences at low-R -- I didn't worry too much about setting clamps. I don't have a strong opinion, but think preserving the noise / high frequency parts of the data is probably preferred. You can smooth it with a variety of methods either before or after background subtraction (and Larch comes with a few common smoothing algorithms including boxcar averaging, convolution and the Savitzky-Golay method). When I do continuous XAFS, I bin the data to energy points selected to be spaced 0.05Ang^-1 apart. That effectively boxcar averages the data at the point of collection. I don't know that there is always a single best way to do that averaging/smoothing, but it would be interesting to hear the approach others take and seeing how we might be able to unify and/or simplify that part of the data reduction. --Matt Best, Yuji From: Ifeffit [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov mailto:ifeffit-bounces@millenia.cars.aps.anl.gov ] On Behalf Of Matt Newville Sent: Tuesday, April 18, 2017 12:45 AM To: XAFS Analysis using Ifeffit mailto:ifeffit@millenia.cars.aps.anl.gov > Subject: Re: [Ifeffit] The difference of EXAFS oscillation when using Athena and when using Larch Hi Yuji, On Mon, Apr 17, 2017 at 9:49 AM, Yuji Mahara mailto:mahara.yuji@e.mbox.nagoya-u.ac.jp > wrote: Dear All, I am Yuji Mahara at Nagoya University, Japan. I am using Athena in Demeter 0.9.24 package and Larch 0.9.33 on windows 10. At that time, I am worrying about the difference of EXAFS oscillation when using Athena and when using Larch. k3-weighted Pd K-edge EXAFS and Fourier transform EXAFS spectrum for PdO bulk are shown in attached file(PdO.png). When XAFS raw data is read using Athena and EXAFS oscillation is obtained, EXAFS spectrum is smooth and noise is small. On the other hand, using Larch, noise is noticeable in k space. This phenomenon occurred when analyzing other XAFS data as well. I tried to analyze XAFS data by Athena and Larch with the almost same conditions. Does this difference of EXAFS results arise from the difference in autobk's algorithm between Athena and Larch? The analysis conditions are shown below. If someone knows this reason, please let me know. There may be subtle differences in the results from autobk() from Ifeffit and autobk() from Larch, but they should not be very big. If I recall correctly, the meaning of the values for the "clamps" is different between the two, but I don't think that accounts for the changes you're seeing. Like you say, the differences seem to be more about the noise in the data which is much more noticeable in the data processed with Larch than with Athena, as if the data is being smoothed or interpolated differently. if the data is very finely spaced or binned in some unexpected way from a QXAFS collection, that might help explain the difference, though I'm not sure of that or what behavior would be preferred. Is it possible to post one of the data files? --Matt Best regards, Yuji Mahara ****************************************** ■ Yuji Mahara Nagoya University(Japan) D3 Graduate School of Engineering Furo-cho, Chikusa-ku, Nagoya-shi, Aichi, 466-8603, Japan ・Phone : +81-52-789-3192 tel:%2B81-52-789-3192 ・E-mail : mailto:mahara.yuji@e.mbox.nagoya-u.ac.jp mahara.yuji@e.mbox.nagoya-u.ac.jp ****************************************** #####Analysis condition of Athena ##### e0 -->[ set e0 to a fraction of the edge step] --> 24355.9037323484 tel:%28903%29%20732-3484 Normalization order: 3 Pre-edge range: from -150 to -60 eV Normalization range: from 150 to 1100 eV Rbkg: 1 k-weight used by spline: 2 Spline clamp: low=None, high=Strong Spline range in k: from 0 to 17 Fourier transform k-range: from 3 to 15, dk=0.5, window=’hanning’ #####Analysis condition of Larch ##### dat.e0 = 24355.9037323484 tel:%28903%29%20732-3484 <-- e0 was estimated by Athena pre_edge(dat.e, dat.mu http://dat.mu , group=dat, e0=dat.e0, pre1=-150, pre2=-60, norm1=150, norm2=1100, nnorm=2, _larch=mylarch) autobk(dat.e, dat.mu http://dat.mu , group=dat, e0=dat.e0, rbkg=1, nclamp=5, clamp_lo=0, clamp_hi=24, kstep=0.05, kmin=0, kmax=17, kweight=2, win='hanning', _larch=mylarch) xftf(dat.k, dat.chi, group=dat, kmin=3, kmax=15, dk=0.5, window='hanning', kweight=3, _larch=mylarch) ######################### _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov mailto:Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov mailto:Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit

Dear Matt Thank you for your reply. I confirmed that chi (k) becomes smooth by the Savitzky-Goley method by doing the attached lar file. I will try on various conditions from now. I may also ask if there is something I do not understand. Thank you for your help again! Bests, Yuji Mahara From: Ifeffit [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of Matt Newville Sent: Thursday, April 20, 2017 5:26 AM To: XAFS Analysis using Ifeffit Subject: Re: [Ifeffit] The difference of EXAFS oscillation when using Athena and when using Larch Hi Yuji, The attached larch script (a modification of your script but note: not python, so you may have to add a bunch of '_larch=mylarch' arguments) to show the effects of smoothing with Savitzky-Golay and of interpolating onto a "classic XAFS energy grid". You might want to play around with this different options. It definitely seems that none of these methods are systematically over-smoothing the data and reducing the amplitudes of the oscillations -- one more reason to use Larch instead of Ifeffit. But it also seems like smoothing this data with a Savitzky-Golay fitler is helpful. Also attached are plots showing k- and R-space generated from the attached script. --Matt