Send Ifeffit mailing list submissions to
        ifeffit@millenia.cars.aps.anl.gov

To subscribe or unsubscribe via the World Wide Web, visit
        https://protect-au.mimecast.com/s/yBQUCL7rxDs7zY2OUBA_Nt?domain=millenia.cars.aps.anl.gov
or, via email, send a message with subject or body 'help' to
        ifeffit-request@millenia.cars.aps.anl.gov

You can reach the person managing the list at
        ifeffit-owner@millenia.cars.aps.anl.gov

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Ifeffit digest..."


Today's Topics:

   1. Re: data after rebinning look worse in k-space than before?
      (Matt Newville)
   2. Re: data after rebinning look worse in k-space than before?
      (Carlo Segre)


----------------------------------------------------------------------

Message: 1
Date: Wed, 27 Jun 2018 13:31:44 -0500
From: Matt Newville <newville@cars.uchicago.edu>
To: Carlo Segre <segre@iit.edu>, XAFS Analysis using Ifeffit
        <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] data after rebinning look worse in k-space than
        before?
Message-ID:
        <CA+7ESbq--_=qjXALgfZ4Bhg4xcwGJDbby5sSshEwMVB=v_bptA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

HI Ilya, Edmund, Carlo,

Ilya and/or Carlo: can you post some example unbinned data?  As it turns
out, I am adding a rebinning feature in the Larch XAS Viewer GUI that
should be ready for a ready-to-try release very soon (for IIT XAFS School
and XAFS2018).

This seems like a good chance to test these procedures out.

My approach for this is to this is to make a "normal XAFS energy grid" of
~5 eV steps, 0.25 eV steps, 0.05 Ang^-1 steps that the downstream
processing needs, and then do one of two strategies -- maybe there should
be more?:
 a) do a straight interpolation onto this array -- that is probably the
"noisy" result.
 b) assign each energy point in the original data to one of these energy
bins, and take the average of all the points in each bin.

I'd also like to try using energy-weighted mean (centroid).  Probably most
of the data is so finely spaced that this won't make much difference, but
it might be a good option.   It might be able to help compensate for energy
jitter, assuming that the recorded energy (probably from an encoder) is
more accurate than the requested energy.

It's also interesting to think about doing a Savitzky-Golay smoothing,
though that might require knowing if the data points are actually uniform
in mono angle or mono energy.  It also makes it easy to over-do the
smoothing, and so a little trickier to prevent bad results.

Do you (or anyone else) have any suggestions for how to best re-bin this
kind of data?

--Matt


On Wed, Jun 27, 2018 at 10:15 AM Carlo Segre <segre@iit.edu> wrote:

>
> Yes, we measure fast and have taken as many as 20000 points.   The problem
> is not in the shifts that you mention.  This is normal and expected.  the
> problem is specificallly in the rebinning algorithm in Demeter.  It seems
> to be different than the one in the old Horae package.  I have done a test
> of this and I attache a coule of figures that show the difference.
>
> I have used 10 continuous scans for this test.  The data were taken at the
> MRCAT beamline, Sector 10 at the APS.  The data are for the Fe K-edge and
> there are about 3400 points per scan with a point density of about 0.35
> eV/step.  I used both versions of Athena and performed the following steps
> to give the data groups shown in the plots
>
> new_athena.png
>   Fe_new_rebin_merge - (blue) all 10 scans rebinned at input and then
>                        merged
>   Fe_new_merge       - (red) all 10 scans merged only
>   Fe_new_merge_rebin - (green) all 10 scans merged then rebinned
>
> old_athena.png
>   Fe_old_rebin_merge - (blue) all 10 scans rebinned at input and then
>                        merged
>   Fe_old_merge       - (red) all 10 scans merged only
>   Fe_old_merge_rebin - (green) all 10 scans merged then rebinned
>
> comp_athena.png
>   Fe_old_rebin_merge - (blue)
>   Fe_new_rebin_merge - (red)
>
> It is clear that the new Athena (Demeter) is not rebinning the same way as
> the old one (Horae).  The contrast is particularly evident with the last
> plot. The new rebinning algorithm is introducing more noise.  For the
> moment, I recommend only merging and perhaps smoothing if you can tolerate
> a bit of amplitude reduction.
>
> I have been thinking that it might even be better to have the data
> acquisition software do the rebinning on the fly so the data does not have
> to be manipulated in Athena.  I am not sure if this is a good idea yet but
> I think it would help my users.
>
> Carlo
>
>
> On Wed, 27 Jun 2018, Edmund Welter wrote:
>
> > Dear Carlo,
> >
> > do you also measure as fast as possible in the sense that for two
> consecutive
> > scans the points on the energy axis are not at the same positions? This
> is
> > what happens at my beamline. The differences are typically very small
> but
> > there are differences and one should not just add all the first points
> and
> > all the second points and so on because they are not necessarily exactly
> at
> > the same energy. Sometimes the beamline computer is doing something else
> in
> > parallel (whatever that might be) and the distance between points A and
> B is
> > significantly larger than the distance between B and C.
> >
> > So, the problem is, at which point does it make sense to merge several
> > spectra of the same sample? I presume that Athena is taking care of this
> when
> > I use it to merge spectra, but it can only do so by interpolating the
> points
> > in the spectrum onto a common grid before summing up the spectra.
> >
> > The best solution might be to rebin/interpolate the spectra onto a fixed
> grid
> > before they are imported into Athena (or any other program), depends on
> what
> > Athena is exactly doing when it is rebinning data.
> >
> > Another aspect is that Athena is not very happy about 8600
> points/spectrum
> > anyway, at least as long as it using Ifeffit.
> >
> > Cheers,
> >
> > Edmund
> >
> >
> >
> > On 27.06.2018 15:14, Carlo Segre wrote:
> >>
> >>
> >> Hi Ilya:
> >>
> >> We always take data in this mode at APS Sector 10 and I have also find
> that
> >> the rebinning function is not working satisfactorily at this time.  I
> find
> >> that for the current version of the software it is better to merge your
> >> data and let IFEFFIT interpolate to the dk=0.05 grid that it uses.
> >>
> >> Carlo
> >>
> >>
> >> On Wed, 27 Jun 2018, Ilya Sinev wrote:
> >>
> >>> Hi all,
> >>>
> >>>
> >>>
> >>> I have a question regarding the chi(k) function isolation and rebinning
> >>> processes. I have some data recorded in ?quasi channel-cut? modus,
> i.e.
> >>> with
> >>> the mono constantly moving and the data points collected with the
> highest
> >>> possible rate. With 180 sec measurement in yields to a spectrum of ca.
> >>> 8600
> >>> point, which obviously needs to be rebinned. The rebinned data,
> however,
> >>> does not look good in k-space even if multiple data are merged.
> Moreover,
> >>> I
> >>> have an impression that the raw spectrum in k-space does not have those
> >>> 8000+ points anymore but significantly less. Is there any reduction of
> the
> >>> data points number that is not seen (e.g. as a preparation step for
> FT)?
> >>> Since the unbinned data has higher quality, does it then make more
> sense
> >>> to
> >>> keep using it for EXAFS analysis?
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> Thank you
> >>>
> >>> Ilya Sinev
> >>>
> >>>
> >>>
> >>>
> >>
> >
> > _______________________________________________
> > Ifeffit mailing list
> > Ifeffit@millenia.cars.aps.anl.gov
> > https://protect-au.mimecast.com/s/yBQUCL7rxDs7zY2OUBA_Nt?domain=millenia.cars.aps.anl.gov
> > Unsubscribe: https://protect-au.mimecast.com/s/vZRDCMwvygs7n6Y8Uk49kH?domain=millenia.cars.aps.anl.gov
> >
>
> --
> Carlo U. Segre -- Duchossois Leadership Professor of Physics
> Interim Chair, Department of Chemistry
> Director, Center for Synchrotron Radiation Research and Instrumentation
> Illinois Institute of Technology
> Voice: 312.567.3498            Fax: 312.567.3494
> segre@iit.edu   https://protect-au.mimecast.com/s/7CwwCNLwzjFzgMpxij0Sb6?domain=phys.iit.edu   segre@debian.org
> _______________________________________________
> Ifeffit mailing list
> Ifeffit@millenia.cars.aps.anl.gov
> https://protect-au.mimecast.com/s/yBQUCL7rxDs7zY2OUBA_Nt?domain=millenia.cars.aps.anl.gov
> Unsubscribe: https://protect-au.mimecast.com/s/vZRDCMwvygs7n6Y8Uk49kH?domain=millenia.cars.aps.anl.gov
>


--
--Matt Newville <newville at https://protect-au.mimecast.com/s/lV6fCOMxAksMB2kXirFAr8?domain=cars.uchicago.edu> 630-252-0431
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20180627/10804256/attachment-0001.html>

------------------------------

Message: 2
Date: Wed, 27 Jun 2018 13:34:43 -0500 (CDT)
From: Carlo Segre <segre@iit.edu>
To: Matt Newville <newville@cars.uchicago.edu>
Cc: XAFS Analysis using Ifeffit <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] data after rebinning look worse in k-space than
        before?
Message-ID: <alpine.DEB.2.21.1806271334150.30761@hydride.segre.home>
Content-Type: text/plain; charset=US-ASCII; format=flowed


Hi Matt:

I'll send the data I used for the figures directly to you.

Carlo

On Wed, 27 Jun 2018, Matt Newville wrote:

> HI Ilya, Edmund, Carlo,
>
> Ilya and/or Carlo: can you post some example unbinned data?  As it turns
> out, I am adding a rebinning feature in the Larch XAS Viewer GUI that
> should be ready for a ready-to-try release very soon (for IIT XAFS School
> and XAFS2018).
>
> This seems like a good chance to test these procedures out.
>
> My approach for this is to this is to make a "normal XAFS energy grid" of
> ~5 eV steps, 0.25 eV steps, 0.05 Ang^-1 steps that the downstream
> processing needs, and then do one of two strategies -- maybe there should
> be more?:
> a) do a straight interpolation onto this array -- that is probably the
> "noisy" result.
> b) assign each energy point in the original data to one of these energy
> bins, and take the average of all the points in each bin.
>
> I'd also like to try using energy-weighted mean (centroid).  Probably most
> of the data is so finely spaced that this won't make much difference, but
> it might be a good option.   It might be able to help compensate for energy
> jitter, assuming that the recorded energy (probably from an encoder) is
> more accurate than the requested energy.
>
> It's also interesting to think about doing a Savitzky-Golay smoothing,
> though that might require knowing if the data points are actually uniform
> in mono angle or mono energy.  It also makes it easy to over-do the
> smoothing, and so a little trickier to prevent bad results.
>
> Do you (or anyone else) have any suggestions for how to best re-bin this
> kind of data?
>
> --Matt
>
>
> On Wed, Jun 27, 2018 at 10:15 AM Carlo Segre <segre@iit.edu> wrote:
>
>>
>> Yes, we measure fast and have taken as many as 20000 points.   The problem
>> is not in the shifts that you mention.  This is normal and expected.  the
>> problem is specificallly in the rebinning algorithm in Demeter.  It seems
>> to be different than the one in the old Horae package.  I have done a test
>> of this and I attache a coule of figures that show the difference.
>>
>> I have used 10 continuous scans for this test.  The data were taken at the
>> MRCAT beamline, Sector 10 at the APS.  The data are for the Fe K-edge and
>> there are about 3400 points per scan with a point density of about 0.35
>> eV/step.  I used both versions of Athena and performed the following steps
>> to give the data groups shown in the plots
>>
>> new_athena.png
>>   Fe_new_rebin_merge - (blue) all 10 scans rebinned at input and then
>>                        merged
>>   Fe_new_merge       - (red) all 10 scans merged only
>>   Fe_new_merge_rebin - (green) all 10 scans merged then rebinned
>>
>> old_athena.png
>>   Fe_old_rebin_merge - (blue) all 10 scans rebinned at input and then
>>                        merged
>>   Fe_old_merge       - (red) all 10 scans merged only
>>   Fe_old_merge_rebin - (green) all 10 scans merged then rebinned
>>
>> comp_athena.png
>>   Fe_old_rebin_merge - (blue)
>>   Fe_new_rebin_merge - (red)
>>
>> It is clear that the new Athena (Demeter) is not rebinning the same way as
>> the old one (Horae).  The contrast is particularly evident with the last
>> plot. The new rebinning algorithm is introducing more noise.  For the
>> moment, I recommend only merging and perhaps smoothing if you can tolerate
>> a bit of amplitude reduction.
>>
>> I have been thinking that it might even be better to have the data
>> acquisition software do the rebinning on the fly so the data does not have
>> to be manipulated in Athena.  I am not sure if this is a good idea yet but
>> I think it would help my users.
>>
>> Carlo
>>
>>
>> On Wed, 27 Jun 2018, Edmund Welter wrote:
>>
>>> Dear Carlo,
>>>
>>> do you also measure as fast as possible in the sense that for two
>> consecutive
>>> scans the points on the energy axis are not at the same positions? This
>> is
>>> what happens at my beamline. The differences are typically very small
>> but
>>> there are differences and one should not just add all the first points
>> and
>>> all the second points and so on because they are not necessarily exactly
>> at
>>> the same energy. Sometimes the beamline computer is doing something else
>> in
>>> parallel (whatever that might be) and the distance between points A and
>> B is
>>> significantly larger than the distance between B and C.
>>>
>>> So, the problem is, at which point does it make sense to merge several
>>> spectra of the same sample? I presume that Athena is taking care of this
>> when
>>> I use it to merge spectra, but it can only do so by interpolating the
>> points
>>> in the spectrum onto a common grid before summing up the spectra.
>>>
>>> The best solution might be to rebin/interpolate the spectra onto a fixed
>> grid
>>> before they are imported into Athena (or any other program), depends on
>> what
>>> Athena is exactly doing when it is rebinning data.
>>>
>>> Another aspect is that Athena is not very happy about 8600
>> points/spectrum
>>> anyway, at least as long as it using Ifeffit.
>>>
>>> Cheers,
>>>
>>> Edmund
>>>
>>>
>>>
>>> On 27.06.2018 15:14, Carlo Segre wrote:
>>>>
>>>>
>>>> Hi Ilya:
>>>>
>>>> We always take data in this mode at APS Sector 10 and I have also find
>> that
>>>> the rebinning function is not working satisfactorily at this time.  I
>> find
>>>> that for the current version of the software it is better to merge your
>>>> data and let IFEFFIT interpolate to the dk=0.05 grid that it uses.
>>>>
>>>> Carlo
>>>>
>>>>
>>>> On Wed, 27 Jun 2018, Ilya Sinev wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>>
>>>>>
>>>>> I have a question regarding the chi(k) function isolation and rebinning
>>>>> processes. I have some data recorded in ?quasi channel-cut? modus,
>> i.e.
>>>>> with
>>>>> the mono constantly moving and the data points collected with the
>> highest
>>>>> possible rate. With 180 sec measurement in yields to a spectrum of ca.
>>>>> 8600
>>>>> point, which obviously needs to be rebinned. The rebinned data,
>> however,
>>>>> does not look good in k-space even if multiple data are merged.
>> Moreover,
>>>>> I
>>>>> have an impression that the raw spectrum in k-space does not have those
>>>>> 8000+ points anymore but significantly less. Is there any reduction of
>> the
>>>>> data points number that is not seen (e.g. as a preparation step for
>> FT)?
>>>>> Since the unbinned data has higher quality, does it then make more
>> sense
>>>>> to
>>>>> keep using it for EXAFS analysis?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Thank you
>>>>>
>>>>> Ilya Sinev
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> Ifeffit mailing list
>>> Ifeffit@millenia.cars.aps.anl.gov
>>> https://protect-au.mimecast.com/s/yBQUCL7rxDs7zY2OUBA_Nt?domain=millenia.cars.aps.anl.gov
>>> Unsubscribe: https://protect-au.mimecast.com/s/vZRDCMwvygs7n6Y8Uk49kH?domain=millenia.cars.aps.anl.gov
>>>
>>
>> --
>> Carlo U. Segre -- Duchossois Leadership Professor of Physics
>> Interim Chair, Department of Chemistry
>> Director, Center for Synchrotron Radiation Research and Instrumentation
>> Illinois Institute of Technology
>> Voice: 312.567.3498            Fax: 312.567.3494
>> segre@iit.edu   https://protect-au.mimecast.com/s/7CwwCNLwzjFzgMpxij0Sb6?domain=phys.iit.edu   segre@debian.org
>> _______________________________________________
>> Ifeffit mailing list
>> Ifeffit@millenia.cars.aps.anl.gov
>> https://protect-au.mimecast.com/s/yBQUCL7rxDs7zY2OUBA_Nt?domain=millenia.cars.aps.anl.gov
>> Unsubscribe: https://protect-au.mimecast.com/s/vZRDCMwvygs7n6Y8Uk49kH?domain=millenia.cars.aps.anl.gov
>>
>
>
>

--
Carlo U. Segre -- Duchossois Leadership Professor of Physics
Interim Chair, Department of Chemistry
Director, Center for Synchrotron Radiation Research and Instrumentation
Illinois Institute of Technology
Voice: 312.567.3498            Fax: 312.567.3494
segre@iit.edu   https://protect-au.mimecast.com/s/7CwwCNLwzjFzgMpxij0Sb6?domain=phys.iit.edu   segre@debian.org


------------------------------

Subject: Digest Footer

_______________________________________________
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: https://protect-au.mimecast.com/s/vZRDCMwvygs7n6Y8Uk49kH?domain=millenia.cars.aps.anl.gov


------------------------------

End of Ifeffit Digest, Vol 184, Issue 21
****************************************