Dear all
I agree with Mike and Scott too.
I might suggest Schalken, M. J. and Chantler, C. T. “Preservation of uncertainty in experiment: structures of Ni (II) coordination complexes” (2018) J. Synchrotron Rad., 25, 920-934 as a partial discussion around this topic if people are interested.

It could be read as 'don't smooth data if you can avoid it'.

Very best wishes to all
Chris

--------------------------------------------------------------------------------------
Christopher Chantler, Professor, FAIP, Fellow American Physical Society
Editor-in-Chief, Radiation Physics and Chemistry
Chair, International IUCr Commission on XAFS; CIT, CCN
IPP, International Radiation Physics Society
School of Physics, University of Melbourne
Parkville Victoria 3010 Australia
+61-3-83445437 FAX +61-3-93474783

From: Ifeffit <ifeffit-bounces@millenia.cars.aps.anl.gov> on behalf of ifeffit-request@millenia.cars.aps.anl.gov <ifeffit-request@millenia.cars.aps.anl.gov>
Sent: Thursday, 23 January 2020 4:05 PM
To: ifeffit@millenia.cars.aps.anl.gov <ifeffit@millenia.cars.aps.anl.gov>
Subject: Ifeffit Digest, Vol 203, Issue 5
 
Send Ifeffit mailing list submissions to
        ifeffit@millenia.cars.aps.anl.gov

To subscribe or unsubscribe via the World Wide Web, visit
        https://protect-au.mimecast.com/s/QAqdCBNZkphLj60guzb-fv?domain=millenia.cars.aps.anl.gov
or, via email, send a message with subject or body 'help' to
        ifeffit-request@millenia.cars.aps.anl.gov

You can reach the person managing the list at
        ifeffit-owner@millenia.cars.aps.anl.gov

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Ifeffit digest..."


Today's Topics:

   1. smoothing XAS data (Daria Boglaienko)
   2. Re: smoothing XAS data (Carlo Segre)
   3. Re: smoothing XAS data (Scott Calvin)
   4. Re: smoothing XAS data (Mike Massey)
   5. Re: smoothing XAS data (Will Bennett)


----------------------------------------------------------------------

Message: 1
Date: Wed, 22 Jan 2020 17:38:43 -0800
From: Daria Boglaienko <dbogl003@fiu.edu>
To: ifeffit@millenia.cars.aps.anl.gov
Subject: [Ifeffit] smoothing XAS data
Message-ID:
        <CAACWq8BmJFr0QNUHk1YrsB7w3BLGhzyus9pXow0Ng+ER3eAZDQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello,

In what case smoothing of the data is OK? My data have a lot of noise and
smoothing really helps (visually). I compared fit done in Artemis for
smoothed and non-smoothed data sets and the result is very similar, however
when I searched about it online, it looks like it is not recommended.
What is a good way to justify it?

Thank you!

-Daria
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20200122/f5449543/attachment-0001.html>

------------------------------

Message: 2
Date: Wed, 22 Jan 2020 19:44:22 -0600 (CST)
From: Carlo Segre <segre@iit.edu>
To: XAFS Analysis using Ifeffit <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] smoothing XAS data
Message-ID:
        <alpine.DEB.2.22.394.2001221942280.10363@hydride.segre.home>
Content-Type: text/plain; charset=US-ASCII; format=flowed


Hi Daria:

I smooth (or boxcar average) the data which it is oversampled, such as in
a continuous scan where the data in the exafs region is spaced closer than
the usual delta k of 0.05.

I generally don't like to smooth XANES data since smoothing does tend to
attenuate sharp features such as the ones which exist near the absorption
edge.

Carlo

On Wed, 22 Jan 2020, Daria Boglaienko wrote:

> Hello,
>
> In what case smoothing of the data is OK? My data have a lot of noise and
> smoothing really helps (visually). I compared fit done in Artemis for
> smoothed and non-smoothed data sets and the result is very similar, however
> when I searched about it online, it looks like it is not recommended.
> What is a good way to justify it?
>
> Thank you!
>
> -Daria
>

--
Carlo U. Segre -- Duchossois Leadership Professor of Physics
Directory, Center for Synchrotron Radiation Research and Instrumentation
Illinois Institute of Technology
Voice: 312.567.3498            Fax: 312.567.3494
segre@iit.edu   https://protect-au.mimecast.com/s/Yw6RCE8knvsZL9OQupTBj2?domain=phys.iit.edu   segre@debian.org


------------------------------

Message: 3
Date: Wed, 22 Jan 2020 23:01:23 -0500
From: Scott Calvin <dr.scott.calvin@gmail.com>
To: Carlo Segre <segre@iit.edu>, XAFS Analysis using Ifeffit
        <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] smoothing XAS data
Message-ID: <2CFE75B8-59D9-472F-95E0-2C618FD3BFC4@gmail.edu>
Content-Type: text/plain; charset="utf-8"

Not only can smoothing XANES data attenuate sharp features, it can also shift them, since many XANES features, particularly in the ?pre-edge? region, are significantly asymmetric about their peak. That can make analysis quite confusing, since, e.g., the size of the boxcar ends up affecting the position of the peak!

Best,
Scott Calvin
Lehman College of the City University of New York

> On Jan 22, 2020, at 8:44 PM, Carlo Segre <segre@iit.edu> wrote:
>
> I generally don't like to smooth XANES data since smoothing does tend to attenuate sharp features such as the ones which exist near the absorption edge.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20200122/4e427a6e/attachment-0001.html>

------------------------------

Message: 4
Date: Wed, 22 Jan 2020 20:50:51 -0800
From: Mike Massey <mmassey@gmail.com>
To: Carlo Segre <segre@iit.edu>, XAFS Analysis using Ifeffit
        <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] smoothing XAS data
Message-ID: <E369E515-D6B6-4E74-8AE5-7D91A3177EA0@gmail.com>
Content-Type: text/plain; charset=utf-8

At this juncture, I'd like to bring up a "rule" I've made for myself regarding data quality and smoothing: I'd rather walk away from a beam run with one good spectrum than a hundred poor ones.

Meaning, for me anyway, if the data aren't smooth enough on their own, I'd rather spend more precious time counting than move on. I'd count for three days on a sample if I had to (I've never had to).

While it has been proven that, contrary to the popular saying, one _can_ polish a turd, I'd personally rather not try.

I realize this approach was more difficult when I was a grad student and everyone in the research group was clamoring for data, but even in that situation, good data > not-good data.

Apologies for the tangent, but I hope someone out there might find it useful.


Cheers,



Mike





> On Jan 22, 2020, at 5:44 PM, Carlo Segre <segre@iit.edu> wrote:
>
> ?
> Hi Daria:
>
> I smooth (or boxcar average) the data which it is oversampled, such as in a continuous scan where the data in the exafs region is spaced closer than the usual delta k of 0.05.
>
> I generally don't like to smooth XANES data since smoothing does tend to attenuate sharp features such as the ones which exist near the absorption edge.
>
> Carlo
>
>> On Wed, 22 Jan 2020, Daria Boglaienko wrote:
>>
>> Hello,
>>
>> In what case smoothing of the data is OK? My data have a lot of noise and
>> smoothing really helps (visually). I compared fit done in Artemis for
>> smoothed and non-smoothed data sets and the result is very similar, however
>> when I searched about it online, it looks like it is not recommended.
>> What is a good way to justify it?
>>
>> Thank you!
>>
>> -Daria
>>
>
> --
> Carlo U. Segre -- Duchossois Leadership Professor of Physics
> Directory, Center for Synchrotron Radiation Research and Instrumentation
> Illinois Institute of Technology
> Voice: 312.567.3498            Fax: 312.567.3494
> segre@iit.edu   https://protect-au.mimecast.com/s/Yw6RCE8knvsZL9OQupTBj2?domain=phys.iit.edu   segre@debian.org
> _______________________________________________
> Ifeffit mailing list
> Ifeffit@millenia.cars.aps.anl.gov
> https://protect-au.mimecast.com/s/QAqdCBNZkphLj60guzb-fv?domain=millenia.cars.aps.anl.gov
> Unsubscribe: https://protect-au.mimecast.com/s/OeSpCJypvAfjZrl7uveus8?domain=millenia.cars.aps.anl.gov



------------------------------

Message: 5
Date: Thu, 23 Jan 2020 05:05:37 +0000
From: Will Bennett <w.bennett@griffith.edu.au>
To: Mike Massey <mmassey@gmail.com>, Carlo Segre <segre@iit.edu>, XAFS
        Analysis using Ifeffit <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] smoothing XAS data
Message-ID:
        <SY4P282MB1051AEAF3F19EE9A41F026A4A80F0@SY4P282MB1051.AUSP282.PROD.OUTLOOK.COM>
       
Content-Type: text/plain; charset="utf-8"

I entirely agree with Mike and Scott.

I never smooth data because it is a slippery slope to start on! How much should you smooth? Should you only smooth data if it looks "bad"? What does "bad data" look like anyway?

If the only point of smoothing is to improve the visual look of data, then you are doing it for the wrong reason - it is better to present the true data and let the reader decide if your interpretation is robust. In my view, smoothing data is somewhat scientifically dishonest unless you have a strong justification that extends beyond trying to make the data look "better"...

As someone who works with environmental samples at low concentrations, I'm very used to seeing noisy data. You just have to get used to long count times and lots of replicate scans!

Cheers,

Will


Dr William W. Bennett

Senior Lecturer
Environmental Futures Research Institute
School of Environment and Science
Griffith University
Gold Coast, Queensland, Australia
M +61 401 186 488
w.bennett@griffith.edu.au

Assistant Professor
Nordcee, Department of Biology
University of Southern Denmark
Odense, Denmark
M +45 8193 8111
wbennett@biology.sdu.dk

________________________________
From: Ifeffit <ifeffit-bounces@millenia.cars.aps.anl.gov> on behalf of Mike Massey <mmassey@gmail.com>
Sent: Thursday, 23 January 2020 2:50 PM
To: Carlo Segre <segre@iit.edu>; XAFS Analysis using Ifeffit <ifeffit@millenia.cars.aps.anl.gov>
Subject: Re: [Ifeffit] smoothing XAS data

At this juncture, I'd like to bring up a "rule" I've made for myself regarding data quality and smoothing: I'd rather walk away from a beam run with one good spectrum than a hundred poor ones.

Meaning, for me anyway, if the data aren't smooth enough on their own, I'd rather spend more precious time counting than move on. I'd count for three days on a sample if I had to (I've never had to).

While it has been proven that, contrary to the popular saying, one _can_ polish a turd, I'd personally rather not try.

I realize this approach was more difficult when I was a grad student and everyone in the research group was clamoring for data, but even in that situation, good data > not-good data.

Apologies for the tangent, but I hope someone out there might find it useful.


Cheers,



Mike





> On Jan 22, 2020, at 5:44 PM, Carlo Segre <segre@iit.edu> wrote:
>
> ?
> Hi Daria:
>
> I smooth (or boxcar average) the data which it is oversampled, such as in a continuous scan where the data in the exafs region is spaced closer than the usual delta k of 0.05.
>
> I generally don't like to smooth XANES data since smoothing does tend to attenuate sharp features such as the ones which exist near the absorption edge.
>
> Carlo
>
>> On Wed, 22 Jan 2020, Daria Boglaienko wrote:
>>
>> Hello,
>>
>> In what case smoothing of the data is OK? My data have a lot of noise and
>> smoothing really helps (visually). I compared fit done in Artemis for
>> smoothed and non-smoothed data sets and the result is very similar, however
>> when I searched about it online, it looks like it is not recommended.
>> What is a good way to justify it?
>>
>> Thank you!
>>
>> -Daria
>>
>
> --
> Carlo U. Segre -- Duchossois Leadership Professor of Physics
> Directory, Center for Synchrotron Radiation Research and Instrumentation
> Illinois Institute of Technology
> Voice: 312.567.3498 Fax: 312.567.3494
> segre@iit.edu https://protect-au.mimecast.com/s/Yw6RCE8knvsZL9OQupTBj2?domain=phys.iit.edu<https://protect-au.mimecast.com/s/Yw6RCE8knvsZL9OQupTBj2?domain=phys.iit.edu> segre@debian.org
> _______________________________________________
> Ifeffit mailing list
> Ifeffit@millenia.cars.aps.anl.gov
> https://protect-au.mimecast.com/s/QAqdCBNZkphLj60guzb-fv?domain=millenia.cars.aps.anl.gov<https://protect-au.mimecast.com/s/QAqdCBNZkphLj60guzb-fv?domain=millenia.cars.aps.anl.gov>
> Unsubscribe: https://protect-au.mimecast.com/s/OeSpCJypvAfjZrl7uveus8?domain=millenia.cars.aps.anl.gov<https://protect-au.mimecast.com/s/OeSpCJypvAfjZrl7uveus8?domain=millenia.cars.aps.anl.gov>

_______________________________________________
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
https://protect-au.mimecast.com/s/QAqdCBNZkphLj60guzb-fv?domain=millenia.cars.aps.anl.gov<https://protect-au.mimecast.com/s/QAqdCBNZkphLj60guzb-fv?domain=millenia.cars.aps.anl.gov>
Unsubscribe: https://protect-au.mimecast.com/s/OeSpCJypvAfjZrl7uveus8?domain=millenia.cars.aps.anl.gov<https://protect-au.mimecast.com/s/OeSpCJypvAfjZrl7uveus8?domain=millenia.cars.aps.anl.gov>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20200123/ffb88fc8/attachment.html>

------------------------------

Subject: Digest Footer

_______________________________________________
Ifeffit mailing list
Ifeffit@millenia.cars.aps.anl.gov
http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Unsubscribe: https://protect-au.mimecast.com/s/OeSpCJypvAfjZrl7uveus8?domain=millenia.cars.aps.anl.gov


------------------------------

End of Ifeffit Digest, Vol 203, Issue 5
***************************************