We recently collected 3000 EXAFS spectra to follow a chemical reaction process, we would like to fit the experimental data with reference spectra using linear combination fitting. Does anyone know there are available or commercial program to normalize the data and make linear combination fitting. We tried to do it with Athena manually, but 3000 spectra take a long time for us and it is not a practical way. Thanks a lot. Qingyu Kong Beamline ODE Société civile Synchrotron SOLEIL L'Orme des Merisiers Saint-Aubin - BP 48 91192 GIF-sur-YVETTE CEDEX Tel: +33 (0)1 69 35 97 84 (office) +33 (0)1 69 35 97 16 (beamline) email: kong@synchrotron-soleil.fr
Hi Qingyu On Fri, 30 Mar 2012, KONG Quingyu wrote:
We recently collected 3000 EXAFS spectra to follow a chemical reaction process, we would like to fit the experimental data with reference spectra using linear combination fitting. Does anyone know there are available or commercial program to normalize the data and make linear combination fitting. We tried to do it with Athena manually, but 3000 spectra take a long time for us and it is not a practical way.
You might wish to look into Bruce Ravel's new suite of programs (called Demeter). I believe they are built to allow easy scripting of massive quantities of data such as yours. Carlo -- Carlo U. Segre -- Duchossois Leadership Professor of Physics Associate Dean for Graduate Admissions, Graduate College Illinois Institute of Technology Voice: 312.567.3498 Fax: 312.567.3494 segre@iit.edu http://phys.iit.edu/~segre segre@debian.org
Hi Qingyu,
On Fri, Mar 30, 2012 at 1:46 AM, KONG Quingyu
We recently collected 3000 EXAFS spectra to follow a chemical reaction process, we would like to fit the experimental data with reference spectra using linear combination fitting. Does anyone know there are available or commercial program to normalize the data and make linear combination fitting. We tried to do it with Athena manually, but 3000 spectra take a long time for us and it is not a practical way.
Ifeffit is designed to be scriptable for batch processing. This can be done with a raw Ifeffit script (which isn't very flexible) or from Perl or Python, which are more flexible. Carlo mentioned Demeter, which is a high-level Perl wrapping of the lower-level functionality. Working with 3000 spectra may be challenging with Ifeffit version 1.2, as the total amount of available memory is limited, but with some care it should be possible. FWIW, the next version of Ifeffit will eliminate this memory limitation, and make both scripting and extending the functionality much easier and better. This is still a work in progress (see https://github.com/xraypy/xraylarch) and poorly documented, but the essential idea is to write all of Ifeffit's functionality in Python, for all the reasons just stated, and this far enough along that I am confident it will be a big improvement. So far (for XAFS), this can read data, do simple pre-edge subtraction, normalization, remove backgrounds ala autobk, and do XAFS Fourier transforms. It can do simple "general purpose fitting", but does not yet expose a simple-to-use linear combination fitting functionality (shouldn't be too hard, as all the linear algebra pieces are readily available) or fitting spectra to Feff calculations (a bit more involved). Work is slow (mostly due to other obligations), but progress is being made, and it's close to ready for an initial release. If you are (or anyone else is) interested in trying this or helping out, let me know. --Matt
On Friday, March 30, 2012 08:56:05 am Matt Newville wrote:
Working with 3000 spectra may be challenging with Ifeffit version 1.2, as the total amount of available memory is limited, but with some care it should be possible.
The trick to using Ifeffit 1.2 with a large data volume is to be very careful to reuse the same Ifeffit group name so that array space gets overwritten when importing the next data file. If you keep generating new group names, you will run out of available memory long before getting to the 3000th scan. This limits the amount of data you can look at at one time. It is almost certainly impossible to make a plot of all 3000 spectra at once with Ifeffit, but it is quite possible to process the data sequentially with Ifeffit and then write the data to a format that some other, more capable plotting program can handle. B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter
On Friday, March 30, 2012 02:46:38 am KONG Quingyu wrote:
We recently collected 3000 EXAFS spectra to follow a chemical reaction process, we would like to fit the experimental data with reference spectra using linear combination fitting. Does anyone know there are available or commercial program to normalize the data and make linear combination fitting. We tried to do it with Athena manually, but 3000 spectra take a long time for us and it is not a practical way.
Hi Qingyu, Carlo is correct, Demeter could be used to write a special purpose program for automating this sort of thing. The performance in terms of speed may not be exactly what you want, but the performance in terms of quickly prototyping a program to process and analyze a large data volume is excellent. The Demeter code base comes with extensive (although still incomplete) documentation as well as plenty of example scripts, including examples for the individual chores you mention above. Here's Demeter's documentation: http://bruceravel.github.com/demeter/ As a closing comment, isn't it somehow strange that you and your colleagues went to all the trouble of building a fancy dispersive beamline without thinking ahead about how you were going to analyze your data? B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter
participants (4)
-
Bruce Ravel
-
Carlo Segre
-
KONG Quingyu
-
Matt Newville