- Ifeffit - millenia.cars.aps.anl.gov

Re: [Ifeffit] Large XAFS data sets
by Jan Stötzel 16 Apr '12

16 Apr '12

Hi Matt, I solved it this way: In my code only the 1-5 reference component spectra for the LCF are permanently imported in IFEFFIT and one variable is reserved for all the spectra that I want to have fitted, which is done consecutively then. Each time a spectrum is fitted, the results are exported into arrays of my software and send to the plot pane. After that everything except the reference component spectra is resetted in IFEFFIT and the fitted spectrum is overwritten by the next one. Going that way IFEFFIT memory is never exceeded, neither when fitting 3000 spectra nor in principal with 3*10^6 spectra (although that would take more than a week I guess). Of course you need some time to code it but it works quite well. Nevertheless, a new IFEFFIT with faster fit routines would be most welcome of course :). Regards, Jan Zitat von Matt Newville <newville(a)cars.uchicago.edu>: > Hi Edmund, Jan, Matthew, > > On Thu, Apr 12, 2012 at 8:51 AM, Jan Stötzel > <j.stoetzel(a)uni-wuppertal.de> wrote: >> Hi Edmund, >> >> in my opinion evaluating 3000 spectra is neither a heroic attempt nor brute >> force. The LCF of 3000 spectra yields one beautiful plot with 3000 * >> components points, where the information of an enormous Gbyte data set is >> included - for me that is indeed very economical. With such a plot each >> reader can decide (or be convinced) which spectra are "boring" and if you >> want to study high order kinetics during your reaction you will be happy to >> have enough points to perform the required fits. Of course, one can also >> write a program that preselects a few spectra by evaluating the differences >> between the spectra. But then I guess you need at least one (avoidable) >> parameter to adjust the tolerance to distinguish between differences caused >> by noise and those due to real variations in sample composition. >> >> That´s why I´d prefer fitting all spectra and afterwards highlighting the >> interesting parts. That might take 2-3 hours for a few thosand spectra >> (extended lunch break) but it is still much more convenient than finding the >> "interesting" spectra manually by going through all spectra (or than finding >> the most suitable tolerance parameter). And the resulting plots are worth >> the time imo. I am rather going to check how much faster I can get by >> parallelizing the processes for multicores... >> >> Best regards, >> >> Jan Stötzel >> >> >> >> Zitat von Edmund Welter <edmund.welter(a)desy.de>: >> >>> Dear XAFS users >>> >>> recently I read several mails on this list which were dealing with the >>> problem of large data sets, as they are produced by Q-EXAFS scans or by >>> dispersive XAFS. The question was if there are tools available to handle >>> data sets of several 1000 spectra and perform a linear combination fit >>> or even a full EXAFS evaluation on each of them. Evaluating 3000 spectra >>> is a heroic attempt, but I wonder if it is also economical. >>> >>> >>> In most (that means not in ALL!) cases, the vast majority of these >>> spectra is boring, because the spectrum with the number X looks exactly >>> like the spectrum with the number X-1 looked and how the spectrum with >>> the number X+1 will look and so on. Evaluating all these (basically >>> identical) spectra is in principle a waste of time and working memory. >>> The interesting spectra are those which were measured when something was >>> happening in the sample. Since we do not always know at which time, or >>> temperature or reactant concentration etc. interesting things will >>> happen it is without any doubt justified to measure x-thousand spectra, >>> but after that we should use a more sophisticated approach than brute >>> force. >>> >>> I think that it would be much more useful to find procedures (that means >>> develop computer programs) that search for the (usually relatively small >>> number of) interesting spectra. The most obvious parameter is how >>> similar is a particular spectrum to the spectra measured before and >>> after. The next step would probably be to identify clusters of related >>> spectra using statistical methods. This is a problem which had to be >>> solved in other areas like the automated analysis of images before and >>> should also be possible with our kind of data. >>> >>> >>> Anyway, how to handle thousands of XAFS spectra will become a very >>> important problem in the future. With all these beamlines that provide >>> 10^12 photons per second we can measure a factor 100 -- 1000 faster than >>> >>> we did with 10^9 photons per second. So, I wonder if anything beyond the >>> brute force approach is going on in the EXAFS software universe to make >>> effective and economical use of the measured data. >>> >>> Best regards, >>> Edmund Welter >>> >>> >>> >>> -- >>> -------------------------------------------------------- >>> Dr. Edmund Welter Deutsches Elektronen-Synchrotron >>> DESY FS-Do >>> >>> Notkestr. 85 Email: edmund.welter(a)desy.de >>> D-22607 Hamburg Phone: +49 40 8998 4510 >>> Germany Fax : +49 40 8998 2787 >>> -------------------------------------------------------- > > > Sorry for the delay, but I just wanted to add my agreement to this > need to be able to work with larger data sets. Handling ~3000 XANES > spectra for linear analysis is definitely something that we need to be > able to do well, and it is not something that Ifeffit can do really at > all. This is a definite weakness. For reference, 3000 EXAFS spectra > can fit in well under 500 Mb of memory, and there really should be no > problem handling this amount of data as a multi-dimensional array in > memory on any modern machine -- in fact, expecting an order of > magnitude larger is not really that absurd. The main problem is > simply that the tools we have are not built with this is mind, and the > hardware + data collection have outpaced analysis software. > > I believe the Ifeffit2 approach will be much better suited for > handling such large datasets, and can easily use any of the linear > analysis tools from scipy (http://scipy.org/) If you or anyone else > has some suggested algorithms to include (PCA, non-negative > least-squares, etc), that would be very helpful. > > --Matt > > _______________________________________________ > Ifeffit mailing list > Ifeffit(a)millenia.cars.aps.anl.gov > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

1 0

difference spectra function in (D)Athena
by Christopher Patridge 12 Apr '12

12 Apr '12

Hello Mailing List, I have a question about the difference spectrum function. I chose difference spectrum in norm mu(E) in the Demeter pull down menu. I set the standard and chose another spectrum. Plotting both the difference and spectra gives me the image attached. Is the difference shifted significantly to lower energy? If so, why does it change its energy scale relative to the plotted spectra? Is there a setting I can change or could this be a small bug? Thank you all, buena salud, Chris Patridge -- Christopher J. Patridge PhD. Contact: (315)-529-0501

3 3

Re: [Ifeffit] Theoretical value of S02 for different absorber
by Rana, Jatinkumar Kantilal 05 Apr '12

05 Apr '12

Dear Dr. Newville, Thank you so much for your reply. Things are very much clear now. With best regards, Jatin ________________________________________ From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [ifeffit-bounces(a)millenia.cars.aps.anl.gov] on behalf of ifeffit-request(a)millenia.cars.aps.anl.gov [ifeffit-request(a)millenia.cars.aps.anl.gov] Sent: Thursday, April 05, 2012 7:00 PM To: ifeffit(a)millenia.cars.aps.anl.gov Subject: Ifeffit Digest, Vol 110, Issue 6 Send Ifeffit mailing list submissions to ifeffit(a)millenia.cars.aps.anl.gov To subscribe or unsubscribe via the World Wide Web, visit http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit or, via email, send a message with subject or body 'help' to ifeffit-request(a)millenia.cars.aps.anl.gov You can reach the person managing the list at ifeffit-owner(a)millenia.cars.aps.anl.gov When replying, please edit your Subject line so it is more specific than "Re: Contents of Ifeffit digest..." Today's Topics: 1. Re: Theoretical value of S02 for different absorber (Rana, Jatinkumar Kantilal) 2. Results Reporting (John Farell) 3. Re: Results Reporting (Bruce Ravel) 4. Re: Theoretical value of S02 for different absorber (Matt Newville) ---------------------------------------------------------------------- Message: 1 Date: Thu, 5 Apr 2012 10:44:05 +0000 From: "Rana, Jatinkumar Kantilal" <jatinkumar.rana(a)helmholtz-berlin.de> To: "ifeffit(a)millenia.cars.aps.anl.gov" <ifeffit(a)millenia.cars.aps.anl.gov> Subject: Re: [Ifeffit] Theoretical value of S02 for different absorber Message-ID: <6FCB59A6A97E13429B863B7014B57E4B10E2E8@didag1> Content-Type: text/plain; charset="utf-8" Dear Dr. Ravel, Thank you very much for your prompt reply. I followed the instruction on (http://leonardo.phys.washington.edu/feff/wiki/index.php?title=EXAFS_calcula…) and activated S02 card by making S02 0.0 in feff.inp file. As a result, FEFF calculated S02. However, I get confused when I compare FEFF calculated S02 with reported value in literature and the one obtained from my own fit. For example, S02 for Mn absorber FEFF calculated 0.94 Fitted value for my sample ~0.75 Fitted value for various Mn-oxides ~0.75 Other reported value from Mn fit ~0.70-0.80 (http://www.sciencedirect.com/science/article/pii/S0020169397057496) S02 for Fe absorber FEFF calculated ~0.94 Fitted value for Fe-foil ~0.70 Fitted value for various Fe-oxides ~0.70 Reported value by calculation 0.69 (http://prola.aps.org/abstract/PR/v169/i1/p27_1) It is well known that S02 is a property of central absorbing atom and is considered to be constant between 0.7 - 1.0. However, in real world, S02 is affected by various systematic errors like thicker samples (in transmission), absorption by other elements in sample at given energy range and many more. These factors can suppress the amplitudes of EXAFS oscillations resulting in lower value of fitted S02 than it should be for a given absorber. The reason I am so much curious about S02 is, if I know what should be (or should be considered) the value of S02 for a given absorber than I can possibly understand the reasons of fitted S02 being too low or too high. For instance, S02 and Sigma2 are highly correlated for first shell. So from fitted value of S02 and Sigma2 of first shell, I could possibly understand changes in local structure. So I would like to put forward very simple but still confusing question that which value of S02 should be considered, FEFF calculated or the one obtained by fitting standards (either reference foils or reference compounds measured on same beamline) and why. Many thanks in advance !!! With best regards, Jatin -----Original Message----- From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of ifeffit-request(a)millenia.cars.aps.anl.gov Sent: Mittwoch, 4. April 2012 19:00 To: ifeffit(a)millenia.cars.aps.anl.gov Subject: Ifeffit Digest, Vol 110, Issue 5 Send Ifeffit mailing list submissions to ifeffit(a)millenia.cars.aps.anl.gov To subscribe or unsubscribe via the World Wide Web, visit http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit or, via email, send a message with subject or body 'help' to ifeffit-request(a)millenia.cars.aps.anl.gov You can reach the person managing the list at ifeffit-owner(a)millenia.cars.aps.anl.gov When replying, please edit your Subject line so it is more specific than "Re: Contents of Ifeffit digest..." Today's Topics: 1. Theoretical value of S02 for different absorber (Jatin Rana) 2. Re: Theoretical value of S02 for different absorber (Bruce Ravel) ---------------------------------------------------------------------- Message: 1 Date: Wed, 4 Apr 2012 17:45:13 +0800 (SGT) From: Jatin Rana <jatin_rana2002(a)yahoo.com> To: "ifeffit(a)millenia.cars.aps.anl.gov" <ifeffit(a)millenia.cars.aps.anl.gov> Subject: [Ifeffit] Theoretical value of S02 for different absorber Message-ID: <1333532713.14873.YahooMailNeo(a)web193501.mail.sg3.yahoo.com> Content-Type: text/plain; charset="utf-8" Dear Ifeffit users, Can anyone recommend a reference where I can get calculated value of S02 for various elements ? I would like to compare them with my fitted values. Many thanks in advance !! With best regards, Jatin Rana

1 0

Re: [Ifeffit] Theoretical value of S02 for different absorber
by Rana, Jatinkumar Kantilal 05 Apr '12

05 Apr '12

Dear Dr. Ravel, Thank you very much for your prompt reply. I followed the instruction on (http://leonardo.phys.washington.edu/feff/wiki/index.php?title=EXAFS_calcula…) and activated S02 card by making S02 0.0 in feff.inp file. As a result, FEFF calculated S02. However, I get confused when I compare FEFF calculated S02 with reported value in literature and the one obtained from my own fit. For example, S02 for Mn absorber FEFF calculated 0.94 Fitted value for my sample ~0.75 Fitted value for various Mn-oxides ~0.75 Other reported value from Mn fit ~0.70-0.80 (http://www.sciencedirect.com/science/article/pii/S0020169397057496) S02 for Fe absorber FEFF calculated ~0.94 Fitted value for Fe-foil ~0.70 Fitted value for various Fe-oxides ~0.70 Reported value by calculation 0.69 (http://prola.aps.org/abstract/PR/v169/i1/p27_1) It is well known that S02 is a property of central absorbing atom and is considered to be constant between 0.7 - 1.0. However, in real world, S02 is affected by various systematic errors like thicker samples (in transmission), absorption by other elements in sample at given energy range and many more. These factors can suppress the amplitudes of EXAFS oscillations resulting in lower value of fitted S02 than it should be for a given absorber. The reason I am so much curious about S02 is, if I know what should be (or should be considered) the value of S02 for a given absorber than I can possibly understand the reasons of fitted S02 being too low or too high. For instance, S02 and Sigma2 are highly correlated for first shell. So from fitted value of S02 and Sigma2 of first shell, I could possibly understand changes in local structure. So I would like to put forward very simple but still confusing question that which value of S02 should be considered, FEFF calculated or the one obtained by fitting standards (either reference foils or reference compounds measured on same beamline) and why. Many thanks in advance !!! With best regards, Jatin -----Original Message----- From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of ifeffit-request(a)millenia.cars.aps.anl.gov Sent: Mittwoch, 4. April 2012 19:00 To: ifeffit(a)millenia.cars.aps.anl.gov Subject: Ifeffit Digest, Vol 110, Issue 5 Send Ifeffit mailing list submissions to ifeffit(a)millenia.cars.aps.anl.gov To subscribe or unsubscribe via the World Wide Web, visit http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit or, via email, send a message with subject or body 'help' to ifeffit-request(a)millenia.cars.aps.anl.gov You can reach the person managing the list at ifeffit-owner(a)millenia.cars.aps.anl.gov When replying, please edit your Subject line so it is more specific than "Re: Contents of Ifeffit digest..." Today's Topics: 1. Theoretical value of S02 for different absorber (Jatin Rana) 2. Re: Theoretical value of S02 for different absorber (Bruce Ravel) ---------------------------------------------------------------------- Message: 1 Date: Wed, 4 Apr 2012 17:45:13 +0800 (SGT) From: Jatin Rana <jatin_rana2002(a)yahoo.com> To: "ifeffit(a)millenia.cars.aps.anl.gov" <ifeffit(a)millenia.cars.aps.anl.gov> Subject: [Ifeffit] Theoretical value of S02 for different absorber Message-ID: <1333532713.14873.YahooMailNeo(a)web193501.mail.sg3.yahoo.com> Content-Type: text/plain; charset="utf-8" Dear Ifeffit users, Can anyone recommend a reference where I can get calculated value of S02 for various elements ? I would like to compare them with my fitted values. Many thanks in advance !! With best regards, Jatin Rana

2 1

Results Reporting
by John Farell 05 Apr '12

05 Apr '12

Hi.. Can you please explain me how to report the errors of EXAFS parameters like bond length, Co - ordination number, disorder etc. (Or how to obtain those errors ??) with regards John

2 1

Theoretical value of S02 for different absorber
by Jatin Rana 04 Apr '12

04 Apr '12

Dear Ifeffit users, Can anyone recommend a reference where I can get calculated value of S02 for various elements ? I would like to compare them with my fitted values. Many thanks in advance !! With best regards, Jatin Rana

2 1

Re: [Ifeffit] Ifeffit Digest, Vol 110, Issue 2
by Carmelo Prestipino 02 Apr '12

02 Apr '12

I suppose the problem is about writing privilege on the folder Program files/Prestopronto. My script in Inno setup theoretically should change it, but evidently practically it doesn't work I'll solve it in the next upgrade. At this time you should change manually the writing privilege or install the program in a different directory. However, it is very difficult to understand what happen, and reproduce the error if the report is "it doesn't work ", (actually I'm not sure at all of diagnosis) normally, all the codes gives clear instruction how to report the error. Ciao Lello On Mon, Apr 2, 2012 at 11:24 AM, <ifeffit-request(a)millenia.cars.aps.anl.gov>wrote: > Send Ifeffit mailing list submissions to > ifeffit(a)millenia.cars.aps.anl.gov > > To subscribe or unsubscribe via the World Wide Web, visit > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > or, via email, send a message with subject or body 'help' to > ifeffit-request(a)millenia.cars.aps.anl.gov > > You can reach the person managing the list at > ifeffit-owner(a)millenia.cars.aps.anl.gov > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Ifeffit digest..." > > > Today's Topics: > > 1. Re: Ifeffit Digest, Vol 110, Issue 1 (KONG Quingyu) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 2 Apr 2012 09:23:31 +0000 > From: KONG Quingyu <kong(a)synchrotron-soleil.fr> > To: "ifeffit(a)millenia.cars.aps.anl.gov" > <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] Ifeffit Digest, Vol 110, Issue 1 > Message-ID: > < > 30F62EE734A9EA45A3F211513CB620033621987A(a)SUN-DAG2.synchrotron-soleil.fr> > > Content-Type: text/plain; charset="iso-8859-1" > > Thanks a lot Santiago, by the way, does the program need windows Vista or > 7, I installed it in windows XP with different versions, b.0.7 or b.0.6, > but it did not work. > > Best regards, > Qingyu > > Beamline ODE > Soci?t? civile Synchrotron SOLEIL > L'Orme des Merisiers > Saint-Aubin - BP 48 > 91192 GIF-sur-YVETTE CEDEX > Tel: +33 (0)1 69 35 97 84 (office) > +33 (0)1 69 35 97 16 (beamline) > email: kong(a)synchrotron-soleil.fr > > ________________________________________ > From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [ > ifeffit-bounces(a)millenia.cars.aps.anl.gov] on behalf of > ifeffit-request(a)millenia.cars.aps.anl.gov [ > ifeffit-request(a)millenia.cars.aps.anl.gov] > Sent: Monday, April 02, 2012 9:09 AM > To: ifeffit(a)millenia.cars.aps.anl.gov > Subject: Ifeffit Digest, Vol 110, Issue 1 > > Send Ifeffit mailing list submissions to > ifeffit(a)millenia.cars.aps.anl.gov > > To subscribe or unsubscribe via the World Wide Web, visit > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > or, via email, send a message with subject or body 'help' to > ifeffit-request(a)millenia.cars.aps.anl.gov > > You can reach the person managing the list at > ifeffit-owner(a)millenia.cars.aps.anl.gov > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Ifeffit digest..." > > > Today's Topics: > > 1. Prestopronto links (Santiago Jos? Alejandro Figueroa) > 2. Re: Prestopronto links (Matt Newville) > 3. Re: Ifeffit Digest, Vol 109, Issue 24 (KONG Quingyu) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 1 Apr 2012 17:53:33 -0300 (BRT) > From: Santiago Jos? Alejandro Figueroa <figueroa(a)fisica.unlp.edu.ar> > To: ifeffit(a)millenia.cars.aps.anl.gov > Subject: [Ifeffit] Prestopronto links > Message-ID: > <37353.189.61.212.168.1333313613.squirrel(a)mail.fisica.unlp.edu.ar> > Content-Type: text/plain;charset=iso-8859-1 > > Hi > > I can add my two cent...(of REAIS) > > The web page of PRESTOPRONTO pointed out by Carmelo is: > > http://www.esrf.eu/computing/scientific/PRESTOPRONTO/Prestopronto.htm > > or > > http://code.google.com/p/prestopronto/ > > We start to use here at the LNLS in the dispersive beamline with > promissing results. > > > Best regards, > > Santiago > > > > ------------------------------ > > Message: 2 > Date: Sun, 1 Apr 2012 17:44:03 -0500 > From: Matt Newville <newville(a)cars.uchicago.edu> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] Prestopronto links > Message-ID: > <CA+7ESbr73fkgSeBCGTZN=5HXZe3P9jHCVgZ36-aCd1PD=8X8VQ(a)mail.gmail.com > > > Content-Type: text/plain; charset=ISO-8859-1 > > Santago, Carmelo, > > 2012/4/1 Santiago Jos? Alejandro Figueroa <figueroa(a)fisica.unlp.edu.ar>: > > Hi > > > > I can add my two cent...(of REAIS) > > > > The web page of PRESTOPRONTO pointed out by Carmelo is: > > > > http://www.esrf.eu/computing/scientific/PRESTOPRONTO/Prestopronto.htm > > > > or > > > > http://code.google.com/p/prestopronto/ > > > > We start to use here at the LNLS in the dispersive beamline with > > promissing results. > > > > Looks great! Thanks for the links! > > --Matt > > > > ------------------------------ > > Message: 3 > Date: Mon, 2 Apr 2012 07:08:37 +0000 > From: KONG Quingyu <kong(a)synchrotron-soleil.fr> > To: "ifeffit(a)millenia.cars.aps.anl.gov" > <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 24 > Message-ID: > < > 30F62EE734A9EA45A3F211513CB6200336217807(a)SUN-DAG2.synchrotron-soleil.fr> > > Content-Type: text/plain; charset="us-ascii" > > Thanks a lot Lello and Matt, > > Now, it is clear to us that we anyway need our own program to > automatically treat the big data set. > > I tried XOP with XAID extension following your suggestions, it is nice to > have the graphic operation, but for a big data set with thousands of > spectra we need a program. Since we have already part of the program and > hope we will finish it soon to a kind of platform for automatic data > analysis. > > Regards, > Qingyu > > > > > -----Original Message----- > From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [mailto: > ifeffit-bounces(a)millenia.cars.aps.anl.gov] On Behalf Of > ifeffit-request(a)millenia.cars.aps.anl.gov > Sent: Saturday, March 31, 2012 9:47 PM > To: ifeffit(a)millenia.cars.aps.anl.gov > Subject: Ifeffit Digest, Vol 109, Issue 24 > > Send Ifeffit mailing list submissions to > ifeffit(a)millenia.cars.aps.anl.gov > > To subscribe or unsubscribe via the World Wide Web, visit > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > or, via email, send a message with subject or body 'help' to > ifeffit-request(a)millenia.cars.aps.anl.gov > > You can reach the person managing the list at > ifeffit-owner(a)millenia.cars.aps.anl.gov > > When replying, please edit your Subject line so it is more specific than > "Re: Contents of Ifeffit digest..." > > > Today's Topics: > > 1. Re: Ifeffit Digest, Vol 109, Issue 23 (Carmelo Prestipino) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 31 Mar 2012 21:47:19 +0200 > From: Carmelo Prestipino <carmelo.prestipino(a)univ-rennes1.fr> > To: ifeffit(a)millenia.cars.aps.anl.gov > Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 23 > Message-ID: > <CAHPDQqzGdbkRaXri5raNG60oiT+h1CPdgZVRHmWZ_CvVGML4ig(a)mail.gmail.com > > > Content-Type: text/plain; charset="iso-8859-1" > > Hi > if I can add my two cent...(of EURO) > > At the ESRF for this kind of studies some work has been done. > > There is the module XAID of the code XOP wrote by Manuel Sanchez del Rio > http://www.esrf.eu/computing/scientific/xop2.1/documentation.html > It is written in IDL from scratch. > It have normalization and linear combination fit features, it is fully > graphically interfaced and in terms of performances is quite impressive. > However, the program has a complicate structure that could difficult at > the first approach (but Manuel is very kind, don't hesitate to contact him > ). > > Otherwise there is the code PrestoPronto. > code.google.com/p/*prestopronto*/ > It is mainly a fully graphic python wrapping of ifeffit for sequential > analysis. > In terms of speed is very similar to Athena but it do automatically the > tricks of cleaning memory But it is still a very young code...... > It still has not any interface to linear combination analysis. > We never tested for more than 1000.. spectra > > Probably Mark Newton could give you some more suggestions Regards Lello > > > On Sat, Mar 31, 2012 at 7:00 PM, > <ifeffit-request(a)millenia.cars.aps.anl.gov>wrote: > > > Send Ifeffit mailing list submissions to > > ifeffit(a)millenia.cars.aps.anl.gov > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > > or, via email, send a message with subject or body 'help' to > > ifeffit-request(a)millenia.cars.aps.anl.gov > > > > You can reach the person managing the list at > > ifeffit-owner(a)millenia.cars.aps.anl.gov > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of Ifeffit digest..." > > > > > > Today's Topics: > > > > 1. Re: Ifeffit Digest, Vol 109, Issue 21 (KONG Quingyu) > > 2. Re: Ifeffit Digest, Vol 109, Issue 21 (Matt Newville) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Fri, 30 Mar 2012 18:32:49 +0000 > > From: KONG Quingyu <kong(a)synchrotron-soleil.fr> > > To: "ifeffit(a)millenia.cars.aps.anl.gov" > > <ifeffit(a)millenia.cars.aps.anl.gov> > > Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 21 > > Message-ID: > > < > > 30F62EE734A9EA45A3F211513CB6200324BD1922(a)SUN-DAG1.synchrotron-soleil.f > > r> > > > > Content-Type: text/plain; charset="iso-8859-1" > > > > Thank you for all your helpful suggestions. > > > > Actually, our beamline has recently installed a fast detector with a > > repetation of 20KHz, which could collected data in a time resolution > > of few tens of microsecond and could easily produce a large amount of > > data. We have already accepted three users for microsecond time > > resolved studies, each of them collected millions of spectra. > > > > Together with the company that provide the detector and our software > > engineer we started to write a Python program to treat the data from > > November last year, however our engineer quitted recently, this make > > us to look for other solutions. > > > > If Demeter would works well for large amount of data treatment, it > > would be great. As I know ID24 at ESRF has also installed the same > > detector and would probably have similar demanding for such a program. > > > > Probably due to the limited memory of my pc, I can only open around > > 200 spectra simultanously with Athena. > > > > Best, > > Qingyu Kong > > > > > > Beamline ODE > > Soci?t? civile Synchrotron SOLEIL > > L'Orme des Merisiers > > Saint-Aubin - BP 48 > > 91192 GIF-sur-YVETTE CEDEX > > Tel: +33 (0)1 69 35 97 84 (office) > > +33 (0)1 69 35 97 16 (beamline) > > email: kong(a)synchrotron-soleil.fr > > > > ________________________________________ > > From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [ > > ifeffit-bounces(a)millenia.cars.aps.anl.gov] on behalf of > > ifeffit-request(a)millenia.cars.aps.anl.gov [ > > ifeffit-request(a)millenia.cars.aps.anl.gov] > > Sent: Friday, March 30, 2012 3:29 PM > > To: ifeffit(a)millenia.cars.aps.anl.gov > > Subject: Ifeffit Digest, Vol 109, Issue 21 > > > > Send Ifeffit mailing list submissions to > > ifeffit(a)millenia.cars.aps.anl.gov > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > > or, via email, send a message with subject or body 'help' to > > ifeffit-request(a)millenia.cars.aps.anl.gov > > > > You can reach the person managing the list at > > ifeffit-owner(a)millenia.cars.aps.anl.gov > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of Ifeffit digest..." > > > > > > Today's Topics: > > > > 1. delr and ss parameterization (Ebrahim Rezaei) > > 2. program (KONG Quingyu) > > 3. Re: program (Carlo Segre) > > 4. Re: program (Matt Newville) > > 5. Re: delr and ss parameterization (Bruce Ravel) > > 6. Re: program (Bruce Ravel) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Thu, 29 Mar 2012 21:57:21 -0700 (PDT) > > From: Ebrahim Rezaei <e_rezaei24(a)yahoo.com> > > To: ifeffit(a)millenia.cars.aps.anl.gov > > Subject: [Ifeffit] delr and ss parameterization > > Message-ID: > > > > <1333083441.66190.YahooMailClassic(a)web120501.mail.ne1.yahoo.com> > > Content-Type: text/plain; charset="utf-8" > > > > Hi everyone, > > > > > > I am working with bulk and nano structures of MnO, MnO2 and Mn2O3. I > > have two questions regarding how to parameterize delr and ss in > > Artemis. > > > > > > It is recommended to > > use either delr=alfa*Reff or relate it to unit cell structure. Can > > anybody tell me how to do the second approach? > > > > > > Also is that a good idea to let all ss to float? Or it should be > > constrained based on type and distance of atoms? I have seen an > > example about ZnO just letting the first two shell?s ss to float and > > let the rest to be the same. I would appreciate it very much if > > anybody could help me with this too. > > > > > > Regards, > > > > > > Ebrahim Rezaei > > > > PhD Candidate, Chem. Eng. Dep., University of Saskatchewan > > > > Saskatoon, SK, Canada > > -------------- next part -------------- An HTML attachment was > > scrubbed... > > URL: < > > http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/2012032 > > 9/e13d64a5/attachment-0001.htm > > > > > > > ------------------------------ > > > > Message: 2 > > Date: Fri, 30 Mar 2012 06:46:38 +0000 > > From: KONG Quingyu <kong(a)synchrotron-soleil.fr> > > To: "ifeffit(a)millenia.cars.aps.anl.gov" > > <ifeffit(a)millenia.cars.aps.anl.gov> > > Subject: [Ifeffit] program > > Message-ID: > > < > > 30F62EE734A9EA45A3F211513CB6200324BD1794(a)SUN-DAG1.synchrotron-soleil.f > > r> > > > > Content-Type: text/plain; charset="iso-8859-1" > > > > We recently collected 3000 EXAFS spectra to follow a chemical reaction > > process, we would like to fit the experimental data with reference > > spectra using linear combination fitting. Does anyone know there are > > available or commercial program to normalize the data and make linear > > combination fitting. We tried to do it with Athena manually, but 3000 > > spectra take a long time for us and it is not a practical way. > > > > Thanks a lot. > > > > Qingyu Kong > > > > Beamline ODE > > Soci?t? civile Synchrotron SOLEIL > > L'Orme des Merisiers > > Saint-Aubin - BP 48 > > 91192 GIF-sur-YVETTE CEDEX > > Tel: +33 (0)1 69 35 97 84 (office) > > +33 (0)1 69 35 97 16 (beamline) > > email: kong(a)synchrotron-soleil.fr > > > > > > > > > > > > ------------------------------ > > > > Message: 3 > > Date: Fri, 30 Mar 2012 01:56:57 -0500 (CDT) > > From: Carlo Segre <segre(a)iit.edu> > > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > > Subject: Re: [Ifeffit] program > > Message-ID: <alpine.DEB.2.02.1203300154590.5339(a)hydride.phys.iit.edu> > > Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed > > > > > > Hi Qingyu > > > > On Fri, 30 Mar 2012, KONG Quingyu wrote: > > > > > We recently collected 3000 EXAFS spectra to follow a chemical > > > reaction process, we would like to fit the experimental data with > > > reference spectra using linear combination fitting. Does anyone know > > > there are available or commercial program to normalize the data and > > > make linear combination fitting. We tried to do it with Athena > > > manually, but 3000 spectra take a long time for us and it is not a > practical way. > > > > You might wish to look into Bruce Ravel's new suite of programs > > (called Demeter). I believe they are built to allow easy scripting of > > massive quantities of data such as yours. > > > > Carlo > > > > > > -- > > Carlo U. Segre -- Duchossois Leadership Professor of Physics Associate > > Dean for Graduate Admissions, Graduate College Illinois Institute of > > Technology > > Voice: 312.567.3498 Fax: 312.567.3494 > > segre(a)iit.edu http://phys.iit.edu/~segre segre(a)debian.org > > > > > > ------------------------------ > > > > Message: 4 > > Date: Fri, 30 Mar 2012 07:56:05 -0500 > > From: Matt Newville <newville(a)cars.uchicago.edu> > > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > > Subject: Re: [Ifeffit] program > > Message-ID: > > > > <CA+7ESbrjp46_kPw9F803MQOGdMRtFbDa-7oGem25+CL2bYznJg(a)mail.gmail.com > > > > > Content-Type: text/plain; charset=ISO-8859-1 > > > > Hi Qingyu, > > > > On Fri, Mar 30, 2012 at 1:46 AM, KONG Quingyu > > <kong(a)synchrotron-soleil.fr> wrote: > > > We recently collected 3000 EXAFS spectra to follow a chemical > > > reaction > > process, we would like to fit the experimental data with reference > > spectra using linear combination fitting. Does anyone know there are > > available or commercial program to normalize the data and make linear > > combination fitting. We tried to do it with Athena manually, but 3000 > > spectra take a long time for us and it is not a practical way. > > > > Ifeffit is designed to be scriptable for batch processing. This can > > be done with a raw Ifeffit script (which isn't very flexible) or from > > Perl or Python, which are more flexible. Carlo mentioned Demeter, > > which is a high-level Perl wrapping of the lower-level functionality. > > Working with 3000 spectra may be challenging with Ifeffit version > > 1.2, as the total amount of available memory is limited, but with some > > care it should be possible. > > > > FWIW, the next version of Ifeffit will eliminate this memory > > limitation, and make both scripting and extending the functionality > > much easier and better. This is still a work in progress (see > > https://github.com/xraypy/xraylarch) and poorly documented, but the > > essential idea is to write all of Ifeffit's functionality in Python, > > for all the reasons just stated, and this far enough along that I am > > confident it will be a big improvement. So far (for XAFS), this can > > read data, do simple pre-edge subtraction, normalization, remove > > backgrounds ala autobk, and do XAFS Fourier transforms. It can do > > simple "general purpose fitting", but does not yet expose a > > simple-to-use linear combination fitting functionality (shouldn't be > > too hard, as all the linear algebra pieces are readily available) or > > fitting spectra to Feff calculations (a bit more involved). Work is > > slow (mostly due to other obligations), but progress is being made, > > and it's close to ready for an initial release. If you are (or > > anyone else is) interested in trying this or helping out, let me know. > > > > --Matt > > > > > > > > ------------------------------ > > > > Message: 5 > > Date: Fri, 30 Mar 2012 09:13:11 -0400 > > From: Bruce Ravel <bravel(a)bnl.gov> > > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > > Subject: Re: [Ifeffit] delr and ss parameterization > > Message-ID: <201203300913.12166.bravel(a)bnl.gov> > > Content-Type: Text/Plain; charset="utf-8" > > > > On Friday, March 30, 2012 12:57:21 am Ebrahim Rezaei wrote: > > > I am working with bulk and nano structures of MnO, MnO2 and Mn2O3. I > > > have two questions regarding how to parameterize delr and ss in > > > Artemis. > > > > > > > > > It is recommended to > > > use either delr=alfa*Reff or relate it to unit cell structure. Can > > anybody > > > tell me how to do the second approach? > > > > > > > > > Also is that a good idea to let all ss to float? Or it should be > > > constrained based on type and distance of atoms? I have seen an > > example > > > about ZnO just letting the first two shell?s ss to float and let the > > > rest to be the same. I would appreciate it very much if anybody > > > could help me with this too. > > > > The alpha*reff trick is only strictly valid for cubic, monoatomic > > materials. For everything else in the world, it is an increasingly > > poor approximation. That's not to say it is not a helpful > > parameterization, just that you need to be mindful of its limitations. > > > > As for the sigma^2 parameter, you have to balance when is physically > > reasonable with what can be reliably extracted from a real data > > analysis. > > > > - Two atoms of a different species at the same distance should have > > different sigma^2 because they have different masses. > > > > - Two atoms of the same species at different distances should have > > different sigma^2 because sigma^2 is a mean square deviation in > > distance. > > > > So, in some sense, all scatterers might have their own sigma^2 > > parameters. But that is not to say that you data are sufficient to > > independently determine that many parameters. When you run up against > > the limited information content of your data, that is when you start > > making approximations of the sort that lead to setting sigma^2 > > parameters for different scatterers to be the same. > > > > However, there are sensible and physically justifiable reasons why the > > sigma^2 for certain multiple scattering paths should be related to > > those of single scattering paths. See > > http://dx.doi.org/10.1103/PhysRevB.54.156 > > > > Two of my favorite papers that show excellent data analysis solving > > real-world problems and which are written in a way that is of > > pedagogical value for XAS beginners are Scott Calvin's paper on doped > > ferrites (http://dx.doi.org/10.1103/PhysRevB.66.224405) and Shelly > > Kelly's paper on uranyl binding to biomass > > (http://dx.doi.org/10.1016/S0016-7037(02)00947-X) For more > > interesting papers, look through the archives of the mailing list and > > see who gives good answers to questions, then look up their papers. > > > > Another extremely useful resource which includes extensive discussion > > of the sorts of things you are asking about is Shelly's review > > chapter. Here's the amazon link to that book: > > > > http://www.amazon.com/Methods-Soil-Analysis-Part-Mineralogical/dp/0891 > > 188460/ The book is a bit pricy, but if you are a clever googler, you > > may find a PDF copy of her chapter. > > > > Regards, > > B > > > > > > -- > > > > Bruce Ravel ------------------------------------ bravel(a)bnl.gov > > > > National Institute of Standards and Technology Synchrotron Methods > > Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, > > 11973 > > > > My homepage: http://xafs.org/BruceRavel > > EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter > > > > > > > > ------------------------------ > > > > Message: 6 > > Date: Fri, 30 Mar 2012 09:29:10 -0400 > > From: Bruce Ravel <bravel(a)bnl.gov> > > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > > Subject: Re: [Ifeffit] program > > Message-ID: <201203300929.11143.bravel(a)bnl.gov> > > Content-Type: Text/Plain; charset="iso-8859-1" > > > > On Friday, March 30, 2012 02:46:38 am KONG Quingyu wrote: > > > We recently collected 3000 EXAFS spectra to follow a chemical > > > reaction process, we would like to fit the experimental data with > > > reference > > spectra > > > using linear combination fitting. Does anyone know there are > > > available or commercial program to normalize the data and make > > > linear combination fitting. We tried to do it with Athena manually, > > > but 3000 spectra take a long time for us and it is not a practical way. > > > > Hi Qingyu, > > > > Carlo is correct, Demeter could be used to write a special purpose > > program for automating this sort of thing. The performance in terms > > of speed may not be exactly what you want, but the performance in > > terms of quickly prototyping a program to process and analyze a large > > data volume is excellent. > > > > The Demeter code base comes with extensive (although still incomplete) > > documentation as well as plenty of example scripts, including examples > > for the individual chores you mention above. > > > > Here's Demeter's documentation: http://bruceravel.github.com/demeter/ > > > > > > As a closing comment, isn't it somehow strange that you and your > > colleagues went to all the trouble of building a fancy dispersive > > beamline without thinking ahead about how you were going to analyze > > your data? > > > > B > > > > > > -- > > > > Bruce Ravel ------------------------------------ bravel(a)bnl.gov > > > > National Institute of Standards and Technology Synchrotron Methods > > Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, > > 11973 > > > > My homepage: http://xafs.org/BruceRavel > > EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter > > > > > > ------------------------------ > > > > _______________________________________________ > > Ifeffit mailing list > > Ifeffit(a)millenia.cars.aps.anl.gov > > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > > > > > > End of Ifeffit Digest, Vol 109, Issue 21 > > **************************************** > > > > > > > > ------------------------------ > > > > Message: 2 > > Date: Fri, 30 Mar 2012 16:36:58 -0500 > > From: Matt Newville <newville(a)cars.uchicago.edu> > > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > > Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 21 > > Message-ID: > > > > <CA+7ESbrZir3zMWAKSx5HPpqjJ5H_qC-V0YAsnCXUzDO512q+aA(a)mail.gmail.com > > > > > Content-Type: text/plain; charset=ISO-8859-1 > > > > Hi Qingyu, > > > > On Fri, Mar 30, 2012 at 1:32 PM, KONG Quingyu > > <kong(a)synchrotron-soleil.fr> wrote: > > > Thank you for all your helpful suggestions. > > > > > > Actually, our beamline has recently installed a fast detector with a > > > repetation of 20KHz, which could collected data in a time resolution > > > of few tens of microsecond and could easily produce a large amount > > > of data. We have already accepted three users for microsecond time > > > resolved studies, each of them collected millions of spectra. > > > > > > Together with the company that provide the detector and our software > > > engineer we started to write a Python program to treat the data from > > > November last year, however our engineer quitted recently, this make > > > us to look for other solutions. > > > > Bruce wrote earlier, > > > > As a closing comment, isn't it somehow strange that you and your > > colleagues went to all the trouble of building a fancy dispersive > > beamline without thinking ahead about how you were going to analyze > > your data? > > > > Sadly, Bruce's observation is not unusual at all. Everyone is > > experiencing data overloads, and almost no one is investing in > > computing resources. Yours may be slightly larger than most XAFS > > users, but only by one or two orders of magnitude. Then again, you > > had a software engineer at your beamline at least for a while. > > > > To be clear, Ifeffit, Athena, Demeter, Sixpack, and so on are written > > primarily by beamline scientists like yourself. Your choices are to > > use what someone else gives you, write your own, or hire someone to do > > it for you. It appears that the last choice rarely works very well. > > That you get to use Bruce's codes at all (and for free!) is not an > > accident: it is part of the design, and an outcome of the programs > > being written by beamline scientists. > > > > The current software base is more than ten years old, and showing its > > age especially with respect to large data sets and multiple > > processors. This is a really good opportunity for you and your > > colleagues (and others in similar situations) to join in. That is, > > you should write code to deal with your fast data rates. If you write > > that with the Ifeffit family of programs, not only will people at ESRF > > be able to use it, but so will anyone else. > > > > > If Demeter would works well for large amount of data treatment, it > > > would be great. As I know ID24 at ESRF has also installed the same > > > detector and would probably have similar demanding for such a program. > > > > Looking into Demeter is one option, though I don't know exactly that > > is perfect for the linear combination analysis you want to use. Most > > of the core algorithms in Demeter still use Ifeffit 1.2, which are > > going to be limited for your needs, though they can be worked around > > if careful. Transferring Demeter to use the newer Ifeffit2 should be > > possible, but neither Bruce or I have done more than > > proof-of-principle tests. Looking into using Larch / IFeffit2 > > might be better for you, and I'd be happy to help you do that. For > > one thing, this approach has no memory limitations beyond your > > machine, is in Python and can read HDF5 files (which SOLEIL uses, as > > far as I understand) directly. It also has the possibility of using > > multicore machines well, but this has not yet been automated. It is > > also possible that the linear algebra techniques used for x-ray > > fluorescence map analysis, including code from Gerd Wellenreuther and > > Armando Sole would be useful for your needs. > > > > > Probably due to the limited memory of my pc, I can only open around > > > 200 spectra simultanously with Athena. > > > > You almost certainly want to not use Athena here anyway. You want > > automated processing programs. > > > > Like Bruce hinted at, you've invested heavily in a detector that > > allows you to collect data at a rate orders of magnitude faster than > > you could before. It would have been incredibly naive to believe that > > decade-old software would have been able to keep up with that. This > > is a good opportunity for you to help out yourself and others. > > > > Cheers, > > > > --Matt > > > > > > ------------------------------ > > > > _______________________________________________ > > Ifeffit mailing list > > Ifeffit(a)millenia.cars.aps.anl.gov > > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > > > > > > End of Ifeffit Digest, Vol 109, Issue 23 > > **************************************** > > > > > > -- > Carmelo Prestipino > Sciences Chimiques de Rennes UMR 6226 > Solid State and Materials Chemistry group (CSM) Campus de Beaulieu, B?t > 10B, bureau 122 > F-35042 RENNES > France > Tel: (+33) 22323 6531 > FAX: (+33) 22323 5959 >

1 0

data from 9.3.1
by Bruce Ravel 02 Apr '12

02 Apr '12

> Hi Dr. Bruce Ravel > > I have Cl K-edge XAS data (as attached for example) that I took from > beamline 9.3.1 ALS four years ago. I can’t import the data into the > Athena, what’s a problem. I want to study XANES of the data. > Thank you, > > Amphol Wongjamras > Department of Physics and Materials Science, Faculty of Science > Chiang Mai University > Chiang Mai, Thailand. Amphol, I am forwarding your question (and my answer) to the Ifeffit mailing list, which is the appropriate venue for asking questions about the software. Please subscribe to and use the mailing list in the future. The problem with the data file you attached is that there are lines with just numbers before the line with the column lables. This confuses Ifeffit and therefore confuses Athena. The simplest solution is to comment out (or, I suppose, delete) the first few lines in the file, like so: # 7 # 008 # # # Mono ev Isignal Izero Ph E 2800.000 -0.05113220 0.30488586 2799.981 2800.500 -0.05165100 0.30612183 2800.482 2801.000 -0.05169678 0.30537415 2800.976 Regards, B -- Bruce Ravel ------------------------------------ bravel(a)bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter

1 0

Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 24
by KONG Quingyu 02 Apr '12

02 Apr '12

Thanks a lot Lello and Matt, Now, it is clear to us that we anyway need our own program to automatically treat the big data set. I tried XOP with XAID extension following your suggestions, it is nice to have the graphic operation, but for a big data set with thousands of spectra we need a program. Since we have already part of the program and hope we will finish it soon to a kind of platform for automatic data analysis. Regards, Qingyu -----Original Message----- From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of ifeffit-request(a)millenia.cars.aps.anl.gov Sent: Saturday, March 31, 2012 9:47 PM To: ifeffit(a)millenia.cars.aps.anl.gov Subject: Ifeffit Digest, Vol 109, Issue 24 Send Ifeffit mailing list submissions to ifeffit(a)millenia.cars.aps.anl.gov To subscribe or unsubscribe via the World Wide Web, visit http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit or, via email, send a message with subject or body 'help' to ifeffit-request(a)millenia.cars.aps.anl.gov You can reach the person managing the list at ifeffit-owner(a)millenia.cars.aps.anl.gov When replying, please edit your Subject line so it is more specific than "Re: Contents of Ifeffit digest..." Today's Topics: 1. Re: Ifeffit Digest, Vol 109, Issue 23 (Carmelo Prestipino) ---------------------------------------------------------------------- Message: 1 Date: Sat, 31 Mar 2012 21:47:19 +0200 From: Carmelo Prestipino <carmelo.prestipino(a)univ-rennes1.fr> To: ifeffit(a)millenia.cars.aps.anl.gov Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 23 Message-ID: <CAHPDQqzGdbkRaXri5raNG60oiT+h1CPdgZVRHmWZ_CvVGML4ig(a)mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Hi if I can add my two cent...(of EURO) At the ESRF for this kind of studies some work has been done. There is the module XAID of the code XOP wrote by Manuel Sanchez del Rio http://www.esrf.eu/computing/scientific/xop2.1/documentation.html It is written in IDL from scratch. It have normalization and linear combination fit features, it is fully graphically interfaced and in terms of performances is quite impressive. However, the program has a complicate structure that could difficult at the first approach (but Manuel is very kind, don't hesitate to contact him ). Otherwise there is the code PrestoPronto. code.google.com/p/*prestopronto*/ It is mainly a fully graphic python wrapping of ifeffit for sequential analysis. In terms of speed is very similar to Athena but it do automatically the tricks of cleaning memory But it is still a very young code...... It still has not any interface to linear combination analysis. We never tested for more than 1000.. spectra Probably Mark Newton could give you some more suggestions Regards Lello On Sat, Mar 31, 2012 at 7:00 PM, <ifeffit-request(a)millenia.cars.aps.anl.gov>wrote: > Send Ifeffit mailing list submissions to > ifeffit(a)millenia.cars.aps.anl.gov > > To subscribe or unsubscribe via the World Wide Web, visit > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > or, via email, send a message with subject or body 'help' to > ifeffit-request(a)millenia.cars.aps.anl.gov > > You can reach the person managing the list at > ifeffit-owner(a)millenia.cars.aps.anl.gov > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Ifeffit digest..." > > > Today's Topics: > > 1. Re: Ifeffit Digest, Vol 109, Issue 21 (KONG Quingyu) > 2. Re: Ifeffit Digest, Vol 109, Issue 21 (Matt Newville) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 30 Mar 2012 18:32:49 +0000 > From: KONG Quingyu <kong(a)synchrotron-soleil.fr> > To: "ifeffit(a)millenia.cars.aps.anl.gov" > <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 21 > Message-ID: > < > 30F62EE734A9EA45A3F211513CB6200324BD1922(a)SUN-DAG1.synchrotron-soleil.f > r> > > Content-Type: text/plain; charset="iso-8859-1" > > Thank you for all your helpful suggestions. > > Actually, our beamline has recently installed a fast detector with a > repetation of 20KHz, which could collected data in a time resolution > of few tens of microsecond and could easily produce a large amount of > data. We have already accepted three users for microsecond time > resolved studies, each of them collected millions of spectra. > > Together with the company that provide the detector and our software > engineer we started to write a Python program to treat the data from > November last year, however our engineer quitted recently, this make > us to look for other solutions. > > If Demeter would works well for large amount of data treatment, it > would be great. As I know ID24 at ESRF has also installed the same > detector and would probably have similar demanding for such a program. > > Probably due to the limited memory of my pc, I can only open around > 200 spectra simultanously with Athena. > > Best, > Qingyu Kong > > > Beamline ODE > Soci?t? civile Synchrotron SOLEIL > L'Orme des Merisiers > Saint-Aubin - BP 48 > 91192 GIF-sur-YVETTE CEDEX > Tel: +33 (0)1 69 35 97 84 (office) > +33 (0)1 69 35 97 16 (beamline) > email: kong(a)synchrotron-soleil.fr > > ________________________________________ > From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [ > ifeffit-bounces(a)millenia.cars.aps.anl.gov] on behalf of > ifeffit-request(a)millenia.cars.aps.anl.gov [ > ifeffit-request(a)millenia.cars.aps.anl.gov] > Sent: Friday, March 30, 2012 3:29 PM > To: ifeffit(a)millenia.cars.aps.anl.gov > Subject: Ifeffit Digest, Vol 109, Issue 21 > > Send Ifeffit mailing list submissions to > ifeffit(a)millenia.cars.aps.anl.gov > > To subscribe or unsubscribe via the World Wide Web, visit > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > or, via email, send a message with subject or body 'help' to > ifeffit-request(a)millenia.cars.aps.anl.gov > > You can reach the person managing the list at > ifeffit-owner(a)millenia.cars.aps.anl.gov > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Ifeffit digest..." > > > Today's Topics: > > 1. delr and ss parameterization (Ebrahim Rezaei) > 2. program (KONG Quingyu) > 3. Re: program (Carlo Segre) > 4. Re: program (Matt Newville) > 5. Re: delr and ss parameterization (Bruce Ravel) > 6. Re: program (Bruce Ravel) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 29 Mar 2012 21:57:21 -0700 (PDT) > From: Ebrahim Rezaei <e_rezaei24(a)yahoo.com> > To: ifeffit(a)millenia.cars.aps.anl.gov > Subject: [Ifeffit] delr and ss parameterization > Message-ID: > > <1333083441.66190.YahooMailClassic(a)web120501.mail.ne1.yahoo.com> > Content-Type: text/plain; charset="utf-8" > > Hi everyone, > > > I am working with bulk and nano structures of MnO, MnO2 and Mn2O3. I > have two questions regarding how to parameterize delr and ss in > Artemis. > > > It is recommended to > use either delr=alfa*Reff or relate it to unit cell structure. Can > anybody tell me how to do the second approach? > > > Also is that a good idea to let all ss to float? Or it should be > constrained based on type and distance of atoms? I have seen an > example about ZnO just letting the first two shell?s ss to float and > let the rest to be the same. I would appreciate it very much if > anybody could help me with this too. > > > Regards, > > > Ebrahim Rezaei > > PhD Candidate, Chem. Eng. Dep., University of Saskatchewan > > Saskatoon, SK, Canada > -------------- next part -------------- An HTML attachment was > scrubbed... > URL: < > http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/2012032 > 9/e13d64a5/attachment-0001.htm > > > > ------------------------------ > > Message: 2 > Date: Fri, 30 Mar 2012 06:46:38 +0000 > From: KONG Quingyu <kong(a)synchrotron-soleil.fr> > To: "ifeffit(a)millenia.cars.aps.anl.gov" > <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: [Ifeffit] program > Message-ID: > < > 30F62EE734A9EA45A3F211513CB6200324BD1794(a)SUN-DAG1.synchrotron-soleil.f > r> > > Content-Type: text/plain; charset="iso-8859-1" > > We recently collected 3000 EXAFS spectra to follow a chemical reaction > process, we would like to fit the experimental data with reference > spectra using linear combination fitting. Does anyone know there are > available or commercial program to normalize the data and make linear > combination fitting. We tried to do it with Athena manually, but 3000 > spectra take a long time for us and it is not a practical way. > > Thanks a lot. > > Qingyu Kong > > Beamline ODE > Soci?t? civile Synchrotron SOLEIL > L'Orme des Merisiers > Saint-Aubin - BP 48 > 91192 GIF-sur-YVETTE CEDEX > Tel: +33 (0)1 69 35 97 84 (office) > +33 (0)1 69 35 97 16 (beamline) > email: kong(a)synchrotron-soleil.fr > > > > > > ------------------------------ > > Message: 3 > Date: Fri, 30 Mar 2012 01:56:57 -0500 (CDT) > From: Carlo Segre <segre(a)iit.edu> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] program > Message-ID: <alpine.DEB.2.02.1203300154590.5339(a)hydride.phys.iit.edu> > Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed > > > Hi Qingyu > > On Fri, 30 Mar 2012, KONG Quingyu wrote: > > > We recently collected 3000 EXAFS spectra to follow a chemical > > reaction process, we would like to fit the experimental data with > > reference spectra using linear combination fitting. Does anyone know > > there are available or commercial program to normalize the data and > > make linear combination fitting. We tried to do it with Athena > > manually, but 3000 spectra take a long time for us and it is not a practical way. > > You might wish to look into Bruce Ravel's new suite of programs > (called Demeter). I believe they are built to allow easy scripting of > massive quantities of data such as yours. > > Carlo > > > -- > Carlo U. Segre -- Duchossois Leadership Professor of Physics Associate > Dean for Graduate Admissions, Graduate College Illinois Institute of > Technology > Voice: 312.567.3498 Fax: 312.567.3494 > segre(a)iit.edu http://phys.iit.edu/~segre segre(a)debian.org > > > ------------------------------ > > Message: 4 > Date: Fri, 30 Mar 2012 07:56:05 -0500 > From: Matt Newville <newville(a)cars.uchicago.edu> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] program > Message-ID: > > <CA+7ESbrjp46_kPw9F803MQOGdMRtFbDa-7oGem25+CL2bYznJg(a)mail.gmail.com > > > Content-Type: text/plain; charset=ISO-8859-1 > > Hi Qingyu, > > On Fri, Mar 30, 2012 at 1:46 AM, KONG Quingyu > <kong(a)synchrotron-soleil.fr> wrote: > > We recently collected 3000 EXAFS spectra to follow a chemical > > reaction > process, we would like to fit the experimental data with reference > spectra using linear combination fitting. Does anyone know there are > available or commercial program to normalize the data and make linear > combination fitting. We tried to do it with Athena manually, but 3000 > spectra take a long time for us and it is not a practical way. > > Ifeffit is designed to be scriptable for batch processing. This can > be done with a raw Ifeffit script (which isn't very flexible) or from > Perl or Python, which are more flexible. Carlo mentioned Demeter, > which is a high-level Perl wrapping of the lower-level functionality. > Working with 3000 spectra may be challenging with Ifeffit version > 1.2, as the total amount of available memory is limited, but with some > care it should be possible. > > FWIW, the next version of Ifeffit will eliminate this memory > limitation, and make both scripting and extending the functionality > much easier and better. This is still a work in progress (see > https://github.com/xraypy/xraylarch) and poorly documented, but the > essential idea is to write all of Ifeffit's functionality in Python, > for all the reasons just stated, and this far enough along that I am > confident it will be a big improvement. So far (for XAFS), this can > read data, do simple pre-edge subtraction, normalization, remove > backgrounds ala autobk, and do XAFS Fourier transforms. It can do > simple "general purpose fitting", but does not yet expose a > simple-to-use linear combination fitting functionality (shouldn't be > too hard, as all the linear algebra pieces are readily available) or > fitting spectra to Feff calculations (a bit more involved). Work is > slow (mostly due to other obligations), but progress is being made, > and it's close to ready for an initial release. If you are (or > anyone else is) interested in trying this or helping out, let me know. > > --Matt > > > > ------------------------------ > > Message: 5 > Date: Fri, 30 Mar 2012 09:13:11 -0400 > From: Bruce Ravel <bravel(a)bnl.gov> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] delr and ss parameterization > Message-ID: <201203300913.12166.bravel(a)bnl.gov> > Content-Type: Text/Plain; charset="utf-8" > > On Friday, March 30, 2012 12:57:21 am Ebrahim Rezaei wrote: > > I am working with bulk and nano structures of MnO, MnO2 and Mn2O3. I > > have two questions regarding how to parameterize delr and ss in > > Artemis. > > > > > > It is recommended to > > use either delr=alfa*Reff or relate it to unit cell structure. Can > anybody > > tell me how to do the second approach? > > > > > > Also is that a good idea to let all ss to float? Or it should be > > constrained based on type and distance of atoms? I have seen an > example > > about ZnO just letting the first two shell?s ss to float and let the > > rest to be the same. I would appreciate it very much if anybody > > could help me with this too. > > The alpha*reff trick is only strictly valid for cubic, monoatomic > materials. For everything else in the world, it is an increasingly > poor approximation. That's not to say it is not a helpful > parameterization, just that you need to be mindful of its limitations. > > As for the sigma^2 parameter, you have to balance when is physically > reasonable with what can be reliably extracted from a real data > analysis. > > - Two atoms of a different species at the same distance should have > different sigma^2 because they have different masses. > > - Two atoms of the same species at different distances should have > different sigma^2 because sigma^2 is a mean square deviation in > distance. > > So, in some sense, all scatterers might have their own sigma^2 > parameters. But that is not to say that you data are sufficient to > independently determine that many parameters. When you run up against > the limited information content of your data, that is when you start > making approximations of the sort that lead to setting sigma^2 > parameters for different scatterers to be the same. > > However, there are sensible and physically justifiable reasons why the > sigma^2 for certain multiple scattering paths should be related to > those of single scattering paths. See > http://dx.doi.org/10.1103/PhysRevB.54.156 > > Two of my favorite papers that show excellent data analysis solving > real-world problems and which are written in a way that is of > pedagogical value for XAS beginners are Scott Calvin's paper on doped > ferrites (http://dx.doi.org/10.1103/PhysRevB.66.224405) and Shelly > Kelly's paper on uranyl binding to biomass > (http://dx.doi.org/10.1016/S0016-7037(02)00947-X) For more > interesting papers, look through the archives of the mailing list and > see who gives good answers to questions, then look up their papers. > > Another extremely useful resource which includes extensive discussion > of the sorts of things you are asking about is Shelly's review > chapter. Here's the amazon link to that book: > > http://www.amazon.com/Methods-Soil-Analysis-Part-Mineralogical/dp/0891 > 188460/ The book is a bit pricy, but if you are a clever googler, you > may find a PDF copy of her chapter. > > Regards, > B > > > -- > > Bruce Ravel ------------------------------------ bravel(a)bnl.gov > > National Institute of Standards and Technology Synchrotron Methods > Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, > 11973 > > My homepage: http://xafs.org/BruceRavel > EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter > > > > ------------------------------ > > Message: 6 > Date: Fri, 30 Mar 2012 09:29:10 -0400 > From: Bruce Ravel <bravel(a)bnl.gov> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] program > Message-ID: <201203300929.11143.bravel(a)bnl.gov> > Content-Type: Text/Plain; charset="iso-8859-1" > > On Friday, March 30, 2012 02:46:38 am KONG Quingyu wrote: > > We recently collected 3000 EXAFS spectra to follow a chemical > > reaction process, we would like to fit the experimental data with > > reference > spectra > > using linear combination fitting. Does anyone know there are > > available or commercial program to normalize the data and make > > linear combination fitting. We tried to do it with Athena manually, > > but 3000 spectra take a long time for us and it is not a practical way. > > Hi Qingyu, > > Carlo is correct, Demeter could be used to write a special purpose > program for automating this sort of thing. The performance in terms > of speed may not be exactly what you want, but the performance in > terms of quickly prototyping a program to process and analyze a large > data volume is excellent. > > The Demeter code base comes with extensive (although still incomplete) > documentation as well as plenty of example scripts, including examples > for the individual chores you mention above. > > Here's Demeter's documentation: http://bruceravel.github.com/demeter/ > > > As a closing comment, isn't it somehow strange that you and your > colleagues went to all the trouble of building a fancy dispersive > beamline without thinking ahead about how you were going to analyze > your data? > > B > > > -- > > Bruce Ravel ------------------------------------ bravel(a)bnl.gov > > National Institute of Standards and Technology Synchrotron Methods > Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, > 11973 > > My homepage: http://xafs.org/BruceRavel > EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter > > > ------------------------------ > > _______________________________________________ > Ifeffit mailing list > Ifeffit(a)millenia.cars.aps.anl.gov > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > > > End of Ifeffit Digest, Vol 109, Issue 21 > **************************************** > > > > ------------------------------ > > Message: 2 > Date: Fri, 30 Mar 2012 16:36:58 -0500 > From: Matt Newville <newville(a)cars.uchicago.edu> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 21 > Message-ID: > > <CA+7ESbrZir3zMWAKSx5HPpqjJ5H_qC-V0YAsnCXUzDO512q+aA(a)mail.gmail.com > > > Content-Type: text/plain; charset=ISO-8859-1 > > Hi Qingyu, > > On Fri, Mar 30, 2012 at 1:32 PM, KONG Quingyu > <kong(a)synchrotron-soleil.fr> wrote: > > Thank you for all your helpful suggestions. > > > > Actually, our beamline has recently installed a fast detector with a > > repetation of 20KHz, which could collected data in a time resolution > > of few tens of microsecond and could easily produce a large amount > > of data. We have already accepted three users for microsecond time > > resolved studies, each of them collected millions of spectra. > > > > Together with the company that provide the detector and our software > > engineer we started to write a Python program to treat the data from > > November last year, however our engineer quitted recently, this make > > us to look for other solutions. > > Bruce wrote earlier, > > As a closing comment, isn't it somehow strange that you and your > colleagues went to all the trouble of building a fancy dispersive > beamline without thinking ahead about how you were going to analyze > your data? > > Sadly, Bruce's observation is not unusual at all. Everyone is > experiencing data overloads, and almost no one is investing in > computing resources. Yours may be slightly larger than most XAFS > users, but only by one or two orders of magnitude. Then again, you > had a software engineer at your beamline at least for a while. > > To be clear, Ifeffit, Athena, Demeter, Sixpack, and so on are written > primarily by beamline scientists like yourself. Your choices are to > use what someone else gives you, write your own, or hire someone to do > it for you. It appears that the last choice rarely works very well. > That you get to use Bruce's codes at all (and for free!) is not an > accident: it is part of the design, and an outcome of the programs > being written by beamline scientists. > > The current software base is more than ten years old, and showing its > age especially with respect to large data sets and multiple > processors. This is a really good opportunity for you and your > colleagues (and others in similar situations) to join in. That is, > you should write code to deal with your fast data rates. If you write > that with the Ifeffit family of programs, not only will people at ESRF > be able to use it, but so will anyone else. > > > If Demeter would works well for large amount of data treatment, it > > would be great. As I know ID24 at ESRF has also installed the same > > detector and would probably have similar demanding for such a program. > > Looking into Demeter is one option, though I don't know exactly that > is perfect for the linear combination analysis you want to use. Most > of the core algorithms in Demeter still use Ifeffit 1.2, which are > going to be limited for your needs, though they can be worked around > if careful. Transferring Demeter to use the newer Ifeffit2 should be > possible, but neither Bruce or I have done more than > proof-of-principle tests. Looking into using Larch / IFeffit2 > might be better for you, and I'd be happy to help you do that. For > one thing, this approach has no memory limitations beyond your > machine, is in Python and can read HDF5 files (which SOLEIL uses, as > far as I understand) directly. It also has the possibility of using > multicore machines well, but this has not yet been automated. It is > also possible that the linear algebra techniques used for x-ray > fluorescence map analysis, including code from Gerd Wellenreuther and > Armando Sole would be useful for your needs. > > > Probably due to the limited memory of my pc, I can only open around > > 200 spectra simultanously with Athena. > > You almost certainly want to not use Athena here anyway. You want > automated processing programs. > > Like Bruce hinted at, you've invested heavily in a detector that > allows you to collect data at a rate orders of magnitude faster than > you could before. It would have been incredibly naive to believe that > decade-old software would have been able to keep up with that. This > is a good opportunity for you to help out yourself and others. > > Cheers, > > --Matt > > > ------------------------------ > > _______________________________________________ > Ifeffit mailing list > Ifeffit(a)millenia.cars.aps.anl.gov > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > > > End of Ifeffit Digest, Vol 109, Issue 23 > **************************************** > -- Carmelo Prestipino Sciences Chimiques de Rennes UMR 6226 Solid State and Materials Chemistry group (CSM) Campus de Beaulieu, B?t 10B, bureau 122 F-35042 RENNES France Tel: (+33) 22323 6531 FAX: (+33) 22323 5959

2 1

Re: [Ifeffit] Ifeffit Digest, Vol 110, Issue 1
by KONG Quingyu 02 Apr '12

02 Apr '12

Thanks a lot Santiago, by the way, does the program need windows Vista or 7, I installed it in windows XP with different versions, b.0.7 or b.0.6, but it did not work. Best regards, Qingyu Beamline ODE Société civile Synchrotron SOLEIL L'Orme des Merisiers Saint-Aubin - BP 48 91192 GIF-sur-YVETTE CEDEX Tel: +33 (0)1 69 35 97 84 (office) +33 (0)1 69 35 97 16 (beamline) email: kong(a)synchrotron-soleil.fr ________________________________________ From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [ifeffit-bounces(a)millenia.cars.aps.anl.gov] on behalf of ifeffit-request(a)millenia.cars.aps.anl.gov [ifeffit-request(a)millenia.cars.aps.anl.gov] Sent: Monday, April 02, 2012 9:09 AM To: ifeffit(a)millenia.cars.aps.anl.gov Subject: Ifeffit Digest, Vol 110, Issue 1 Send Ifeffit mailing list submissions to ifeffit(a)millenia.cars.aps.anl.gov To subscribe or unsubscribe via the World Wide Web, visit http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit or, via email, send a message with subject or body 'help' to ifeffit-request(a)millenia.cars.aps.anl.gov You can reach the person managing the list at ifeffit-owner(a)millenia.cars.aps.anl.gov When replying, please edit your Subject line so it is more specific than "Re: Contents of Ifeffit digest..." Today's Topics: 1. Prestopronto links (Santiago Jos? Alejandro Figueroa) 2. Re: Prestopronto links (Matt Newville) 3. Re: Ifeffit Digest, Vol 109, Issue 24 (KONG Quingyu) ---------------------------------------------------------------------- Message: 1 Date: Sun, 1 Apr 2012 17:53:33 -0300 (BRT) From: Santiago Jos? Alejandro Figueroa <figueroa(a)fisica.unlp.edu.ar> To: ifeffit(a)millenia.cars.aps.anl.gov Subject: [Ifeffit] Prestopronto links Message-ID: <37353.189.61.212.168.1333313613.squirrel(a)mail.fisica.unlp.edu.ar> Content-Type: text/plain;charset=iso-8859-1 Hi I can add my two cent...(of REAIS) The web page of PRESTOPRONTO pointed out by Carmelo is: http://www.esrf.eu/computing/scientific/PRESTOPRONTO/Prestopronto.htm or http://code.google.com/p/prestopronto/ We start to use here at the LNLS in the dispersive beamline with promissing results. Best regards, Santiago ------------------------------ Message: 2 Date: Sun, 1 Apr 2012 17:44:03 -0500 From: Matt Newville <newville(a)cars.uchicago.edu> To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> Subject: Re: [Ifeffit] Prestopronto links Message-ID: <CA+7ESbr73fkgSeBCGTZN=5HXZe3P9jHCVgZ36-aCd1PD=8X8VQ(a)mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Santago, Carmelo, 2012/4/1 Santiago Jos? Alejandro Figueroa <figueroa(a)fisica.unlp.edu.ar>: > Hi > > I can add my two cent...(of REAIS) > > The web page of PRESTOPRONTO pointed out by Carmelo is: > > http://www.esrf.eu/computing/scientific/PRESTOPRONTO/Prestopronto.htm > > or > > http://code.google.com/p/prestopronto/ > > We start to use here at the LNLS in the dispersive beamline with > promissing results. > Looks great! Thanks for the links! --Matt ------------------------------ Message: 3 Date: Mon, 2 Apr 2012 07:08:37 +0000 From: KONG Quingyu <kong(a)synchrotron-soleil.fr> To: "ifeffit(a)millenia.cars.aps.anl.gov" <ifeffit(a)millenia.cars.aps.anl.gov> Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 24 Message-ID: <30F62EE734A9EA45A3F211513CB6200336217807(a)SUN-DAG2.synchrotron-soleil.fr> Content-Type: text/plain; charset="us-ascii" Thanks a lot Lello and Matt, Now, it is clear to us that we anyway need our own program to automatically treat the big data set. I tried XOP with XAID extension following your suggestions, it is nice to have the graphic operation, but for a big data set with thousands of spectra we need a program. Since we have already part of the program and hope we will finish it soon to a kind of platform for automatic data analysis. Regards, Qingyu -----Original Message----- From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of ifeffit-request(a)millenia.cars.aps.anl.gov Sent: Saturday, March 31, 2012 9:47 PM To: ifeffit(a)millenia.cars.aps.anl.gov Subject: Ifeffit Digest, Vol 109, Issue 24 Send Ifeffit mailing list submissions to ifeffit(a)millenia.cars.aps.anl.gov To subscribe or unsubscribe via the World Wide Web, visit http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit or, via email, send a message with subject or body 'help' to ifeffit-request(a)millenia.cars.aps.anl.gov You can reach the person managing the list at ifeffit-owner(a)millenia.cars.aps.anl.gov When replying, please edit your Subject line so it is more specific than "Re: Contents of Ifeffit digest..." Today's Topics: 1. Re: Ifeffit Digest, Vol 109, Issue 23 (Carmelo Prestipino) ---------------------------------------------------------------------- Message: 1 Date: Sat, 31 Mar 2012 21:47:19 +0200 From: Carmelo Prestipino <carmelo.prestipino(a)univ-rennes1.fr> To: ifeffit(a)millenia.cars.aps.anl.gov Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 23 Message-ID: <CAHPDQqzGdbkRaXri5raNG60oiT+h1CPdgZVRHmWZ_CvVGML4ig(a)mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Hi if I can add my two cent...(of EURO) At the ESRF for this kind of studies some work has been done. There is the module XAID of the code XOP wrote by Manuel Sanchez del Rio http://www.esrf.eu/computing/scientific/xop2.1/documentation.html It is written in IDL from scratch. It have normalization and linear combination fit features, it is fully graphically interfaced and in terms of performances is quite impressive. However, the program has a complicate structure that could difficult at the first approach (but Manuel is very kind, don't hesitate to contact him ). Otherwise there is the code PrestoPronto. code.google.com/p/*prestopronto*/ It is mainly a fully graphic python wrapping of ifeffit for sequential analysis. In terms of speed is very similar to Athena but it do automatically the tricks of cleaning memory But it is still a very young code...... It still has not any interface to linear combination analysis. We never tested for more than 1000.. spectra Probably Mark Newton could give you some more suggestions Regards Lello On Sat, Mar 31, 2012 at 7:00 PM, <ifeffit-request(a)millenia.cars.aps.anl.gov>wrote: > Send Ifeffit mailing list submissions to > ifeffit(a)millenia.cars.aps.anl.gov > > To subscribe or unsubscribe via the World Wide Web, visit > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > or, via email, send a message with subject or body 'help' to > ifeffit-request(a)millenia.cars.aps.anl.gov > > You can reach the person managing the list at > ifeffit-owner(a)millenia.cars.aps.anl.gov > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Ifeffit digest..." > > > Today's Topics: > > 1. Re: Ifeffit Digest, Vol 109, Issue 21 (KONG Quingyu) > 2. Re: Ifeffit Digest, Vol 109, Issue 21 (Matt Newville) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 30 Mar 2012 18:32:49 +0000 > From: KONG Quingyu <kong(a)synchrotron-soleil.fr> > To: "ifeffit(a)millenia.cars.aps.anl.gov" > <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 21 > Message-ID: > < > 30F62EE734A9EA45A3F211513CB6200324BD1922(a)SUN-DAG1.synchrotron-soleil.f > r> > > Content-Type: text/plain; charset="iso-8859-1" > > Thank you for all your helpful suggestions. > > Actually, our beamline has recently installed a fast detector with a > repetation of 20KHz, which could collected data in a time resolution > of few tens of microsecond and could easily produce a large amount of > data. We have already accepted three users for microsecond time > resolved studies, each of them collected millions of spectra. > > Together with the company that provide the detector and our software > engineer we started to write a Python program to treat the data from > November last year, however our engineer quitted recently, this make > us to look for other solutions. > > If Demeter would works well for large amount of data treatment, it > would be great. As I know ID24 at ESRF has also installed the same > detector and would probably have similar demanding for such a program. > > Probably due to the limited memory of my pc, I can only open around > 200 spectra simultanously with Athena. > > Best, > Qingyu Kong > > > Beamline ODE > Soci?t? civile Synchrotron SOLEIL > L'Orme des Merisiers > Saint-Aubin - BP 48 > 91192 GIF-sur-YVETTE CEDEX > Tel: +33 (0)1 69 35 97 84 (office) > +33 (0)1 69 35 97 16 (beamline) > email: kong(a)synchrotron-soleil.fr > > ________________________________________ > From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [ > ifeffit-bounces(a)millenia.cars.aps.anl.gov] on behalf of > ifeffit-request(a)millenia.cars.aps.anl.gov [ > ifeffit-request(a)millenia.cars.aps.anl.gov] > Sent: Friday, March 30, 2012 3:29 PM > To: ifeffit(a)millenia.cars.aps.anl.gov > Subject: Ifeffit Digest, Vol 109, Issue 21 > > Send Ifeffit mailing list submissions to > ifeffit(a)millenia.cars.aps.anl.gov > > To subscribe or unsubscribe via the World Wide Web, visit > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > or, via email, send a message with subject or body 'help' to > ifeffit-request(a)millenia.cars.aps.anl.gov > > You can reach the person managing the list at > ifeffit-owner(a)millenia.cars.aps.anl.gov > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Ifeffit digest..." > > > Today's Topics: > > 1. delr and ss parameterization (Ebrahim Rezaei) > 2. program (KONG Quingyu) > 3. Re: program (Carlo Segre) > 4. Re: program (Matt Newville) > 5. Re: delr and ss parameterization (Bruce Ravel) > 6. Re: program (Bruce Ravel) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 29 Mar 2012 21:57:21 -0700 (PDT) > From: Ebrahim Rezaei <e_rezaei24(a)yahoo.com> > To: ifeffit(a)millenia.cars.aps.anl.gov > Subject: [Ifeffit] delr and ss parameterization > Message-ID: > > <1333083441.66190.YahooMailClassic(a)web120501.mail.ne1.yahoo.com> > Content-Type: text/plain; charset="utf-8" > > Hi everyone, > > > I am working with bulk and nano structures of MnO, MnO2 and Mn2O3. I > have two questions regarding how to parameterize delr and ss in > Artemis. > > > It is recommended to > use either delr=alfa*Reff or relate it to unit cell structure. Can > anybody tell me how to do the second approach? > > > Also is that a good idea to let all ss to float? Or it should be > constrained based on type and distance of atoms? I have seen an > example about ZnO just letting the first two shell?s ss to float and > let the rest to be the same. I would appreciate it very much if > anybody could help me with this too. > > > Regards, > > > Ebrahim Rezaei > > PhD Candidate, Chem. Eng. Dep., University of Saskatchewan > > Saskatoon, SK, Canada > -------------- next part -------------- An HTML attachment was > scrubbed... > URL: < > http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/2012032 > 9/e13d64a5/attachment-0001.htm > > > > ------------------------------ > > Message: 2 > Date: Fri, 30 Mar 2012 06:46:38 +0000 > From: KONG Quingyu <kong(a)synchrotron-soleil.fr> > To: "ifeffit(a)millenia.cars.aps.anl.gov" > <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: [Ifeffit] program > Message-ID: > < > 30F62EE734A9EA45A3F211513CB6200324BD1794(a)SUN-DAG1.synchrotron-soleil.f > r> > > Content-Type: text/plain; charset="iso-8859-1" > > We recently collected 3000 EXAFS spectra to follow a chemical reaction > process, we would like to fit the experimental data with reference > spectra using linear combination fitting. Does anyone know there are > available or commercial program to normalize the data and make linear > combination fitting. We tried to do it with Athena manually, but 3000 > spectra take a long time for us and it is not a practical way. > > Thanks a lot. > > Qingyu Kong > > Beamline ODE > Soci?t? civile Synchrotron SOLEIL > L'Orme des Merisiers > Saint-Aubin - BP 48 > 91192 GIF-sur-YVETTE CEDEX > Tel: +33 (0)1 69 35 97 84 (office) > +33 (0)1 69 35 97 16 (beamline) > email: kong(a)synchrotron-soleil.fr > > > > > > ------------------------------ > > Message: 3 > Date: Fri, 30 Mar 2012 01:56:57 -0500 (CDT) > From: Carlo Segre <segre(a)iit.edu> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] program > Message-ID: <alpine.DEB.2.02.1203300154590.5339(a)hydride.phys.iit.edu> > Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed > > > Hi Qingyu > > On Fri, 30 Mar 2012, KONG Quingyu wrote: > > > We recently collected 3000 EXAFS spectra to follow a chemical > > reaction process, we would like to fit the experimental data with > > reference spectra using linear combination fitting. Does anyone know > > there are available or commercial program to normalize the data and > > make linear combination fitting. We tried to do it with Athena > > manually, but 3000 spectra take a long time for us and it is not a practical way. > > You might wish to look into Bruce Ravel's new suite of programs > (called Demeter). I believe they are built to allow easy scripting of > massive quantities of data such as yours. > > Carlo > > > -- > Carlo U. Segre -- Duchossois Leadership Professor of Physics Associate > Dean for Graduate Admissions, Graduate College Illinois Institute of > Technology > Voice: 312.567.3498 Fax: 312.567.3494 > segre(a)iit.edu http://phys.iit.edu/~segre segre(a)debian.org > > > ------------------------------ > > Message: 4 > Date: Fri, 30 Mar 2012 07:56:05 -0500 > From: Matt Newville <newville(a)cars.uchicago.edu> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] program > Message-ID: > > <CA+7ESbrjp46_kPw9F803MQOGdMRtFbDa-7oGem25+CL2bYznJg(a)mail.gmail.com > > > Content-Type: text/plain; charset=ISO-8859-1 > > Hi Qingyu, > > On Fri, Mar 30, 2012 at 1:46 AM, KONG Quingyu > <kong(a)synchrotron-soleil.fr> wrote: > > We recently collected 3000 EXAFS spectra to follow a chemical > > reaction > process, we would like to fit the experimental data with reference > spectra using linear combination fitting. Does anyone know there are > available or commercial program to normalize the data and make linear > combination fitting. We tried to do it with Athena manually, but 3000 > spectra take a long time for us and it is not a practical way. > > Ifeffit is designed to be scriptable for batch processing. This can > be done with a raw Ifeffit script (which isn't very flexible) or from > Perl or Python, which are more flexible. Carlo mentioned Demeter, > which is a high-level Perl wrapping of the lower-level functionality. > Working with 3000 spectra may be challenging with Ifeffit version > 1.2, as the total amount of available memory is limited, but with some > care it should be possible. > > FWIW, the next version of Ifeffit will eliminate this memory > limitation, and make both scripting and extending the functionality > much easier and better. This is still a work in progress (see > https://github.com/xraypy/xraylarch) and poorly documented, but the > essential idea is to write all of Ifeffit's functionality in Python, > for all the reasons just stated, and this far enough along that I am > confident it will be a big improvement. So far (for XAFS), this can > read data, do simple pre-edge subtraction, normalization, remove > backgrounds ala autobk, and do XAFS Fourier transforms. It can do > simple "general purpose fitting", but does not yet expose a > simple-to-use linear combination fitting functionality (shouldn't be > too hard, as all the linear algebra pieces are readily available) or > fitting spectra to Feff calculations (a bit more involved). Work is > slow (mostly due to other obligations), but progress is being made, > and it's close to ready for an initial release. If you are (or > anyone else is) interested in trying this or helping out, let me know. > > --Matt > > > > ------------------------------ > > Message: 5 > Date: Fri, 30 Mar 2012 09:13:11 -0400 > From: Bruce Ravel <bravel(a)bnl.gov> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] delr and ss parameterization > Message-ID: <201203300913.12166.bravel(a)bnl.gov> > Content-Type: Text/Plain; charset="utf-8" > > On Friday, March 30, 2012 12:57:21 am Ebrahim Rezaei wrote: > > I am working with bulk and nano structures of MnO, MnO2 and Mn2O3. I > > have two questions regarding how to parameterize delr and ss in > > Artemis. > > > > > > It is recommended to > > use either delr=alfa*Reff or relate it to unit cell structure. Can > anybody > > tell me how to do the second approach? > > > > > > Also is that a good idea to let all ss to float? Or it should be > > constrained based on type and distance of atoms? I have seen an > example > > about ZnO just letting the first two shell?s ss to float and let the > > rest to be the same. I would appreciate it very much if anybody > > could help me with this too. > > The alpha*reff trick is only strictly valid for cubic, monoatomic > materials. For everything else in the world, it is an increasingly > poor approximation. That's not to say it is not a helpful > parameterization, just that you need to be mindful of its limitations. > > As for the sigma^2 parameter, you have to balance when is physically > reasonable with what can be reliably extracted from a real data > analysis. > > - Two atoms of a different species at the same distance should have > different sigma^2 because they have different masses. > > - Two atoms of the same species at different distances should have > different sigma^2 because sigma^2 is a mean square deviation in > distance. > > So, in some sense, all scatterers might have their own sigma^2 > parameters. But that is not to say that you data are sufficient to > independently determine that many parameters. When you run up against > the limited information content of your data, that is when you start > making approximations of the sort that lead to setting sigma^2 > parameters for different scatterers to be the same. > > However, there are sensible and physically justifiable reasons why the > sigma^2 for certain multiple scattering paths should be related to > those of single scattering paths. See > http://dx.doi.org/10.1103/PhysRevB.54.156 > > Two of my favorite papers that show excellent data analysis solving > real-world problems and which are written in a way that is of > pedagogical value for XAS beginners are Scott Calvin's paper on doped > ferrites (http://dx.doi.org/10.1103/PhysRevB.66.224405) and Shelly > Kelly's paper on uranyl binding to biomass > (http://dx.doi.org/10.1016/S0016-7037(02)00947-X) For more > interesting papers, look through the archives of the mailing list and > see who gives good answers to questions, then look up their papers. > > Another extremely useful resource which includes extensive discussion > of the sorts of things you are asking about is Shelly's review > chapter. Here's the amazon link to that book: > > http://www.amazon.com/Methods-Soil-Analysis-Part-Mineralogical/dp/0891 > 188460/ The book is a bit pricy, but if you are a clever googler, you > may find a PDF copy of her chapter. > > Regards, > B > > > -- > > Bruce Ravel ------------------------------------ bravel(a)bnl.gov > > National Institute of Standards and Technology Synchrotron Methods > Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, > 11973 > > My homepage: http://xafs.org/BruceRavel > EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter > > > > ------------------------------ > > Message: 6 > Date: Fri, 30 Mar 2012 09:29:10 -0400 > From: Bruce Ravel <bravel(a)bnl.gov> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] program > Message-ID: <201203300929.11143.bravel(a)bnl.gov> > Content-Type: Text/Plain; charset="iso-8859-1" > > On Friday, March 30, 2012 02:46:38 am KONG Quingyu wrote: > > We recently collected 3000 EXAFS spectra to follow a chemical > > reaction process, we would like to fit the experimental data with > > reference > spectra > > using linear combination fitting. Does anyone know there are > > available or commercial program to normalize the data and make > > linear combination fitting. We tried to do it with Athena manually, > > but 3000 spectra take a long time for us and it is not a practical way. > > Hi Qingyu, > > Carlo is correct, Demeter could be used to write a special purpose > program for automating this sort of thing. The performance in terms > of speed may not be exactly what you want, but the performance in > terms of quickly prototyping a program to process and analyze a large > data volume is excellent. > > The Demeter code base comes with extensive (although still incomplete) > documentation as well as plenty of example scripts, including examples > for the individual chores you mention above. > > Here's Demeter's documentation: http://bruceravel.github.com/demeter/ > > > As a closing comment, isn't it somehow strange that you and your > colleagues went to all the trouble of building a fancy dispersive > beamline without thinking ahead about how you were going to analyze > your data? > > B > > > -- > > Bruce Ravel ------------------------------------ bravel(a)bnl.gov > > National Institute of Standards and Technology Synchrotron Methods > Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, > 11973 > > My homepage: http://xafs.org/BruceRavel > EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter > > > ------------------------------ > > _______________________________________________ > Ifeffit mailing list > Ifeffit(a)millenia.cars.aps.anl.gov > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > > > End of Ifeffit Digest, Vol 109, Issue 21 > **************************************** > > > > ------------------------------ > > Message: 2 > Date: Fri, 30 Mar 2012 16:36:58 -0500 > From: Matt Newville <newville(a)cars.uchicago.edu> > To: XAFS Analysis using Ifeffit <ifeffit(a)millenia.cars.aps.anl.gov> > Subject: Re: [Ifeffit] Ifeffit Digest, Vol 109, Issue 21 > Message-ID: > > <CA+7ESbrZir3zMWAKSx5HPpqjJ5H_qC-V0YAsnCXUzDO512q+aA(a)mail.gmail.com > > > Content-Type: text/plain; charset=ISO-8859-1 > > Hi Qingyu, > > On Fri, Mar 30, 2012 at 1:32 PM, KONG Quingyu > <kong(a)synchrotron-soleil.fr> wrote: > > Thank you for all your helpful suggestions. > > > > Actually, our beamline has recently installed a fast detector with a > > repetation of 20KHz, which could collected data in a time resolution > > of few tens of microsecond and could easily produce a large amount > > of data. We have already accepted three users for microsecond time > > resolved studies, each of them collected millions of spectra. > > > > Together with the company that provide the detector and our software > > engineer we started to write a Python program to treat the data from > > November last year, however our engineer quitted recently, this make > > us to look for other solutions. > > Bruce wrote earlier, > > As a closing comment, isn't it somehow strange that you and your > colleagues went to all the trouble of building a fancy dispersive > beamline without thinking ahead about how you were going to analyze > your data? > > Sadly, Bruce's observation is not unusual at all. Everyone is > experiencing data overloads, and almost no one is investing in > computing resources. Yours may be slightly larger than most XAFS > users, but only by one or two orders of magnitude. Then again, you > had a software engineer at your beamline at least for a while. > > To be clear, Ifeffit, Athena, Demeter, Sixpack, and so on are written > primarily by beamline scientists like yourself. Your choices are to > use what someone else gives you, write your own, or hire someone to do > it for you. It appears that the last choice rarely works very well. > That you get to use Bruce's codes at all (and for free!) is not an > accident: it is part of the design, and an outcome of the programs > being written by beamline scientists. > > The current software base is more than ten years old, and showing its > age especially with respect to large data sets and multiple > processors. This is a really good opportunity for you and your > colleagues (and others in similar situations) to join in. That is, > you should write code to deal with your fast data rates. If you write > that with the Ifeffit family of programs, not only will people at ESRF > be able to use it, but so will anyone else. > > > If Demeter would works well for large amount of data treatment, it > > would be great. As I know ID24 at ESRF has also installed the same > > detector and would probably have similar demanding for such a program. > > Looking into Demeter is one option, though I don't know exactly that > is perfect for the linear combination analysis you want to use. Most > of the core algorithms in Demeter still use Ifeffit 1.2, which are > going to be limited for your needs, though they can be worked around > if careful. Transferring Demeter to use the newer Ifeffit2 should be > possible, but neither Bruce or I have done more than > proof-of-principle tests. Looking into using Larch / IFeffit2 > might be better for you, and I'd be happy to help you do that. For > one thing, this approach has no memory limitations beyond your > machine, is in Python and can read HDF5 files (which SOLEIL uses, as > far as I understand) directly. It also has the possibility of using > multicore machines well, but this has not yet been automated. It is > also possible that the linear algebra techniques used for x-ray > fluorescence map analysis, including code from Gerd Wellenreuther and > Armando Sole would be useful for your needs. > > > Probably due to the limited memory of my pc, I can only open around > > 200 spectra simultanously with Athena. > > You almost certainly want to not use Athena here anyway. You want > automated processing programs. > > Like Bruce hinted at, you've invested heavily in a detector that > allows you to collect data at a rate orders of magnitude faster than > you could before. It would have been incredibly naive to believe that > decade-old software would have been able to keep up with that. This > is a good opportunity for you to help out yourself and others. > > Cheers, > > --Matt > > > ------------------------------ > > _______________________________________________ > Ifeffit mailing list > Ifeffit(a)millenia.cars.aps.anl.gov > http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit > > > End of Ifeffit Digest, Vol 109, Issue 23 > **************************************** > -- Carmelo Prestipino Sciences Chimiques de Rennes UMR 6226 Solid State and Materials Chemistry group (CSM) Campus de Beaulieu, B?t 10B, bureau 122 F-35042 RENNES France Tel: (+33) 22323 6531 FAX: (+33) 22323 5959

1 0