- Ifeffit - millenia.cars.aps.anl.gov

Re: [Ifeffit] Question about statistics
by Olga Kashurnikova 08 Mar '15

08 Mar '15

Hello, Dr Matt Newville, Thank you very much for the answer. There is a message with some corrections in the next thread - the numbers of my hand try to check am I right about chi_square and error calculation in IFEFFIT, were not right in the first time. I will try to attach the calculation files later, it is not clear for me how they will be shown here. I will write the main question and then about Bayesian analysis for thread splitting if you don't mind. These things are interconnected that's why I didn't split them. The main question was about how IFEFFIT calculates chi_square and to what minimization function it adds restraints, in case of k_space. What is the normalization? I thought it should be (N_idp/N_data)sum[(dat-model)^2/eps^2], and in k space only the real part of data compared with imaginary part of model, but my hand calculation for check this gave the different result. It is necessary for the choice of optimal restraint (it is chi^2 without normalization that I need to compare) and for normalization of errors for them to be covariance matrix of this chi^2, as of Krappe and Rossner. Restraint will change errors, of course, because covariance matrix is (Q+A)^(-1) where A is regularizer in restraint and Q is what should be without restraint. If Q is not invertible (in case of some parametra don't influence spectrum) it is critical, and optimal A can be found to strictly divide the 'parametra space' and verify models. That's why I need to rescale them well, because for bayes it should be chi^2=sum[(dat-model)^2/eps^2]+A*sum(x-x0)**2 (A is connected with a priori ranges of parametra, it should be not one number, but algorithm of finding optimal diagonal A matrix is harder and I think not with IFEFFIT/Larch help). It is the same as your (paramA-A_0)/eps_A restraint, I understood what formula should be for a restraint, but don't understand how to weight data and restraint part. It were Krappe and Rossner who mentioned Tikhonov regularization if I remember right, and it is very close, if I understood the paper about this regularization. That's what I mean in 'doesn't fit': 4 coordination spheres (Gd-O, Gd-Gd,Gd-Hf, next Gd-O for instance, and only first is in separate peak) may be in the N_idp range, but are so correlated that without constraints or other structure model the fit give bad results - some parametra will leave the acceptable range etc, and I'm not sure that without a Bayesian analysis I can define the better model of constraints, and the sphere splitting is even more difficult to define. So, the main question is 'how IFEFFIT calculates this, what is the formula', because FEFFIT manual doesn't give the k-space case, and the check was wrong. I can append calculation files in a while if needed (I need to convert from a program to iff file), though I wrote formulae and they may be can be verified without numerical values. Chi_square in your method is close to what should be minimized and for what the covariance matrix should be found in Bayesian analysis, with the right normalization, that's why I ask if I could use IFEFFIT to calculate this with renormalization of the results. If I put the right normalization to restraint, I need only rescale chi_square and errors and then calculate what I need. It is not that I think IFEFFIT use Bayesian statistics, I understand there are different approaches. Thankful for your assistance, Olga Kashurnikova, MEPhI, Moscow

3 7

Question about statistics
by Ольга Кашурникова 06 Mar '15

06 Mar '15

Good day to developers and users of Ifeffit, I'd like to get help with my question of statistical calculations in Ifeffit (I use the version embedded in Demeter, but in console mode or from my Fortran little program: think of Larch using but there are no some features and I'm not sure). The task is to try making Bayes statistics, in the sense that was published in Krappe et al. papers from 1999,2002,2004 years and the book of Bretthorst. I haven't found the software for it and my own tries to write it are not completed because of no experience. I saw something in papers about BFEFFIT, an extension for such statistics, but haven't found it in the web or any other information. May be somebody knows what stage is it on. I need it because of complicate model (4 spheres, may be splitting of spheres because it is many-phase system with dopant phase) and need of regularization in Tikhonov sense (parametra are correlated and I can not determine which constraint or model is better without statistics, and with nonconstrained parametra it simply will not fit) I understood that it can be emulated in Ifeffit with help of restraints. All thing is to: -minimize chi**2=(dat-model)**2/epsilon_k**2+A*sum(xi-x0i)**2, where A is regularizer, xi - parametra in fit end, x0i - a priori values of parametra. -calculate pair correlations and errors so that we can calculate the second derivatives and variance matrix of chi**2 determines in this sence. But I can not understand about how restraint goes in fit and how IFEFFIT calculates chi_square (not reduced) and errors. From manuals of FEFFIT and IFEFFIT I supposed that there goes: -LS minimisation of (dat-model)**2+restraint**2, may be normed, but it doesn't matter -chi_square is this value divided by eps_k**2, may be divided by points num -for errors found variance of chi_square defined above, but then error bars are scaled to chi_reduced to eliminate noise dependence -pair correlations are corr(i,j)/sqrt(corr(i,i)*corr(j,j) and doesn't depend of normalisations What is right? I need the above defined chi**2=(dat-model)**2/epsilon_k**2+A*sum(xi-x0i)**2 and its variance matrix, because it defines my statistics, optimal A,errors and 'projections' of parametra that I need to verify model. I use the fit in k space (not q) without Fourier transforms or k weight, because it is hard to do for this statistics. (Filtration of high fr3equency components I can do with rectangle windows if needed, I think, but it is not much bigger than noise) I thought (dat-model)**2 in k space is in real part for points from kmax to kmin for defined dat array(my_spectrum.chi for instance) and fit.chi array. But I calculated chi_square in hand from these arrays and there was no right number. for test of 1 sphere chi_square=8e-3, (dat-model)**2=3e-4, (dat-model)**2/noise**2=2,37e1, (dat-model)**2/noise**2=9.5e-2 and so on (regularizer part A(x-x0)**2=0.91). For instance. But if it calculates in real and imaginary part of Fourier transform no meaning what space I use - and this function is used to find variance - then I'm not sure how to find error bars and variance matrix I need. There are many other questions and limitations, I want to make this calculation approximately with what I can (constaint noise, may be +bkg part as Ifeffit gives, two or more spectra - and the other things later), but I hope someone could help the beginner to solve this problem or find the simplest way in existing software... I need to know what is the stage of this topic and need I end my program for it or not. There in mind are such things as mu0 polynome fit, many-spectra fit with PDF and diffraction spectra for model and so on, but may be there is the best program in one step from completeness and I need not to do it. Thanks sincerely, Olga Kashurnikova (from MePHI XAFS group in Moscow)

3 3

calculation of statistics-2
by Ольга Кашурникова 06 Mar '15

06 Mar '15

Good afternoon or other time of day again, I don't know how to add message to the thread (I use online mail service), so will write additions here, sorry, may be somebody will correct me. The chi_square definition in FEFFIT manual is chi_square=N_idp/N_data*sum((dat-model)^2/epsilon_k^2), and to this sum IFEFFIT adds restraint^2 (as of source code). But with 0 restraint I didn't get the right value. If there is 2*N_data, I get the right order, but not exactly the same value (0.6 and 0.8 for instance). chi_reduced=chi_square/(N_idp-N_param) by definition, and it is exact. (N_idp=18, N_param=4 in my case, eps_k=3.6e-3 from FT high-frequency compared with mu0 calculation in real spectrum. I'm not sure about small values, but the test is generated oxygen sphere without noise, so only systematic errors. I will correct it anyhow, but can it be a problem I'm not sure) I found that I need to set Rmin and Rmax for N_idp calculus even if I fit in k space, but it doesn't help. The adding of real part of fit chi (that exists, I don't know why) with 0 data made things only worse. I try to find maths in source code, but it is big, and I can not find all definitions and understand quickly what is the beginning and what is the chain of calls I need. Are files in feffit and lib directories used together? That seems that the fit vector that is LS-fitted is (dat-model)/weight and restraint components, but what of weight. In fitnls.f file there are weight(id) = sigdtk(id) * sigdtk(id) / sigwgt(id) !in k space case. Is it epsilon_k**2 if user-defined epsilon_k? what is this normalization? weight(id) = sqrt ( nptfit(id) * weight(id) / xinfo(id)) !nptfit=N_data without 2 from kmin to kmax? xinfo=N_idp ifeffit variable? !We sum chi_square in m_fit points, is it the same? in fft file there was 2*N_data even in k space? So can I think that weight=sqrt(N_data/N_idp)*epsilon_k, or not? I didn't get the chi_square right with this formula. Also, dat-model is multiplied by window, but I make Hanning window with dk=0 and it doesn't help. It is the main question I wish to find the answer now, because need to make this calculus to publish, for results in my compounds to be confident. I thought of suggesting that chi_square is different only by norm and restraint is added to chi_square and to calculate and renormalize errors and A parameter in restraint for calculation. But it can be a problem if is not proportional, and I'm not sure if A renormalization can help... The question of extending the method can wait a while. I will be very thankful if somebody could help... Thanks sincerely, O.V. Kashurnikova, XAFS group of the MEPhI, Moscow (Moscow Engeneering-Physics Institute)

1 0

Artemis error - xdi is not writable (0.9.21, Windows 8)
by Godfrey, Ian 05 Mar '15

05 Mar '15

Dear All, I've encountered a problem in Artemis 0.9.21 (x64 running under Win 8.1 Pro x64) when I try and load a project file to which I've previously saved changes using this version of Artemis. When double clicking the file to open it the programme begins to open as normal, until it reaches "importing paths" - at which point it simply closes. The log file (attached) shows an error that "xdifile is not writable, no setter defined at...". Opening files saved in older versions of Artemis seems fine though. I've attached a log file and a problematic project file. Regards, Ian

2 3

Two plotting related glitches in Athena
by Godfrey, Ian 04 Mar '15

04 Mar '15

?Dear All, I encountered two (apparent) bugs related to plotting data in Athena. In both cases I think they're best expressed via video screencaps so please see the links with each bug report (both of these links are valid for 10 days only). Both of these are occurring in Athena 0.9.21 x64 on Windows 8.1 Enterprise x64. An Athena log file is attached. 1) Sometimes lines on a plot disappear when zooming/scrolling. This typically happens with steeper lines. In the video you can see it occurring when I scroll up and down - sometimes it also occurs immediately when I try and zoom in. (I guess this could be an issue with GNUPlot, rather than Athena?) Video: https://wwwa-e.ucl.ac.uk/cgi-bin/dropbox/dropbox.cgi?state=pickup_info&id=1… Password: 2725d459 2) When using the pluck function to set indicators (possibly in other situations too, although I haven't noticed it there) if you don't double click to pluck the point fast enough after clicking the pluck button the software plucks a nonsense value instead. (The halo around the cursor is supposed to show when I click, but it seems to show up when I leave the cursor stationary and at some other times too, so don't rely too much on that... when I don't double click the point I've left the cursor entirely off the Athena windows). (Sometimes when this happens it seems that you need to replot the graph to be able to pluck again, but I can't reliably reproduce that behaviour.) Video: https://wwwa-e.ucl.ac.uk/cgi-bin/dropbox/dropbox.cgi?state=pickup_info&id=3… Password: 2725d459 Regards, Ian ---- Ian Godfrey PhD Student, UCL/JAIST Programme Industrial Doctorate Centre in Molecular Modelling and Materials Science, Department of Chemistry, University College London and School of Materials Science, Japan Advanced Institute of Science and Technology i.godfrey(a)ucl.ac.uk<mailto:i.godfrey@ucl.ac.uk> i.godfrey(a)jaist.ac.jp<mailto:i.godfrey@jaist.ac.jp>

2 2

International EXAFS Conference XAFS16
by Lichtenberg, Henning (IKFT) 04 Mar '15

04 Mar '15

Abstract Deadline for the International XAFS Conference in Karlsruhe (www.xafs16.org<http://www.xafs16.org>) extended until March 15,2015 Dear colleagues of earlier XAFS conferences Good news about the XAFS16 conference, which will take place in Karlsruhe/Germany 23 – 28 August 2015: More than 300 abstracts have arrived in our system during the last 3 days just before the deadline. Most popular at the moment are topics like EXAFS in materials science, instrumentation, new spectroscopic methods, energy-related materials and catalysis (please refer to the site: http://www.xafs16.org) The strength of the conference is that theory and instrumentation on the one hand and applications on the other hand talk to each other! Many have asked us about the extension of the deadline so that we have prolonged the abstract submission deadline until March 15, 2015. Please submit your abstract in time! The reviewing process will start immediately afterwards. Later abstracts may be considered as poster but this also depends on whether we still have enough space. During the next days we will also update the plenary and keynote speaker list. So please have a glance on the homepage www.xafs16.org<http://www.xafs16.org> from time to time. Special sessions like an industrial session as well as 3 satellite meetings are planned. We already now look forward to a successful conference and look forward to contributions in all 16 topics. Best regards, in the name of the IXAS society and the local organizing committee. Jan-Dierk Grunwaldt Chairman XAFS16 [cid:image001.jpg@01D0553A.FD9D8F50]<http://www.xafs16.org/> --------------------------------------------------------------------- Prof. Dr. Jan-Dierk Grunwaldt Chair in Chemical Technology and Catalysis grunwaldt(a)kit.edu<mailto:grunwaldt@kit.edu>; www.itcp.kit.edu/grunwaldt/<http://www.itcp.kit.edu/grunwaldt/> Tel.: +49 721 608-42120, Fax: -44805, Sekr. Tel.: -42121, -43064 Karlsruhe Institute of Technology (KIT) Director at the Institute for Chemical Technology and Polymer Chemistry (ITCP) Engesserstr. 20, 76131 Karlsruhe, Germany Director at the Institute for Catalysis Research and Technology (IKFT) Herman-Helmholtz-Platz 1, 76344 Eggenstein-Leopoldshafen, Germany KIT - Universität des Landes Baden-Württemberg und nationales Großforschungszentrum in der Helmholtz-Gemeinschaft

1 0

problem plotting in Athena after installation of Demeter 0.9.21 on Windows 64bit
by kirshstein＠gmail.com 03 Mar '15

03 Mar '15

Hello, I have just installed Demeter and the first thing Athena does, when asked to plot (mu(E)) of an averaged standard spectrum (S-K-edge of CaSO4, averaged in sixpack), is to give the following error message: "Use of uninitialized value in multiplication(*) at C:/strawberry/perl/site/lib/Math/Spline.pm line 36." Also E0 is 0 and in front of a red background. So it seems to have trouble finding E0. But why? What does this mean? What do I have to do to get athena to work? attached is the CaSO4 spectrum in question. Thanks! Yael Kirsch

2 2

Feature request - Hephaestus: Plot multiple standards at once
by Godfrey, Ian 02 Mar '15

02 Mar '15

Dear All, I'm using Hephaestus' standards library to help with some XANES fingerprinting and it's very useful. I would find it quite helpful to be able to select two (or more) standards from the list and plot them on the same graph for comparison. Regards, Ian ---- Ian Godfrey PhD Student, UCL/JAIST Programme Industrial Doctorate Centre in Molecular Modelling and Materials Science, Department of Chemistry, University College London and School of Materials Science, Japan Advanced Institute of Science and Technology i.godfrey(a)ucl.ac.uk<mailto:i.godfrey@ucl.ac.uk> i.godfrey(a)jaist.ac.jp<mailto:i.godfrey@jaist.ac.jp>

2 5

Pb compound determination fitting issue
by Pieter Tack 25 Feb '15

25 Feb '15

Hi everyone, I recently acquired an EXAFS data set (albeit somewhat noisy) on the Pb-L3 edge. I have normalized it using Athena, and tried to fit the first shell using Artemis. I attached the Athena project file as well as the Artemis project file of what I obtained thus far. We have reason to believe the unknown compound is PbO or PbS (or a mixture). However, when I look at the R space curve, the first coordination shell is at rather low path distance (1.7 A), whereas one would expect larger bond distances for PbO and PbS. I found that a similar bond distance as to what the experimental spectrum shows can be found in PbCO3, but a fit using the mineral data of PbCO3 also doesn't do the trick. In any case, no matter what I try to fit in Artemis, nothing seems decent. Is the EXAFS fitting simply impossible due to the noisy data, did I extract my Chi(k) wrong or am I encountering some other issue? I realize it's probably not much to go on, but I'm rather new to EXAFS analysis so any help/advice is appreciated. Thanks a lot in advance! Best regards, Pieter Tack -- X-Ray Micro-Spectroscopy and Imaging Group Department of Analytical Chemistry Ghent University Krijgslaan 281 S12 B-9000 Ghent Belgium Phone: +32 (0)9 264 4723 Fax: +32 (0)9 264 49 60

2 2

Re: [Ifeffit] Pb compound determination fitting issue
by fred.mosselmans＠diamond.ac.uk 25 Feb '15

25 Feb '15

-----Original Message----- From: ifeffit-bounces(a)millenia.cars.aps.anl.gov [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of ifeffit-request(a)millenia.cars.aps.anl.gov Sent: 25 February 2015 10:48 To: ifeffit(a)millenia.cars.aps.anl.gov Subject: Ifeffit Digest, Vol 144, Issue 11 Hi Pieter Apologies for confusing you, By 1 and 3 I meant fit 3 O, fit 5 S in the history. I had to fix S02 as the data range gives only 3 -4 paras and it flew off initially. Sorry Fred ---------------------------------------------------------------------- Message: 1 Date: Wed, 25 Feb 2015 10:45:15 +0000 From: <fred.mosselmans(a)diamond.ac.uk> To: <ifeffit(a)millenia.cars.aps.anl.gov> Subject: Re: [Ifeffit] Pb compound determination fitting issue Message-ID: <CF65047C5B0562499EAFF9FBBC542D6485DF0819(a)EXCHMBX01.fed.cclrc.ac.uk> Content-Type: text/plain; charset="us-ascii" Dear Pieter, You wouldn't start with Pb exafs if you had the choice. Pb has a lot of structural disorder in its compounds and is very thermally wobbly leading to poor signal to noise data. I think your dataset is only good to 7 -8 inverse A. That said I think you can learn something about the sample. Was the sample cold? eg in a cryostat when measured this is about the only way to get OK Pb data from anything but a solid state system with ordered atoms. In fit 1 (see the history) in the attached project file I use the first shell of massicot and this doesn't give a good fit but nothing ridiculous , in fit 3 I sued the cerrusite 1st shell and e0 goes along way and the first shell distance contracts a lot. This suggests you are likely have O in the first shell and by playing around more, e.g. Shortening the data range in Athena to improve the data reduction you should be able to get a better fit than with just one button press, however the useable data range is small so it will be not great In accuracy terms. Also you might try comparing the XANES to literature data though the core hole size in Pb L3 washes out xanes a bit so its not the best. Regards Fred ---------------------------------------------------------------------- Message: 1 Date: Wed, 25 Feb 2015 11:20:17 +0100 From: Pieter Tack <Pieter.Tack(a)ugent.be> To: ifeffit(a)millenia.cars.aps.anl.gov Subject: [Ifeffit] Pb compound determination fitting issue Message-ID: <20150225112017.Horde.GS9lWdD6UQ5U7aHhPZgmRhA(a)webmail.ugent.be> Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"; DelSp="Yes" Hi everyone, I recently acquired an EXAFS data set (albeit somewhat noisy) on the Pb-L3 edge. I have normalized it using Athena, and tried to fit the first shell using Artemis. I attached the Athena project file as well as the Artemis project file of what I obtained thus far. We have reason to believe the unknown compound is PbO or PbS (or a mixture). However, when I look at the R space curve, the first coordination shell is at rather low path distance (1.7 A), whereas one would expect larger bond distances for PbO and PbS. I found that a similar bond distance as to what the experimental spectrum shows can be found in PbCO3, but a fit using the mineral data of PbCO3 also doesn't do the trick. In any case, no matter what I try to fit in Artemis, nothing seems decent. Is the EXAFS fitting simply impossible due to the noisy data, did I extract my Chi(k) wrong or am I encountering some other issue? I realize it's probably not much to go on, but I'm rather new to EXAFS analysis so any help/advice is appreciated. Thanks a lot in advance! Best regards, Pieter Tack -- X-Ray Micro-Spectroscopy and Imaging Group Department of Analytical Chemistry Ghent University Krijgslaan 281 S12 B-9000 Ghent Belgium Phone: +32 (0)9 264 4723 Fax: +32 (0)9 264 49 60

1 0