- Ifeffit - millenia.cars.aps.anl.gov

continuous scan data
by Matt Newville 28 Jul '02

28 Jul '02

Hi Everyone, I made some small, probably incomplete, progress on dealing with the problem of continuous scan (or QEXAFS) XAFS data for which the XAFS data is sampled on a fine energy grid -- 1eV was used for the data Raul posted. As mentioned earlier, Ifeffit currently uses a fixed k-grid of 0.05Ang^-1 for chi(k). At k = 10Ang^-1, this corresponds to energy steps of nearly 4eV. Any scheme to use "all the data" to map to chi(k=10) may be tricky. The basic approaches are 1) better interpolation algorithm. 2) change the k-space grid size. 3) doing some sort of averaging of the data around a particular k value. At this point I've tried the first 2 of these. More details are posted at http://cars9.uchicago.edu/ifeffit/qexafs/ including data files and some plots figures. This is all with 'experimental' versions of ifeffit. First, I made the interpolation scheme used for the translation of chi(E) to chi(k) in the spline command have a few options: 'linear' using two surroundings data points 'quadratic' or parabolic using three data points 'cubic spline' put a cubic spline through all the data points Linear interpolation is not very satisfying. The results from 'quadratic' and 'cubic spline' are very similar. The next thing I did was to change the k-grid spacing from 0.050 to 0.025 and to 0.010 Ang^-1, and repeated the use all 3 interpolation schemes. Using 0.050 is noticeably different from 0.025, while 0.025 and 0.010 seem pretty similar to me. This was all done by changing the kgrid at compile time, which will be possible in 1.0073. I also played around with trying to make the kgrid and size of the FFT array variables that can be set at run-time. This seems highly desirable for cases like this. Surprisingly to me, it also seems possible. I have a *highly* experimental version that allow kgrid to be set at run time. In favorable conditions, it even works correctly. There are still some severe bugs with this version that I do not fully understand, and I doubt I'll have much time to work on it soon. But if anyone is very eager to try it, I'd be willing to send it. I have not yet tried implementing any 'averaging of adjacent points' such based on a rolling boxcar or a convolution with a Lorenztian. I could be persuaded to try such things, but at this point I'm not sure what the right thing is. Finally, coming back to the interpolation of the continuous scan data, the data Raul sent has some systematic effects going on that might complicate these comparisons. The effects can be seen from the un-interpolated chi(k) data: http://cars9.uchicago.edu/ifeffit/qexafs/orig_data_zoom.png and the attempts to interpolate this data: http://cars9.uchicago.edu/ifeffit/qexafs/kgrid_zoom.png It looks like there are systematic trends due to insufficient precision (only 4 digits in the mu(E) data). I don't know enough about the electronics used to know if this is intrinsic to the measurements or not. Also, does anyone know how accurate the energies are with this kind of data? Thanks, --Matt

1 0

Re: GIFeffit bugs
by Matt Newville 28 Jul '02

28 Jul '02

Hi Raul, On Fri, 26 Jul 2002, Raul Barrea wrote: > I was working with Gifeffit for windows and I save a session > file as ifeffit.sav but I couldn't open it again. Could you > please check it to see what can be wrong? Yes, GIFEFFIT is not handling the opening of files in directories with spaces in their name. It appears that save files are written correctly and can be read from a directory without a space in the name, but not if there is a space in the name. Should be easy to fix. > I found that log(my_logfile.log) is not working properly in > the windows version of GIfeffit. The program literally > crashes. A visual fortran run time error window appears. This > is not a serious bug but I think you must know it. I think this is related to all the other problems that result in a Visual Fortran Error of the form: forttl: severe (38): error during write, unit -1, file CONOUT$ .... which are come from GUIs that have no DOS console associated with them for simple print statements to be sent. These messages should not be necessary, but making sure that they never happen is harder. A solution should be fairly easy. I hope to post a version with fixes for both of these problems later today or tomorrow. > By the way, have you had any time to find a solution to the > interpolation routine to include all the statistics available > in the experimental data? I've made some progress, but not a complete solution. I'll send a separate post on this later today. --Matt

1 0

Re: [Ifeffit] athena / artemis
by Bruce Ravel 25 Jul '02

25 Jul '02

Hi folks, On Monday I was rushing to get updated versions of Athena and Artemis finished so I could go on a two day trip to NSLS on Tuesday and Wednesday. In my rush, I posted broken versions of Athena and Artemis on my web site. Upon my return, I found my mailbox overflowing with reports of problems. For the Windows users: I belive that Matt correctly identified the problem. I, of course, installed the logos for the splashscreens on *my* computer, but forgot that no one else would have them installed on theirs. That's none too bright! Matt sent me personal email while I was gone stating that he is almost ready to release 1.0073. So here is what I propose. I'll build new executables that do not use the splashscreens and wait for 1.0073 to release that feature. In the future, I'll try very hard to remember to post any new files that need to go into the distribution. Thanks to Mark Jensen, Raul Barrea, and Matt for downloading the executables and reporting back with their problems. For the Unix users: The splashscreens appear to be a problem for unix types as well. The code which handles the splashscreen makes use of a perl module that I had thought was part of the standard distribution but which, in fact, is not. The solution is simple. I'll include the missing module in the source tarballs for athena and artemis and post the new tarballs later today. Thanks to Sher Alam for pointing out this problem. I'll send mail out to the list later today when the new executables and tarballs are in place. Once again, thanks to all of you for taking the time to provide feedback. Your help is very much appreciated. B -- Bruce Ravel ----------------------------------- ravel(a)phys.washington.edu Code 6134, Building 3, Room 222 Naval Research Laboratory phone: (1) 202 767 5947 Washington DC 20375, USA fax: (1) 202 767 1697 NRL Synchrotron Radiation Consortium (NRL-SRC) Beamlines X11a, X11b, X23b, X24c, U4b National Synchrotron Light Source Brookhaven National Laboratory, Upton, NY 11973 My homepage: http://feff.phys.washington.edu/~ravel EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/

1 0

Re: FT padding
by Matt Newville 23 Jul '02

23 Jul '02

Dear Marcus, Recently, Marcus Karolewski wrote: > Dear Matt, > > I have been looking thro' your (highly informative) PhD thesis today in the > hope of finding some discussion about the zero padding used in FT > operations on XAFS data, but there doesn't seem to be any direct reference > to the reasoning behind it, although I gather you padded your data up to 25 > A-1. > > I realise that FT padding makes the data look nice in R-space, but I'd like > to ask whether you know of any other benefit to be gained from it. To my > way of thinking, there seems to be a definite drawback to padding insofar > as one ends up with data that consist mostly of interpolated points in R > space. This would seem to make a mockery of the fitting criteria which > strictly should be based on the actual number of points justified by the > range of the expl data. I would be very interested to hear your comments if > you can find the time. Excellent question! I hope it's OK if I CC this to the ifeffit mailing list, as it's relevant to a current discussion there on interpolating QEXAFS data, and because others might be interested or have opinions on this as well. Short answer: Yes, zero-padding is mostly to make the data look nice in R-space. A mockery of the fitting criteria?? I wouldn't go that far! Just for reference, the important values for the FTs are: kgrid: the k-grid spacing needs to get 'all the features'. N_fft: the FFT should use a number of points = a power of 2, even if zero-padding is used. rmax: the largest frequency that can be seen. rgrid: the resulting grid spacing in R should be small enough to avoid missing real features in R-space. Ifeffit uses kgrid=0.05Ang^-1 and zero-pads chi(k) to have N_fft=2048. The outputs from feffit also use these values. rmax and kgrid are simply related: rmax = pi /(2* kgrid) ~= 31.4 Ang for kgrid=0.05 The spectra above Rmax are a mirror image of the lower-R portion of the spectra: chi(R) = chi(2*Rmax-R) for R > Rmax 31.4Ang is much further than the XAFS really extends, but we can use the high portion to estimate the noise in the signal (feffit does this). So, we probably wouldn't want to go to kgrid=0.10 & rmax=16Ang. In fact, we might be tempted to use a smaller kgrid. The spacing of the R-space data, rgrid, is given as rgrid = pi / (kgrid * N_fft) For N_fft=2048 and kgrid=0.05Ang^-1, rgrid~=0.03Ang. If we had N_fft = 512, we'd have kmax = 0.05 * 512 = 25.6Ang-1 which is probably reasonable -- I've seen a few data sets past 25Ang^-1, but these are rare. We'd also have rgrid = pi / (0.05 * 512 ) = 0.122 Ang This has the advantages of giving data points in R that is closer to the 'real spacing' of "pairs of independent data points" in R-space: Delta R ~= pi / (K-range). The emphasis here should be on the 'approximate' sign. It is tempting to interpret this formula as "my independent data points are exactly 0.XXX Ang apart", but this is a serious overstatement. A better interpretation is that distances separated by less than (2pi/K-range) will be very difficult to distinguish. Using a coarser R spacing does make the fits faster. In fact, feffit (the program) does NOT use N_fft = 2048 when doing the fits -- it uses either 256, 512, 1024, or 2048 depending on the data k-range. This does indeed speed up the fits (for all output arrays, N_fft=2048 is used to make the plots prettier). Long ago (when computers ran on diesel), N_fft=128 was also allowed, but this was not reliable because there was a good chance of getting stuck in a 'false minima' with data that sparsely spaced in R. I've seen such false minima happen a few times, and took out the option of using N_fft = 128. Because it's painful to constantly switch N_fft, ifeffit currently uses kgrid=0.05 and N_fft = 2048 throughout. I have started working toward having these be changeable at will inside the program, but this still causes a few problems at the moment. Anyway, I wouldn't say that using zero padding and a large N_fft is really cheating much. It's really just oversampling the data. An FT is linear, so zero-padding does not change your data (much), but just convolves the 'independent data points' with Gaussian functions. Finally, zero-padding does change your data some because it means that you are asserting that chi(k)=0 for k>kmax. A better approach would be to pad with random noise at a level consistent with your data. This could be done, I suppose, but has not been done yet. Hope that helps, --Matt

1 0

serious athena bug fix
by Bruce Ravel 22 Jul '02

22 Jul '02

Hi folks, Today Shelly Kelly pointed out a serious bug in the code that merges spectra. Spectra merged as mu(E) or norm(E) have not ever been calculated correctly. There were several problems -- the worst was that the e0 shifts for all groups involved in the merge were ignored. This resulted in numerous problems with the merged spectra. I think I have fixed the problem -- at least the sample data that Shelly gave me now gets merged correctly. There is a new source tarball on my web site. I also added a couple of trivial features to both athena and artemis. The first is a splash-screen, i.e. something to look at in those endless seconds between double clicking on the icon and having the program spring to life on the screen. I also added several keyboard shortcuts for common tasks in both programs. You can see what these are by examining the menus in the menubar. I also added something new to my web site. Athena and Artemis change much faster than Ifeffit these days. The bad part of this is that Windows users don't get to take advantage of my updates as often as unix and OSX users. So today I added a section to the downloads page with links to the executables of my four programs. I doubt I will update the executables as often as I update the source tarballs. But when I do, I'll post the executables on the downloads page. The idea of these individual executables is that you replace the executable in the C:\Program Files\Ifeffit\ folder with the new one. These executables now found on my downloads WILL NOT RUN CORRECTLY outside the context of a properly installed ifeffit package. But if you have installed Ifeffit using the installer from Matt's web page, you will be able to drop my new executables in place. Regards, B -- Bruce Ravel ----------------------------------- ravel(a)phys.washington.edu Code 6134, Building 3, Room 222 Naval Research Laboratory phone: (1) 202 767 5947 Washington DC 20375, USA fax: (1) 202 767 1697 NRL Synchrotron Radiation Consortium (NRL-SRC) Beamlines X11a, X11b, X23b, X24c, U4b National Synchrotron Light Source Brookhaven National Laboratory, Upton, NY 11973 My homepage: http://feff.phys.washington.edu/~ravel EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/

1 0

Re: continuous scan data
by Webb, Samuel M. 19 Jul '02

19 Jul '02

Matt -- What we've used at DND-CAT is a box-car type averaging to grid the data. Basically, impose a 0.05 A-1 grid and bin all points that lie between a starting and ending point of each point on the grid. Of course, at DND we have data that has been collected at 12.5 kHz, so there are plenty of points to average and get statistical information with... Sam

1 0

Re: Ifeffit digest, Vol 1 #38 - 6 msgs
by Raul Barrea 18 Jul '02

18 Jul '02

Hi Matt, first of all I'd like to say that I have been using Iffefit and Athena for a while and I found both packages very easy to learn and use. I've been following the discussion about how Ifeffit and also athena manage continuous scans and finer grid in energy space. We work in continuous scan mode at BioCAT, so our energy grid cannot be change during the scan as step scans usually do and our energy grid is aprox 1 eV per point, so when we transform this grid to k space we get a not even grid and for most of our point the grid is really smaller than 0.05 A-1. I was very curious about the statistics and how ifeffit and Athena calculate the interpolation. I took one of our scans taken at the Cu K Edge and I remove every other point from it. In order to preserve the grid in the very first region of K I just substitued the high K region from 6 to 13 A-1 with this "new" grid. I found that the results of the original data and the modified data are exactly the same, so the interpolation is not considering the missing points. I've also removed a group of three consecutive points of a goupr of five from the grid (in the same k region) and I found the same results. It will be very usefull if you could include into your package other method that consider every measured point. As you said in your last mail working with finer grid could be a non trivial change but if you could include a better interpolation method will be great. Attached you will fine the three files with the original data and the modified ones (senc-full.xmu; senc-1.xmu and senc-3.xmu). Thanks Raul Raul A. Barrea, PhD Senior Research Associate Illinois Institute of Technology 7900 S Cass Av. BioCAT/APS, Bldg 435B Argonne, IL 60439

2 1

Re: continuous scan data
by Webb, Samuel M. 18 Jul '02

18 Jul '02

Hi all... I've been reading the list about the continuous scan data lately and it looks as if the main issue getting a oversampled grid onto the "standard EXAFS" grid. We've collected a lot of continous scan data DND-CAT at the APS and basically have a routine to bin the data so it resembles a normal step scan (this way you don't loose the statistics and use every point) and then feed it into IFEFFIT. I'm not sure how similar the MR-CAT setup is to the DND collection, but it may be that the conversion process is very similar. To that effect, I have integrated those scipts into an averaging program (now called Sixpack) and analysis suite that uses IFEFFIT. One future development that I hope to add to the program is to be able to read a variety of formats, and allow user definable formats. Until that is complete, it would be simple for me to expand the functionality of what's there to include the MR-CAT continuous scan format... This solution might be able to circumvent the current problems of the splining you have. *********************************************** Street address for courier deliveries: Sam Webb SSRL/SLAC Bldg. 137, MS 69 2575 Sand Hill Rd. Menlo Park, CA 94025 Letters and other mail: Stanford Synchrotron Radiation Laboratory P.O. Box 20450, MS69 Stanford, CA 94309-0210 Phone: (650)-926-3734 FAX: (650)-926-4100 E-mail: samwebb(a)slac.stanford.edu ***********************************************

1 0

Ifeffit 1.0072 released
by Matt Newville 18 Jul '02

18 Jul '02

Hi Everyone, I figured that it had been long enough for version 1.0071, so posted version 1.0072 today. This version has several small feature improvements and bug fixes. There is also a fairly substantial changes to the 'configure ; make ' on Unix, so that the perl and python extensions are no longer built during this process, but have to be built after installation of the library, with the 'normal method' for these languages: cd wrappers/python python setup.py install cd wrappers/perl perl Makefile.PL ; make install This change greatly simplifies the work of the configuration script and is much more reliable. The other major change is that the internals of macro handling were largely rewritten so that nested macros work correctly in all cases (including on windows). Related to that, macros that are run by feffit() and minimize() at each iteration of the fitting loop now run correctly as well. I have a Windows dll ready, but am having difficulty making standalone executables from the ravelware perl scripts. Hopefully that will get worked out soon so that a new Windows installer with the latest athena, artemis, and tkatoms, as well as gifeffit and samxas will be possible. I've also started (but haven't gotten very far on) an annoted collection of ifeffit scripts, intended to be something of a 'how-to' or 'cookbook' for ifeffit commands that will complement the documents and FAQ. Any suggestions or comments on this would be very welcome. Finally, I am really expecting that 1.0072 represents a nearly complete version of Ifeffit 1.0. I believe the XAFS analysis functionality is more-or-less complete at this point. I'm willing to fix bugs, and add small features, but would like to try to leave this version stable and to start working (even if slowly!) on much more significant changes that would eventually become Ifeffit 2. Thanks, --Matt

3 9

Dopants in ATOMS
by Bruce Ravel 18 Jul '02

18 Jul '02

Hi, One of the most common inquiries that finds its way into my mailbox is why the dopants capability in Atoms does not do what the user expects. Typically, the user expects Atoms to write a magical feff.inp file that correctly accounts for the dopants in the material such that FEFF can be run using this magic input file to generate fitting standards or a XANES calculations for the doped material. I get this question so often, that I want to post a response to both the feff and ifeffit mailing lists with an obvious subject line. Then, in the future, I (or even you!) can just point the curious to this posting. ----------- O ----------- First off, this topic is covered in the Atoms document http://leonardo.phys.washington.edu/~ravel/software/exafs/doc/Atoms/Atoms/A… See section 3.4, Dopants. I also discuss this issue in Chapter 10 of the lecture notes for my course on EXAFS Analysis Using FEFF and FEFFIT. http://leonardo.phys.washington.edu/~ravel/course/notes.pdf The discussion in this posting applies to the old versions (2.50 and earlier) of Atoms as well as to the current version. ----------- O ----------- Atoms is, except in extremely contrived situations, not capable of writing a proper feff.inp file for a doped material. This is not a programming shortcoming of Atoms, but a number theoretic limitation imposed by the physical model used by FEFF. In the feff.inp file, there is a big list of atomic coordinates. The reason that people like using Atoms is because, without Atoms, it is a pain in the ass to generate that list. The virtue of Atoms is that it automates that annoying task for a certain class of matter, i.e. crystals. FEFF expects a point in space to be either unoccupied or occupied by a specific atom. A given point may be occupied neither by a fraction of an atoms nor by two different kinds of atoms. Let's use a very simple example -- gold doped into fcc copper. In fcc copper, there are 12 atoms in the first shell. If the level of doping was, say, 25%, then Atoms could reasonably use a random number generator to choose three of the 12 first neighbor sites and replace them with gold atoms. However, what should Atoms do with the second shell, which contains 6 atoms? 25% of 6 is 1.5. Feff does not allow a site to be half occupied by an atomic species, thus Atoms would have to decide either to over-dope or under-dope the second shell. This problem only gets worse if the doping fraction is not a rational (in the number theory sense) fraction, if the material is non-isotropic, or if the materials has multiple sites that the dopant might want to go to. Because Atoms cannot solve this problem correctly except in the most contrived of situations, I decided that Atoms would not attempt to solve it in any situation. If you specify dopants in Atoms' input data, the list in the feff.inp file will be be made as if there are no dopants. This leads to two big questions: 1. Why are dopants allowed in Atoms? 2. How do one deal with a doped sample? The first question is the easy one. Atoms does many things other besides feff.inp generation. Calculations involving tables of absorption coefficients and simulations of powder diffraction and DAFS spectra can use dopant information quite well. The second question is the tricky one and the answer is somewhat different for EXAFS as for XANES. The chapter in the PDF file mentioned at the beginning of this posting discusses one approach to analyzing EXAFS of doped materials. In this posting http://cars9.uchicago.edu/feff/feffusers/msg00022.html Scott Calvin adds his two cents worth. The bottom line for EXAFS analysis is that most dopant problems require running FEFF many times to deal with the wide variety of local environments one finds in a doping problem. One of my papers, A Combined EXAFS and First Principles Theory Study of Pb(1-x)Ge(x)Te, B. Ravel, E. Cockayne, M. Newville, and K.M. Rabe, Phys. Rev. B 60, #21, pp. 14632-14642 (1999). discusses a very thorough approach to one particular case of a doped material. The best approach to simulating a XANES spectrum on a doped material that I am aware of also involves running FEFF many times. One problem a colleague of mine asked me about some time ago was the situation of oxygen vacancies in Au2O3. After some discussion, the solution we came up with was to use Atoms to generate the feff.inp for the pure material. My friend then wrote a little computer program that would read in the feff.inp file, randomly remove oxygen atoms from the list, write the feff.inp file back out with the missing oxygens, and run FEFF. He would do this repeatedly, each time replacing a different set of randomly selected atoms and each time saving the result. This set of computed spectra was then averaged. New calculations were made and added to the running average until the result stopped changing. If I remember, it took about 10 calculations to converge. This random substitution approach would work just as well for dopants as for vacancies. I hope that is helpful. B -- Bruce Ravel ----------------------------------- ravel(a)phys.washington.edu Code 6134, Building 3, Room 222 Naval Research Laboratory phone: (1) 202 767 5947 Washington DC 20375, USA fax: (1) 202 767 1697 NRL Synchrotron Radiation Consortium (NRL-SRC) Beamlines X11a, X11b, X23b, X24c, U4b National Synchrotron Light Source Brookhaven National Laboratory, Upton, NY 11973 My homepage: http://feff.phys.washington.edu/~ravel EXAFS software: http://feff.phys.washington.edu/~ravel/software/exafs/

1 0