Athena- Demeter 0.9.14 importing data
Hi All, I am using Athena (Demeter 0.9.14) to process the EXAFS data of Copper samples. Athena reads the data correctly however my problem is concerned with the importing the data. When I import the data, in the Athena: Column Selection dialog box the Energy units is shown in keV by default. Since my data is in eV, I can always change the keV to eV and work when it's a single file. My problem starts when I try to import multiple files where I choose Energy units as eV in the dialog box however this setting gets applied to the first file only. Remaining data files are read as keV which creates a problem for me to process the data. Is there any way to apply same Energy units to all the files while they are getting imported ? Or is it possible to change the Energy units after importing files ? I appreciate all your help. Thanks in advance ! Best regards Atul
As I have said many, many, many times on this mailing list, there is nothing I can do to fix -- much less understand -- a problem without being able to reproduce it on my own computer. Atul, the reason you have not gotten a response to your post is because you did not provide an example of the data that triggers the problem. Without that, there truly is nothing I or anyone else can do to help you. If anyone out there has a suggestion for how I can explain this point, either in this forum in in another location, in a way that will result in people submitting actionable bug reports, I would be very grateful. B ________________________________ From: ifeffit-bounces@millenia.cars.aps.anl.gov [ifeffit-bounces@millenia.cars.aps.anl.gov] on behalf of Atul Bansode [abansode@iciq.es] Sent: Friday, May 03, 2013 6:28 AM To: ifeffit@millenia.cars.aps.anl.gov Subject: [Ifeffit] Athena- Demeter 0.9.14 importing data Hi All, I am using Athena (Demeter 0.9.14) to process the EXAFS data of Copper samples. Athena reads the data correctly however my problem is concerned with the importing the data. When I import the data, in the Athena: Column Selection dialog box the Energy units is shown in keV by default. Since my data is in eV, I can always change the keV to eV and work when it’s a single file. My problem starts when I try to import multiple files where I choose Energy units as eV in the dialog box however this setting gets applied to the first file only. Remaining data files are read as keV which creates a problem for me to process the data. Is there any way to apply same Energy units to all the files while they are getting imported ? Or is it possible to change the Energy units after importing files ? I appreciate all your help. Thanks in advance ! Best regards Atul
Hi Bruce, Thanks for the reply. Well I wasn’t aware of that actually ! Herewith I am attaching 4 data files. The energy is column no. 2, numerator -12 and denominator is 13,14 Thanks Atul From: Ravel, Bruce Sent: Saturday, May 4, 2013 4:09 PM To: XAFS Analysis using Ifeffit Subject: Re: [Ifeffit] Athena- Demeter 0.9.14 importing data As I have said many, many, many times on this mailing list, there is nothing I can do to fix -- much less understand -- a problem without being able to reproduce it on my own computer. Atul, the reason you have not gotten a response to your post is because you did not provide an example of the data that triggers the problem. Without that, there truly is nothing I or anyone else can do to help you. If anyone out there has a suggestion for how I can explain this point, either in this forum in in another location, in a way that will result in people submitting actionable bug reports, I would be very grateful. B -------------------------------------------------------------------------------- From: ifeffit-bounces@millenia.cars.aps.anl.gov [ifeffit-bounces@millenia.cars.aps.anl.gov] on behalf of Atul Bansode [abansode@iciq.es] Sent: Friday, May 03, 2013 6:28 AM To: ifeffit@millenia.cars.aps.anl.gov Subject: [Ifeffit] Athena- Demeter 0.9.14 importing data Hi All, I am using Athena (Demeter 0.9.14) to process the EXAFS data of Copper samples. Athena reads the data correctly however my problem is concerned with the importing the data. When I import the data, in the Athena: Column Selection dialog box the Energy units is shown in keV by default. Since my data is in eV, I can always change the keV to eV and work when it’s a single file. My problem starts when I try to import multiple files where I choose Energy units as eV in the dialog box however this setting gets applied to the first file only. Remaining data files are read as keV which creates a problem for me to process the data. Is there any way to apply same Energy units to all the files while they are getting imported ? Or is it possible to change the Energy units after importing files ? I appreciate all your help. Thanks in advance ! Best regards Atul -------------------------------------------------------------------------------- _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Hi Atul,
On Sat, May 4, 2013 at 10:21 AM,
Hi Bruce, Thanks for the reply. Well I wasn’t aware of that actually !
Sorry, and I don't mean to pick on you in particular, but just to emphasize Bruce's response. Did you really think your question could be answered with the data files? DAthena tries to read any data file, but beamlines seem to be able to write very odd data files, and it is inevitable that some raw data files will cause problems.
Herewith I am attaching 4 data files. The energy is column no. 2, numerator -12 and denominator is 13,14
Well, what would be your guess at could be going on? What have you tried so far? I look at those files and two thing come to my mind: a) Column 1 appears to be the highly useless point number. It seems perfectly believable that DAthena might think this column is the energy column, and it appears to be in keV, not eV. b) Columns 3 to 10 have 3 significant digits each. Thus, without even trying to read the file, it's clear that 9 of the 14 columns are of no use. The file also has an extra line at the end without data. It might be worthwhile to convert the files to simpler format that DAthena. Cheers, --Matt
Hi Matt, Thanks for your reply ! a.. Sorry, and I don't mean to pick on you in particular, but just to b.. emphasize Bruce's response. Did you really think your question could c.. be answered with the data files? Well, I suppose the data file were necessary to understood what kind of problem I am seeing and if its really reproducible on some other PC, to be called is as a problem. That’s what I understood from Bruce´s message. a.. DAthena tries to read any data file, but beamlines seem to be able to write very odd data files, and b.. it is inevitable that some raw data files will cause problems. I fully understand that different beamlines will produce different type of files depending on their preference. My message was to check if I can get some help if I have this kind of data set ! a.. Well, what would be your guess at could be going on? What have you b.. tried so far? c.. I look at those files and two thing come to my mind: a) Column 1 d.. appears to be the highly useless point number. It seems perfectly e.. believable that DAthena might think this column is the energy column, f.. and it appears to be in keV, not eV. b) Columns 3 to 10 have 3 g.. significant digits each. Thus, without even trying to read the file, h.. it's clear that 9 of the 14 columns are of no use. The file also i.. has an extra line at the end without data. What I understood so far is dathena doesn’t have an problem to read a single data file. It works like charm when I open files one by one and select the energy units in eV. The problem occurs during import of multiple files where I can choose Energy unit to eV but its gets applied to only first file in import. Having an Energy unit option in preprocessing tab could be useful ( its just some wild thought ) so that same setting will be applied to all the files imported. Well in the end I can import files one by one but then I have hundreds of data files, and that’s what made me to subscribe and mail to Ifeffit mailing list to see if I can get any help. Cheers _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
Hi Atul,
On Sat, May 4, 2013 at 5:10 PM,
Hi Matt, Thanks for your reply !
Sorry, and I don't mean to pick on you in particular, but just to emphasize Bruce's response. Did you really think your question could be answered with the data files?
Well, I suppose the data file were necessary to understood what kind of problem I am seeing and if its really reproducible on some other PC, to be called is as a problem. That’s what I understood from Bruce´s message.
DAthena tries to read any data file, but beamlines seem to be able to write very odd data files, and it is inevitable that some raw data files will cause problems.
I fully understand that different beamlines will produce different type of files depending on their preference. My message was to check if I can get some help if I have this kind of data set !
Well, what would be your guess at could be going on? What have you tried so far? I look at those files and two thing come to my mind: a) Column 1 appears to be the highly useless point number. It seems perfectly believable that DAthena might think this column is the energy column, and it appears to be in keV, not eV. b) Columns 3 to 10 have 3 significant digits each. Thus, without even trying to read the file, it's clear that 9 of the 14 columns are of no use. The file also has an extra line at the end without data.
What I understood so far is dathena doesn’t have an problem to read a single data file. It works like charm when I open files one by one and select the energy units in eV. The problem occurs during import of multiple files where I can choose Energy unit to eV but its gets applied to only first file in import. Having an Energy unit option in preprocessing tab could be useful ( its just some wild thought ) so that same setting will be applied to all the files imported.
Well in the end I can import files one by one but then I have hundreds of data files, and that’s what made me to subscribe and mail to Ifeffit mailing list to see if I can get any help.
OK, I see that Athena has slightly different behavior when reading one and many spectra. Just as a warning, I think you may run into problems using the standard version of Athena to handle hundreds of spectra (It's not Bruce's fault -- it's a limitation of Ifeffit). It might be worth your time (or someone at the beamline producing these files) to write a dedicated conversion-and-averaging script. --Matt
Hi Matt,
I had similar thing in my mind but I thought before I spend lot of time in
writing and debugging the script I could ask if any one ran in to same
problem.
And eventually some solution.
Indeed, its good idea !
Thanks for your time.
Best regards
Atul
-----Original Message-----
From: Matt Newville
Sent: Monday, May 6, 2013 9:51 PM
To: XAFS Analysis using Ifeffit
Subject: Re: [Ifeffit] Athena- Demeter 0.9.14 importing data
Hi Atul,
On Sat, May 4, 2013 at 5:10 PM,
Hi Matt, Thanks for your reply !
Sorry, and I don't mean to pick on you in particular, but just to emphasize Bruce's response. Did you really think your question could be answered with the data files?
Well, I suppose the data file were necessary to understood what kind of problem I am seeing and if its really reproducible on some other PC, to be called is as a problem. That’s what I understood from Bruce´s message.
DAthena tries to read any data file, but beamlines seem to be able to write very odd data files, and it is inevitable that some raw data files will cause problems.
I fully understand that different beamlines will produce different type of files depending on their preference. My message was to check if I can get some help if I have this kind of data set !
Well, what would be your guess at could be going on? What have you tried so far? I look at those files and two thing come to my mind: a) Column 1 appears to be the highly useless point number. It seems perfectly believable that DAthena might think this column is the energy column, and it appears to be in keV, not eV. b) Columns 3 to 10 have 3 significant digits each. Thus, without even trying to read the file, it's clear that 9 of the 14 columns are of no use. The file also has an extra line at the end without data.
What I understood so far is dathena doesn’t have an problem to read a single data file. It works like charm when I open files one by one and select the energy units in eV. The problem occurs during import of multiple files where I can choose Energy unit to eV but its gets applied to only first file in import. Having an Energy unit option in preprocessing tab could be useful ( its just some wild thought ) so that same setting will be applied to all the files imported.
Well in the end I can import files one by one but then I have hundreds of data files, and that’s what made me to subscribe and mail to Ifeffit mailing list to see if I can get any help.
OK, I see that Athena has slightly different behavior when reading one and many spectra. Just as a warning, I think you may run into problems using the standard version of Athena to handle hundreds of spectra (It's not Bruce's fault -- it's a limitation of Ifeffit). It might be worth your time (or someone at the beamline producing these files) to write a dedicated conversion-and-averaging script. --Matt _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
On Saturday, May 04, 2013 05:21:51 PM abansode@iciq.es wrote:
Well I wasn’t aware of that actually ! Herewith I am attaching 4 data files. The energy is column no. 2, numerator -12 and denominator is 13,14
Atul, You are correct. Athena does the wrong thing when importing multiple files. Although it remembers your column selection, it keeps resetting the choice of energy units. As Matt pointed out, the reason for this is the annoying and useless first column of your data files. Point number? That's kind of silly. Athena attempts to use some smarts when importing data. One of the tests it does is on the contents of the first column in an effort to determine whether the data were recorded in eV, keV, or some other unit. "Point number" matches the simple test Athena uses to recognize keV units. That's not the part that needs fixing. (Well, OK, maybe it could be better, but it's not the thing I am going to fix!) The part that needs fixing is that, while Athena is rememberiong the column selection from the last time you imported a similar data file, it is not remembering the energy units. I am going to work on that this afternoon. Whenever I make a new release, this will be fixed. For now, the work-around is to import you data one at a time. Alternately, write yourself a little program that strips the first column out of all your data files. Alternate #2, write yourself a Athena filetype plugin that recognizes the file type and scrubs the first column. B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 Homepage: http://xafs.org/BruceRavel Software: https://github.com/bruceravel
On Monday, May 13, 2013 02:49:21 PM Bruce Ravel wrote:
The part that needs fixing is that, while Athena is rememberiong the column selection from the last time you imported a similar data file, it is not remembering the energy units. I am going to work on that this afternoon. Whenever I make a new release, this will be fixed.
Well, that didn't take long. Thanks, Atul, for discovering and reporting this bug. B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 Homepage: http://xafs.org/BruceRavel Software: https://github.com/bruceravel
Hi Bruce, Thanks for the message ! I fully agree with you regarding the first column of data files. At the end, I was averaging my data files with written macro but seems it would be much more easier to process the files directly. I highly appreciate your efforts towards the solution of this problem. Thank a ton ! Best regards Atul -----Original Message----- From: Bruce Ravel Sent: Monday, May 13, 2013 8:49 PM To: XAFS Analysis using Ifeffit Subject: Re: [Ifeffit] Athena- Demeter 0.9.14 importing data On Saturday, May 04, 2013 05:21:51 PM abansode@iciq.es wrote:
Well I wasn’t aware of that actually ! Herewith I am attaching 4 data files. The energy is column no. 2, numerator -12 and denominator is 13,14
Atul, You are correct. Athena does the wrong thing when importing multiple files. Although it remembers your column selection, it keeps resetting the choice of energy units. As Matt pointed out, the reason for this is the annoying and useless first column of your data files. Point number? That's kind of silly. Athena attempts to use some smarts when importing data. One of the tests it does is on the contents of the first column in an effort to determine whether the data were recorded in eV, keV, or some other unit. "Point number" matches the simple test Athena uses to recognize keV units. That's not the part that needs fixing. (Well, OK, maybe it could be better, but it's not the thing I am going to fix!) The part that needs fixing is that, while Athena is rememberiong the column selection from the last time you imported a similar data file, it is not remembering the energy units. I am going to work on that this afternoon. Whenever I make a new release, this will be fixed. For now, the work-around is to import you data one at a time. Alternately, write yourself a little program that strips the first column out of all your data files. Alternate #2, write yourself a Athena filetype plugin that recognizes the file type and scrubs the first column. B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 Homepage: http://xafs.org/BruceRavel Software: https://github.com/bruceravel _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
participants (5)
-
abansode＠iciq.es -
Atul Bansode
-
Bruce Ravel -
Matt Newville -
Ravel, Bruce