Athena (Demeter 0.9.26) does not import all datapoints from ascci file
Hello, I got a new trouble by importing datafile in Athena (Demeter 0.9.26). Athena from Demeter version 0.9.26 does not import all points from the datafile but some of them yielding a very poor resolution especially for the edge region. The given file has 275 points and only 143points are imported (see derivative after import with Athena(26) and with pymca were all points are imported). Importing data using Athena from an older version of Demeter and saving the file solved my import problem temporary. But there I notice that a priori only 202 instead 275 were imported but it was not obvious from the plot appearance that some points are missing . I joined some screenshots as well as Athena log file after importing the data (error message about a file not found but the name in stash is not the one Athena is looking for..?). I joined both Athena file (Demeter v26 and Demeter v19 after import of the same asci file) as well as the asci file. I got the problem of "partial import" with all my datafiles. Did somebody experienced this trouble too? Thanks a lot Kathy Dardenne Operating system : Windows 7 entreprise - Service pack 1 Programme : Demeter 0.9.26 last release Karlsruhe Institute of Technology (KIT) Institute for nuclear waste disposal (INE) Dr. Kathy DARDENNE Hermann-von-Helmholtz-Platz 1, Building 712 76344 Eggenstein-Leopoldshafen, Germany Tel. : +49 721 608-26669 Fax : +49 721 608-23927 E-mail : mailto:kathy.dardenne@kit.edu dardenne@kit.edu Internet : http://www.kit.edu/ www.ine.kit.edu KIT - The Research University in the Helmholtz Association.
Hi, I can confirm Kathy's problem. On my computer with Athena (Demeter 0.9.26) only 109 of 275 points are imported. Reinstalling Demeter 0.9.25 fixes the problem again. Tim From: Ifeffit [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of Kathy Dardenne Sent: Wednesday, February 28, 2018 9:19 PM To: ifeffit@millenia.cars.aps.anl.gov Subject: [Ifeffit] Athena (Demeter 0.9.26) does not import all datapoints from ascci file Hello, I got a new trouble by importing datafile in Athena (Demeter 0.9.26). Athena from Demeter version 0.9.26 does not import all points from the datafile but some of them yielding a very poor resolution especially for the edge region. The given file has 275 points and only 143points are imported (see derivative after import with Athena(26) and with pymca were all points are imported). Importing data using Athena from an older version of Demeter and saving the file solved my import problem temporary. But there I notice that a priori only 202 instead 275 were imported but it was not obvious from the plot appearance that some points are missing . I joined some screenshots as well as Athena log file after importing the data (error message about a file not found but the name in stash is not the one Athena is looking for....?). I joined both Athena file (Demeter v26 and Demeter v19 after import of the same asci file) as well as the asci file. I got the problem of "partial import" with all my datafiles. Did somebody experienced this trouble too? Thanks a lot Kathy Dardenne Operating system : Windows 7 entreprise - Service pack 1 Programme : Demeter 0.9.26 last release Karlsruhe Institute of Technology (KIT) Institute for nuclear waste disposal (INE) Dr. Kathy DARDENNE Hermann-von-Helmholtz-Platz 1, Building 712 76344 Eggenstein-Leopoldshafen, Germany Tel. : +49 721 608-26669 Fax : +49 721 608-23927 E-mail : dardenne@kit.edumailto:kathy.dardenne@kit.edu Internet : www.ine.kit.eduhttp://www.kit.edu/ KIT - The Research University in the Helmholtz Association.
Tim, Kathy, I did not respond to Kathy's earlier email, and I apologize for letting it falling off my radar screen. FWIW, I do not see the same problem. I can read the file (with the Spec plugin enabled) and plot the data. There are 275 data points. Reading multi-column files from can be tricky (should we know which columns you chose or what processing steps you took). And spec files can cause no end of trouble. But from the screenshots aattached, it looks like maybe Time meant to rebin the data. Is that related to what Kathy was trying to do? Perhaps you could explain what steps you took in reading the data file and more clearly explain the problem? Like, did you intend to re-bin the data? For sure, the plot labeled as coming from PyMCA looks strange to me --- the derivative of the data has clearly been smoothed from the raw data in the file. I don't know what to make of that. Hope that helps, --Matt
Hi Matt, the rebin screen was just to show the number of data points. We used column 1 as energy and column 9 as numerator; data type ‘xanes’; no preprocessing. Overall we used identical parameters in 0.9.25 and 0.9.26 The “plan” is to do the standard calibration and normalization steps, but we lose data points by just importing the data. In the example Kathy sent, we go from 275 “real” points to 109 imported points. This effect appears with all Spec files we have tested (measured at different times and to different beamlines -> different number of columns). Importing data from FEFF and FDMNES works, we get the same number of points as in the ascii file. I also tried the XAS Viewer in Larch (0.9.37), which read all points of the Spec file. I exported the data as Athena project and .csv Importing the project file into Athena works and the “about current group” function shows 275 points. However, nothing can be plotted, the plot window stays on the overview from the import screen. The csv file contains all 275 points, but importing to Athena again reduces it to 109 points. Tim From: Ifeffit [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of Matt Newville Sent: Tuesday, March 27, 2018 2:12 AM To: XAFS Analysis using Ifeffit Subject: Re: [Ifeffit] Athena (Demeter 0.9.26) does not import all datapoints from ascci file Tim, Kathy, I did not respond to Kathy's earlier email, and I apologize for letting it falling off my radar screen. FWIW, I do not see the same problem. I can read the file (with the Spec plugin enabled) and plot the data. There are 275 data points. Reading multi-column files from can be tricky (should we know which columns you chose or what processing steps you took). And spec files can cause no end of trouble. But from the screenshots aattached, it looks like maybe Time meant to rebin the data. Is that related to what Kathy was trying to do? Perhaps you could explain what steps you took in reading the data file and more clearly explain the problem? Like, did you intend to re-bin the data? For sure, the plot labeled as coming from PyMCA looks strange to me --- the derivative of the data has clearly been smoothed from the raw data in the file. I don't know what to make of that. Hope that helps, --Matt
Hi Tim, Hm, that's all weird. I still cannot reproduce this. Reading Kathy's original file, I see 275 data points in Athena. Reading the Larch-generated project file in Athena, I also see 275 points... That's running both Athena+Ifeffit on Windows or running Athena+Larch on MacOS. One thing that might be worth looking into is whether the data being in keV rather than eV might make a difference. That's sort of grasping at straws, but I'm not sure what else to try. On Tue, Mar 27, 2018 at 10:22 AM, Prüßmann, Tim (IKFT) < tim.pruessmann@kit.edu> wrote:
Hi Matt,
the rebin screen was just to show the number of data points.
We used column 1 as energy and column 9 as numerator; data type ‘xanes’; no preprocessing.
Overall we used identical parameters in 0.9.25 and 0.9.26
The “plan” is to do the standard calibration and normalization steps, but we lose data points by just importing the data. In the example Kathy sent, we go from 275 “real” points to 109 imported points.
This effect appears with all Spec files we have tested (measured at different times and to different beamlines -> different number of columns).
Importing data from FEFF and FDMNES works, we get the same number of points as in the ascii file.
I also tried the XAS Viewer in Larch (0.9.37), which read all points of the Spec file. I exported the data as Athena project and .csv
Importing the project file into Athena works and the “about current group” function shows 275 points. However, nothing can be plotted, the plot window stays on the overview from the import screen.
The csv file contains all 275 points, but importing to Athena again reduces it to 109 points.
Tim
*From:* Ifeffit [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] *On Behalf Of *Matt Newville *Sent:* Tuesday, March 27, 2018 2:12 AM *To:* XAFS Analysis using Ifeffit *Subject:* Re: [Ifeffit] Athena (Demeter 0.9.26) does not import all datapoints from ascci file
Tim, Kathy,
I did not respond to Kathy's earlier email, and I apologize for letting it falling off my radar screen.
FWIW, I do not see the same problem. I can read the file (with the Spec plugin enabled) and plot the data. There are 275 data points. Reading multi-column files from can be tricky (should we know which columns you chose or what processing steps you took). And spec files can cause no end of trouble. But from the screenshots aattached, it looks like maybe Time meant to rebin the data. Is that related to what Kathy was trying to do?
Perhaps you could explain what steps you took in reading the data file and more clearly explain the problem? Like, did you intend to re-bin the data?
For sure, the plot labeled as coming from PyMCA looks strange to me --- the derivative of the data has clearly been smoothed from the raw data in the file. I don't know what to make of that.
Hope that helps,
--Matt
_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit
--Matt
Hi Matt,
that’s it. I manually changed the energy column from keV to eV and it works.
Tim
From: Ifeffit [mailto:ifeffit-bounces@millenia.cars.aps.anl.gov] On Behalf Of Matt Newville
Sent: Tuesday, March 27, 2018 7:50 PM
To: XAFS Analysis using Ifeffit
Subject: Re: [Ifeffit] Athena (Demeter 0.9.26) does not import all datapoints from ascci file
Hi Tim,
Hm, that's all weird. I still cannot reproduce this. Reading Kathy's original file, I see 275 data points in Athena. Reading the Larch-generated project file in Athena, I also see 275 points... That's running both Athena+Ifeffit on Windows or running Athena+Larch on MacOS.
One thing that might be worth looking into is whether the data being in keV rather than eV might make a difference. That's sort of grasping at straws, but I'm not sure what else to try.
On Tue, Mar 27, 2018 at 10:22 AM, Prüßmann, Tim (IKFT)
Hi Tim, On Wed, Mar 28, 2018 at 8:33 AM, Prüßmann, Tim (IKFT) < tim.pruessmann@kit.edu> wrote:
Hi Matt,
that’s it. I manually changed the energy column from keV to eV and it works.
Huh, OK -- I don't know why that should sometimes cause a problem, but it should be a good clue. We should probably make sure that when Larch or XAS Viewer writes an Athena Project file that it forces energy to be in eV. And somewhat related, we were just having a discussion for Larch about adding more and better routines for reading Spec files. Anyway, I'm glad you sort of found a workaround. --Matt
participants (3)
-
Kathy Dardenne -
Matt Newville -
Prüßmann, Tim (IKFT)