John, I think the problem is different from what you presume. You have measured a *very* short data range. The archive post you cite is not really relevant. This one is: http://www.mail-archive.com/ifeffit@millenia.cars.aps.anl.gov/msg03717.html There are two issues that need addressing given the data you provided. Issue #1: As I explain in that mailing list posting, you are doing yourself no favors measuring data in a range from -40 to +70 or so. Such data are almost impossible to normalize consistently and defensibly. Issue #2: Your data point out some shortcomings in Athena. Athena should deal more gracefully when given ridiculous input like your much-too-short data files. Here is a short list of things that Athena is doing wrongly: 1. The default values for the pre-edge line are both allowed to be outside the data range. This is actually the primary cause of your problems. Because both pre-edge parameters are before the beginning of the data, the determination of the pre-edge line fails. Trying to plot the pre-edge line or normalized data, then also fails. This problem is exacerbated by the fact that you are measuing Mo. For such a heavy element, the default value for the upper bound of the pre-edge region is -45. Your data begin at about -38. 2. As long as the data are flagged as being XANES data (something Athena does to avoid wasting time attempting to find a background function for very short data range), it is not possible to recover from problem 1. I had to change the data type [1] to mu(E) in order to correctly compute the pre-edge line. 3. Athena provides no feedback that the source of the problem is extremely short data range and poor choice of pre- and/or post-edge line parameters. Athena should do some sanity checking and complain about the potential problem. I will attempt to address the problems Athena has with data like this for the next release. You should reconsider your measurement strategy the next time you go to the synchrotron. B [1] http://bruceravel.github.com/demeter/aug/ui/glist.html#informationaboutitems... On Tuesday, February 19, 2013 03:33:26 PM John Hayes wrote:

Hello,

I am having some issues when importing multiple data sets into Athena. When I try to import more than one file at a time only the reference sample data from the first file are imported and the subsequent files do not import the reference data (only a straight line is imported). There is an error that is reported, and I have saved the error log [log.txt] and attached it in the zip file (as well as the dathena.log file and some sample spectra files). The spectra files are of the Mo K-edge and the reference is an Mo foil. These spectra were collected using the HXMA beamline, and as such, I am using the HXMA plug-in to import the data. The relevant columns for the reference are:

col 2: energy col 6: I1 col 7: I2

I searched the archive and found what I think is a similar issue (link: www.mail-archive.com/ifeffit@millenia.cars.aps.anl.gov/msg03454.html), and hope I'm not over-reporting the error (sorry in advance if I am), but the previous report seemed somewhat different, in that there was an issue with the wrong edge being chosen for the test and ref data (best I can tell). I do have the work around of importing each file individually, so this is no more than an inconvenience, but I thought I'd report it and see if maybe someone had some up with a more elegant solution.

I am currently using version 9.14 on a 64-bit windows platform.

Thanks in advance for any advice or help the community can provide!

-- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 Homepage: http://xafs.org/BruceRavel Software: https://github.com/bruceravel

On Wednesday, February 20, 2013 11:56:25 AM John Hayes wrote:

Thank you for your reply. I find your lack of faith...disturbing,

You are criticizing me for not having faith in you and your email address is ginger.jesus@. I think I would be well justified in ignoring you out of hand. Begrudgingly, I won't. And I will address your problem as best as I can. I want to stress that the quality of a response to an issue raised on this mailing list is highly correlated to the quality of the bug report. You wanted me to look into a problem importing the reference channel and you presented a problem that appeared to be one of experimental design -- a problem common enough to have been discussed within the last few weeks. Reasonably enough, I responded to the problem as I understood it.

but understand your concerns about the data I sent as they were just test files we collected when we were setting up the beam. However, I do not think the abridged range is the cause of the problem, as I wasl having issues importing real data as well (i.e. -150 eV pre-edge and k=12 post edge). These spectra files are attached, along with the error log file and the dathena.log file that was recorded when importing these files.

You are correct. When importing multiple data sets through the HXMA filetype plugin, the reference channels for all but the first file are imported incorrectly. I am not yet sure what is causing the problem, but I will look into it and try to have a solution in place for the next release. Unfortunately, the only workaround entirely within Athena is to import the data one at a time. Another possibility is to write yourself a little program that strips out columns 2 (or possibly 4), 6, 7, and 13-20 from the original files and writes out files that Athena can read without the plugin. B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 Homepage: http://xafs.org/BruceRavel Software: https://github.com/bruceravel

I can't help but wonder how much of this message chain is set forth by a missed Star Wars reference. On 2013-02-21, at 12:01 AM, Matt Newville wrote:

...

On Wednesday, February 20, 2013 11:56:25 AM John Hayes wrote:

...

Thank you for your reply. I find your lack of faith...disturbing,

I'd like to take this opportunity to emphasize two points I've made several times before. John, I apologize if it sounds like I'm picking on you. I don't mean to be, and I might be willing to guess that you meant this in a joking way, but I also agree fully with Bruce that the easiest action (simply ignoring you as if you were an ingrate) is worth resisting.

First, it's not about faith. John sent files. Bruce responded to the question with the facts available. I can tell you that Bruce and I just barely have time to even read messages like yours, and it is very hard for us to respond to all of them. There have been several recent questions to the mailing list that I just ignored because the questions are too vague or outside my immediate interest, and I don't have time to write "that question is too vague, please be more specific" to every question.

if you don't like the response based of the files you sent, and expect an answer based on files you didn't send, then let me remind you that you are free to solve the problem yourself. In fact, and quite seriously, you and anyone else are free to solve the problem yourself. The code is available and documented, and you can investigate what it is doing, find a solution, and change the code. If you do, please send a patch back to us.

Second, while the files from HXMA (not unlike some of the files from my own beamline) are ASCII files, they have enough non-trivial structure to them that they can not be treated as "plain", un-structured files for Athena/Artemis/Ifeffit to read. If you don't like the way Athena reads the file, write a plugin, or alter an existing one.

The essential issue is entirely with the expectation that Athena will read whatever data files you give it in the complex ways you want (for example, extracting two separate spectra from a single file and knowing that one is "reference", but also knowing how you want it use the different columns to average data or do deadtime corrections). If it doesn't do what you want, fix it: change the HXMA plugin or pre-parse the HXMA files. Or, wait for someone else to fix your problem for you. That seems like a crowded part of Hell to me.

--Matt

PS: I've made similar appeals for more people to get involved several time over the past year. The results are not good. My sense is that we really need new people involved, and have reached a good time to think about changes to the way we operate. If you're interested in the future of this software suite (like, should development continue?), this is the time to get involved. _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

Hi All, It seems this has blown up in a way that I never intended. I had sent an email privately to Bruce, but it seems I should send it to the group as well (with a few changes to address some points already brought up): I wanted to apologize for being flippant (I think that's the right word, but articulation is not my strong suit) in my last email to you. The line "I find your lack of faith...disturbing" was meant to be an allusion to Star Wars, and was meant to be a joke rather than an attack. I realize, upon re-reading the email, you do not know me and my...unique...sense of humor, and that the tone I had intended (genial), the allusion (which it seems was picked up), and ultimately the joke (which was a bad one), probably did not come through. Anyways, to put it plainly, I'm sorry I offended you. To Matt: The implication was never meant to be that Bruce should do what I wanted or be punished. (I realize that Vader was force choking the Admiral when he said the line, but when I wrote the original email the line just sort of popped into my head and I wrote it, without thinking about the context of the moment.) I think this may be one of those times where a tone of voice that seems clear in the head does not get communicated on paper. Your point about actively contributing to Demeter is well taken, and if my coding knowledge was anywhere above non-existent, I would have been more than willing to help fix the problem and further contribute to the project. But at this time, the only meaningful contribution I feel I can make is to report bugs as I find them. Once again, my apologies to everyone. John On Wed, Feb 20, 2013 at 11:17 PM, Matt Newville wrote:

On Wed, Feb 20, 2013 at 10:31 PM, Jeremy Thorbahn wrote:

...

I can't help but wonder how much of this message chain is set forth by a

missed Star Wars reference.

Oh, I think if the reference had been missed, neither Bruce nor I would have responded. The implication was that John, perhaps in jest, feels that Bruce should do what John wants or be punished. John is woefully misunderstanding the nature of the relationship here.

Of course, anything that appears in any way to be threatening is inappropriate. I assume it was a bad joke, but if it occurs again, it will be dealt with in the easiest manner (banishment from the list), rather than spending the time to write a response.

--Matt _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

-- John Hayes The Grosvenor Group Dept. of Chemistry University of Saskatchewan Two fish are in a tank. One turns to the other and says, "You man the guns, I'll drive."