Hi Logan, Matt, I can reproduce Logan's error with his data files. Using 0.9.22, I enable the SSRL plugins (A,B) and try to import. The 2013 binary imports without issue. The 2015 binary gives the error message Logan gave: At line 316 of file iff_rddata.f Fortran runtime error: End of record I have issues with both ascii files when trying to import as well in that the import function finds many lines of 0.0000s (3 columns of zeros) that it did not find during the binary import. Screen captures in the attached pdf illustrate this. -R. On 9/17/2015 9:06 AM, Logan Giles wrote:

Hi Matt,

Thanks for the response. I did have the SSRLB file plugin enabled. The 2013 binary wouldn't open if that plugin wasn't enabled anyways, I think. I am using Demeter 0.9.22, is that the version you are using as well? I tried running the program in "run as administrator" as well. No luck.

Any other suggestions?

-Logan

On Thu, Sep 17, 2015 at 5:14 AM, Matt Newville mailto:newville@cars.uchicago.edu> wrote:

Hi Logan,

On Wed, Sep 16, 2015 at 5:26 PM, Logan Giles mailto:gileslo@slac.stanford.edu> wrote:

Hello everyone,

I am running Demeter 0.9.22 and trying import recent SSRL Binary files. I have the SSRL plugins enabled as well. The problem is when I try to import binary files, the program crashes.

It seems that something might have changed recently in SSRL BL 7-3 files because I can import July 2013 files but not July 2015. I have attached a NaMoO4 sweep at the Mo K-edge from each run for your testing. Immediately after trying to import the binary July 2015 files, the program immediately crashes and exits. The July 2013 binary opens right up.

When I look at the dathena.log file for the "2015 crash", everything looks normal until the end:

"Demeter version 0.9.22

At line 316 of file iff_rddata.fFortran runtime error: End of record"

Also, when I try to import the ascii files of the July 2013 or July 2015 data, I get an error as well, but the same error. I can fix this by changing two of the blank rows, which are the column labels for the data, to "BLANK" or something. I have attached these unedited ascii files as well, but I am more concerned with my binary problem. The reason for this is that I was recently sent some other SSRL binary files without their corresponding ascii files that also crash.

Let me know that you think, or if you can reproduce the error in trying to import the binary files.

Thanks for your assistance and best regards,

Did you enable the SSRLB file plugin? From the Main Window top drop down list, the 2nd from the bottom choice is "Plugin Registry". There are plugins for SSRL ASCII and SSRL Binary files, and several others.

If I do that, I *can* read in the 2015 Binary file. HTH,

--Matt

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov mailto:Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

-- Logan Giles, PhD Postdoctoral Scholar Stanford University Cell: (406) 600-6883 Work: (650) 926-5033

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

Hi Logan, A followup to clarify my earlier message: I am able to read at least one SSRL binary format file with Athena by using the provided plugin. This is pretty darn AMAZING. If you're having problems reading data from SSRL with Athena, contact the beamline staff to get the data formatted in a way that can be read by Athena. An undocumented binary format without a public, supported API for reading it is completely unacceptable, and you should not in any way rely on that data. A beamline that expects anyone to use such a non-supported binary format is NOT giving you the data you need from the beamline. You should get data in a readable ASCII format or stop using the facility. You should complain LOUDLY to the facility on End-of-Experiment forms. It is not forgivable to use this format. So, while it is impressive that Athena can read some SSRL binary data, SSRL is solely and completely to blame if Athena cannot read other SSRL binary data. As for the ASCII data: Well, at least you can edit it until it does get read in. You might want to complain about that format too -- it's sort of a mess. Cheers, --Matt On Thu, Sep 17, 2015 at 11:06 AM, Logan Giles wrote:

Hi Matt,

Thanks for the response. I did have the SSRLB file plugin enabled. The 2013 binary wouldn't open if that plugin wasn't enabled anyways, I think. I am using Demeter 0.9.22, is that the version you are using as well? I tried running the program in "run as administrator" as well. No luck.

Any other suggestions?

-Logan

On Thu, Sep 17, 2015 at 5:14 AM, Matt Newville

...

wrote:

...

Hi Logan,

On Wed, Sep 16, 2015 at 5:26 PM, Logan Giles wrote:

...

Hello everyone,

I am running Demeter 0.9.22 and trying import recent SSRL Binary files. I have the SSRL plugins enabled as well. The problem is when I try to import binary files, the program crashes.

It seems that something might have changed recently in SSRL BL 7-3 files because I can import July 2013 files but not July 2015. I have attached a NaMoO4 sweep at the Mo K-edge from each run for your testing. Immediately after trying to import the binary July 2015 files, the program immediately crashes and exits. The July 2013 binary opens right up.

When I look at the dathena.log file for the "2015 crash", everything looks normal until the end:

"Demeter version 0.9.22

At line 316 of file iff_rddata.fFortran runtime error: End of record"

Also, when I try to import the ascii files of the July 2013 or July 2015 data, I get an error as well, but the same error. I can fix this by changing two of the blank rows, which are the column labels for the data, to "BLANK" or something. I have attached these unedited ascii files as well, but I am more concerned with my binary problem. The reason for this is that I was recently sent some other SSRL binary files without their corresponding ascii files that also crash.

Let me know that you think, or if you can reproduce the error in trying to import the binary files.

Thanks for your assistance and best regards,

...

Did you enable the SSRLB file plugin? From the Main Window top drop down list, the 2nd from the bottom choice is "Plugin Registry". There are plugins for SSRL ASCII and SSRL Binary files, and several others.

If I do that, I *can* read in the 2015 Binary file. HTH,

--Matt

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

-- Logan Giles, PhD Postdoctoral Scholar Stanford University Cell: (406) 600-6883 Work: (650) 926-5033

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

-- --Matt Newville <newville at cars.uchicago.edu> 630-252-0431

Thanks Bruce. I'll speak to the beam-line scientists here and see what they say. -Logan On Thu, Sep 17, 2015 at 9:54 AM, Bruce Ravel wrote:

On 09/16/2015 06:26 PM, Logan Giles wrote:

...

I am running Demeter 0.9.22 and trying import recent SSRL Binary files. I have the SSRL plugins enabled as well. The problem is when I try to import binary files, the program crashes.

It seems that something might have changed recently in SSRL BL 7-3 files because I can import July 2013 files but not July 2015. I have attached a NaMoO4 sweep at the Mo K-edge from each run for your testing. Immediately after trying to import the binary July 2015 files, the program immediately crashes and exits. The July 2013 binary opens right up.

When I look at the dathena.log file for the "2015 crash", everything looks normal until the end:

"Demeter version 0.9.22

At line 316 of file iff_rddata.fFortran runtime error: End of record"

Also, when I try to import the ascii files of the July 2013 or July 2015 data, I get an error as well, but the same error. I can fix this by changing two of the blank rows, which are the column labels for the data, to "BLANK" or something. I have attached these unedited ascii files as well, but I am more concerned with my binary problem. The reason for this is that I was recently sent some other SSRL binary files without their corresponding ascii files that also crash.

Logan,

The fundamental problem here is that the SSRL binary and ascii files are not well documented. And, they both suck really hard as file formats.

When I made the SSRLA and SSRLB plugins, I was working from examples that people had provided to me. (It has been something like 22 years since I was last at SSRL for an experiment, so I have no direct experience with these files.) To interpret the wacky binary formats, I followed the example that Sam had programed into sixpack. The only sense in which the plugins ever worked was in the sense that "they worked on the examples in front of me".

On the computer I am sitting at, I am able to read the 2015 binary file.

The 2013 binary file fails. I am guessing that is because there are a lot of ^M characters in that file that are not in my example data or in the 2015 file.

Both ascii files fail. Your explanation appears to be correct. The plugin expects all of the columns to be named. It expects that the list of column labels ends in a blank line, with the data to follow. Your files have blank lines in the middle of the column labels. That breaks the plugin.

I am not clear that any of this is my problem. I don't actually know the specification for either format. Three of your four examples fail to meet my expectation based off of examples I was given some years back. I don't think I should be expected to meet an ill-defined and moving target.

If you or someone else wants to update the SSRLA or SSRLB filetype plugins to work better with actual files from SSRL, I would be very happy to accept that contribution.

But fixing them myself on the basis of this small sample set is very low on my to-do list. If you intend to use Athena, your best bet would be to write yourself a little conversion program that leaves the data in a state that Athena can interpret. Another option is to use sixpack.

B

-- Bruce Ravel ------------------------------------ bravel@bnl.gov

National Institute of Standards and Technology Synchrotron Science Group at NSLS-II Building 535A Upton NY, 11973

Homepage: http://bruceravel.github.io/home/ Software: https://github.com/bruceravel Demeter: http://bruceravel.github.io/demeter/

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

-- Logan Giles, PhD Postdoctoral Scholar Stanford University Cell: (406) 600-6883 Work: (650) 926-5033