[Ifeffit] Demeter-devel MacPorts Athena issues & work-around

[Matt Newville]

Hi,


On Sat, Jul 5, 2014 at 2:06 PM, Bruce Ravel <bravel at bnl.gov> wrote:

> On 07/04/2014 12:11 PM, Margaret Hinkle wrote:
>
>> Thanks for your help & your quick reply. I did not expect my altering
>> one character in the code to fix the problem -- I had just hoped it
>> would tell me if the Tip of the Day was indeed the culprit. I did as you
>> suggested, and the problem is completely fixed. Thanks!
>>
>> Per your comment regarding Issue 2 -- I have attached two files, one
>> with the header removed (filename: '...NoHeader_exafsscan.dat'), and the
>> other with the header left in with #s commenting it out (filename:
>> '...Header_exafsscan.dat'). Note that these are transmission samples,
>> with I0 in column 56, and I1 in column 57.
>>
>>
> Margaret Anne,
>
> By the end of this email, I am going to thoroughly solve your problem.
> But, first, I am going to step up on my soapbox and pontificate.  It's
> kinda what I do on this mailing list....
>
>
>
> You started this by writing "I can only load data files into Athena if
> there are no headers of any sort".  That is a demonstrably untrue
> statement, and easily so.  Were it true, this mailing list would be either
> (a) filled with nothing but posts about people complaining that they cannot
> import their data, or (b) silent because no one would use software so
> completely fucked up and useless.
>
> A true statement might have been that you had examples of data files that
> could not be imported into Athena.  But you did not include an example of
> the troublesome data file -- I had to ask for it.
>
> When you ask a question, you should try to put yourself in the shoes of
> the person answering.  I am not in the same room as you, I am not looking
> at your computer, and I do not have the example that triggers the problem
> here in front of me.  And I do not possess telepathy.  When you ask a
> question, you have to give the person who wants to help a fighting chance
> of being able to help.
>

Grouchy?   Didn't Margaret Anne post an example file when asked?  I believe
she also tried to solve the Splash screen issue herself, and explained what
she tried in pretty good detail...  It may not have worked as well as she'd
hoped but she definitely tried something plausible (admirable) and
explained what she did (even more so).   I would call that an exemplary
report, myself.


> All right.  Enough of that.  Let's solve your problem.
>
Find the installation location of Demeter on your computer.  If you do not
> know where that is, then open a command terminal and enter the following
> command:
>
>    perl -e 'use Demeter; print $INC{"Demeter.pm"}, $/'
>
> The thing that gets printed is the folder containing the main Demeter
> module file.  Everything else is beneath that folder.
>
> Download this file:
>
>    https://s3.amazonaws.com/demeter4xas/SpecFileLongLine.pm
>
> and save it as
>
>    Demeter/Plugins/SpecFileLongLine.pm
>
> where "Demeter/" is a folder inside the folder we identified above.
>
> Make sure that the file is called "SpecFileLongLines.pm" -- exactly as
> written there -- and that your web browser does not "help you out" by
> changing the extension of the file to ".txt" or any other such shenanigans.
>
> Now fire up Athena and select "Plugin Registry" from the main menu. This
> will display the view shown here:
>
>
> http://bruceravel.github.io/demeter/aug/other/plugin.html#
> athena%27spluginregistry
>
> There should be an row that says "SpecFileLongLine".  Click it on.
>
> You should now be able to import your files without needing to edit the
> header.
>
> Note that if you attempt to import one of your data files *without*
> enabling the "SpecFileLongLine" plugin, Athena will continue to crash. The
> mere presence of the SpecFileLongLine.pm file is not enough -- you *must*
> enable in the plugin registry.
>
>
>
> So, what was the problem?
>
> Well, Ifeffit -- which is the math and XAS engine underneath the hood of
> Athena -- is written in Fortran.  Fortran does not have dynamic memory
> allocation.  Thus there are some hardwired limits to things in Ifeffit, not
> all of which get handled gracefully when exceeded.
>
> The line in your data file causing the problem is the one that starts with
> "#L".  It contains the column labels and is about 330 characters long.  As
> one point, Ifeffit attempts to stuff those 330 characters in a variable
> configured to be 256 characters long.  That is the cause and location of
> the error message you cited.
>

Wait, what?   What text string is configured to be only 256 characters?
Something must be very wrong with the Macports  installation - this file
opens just fine for me, though I haven't tested it with the Macports
version.

The "read_data()" command in ifeffit uses a line length of 2048 characters,
and should not crash on a longer line, but truncate lines to 2048
characters (silently, which is irritating when encountered, but not as bad
as crashing).     I've written and used thousands of data files with
comment and data lines longer than 256 characters.    I believe there is
(and should be) nothing in the code for reading data that sets a limit as
low as 256 characters (File names are allowed to be 512 characters!), and
nothing that should cause a crash on larger lines.

Of course it's fine to use a plugin for specific file types, and probably
wise to not rely of ifeffit's read_data() for really big files (though,
again, I regularly read files of 35 to 40 columns without a plugin).   But
this seems to me like its a Macports-only problem to me that should be
fixed.

--Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20140706/365f4dea/attachment.html>