[Ifeffit] data point limits in Athena?

[Robert Gordon]

Hi Jorg,

Matt is on to the problem...Too many numbers... I deleted
columns 2 - 58. Working with columns labelled Ioni1,2 and 3
was ok, with little problem reading it in to Athena 0.9.17 even with the
2460
lines. I say "little" because it does take several seconds on my system
for Athena to read in the file so I can select columns. If I reduce it to
just the 4 columns that are of most value to an end user (E, and detectors),
it opens in about 1s. Notepad takes about that long to open the original
file.
So the problem seems to be not so much one of length, but girth - an obese
datafile trying to squeeze into a hefty array-size/memory limit.

I would not consider this an Athena issue. There are numerous columns
in the datafile that are constants, many zero. The number of lines could
also be improved.
The data was collected at 0.3 eV steps from 6932 to 7700 eV and that
certainly
oversamples for most of that range.

..and this is the second instance I have seen of obese datafiles...starting
to wonder how prevalent this issue is or will be, particularly when
beamlines
use large multi-element detectors.

regards,
Robert




On Sun, Aug 4, 2013 at 3:00 PM, Matt Newville <newville at cars.uchicago.edu>wrote:

> Jorg,
>
> On Sun, Aug 4, 2013 at 12:49 PM, Göttlicher, Jörg
> <joerg.goettlicher at kit.edu> wrote:
> > Hello Bruce,
> >
> > thanks a lot for your fast reply.
> > I did not want to attach an example file immediately when submiting a
> question to the mailing list.
> > But there is no problem to send you some files. In the attachments you
> will now find the original data file for trying to reproduce my problems
> and hopefully to find a solution.
> >
> > I attach also the files reduced to 1711 and 1627 data points (just by
> trial and error. I checked 1711 (didn't work), and 1627 (worked). The limit
> might be somewhere inbetween.
> >
> > Attached is their read in in Athena, too.
> >
> > The counters are the following:
> > 1 Energy
> > Nominator 62 Absorption (Transmission), sample
> >
> > Reference
> > Nominator 63 Reference (Fe foil)
>
> If you're having difficulty reading these data files, I think you may
> want to convert them into a more sensible format with say two or three
> columns before trying to read it into athena.   That might help
> isolate where any problem might be.
>
> --Matt
>
> _______________________________________________
> Ifeffit mailing list
> Ifeffit at millenia.cars.aps.anl.gov
> http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20130805/fada54fd/attachment.html>