[Ifeffit] Decrease LCF Fitting Computational Time in Athena

[Matt Newville]

Hi Bradley, Bruce,

On Wed, Mar 21, 2012 at 8:44 AM, Bruce Ravel <bravel at bnl.gov> wrote:
>
> Hi Bradley,
>
> Athena cannot use multiple CPUs.  No threads, no message passing.
> Sorry.
>
> As for why Athena uses only 25% of the CPU in your VM is a mystery to
> me.  Thatis not a restriction I put on the code and it is not
> something that normally happens with programs written using the tools
> Athena uses.  My suspicion is that it is a VM configuration issue.

I think Bradley said 'using at most 25% of the available CPU', which
probably means 'at most 1 CPU core at a time', which is correct.  The
OS may be able to switch which core is used for Athena/Ifeffit, but
only one can be used at any time.

> As for the memory issue, the dominant use of memory by a long stretch
> in Athena is the memory that is statically allocated for the Ifeffit
> library.  While it is possible to compile up Ifeffit to use more
> memory, that isn't likely to have an impact on how fast you LCF runs.

I doubt memory (22% of 8Gb) is an issue here.   Ifeffit is 32-bit, and
will not ever use more than ~2Gb on Windows, though the GUI could add
to that, and might be able to go above 2Gb.  Anyway, memory is not
being swapped from RAM to disk. Low-level cache memory will definitely
be swapped to RAM, which is likely to actually limit run time, not
exactly the number of cores.  That's all to say that the mathematical
problem probably could be solved much better by making use of multiple
cores.  But Athena/Ifeffit won't be doing that anytime soon.

> As for the underlying problem -- the fact that it takes an hour and 40
> minutes to run a large combinatorial set, the best advice I can offer
> is to use some prior knowledge to restrict the scope of the problem.
> If you know that a standard is present in a sample, you can restrict
> the combinatorial set to always include that one.  Similarly, if you
> can reduce the size of you set of standards by excluding ones that you
> know to be unlikely or absent, that too will help.  Combinatorial
> fitting is a blunt instrument and you seem to be using it in the most
> blunt way.

Right, throwing more spectra at Athena's LCF is going to be expensive.
 The combinations are tried one at a time, and that won't ever be
changed.

OTOH, recasting the basic problem in terms of linear algebra might
help, and might be able to make use of multiple cores, but that is not
something Athena is going to do with Ifeffit 1.2, or probably ever.

> PS:  One thing that occurs to me is that you could run 2 or three
> instances of Athena.  Each one will use its own CPU and you can be a
> human load leveler.

Yes, running 4 instances of Athena would help, as each could
presumably use a different core.  Even if they're sharing physical
cores, it might actually go faster, depending on caching.   Splitting
the problem in half 4 different ways and ensuring overlap is likely to
take less real time.

--Matt