Hi Bradley, Athena cannot use multiple CPUs. No threads, no message passing. Sorry. As for why Athena uses only 25% of the CPU in your VM is a mystery to me. Thatis not a restriction I put on the code and it is not something that normally happens with programs written using the tools Athena uses. My suspicion is that it is a VM configuration issue. As for the memory issue, the dominant use of memory by a long stretch in Athena is the memory that is statically allocated for the Ifeffit library. While it is possible to compile up Ifeffit to use more memory, that isn't likely to have an impact on how fast you LCF runs. As for the underlying problem -- the fact that it takes an hour and 40 minutes to run a large combinatorial set, the best advice I can offer is to use some prior knowledge to restrict the scope of the problem. If you know that a standard is present in a sample, you can restrict the combinatorial set to always include that one. Similarly, if you can reduce the size of you set of standards by excluding ones that you know to be unlikely or absent, that too will help. Combinatorial fitting is a blunt instrument and you seem to be using it in the most blunt way. Sorry I didn't have happier new for you. B PS: One thing that occurs to me is that you could run 2 or three instances of Athena. Each one will use its own CPU and you can be a human load leveler. If you are up for a bit of programming, I can suggest another solution which uses my new Demeter code to do the LCF. On the plus side, it would be much easier to automate your large amount of work. On the minuis side, you would have to do some programming. On Tuesday, March 20, 2012 04:22:24 pm BradleyW Miller wrote:
Hi,
I have been given access to a Virtual Machine (VM) with 8 GB of RAM and 4 CPU's with the hopes of decreasing the time required to do LCF fittings. I have large set of environmental samples from different waste streams and/or contaminate sources. Currently I have 14 standards and I wanted to run a LCF model fitting up to 4 standards. This results in 1456 fits if I selected "fit all combinations". This takes about 100 mins. to run 1 samples (I have 60+ and more coming).
It appears that Athena is constrained to using at most 25% of the available CPU and the physical memory is limited to 22% .
1) Is there a way I could change Athena's settings to take advantage of this VM's multiple CPU's and larger RAM?
2) Or if Athena can't handle multiple CPU's is there a way to increase the maximum percentage of the CPU being used? It appears that Athena is limited to accessing up to 25% of the CPU's. One CPU of the four appears to be processing a little more data than the other 3 CPU's (when viewed on the Task Manager, Performance tab). When more CPU's were added to the VM, computational time remained similar... if not worse.
3) Similarly the physical memory is limited to 22% of available memory (when viewed on the Task Manager, Performance tab) when Athena is running the LCF models as well. Could Athena's setting be changed to increase the % of RAM available for LCF fitting?
4) Any other suggestions?
I spoke with my IT guy who said he believes the software is limiting my hopes of decreasing the computational time. I'd love to call it all contaminated dirt, rent some big equipment like D-9 dozers, big trucks, a couple barges and dump it into the ocean... but we don't have that much $$$ for remediation and my kids eat fish. That's is a joke of course... my kids don't eat fish :-)
btw Thanks for your time and for use of the software!
Respectfully, Bradley W. Miller, Ph.D. Post Doctoral Fellow Oak Ridge Institute for Science and Education U. S. Environmental Protection Agency National Risk Management Research Laboratory Land Remediation and Pollution Control Division 5995 Center Hill Avenue, Cincinnati, OH 45224-1702
www.tinyurl.com/bwmiller
The great tragedy of Science—the slaying of beautiful hypothesis by an ugly fact.— Thomas H. Huxley
Views or Opinions expressed in this email is solely representative of the sender and does not represents those of the EPA or any other agency.
-- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/ifeffit/Demeter