Bruce, I'm afraid I'm not actually a Athena user, so could you clarify something for me in regards the discussion of data groups in the "Care and feeding of Athena" thread? It looks like users should be wary starting around 45-50 data groups. Does that translate to 45-50 raw datafile that are "open" for active processing/graphing (i.e. are checked in the top right hand pane), or 45-50 raw datafiles that athena is aware of (are available to BE checked in the upper right hand pane)? Was there a resolution to the issue that Mark was saying that 150+ data groups still only uses up 5% of IEFFIT's heap? Thanks!! David. On Mon, 15 Feb 2010, Bruce Ravel wrote:
Hi Dave,
I suspect that the issue is related to the fact that, under the hood, Athena is using a library written in Fortran (Ifeffit is written in Fortran). Fortran does not do dynamic memory allocation. As Athena projects get bigger and bigger, you will eventually use up all of the statically allocated memory. Unless you have done something special in the compilation or run-time environment, there is probably nothing stopping the process from then touching memory outside of its static block. Then weird things will happen.
This is explained in some detail here:
http://millenia.cars.aps.anl.gov/pipermail/ifeffit/2007-May/007577.html
(Yes, this does indeed represent a way to attack a computer, but its a hammer, not a scalpel. It is hard to imagine how to make something worse than what you have already observed happen.)
This issue is well known and long discussed on the mailing list. It is also known (or should be...) to all the staff scientists at your beamline, who should be warning their users to limit the size of their project files by saving and restarting every 40 or 50 or so scans.
Alternately, you could compile up a version of Ifeffit with a much larger statically allocated block of memory for use at the beamlines. (Carlo would certainly know how to do so, Ken also.)
I am open to other suggestions, but your statement that it only happens after long use suggests it is the memory thing. It seems odd but certainly not impossible that it would freeze X.
B
On Monday 15 February 2010, 02:28:35 pm, David Ehle wrote:
Bruce,
Sorry for the accusatory subject line - I can't prove Athena is the culprit but I figured that others with the same problem would be searching for similar terms.
Running Athena from the current debian package for Lenny (2:1.2.11d-1) we have been seeing desktop freezes after some hard to characterize period of use. The time period seems to usually be a few hours, and there is a belief that it is connected to having more than a few raw data files open in Athena.
Users have previously seen Athena crash/freeze when they have "too many" files open, or Athena open for "too long" but I haven't been able to get any firm numbers. The concern is that while previously when Athena crashed or froze, only Athena was effected. Now it seems that the whole desktop becomes non-responsive.
The reason I'm suggesting there is a link between Athena and the Desktop (XFCE4) problem is that if you connect to the system via SSH and kill Athena, the Desktop becomes unfrozen and the user can continue with what they are doing.
We have seen this occur on both beamlines, with different users and different data sets so I doubt it is data specific, but can send your some sample files if that will be useful.
In one case I was able to collect an error message from the .xsession-errors file of a user who was experiencing the problem: ** (xfwm4:21854): WARNING **: Unmanaged net_wm_state (window 0x3800008) X Error: BadWindow (invalid Window parameter) 3 Major opcode: 7 Minor opcode: 0 Resource id: 0x329 kbuildsycoca running... Reusing existing ksycoca kbuildsycoca running... Reusing existing ksycoca kbuildsycoca running... Reusing existing ksycoca Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 3891. Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 4700. Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 3891. Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 4700. Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 3891. Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 4700. Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 3891. Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 4700. Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 3891. Use of uninitialized value in concatenation (.) or string at /usr/bin/athena line 4700.
I'm afraid I don't know if the lines before the lines that explicitly mention athena are related, or if these messages have anything to do with the desktop freezing problem - I think I have the seen the freeze without the errors.
Please advise if we can provide you with other information to make figuring out easier.
When this problem occurs at 3AM it really upsets our users (which tends to upset the staff when they get called about it...) so we have started running a second system just for Athena. Can you make any suggestions on the minimum vs optimal hardware for Athena? For tracking many (30+) raw data files to see how a run is progressing, can you suggest minimum RAM, CPU power, # of Cores?
Thanks!! Please CC ehle@iit.edu with any responses as I'm not an active subscriber to this list.
David Ehle MRCAT -IIT _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
-- Bruce Ravel ------------------------------------ bravel@bnl.gov
National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973
My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/