On 06/02/2014 10:17 AM, Denis Testemale wrote:

I'd like to raise a related issue to complete my initial bug report and try to understand better those problems (which could occur again in the future). To install demeter initially (last year I think) I followed this procedure: https://gist.github.com/waveform80/3959252 One thing that I could not understand (and still cannot, it is somehow beyond my limited knowledge of linux) are the subtle differences between installing the code locally or system-wide and in particular this cryptic (to me) command:

eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)

Indeed, the procedure I followed created a perl5 folder in my home folder. That is the reason why, when I mentionned that I erased all the folders to reinstall everything, I was referring to the folders demeter and perl5 in my home folder, as suggested in the 'Removing installation' chapter in the same page. And I'm pretty sure that many of my problems are related to that. As always, being a numb, I tried different fixes, in particular a root installation. Everything is probably a mixture of approximate fixes now and makes it hard to source the problems. The thing is I actually don't need this perl5 folder anymore. Everything now runs smoothly (with the en_US LANGUAGE trick), but I sometimes have warning messages (in dathena for example) that it can't locate Encoding/FixLatin/XS.pm in @INC for example. I copied the file from the local perl5 folder to the system wide folder in /usr/local/lib and the warning disappeared.

In fact I was wondering if I should reinstall from zero with a system-wide install, or if the local install made sense in some situations. As I understood it at the time, it was to be able to erase the local perl install.

Sorry for the long message. The bottom line is that you unlocked my situation.

Denis, You thought that was long? Just wait until you see my response! This is a valid question. Perl (like python, lua, ruby, and many others) is an interpreted (as opposed to a compiled language. One of the great strengths of an interpreted language is that its behavior can be modified or enhanced at run time simply by making more code available to the interpreter. In the context of perl (the story for other languages differs only in the details, not in the concepts), these language enhancements are contained in files called modules. Modules are just files containing perl code but which live in special locations, have names that end in ".pm", and contain some special lines which make them behave as modules. Demeter itself is an example of a module. If Demeter is installed in the correct location, any perl script can be enhanced to handle XAS data. Demeter, in turn, uses many dozens of such modules to add all the various features needed to make Athena, Artemis, and Hephaestus work the way they work. Sometimes, a module author finds it useful -- usually for reasons of speed and performance -- to provide some functionality in the form of code written in C, C++, or some other compiled language. In that case, there is a small module with some special code in it to provide a way of making perl communicate with the compiled portion of the code. So, there are a two substantial problems that must somehow be solved: (1) the modules must be in a location where a perl program can find them, (2) the compiled bits must be compiled in a way that is compatible with the version of perl being used. When you get to use a packaging system -- apt-get for Ubuntu, yum on Fedora, and so on -- both of those problems are guaranteed (by some definition of "guarantee"!) to be solved correctly. That is, if you ask to install an application package, its dependencies will include a certain version of perl and certain versions of packages for each of the modules that the application requires. Presumably, both of the problems are solved correctly by the person maintaining the packages. Sadly, there are not packages for Demeter for any linux system. The discussion of why is best left for another posting on the mailing list. The bottom line is that, when installing from source on a linux machine, the chore of solving both problems falls on you, the end-user. The perl environment tries to help you solve this problem, but there are pitfalls. If you can run the "./Build install" step as root, then it is likely that you will have no problem. If you do not have root, then solving the problems is a bit harder. Simply making a "perl5/" folder in your home directory is a good solution, but it is likely to cause problems when you upgrade your system. This, in fact, was the problem you ran into. The problem is that when you upgraded Ubuntu, you upgraded to a later version of perl. When Demeter's build script checked to see what modules needed to be installed, it found the modules built for the earlier version of perl and decided -- incorrectly -- that it did not need to reinstall those modules. Some of them, however, had compiled bits. Those had been compiled against the earlier version of perl and could not be used with the new version. To solve this versioning problem on a computer for which the user does not have the authority to install as root, the perl community came up with a solution called "local::lib". This sets up a "perl5/" folder in your home directory *AND* uses it in a way that deals with perl versions correctly *AND* modifies your shell login script to use it correctly. The cryptic command you quoted is part of the magic of setting up local::lib to do all that. Finally, how would you recover gracefully from the situation that you found yourself in? Well, there are several things that help: 1. The command "perl -V" displays the full list of places where perl looks for modules. It's the part under "@INC:". On my Ubuntu computer, perl modules installed by the user go to /usr/local/share/perl/5.14.2/. If I wanted to, say, simply remove the Color::Rgb module, I would do sudo rm /usr/local/share/perl/5.14.2/Color/Rgb.pm 2. The same thing can be done using the "cpanp" program, which is standard part of the perl installation. Doing sudo cpanp u Color::Rgb will uninstall the module. The advantage of using cpanp for this is that it will unistall every part of the module. For some complicated things, that could involved dozens or even hundreds of files. 3. If you suspect that you have a version clash (you'll recall that the error messages you sent me stated that somewhat clearly), you can force an update of a module. For example, you ran into this problem with the module DateTime. You could force an upgrade to that module, again using cpanp: sudo cpanp i DateTime --force Those are all useful tools to solving perl problems, but really the best solution (if system-specfic packages are not available) is to install normally as root or to use local::lib. An ad hoc solution (i.e. making your own perl5/ directory and installing there) sort of works, but can lead to problems if you're not careful. Hope that helps, B PS: If your take-away message from this absurdly long post is that it would be helpful to have Demeter packages maintained for the major Linux systems, then you read it correctly! -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Science Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 Homepage: http://xafs.org/BruceRavel Software: https://github.com/bruceravel