[Ifeffit] Demeter on Ubuntu 14.04

[Bruce Ravel]

On 06/02/2014 10:17 AM, Denis Testemale wrote:
> I'd like to raise a related issue to complete my initial bug report and
> try to understand better those problems (which could occur again in the
> future). To install demeter initially (last year I think) I followed
> this procedure:
> https://gist.github.com/waveform80/3959252
> One thing that I could not understand (and still cannot, it is somehow
> beyond my limited knowledge of linux) are the subtle differences between
> installing the code locally or system-wide and in particular this
> cryptic (to me) command:
>
> eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)
>
> Indeed, the procedure I followed created a perl5 folder in my home
> folder. That is the reason why, when I mentionned that I erased all the
> folders to reinstall everything, I was referring to the folders demeter
> and perl5 in my home folder, as suggested in the 'Removing installation'
> chapter in the same page.  And I'm pretty sure that many of my problems
> are related to that. As always, being a numb, I tried different fixes,
> in particular a root installation. Everything is probably a mixture of
> approximate fixes now and makes it hard to source the problems.
> The thing is I actually don't need this perl5 folder anymore. Everything
> now runs smoothly (with the en_US LANGUAGE trick), but I sometimes have
> warning messages (in dathena for example) that it can't locate
> Encoding/FixLatin/XS.pm in @INC for example. I copied the file from the
> local perl5 folder to the system wide folder in /usr/local/lib and the
> warning disappeared.

> In fact I was wondering if I should reinstall from zero with a
> system-wide install, or if the local install made sense in some
> situations. As I understood it at the time, it was to be able to erase
> the local perl install.

> Sorry for the long message. The bottom line is that you unlocked my
> situation.


Denis,

You thought that was long?  Just wait until you see my response!

This is a valid question.  Perl (like python, lua, ruby, and many
others) is an interpreted (as opposed to a compiled language.  One of
the great strengths of an interpreted language is that its behavior
can be modified or enhanced at run time simply by making more code
available to the interpreter.

In the context of perl (the story for other languages differs only in
the details, not in the concepts), these language enhancements are
contained in files called modules.  Modules are just files containing
perl code but which live in special locations, have names that end in
".pm", and contain some special lines which make them behave as
modules.

Demeter itself is an example of a module.  If Demeter is installed in
the correct location, any perl script can be enhanced to handle XAS
data.  Demeter, in turn, uses many dozens of such modules to add all
the various features needed to make Athena, Artemis, and Hephaestus
work the way they work.

Sometimes, a module author finds it useful -- usually for reasons of
speed and performance -- to provide some functionality in the form of
code written in C, C++, or some other compiled language.  In that
case, there is a small module with some special code in it to provide
a way of making perl communicate with the compiled portion of the
code.

So, there are a two substantial problems that must somehow be solved:
(1) the modules must be in a location where a perl program can find
them, (2) the compiled bits must be compiled in a way that is
compatible with the version of perl being used.

When you get to use a packaging system -- apt-get for Ubuntu, yum on
Fedora, and so on -- both of those problems are guaranteed (by some
definition of "guarantee"!) to be solved correctly.  That is, if you
ask to install an application package, its dependencies will include a
certain version of perl and certain versions of packages for each of
the modules that the application requires.  Presumably, both of the
problems are solved correctly by the person maintaining the packages.

Sadly, there are not packages for Demeter for any linux system.  The
discussion of why is best left for another posting on the mailing
list.  The bottom line is that, when installing from source on a linux
machine, the chore of solving both problems falls on you, the
end-user.  The perl environment tries to help you solve this problem,
but there are pitfalls.

If you can run the "./Build install" step as root, then it is likely
that you will have no problem.

If you do not have root, then solving the problems is a bit harder.
Simply making a "perl5/" folder in your home directory is a good
solution, but it is likely to cause problems when you upgrade your
system.  This, in fact, was the problem you ran into.  The problem is
that when you upgraded Ubuntu, you upgraded to a later version of
perl.  When Demeter's build script checked to see what modules needed
to be installed, it found the modules built for the earlier version of
perl and decided -- incorrectly -- that it did not need to reinstall
those modules.  Some of them, however, had compiled bits.  Those had
been compiled against the earlier version of perl and could not be
used with the new version.

To solve this versioning problem on a computer for which the user does
not have the authority to install as root, the perl community came up
with a solution called "local::lib".  This sets up a "perl5/" folder
in your home directory *AND* uses it in a way that deals with perl
versions correctly *AND* modifies your shell login script to use it
correctly.  The cryptic command you quoted is part of the magic of
setting up local::lib to do all that.


Finally, how would you recover gracefully from the situation that you
found yourself in?  Well, there are several things that help:

  1. The command "perl -V" displays the full list of places where perl
     looks for modules.  It's the part under "@INC:".  On my Ubuntu
     computer, perl modules installed by the user go to
     /usr/local/share/perl/5.14.2/.  If I wanted to, say, simply remove
     the Color::Rgb module, I would do

        sudo rm /usr/local/share/perl/5.14.2/Color/Rgb.pm

  2. The same thing can be done using the "cpanp" program, which is
     standard part of the perl installation.  Doing

        sudo cpanp u Color::Rgb

     will uninstall the module.  The advantage of using cpanp for this
     is that it will unistall every part of the module.  For some
     complicated things, that could involved dozens or even hundreds of
     files.

  3. If you suspect that you have a version clash (you'll recall that
     the error messages you sent me stated that somewhat clearly), you
     can force an update of a module.  For example, you ran into this
     problem with the module DateTime.  You could force an upgrade to
     that module, again using cpanp:

         sudo cpanp i DateTime --force

Those are all useful tools to solving perl problems, but really the
best solution (if system-specfic packages are not available) is to
install normally as root or to use local::lib.  An ad hoc solution
(i.e. making your own perl5/ directory and installing there) sort of
works, but can lead to problems if you're not careful.

Hope that helps,
B

PS: If your take-away message from this absurdly long post is that it
would be helpful to have Demeter packages maintained for the major
Linux systems, then you read it correctly!


-- 
  Bruce Ravel  ------------------------------------ bravel at bnl.gov

  National Institute of Standards and Technology
  Synchrotron Science Group at NSLS --- Beamlines U7A, X24A, X23A2
  Building 535A
  Upton NY, 11973

  Homepage:    http://xafs.org/BruceRavel
  Software:    https://github.com/bruceravel