[Ifeffit] How to use Larch as backend for Athena and Artemis on Windows

Thu Aug 17 21:27:34 CDT 2017

Hi Hiroyuki, Bruce, Carmelo, Luke, all,

Sorry for the delay in responding.  I was at the Q2XAFS workshop spending
some of my time discussing Larch.

As a relevant aside, at this meeting Yohei Uemura  showed me a very nice
Python GUI that he's been working on for fitting multiple EXAFS spectra.
He also had a very interesting poster of results of a rather amazing study
of photo-excited WO3.  If I understand correctly, Yohei used Larch as the
EXAFS calculation engine for ~30 million configurations using around 30
core-hours (that is, it took around 2 real hours on a 14 core machine).
I think we can conclude it will be good to handle 100 spectra ;).

For installing Larch, Hiroyuki's approach can work but is pretty involved.
We're recommending Anaconda Python (as Luke mentioned), as it includes many
of the needed packages by default, and has a good package manager to
install Larch and everything it needs.  It also makes pushing updates easy
for us (and easy for the end user to pull and install the updates too).

Also, Larch on Python 3.6 works for me, and I believe the XAFS code should
be clean.  But I also know we're not testing every corner of the code as
well as we should, and I would not be surprised to see some Python3 bugs.
These may be more likely if you're using HDF5 files and/or the XRF Mapping
GUI tools.

Anyway, I'm very happy to hear others getting the Larch+Demeter combination
working on Windows! For sure, please let us know what could make that
easier and better.

Cheers,

On Thu, Aug 17, 2017 at 4:21 AM, ASAKURA, Hiroyuki <
asakura at moleng.kyoto-u.ac.jp> wrote:

> Dear Bruce,
>
> I do appreciate your prompt response!
>
> Now, I managed to run Athena with Larch backend on Windows 10 and handle
> about 100 XAS data at once.
> Athena with Ifeffit backend stalls when handling about 40 XAS spectra at
> least in my case.
> Unfortunately, there seems to be a bit severe performance issue with a
> large number of spectra, as noted.
>
> Here is what I did.
>
> 0. Install an official Git package
> 1-1. Download "64-bit Git for Windows Setup" via
> https://git-scm.com/download/win
> 1-2. Install Git with default options except for "Use Git from the Windows
> Command Prompt" and "Use Windows' default console window"
> Note: You can use any options you want.
>
> 1. Install an official Demeter package
> 1-1. Download "Demeter 0.9.25 (32 bit)" via https://bruceravel.github.io/d
> emeter/
> 1-2. Install Demeter with default options
> Note: I cannot run 64 bit version with Larch.
>
> 2. Install an official Python package
> 2-1. Download "Windows x86-64 web-based installer" via
> https://www.python.org/downloads/release/python-362/
> 2-2. Custom Install Python with default options except for "Add Python 3.6
> to PATH" (checked)
>
> 3. Install unofficial Numpy and Scipy, Python libraries
> 3-1. Download numpy‑1.13.1+mkl‑cp36‑cp36m‑win_amd64.whl and
> scipy‑0.19.1‑cp36‑cp36m‑win_amd64.whl via http://www.lfd.uci.edu/~gohlke
> /pythonlibs/
> 3-2. pip install numpy-1.13.1+mkl-cp36-cp36m-win_amd64.whl
> 3-3. pip install scipy‑0.19.1‑cp36‑cp36m‑win_amd64.whl
>
> 4. Install Larch and its dependencies
> 4-1. Start a command prompt
> 4-2. git clone https://github.com/xraypy/xraylarch.git
> 4-3. cd xraylarch
> 4-4. pip install lmfit
> 4-5. pip install h5py
> 4-6. pip install pillow
> 4-7. pip install requests
> 4-8. pip install sqlalchemy
> 4-9. pip install matplotlib
> 4-10. pip install pypiwin32
> 4-11. pip install wxpython
> 4-12. pip install wxmplot
> 4-13. pip install wxutils
> 4-14. python setup.py install
> 4-15. larch --version
> 4-16. You will see "larch command-line version 0.2".
> Note: I installed Larch dependencies except for pyFAI, CifFile, fabio,
> tomopy, scikit-image.
>
> 5. Run Larch and Athena with Larch
> 5-1. Start a new command prompt
> 5-2. larch -rc
> 5-3. Start another new command prompt
> 5-4.cd %APPDATA%\DemeterPerl\perl\site\bin
> 5-5. lathena.bat
>
> It works.
> Athena is the most user-friendly XAS processing tool I have used.
>
> Thanks so much!
>
> Best regards,
>
> ASAKURA, Hiroyuki
>
>
>
> On 2017/08/17 2:17, Bruce Ravel wrote:
>
>>
>> Hi Hiroyuki,
>>
>> Sorry I didn't respond yesterday.  Yesterday was a bit of crazy day...
>>
>> I suppose that this is still a work in progress.  I believe that Demeter
>> works with Larch on Windows, but it is certainly not as easy or convenient
>> yet as it should be.
>>
>> In the place where Demeter got installed on your computer, there is a
>> folder called \perl\site\bin -- you can look at the properties of the
>> Athena desktop icon to see where that is.  In that folder is the
>> "dathena.bat" file which the Athena desktop icon points at.
>>
>> You should also find a file called "lathena.bat".  Try running that or
>> pointing your desktop icon at it instead.
>>
>> The difference between "lathena.bat" and "dathena.bat" is line 25 that
>> sets an environment variable that Demeter uses to decide which of ifeffit
>> or larch to use.
>>
>> Obviously, there are number of other ways to get this environment
>> variable set.  The bottom line is, yes, there is a switch.  The switch is
>> the DEMETER_BACKEND environment variable.
>>
>> Unfortunately, that switch cannot be in the configuration files, although
>> what you tired in item 2 of your email was a sensible guess. It turns out
>> that the choice of larch/ifeffit has to made long before the configuration
>> files are read.
>>
>> Do let me know how you get on.  While Demeter + larch should work, there
>> are still almost certainly problems.  I need to get a new version of the
>> Demeter installer out soon, so it would be nice to get some feedback in
>> this area.
>>
>> As for using Demeter + larch + 100s of file -- it should work after a
>> fashion.  But I worry that Athena's performance might not be what you are
>> hoping for.  Athena does a lot of things that are more user-friendly than
>> performance-friendly....
>>
>>
>> Cheers,
>> B
>>
>>
>> On 08/15/2017 06:10 AM, ASAKURA, Hiroyuki wrote:
>>
>>> Dear all,
>>>
>>> I would like to use the latest Larch as backend for Athena and Artemis
>>> on Windows to handle hundreds of XAS data (, hopefully).
>>> But, I'm stuck. I would appreciate any comments and suggestions.
>>>
>>> Matt announced it is now possible on this list, but I still cannot do it
>>> on Windows.
>>>
>>> [Ifeffit] Larch 0.9.33
>>> http://millenia.cars.aps.anl.gov/pipermail/ifeffit/2017-Apri
>>> l/009091.html
>>>
>>> The "server mode" for Athena and Artemis now works on all platforms and I
>>>> strongly encourage everyone to use this over the ifeffit back end.
>>>>
>>>
>>> Here is what I did.
>>>
>>> Environment:
>>> Windows 10 Pro (Japanese)
>>> Demeter 0.9.25 (via official website)
>>> Python 3.6 (official distribution)
>>>
>>> 1. Installation of the latest Larch
>>>
>>> git clone https://github.com/xraypy/xraylarch.git (sha1:
>>>> 5a372cfe1368a8ac10c2ebfc5bef4e9a9a325f6d)
>>>> cd xraylarch
>>>> python setup.py install
>>>>
>>> # Some dependency related to XRD is not fulfilled.
>>>
>>> 2. Start Larch server
>>>
>>> cd %LOCALAPPDATA%\Programs\Python\Python36\Scripts
>>>> python larch_server start
>>>>
>>> larch_server port=4966: started
>>>
>>> I found a parameter of peakfit backend in demeter.ini and changed to
>>> larch, but did not work (as expected).
>>>
>>> In the dathena.log, Athena still uses ifeffit as backend.
>>>
>>> perl version: v5.18.2
>>>> backend: ifeffit
>>>>
>>>
>>> Should I compile Demeter package by myself to use the Larch backend?
>>> Or, is there any "switch" to use Larch?
>>>
>>> Thanks in advance,
>>>
>>> ASAKURA, Hiroyuki
>>>
>>>
>>>
>>
>>
> --
> ASAKURA, Hiroyuki (Ph.D)
> Program-Specific Assistant Professor (ESICB)
> T. Tanaka Lab., ESICB, Kyoto University, Japan
> asakura at moleng.kyoto-u.ac.jp
> http://www.moleng.kyoto-u.ac.jp/~moleng_04/asakura/
> _______________________________________________
> Ifeffit mailing list
> Ifeffit at millenia.cars.aps.anl.gov
> http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
> Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit
>

-- 
--Matt Newville <newville at cars.uchicago.edu> 630-252-0431
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://millenia.cars.aps.anl.gov/pipermail/ifeffit/attachments/20170817/a84c43d9/attachment.html>