How to use Larch as backend for Athena and Artemis on Windows
Dear all, I would like to use the latest Larch as backend for Athena and Artemis on Windows to handle hundreds of XAS data (, hopefully). But, I'm stuck. I would appreciate any comments and suggestions. Matt announced it is now possible on this list, but I still cannot do it on Windows. [Ifeffit] Larch 0.9.33 http://millenia.cars.aps.anl.gov/pipermail/ifeffit/2017-April/009091.html
The "server mode" for Athena and Artemis now works on all platforms and I strongly encourage everyone to use this over the ifeffit back end.
Here is what I did. Environment: Windows 10 Pro (Japanese) Demeter 0.9.25 (via official website) Python 3.6 (official distribution) 1. Installation of the latest Larch
git clone https://github.com/xraypy/xraylarch.git (sha1: 5a372cfe1368a8ac10c2ebfc5bef4e9a9a325f6d) cd xraylarch python setup.py install # Some dependency related to XRD is not fulfilled.
2. Start Larch server
cd %LOCALAPPDATA%\Programs\Python\Python36\Scripts python larch_server start larch_server port=4966: started
I found a parameter of peakfit backend in demeter.ini and changed to larch, but did not work (as expected). In the dathena.log, Athena still uses ifeffit as backend.
perl version: v5.18.2 backend: ifeffit
Should I compile Demeter package by myself to use the Larch backend? Or, is there any "switch" to use Larch? Thanks in advance, ASAKURA, Hiroyuki -- ASAKURA, Hiroyuki (Ph.D) T. Tanaka Lab., ESICB, Kyoto University, Japan asakura@moleng.kyoto-u.ac.jp http://www.moleng.kyoto-u.ac.jp/~moleng_04/asakura/
Hi Hiroyuki, Sorry I didn't respond yesterday. Yesterday was a bit of crazy day... I suppose that this is still a work in progress. I believe that Demeter works with Larch on Windows, but it is certainly not as easy or convenient yet as it should be. In the place where Demeter got installed on your computer, there is a folder called \perl\site\bin -- you can look at the properties of the Athena desktop icon to see where that is. In that folder is the "dathena.bat" file which the Athena desktop icon points at. You should also find a file called "lathena.bat". Try running that or pointing your desktop icon at it instead. The difference between "lathena.bat" and "dathena.bat" is line 25 that sets an environment variable that Demeter uses to decide which of ifeffit or larch to use. Obviously, there are number of other ways to get this environment variable set. The bottom line is, yes, there is a switch. The switch is the DEMETER_BACKEND environment variable. Unfortunately, that switch cannot be in the configuration files, although what you tired in item 2 of your email was a sensible guess. It turns out that the choice of larch/ifeffit has to made long before the configuration files are read. Do let me know how you get on. While Demeter + larch should work, there are still almost certainly problems. I need to get a new version of the Demeter installer out soon, so it would be nice to get some feedback in this area. As for using Demeter + larch + 100s of file -- it should work after a fashion. But I worry that Athena's performance might not be what you are hoping for. Athena does a lot of things that are more user-friendly than performance-friendly.... Cheers, B On 08/15/2017 06:10 AM, ASAKURA, Hiroyuki wrote:
Dear all,
I would like to use the latest Larch as backend for Athena and Artemis on Windows to handle hundreds of XAS data (, hopefully). But, I'm stuck. I would appreciate any comments and suggestions.
Matt announced it is now possible on this list, but I still cannot do it on Windows.
[Ifeffit] Larch 0.9.33 http://millenia.cars.aps.anl.gov/pipermail/ifeffit/2017-April/009091.html
The "server mode" for Athena and Artemis now works on all platforms and I strongly encourage everyone to use this over the ifeffit back end.
Here is what I did.
Environment: Windows 10 Pro (Japanese) Demeter 0.9.25 (via official website) Python 3.6 (official distribution)
1. Installation of the latest Larch
git clone https://github.com/xraypy/xraylarch.git (sha1: 5a372cfe1368a8ac10c2ebfc5bef4e9a9a325f6d) cd xraylarch python setup.py install # Some dependency related to XRD is not fulfilled.
2. Start Larch server
cd %LOCALAPPDATA%\Programs\Python\Python36\Scripts python larch_server start larch_server port=4966: started
I found a parameter of peakfit backend in demeter.ini and changed to larch, but did not work (as expected).
In the dathena.log, Athena still uses ifeffit as backend.
perl version: v5.18.2 backend: ifeffit
Should I compile Demeter package by myself to use the Larch backend? Or, is there any "switch" to use Larch?
Thanks in advance,
ASAKURA, Hiroyuki
-- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Science Group at NSLS-II Building 743, Room 114 Upton NY, 11973 Homepage: http://bruceravel.github.io/home/ Software: https://github.com/bruceravel Demeter: http://bruceravel.github.io/demeter/
Dear Bruce, I do appreciate your prompt response! Now, I managed to run Athena with Larch backend on Windows 10 and handle about 100 XAS data at once. Athena with Ifeffit backend stalls when handling about 40 XAS spectra at least in my case. Unfortunately, there seems to be a bit severe performance issue with a large number of spectra, as noted. Here is what I did. 0. Install an official Git package 1-1. Download "64-bit Git for Windows Setup" via https://git-scm.com/download/win 1-2. Install Git with default options except for "Use Git from the Windows Command Prompt" and "Use Windows' default console window" Note: You can use any options you want. 1. Install an official Demeter package 1-1. Download "Demeter 0.9.25 (32 bit)" via https://bruceravel.github.io/demeter/ 1-2. Install Demeter with default options Note: I cannot run 64 bit version with Larch. 2. Install an official Python package 2-1. Download "Windows x86-64 web-based installer" via https://www.python.org/downloads/release/python-362/ 2-2. Custom Install Python with default options except for "Add Python 3.6 to PATH" (checked) 3. Install unofficial Numpy and Scipy, Python libraries 3-1. Download numpy‑1.13.1+mkl‑cp36‑cp36m‑win_amd64.whl and scipy‑0.19.1‑cp36‑cp36m‑win_amd64.whl via http://www.lfd.uci.edu/~gohlke/pythonlibs/ 3-2. pip install numpy-1.13.1+mkl-cp36-cp36m-win_amd64.whl 3-3. pip install scipy‑0.19.1‑cp36‑cp36m‑win_amd64.whl 4. Install Larch and its dependencies 4-1. Start a command prompt 4-2. git clone https://github.com/xraypy/xraylarch.git 4-3. cd xraylarch 4-4. pip install lmfit 4-5. pip install h5py 4-6. pip install pillow 4-7. pip install requests 4-8. pip install sqlalchemy 4-9. pip install matplotlib 4-10. pip install pypiwin32 4-11. pip install wxpython 4-12. pip install wxmplot 4-13. pip install wxutils 4-14. python setup.py install 4-15. larch --version 4-16. You will see "larch command-line version 0.2". Note: I installed Larch dependencies except for pyFAI, CifFile, fabio, tomopy, scikit-image. 5. Run Larch and Athena with Larch 5-1. Start a new command prompt 5-2. larch -rc 5-3. Start another new command prompt 5-4.cd %APPDATA%\DemeterPerl\perl\site\bin 5-5. lathena.bat It works. Athena is the most user-friendly XAS processing tool I have used. Thanks so much! Best regards, ASAKURA, Hiroyuki On 2017/08/17 2:17, Bruce Ravel wrote:
Hi Hiroyuki,
Sorry I didn't respond yesterday. Yesterday was a bit of crazy day...
I suppose that this is still a work in progress. I believe that Demeter works with Larch on Windows, but it is certainly not as easy or convenient yet as it should be.
In the place where Demeter got installed on your computer, there is a folder called \perl\site\bin -- you can look at the properties of the Athena desktop icon to see where that is. In that folder is the "dathena.bat" file which the Athena desktop icon points at.
You should also find a file called "lathena.bat". Try running that or pointing your desktop icon at it instead.
The difference between "lathena.bat" and "dathena.bat" is line 25 that sets an environment variable that Demeter uses to decide which of ifeffit or larch to use.
Obviously, there are number of other ways to get this environment variable set. The bottom line is, yes, there is a switch. The switch is the DEMETER_BACKEND environment variable.
Unfortunately, that switch cannot be in the configuration files, although what you tired in item 2 of your email was a sensible guess. It turns out that the choice of larch/ifeffit has to made long before the configuration files are read.
Do let me know how you get on. While Demeter + larch should work, there are still almost certainly problems. I need to get a new version of the Demeter installer out soon, so it would be nice to get some feedback in this area.
As for using Demeter + larch + 100s of file -- it should work after a fashion. But I worry that Athena's performance might not be what you are hoping for. Athena does a lot of things that are more user-friendly than performance-friendly....
Cheers, B
On 08/15/2017 06:10 AM, ASAKURA, Hiroyuki wrote:
Dear all,
I would like to use the latest Larch as backend for Athena and Artemis on Windows to handle hundreds of XAS data (, hopefully). But, I'm stuck. I would appreciate any comments and suggestions.
Matt announced it is now possible on this list, but I still cannot do it on Windows.
[Ifeffit] Larch 0.9.33 http://millenia.cars.aps.anl.gov/pipermail/ifeffit/2017-April/009091.html
The "server mode" for Athena and Artemis now works on all platforms and I strongly encourage everyone to use this over the ifeffit back end.
Here is what I did.
Environment: Windows 10 Pro (Japanese) Demeter 0.9.25 (via official website) Python 3.6 (official distribution)
1. Installation of the latest Larch
git clone https://github.com/xraypy/xraylarch.git (sha1: 5a372cfe1368a8ac10c2ebfc5bef4e9a9a325f6d) cd xraylarch python setup.py install # Some dependency related to XRD is not fulfilled.
2. Start Larch server
cd %LOCALAPPDATA%\Programs\Python\Python36\Scripts python larch_server start larch_server port=4966: started
I found a parameter of peakfit backend in demeter.ini and changed to larch, but did not work (as expected).
In the dathena.log, Athena still uses ifeffit as backend.
perl version: v5.18.2 backend: ifeffit
Should I compile Demeter package by myself to use the Larch backend? Or, is there any "switch" to use Larch?
Thanks in advance,
ASAKURA, Hiroyuki
-- ASAKURA, Hiroyuki (Ph.D) Program-Specific Assistant Professor (ESICB) T. Tanaka Lab., ESICB, Kyoto University, Japan asakura@moleng.kyoto-u.ac.jp http://www.moleng.kyoto-u.ac.jp/~moleng_04/asakura/
Hi Hiroyuki, Bruce, Carmelo, Luke, all, Sorry for the delay in responding. I was at the Q2XAFS workshop spending some of my time discussing Larch. As a relevant aside, at this meeting Yohei Uemura showed me a very nice Python GUI that he's been working on for fitting multiple EXAFS spectra. He also had a very interesting poster of results of a rather amazing study of photo-excited WO3. If I understand correctly, Yohei used Larch as the EXAFS calculation engine for ~30 million configurations using around 30 core-hours (that is, it took around 2 real hours on a 14 core machine). I think we can conclude it will be good to handle 100 spectra ;). For installing Larch, Hiroyuki's approach can work but is pretty involved. We're recommending Anaconda Python (as Luke mentioned), as it includes many of the needed packages by default, and has a good package manager to install Larch and everything it needs. It also makes pushing updates easy for us (and easy for the end user to pull and install the updates too). Also, Larch on Python 3.6 works for me, and I believe the XAFS code should be clean. But I also know we're not testing every corner of the code as well as we should, and I would not be surprised to see some Python3 bugs. These may be more likely if you're using HDF5 files and/or the XRF Mapping GUI tools. Anyway, I'm very happy to hear others getting the Larch+Demeter combination working on Windows! For sure, please let us know what could make that easier and better. Cheers, On Thu, Aug 17, 2017 at 4:21 AM, ASAKURA, Hiroyuki < asakura@moleng.kyoto-u.ac.jp> wrote:
Dear Bruce,
I do appreciate your prompt response!
Now, I managed to run Athena with Larch backend on Windows 10 and handle about 100 XAS data at once. Athena with Ifeffit backend stalls when handling about 40 XAS spectra at least in my case. Unfortunately, there seems to be a bit severe performance issue with a large number of spectra, as noted.
Here is what I did.
0. Install an official Git package 1-1. Download "64-bit Git for Windows Setup" via https://git-scm.com/download/win 1-2. Install Git with default options except for "Use Git from the Windows Command Prompt" and "Use Windows' default console window" Note: You can use any options you want.
1. Install an official Demeter package 1-1. Download "Demeter 0.9.25 (32 bit)" via https://bruceravel.github.io/d emeter/ 1-2. Install Demeter with default options Note: I cannot run 64 bit version with Larch.
2. Install an official Python package 2-1. Download "Windows x86-64 web-based installer" via https://www.python.org/downloads/release/python-362/ 2-2. Custom Install Python with default options except for "Add Python 3.6 to PATH" (checked)
3. Install unofficial Numpy and Scipy, Python libraries 3-1. Download numpy‑1.13.1+mkl‑cp36‑cp36m‑win_amd64.whl and scipy‑0.19.1‑cp36‑cp36m‑win_amd64.whl via http://www.lfd.uci.edu/~gohlke /pythonlibs/ 3-2. pip install numpy-1.13.1+mkl-cp36-cp36m-win_amd64.whl 3-3. pip install scipy‑0.19.1‑cp36‑cp36m‑win_amd64.whl
4. Install Larch and its dependencies 4-1. Start a command prompt 4-2. git clone https://github.com/xraypy/xraylarch.git 4-3. cd xraylarch 4-4. pip install lmfit 4-5. pip install h5py 4-6. pip install pillow 4-7. pip install requests 4-8. pip install sqlalchemy 4-9. pip install matplotlib 4-10. pip install pypiwin32 4-11. pip install wxpython 4-12. pip install wxmplot 4-13. pip install wxutils 4-14. python setup.py install 4-15. larch --version 4-16. You will see "larch command-line version 0.2". Note: I installed Larch dependencies except for pyFAI, CifFile, fabio, tomopy, scikit-image.
5. Run Larch and Athena with Larch 5-1. Start a new command prompt 5-2. larch -rc 5-3. Start another new command prompt 5-4.cd %APPDATA%\DemeterPerl\perl\site\bin 5-5. lathena.bat
It works. Athena is the most user-friendly XAS processing tool I have used.
Thanks so much!
Best regards,
ASAKURA, Hiroyuki
On 2017/08/17 2:17, Bruce Ravel wrote:
Hi Hiroyuki,
Sorry I didn't respond yesterday. Yesterday was a bit of crazy day...
I suppose that this is still a work in progress. I believe that Demeter works with Larch on Windows, but it is certainly not as easy or convenient yet as it should be.
In the place where Demeter got installed on your computer, there is a folder called \perl\site\bin -- you can look at the properties of the Athena desktop icon to see where that is. In that folder is the "dathena.bat" file which the Athena desktop icon points at.
You should also find a file called "lathena.bat". Try running that or pointing your desktop icon at it instead.
The difference between "lathena.bat" and "dathena.bat" is line 25 that sets an environment variable that Demeter uses to decide which of ifeffit or larch to use.
Obviously, there are number of other ways to get this environment variable set. The bottom line is, yes, there is a switch. The switch is the DEMETER_BACKEND environment variable.
Unfortunately, that switch cannot be in the configuration files, although what you tired in item 2 of your email was a sensible guess. It turns out that the choice of larch/ifeffit has to made long before the configuration files are read.
Do let me know how you get on. While Demeter + larch should work, there are still almost certainly problems. I need to get a new version of the Demeter installer out soon, so it would be nice to get some feedback in this area.
As for using Demeter + larch + 100s of file -- it should work after a fashion. But I worry that Athena's performance might not be what you are hoping for. Athena does a lot of things that are more user-friendly than performance-friendly....
Cheers, B
On 08/15/2017 06:10 AM, ASAKURA, Hiroyuki wrote:
Dear all,
I would like to use the latest Larch as backend for Athena and Artemis on Windows to handle hundreds of XAS data (, hopefully). But, I'm stuck. I would appreciate any comments and suggestions.
Matt announced it is now possible on this list, but I still cannot do it on Windows.
[Ifeffit] Larch 0.9.33 http://millenia.cars.aps.anl.gov/pipermail/ifeffit/2017-Apri l/009091.html
The "server mode" for Athena and Artemis now works on all platforms and I
strongly encourage everyone to use this over the ifeffit back end.
Here is what I did.
Environment: Windows 10 Pro (Japanese) Demeter 0.9.25 (via official website) Python 3.6 (official distribution)
1. Installation of the latest Larch
git clone https://github.com/xraypy/xraylarch.git (sha1:
5a372cfe1368a8ac10c2ebfc5bef4e9a9a325f6d) cd xraylarch python setup.py install
# Some dependency related to XRD is not fulfilled.
2. Start Larch server
cd %LOCALAPPDATA%\Programs\Python\Python36\Scripts
python larch_server start
larch_server port=4966: started
I found a parameter of peakfit backend in demeter.ini and changed to larch, but did not work (as expected).
In the dathena.log, Athena still uses ifeffit as backend.
perl version: v5.18.2
backend: ifeffit
Should I compile Demeter package by myself to use the Larch backend? Or, is there any "switch" to use Larch?
Thanks in advance,
ASAKURA, Hiroyuki
-- ASAKURA, Hiroyuki (Ph.D) Program-Specific Assistant Professor (ESICB) T. Tanaka Lab., ESICB, Kyoto University, Japan asakura@moleng.kyoto-u.ac.jp http://www.moleng.kyoto-u.ac.jp/~moleng_04/asakura/ _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit
-- --Matt Newville <newville at cars.uchicago.edu> 630-252-0431
participants (3)
-
ASAKURA, Hiroyuki -
Bruce Ravel -
Matt Newville