how can i call the function of Athena or Artemis in Python
Such as I want to call calibrate or merge function of Athena in python. Can python glue them together? any document? Jack Song
On Thursday 04 December 2008 11:58:48 am goodhei8@gmail.com wrote:
Such as I want to call calibrate or merge function of Athena in python. Can python glue them together? any document?
Jack Song
Jack, That's the most interesting question I've seen here in weeks! :) The fact that you cannot do what you are asking is something that has been bothering me for quite some time. So much so, in fact, that I am in the process of rewriting the entire functionality of Athena and Artemis so that clever ideas like yours are possible. You can peruse my Demeter wiki pages for details. One of my goals for the new software is to write a XML-RPC wrapper around it so that you can have the new software running as a service on your computer or on a computer on the network. This is a nifty idea because it is language neutral in the sense that you could write a script in python or any other language (Ruby, C, Lua, C#, even something crazy like perl!) and process your data identically to how it is done in Athena. Alas, what I just described is still many months away. The code base is mostly written, but I need to spend some time writing GUIs so the code base can be tested more methodically. But someday.... One option you might consider in the short term is to drive Ifeffit directly from your python script. Try watching what goes on in the Ifeffit buffer (see Athena's Edit menu) while you are calibrating or merging or whatever. That will give you a sense of how my perl programs are driving Ifeffit and might help you hack something usable together in python. Another option would be to try to use some of Sam's work in SixPack as the basis for your own python scripts. HTH B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/
Got you. Thanks for your quick reply. since i am not ever a software developer(as most people), i really suggest to do a function lib first(which is the key parts of Athena etc. and gap between ifeffit and user. then users can use them in python etc first), then GUI. PS, You just made the next version more exciting to me. Glad to know we are thinking in the same way for this question : ) Thanks Jack Bruce Ravel wrote:
On Thursday 04 December 2008 11:58:48 am goodhei8@gmail.com wrote:
Such as I want to call calibrate or merge function of Athena in python. Can python glue them together? any document?
Jack Song
Jack,
That's the most interesting question I've seen here in weeks! :)
The fact that you cannot do what you are asking is something that has been bothering me for quite some time. So much so, in fact, that I am in the process of rewriting the entire functionality of Athena and Artemis so that clever ideas like yours are possible. You can peruse my Demeter wiki pages for details.
One of my goals for the new software is to write a XML-RPC wrapper around it so that you can have the new software running as a service on your computer or on a computer on the network. This is a nifty idea because it is language neutral in the sense that you could write a script in python or any other language (Ruby, C, Lua, C#, even something crazy like perl!) and process your data identically to how it is done in Athena.
Alas, what I just described is still many months away. The code base is mostly written, but I need to spend some time writing GUIs so the code base can be tested more methodically. But someday....
One option you might consider in the short term is to drive Ifeffit directly from your python script. Try watching what goes on in the Ifeffit buffer (see Athena's Edit menu) while you are calibrating or merging or whatever. That will give you a sense of how my perl programs are driving Ifeffit and might help you hack something usable together in python.
Another option would be to try to use some of Sam's work in SixPack as the basis for your own python scripts.
HTH B
HI Jack,
since i am not ever a software developer(as most people), i really suggest to do a function lib first(which is the key parts of Athena etc.and gap between ifeffit and user. then users can use them in python etc first), then GUI.
It's a little confusing to read "I'm not a software developer" and "I want to run Athena from Python". Nobody here is actually a software developer... we're all scientists, and some of us write programs to get stuff done. It seems like you'd also like to write programs to get stuff done. The basic Ifeffit interface may seem fairly low level at first glance, but it really does allow you to write fairly high-level functions very easily: Below is a primitive "Library" that does some of what you want (merging data). I often use and write such scripts and I don't think it can ever get a lot simpler than this and still be generally useful. Put another way: writing a library that is all of "simple to use" and "high level" and "generally useful" is hard. Of course, Athena, Artemis, and SixPack all have a lot of "higher level" functionality" on top of Ifeffit, but (as far as I can tell) neither fully separated processing approaches from the GUI interface, and also are not (yet) intended for general reuse. Again, creating such a general-purpose library is not a small amount of work. I do agree that it would be very nice to have a set of standard functions and classes that were not tied to a particular GUI. I think Bruce agrees too, hence Demeter. For now, if you'd like to work on a higher level Python interface that did many of the "Athena Steps", that would be great. The code below might give a reasonable start, though it is a set of functions, and not class-based. I think that defining a class for "DataGroup" would help a lot. As Bruce mentioned, we've played with "Python-Perl" interaction with XML-RPC and are planning on using more of this in the future, as Ifeffit2 will be implemented in Python, Bruce's Perl codes will need to interact with the base library through XML-RPC. That's a ways off, but if your (or anyone else) is interested in getting involved with these developments, let us know. --Matt #!/usr/bin/python from Ifeffit import Ifeffit iff = Ifeffit() def run_iff(commands,debug=False): """basic wrapper around ifeffit commands: send a command string or a list of command strings """ cmds = commands if isinstance(commands,(list,tuple)): cmds = "\n".join(commands) elif not isinstance(commands,str): print "cannot pass commands to Ifeffit:" print commands if debug: print cmds iff.ifeffit(cmds) def show_arrays(): run_iff("show @arrays") def show_scalars(): run_iff("show @scalars") def show_group(group): run_iff("show @group=%s"%group) def plot(group,x,y, range=(None,None), new=False): """ simple plotting wrapper""" cmd = ["%s.%s, %s.%s" % (group,x, group,y)] if new: cmd.append("new=1,xlabel=%s,ylabel=%s" % (x,y)) run_iff("plot(%s)" % ','.join(cmd)) if range is not (None,None): opts = '' if range[0] is not None: opts = "%s,xmin=%s" % (opts,str(range[0])) if range[1] is not None: opts = "%s,xmax=%s" % (opts,str(range[1])) run_iff("plot %s" % opts) def read_GSE_trans(fname,group='a'): """read transmission XAFS data from GSECARS """ cmds = ["read_data(file=%s, group=%s)" % (fname,group)] cmds.append("set %s.energy = %s.p1" % (group,group)) cmds.append("set %s.xmu = -log(%s.d2/%s.d1)" % (group,group,group)) run_iff(cmds) def read_GSE_fluor(fname,group='a',i0=1,d=(3,16)): """Simple read of fluor XAFS data from GSECARS (see escan data for more elaborate approach) i0 = detector column for i0 channel [default=1] d = (c1, c2) detector column range for fluorescence channels to add together """ cmds = ["read_data(file=%s, group=%s)" % (fname,group)] cmds.append("set %s.energy = %s.p1" % (group,group)) cmds.append("set %s.i0 = %s.d1" % (group,group)) fcols = range(d[0],d[1]+1) dcol = '+'.join(['%s.d%i' % (group,i) for i in fcols]) cmds.append("set %s.fl = %s" % (group,dcol)) cmds.append("set %s.xmu = %s.fl/%s.i0" % (group,group,group)) run_iff(cmds) def autobk(group,rbkg=1.0,kweight=2,kmin=0,kmax=0): """ simple autobk wrapper""" cmds = ["""spline(%s.energy, %s.xmu, rbkg=%f, kweight=%f,kmin=%f,kmax=%f)""" % (group,group,rbkg,kweight,kmin,kmax)] cmds.append("set %s.chik = %s.chi* %s.k^2" % (group,group,group)) run_iff(cmds) def fftf(group,kmin=0,kmax=0,kweight=2,dk=2,window='kaiser',rmax=10): """ simple fftf wrapper""" cmds = ["""fftf(%s.chi,kweight=%f,kmin=%f,kmax=%f, dk=%f,kwindow=%s,rmax_out=%f)""" % (group,kweight,kmin,kmax,dk,window,rmax)] run_iff(cmds) def pause(t=5): import time time.sleep(t) def merge_start(group,type='xmu',output=None): cmds = [] ; add = cmds.append add("set $merge_type=%s" % type) add("set merge_count=1") if output is None: output='merge' add("set $merge_output=%s" % output) if type=='xmu': add("set __tmp.en = %s.energy" % group) add("set __tmp.xmu = %s.xmu" % group) elif type=='chi': add("set __tmp.k = %s.k" % group) add("set __tmp.chi = %s.chi" % group) else: print "cannot merge data of type: ", mtype return run_iff(cmds) def merge_add(group): mtype = iff.get_string("$merge_type") cmds = [] ; add = cmds.append if mtype=='xmu': add("set __tmp.xmu = __tmp.xmu + lintrp(%s.energy,%s.xmu,__tmp.xmu)" % (group,group)) elif mtype=='chi': add("set __tmp.chi = __tmp.chi + lintrp(%s.k,%s.chi,__tmp.chi)" % (group,group)) else: print "cannot merge data of type: ", mtype return add("set merge_count=merge_count+1") run_iff(cmds) def merge_finish(group,output=None): merge_add(group) cmds = [] ; add = cmds.append mtype = iff.get_string("$merge_type") mout = iff.get_string("$merge_output") if output is not None: mout = output if mtype=='xmu': add("set %s.energy = __tmp.en" % mout) add("set %s.xmu = __tmp.xmu / merge_count" % mout) elif mtype=='chi': add("set %s.k = __tmp.k" % mout) add("set %s.chi = __tmp.chi / merge_count" % mout) else: print "cannot merge data of type: ", mtype return run_iff(cmds) if __name__ == '__main__': read_GSE_trans('as2o5_xafs.001',group='a5') autobk('a5',rbkg=0.9) plot('a5','energy','xmu',new=True) plot('a5','energy','bkg') pause(3) plot('a5','k','chik',new=True) pause(3) fftf('a5',kmin=2,kmax=11,dk=3) plot('a5','r','chir_mag',new=True,range=(0,7)) ## ## Another Example Use: (assumes the file is saved as Ifflib.py) # import Ifflib as IFL # # IFL.read_GSE_fluor('04_as_xafs.001', group='as04_1', d=(3,15)) # IFL.read_GSE_fluor('04_as_xafs.002', group='as04_2', d=(3,15)) # IFL.read_GSE_fluor('04_as_xafs.003', group='as04_3', d=(3,15)) # IFL.read_GSE_fluor('04_as_xafs.004', group='as04_4', d=(3,15)) # # IFL.merge_start('as04_4') # IFL.merge_add('as04_3') # IFL.merge_add('as04_2') # IFL.merge_finish('as04_1',output='as04_ave') # # IFL.run_iff("save as04.sav") # # IFL.plot('as04_ave','energy','xmu',new=True) # ##
On Thursday 04 December 2008 05:52:09 pm Matt Newville wrote:
I do agree that it would be very nice to have a set of standard functions and classes that were not tied to a particular GUI. I think Bruce agrees too, hence Demeter.
It seems worth mentioning that Demeter is already highly usable and ships with numerous examples, including examples of doing the specific things Jack asked about. I use it for my own data analysis chores all the time. In fact, just today I wrote this rather elaborate data analysis example to show off some of Demeter's cool new Feff-related capabilities: http://cars9.uchicago.edu/iffwiki/Demeter/Cookbook/06Uranyl Of course, using Demeter means writing perl programs. But that ain't *so* bad, is it...? B -- Bruce Ravel ------------------------------------ bravel@bnl.gov National Institute of Standards and Technology Synchrotron Methods Group at NSLS --- Beamlines U7A, X24A, X23A2 Building 535A Upton NY, 11973 My homepage: http://xafs.org/BruceRavel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/
Good stuff. Appreciate it. Jack Bruce Ravel wrote:
On Thursday 04 December 2008 05:52:09 pm Matt Newville wrote:
I do agree that it would be very nice to have a set of standard functions and classes that were not tied to a particular GUI. I think Bruce agrees too, hence Demeter.
It seems worth mentioning that Demeter is already highly usable and ships with numerous examples, including examples of doing the specific things Jack asked about.
I use it for my own data analysis chores all the time. In fact, just today I wrote this rather elaborate data analysis example to show off some of Demeter's cool new Feff-related capabilities: http://cars9.uchicago.edu/iffwiki/Demeter/Cookbook/06Uranyl
Of course, using Demeter means writing perl programs. But that ain't *so* bad, is it...?
B
Matt, I can not agree more or can not expect more. Thanks for the info. When I can, I definitely would like to contribute some. (I am not just saying it.) PS, sorry for my bad English Jack Matt Newville wrote:
HI Jack,
since i am not ever a software developer(as most people), i really suggest to do a function lib first(which is the key parts of Athena etc.and gap between ifeffit and user. then users can use them in python etc first), then GUI.
It's a little confusing to read "I'm not a software developer" and "I want to run Athena from Python". Nobody here is actually a software developer... we're all scientists, and some of us write programs to get stuff done. It seems like you'd also like to write programs to get stuff done. The basic Ifeffit interface may seem fairly low level at first glance, but it really does allow you to write fairly high-level functions very easily:
Below is a primitive "Library" that does some of what you want (merging data). I often use and write such scripts and I don't think it can ever get a lot simpler than this and still be generally useful. Put another way: writing a library that is all of "simple to use" and "high level" and "generally useful" is hard. Of course, Athena, Artemis, and SixPack all have a lot of "higher level" functionality" on top of Ifeffit, but (as far as I can tell) neither fully separated processing approaches from the GUI interface, and also are not (yet) intended for general reuse. Again, creating such a general-purpose library is not a small amount of work.
I do agree that it would be very nice to have a set of standard functions and classes that were not tied to a particular GUI. I think Bruce agrees too, hence Demeter. For now, if you'd like to work on a higher level Python interface that did many of the "Athena Steps", that would be great. The code below might give a reasonable start, though it is a set of functions, and not class-based. I think that defining a class for "DataGroup" would help a lot.
As Bruce mentioned, we've played with "Python-Perl" interaction with XML-RPC and are planning on using more of this in the future, as Ifeffit2 will be implemented in Python, Bruce's Perl codes will need to interact with the base library through XML-RPC. That's a ways off, but if your (or anyone else) is interested in getting involved with these developments, let us know.
--Matt
#!/usr/bin/python
from Ifeffit import Ifeffit iff = Ifeffit()
def run_iff(commands,debug=False): """basic wrapper around ifeffit commands: send a command string or a list of command strings """ cmds = commands if isinstance(commands,(list,tuple)): cmds = "\n".join(commands) elif not isinstance(commands,str): print "cannot pass commands to Ifeffit:" print commands if debug: print cmds iff.ifeffit(cmds)
def show_arrays(): run_iff("show @arrays") def show_scalars(): run_iff("show @scalars") def show_group(group): run_iff("show @group=%s"%group)
def plot(group,x,y, range=(None,None), new=False): """ simple plotting wrapper""" cmd = ["%s.%s, %s.%s" % (group,x, group,y)] if new: cmd.append("new=1,xlabel=%s,ylabel=%s" % (x,y))
run_iff("plot(%s)" % ','.join(cmd)) if range is not (None,None): opts = '' if range[0] is not None: opts = "%s,xmin=%s" % (opts,str(range[0])) if range[1] is not None: opts = "%s,xmax=%s" % (opts,str(range[1])) run_iff("plot %s" % opts)
def read_GSE_trans(fname,group='a'): """read transmission XAFS data from GSECARS """ cmds = ["read_data(file=%s, group=%s)" % (fname,group)] cmds.append("set %s.energy = %s.p1" % (group,group)) cmds.append("set %s.xmu = -log(%s.d2/%s.d1)" % (group,group,group)) run_iff(cmds)
def read_GSE_fluor(fname,group='a',i0=1,d=(3,16)): """Simple read of fluor XAFS data from GSECARS (see escan data for more elaborate approach) i0 = detector column for i0 channel [default=1] d = (c1, c2) detector column range for fluorescence channels to add together """
cmds = ["read_data(file=%s, group=%s)" % (fname,group)]
cmds.append("set %s.energy = %s.p1" % (group,group)) cmds.append("set %s.i0 = %s.d1" % (group,group))
fcols = range(d[0],d[1]+1) dcol = '+'.join(['%s.d%i' % (group,i) for i in fcols]) cmds.append("set %s.fl = %s" % (group,dcol)) cmds.append("set %s.xmu = %s.fl/%s.i0" % (group,group,group))
run_iff(cmds)
def autobk(group,rbkg=1.0,kweight=2,kmin=0,kmax=0): """ simple autobk wrapper""" cmds = ["""spline(%s.energy, %s.xmu, rbkg=%f, kweight=%f,kmin=%f,kmax=%f)""" % (group,group,rbkg,kweight,kmin,kmax)] cmds.append("set %s.chik = %s.chi* %s.k^2" % (group,group,group))
run_iff(cmds)
def fftf(group,kmin=0,kmax=0,kweight=2,dk=2,window='kaiser',rmax=10): """ simple fftf wrapper""" cmds = ["""fftf(%s.chi,kweight=%f,kmin=%f,kmax=%f, dk=%f,kwindow=%s,rmax_out=%f)""" % (group,kweight,kmin,kmax,dk,window,rmax)]
run_iff(cmds)
def pause(t=5): import time time.sleep(t)
def merge_start(group,type='xmu',output=None): cmds = [] ; add = cmds.append add("set $merge_type=%s" % type) add("set merge_count=1") if output is None: output='merge' add("set $merge_output=%s" % output) if type=='xmu': add("set __tmp.en = %s.energy" % group) add("set __tmp.xmu = %s.xmu" % group) elif type=='chi': add("set __tmp.k = %s.k" % group) add("set __tmp.chi = %s.chi" % group) else: print "cannot merge data of type: ", mtype return run_iff(cmds)
def merge_add(group): mtype = iff.get_string("$merge_type") cmds = [] ; add = cmds.append if mtype=='xmu': add("set __tmp.xmu = __tmp.xmu + lintrp(%s.energy,%s.xmu,__tmp.xmu)" % (group,group)) elif mtype=='chi': add("set __tmp.chi = __tmp.chi + lintrp(%s.k,%s.chi,__tmp.chi)" % (group,group)) else: print "cannot merge data of type: ", mtype return add("set merge_count=merge_count+1") run_iff(cmds)
def merge_finish(group,output=None): merge_add(group) cmds = [] ; add = cmds.append mtype = iff.get_string("$merge_type") mout = iff.get_string("$merge_output") if output is not None: mout = output if mtype=='xmu': add("set %s.energy = __tmp.en" % mout) add("set %s.xmu = __tmp.xmu / merge_count" % mout) elif mtype=='chi': add("set %s.k = __tmp.k" % mout) add("set %s.chi = __tmp.chi / merge_count" % mout) else: print "cannot merge data of type: ", mtype return run_iff(cmds)
if __name__ == '__main__': read_GSE_trans('as2o5_xafs.001',group='a5') autobk('a5',rbkg=0.9)
plot('a5','energy','xmu',new=True) plot('a5','energy','bkg') pause(3)
plot('a5','k','chik',new=True) pause(3)
fftf('a5',kmin=2,kmax=11,dk=3) plot('a5','r','chir_mag',new=True,range=(0,7))
## ## Another Example Use: (assumes the file is saved as Ifflib.py) # import Ifflib as IFL # # IFL.read_GSE_fluor('04_as_xafs.001', group='as04_1', d=(3,15)) # IFL.read_GSE_fluor('04_as_xafs.002', group='as04_2', d=(3,15)) # IFL.read_GSE_fluor('04_as_xafs.003', group='as04_3', d=(3,15)) # IFL.read_GSE_fluor('04_as_xafs.004', group='as04_4', d=(3,15)) # # IFL.merge_start('as04_4') # IFL.merge_add('as04_3') # IFL.merge_add('as04_2') # IFL.merge_finish('as04_1',output='as04_ave') # # IFL.run_iff("save as04.sav") # # IFL.plot('as04_ave','energy','xmu',new=True) # ## _______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit
participants (3)
-
Bruce Ravel -
goodhei8＠gmail.com -
Matt Newville