Dear all, I would like to use Larch as an external function server. I'm a Python novice, but if I understand correctly, the present server (larch -r) provides access to a Larch process controllable with xmlrpc from outside. However, I would like to do something like below to achieve looser coupling with my future code (maybe not in python). How can I do that? I appreciate any suggestion for my first step to go, though I guess it is a bit far from the Larch way. ### client.py (not a working code) from six.moves.xmlrpc_client import ServerProxy import os, json def main(): s = ServerProxy("http://127.0.0.1:4966/", allow_none = True) response = s.read_ascii(os.path.abspath("/path/to/xas/data")) # I expect a JSON or python dict with keys like "energy", "mu", etc. data = response response = s.pre_edge(data) # data is a JSON or python dict with keys like "energy", "mu", etc. if __name__ == '__main__': main() ### client.py end I guess one can write some sort of function wrapper, if one wants. ### server.py (not a working code) from six.moves.socketserver import ThreadingMixIn from six.moves.xmlrpc_server import SimpleXMLRPCServer, SimpleXMLRPCRequestHandler import json import larch from larch_plugins.xafs import find_e0, pre_edge, autobk from larch_plugins.io import read_ascii class LarchServer(ThreadingMixIn, SimpleXMLRPCServer): def __init__(self, *args, **kw): super().__init__(*args, **kw) self.larch = larch.Interpreter(with_plugins=True) self.larch.run_init_scripts() self.register_introspection_functions() class LarchRequestHandler(SimpleXMLRPCRequestHandler): def __init__(self, *args, **kw): super().__init__(*args, **kw) def _dispatch(self, method_name, args): return getattr(self, method_name)(*args) ### wrapper function to Larch function ### Do something nice to convert the file into processed data as JSON or a python dict def read_ascii(self, filepath): data = read_ascii(filepath, _larch=larch) # Return processed data as JSON, for example. return json.dumps(data) def main(): server = LarchServer( ("127.0.0.1", 4966), requestHandler = LarchRequestHandler, logRequests = None, allow_none = True ) server.serve_forever() if __name__ == '__main__': main() ### server.py end -- ASAKURA, Hiroyuki (Ph.D) Program-Specific Senior Lecturer (ESICB) T. Tanaka Lab., ESICB, Kyoto University, Japan asakura@moleng.kyoto-u.ac.jp http://www.moleng.kyoto-u.ac.jp/~moleng_04/asakura/
Hi Hiroyuki, On Sat, Mar 2, 2019 at 11:40 PM ASAKURA, Hiroyuki < asakura@moleng.kyoto-u.ac.jp> wrote:
Dear all,
I would like to use Larch as an external function server. I'm a Python novice, but if I understand correctly, the present server (larch -r) provides access to a Larch process controllable with xmlrpc from outside.
However, I would like to do something like below to achieve looser coupling with my future code (maybe not in python). How can I do that? I appreciate any suggestion for my first step to go, though I guess it is a bit far from the Larch way.
Running Larch in "server mode" is definitely possible but also sort of advanced programming. I should clarify something: you should start a server with the `larch_server start`, not `larch -r`. This should be removed from the `larch` command and fixed in the docs. Once the server process is running, client processes can send commands to run and receive data from the larch session running in the server process. The server process can be on a different machine, but I should admit that I've only done this in tests. The functions available to the client are pretty limited (definitely not all larch commands) and meant to be "high level". They include: larch send larch commands as strings to the server get_data get data encoded in json get_rawdata get data with json encoding get_messages, len_messages get messages printed by the server ls, chdir, cd, cwd setting the working directory for the server shutdown, set_keepalive_time shutdown server, delay shutdown due to no activity set_client_info, get_client_info set/get information about the client using this server There isn't anything as specific as "read_ascii()" function. Instead, you would do from xmlrpc.client import ServerProxy server = ServerProxy("http://127.0.0.1:4966") # send a string to run as a larch command server.larch("grp = read_ascii('/path/to/xas/data')") We could consider adding more exposed functions, but the server has to register each exposed function (see https://github.com/xraypy/xraylarch/blob/master/larch/xmlrpc_server.py#L121 ), so I would not want to try to expose every function in the Larch API. The client application can be in any language that supports XML-RPC communication. Many languages do have an XML-RPC library, though they might differ in some features like how to unpack data into native datatypes. For example, a Perl implementation so that Demeter can use a larch server as its backend is at https://github.com/bruceravel/demeter/blob/master/lib/Larch.pm If you are interested in doing that for another language let me know if I can be of any help. Hope that helps, but if I missed the some part of your question, please let me know.... --Matt
Dear Matt, Thank you for your prompt response. If I understand correctly, Athena works with Larch by making command strings (Text::Template?) and submitting it to Larch via XML-RPC. So, it would be the best way, as you suggested.
server.larch("grp = read_ascii('/path/to/xas/data')")
I understand you don't plan to expose (all of) "low" functions, too. I intended to use Larch in loose-coupling (without knowledge of a Larch session status) with my app, so I have to go another way. Thanks again, Hiroyuki On 2019/03/03 23:26, Matt Newville wrote:
Hi Hiroyuki,
On Sat, Mar 2, 2019 at 11:40 PM ASAKURA, Hiroyuki < asakura@moleng.kyoto-u.ac.jp> wrote:
Dear all,
I would like to use Larch as an external function server. I'm a Python novice, but if I understand correctly, the present server (larch -r) provides access to a Larch process controllable with xmlrpc from outside.
However, I would like to do something like below to achieve looser coupling with my future code (maybe not in python). How can I do that? I appreciate any suggestion for my first step to go, though I guess it is a bit far from the Larch way.
Running Larch in "server mode" is definitely possible but also sort of advanced programming. I should clarify something: you should start a server with the `larch_server start`, not `larch -r`. This should be removed from the `larch` command and fixed in the docs.
Once the server process is running, client processes can send commands to run and receive data from the larch session running in the server process. The server process can be on a different machine, but I should admit that I've only done this in tests. The functions available to the client are pretty limited (definitely not all larch commands) and meant to be "high level". They include:
larch send larch commands as strings to the server get_data get data encoded in json get_rawdata get data with json encoding get_messages, len_messages get messages printed by the server ls, chdir, cd, cwd setting the working directory for the server shutdown, set_keepalive_time shutdown server, delay shutdown due to no activity set_client_info, get_client_info set/get information about the client using this server
There isn't anything as specific as "read_ascii()" function. Instead, you would do
from xmlrpc.client import ServerProxy server = ServerProxy("http://127.0.0.1:4966")
# send a string to run as a larch command server.larch("grp = read_ascii('/path/to/xas/data')")
We could consider adding more exposed functions, but the server has to register each exposed function (see https://github.com/xraypy/xraylarch/blob/master/larch/xmlrpc_server.py#L121 ), so I would not want to try to expose every function in the Larch API.
The client application can be in any language that supports XML-RPC communication. Many languages do have an XML-RPC library, though they might differ in some features like how to unpack data into native datatypes. For example, a Perl implementation so that Demeter can use a larch server as its backend is at https://github.com/bruceravel/demeter/blob/master/lib/Larch.pm
If you are interested in doing that for another language let me know if I can be of any help.
Hope that helps, but if I missed the some part of your question, please let me know....
--Matt
_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit Unsubscribe: http://millenia.cars.aps.anl.gov/mailman/options/ifeffit
-- ASAKURA, Hiroyuki (Ph.D) Program-Specific Senior Lecturer (ESICB) T. Tanaka Lab., ESICB, Kyoto University, Japan asakura@moleng.kyoto-u.ac.jp http://www.moleng.kyoto-u.ac.jp/~moleng_04/asakura/
Hi, On Wed, Mar 13, 2019 at 10:26 PM ASAKURA, Hiroyuki < asakura@moleng.kyoto-u.ac.jp> wrote:
Dear Matt,
Thank you for your prompt response.
If I understand correctly, Athena works with Larch by making command strings (Text::Template?) and submitting it to Larch via XML-RPC. So, it would be the best way, as you suggested.
server.larch("grp = read_ascii('/path/to/xas/data')")
I understand you don't plan to expose (all of) "low" functions, too.
Well, that would be a large number of functions to expose, each needing error checking, and support for how to map the various data types. I think that would end up repeating nearly all of Larch and Python for the RPC interface. The "read_ascii" example might be possible, as it takes a string for a filename (then again, for RPC, is it obvious that files are accessible from the same path?). But if you want to do any data processing, you'd have to be able to handle more complex Python objects (dictionaries, lists, numpy arrays). The main motivation for the RPC interface was so other languages (say, Perl) could run the Larch functions. For Demeter, having a "session" makes sense -- the GUI is an interface that wants to keep track of analysis steps and be able to apply functions on data in memory. I intended to use Larch in loose-coupling (without knowledge of a Larch
session status) with my app, so I have to go another way.
I guess it depends what you mean by "loose-coupling". The RPC interface supports using Larch from another language and/or from a different process (even different user account or machine). To use Larch from outside Python, some sort of RPC is needed. I can believe it could be better, but there have not been a lot of requests for that. Maybe you can describe what your trying to do? Cheers, --Matt
participants (2)
-
ASAKURA, Hiroyuki -
Matt Newville