data file format

emmanuelle montarges-pelletier

21 Jun 2006 21 Jun '06

3:25 a.m.

Hello I have some problems with data file format as I try to import exafs files in Artemis or in Athena. I am working on a powerbook G4, with OSX 10.4.6. I installed the ifeffit package, including Athena and Artemis. Athena could open any XAS "raw" file from fluorescence or transmission measurements. However, if I perform any modifications on these raw files with exterior programs (Michalowicz programs, Kaleidagraph) and try to import the new files , saved as text files, Athena or Arthemis can not open them. I guess it may come from the encoding of the text file. Do you have any clue on this encoding issue? Do you know any program which would allow me to transform these text files in readable files for Artemis? Thank you for your answer sincerely Emmanuelle ********************************************* Emmanuelle Montargés-Pelletier CR CNRS LEM UMR7569 CNRS-INPL-ENSG 15 avenue du charmois BP40 54501 Vandoeuvre les Nancy cedex France Tel 33 (0)3 83 59 62 52 fax 33 (0)3 83 59 62 55 emmanuelle.montarges@ensg.inpl-nancy.fr http://www.ensg.inpl-nancy.fr/lem/ **************************************************

Show replies by date

Matthew

21 Jun 21 Jun

3:36 a.m.

There are two types of encodings which may differ among text files from different countries and OSs. I don't know how Athena&Co. handle either of them. One is that numbers and decimals can differ per country. If Kgraph is using a local encoding other than the American standard, say, using commas instead of periods, it's possible that Artemis won't understand it. I have the same problem with my analysis programs. Every so often I get a panicky email from a user whose files won't open. The other variation is in what constitutes the end of a line, which is OS-dependent. It could be carriage return, linefeed, or CR-LF. mam ----- Original Message ----- From: "emmanuelle montarges-pelletier" To: Sent: Wednesday, June 21, 2006 1:25 AM Subject: [Ifeffit] data file format

...

Hello I have some problems with data file format as I try to import exafs files in Artemis or in Athena. I am working on a powerbook G4, with OSX 10.4.6. I installed the ifeffit package, including Athena and Artemis. Athena could open any XAS "raw" file from fluorescence or transmission measurements. However, if I perform any modifications on these raw files with exterior programs (Michalowicz programs, Kaleidagraph) and try to import the new files , saved as text files, Athena or Arthemis can not open them. I guess it may come from the encoding of the text file. Do you have any clue on this encoding issue? Do you know any program which would allow me to transform these text files in readable files for Artemis? Thank you for your answer sincerely Emmanuelle ********************************************* Emmanuelle Montargés-Pelletier CR CNRS LEM UMR7569 CNRS-INPL-ENSG 15 avenue du charmois BP40 54501 Vandoeuvre les Nancy cedex France Tel 33 (0)3 83 59 62 52 fax 33 (0)3 83 59 62 55

emmanuelle.montarges@ensg.inpl-nancy.fr http://www.ensg.inpl-nancy.fr/lem/ **************************************************

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

Bruce Ravel

8:52 a.m.

On Wednesday 21 June 2006 03:25, emmanuelle montarges-pelletier wrote:

...

Hello I have some problems with data file format as I try to import exafs files in Artemis or in Athena. I am working on a powerbook G4, with OSX 10.4.6. I installed the ifeffit package, including Athena and Artemis. Athena could open any XAS "raw" file from fluorescence or transmission measurements. However, if I perform any modifications on these raw files with exterior programs (Michalowicz programs, Kaleidagraph) and try to import the new files , saved as text files, Athena or Arthemis can not open them. I guess it may come from the encoding of the text file. Do you have any clue on this encoding issue? Do you know any program which would allow me to transform these text files in readable files for Artemis? Thank you for your answer

Salut Emmanuelle, Matthew's suggestion of floats with commas is a good suggestion. A&A make no attempt at respecting localization. (That is, a number between one and two might be written as 1,234 -- which would confuse A&A -- rather than 1.234.) If that is the case, your best bet is to go back to the exterior program and ask it nicely to write out points instead of commas. Or you might need to edit the file by hand. I thought that the end-of-line issue (Matthew's other suggestion) was one that was solved in A&A years ago. If that proves not to be the case, please let me know. Another possibility is that the exterior program added other text to the file that is confusing A&A. Here is my explanation of how Athena reads ascii data files http://cars9.uchicago.edu/iffwiki/FAQ/HoraeQuestions and here is the explanation of the file-reading function in Ifeffit which Athena uses: http://cars9.uchicago.edu/~ifeffit/refman/node24.html The situation in Artemis is similar, although restricted in the sense that Artemis is expecting its data to be chi(k). If none of that clarifies the issue for you, I'll need to see an example of the data file which is causing the problem. The easiest way to move data from Athena to Artemis is to have Artemis import its data directly from the Athena project file. If there is some data processing feature that Athena does not do but which you require before moving on to data analysis, you should consider making a feature request here on the mailing list. Athena will never get the features that you need if I am not aware that they are needed! B -- Bruce Ravel ---------------------------------------------- bravel@anl.gov Molecular Environmental Science Group, Building 203, Room E-165 MRCAT, Sector 10, Advanced Photon Source, Building 433, Room B007 Argonne National Laboratory phone and voice mail: (1) 630 252 5033 Argonne IL 60439, USA fax: (1) 630 252 9793 My homepage: http://cars9.uchicago.edu/~ravel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/

Matthew

12:43 p.m.

Thanks. I didn't mean to imply that the EOL problem was not dealt with properly by A&A, just that I didn't know if it was dealt with properly. That was a confession of ignorance rather than a statement of fact. mam ----- Original Message ----- From: "Bruce Ravel" To: "XAFS Analysis using Ifeffit" Sent: Wednesday, June 21, 2006 6:52 AM Subject: Re: [Ifeffit] data file format

...

On Wednesday 21 June 2006 03:25, emmanuelle montarges-pelletier wrote:

...
Hello I have some problems with data file format as I try to import exafs files in Artemis or in Athena. I am working on a powerbook G4, with OSX 10.4.6. I installed the ifeffit package, including Athena and Artemis. Athena could open any XAS "raw" file from fluorescence or transmission measurements. However, if I perform any modifications on these raw files with exterior programs (Michalowicz programs, Kaleidagraph) and try to import the new files , saved as text files, Athena or Arthemis can not open them. I guess it may come from the encoding of the text file. Do you have any clue on this encoding issue? Do you know any program which would allow me to transform these text files in readable files for Artemis? Thank you for your answer

Salut Emmanuelle,

Matthew's suggestion of floats with commas is a good suggestion. A&A make no attempt at respecting localization. (That is, a number between one and two might be written as 1,234 -- which would confuse A&A -- rather than 1.234.) If that is the case, your best bet is to go back to the exterior program and ask it nicely to write out points instead of commas. Or you might need to edit the file by hand.

I thought that the end-of-line issue (Matthew's other suggestion) was one that was solved in A&A years ago. If that proves not to be the case, please let me know.

Another possibility is that the exterior program added other text to the file that is confusing A&A. Here is my explanation of how Athena reads ascii data files

http://cars9.uchicago.edu/iffwiki/FAQ/HoraeQuestions

and here is the explanation of the file-reading function in Ifeffit which Athena uses:

http://cars9.uchicago.edu/~ifeffit/refman/node24.html

The situation in Artemis is similar, although restricted in the sense that Artemis is expecting its data to be chi(k).

If none of that clarifies the issue for you, I'll need to see an example of the data file which is causing the problem.

The easiest way to move data from Athena to Artemis is to have Artemis import its data directly from the Athena project file.

If there is some data processing feature that Athena does not do but which you require before moving on to data analysis, you should consider making a feature request here on the mailing list. Athena will never get the features that you need if I am not aware that they are needed!

B

-- Bruce Ravel ---------------------------------------------- bravel@anl.gov

Molecular Environmental Science Group, Building 203, Room E-165 MRCAT, Sector 10, Advanced Photon Source, Building 433, Room B007

Argonne National Laboratory phone and voice mail: (1) 630 252 5033 Argonne IL 60439, USA fax: (1) 630 252 9793

My homepage: http://cars9.uchicago.edu/~ravel EXAFS software: http://cars9.uchicago.edu/~ravel/software/exafs/

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

Matt Newville

5:19 p.m.

Hi Everyone, I doubt line endings are the a problem. But "local encoding" and/or the use of unicode as the default "text formatting" will definitely be a problem. We've seen this with Asian versions of Windows, and I can believe it will happen with other systems too. Emmanuelle, is there an easy way for you to "save as ASCII" or "save as Unix file" from TextEdit or some other editor? Cheers, --Matt

emmanuelle montarges-pelletier

22 Jun 22 Jun

3:10 a.m.

Hello Thanks to everyone for the helpful answers. I finally followed the advice of Michel Jaouen and Matthew Newville, I downloaded TEXTWRANGLER, opened the text files and saved them as unix files. With this format, Artemis was able to read them and plot them. thanks again sincerely emmanuelle ********************************************* Emmanuelle Montargés-Pelletier CR CNRS LEM UMR7569 CNRS-INPL-ENSG 15 avenue du charmois BP40 54501 Vandoeuvre les Nancy cedex France Tel 33 (0)3 83 59 62 52 fax 33 (0)3 83 59 62 55 emmanuelle.montarges@ensg.inpl-nancy.fr http://www.ensg.inpl-nancy.fr/lem/ **************************************************

Matthew

3:52 a.m.

So was it the localization or the end-of-line or what? mam ----- Original Message ----- From: "emmanuelle montarges-pelletier" To: "XAFS Analysis using Ifeffit" Sent: Thursday, June 22, 2006 1:10 AM Subject: Re: [Ifeffit] data file format

...

Hello Thanks to everyone for the helpful answers. I finally followed the advice of Michel Jaouen and Matthew Newville, I downloaded TEXTWRANGLER, opened the text files and saved them as unix files. With this format, Artemis was able to read them and plot them. thanks again sincerely emmanuelle

********************************************* Emmanuelle Montargés-Pelletier CR CNRS LEM UMR7569 CNRS-INPL-ENSG 15 avenue du charmois BP40 54501 Vandoeuvre les Nancy cedex France Tel 33 (0)3 83 59 62 52 fax 33 (0)3 83 59 62 55

emmanuelle.montarges@ensg.inpl-nancy.fr http://www.ensg.inpl-nancy.fr/lem/ **************************************************

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

emmanuelle montarges-pelletier

5:31 a.m.

the end of line seems to be return carriage even on unix files... For sure, the problem was the text-formatting, I don't know why but with textedit I could not save as "ascii file", and the code of the text file seemed to be a unicode UTF8 . As I saved the same data as unix file with TEXTWRANGLER, Athena can open it. The unix file presents "return carriage" at the end of line. I did not try with text file from a windows station...There is still a tiny problem as the first numberof the first column is preceeded of weird characters (Ôªø3.44 ), with subsequent addition of a datapoint (0, 0) on the chi file, but it works. Artemis can open the same file but additional data points appeared which makes the data unexploitable on artemis. Le 22 juin 06 à 10:52, Matthew a écrit :

...

So was it the localization or the end-of-line or what? mam ----- Original Message ----- From: "emmanuelle montarges-pelletier" To: "XAFS Analysis using Ifeffit" Sent: Thursday, June 22, 2006 1:10 AM Subject: Re: [Ifeffit] data file format

...
Hello Thanks to everyone for the helpful answers. I finally followed the advice of Michel Jaouen and Matthew Newville, I downloaded TEXTWRANGLER, opened the text files and saved them as unix files. With this format, Artemis was able to read them and plot them. thanks again sincerely emmanuelle

********************************************* Emmanuelle Montargés-Pelletier CR CNRS LEM UMR7569 CNRS-INPL-ENSG 15 avenue du charmois BP40 54501 Vandoeuvre les Nancy cedex France Tel 33 (0)3 83 59 62 52 fax 33 (0)3 83 59 62 55

emmanuelle.montarges@ensg.inpl-nancy.fr http://www.ensg.inpl-nancy.fr/lem/ **************************************************

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

_______________________________________________ Ifeffit mailing list Ifeffit@millenia.cars.aps.anl.gov http://millenia.cars.aps.anl.gov/mailman/listinfo/ifeffit

********************************************* Emmanuelle Montargés-Pelletier CR CNRS LEM UMR7569 CNRS-INPL-ENSG 15 avenue du charmois BP40 54501 Vandoeuvre les Nancy cedex France Tel 33 (0)3 83 59 62 52 fax 33 (0)3 83 59 62 55 emmanuelle.montarges@ensg.inpl-nancy.fr http://www.ensg.inpl-nancy.fr/lem/ **************************************************

emmanuelle montarges-pelletier

7:22 a.m.

Finally, I saved the text file as a unix file with encoding unicode 8bytes, without byte order mark (BOM), and it works perfectly on Athena. ********************************************* Emmanuelle Montargés-Pelletier CR CNRS LEM UMR7569 CNRS-INPL-ENSG 15 avenue du charmois BP40 54501 Vandoeuvre les Nancy cedex France Tel 33 (0)3 83 59 62 52 fax 33 (0)3 83 59 62 55 emmanuelle.montarges@ensg.inpl-nancy.fr http://www.ensg.inpl-nancy.fr/lem/ **************************************************

6758

Age (days ago)

6759

Last active (days ago)

List overview

Download

8 comments

4 participants

participants (4)

Bruce Ravel
emmanuelle montarges-pelletier
Matt Newville
Matthew

data file format

emmanuelle montarges-pelletier

Matthew

Bruce Ravel

Matthew

Matt Newville

emmanuelle montarges-pelletier

Matthew

emmanuelle montarges-pelletier

emmanuelle montarges-pelletier

tags

participants (4)