Athena and Artemis Questions
Athena and Artemis Questions
- How do I transfer data from Athena to Artemis?
- How can I preprocess groups of related data?
- What kind of data files does Athena import?
- How do I read data into Athena or Artemis on a computer using a non-ASCII language?
- Athena says my data file has fewer than 10 data points, but I know this is not true. How do I import these data into Athena?
- What's the deal with Athena's compressed project files?
- How can I manage multiple FEFF path calculations in Artemis?
- How do I use the 'Fit background' option?
- Why does Athena become unresponsive and consume 100% of the CPU
- How do I make publication quality plots with Athena/Artemis? (Alternately: Why are the plots so ugly in Athena/Artemis?)
- Fine, Athena and Artemis do not make publication quality plots. How can I modify the plots that are made?
- Why are the fonts so ugly in Athena and/or Artemis?
- Athena does not fit on my small screen! I cannot see the OK button in the column selection dialog! How can I fix this?
- How do I add a new FAQ entry?
How do I transfer data from Athena to Artemis?
One of the features of Artemis 0.8 is the ability to directly import an Athena project file into Artemis. Just select the Athena project using the normal file selection dialog. When you do this, you see a list of all the groups in the Athena project file. You can select one and its data along with its Fourier transform parameters get imported into your Artemis project.
Alternately, you can save a chi(k) file in Athena and import that into Artemis, but that is more work and less convenient that reading from the project file. If you have chi(k) data from some other source, you can read that directly into Artemis, so long as it is not k-weighted.
How can I preprocess groups of related data?
Here are some suggestions from Bruce
The details of what you might do vary from situation to situation, but here is the outline:
- Import one scan. This is going to be your "standard" against which the rest will be compared, so it should be a trustworthy one. Calibrate it by hand using the Calibration option in the Data menu. Play around with the background removal and FT parameters until you are reasonably happy with the data processing.
- Use the "Open many files" option in the Files menu to import several more of the data files in that ensemble of related data files.
- When the column selection dialog comes up, make sure that the columns are properly selected. Before hitting the OK button, hit the button that says "Set preprocessing parameters". This will expand the column selection dialog, showing a new menu and bunch of checkbuttons.
- From the menu labeled "Standard" select the first data file that you read in. This will activate all the checkbuttons below. Now select the buttons for "Align to the standard" and "Set parameters to the standard".
- Now click the OK button. As each data file is imported, it will be aligned and the parameters, including e0, the background parameters, and the FT parameters, will all be set to the values you settled on for the first data set. Magic!!
If all went well, then each data set will be calibrated the same way as the first one and the background and FT parameters will be well set. The caveat is that the automated alignment does not always work well. A quick plot will let you know if it failed and you need to go back and align by hand. But with data that is low noise and similar, it tends to work ok.
Depending on your data set and what you are aiming to do, you may need to do some variation of what I described above. Of course, a change in Ifeffit's algorithm might still be in order. But, in any case, I encourage you to resolve your e0 problems by explicitly examining one data file and using preprocessing to make the rest consistent with the first.
What kind of data files does Athena import?
See AthenaFiletypePlugins for details about how to teach Athena to read arbitrary data files.
Athena is very versatile in how she reads in data files. Pretty much any data in the form of columns of numbers can be successfully read. With a few exceptions, Athena relies upon Ifeffit's read_data() command to handle the details of data import. Ifeffit is clever about recognizing which part of a file is columns of numbers and which part is not. In the following, I'll explain how the read_data() command interprets files, explain the limits on its and Athena's abilities to interpret a data file, and discuss the kinds of manipulations of data that can and cannot be performed by Athena as data are imported.
Athena expects data of one of a few types. Column data in which the columns represent such things as the energy grid and the scalars measured during the experiment are the most common sort of data that most people use import into Athena. Athena's column selection dialog is used to convert the raw scalars into mu(E) data. Other common kinds of data files that might be read into Athena are files that contain mu(E) or chi(k) data in columns or the output files from Feff, xmu.dat and chi.dat.
Here is an example of a data file that will make Athena as happy as can be. There are some header lines, followed by a line of dashes, followed by a line of column labels, followed by lines containing columns of data.
# X15B project: MT 9/23/04 # original file: STD1.001 # unpacked from original data as a sequence of 4-byte floats # ----------------------------------------------------------- # energy I0 narrow wide 2400.0020 60183.3008 38.5000 83.0000 2401.5088 60241.0508 41.5000 82.0000 2403.0078 60347.5508 40.0000 83.7500 2404.5039 60531.0508 42.2500 78.2500
In this example of a perfectly formatted file, the header lines, the line of dashes, and the column labels line are all preceded by a hash (#) mark. Ifeffit is thus able to recognize these as header lines. Since Ifeffit recognizes them as such, Athena will store them in the project along with the data. Because there is a line of dashes and because it is followed immediately by column labels, Athena is able to use these labels in the column selection dialog. Other marks, such as % or *, will also be understood as marking header lines.
The numbers in the columns can be integers, floats (such as 1.234, -1.234, .1234 or -.1234), or exponentials (such as 1.23e45 or -1.23E-45). Anything interpretable as a number in the C programming language will be interpretable in this context. The columns of numbers go to the end of the file. There is no text following the data.
When data is recorded as described above, it will be fully utilized by Athena. The headers will be recorded, the column labels will be used, the data will be interpreted. Athena can, however, accommodate significant deviation from the format described above.
- If the header lines are not marked by a # or some other recognizable marking character, Ifeffit will not be able to recognize headers or column labels. As long as no text follows the data, the columns will still be understood as columns of data and the data can be imported by Athena.
- If the line of dashes is missing, again the headers and column labels will not be recognized, but the columns of data will be.
- If no headers are in the file, the columns of data will still be understood as data.
Athena expects that the data are recorded as a function of energy and that one of the columns contains energy values. The assumption is that the first column is the energy column, but that can be changed in the column selection dialog. Athena works in eV. If data are recorded in keV, there is a menu in the column selection dialog that must be set accordingly.
Here are some operations that can be performed as data is imported.
- Data can be imported as a function of pixel position on an area or linear detector.
- Data from a multi-element detector can be summed on the fly.
- Data from a multi-element detector can be imported such that each detector channel is imported into its own data group.
- Data can be negated, i.e. multiplied by -1.
- A reference channel can be read from the the same file.
- Data can be preprocessed. That is, data can be truncated, deglitched, aligned to a standard, and have its parameters constrained to a standard.
Here are some operations that can be handled using the Plugin architecture:
- Conversion from wavelength to energy.
- Conversion from encoder reading or motor steps to energy.
- Dead-time corrections using columns from the data file.
- Any math expression more complicated than sums of columns in the numerator and denominator (e.g. plugins allow you to multiply the If column by 7 and divide by the sine of the I0 column, if that's what you want).
If some of the criteria for the data file format are not met, for example if there is text following the data columns or if you need to perform one of the operations not yet supported, you will need to process you data before trying to import into Athena.
There are examples of data files that Athena will process before sending off to Ifeffit for import. An example is the data file format from beamline X10C at NSLS. Files from that beamline cannot be imported as written by Ifeffit's read_data command. Athena will recognize such a file and process it as needed before importing it. This can be done with other beamlines. You should contact Bruce if you are the beamline scientist of a frequent user of some beamline which writes data in a way that read_data cannot import.
As a final comment, I would encourage beamline scientists and the authors of data acquisition software to consider their users when designing data file formats. While I certainly will not say that beamlines should be required to accommodate Athena or even that beamline staff have any obligation to recommend Athena to their users, the truth is that Athena is becoming an increasingly common tool in the EXAFS community. The format that best serves Athena is actually a fine format that can be imported by a very wide variety of EXAFS software, plotting software, spreadsheets, and other programs. It's a good format and your users would be well served by your adopting it.
How do I read data into Athena or Artemis on a computer using a non-ASCII language?
Several users from countries whose languages cannot be represented in ASCII characters (Japan and China for instance) have reported problems importing data files when the data file lies in a folder whose name cannot be represented in ASCII. As explained here, the work-around is to move your data into a folder with an entirely ASCII name.
Athena says my data file has fewer than 10 data points, but I know this is not true. How do I import these data into Athena?
It is possible that you have tried to import a data file from a scan that was aborted soon after it started. In that case, it is possible the file truly does have fewer than 10 data points. More likely, however, Ifeffit (and thus Athena) is unable to determine what part of the file is the header and what part is the data.
The Ifeffit Reference Manual explain how Ifeffit interprets the header and data sections of a data file. See this section and its subsections.
Usually the cause of this problem is that one or more lines of the header meets Ifeffit's criteria for a line of data. When those lines are followed by lines that do not meet that criteria, Ifeffit then stops reading the file as data. The solution to the problem is to do something to preserve the data file in a state that will not trigger this confusion in Ifeffit. Some suggestions
- Edit the data file to remove all the header lines.
Edit the data file such that each header line begins with a hash (#) character.
- Change the data acquisition software so that it writes files that can be read unedited into Athena.
- Write a small program, macro, or script that preprocesses the header in one of the ways suggested above. This is a fine solution if you have large quantities of data.
Another possible explanation is that you are running Athena on Windows and another program is currently using the data file. Windows will usually not allow you to open a file with two programs at the same time. Athena might misinterpret this situation as a data file with 0 lines of data.
Some beamlines, such as X10c at NSLS, use strange file formats that Ifeffit cannot read. Athena ships with a filetype plugin for reading data from that beamline. If you have lots of data in a format that confuses Ifeffit and you are very nice to Bruce, he might write a special preprocessor for you as well. Or, even better, write a preprocessor in perl and share it with Bruce so he can incorporate it into Athena.
See AthenaFiletypePlugins for details about how to teach Athena to read arbitrary data files.
Should you choose to write email to Bruce asking about this situation, you must include an example of the data file or he will not be able to help.
What's the deal with Athena's compressed project files?
Starting with version 0.8.050, Athena writes out its project files using compression. The files written are identical to the format used by the popular gzip program, using the highest level of compression. Athena and Artemis will import these files or uncompressed project files from earlier versions of Athena in a transparent manner. It is quite possible that the user will never even realize that this change in the project file format occured.
One possible problem is if someone using the latest version of Athena wants to share a project file with someone who is not using the latest version. In that case, the project file will be written using compression, but the person with the older version will not be able to import it into his version of Athena. There are two solutions to this problem (other than the obvious solution of encouraging your colleague to update to the most recent version of the code).
The better solution is to rename the file from whatever.prj to whatever.prj.gz then use a decompression program to decompress the project. On unix, gzip is a good choice. On the Mac, there are a number of options, including gzip or stuffit. On Windows, WinZip works fine, as will any of the many other popular programs for dealing with compressed files. Simply run the renamed file through the decompression program and use the resulting file. There are no changes to the textual content of the project file -- the only change is that Athena uses compression by default.
The less good solution is for the person writing the project file to set the General->compress_prj preference parameter to false. This will get you over the hurdle of communicating with your colleague, but it is an inferior solution. Compression reduces the size of the project file by up to a factor of 4, which is a very nice feature indeed.
This issue was discussed in the release notes for version 0.8.050: HoraeVersion062.
How can I manage multiple FEFF path calculations in Artemis?
Some hints from the Ifeffit mailing list
Bruce says: Note that the discussion in the mailing list entries linked above refer to the version of Artemis that was current in November of 2003. Things have changed a lot since then. There is good advice in those links, but be mindful that some of the features of Artemis refered to have changed. (8 June, 2005)
How do I use the 'Fit background' option?
- Co-refine the background by selecting the fit background button.
- Play around with the value of Rmin to get a good background co-refinement without undue correlations with the fitting parameters. These correlations are reported in the log file. I would say that this procedure helps remove some of the mystery as the "correct" value for Rbkg -- the cutoff in Fourier space between background and data. If the correlations are quite high, that value is probably too high since you may be damaging the data by allowing the spline too much freedom.
- In the Data menu is an option that says 'Save background subtracted as chi(k)'. This will write a file containing the data with the fitted spline subtracted out to the project space. You will then be offered an opportunity to replace the data with the background subtracted data.
- Choosing to import the background subtracted data will change the data and unselect the "fit background" button.
- You can now continue along with your data analysis, using the newly background-subtracted data.
If you ever want to return to the non-background-subtracted data, select "Project data -> Import project data" from the File menu. This will open the file selection dialog in the folder where the project's data is stored. You will see the original data along with the background-subtracted data and any other data you might have (at some point) imported into your project.
From the Ifeffit mailing list http://millenia.cars.aps.anl.gov/pipermail/ifeffit/2004-October/001431.html
Why does Athena become unresponsive and consume 100% of the CPU
The situation described here should be fixed in Athena 0.8.051.
There is a situation where Athena enters a race condition that some users might run into. The culprit is the Groups List. When the mouse passes over an item in the groups list, you get a bit of visual feedback. The text turns blue and displays in a slightly larger font. The purpose of this is to provide a visual clue about when clicking the mouse button will select that group.
On some machine, the font used for this visual clue might not exist. perl/Tk deals with that situation particularly poorly. It seems to enter into a race condition from which it will never exit and which will consume the entire CPU. That's bad.
For folks using a version prior to 0.8.051, the simplest solution is to change the font used for the mouse-over event. Doing so is not trivial, but it is much better than having Athena be useless.
On a unix machine, go to the $HOME/.horae directory and open the file athenarc in a text editor. Find the fonts section, which should be near the end of the file. Change the "large" font to be identical to the "med" font. (On Windows the equivalent file is C:\Program Files\Ifeffit\horae\athena.ini).
I believe that this problem will only effect linux users and probably only a small subset of linux users. I saw this issue on a new installation of SUSE 10.1 and had never seen it on any other linux machine. I am mostly certain that Windows users will not see this bug and I think that Mac users will not see it either.
How do I make publication quality plots with Athena/Artemis? (Alternately: Why are the plots so ugly in Athena/Artemis?)
I completely agree that the plots made by pgplot are not pretty enough for publication and I suspect that Matt would not argue either. We use pgplot because it works well within the context of Ifeffit, because it is free and redistributable, and because it works equivalently on Windows, Unix, and the Mac. Matt and I both prefer to spend our limited time making good XAS software -- neither of us really felt particularly inclined to make and maintain publication quality plotting software. There are, after all, plenty of excellent packages out there for making beautiful plots.
The trick, then, is the link between Athena and your favorite plotting package. In the File menu, there are a number of option for exporting data into plain-text, column-data files. These can then be imported easily into your favorite plotting program. The File menu options from Save mu(E) through Save chi(q) will save the current data group (i.e. the one highlighted in the groups list and the one whose parameters are currently on display in the main window). The mu(E) file, for instance, contains columns for
energy data background I0 pre-edge line post-edge polynomial
You can import these columns into your favorite plotting program and have your way with them. The chi(R) file, as another example, has columns of
R real part imaginary part magnitude phase back-transform window
Below those options is a submenu labeled Save marked groups as data. The idea there is to save column data file where each column contains the data from a marked group (i.e. the ones with the little purple button in the groups list clicked). There are options for saving data in this way in each of the plotting spaces.
Finally, it is possible to have some control over how the plots are made. In the Settings menu, select Edit preferences. Then open up the Plot branch by clicking on its little plus sign. The options c0 through c9 and the linetypes option might be of interest to your. Nothing among the preferences will magically make pgplot prettier, but it may help you visualize your data better.
- Richard Mayes adds this helpful hint
- There's another way to get better looking plots out of Athena and Artemis (for those of us who can't completely give up microsoft just yet), although it's probably a little more time intensive than the options Bruce mentioned...in the pgplot menu, save the image as a metafile, then import the metafile into Powerpoint and convert it into a drawing object with the drawing toolbar (click on the image, then click "ungroup"). That gives you the options of broadening the lines, changing the colors, enlarging the numbers, etc. The drawing object can then be saved as a jpeg and imported into various other programs.
Fine, Athena and Artemis do not make publication quality plots. How can I modify the plots that are made?
Neither Athena nor Artemis offer controls for many aspects of the plot. For example, there are no controls for adding arrows or text labels to a plot; the title and axis labels are generated automatically, and there is scant control (via the preferences) over lines types and colors. That does not mean, however, that such things are impossible -- they are merely inconvenient
Athena and Artemis work by sending text commands to Ifeffit. If you open the Ifeffit buffer (see the Edit menu in either program), you will see a text entry box at the bottom of the palettes window. This is an ifeffit command line. That is, it is a place from which you can send arbitrary command to Ifeffit. For example, typing this command
plot_text(x=17200, y=-0.7, text="Hi mom!")
into the ifeffit command line in the Ifeffit buffer added the text in this plot:
This chapter of the Ifeffit Reference Manual has all the Ifeffit commands. Of particular interest are plot_arrow, plot_marker and plot_text. You can also cut-and-paste command from the Ifeffit buffer into the Ifeffit command line and modify them as desired. As Adam Webb suggested on the mailing list, you can make extensive modifications to your plots in that manner.
Why are the fonts so ugly in Athena and/or Artemis?
A lot of work went into horae release 065 to fix the situation with fonts and display of widgets in all the programs. Success was good, but not complete. The central issue is that Tk does not recognize font hinting or perform antialiasing. If a chosen font on a given screen resolution results in line widths of less than a pixel, the display of fonts will be somewhere between poor and illegible.
While it is certainly reasonable to suggest that a toolkit other than Tk be used, that is not a short term solution. If fonts fail on your system, send Bruce a screenshot and as much information about your system as possible. Using that, he can try to refine the solutions implemented in release 065.
Probably the best short term solution is to increase the font sizes used. Unfortunately, this cannot be done through the preferences dialog at this time. You will need to edit your initialization file by hand. Fortunately, that's not hard. On linux, edit ~/.horae/athenarc or ~/.horae/artemisrc. On Windows, look in C:\Program Files\Ifeffit\horae\. Increase the sizes of the fonts (which are found at the end of the file) and restart the program. Hopefully, that'll do the trick.
Athena does not fit on my small screen! I cannot see the OK button in the column selection dialog! How can I fix this?
It is certainly true that Athena and Artemis do not handle fonts as a well-behaved program should. Indeed, this is one of the weaker aspects of the programs. Unfortunately, poor font management is built very deeply into the code due to some poor decisions Bruce made way back when he started writing Athena in 2001. While there is no proper solution to this problem, there is a work-around.
Find your Athena configuration file. On a unix or Mac machine, it will be at $HOME/.horae/athenarc. On a Windows machine, it will be %APPDATA%\horae\athena.ini, where %APPDATA% is something like C:\Documents and Settings\bravel\Application Data\ (that is the location on my computer, yours should be something similar).
Open this file with a text editor (not a word processor! on Windows use something like Wordpad rather Word) and scroll to the bottom, where you will find lines like this:
[fonts] smbold=Helvetica 10 bold tiny=Helvetica 8 normal med=Helvetica 12 normal medit=Helvetica 12 italic bold=Helvetica 12 bold boldit=Helvetica 12 bold italic large=Helvetica 14 normal fixed=Courier 12 entry=Courier 10 entrybold=Courier 10 bold small=Helvetica 10 normal
These lines define the fonts used by Athena. Decrease each of the numbers. Turn the 10s into 9s, turn the 12s into 11s, and so on. Restart Athena and see if everything fits on the screen. If so -- yay! you are done. If not, decrease all the numbers until Athena looks ok once restarted.
The procedure for Artemis is the same, except that you need to edit the font size in the Artemis configuration file, artemisrc or artemis.ini.
How do I add a new FAQ entry?
Edit this section, use copy and paste! Use a format like this, using the same level of header (i.e. the same number of = signs) and keeping the link back to the contents at the top of the page:
== How do I add a new FAQ entry? == Edit this section, use copy and paste! [#contents Contents] ----