Anatoly, On Fri, 26 Aug 2005, Anatoly Frenkel wrote:
Hi Scott,
Not exactly related to my last posint, here is an argument against any tampering with integration time, whether k^n or k^2n weighting...
The Fourier transform of white noise is easy to calculate: it's magnitude does not depend on r, and this property is used as a measure of statistical errors in experimental data, by using the FT magnitude at higher r, as described in this document (page 5, eq. (6)):
http://ixs.iit.edu/subcommittee_reports/sc/err-rep.pdf
Since k-weighted integration time alters the noise, it is no longer white. That means, its FT at high r is not indicative of the noise at low r, because it varies with r.
Err, well, what we care about the noise in chi(R) which is related by Parseval's theorem to the noise in chi(k)*k^n, the k-weighted chi(k). We assume the noise in chi(R) is white, and so also that the noise in chi(k)*k^n is white. If you want white noise (ie, independent of k) in k-weighted chi(k), then k-weighting the collection time is a good approach. It's not perfect, but then the noise won't really be white or dominated by statistical noise unless the spectra are really bad.
Moreover, according to literature, this behavior is strongly dependent on the Fourier window function used.
What literature is that? The window function should have very little effect on the estimated noise.
Thus, Eq. (6)that is implemented in IFEFFIT, is no longer valid for statistical error analysis if variable integration time used. Eqs. (7) and (8) in this summary, that also can be used to estimate statistical errors do not allow to account for a measurement with variable integration times either.
Well, we want the noise in chi(R) (assuming we're fitting in R-space). The Section 3 of the Error Report tries to use 'chi' where it may mean 'chi(R)' or 'chi(k)' or 'chi(E)'. Eq. 6 gives the estimated uncertainty in chi(k) [un-weighted] from the estimated uncertainty in chi(R), assuming the noise in chi(R) is white (independent of R). You read the report, right? Eq. 7 (epsilon^2 = 1/N0 + 1/N, for N0 counts in I0 and N in If or I) and 8 (a variation on Eq. 7) assumes the noise is dominated by shot noise, and is most useful to estimate the noise in mu(E), as the report states. But it can also be applied point-by-point, and thus vary with counting time. The report also says right after Eq. 8: The average statistical error should be estimated from the r.m.s. value of Eq. (7) or Eq. (8) over data segments with similar statistical weight, e.g., over segments with a constant integration time. which does imply that there may be varying integration time. Of course, the people writing the report knew about k-weighting the collection time.
In summary, to my opinion, one may improve the data quality by varying integration time, but it is not straightforward how to quantify such an improvement in terms of the effect on statistical errors in the data made by such trick.
It is straightforward to take data with different collection times and different k-weightings, do a bunch of FTs and compare the values of 'epsilon_r' (the estimated noise in chi(R)), 'epsilon_k' (the estimated noise in un-weighted chi(k)), and 'kmax_suggest' (the estimated k above which the signal is smaller than the noise). --Matt