[Ifeffit] A basic question about data collection
newville at cars.uchicago.edu
Fri Aug 26 13:45:59 CDT 2005
On Fri, 26 Aug 2005, Anatoly Frenkel wrote:
> Hi Scott,
> Not exactly related to my last posint, here is an argument against any
> tampering with integration time, whether k^n or k^2n weighting...
> The Fourier transform of white noise is easy to calculate: it's magnitude
> does not depend on r, and this property is used as a measure of statistical
> errors in experimental data, by using the FT magnitude at higher r, as
> described in this document (page 5, eq. (6)):
> Since k-weighted integration time alters the noise, it is no longer white.
> That means, its FT at high r is not indicative of the noise at low r,
> because it varies with r.
Err, well, what we care about the noise in chi(R) which is
related by Parseval's theorem to the noise in chi(k)*k^n, the
k-weighted chi(k). We assume the noise in chi(R) is white, and
so also that the noise in chi(k)*k^n is white. If you want
white noise (ie, independent of k) in k-weighted chi(k), then
k-weighting the collection time is a good approach. It's not
perfect, but then the noise won't really be white or dominated
by statistical noise unless the spectra are really bad.
> Moreover, according to literature, this behavior
> is strongly dependent on the Fourier window function used.
What literature is that? The window function should have very
little effect on the estimated noise.
> Thus, Eq. (6)that is implemented in IFEFFIT, is no longer
> valid for statistical error analysis if variable integration
> time used. Eqs. (7) and (8) in this summary, that also can be
> used to estimate statistical errors do not allow to account
> for a measurement with variable integration times either.
Well, we want the noise in chi(R) (assuming we're fitting in
R-space). The Section 3 of the Error Report tries to use 'chi'
where it may mean 'chi(R)' or 'chi(k)' or 'chi(E)'. Eq. 6 gives
the estimated uncertainty in chi(k) [un-weighted] from the
estimated uncertainty in chi(R), assuming the noise in chi(R) is
white (independent of R). You read the report, right?
Eq. 7 (epsilon^2 = 1/N0 + 1/N, for N0 counts in I0 and N in If
or I) and 8 (a variation on Eq. 7) assumes the noise is
dominated by shot noise, and is most useful to estimate the
noise in mu(E), as the report states. But it can also be
applied point-by-point, and thus vary with counting time. The
report also says right after Eq. 8:
The average statistical error should be estimated from the
r.m.s. value of Eq. (7) or Eq. (8) over data segments with
similar statistical weight, e.g., over segments with a
constant integration time.
which does imply that there may be varying integration time. Of
course, the people writing the report knew about k-weighting the
> In summary, to my opinion, one may improve the data quality by
> varying integration time, but it is not straightforward how to
> quantify such an improvement in terms of the effect on
> statistical errors in the data made by such trick.
It is straightforward to take data with different collection
times and different k-weightings, do a bunch of FTs and compare
the values of 'epsilon_r' (the estimated noise in chi(R)),
'epsilon_k' (the estimated noise in un-weighted chi(k)), and
'kmax_suggest' (the estimated k above which the signal is
smaller than the noise).
More information about the Ifeffit