Diagnose computational and evidence readiness

This diagnostic checks whether a numerical result is supported—not whether you know a particular programming language. Use a calculator, spreadsheet, notebook, short program, or documented manual computation. Your work should make precision, convergence, tests, correlated uncertainty, and reproduction visible enough that another reader can distinguish “it ran” from “the stated claim is justified.”

The tasks are untimed. A table can replace a plot, and a spoken explanation can be paired with equations or code in an accessible companion format. Statistical and probability reasoning is helpful background; if effective sample size or correlated uncertainty is the main obstacle, take the statistics diagnostic first.

Precision and convergence

Approximate $f'(1)$ for $f(x)=e^x$ with the central difference

$$D(h)=\frac{f(1+h)-f(1-h)}{2h}.$$

1. Derive its leading truncation error by Taylor expansion.
2. Evaluate a sequence of decreasing positive $h$ values. Tabulate $h$, $D(h)$, error relative to $e$, and an observed convergence order.
3. Identify the range in which the expected order is visible and any small-$h$ turnover caused by cancellation or roundoff.
4. Repeat selected points at higher precision. State a tolerance justified by the observed regime rather than chosen after seeing a preferred answer.

Answer guide

Taylor expansion about $x=1$ gives

$$D(h)=f'(1)+\frac{h^2}{6}f^{(3)}(1)+\mathcal O(h^4) =e+\frac{e}{6}h^2+\mathcal O(h^4).$$

Thus exact arithmetic predicts second-order error. For two steps $h$ and $h/2$ in the truncation-dominated regime, an observed order $p=\log_2(|E(h)|/|E(h/2)|)$ should approach $2$. At sufficiently small $h$, the subtraction loses significant digits and roundoff grows roughly like machine precision divided by $h$. Higher precision should move that turnover to smaller $h$. A defensible tolerance is tied to the stable refinement range and precision repeat; “the smallest $h$” is not automatically best.

Tests that can fail

Using the same implementation, add three checks:

• compare with the analytic derivative at more than one point;
• verify exactness within declared precision for a constant and a linear function; and
• introduce a deliberate sign, argument, or indexing error, show a test failing because of it, then repair the implementation.

For each check, state the expected value, tolerance, observed result, failure message or discrepancy, and one defect that the check would not detect.

What good evidence looks like

The analytic comparisons test both the formula and point handling, while the polynomial cases isolate simple structural behavior. The deliberately broken case demonstrates that the test is sensitive to at least one realistic failure rather than merely documenting successful execution. No finite test set proves the implementation correct: for example, a derivative routine can pass tests at symmetric points while mishandling an adaptive step rule elsewhere. State that remaining scope explicitly.

Correlated uncertainty

Generate or inspect a stationary sequence with positive autocorrelation—for example, an autoregressive process $x_t=\rho x_{t-1}+\epsilon_t$ with $0<\rho<1$ and independent Gaussian innovations. Estimate its mean in two ways:

1. use the independent-sample standard error $s/\sqrt N$;
2. use blocking, the autocorrelation function, or another covariance-aware method.

Record the generation process, discarded initial segment, observable, chain length, autocorrelation or block choice, effective sample size, and uncertainty on the mean. Vary the window or block size and discuss stability.

Answer guide

For a stationary AR(1) process, $\rho_k=\rho^k$ and the integrated autocorrelation time under the convention $\tau_{\mathrm{int}}=\tfrac12+\sum_{k\ge1}\rho_k$ is

$$\tau_{\mathrm{int}}=\frac{1+\rho}{2(1-\rho)}.$$

Consequently $N_{\mathrm{eff}}\approx N/(2\tau_{\mathrm{int}}) =N(1-\rho)/(1+\rho)$ and

$$\operatorname{SE}(\bar x) \approx\frac{s}{\sqrt{N_{\mathrm{eff}}}} =\frac{s}{\sqrt N}\sqrt{\frac{1+\rho}{1-\rho}}.$$

Positive correlation therefore makes the independent-sample error too small. Finite data require an estimated window or block size; stability under reasonable changes is part of the result. A stable mean alone does not prove stationarity or equilibration.

Reproduce the calculation from a clean description

Prepare a compact reproduction packet for the preceding tasks:

mathematical target and reference value:
formula or source version:
exact inputs and preprocessing:
software, libraries, platform, and precision:
command or interaction sequence:
tolerances, stopping rules, and failure handling:
random generator and seed policy, if used:
raw outputs and analysis decisions:

Use only this packet to repeat the work in a clean environment, or ask another person or implementation to do so. Note every missing instruction. State whether the result is a rerun of the same implementation, a reanalysis of the same outputs, an independent implementation, or a regeneration from new data; these provide different degrees of independence.

How to interpret the repeat

A successful rerun shows that the recorded implementation and inputs can be executed again in the tested environment. An independent implementation adds protection against shared code defects, but may still share the same model or formula. Agreement within a predeclared tolerance supports the bounded numerical target; it does not automatically validate an analogous QFT model, regulator, observable, or continuum limit.

Keep each claim within its evidence

For your work, write one sentence about each claim below and mark it supported, partly supported, or unsupported:

1. The procedure executed in the described environment.
2. The implementation passed the stated tests.
3. The approximation converged with the expected order over the tested range.
4. The correlated mean has a defensible uncertainty.
5. The result was repeated or independently reproduced.
6. An analogous method establishes a particular scientific conclusion in QFT.

The sixth claim should normally remain unsupported: these elementary tasks do not specify a physical theory, regulator, observable, regime, continuum limit, or systematic-error model. This “claim ceiling” is the central habit the diagnostic is testing.

Interpret the work

• Demonstrated: the refinement study matches its analytic error model; tests expose their intended defect; dependence changes the uncertainty in a justified way; a clean or independent repeat succeeds; and every conclusion stays within the evidence.
• Uncertain: the results are plausible, but a tolerance, order estimate, stationarity choice, missing reproduction step, or degree of independence is implicit.
• Not yet demonstrated: one resolution or a successful run is the only check; correlated samples are treated as independent; the work cannot be reconstructed; or numerical agreement is asserted as proof of a physical theory.

These labels describe this work product, not general programming ability. For either of the last two results, use Numerical methods and reproducibility. If correlated uncertainty is the blocking step, first use Statistical ensembles and probability. Then repeat the exercise with a numerical integral known analytically and a fresh correlated sequence. Return to the path that requested the check.

References

• Nicholas J. Higham, Accuracy and Stability of Numerical Algorithms, second edition, Society for Industrial and Applied Mathematics, 2002, doi:10.1137/1.9780898718027.
• David W. Hogg and Daniel Foreman-Mackey, “Data Analysis Recipes: Using Markov Chain Monte Carlo,” The Astrophysical Journal Supplement Series 236, no. 1 (2018): 11, doi:10.3847/1538-4365/aab76e.
• David J. C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003, author-hosted open PDF.
• Alan D. Sokal, “Monte Carlo Methods in Statistical Mechanics: Foundations and New Algorithms,” 1997, arXiv:hep-lat/9405016.