Hypothesis Testing and Asymptotic Distinguishability

Quantum hypothesis testing asks a concrete question: which of two declared state preparations produced the data? The answer depends not only on the states, but also on the allowed observable algebra, the tolerated false-alarm probability, the number and independence of preparations, and any detector or energy constraint. Relative entropy controls an important many-copy limit; it is not itself a one-shot error probability.

Required background. Start from Relative Entropy for QFT States.

The chapter’s task-comparison table separates asymmetric testing from fidelity, bounded-observable bias, and channel discrimination. The validity map records the resources that must remain fixed.

Binary tests on a declared algebra

Let $\rho$ be the null hypothesis and $\sigma$ the alternative. A two-outcome effect $0\leq Q\leq I$ means “accept $\rho$” on outcome $Q$ and “accept $\sigma$” on outcome $I-Q$. The two errors are

$$\alpha(Q)=\operatorname{Tr}[(I-Q)\rho], \qquad \beta(Q)=\operatorname{Tr}(Q\sigma).$$

Thus $\alpha$ is the probability of rejecting $\rho$ when $\rho$ is true, while $\beta$ is the probability of accepting $\rho$ when $\sigma$ is true. For a fixed tolerance $0<\varepsilon<1$, set

$$\beta_\varepsilon(\rho\Vert\sigma) =\inf\left\{ \operatorname{Tr}(Q\sigma): 0\leq Q\leq I, \operatorname{Tr}(Q\rho)\geq1-\varepsilon \right\},$$

and define the hypothesis-testing divergence, in the site’s natural-log convention, by

$$D_H^\varepsilon(\rho\Vert\sigma) =-\log\beta_\varepsilon(\rho\Vert\sigma).$$

The quantum Neyman–Pearson test is a threshold effect built from the positive part of $\rho-e^t\sigma$; randomization on the zero eigenspace may be needed to attain the prescribed $\varepsilon$. The definition itself is more general than a projective measurement and already includes that boundary randomization.

For normal states $\omega,\varphi$ on a von Neumann algebra $\mathfrak M$, replace the traces by

$$\alpha_\omega(Q)=\omega(I-Q), \qquad \beta_\varphi(Q)=\varphi(Q), \qquad Q\in\mathfrak M.$$

The choice of $\mathfrak M$ is operational: it is exactly the collection of bounded observables from which the test may be assembled. Restricting to a detector subalgebra can only make the optimal test worse. General von Neumann-algebra versions of the key testing lemma and Stein asymptotics are established by Pautrat and Wang 2023, Theorem 1, Remark 2, and Theorems 4–6, pp. 2325–2335.

One normalization check prevents a common misconception. If $\rho=\sigma$, then $\beta(Q)=1-\alpha(Q)$, so

$$D_H^\varepsilon(\rho\Vert\rho)=-\log(1-\varepsilon),$$

not zero. The one-shot quantity includes the allowed error budget; its per-copy rate is what vanishes.

One-shot divergences and smoothing

Hypothesis-testing divergence is one member of a task-dependent one-shot family. Two other relative quantities are

$$\begin{aligned} D_{\max}(\rho\Vert\sigma) &=\inf\{\lambda\in\mathbb R:\rho\leq e^\lambda\sigma\},\\ D_{\min}(\rho\Vert\sigma) &=-\log\operatorname{Tr}(\Pi_\rho\sigma), \end{aligned}$$

where $\Pi_\rho$ is the support projection of $\rho$. With the definitions above,

$$D_H^0(\rho\Vert\sigma)=D_{\min}(\rho\Vert\sigma).$$

Smoothing replaces $\rho$ by nearby subnormalized states before optimizing. For positive operators of trace at most one, the generalized root fidelity is

$$\overline F(\widetilde\rho,\rho) = \left\lVert\sqrt{\widetilde\rho}\sqrt\rho\right\rVert_1 +\sqrt{ (1-\operatorname{Tr}\widetilde\rho) (1-\operatorname{Tr}\rho) }.$$

When $\rho$ is normalized, the second term vanishes. After declaring the purified distance

$$P(\widetilde\rho,\rho) =\sqrt{1-\overline F(\widetilde\rho,\rho)^2},$$

one may define

$$D_{\max}^{\varepsilon,P}(\rho\Vert\sigma) =\inf_{\widetilde\rho:\,P(\widetilde\rho,\rho)\leq\varepsilon} D_{\max}(\widetilde\rho\Vert\sigma).$$

The superscript $P$ is useful here because a trace-distance ball with the same numerical radius is a different set. Normalized versus subnormalized smoothing also changes boundary terms. Relative $D_{\min/\max}$ should not be conflated with conditional $H_{\min/\max}$, whose optimizations and operational tasks are different. Datta 2009, Definitions 1–2 and 4, pp. 2818 and 2823 uses base-two logarithms and a trace-norm smoothing convention; Tomamichel and Hayashi 2013, Definitions 2–5 and Eq. (6), pp. 7697–7700 uses purified-distance smoothing. Every imported numerical formula must translate both choices.

The structure map shows why these quantities should not be merged with symmetric overlap or recovery measures.

Hypothesis-testing divergence is operational for a declared effect algebra and error tolerance. Its relative-entropy limit requires an independent-copy experiment; overlap, correlation, and recovery branches have different inputs and conclusions. Schematic.

What n copies means in QFT

An iid experiment with $n$ preparations uses

$$\rho_n=\rho^{\otimes n}, \qquad \sigma_n=\sigma^{\otimes n},$$

and permits a declared class of effects on the product algebra. A collective effect on all copies, a product of one-copy effects, and an adaptive sequence are different measurement resources. The usual quantum Stein and Chernoff theorems optimize over unrestricted collective effects.

This tensor power is a preparation model, not a geometric fact. Spacelike-separated regions of one field state are generally correlated, so their joint restriction is not automatically $\omega^{\otimes n}$. Enlarging a lattice region, repeating one detector on freshly prepared systems, and measuring $n$ subsystems of one many-body state define three different asymptotic sequences. The QFT-facing distinctions and collective-measurement protocol are developed in de Boer et al. 2021, §§ 2.2, 4, and 7–8.

For a continuum claim, report:

• how the independent preparations are produced;
• the product algebra and admissible collective tests;
• the state order $\rho$ versus $\sigma$ and the support condition;
• the fixed $\varepsilon$, smoothing convention, detector support, and energy set; and
• the order of the limits in copy number, volume, and ultraviolet cutoff.

Without those data, “many measurements” does not identify a theorem.

Stein’s threshold and the second-order term

Assume finite-dimensional density operators with $\operatorname{supp}\rho\subseteq\operatorname{supp}\sigma$. For a sequence of tests, call

$$r=\liminf_{n\to\infty}-\frac1n\log\beta_n$$

the achieved type-II error exponent. Every rate $r<D(\rho\Vert\sigma)$ is achievable with $\alpha_n\to0$, while demanding a rate $r>D(\rho\Vert\sigma)$ forces $\alpha_n\to1$. This direct/strong-converse threshold is quantum Stein’s lemma; see Ogawa and Nagaoka 2000, Theorems 2–3 and Corollary 1, pp. 2429–2432.

The fluctuations are governed by the relative-entropy variance

$$V(\rho\Vert\sigma) =\operatorname{Tr}\rho \left( \log\rho-\log\sigma-D(\rho\Vert\sigma) \right)^2.$$

For fixed $0<\varepsilon<1$ and nondegenerate $0<V(\rho\Vert\sigma)<\infty$,

$$D_H^\varepsilon(\rho^{\otimes n}\Vert\sigma^{\otimes n}) =nD(\rho\Vert\sigma) +\sqrt{nV(\rho\Vert\sigma)}\,\Phi^{-1}(\varepsilon) +O(\log n),$$

where $\Phi$ is the standard-normal cumulative distribution. This is Li 2014, Theorem 2 and Eqs. (1)–(9), pp. 175–179; Theorem 5, pp. 185–186. If $\varepsilon<1/2$, then $\Phi^{-1}(\varepsilon)<0$, so the second-order correction lowers the finite-copy exponent. The formula is an asymptotic expansion, not a finite-$n$ upper or lower bound. The case $V=0$ is degenerate and requires a separate remainder analysis.

Support failure is not a small correction. Let $P$ project onto $\ker\sigma$ and suppose $p=\operatorname{Tr}(P\rho)>0$. The test

$$Q_n=I-(I-P)^{\otimes n}$$

accepts $\rho$ if any copy lands in that singular sector. It has $\beta_n=0$ and $\alpha_n=(1-p)^n$. Hence, for every fixed $\varepsilon>0$, $D_H^\varepsilon=+\infty$ for all sufficiently large $n$; a finite $D,V$ expansion is then inapplicable.

Exact wavepacket detector benchmark

Consider one normalized bosonic wavepacket mode of a finite-volume regulated free field,

$$[a,a^\dagger]=1, \qquad q=\frac{a+a^\dagger}{\sqrt2}, \qquad H=\omega\left(a^\dagger a+\frac12\right).$$

Let $\tau_N$ be the thermal state of mean occupation $N=1/2$, and compare

$$\sigma=\tau_N, \qquad \rho=D(1)\tau_ND(1)^\dagger.$$

Both are faithful Gaussian states with finite energy:

$$\langle H\rangle_\sigma=\omega, \qquad \langle H\rangle_\rho=2\omega.$$

An ideal homodyne detector for the smeared quadrature $q$ produces the classical laws

$$q\mid\rho\sim\mathcal N(\sqrt2,1), \qquad q\mid\sigma\sim\mathcal N(0,1).$$

The detector-restricted log-likelihood therefore has

$$D_q=1, \qquad V_q=2.$$

The unrestricted quantum pair retains more information:

$$D(\rho\Vert\sigma)=\log3, \qquad V(\rho\Vert\sigma)=2(\log3)^2.$$

This strict gap is data processing in numbers: measuring only $q$ discards part of the state distinguishability.

For $n$ independent preparations and $\varepsilon=0.1$, accepting $\rho$ above the Gaussian sample-mean threshold gives the exact detector-restricted error

$$\beta_{n,q}^{0.1} =\Phi\!\left[-\sqrt{2n}-\Phi^{-1}(0.1)\right].$$

The comparison with $nD_q=n$ and $nD_q+\sqrt{nV_q}\,\Phi^{-1}(0.1)$ is:

Copies $n$	Exact $\beta_{n,q}^{0.1}$	Exact $D_{H,q}^{0.1}$	First order	Second order
1	0.44723035	0.80468149	1	−0.81238760
4	0.060946628	2.79775675	4	0.37522479
16	$6.0632046\times10^{-6}$	12.0132721	16	8.75044958
64	$5.5042538\times10^{-24}$	53.5565210	64	49.5008992

The negative $n=1$ second-order value is not a paradox: it visibly demonstrates that an asymptotic truncation need not be a valid one-copy bound. The machine-readable benchmark freezes the state order, covariance and energy conventions, detector algebra, thresholds, both errors, and all first- and second-order values.

Detector, smoothing, and ultraviolet failures

Rotate the detector. Measuring the quadrature orthogonal to the real displacement gives identical outcome laws. The full states still differ, but the detector sees $D_H^\varepsilon=-\log(1-\varepsilon)$ for every $n$, so its per-copy exponent tends to zero.

Change the smoothing ball. A purified-distance radius and a trace-distance radius with the same numeral do not select the same nearby states. Reusing a smooth-entropy formula after silently changing the metric changes the optimization problem.

Keep only a mean-energy bound. Let diagonal oscillator distributions satisfy

$$p_m\propto(m+1)^{-3}, \qquad q_m\propto p_m e^{-m}.$$

Both have finite mean energy, and $\log(p_m/q_m)=m+\text{constant}$ has a finite expectation under $p$. Its variance is infinite because $\sum_m m^2p_m$ diverges. Thus finite mean energy and finite relative entropy do not by themselves license the $\sqrt n$ expansion.

Move the cutoff with $n$. A detector support, lattice spacing, or ultraviolet projection that changes along the sequence defines a new sequence of tests and states. Its effect must be controlled by a theorem; it cannot be hidden inside the $O(\log n)$ remainder.

For the operator-algebraic divergence hierarchy behind these restrictions, continue to the rigorous noncommutative treatment. The next pages compare fidelity and Chernoff testing and operational norm bounds.

Common pitfalls

Calling repeated regions independent copies. Correlated subregions of one field state do not form an iid tensor power. Declare the preparation channel that produces independent replicas.

Treating the second-order expression as a bound. Its $O(\log n)$ remainder is uncontrolled until a finite-blocklength theorem supplies constants. At small $n$, the truncated expression can even be negative.

Forgetting the state order. Stein testing is asymmetric: swapping $\rho$ and $\sigma$ changes the relative entropy, variance, support branch, and meaning of the two errors.

Assuming finite energy implies finite variance. A first moment of $H$ need not control the second moment of the log-likelihood operator. Check the moment used by the theorem, not a nearby physical quantity.

Exercises

1. Error tolerance and min-relative entropy

Show that $D_H^\varepsilon(\rho\Vert\sigma)$ is nondecreasing in $\varepsilon$ and that $D_H^0=D_{\min}$ with the definitions on this page.

Solution

If $\varepsilon_2\geq\varepsilon_1$, every test feasible at $\varepsilon_1$ is feasible at $\varepsilon_2$. The infimum of $\beta$ can therefore only decrease, so its negative logarithm can only increase.

At $\varepsilon=0$, feasibility requires $\operatorname{Tr}(Q\rho)=1$. Because $0\leq Q\leq I$, this forces $Q$ to act as the identity on $\operatorname{supp}\rho$, hence $Q\geq\Pi_\rho$. The smallest type-II error is attained by $Q=\Pi_\rho$:

$$\beta_0=\operatorname{Tr}(\Pi_\rho\sigma), \qquad D_H^0=-\log\operatorname{Tr}(\Pi_\rho\sigma)=D_{\min}.$$

2. Derive the Gaussian test

For $p=\mathcal N(\mu,v)$ and $q=\mathcal N(0,v)$ with $\mu>0$, derive the likelihood-ratio threshold at type-I tolerance $\varepsilon$. Show that

$$\beta_n^\varepsilon =\Phi\!\left(-\sqrt n\,\frac{\mu}{\sqrt v} -\Phi^{-1}(\varepsilon)\right).$$

Evaluate it for $\mu=\sqrt2$, $v=1$, $n=4$, and $\varepsilon=0.1$.

Solution

The log-likelihood ratio is affine and increasing in the sample mean $\bar q$. Accept $p$ when $\bar q\geq c$. Under $p$, $\bar q\sim\mathcal N(\mu,v/n)$, so

$$\varepsilon =\Pr_p(\bar q<c) =\Phi\!\left(\sqrt{\frac nv}(c-\mu)\right),$$

which gives $c=\mu+\sqrt{v/n}\,\Phi^{-1}(\varepsilon)$. Under $q$,

$$\beta_n^\varepsilon =\Pr_q(\bar q\geq c) =\Phi\!\left(-\sqrt{\frac nv}\,c\right),$$

and substitution yields the result. For the stated values, $c\simeq0.77343778$, $\beta_4^{0.1}\simeq0.060946628$, and $D_{H,q}^{0.1}\simeq2.79775675$ nats.

3. Detect singular support

Let $P$ project onto $\ker\sigma$ and let $p=\operatorname{Tr}(P\rho)>0$. Verify the errors of $Q_n=I-(I-P)^{\otimes n}$ and find the smallest $n$ for which it is feasible at tolerance $\varepsilon$.

Solution

Because $\sigma$ has no support in $P$,

$$\beta_n=\operatorname{Tr}(Q_n\sigma^{\otimes n})=0.$$

Under $\rho^{\otimes n}$, the test rejects $\rho$ only if no copy lands in $P$, so

$$\alpha_n=(1-p)^n.$$

It is feasible when $(1-p)^n\leq\varepsilon$, equivalently

$$n\geq \frac{\log\varepsilon}{\log(1-p)}.$$

Taking the ceiling gives the smallest integer $n$. At and above that value, $D_H^\varepsilon=+\infty$.

4. Blind detector and vanishing rate

Suppose the two measured outcome distributions are identical on every copy. Prove that the optimal $n$-copy hypothesis-testing divergence is $-\log(1-\varepsilon)$ and explain why this is compatible with a zero Stein rate.

Solution

For identical outcome laws, every test satisfies $\beta=1-\alpha$. The constraint $\alpha\leq\varepsilon$ implies $\beta\geq1-\varepsilon$, and equality is reached by accepting the null with constant probability $1-\varepsilon$. Hence

$$D_H^\varepsilon=-\log(1-\varepsilon)$$

for every $n$. Dividing by $n$ sends this fixed normalization term to zero, so the asymptotic rate vanishes even though the one-shot divergence does not.

References

• Datta, Nilanjana. “Min- and Max-Relative Entropies and a New Entanglement Monotone.” IEEE Transactions on Information Theory 55, no. 6 (2009): 2816–2826. DOI. Open preprint.
• de Boer, Jan, Victor Godet, Jani Kastikainen, and Esko Keski-Vakkuri. “Quantum Hypothesis Testing in Many-Body Systems.” SciPost Physics Core 4 (2021): 019. DOI. Open preprint.
• Li, Ke. “Second-Order Asymptotics for Quantum Hypothesis Testing.” The Annals of Statistics 42, no. 1 (2014): 171–189. DOI. Open preprint.
• Ogawa, Tomohiro, and Hiroshi Nagaoka. “Strong Converse and Stein’s Lemma in Quantum Hypothesis Testing.” IEEE Transactions on Information Theory 46, no. 7 (2000): 2428–2433. DOI. Open preprint.
• Pautrat, Yan, and Simeng Wang. “Ke Li’s Lemma for Quantum Hypothesis Testing in General von Neumann Algebras.” Annales Henri Poincaré 24 (2023): 2323–2339. DOI. Open preprint.
• Tomamichel, Marco, and Masahito Hayashi. “A Hierarchy of Information Quantities for Finite Block Length Analysis of Quantum Tasks.” IEEE Transactions on Information Theory 59, no. 11 (2013): 7693–7710. DOI. Open preprint.