Von Neumann entropy

Intuition

The von Neumann entropy $S=-\operatorname{tr}(\hat{\rho}\ln\hat{\rho})$ measures how mixed a state is in a way that adds up properly: it is the Shannon entropy of the eigenvalues of $\hat{\rho}$ . It is zero exactly for pure states and largest for the completely mixed state, where it is $\ln d$ . Counted in bits, with logarithms to base 2, a spin $\tfrac12$ holds at most one bit. Isolated evolution leaves it unchanged; and among all states with a given mean energy, the thermal state has the most entropy — which is what makes it the state of equilibrium.

How many yes-or-no questions would you need, on average, to learn which member of an ensemble you hold? That count is the entropy in bits: none if you already know, one for a fair coin, more for a die.

The entropy in bits of a spin whose density matrix has eigenvalues $p$ and $1-p$ : zero for a pure state at either end, and one bit, its largest, for the even mixture $p=\tfrac12$ .

For a density matrix with eigenvalues $\lambda_{i}$ , taking $0\ln0=0$ :

S(\hat{\rho})=-\operatorname{tr}\bigl(\hat{\rho}\ln\hat{\rho}\bigr)=-\sum_{i}\lambda_{i}\ln\lambda_{i}

Properties

$0\le S\le\ln d$ : zero only for pure states, $\ln d$ only for $\hat{\rho}=\hat{1}/d$ .

The entropy is at most \ln d

S(\hat{\rho})\le\ln d,\qquad S=\ln d\iff\hat{\rho}=\hat{1}/d

Compare the entropy with $\ln d$ term by term. The logarithm lies below its tangent at one, $\ln x\le x-1$ , and applying that to each term turns the difference into a sum that cannot be positive, because there are at most $d$ nonzero eigenvalues adding to one. Equality needs every term at the tangent point, which means all $d$ eigenvalues equal .

Proof steps

S-\ln d=\sum_{\lambda_{i}>0}\lambda_{i}\ln\frac{1}{d\,\lambda_{i}}

Write $\ln d=\sum\lambda_{i}\ln d$ , since the $\lambda_{i}$ add to one.

\ln x\le x-1\qquad(x>0)

The logarithm lies below its tangent at $x=1$ , touching it only there.

S-\ln d\le\sum_{\lambda_{i}>0}\lambda_{i}\Bigl(\frac{1}{d\,\lambda_{i}}-1\Bigr)=\sum_{\lambda_{i}>0}\frac1d-1

Apply it with $x=1/(d\lambda_{i})$ .

\sum_{\lambda_{i}>0}\frac1d\le1\ \Longrightarrow\ S\le\ln d

At most $d$ eigenvalues are nonzero.

S=\ln d\iff\lambda_{i}=\tfrac1d\ \ \text{for every}\ i

Equality needs $d\lambda_{i}=1$ in every term, with all $d$ terms present.

Applications

Practice

Entropy of the Eigenvalues

The von Neumann entropy is the Shannon entropy of the eigenvalues of $\hat{\rho}$ . In bits, an even mixture of two states has entropy one.

S=-\sum_{i}\lambda_{i}\log_{2}\lambda_{i}\ \ \text{bits}

Try it

What is the entropy, in bits, of the completely mixed state of a spin $\tfrac12$ ?