The Csiszár-Kullback inequality

Today we give an inequality, which is very useful in proving the convergence of a solution to PDEs tends to the stationary solution as time tends to infinity.

Lemma. Let $\Omega\in\mathbb R^d$ be a domain and let $f, g\in L^1(\Omega)$ satisfy $f\geq 0, g>0$ and $\int_\Omega fdx = \int_\Omega gdx = 1$. Furthermore, let $\Phi\in C^1(\mathbb R)$ satisfy

$\Phi(s) \geq \Phi(1) + \Phi'(1)(s-1) + \gamma^2(s-1)^21_{\{s<1\}}$

for all $s\in\mathbb R$ and some $\gamma>0$, where $1_{\{s<1\}}$ is the characteristic function on $A\subset \mathbb R$. Finally, let

$H_\Phi[f] = \int_\Omega \Phi\left(\dfrac{f}{g}\right)gdx$.

Then

$\|f - g\|^2_{L^1(\Omega)} \leq \dfrac{4}{\gamma^2}(H_\Phi[f] - H_\Phi[g])$

Proof. First, we try to estimate $H_\Phi[f] - H_\Phi[g]$.

$H_\Phi[f] - H_\Phi[g] = \int_{\Omega}(\Phi(f/g) - \Phi(1))gdx$

$\geq \int_{\Omega}(\Phi'(1)(f/g-1) + \gamma^2(f/g-1)^21_{\{f/g<1\}})gdx$

$= \Phi'(1)\int_{\Omega}(f - g)dx + \gamma^2\int_{f

$= \gamma^2\int_{f (since $\int_\Omega fdx = \int_\Omega gdx =1$)

Now, it is sufficient to prove that

$\gamma^2\int_{f

or equivalently

$4\int_{f

We have

$\|f - g\|_{L^1(\Omega)} = \int_{\Omega}|f-g|dx$

$= \int_{f

$= \int_{f

$= \int_{f

$= 2\int_{f (since $\int_\Omega fdx = \int_\Omega gdx = 1$).

We now have to show that

$\int_{f.

Using Hölder’s inequality and $\int_\Omega gdx = 1$, we have

$\int_{f

$\leq \int_{f

$\leq \left(\int_{f

$= \left(\int_{f

Thus,

$(\int_{f.

This completes the proof.