We present a result on the local stability of the Bethe CCCP algorithm, proposed by Yuille as an alternative to the belief propagation for solving minimization problems of Bethe free energies. We first show that the Bethe CCCP algorithm can be interpreted within our framework, based on information geometry, to analyze the belief propagation and related algorithms, and then establish the local stability of the Bethe CCCP algorithm by showing that the Bethe CCCP algorithm is derived as the first-order implicit discretization scheme for the natural gradient algorithm to minimize Bethe free energy.