New perspectives, extensions and applications of De Bruijn identity Academic Article uri icon


  • Two significant identities-de Bruijn and Stein-were independently studied in information theory and statistics. De Bruijn identity shows a connection between two fundamental concepts in information theory and signal processing: differential entropy and Fisher information. On the other hand, Stein identity represents a relationship between the expectation of a function and its first-order derivative. Due to their several applications in statistics, information theory, probability theory, and economics, de Bruijn and Stein identities have attracted a lot of interest. In this study, two different extensions of de Bruijn identity and its relationship with Stein identity will be established. In addition, a number of applications using de Bruijn identity and its extensions will be introduced. The main theme of this study is to prove the equivalence between de Bruijn identity and Stein identity, in the sense that each identity can be derived from the other one. In a particular case, not only are de Bruijn and Stein identities equivalent, but they are also equivalent to the heat equation identity, which is another important result in statistics. The second major goal of this study is to extend de Bruijn identity in two distinctive ways. Given an additive non-Gaussian noise channel, the first-order derivative of differential entropy of the output signal is expressed as a function of the posterior mean, and the second-order derivative of differential entropy of the output signal is represented in terms of Fisher information. The third most important result is to introduce practical applications based on the results mentioned above. First, two fundamental bounds-the Bayesian Cramér-Rao lower bound (BCRLB) and the Cramér-Rao lower bound (CRLB)-in statistical signal processing, and a novel lower bound, tighter than BCRLB, are presented. Second, Costa's entropy power inequality is proved in two distinctive ways. Finally, min-max optimal training sequences for channel estimation and synchronization in the presence of unknown noise distribution are designed.

published proceedings

  • Qatar Foundation Annual Research Forum Volume 2012 Issue 1

author list (cited authors)

  • Park, S., Serpedin, E., & Qaraqe, K.

citation count

  • 0

complete list of authors

  • Park, Sangwoo||Serpedin, Erchin||Qaraqe, Khalid

publication date

  • October 2012