Trellis source code design as an optimization problem

The design of time-invariant trellis codes for stationary ergodic discrete-time sources is cast as an unconstrained, nonlinear optimization problem, where the objective function and its derivatives are evaluated by simulation. Using classical real analysis and the ergodic theorem, convergence of the sample encoding distortion and its partial derivatives (with respect to the code quantization levels) to their ensemble average values is investigated. It is found that in the common code design situation, the expected per-symbol distortion and its first derivatives are available and piecewise continuous, but second-derivative information is unreliable. This indicates that efficient optimization should be performed using a nonderivative or first-derivative method that does not compute approximate second derivatives to determine search directions.< >