Characterizing Solutions

To solve either or , we take as our guiding philosophy: do not discretize until (unless) necessary. In many cases, we can actually characterize solutions to and by solving a system of nonlinear equations in , which we can then solve numerically.

In short, our characterization is obtained by considering the dual problems: and where can be explicitly calculated, and .

For example, in maximum entropy problems, we begin with with dual problem To solve , we observe that is concave, and so a maximum occurs exactly when . That is, we solve for , which is a system of nonlinear equations:

Once the optimal dual vector is obtained, we can recover the optimal function from the formula when .

In the ME case, , , so the maximum entropy solution is given by which is a functional form of the solution.