The optimal production policy

By the choice of the action space, all stationary policies have transition probability matrices representing recurrent aperiodic Markov chains. If the number of possible states is limited, i.e. if the one-period demand is bounded, we can determine the optimal production policy. The optimal policy can be determined by a policy iteration method, but we will use the method of successive iteration, as described by Odoni(1969), since this method is faster in our situation. The optimal policy is the policy which achieves the minimum expected costs per transition, which will be denoted by g. Defining the quantity v (r) as the total expected costs from the next n transitions if the current state is r and if an optimal policy is followed, the iteration scheme takes the form described in the optimality principle by Bellman (1957)  

Then according to Odoni for any choice of starting conditions vo(r)  

This function v(r) will be called the relative costs of a state r. 

It is not necessary to consider all possible values for first component of the order state vector ri. If for some state reR, action a 0 is optimal, then the same action will be optimal for all states (y,rz,r i.rt ) with y ri, because is not involved in the holding costs. Beginning with vo(r)=0 for all reR, v/c repeat (4.2.4) until a satisfactory 

---

There are numerous papers that assume that demand is a Poisson process. One important early one is Li [97], who studied price and production problems with capacity investment at the beginning of the horizon. Given an initial production capacity, the demand and production rates are Poisson counting processes if there is demand in excess of production, sales are lost. Li shows that the optimal production policy under a single fixed price is a barrier or threshold policy, where production is optimal if inventory is below a certain value. Further, he characterizes the optimal policy when price is dynamic over time, and he shows that the stochastic price is always higher than the deterministic price. In extensions, he considers the case with production learning effects that is, the production rate over time becomes closer to the ideal capacity. 

---