Modeling the System Acquisition Using Deep Reinforcement Learning

The process of acquiring large-scale complex systems is usually characterized by cost and schedule overruns. We develop and evaluate a model of the acquisition process that accounts for the strategic behavior of different parties. Specifically, we cast our model in terms of government-funded projects and assume the following steps. First, the government publishes a request for bids. Then, private firms offer their proposals in a bidding process and the winner bidder enters in a contract with the government. The contract describes the system requirements and the corresponding monetary transfers for meeting them. The winner firm devotes effort to deliver a system that fulfills the requirements. This can be assumed as a game that the government plays with the bidder firms. The objective of this paper is to study how different parameters in the acquisition procedure affect the bidders' behaviors and therefore, the utility of the government. Using reinforcement learning, we seek to learn the optimal policies of involved actors in this game. In particular, we study how the requirements, contract types such as cost-plus and incentive-based contracts, number of bidders, problem complexity, etc., affect the acquisition procedure. Furthermore, we study the bidding strategy of the private firms and how the contract types affect their strategic behavior. Also, we study the effects of different contract types on the winner's optimal effort level necessary to meet the system requirements. We run exhaustive numerical simulations, which show that cost-plus contracts are particularly prone to strategic misrepresentation. This analysis can be expanded to help the government select procedures that achieve specific goals, such us minimizing cost overruns.