Problem definition: Managers in digital ad agencies face the complex problem of setting prices for new ad campaigns, bidding for viewers on ad exchanges, and allocating them appropriately to campaigns. We have developed an analytical methodology for optimizing the agency's profits, with uncertainties in the arrival of potential viewers and new ad campaigns and in winning bids and obtaining viewer actions on ad exchanges. Methodology/results: The ad agency manages a series of contracts with advertisers who wish to display their ads to randomly arriving targeted viewers. Advertisers differ in their delay costs and willingness to pay for obtaining various actions by these viewers. We formulate the ad agency's sequential decision problem as a Markov decision process and develop exact solutions for both finite-horizon and steady-state cases. In our numerical analysis, we find that the finite-horizon dynamic programming (DP) solutions converge quickly to time-invariant policies, which are shown to be equivalent to the steady-state solutions. We also develop and test two heuristics that allow our solution methods to be scaled for large problems. Managerial implications: Several managerial implications follow from our paper, which is the first to address this multifaceted ad agency decision problem: (i) managers can exercise three key operational levers to maximize profits: campaign pricing, bidding strategy, and the queue capacity for campaign outcomes; (ii) the effectiveness of these levers varies in different situations, as illustrated by our numerical sensitivity analysis; (iii) optimal dynamic policies that depend on the queue length deliver significantly better profits than optimal static policies; (iv) appropriately designed heuristics can provide accurate solution methods that can be scaled to accommodate the large numbers of campaign outcomes frequently requested in practice; and (v) combining ad campaigns or merging ad agencies leads to higher total profits for the ad agency, lower prices for ad campaigns, and decreased delays in obtaining campaign outcomes.