About Q-values of Monte Carlo method

Profit Sharing method is one of the reinforcement learning methods. Profit Sharing can work well on the Partially Observable Markov Decision Processes(POMDPs). Because it is the typical non-bootstrap method, and it's Q-value is usually handled accumulative. Profit Sharing, however, does not work well on the probabilistic state transition. This paper we propose the novel learning method which can work well on the probabilistic state transition. It is similar to the Monte Carlo method. So we discuss about Q-values of our proposed method. In the environment with deterministic state transitions, we show the same performance both the conventional Profit Sharing and proposed method. And show the good performance of proposed method against the conventional Profit Sharing.