Optimizing Automated Negotiation: Integrating Opponent Modeling with Reinforcement Learning for Strategy Enhancement

Agent-based automated negotiation aims to enhance decision-making processes by predefining negotiation rules, strategies, and objectives to achieve mutually acceptable agreements. However, most existing research primarily focuses on modeling the formal negotiation phase, while neglecting the critical role of opponent analysis during the pre-negotiation stage. Additionally, the impact of opponent selection and classification on strategy formulation is often overlooked. To address these gaps, we propose a novel automated negotiation framework that enables the agent to use reinforcement learning, enhanced by opponent modeling, for strategy optimization during the negotiation stage. Firstly, we analyze the node and network topology characteristics within an agent-based relational network to uncover the potential strength and types of relationships between negotiating parties. Then, these analysis results are used to inform strategy adjustments through reinforcement learning, where different negotiation strategies are selected based on the opponent's profile. Specifically, agents' expectations are adjusted according to relationship strength, ensuring that the expectations of negotiating parties are accurately represented across varying levels of relationship strength. Meanwhile, the relationship classification results are used to adjust the discount factor within a Q-learning negotiation algorithm. Finally, we conducted a series of experiments, and comparative analysis demonstrates that our proposed model outperforms existing negotiation frameworks in terms of negotiation efficiency, utility, and fairness.