In order to deliver real-time feedback to support employee development and rapid innovation, many companies are replacing formal, review-based performance management with systems that enable frequent and continuous employee evaluation. Real-time feedback applications enable supervisors and employees to give, seek, and receive competency-based feedback using their computers, smartphones, or other devices. In this study, we examine the role of one such real-time feedback application to understand its effects on employee performance appraisals. First, we find that relationship source (peer, subordinate, or supervisor) impacts real-time feedback: feedback tends to be more critical when it is from supervisors. What is more interesting is the effect of favoritism and retribution in real-time feedback: supervisors adopt tit-for-tat strategies, but peers do not. We also uncover that men rate women higher than men and that women rate men and women similar to how men rate men. In addition, positive real-time feedback has a stronger effect on future ratings than negative feedback. Our findings have direct implications for the design and implementation of performance management systems. Further, we highlight how companies can use information systems to create an innovative human resource operation that delivers flexibility and agility.