With the rapid developments in information and communication technologies, knowledge management activities are becoming significant in managing knowledge for achieving competitive advantage in the organizational environment. Knowledge management has become more and more recognized in today's business environment and facilitates decision-making. It cannot be overstated that knowledge management constitute a vital resource for organizations to flourish, however, it is not without challenges and hindrances. Interestingly, many organizations, including public government organizations are perforated with a plethora of challenges that makes it difficult to manage their knowledge. The manner in which knowledge is managed in organizations is still wanting. The management of knowledge in South African organizations has always been a cause for concern that needed an immediate intervention. Therefore, this study seeks to identify challenges related to the management of knowledge in the government departments of South Africa. This study is qualitative in nature and used content analysis from desk top review of literature to identify the challenges and suggested solutions. Among others, the findings indicates that the South African public sector does encounter challenges such as corruption, the difficulties of handling knowledge in different formats, departments are still rigid and bureaucratic in their processes and makes it difficult when continuing to evolve through the implementation of public sector reforms into knowledge-based work environments. Therefore, the study recommends the development of policy frameworks nor strategies aligned to the organizational goals.