Human detection is an essential task in so many applications, especially surveillance systems. Recently, ConvNets (Convolutional Neural Networks)-based YOLO model is a successful method applied for object (including human) detection. It is one of the fastest way to detect directly objects from the input image. However, compared to the ConvNets-based state-of-the-art object detection methods, YOLO model-based object detection method achieved less accuracy. In this paper, we propose a new real-time human detection under fisheye cameras for surveillance purpose based on YOLO model. However, we improve the preciseness by using 2-D input channels consisting of grey-level image channel and foreground-background context information extracted by AGMM (Adaptive Gaussian Mixture Model) instead of original 3-D color input channels for ConvNets-based YOLO model. It is shown through experiments that the proposed method performs better with respect to accuracy and more robust to background scene changes without processing speed degradation compared to YOLO model-based human detection so that it can be successfully employed for embedded surveillance application.