Recent experiments revealed that the neocortical inhibitory neurons, which are believed to play a crucial role in controlling the cortical coherent activity, are coupled not only with the conventional synapses but also with gap junctions. In this letter, we study the synchronization properties in the network of Hodgkin-Huxley neurons coupled with both conventional chemical synapses and gap junctions, and explore the putative functional role of the gap junctions. In general, gap junction coupling is modeled as producing currents proportional to the differences in the membrane voltages between the two neurons. Usually, it is expected that the diffusive form of this coupling leads to synchronization of two neurons. However, as a result of phase oscillator analysis, we found that, in the two-neuron network, the gap junction stabilizes not only the in-phase state but also the antiphase one. Furthermore, in the population of inhibitory neurons, we numerically demonstrate that if the strength of the gap junctions is not too strong, the network can converge to a two-cluster stable state. Furthermore, it is also shown that the presence of gap junctions can facilitate the rapid convergence to a two-cluster state. Since any one of many possible two-cluster states can be selectively realized by choosing the appropriate initial condition, these results suggest that the gap junction in inhibitory neural networks may play a significant role in the neocortical activity for information processing.