We present near-IR (J, H, Ks) photometry for 27 of the 28 candidate Herbig Ae/Be stars in the Small and Large Magellanic Clouds (SMC and LMC) identified via the EROS1 and EROS2 surveys as well as near-contemporaneous optical (Ha) spectroscopy for 21 of these 28 candidates. Our observations extend previous efforts to determine the evolutionary status of these objects. We compare the IR brightness and colors of a subset of our sample with archival ground-based IR data and find evidence of statistically significant photometric differences for EROS LMC HAeBe candidates (ELHC) 5, 7, 12, 18, and 21 in one or more filters. In all cases, these near-IR photometric variations exhibit a gray color as compared to earlier epoch data. The similar to 1 mag IR brightening and minimal change in the H alpha emission strength we observe in ELHC 7 is consistent with previous claims that it is a UX Ori type HAe/Be star, which is occasionally obscured by dust clouds. We also detect a similar to 1 mag IR brightening of ELHC 12, but find little evidence of a similar large-scale change in its Ha line strength, suggesting that its behavior could also be caused by a UX Ori-like event. The similar to 0.5 mag IR variability we observe for ELHC 21, which also exhibited little evidence of a change in its Ha emission strength, could conceivably be caused by a major recent enhancement in the density of the inner disk region of a classical Be star. We also report the first near-IR photometry for two EROS SMC HAeBe Candidates (ESHC) stars and the first H alpha spectroscopy for one ELHC and five ESHC stars. Although H alpha emission is detected in all of these new observations, they do not exhibit a strong near-IR excess. It is therefore possible that many of these objects may be classical Be stars rather than Herbig Ae/Be stars.