Clustering of the diffuse infrared light from the COBE DIRBE maps .1. C(0) and limits on the near-infrared background

The cosmic infrared background (CIB) and its anisotropy have not yet been measured but are important signatures of the early evolution and clustering of galaxies. The near-IR is particularly interesting because redshift effects bring the peak luminosity of distant galaxies into the near-IR, allowing high-redshift objects to dominate the theoretical predictions of the CIB and its fluctuations. This paper is devoted to studying the CIB through its correlation properties. We studied the limits on CIB anisotropy in the near-IR (1.25, 2.2, and 3.5 mu m, or J, K, L) bands at a scale of 0.degrees 7 using the COBE(5) Diffuse Infrared Background Experiment (DIRBE) data. In single bands, we obtain the upper limits on the zero-lag correlation signal C(0)= [(v delta I-v)(2)] < 3.6 x 10(-16), 5.1 x 10(-17), and 5.7 x 10(-18) W-2 m(-4) sr(-2) for the J, K, and L bands, respectively. The DIRBE data exhibit a clear color between the various bands with a small dispersion. On the other hand, most of the CIB is expected to come from redshifted galaxies, and thus it should have different color properties. We use this observation to develop a ''color subtraction'' method of linear combinations of maps at two different bands. This method is expected to suppress the dominant fluctuations from foreground stars and nearby galaxies, while not reducing (or perhaps even amplifying) the extragalactic contribution to C(0). Applying this technique gives significantly lower and more isotropic limits. For the J-K, J-L, and K-L combinations, these limits are C(0)< 6.3 x 10(-17) 1.4 x 10(-16), and 1.2 x 10(-17) W-2 m(-4) sr(-2), respectively. We also use simple no-evolution models to interpret these numbers in terms of the total CIB levels, postponing a more detailed interpretation to a forthcoming paper. From the single-band fluctuations, we estimate upper limits on the CIB from clustered matter of (vI(v))(z,rms) =[integral(dvI(v)/dz)(2) dz](1/2) < 200, 78, and 26 nW m(-2) sr(-1) in the J, K, and L bands independently of the evolution history or spectral energy distribution. The color-subtracted signals constrain strongly the color evolution of galaxy populations and, if their degree of isotropy is indicative of a cosmological origin, they could allow determination of the total diffuse fluxes due to clustered material.