Razor-thin dust layers in protoplanetary disks: Limits on the vertical shear instability

Context. Recent observations with the Atacama Large Millimeter Array (ALMA) have shown that the large dust aggregates observed at millimeter wavelengths settle to the midplane into a remarkably thin layer. This sets strong limits on the strength of the turbulence and other gas motions in these disks.Aims. We intend to find out if the geometric thinness of these layers is evidence against the vertical shear instability (VSI) operating in these disks. We aim to verify if a dust layer consisting of large enough dust aggregates could remain geometrically thin enough to be consistent with the latest observations of these dust layers, even if the disk is unstable to the VSI. If this is falsified, then the observed flatness of these dust layers proves that these disks are stable against the VSI, even out to the large radii at which these dust layers are observed.Methods. We performed hydrodynamic simulations of a protoplanetary disk with a locally isothermal equation of state, and let the VSI fully develop. We sprinkled dust particles with a given grain size at random positions near the midplane and followed their motion as they got stirred up by the VSI, assuming no feedback onto the gas. We repeated the experiment for different grain sizes and determined for which grain size the layer becomes thin enough to be consistent with ALMA observations. We then verified if, with these grain sizes, it is still possible (given the constraints of dust opacity and gravitational stability) to generate a moderately optically thick layer at millimeter wavelengths, as observations appear to indicate.Results. We found that even very large dust aggregates with Stokes numbers close to unity get stirred up to relatively large heights above the midplane by the VSI, which is in conflict with the observed geometric thinness. For grains so large that the Stokes number exceeds unity, the layer can be made to remain thin, but we show that it is hard to make dust layers optically thick at ALMA wavelengths (e.g., tau(1.3mm) greater than or similar to 1) with such large dust aggregates.Conclusions. We conclude that protoplanetary disks with geometrically thin midplane dust layers cannot be VSI unstable, at least not down to the disk midplane. Explanations for the inhibition of the VSI out to several hundreds of au include a high dust-to-gas ratio of the midplane layer, a modest background turbulence, and/or a reduced dust-to-gas ratio of the small dust grains that are responsible for the radiative cooling of the disk. A reduction of small grains by a factor of between 10 and 100 is sufficient to quench the VSI. Such a reduction is plausible in dust growth models, and still consistent with observations at optical and infrared wavelengths.