This paper introduces a framework for analysis of cross-sectional dependence in the idiosyncratic volatilities of assets using high frequency data. We first consider the estimation of standard measures of dependence in the idiosyncratic volatilities such as covariances and correlations. Next, we study an idiosyncratic volatility factor model, in which we decompose the co-movements in idiosyncratic volatilities into two parts: those related to factors such as the market volatility, and the residual co-movements. When using high frequency data, naive estimators of all of the above measures are biased due to the estimation errors in idiosyncratic volatilities. We provide bias-corrected estimators and establish their asymptotic properties. We apply our estimators to high-frequency data on the 30 Dow Jones Industrial Average components, and document strong cross-sectional dependence in their idiosyncratic volatilities. We consider two different sets of idiosyncratic volatility factors, and we find that they cannot fully account for the cross-sectional dependence in idiosyncratic volatilities. We map the network of dependencies in residual idiosyncratic volatilities across the stocks.