The observation of Cosmic Microwave Background (CMB) has been one of the cornerstones in establishing the current understanding of the Universe. This valuable source of information consists of primary and secondary effects. While the primary source of information in CMB (as a Gaussian random field) can be efficiently analyzed using established statistical methods, CMB is also host to secondary sources of information that are more complex to analyze and understand. Here, we report encouraging preliminary results as well as some difficulties in using deep learning for prediction of the cosmological parameters and uncertainty estimates from the primary CMB. This opens the way to application of deep models in analysis of the secondary CMB and joint analysis of CMB with other modalities such as the large-scale structure