In recent years, breast cancer is causing the death of many women. The major reasons for breast cancer are an increase in age, dense breast tissue, alcoholism, body weight, and radiation exposure. If it is detected at an early stage, then the chances of survival increase. Machine Learning and deep learning can help us to detect breast cancer at initial stage. In this paper, we propose a novel deep neural network CanNet to detect cancer with high accuracy. CanNet has accuracy of 99% with 98.8% specmcity and 99.1% sensitivity. We also compared our experimental results with other state-of-the-art methods. The comparison shows that CanNet is more efficient than all other methods.