Diabetic Retinopathy (DR) is an eye disorder in patients with diabetes. Detection of DR presence and its complications using fundus images at an early stage helps prevent its progression to the advanced levels. In the recent years, several well-designed Convolutional Neural Networks (CNN) have been proposed to detect the presence of DR with the help of publicly available datasets. However, these existing CNN-based classifiers focus on utilizing different architectural settings to improve the performance of detection task only i.e. presence or absence of DR. The further classification of the severity and type of the disease, however, remains a non-trivial task. To this end, we propose a multi-stream ensemble deep network to classify diabetic retinopathy severity. The proposed approach takes advantages of the deep networks and principal component analysis (PCA) to learn inter-class and intra-class variations from the raw image features. Ensemble machine learning classifiers are then applied to achieve high classification accuracy and robust performance on the obtained deep features. Specifically, a multi-stream network is made using pre-trained deep learning architectures i.e. ResNet-50 and DenseNet-121 to serve as the main feature extractors. Further application of PCA reduces the dimensionality of features and effectively separates the variation space of inter-class and intra-class images. Finally, an ensemble machine learning classifier using AdaBoost and random forest algorithms is built to further improve classification accuracy. The proposed approach has been compared with multiple conventional CNN-based approaches on Messidor-2 (two categories) and EyePACS (two, five categories) datasets. The experiment results show that our proposed approach achieves superior performance (upto 95.58% accuracy) and can be considered a promising method for automatic diabetic retinopathy detection.