Dysarthria, a speech disorder caused by neuro-motor problems resulting in impaired articulation, requires an assessment of its severity for diagnostic and monitoring purposes. Additionally, accurate severity classification facilitates the development of automated dysarthric speech detection and classification systems. This paper presents a comprehensive investigation into detecting dysarthric voices within a collection of normal voice samples, followed by the dysarthria severity classification utilizing neural network frameworks, specifically long short-term memory network (LSTM) and recurrent neural network (RNN). The study employs various features including Mel frequency cepstral coefficients (MFCC), formants, prosodic parameters, and voice quality. The performance of these models is evaluated against a baseline support vector machine (SVM) classifier using the Nemours corpus database. Remarkably, the highest classification accuracy achieved for this corpus is 99.69%. Detailed analysis demonstrates that selecting an appropriate neural network architecture yields superior performance compared to the conventional SVM classifier.