The aim of this paper is to describe an algorithm to recognize Assamese handwritten numerals using mathematical morphology. The digits are classified into two groups. One group contains digits which contains one or more blobs or/and stems in its structure. The other group does not contain any blobs. The number of blobs is determined with the help of morphological boundary finding method considering the property as hole. We also use the concept called `connected component' of morphology to recognize digits without blobs. Digits without blobs are extended to blobs by using connected component approach of morphology. Digits with blobs and stems need to recognize the number of stems. The present study shows that stems need not to be exactly vertical or horizontal to detect it. The proposed algorithm has been applied and tested for various handwritten digits from ISI Kolkata database. We also compare this algorithm for various printed Assamese digits. Experiment result shows that the average recognition rate that can be achieved by this algorithm is 80% for handwritten numerals and almost 100% for printed numerals. The result is obtained by using 50 handwritten samples for each digit and different printed Assamese digits.