Since the introduction of the Algebraic MultiGrid algorithm (AMG) over twenty years ago, significant progress has been made in improving the coarsening and the convergence behavior of the method. In this paper, an AMG method is introduced that utilizes a new generic approximate inverse algorithm as a smoother in conjunction with common coarsening techniques, such as classical Ruge-Stüben coarsening, CLJP and PMIS coarsening. The proposed approximate inverse scheme, namely Generic Approximate Banded Inverse (GenAbI), is a banded approximate inverse based on Incomplete LU factorization with zero fill–in (ILU (0)). The new class of Generic Approximate Banded Inverse can be computed for any sparsity pattern of the coefficient matrix, in an analogous way as the explicit approximate inverse, yielding a suitable smoother to be used in conjunction with an Algebraic Multigrid method. The proposed smoother is parameterized and thus by increasing the “retention” parameter the smoothing scheme becomes more effective in terms of required number of cycles for convergence. Finally, the applicability and effectiveness of the proposed AMG method along with implementation issues, based on the Generic Approximate Banded Inverse matrix, is demonstrated by solving two and three dimensional problems and numerical results on the convergence behavior and convergence factor are given.