Time-varying matrix reciprocal problems are widely appeared in different matrix computations and engineering fields. Neural networks as a powerful tool have been developed to solve the time-varying problems. Recurrent neural networks (RNNs) are designed considering mainly for two aspects: 1) convergent precision and 2) convergent time. The core part of the existed neural methods is to design various kinds of activation function for time-varying matrix solving. However, most of the activation functions of neural networks are with infinite value, which demands long convergent time and are not applicable in practical engineering fields. This note proposes theoretical analyses and simulation results on the performance of terminal RNN (TRNN) and accelerated TRNN (ATRNN) with finite-time convergence, which is not only designed for constant matrix inversions but also for time-varying reciprocal matrix. Compared to the traditional RNNs, TRNNs are of limit-valued activation function and possess a finite time convergence property. The simulation results for time-varying reciprocal solving validate the perfect performance solved by TRNN and ATRNN. In addition, a quadratic program (QP) of velocity minimization based on TRNN is proposed to solve the trajectory tracking problems without considering the initial position error of the redundant manipulators. Finally, practical experiments of the redundant manipulators based on PUMA560 show the effectiveness and accuracy of the proposed approaches.