This paper presents a new type of endocrine neural network (ENN). ENN utilizes artificial glands which enable the network to be adaptive to external disturbances. Sensitivity is controlled by the hormone decay rate and the value of the sensitivity parameter. The network presented in this paper is improved by making the sensitivity parameter self-tuning and implementing orthogonal activation functions inside the network structure. Automatic tuning is performed on the basis of the biological principle of postsynaptic potentials by implementing inhibitory and excitatory glands inside the standard backpropagation learning algorithm of developed orthogonal ENN. These additional network functionalities enable extra sensitivity to external conditions and an additional network feature of activation sharpening. The network was tested on real-time series of experimental data with a purpose to forecast exchange rate of the three widely used international currencies.