We study two equivalent characterizations of the strong Feller property for a Markov process and of the associated sub-Markovian semigroup. One is described in terms of locally uniform absolute continuity, whereas the other uses local Orlicz-ultracontractivity. These criteria generalize many existing results on strong Feller continuity and seem to be more natural for Feller processes. By establishing the estimates of the first exit time from balls, we also investigate the continuity of harmonic functions for Feller processes which enjoy the strong Feller property.