We develop a Bayesian procedure for analyzing stationary long-range dependent processes. Specifically, we consider the fractional exponential model (FEXP) to estimate the memory parameter of a stationary long-memory Gaussian time series. In particular, we propose a hierarchical Bayesian model and make it fully adaptive by imposing a prior distribution on the model order. Further, we describe a reversible jump Markov chain Monte Carlo algorithm for variable dimension estimation and show that, in our context, the algorithm provides a reasonable method of model selection (within each repetition of the chain). Therefore, through an application of Bayesian model averaging, we incorporate all possible models from the FEXP class (up to a given finite order). As a result we reduce the underestimation of uncertainty at the model-selection stage as well as achieve better estimates of the long memory parameter. Additionally, we establish Bayesian consistency of the memory parameter under mild conditions on the data process. Finally, through simulation and the analysis of two data sets, we demonstrate the effectiveness of our approach.