The boundary effect on the diffusiophoretic behavior of a particle is analyzed theoretically by considering the diffusiophoresis of a charged sphere under arbitrary surface potential and double-layer thickness at an arbitrary position in an uncharged spherical cavity. We show that the phenomenon under consideration is governed by double-layer relaxation, chemiosmotic/diffusioosmotic flow, and two types of competing double-layer polarization. The presence of the cavity has a profound influence on the diffusiophoretic behavior of the particle, especially when the surface potential is high. For instance, the scaled diffusiophoretic velocity of the particle has a local maximum as the position of the particle varies; it may have a local maximum and local minimum as the thickness of the double-layer varies. The significance of the effect of double-layer relaxation depends upon the level of surface potential and magnitude of the electric Peclet number.