We present a model that assesses the different elastic responses of a semiflexible network, which either (i) is constrained to deform in an affine way or (ii) is permitted to thermally fluctuate and deviate from affine response. The thermal, non-affine response of the network is achieved using a Metropolis Monte Carlo algorithm with dynamic step size. We find that non-affine deformations soften the network dramatically at low strains and make the eventual nonlinear strain stiffening far more pronounced. We show that the effect of these non-affine deformations are very sensitive to the degree variation in the lengths of filaments connecting cross-links. Where there is high variation, non-affine deformations allow internal stresses to relax, giving rise to a smaller range of tensile forces in filaments and a dramatic reduction of network stiffness. This highlights that non-affine deformations are crucial in small strain response of stiff polymer networks.