The addition of silicon carbide (SiC) micro powder into silicone rubber (SiR) matrix has been found to introduce a marked nonlinear conductivity into the resulting composites, without a degradation of the insulating properties under low electric field. Owing to these properties, the SiR-based nonlinear conductive composites have a potential use as the field grading material (FGM) layer in high voltage direct current (HVDC) cable accessories, which is employed to uniform electric field and suppress charge accumulation at the interface between crosslinked polyethylene (XLPE) and accessory insulation. In this paper, the space charge transportation and dissipation behaviors of SiC/SiR composites were investigated by pulsed electro-acoustic (PEA) test, in view of the effects of SiC filler size and crystal morphology. The results indicated that with the enhancement of nonlinear conductivity, the charge transportation and dissipation processes were accelerated. Compared with α-SiC particles, the β-SiC whiskers could provide a larger nonlinear conductivity. Meanwhile, the trap level distributions of SiC/SiR composites showed that the β-SiC whiskers introduced a shallower trap level at 0.81 eV than the α-SiC particles (0.85 eV). In addition, for the α-SiC particles filled composites, with the increase of the filler size from 0.45 μm to 5.0 μm, the shallow trap density increased, and the deep trap density decreased, resulting in an enhancement of space charge dissipation.