Bismuth telluride‐based materials are already being commercially developed for thermoelectric (TE) cooling devices and power generators. However, the relatively low efficiency, which is characterized by a TE figure of merit, zT, is the main obstacle to more widespread application. Significant advances in the TE performance have been made through boundary engineering via embedding nanoinclusions or nanoscale grains. Herein, an effective approach to greatly enhance the TE performance of p‐type BiSbTe material by incorporating carbon microfibers is reported. A high zT of 1.4 at 375 K and high average zT of 1.25 for temperatures in the range of 300 to 500 K is achieved in the BiSbTe/carbon microfiber (BST/CF) composite materials. Their superior TE performance originates from the low thermal conductivity and the relatively high power factor. A TE unicouple device based on the p‐type BST/CF composite material and the commercially available n‐type bismuth telluride‐based material shows a huge cooling temperature drop in the operating temperature range of 299–375 K, and is greatly superior to the unicouple device made of both commercial p‐type and n‐type bismuth telluride‐based material. The materials demonstrate a high average zT and excellent mechanical properties and are strong candidates for practical applications.