Advances in Field-Programmable Gate Array (FPGA) technology in recent years have resulted in an expansion of its usage in a very wide spectrum of applications. Apart from serving the traditional prototyping purposes, FPGAs are currently regarded as an integral part of embedded systems used in many industries, including communication, medical, aerospace, automotive, and military. Moreover, the emerging trend of AI has found FPGAs to be at the technological forefront with their use as deep learning acceleration platforms. The demand for FPGAs has grown to the point that major companies (e.g. Amazon) are offering cloud-based access to FPGAs, known as FPGA-as-a-Service. In many applications, FPGAs handle sensitive data and/or host cryptographic algorithm implementations. These FPGAs are not always located in a tamper-resistant environment, which makes their security a major concern, especially in light of the ever-growing number of publications demonstrating effective attacks specifically tailored to exploit the physical traits of FPGA implementations. In this survey, we cover the subset of those attacks that involve tampering with the FPGA configuration bitstream. We start by discussing how the FPGA vendors attempt to protect their products and how malicious parties try to overcome this protection. We then proceed to present the different bitstream modification attacks that can be found in the literature organized according to their targets. Finally, we present various countermeasures that can be deployed, drawing on bibliographic references from works specifically focused on FPGA bitstream protection, as well as those initially proposed for different purposes or devices that can be adapted for bitstream protection.