Network function virtualization (NFV) is an emerging technology that is becoming increasingly important due to its many advantages. NFV transforms legacy hardware-based network infrastructure into software-based virtualized networks. This transformation increases the flexibility and scalability of networks, at the same time reducing the time for the creation of new networks. However, the attack surface of the network increases, which requires the definition of a clear map of where attacks may happen. ETSI standards precisely define many security aspects of this architecture, but these publications are very long and provide many details which are not of interest to software architects. We start by conducting threat analysis of some of the NFV use cases. The use cases serve as scenarios where the threats to the architecture can be enumerated. Representing threats as misuse cases that describe the modus operandi of attackers, we can find countermeasures to them in the form of security patterns, and we can build a security reference architecture (SRA). Until now, only imprecise models of NFV architectures existed; by making them more detailed and precise it is possible to handle not only security but also safety and reliability, although we do not explore those aspects. Because security is a global property that requires a holistic approach, we strongly believe that architectural models are fundamental to produce secure networks and allow us to build networks which are secure by design. The resulting SRA defines a roadmap to implement secure concrete architectures.