The IP-based Internet of Things refers to the interconnection of smart objects in a Low-power and Lossy Network (LLN) with the Internet by means of protocols such as 6LoWPAN or CoAP. The provisioning of an end-to-end security connection is the key to ensure basic functionalities such as software updates or network access. This is, however, very challenging due to the asymmetry of the devices in the system (smart objects are resource constrained when compared with traditional Internet devices), and the interaction of possibly different security protocols such as TLS and DTLS. This paper describes the security and threat model for this scenario to arrive at the following conclusion: end-to-end security in the IP-based Internet of Things is more than a simple end-to-end handshake and requires additional measures to protect the LLN while performing the handshake. We further describe some simple solutions and give hints for further work.