Not only software development, but also many companies developing physical products (not purely software) face high uncertainties and dynamics which raises the need for agile practices well-proven in software development. However, so-called constraints of physicality (e.g. duration to build potentially shippable increments) make it very difficult to become agile. Challenges associated with the constraints of physicality are highly interdependent and form an entangled, complex system. It is not obvious to find the root or pinpoint challenges with extraordinary high influence on others. Therefore, the investigation's goal is to identify most important challenges by separating between causes and effects. Knowing this can increase the effectiveness of research efforts in the realm of agile development. The investigation identifies 153 challenges and 160 interdependencies, and detects four backbones through the network that experts from industry rate highly influencing. Those point in particular to issues with product separation into increments, flexibility and scaling that represent very effective directions to overcome or reduce the impact of the constraints of physicality.