Machine Learning (ML) is often used invisibly in everyday applications with little opportunity for consumers to investigate how it works. In this paper, we expand recent efforts to unfold what students should know about ML and how to design tools and activities allowing them to engage with ML. To do so, we explore how to make processes and aspects of ML tangible through the design of the Machine Learning Machine (MLM); a tangible user interface which enables students to create their own data-sets using pen and paper and to iteratively build and test ML models using this data. Based on insights from the design process and a preliminary pilot study with the MLM, we discuss how a tangible approach to engaging with ML can spur curiosity in students and how the iterative process of improving ML models can encourage students to reflect on the relation between data, model and predictions.