It is worthwhile to incorporate human knowledge with conventional machine learning approaches for big data analytics. Focusing on big video data understanding, this paper presents a formal scenario recognition framework where knowledge-based logic representation and reasoning is combined with data-based learning approach to enhance scenario recognition capabilities. This is achieved via multi-layered (hierarchical) processing. This approach constructs the hierarchical representation structure based on the semantic understanding of considered scenario, and transforms the structure into logic formulas. After applying conventional computer vision methods for low-level events classification, we apply logic based uncertainty reasoning to determine scene content. Experimental results on a benchmark dataset are provided to show the rationality of the proposed approach.