Modeling user interfaces (UIs) from visual information allows systems to make inferences
about the functionality and semantics needed to support use cases in accessibility, app …

Mobile apps bring us many conveniences, such as online shopping and communication, but
some use malicious designs called dark patterns to trick users into doing things that are not …

Human-centered AI plays a vital role in ensuring that human capabilities and ideas are
tailored to meet efficiently the data requirements. The main idea is focusing on making …

In the current landscape of pervasive smartphones and tablets, apps frequently exist across
both platforms. Although apps share most graphic user interfaces (GUIs) and functionalities …

We present a new task and dataset, ScreenQA, for screen content understanding via
question answering. The existing screen datasets are focused either on structure and …

AI models that can recognize and understand the semantics of graphical user interfaces
(GUIs) enable a variety of use cases ranging from accessibility to automation. Recent efforts …

With the popularity of smartphones and tablets, users have become accustomed to using
different devices for different tasks, such as using their phones to play games and tablets to …