Recent interest in design through the artificial intelligence (AI) lens is rapidly increasing.
Designers, as a special user group interacting with AI, have received more attention in the …

Mobile User Interface Summarization generates succinct language descriptions of mobile
screens for conveying important contents and functionalities of the screen, which can be …

Modeling user interfaces (UIs) from visual information allows systems to make inferences
about the functionality and semantics needed to support use cases in accessibility, app …

The field of generative artificial intelligence has seen significant advancements in recent
years with the advent of large language models, which have shown impressive results in …

Machine learning models have been trained to predict semantic information about user
interfaces (UIs) to make apps more accessible, easier to test, and to automate. Currently …

Rapid GUI prototyping has evolved into a widely applied technique in early stages of
software development to facilitate the clarification and refinement of requirements …

Inspiration from design examples plays a crucial role in the creative process of user interface
design. However, current tools and techniques that support inspiration usually only focus on …

All graphical user interfaces are comprised of one or more screens that may be shown to the
user depending on their interactions. Identifying different screens of an app and …

We propose the problem of conversational web navigation, where a digital agent controls a
web browser and follows user instructions to solve real-world tasks in a multi-turn dialogue …

In the current landscape of pervasive smartphones and tablets, apps frequently exist across
both platforms. Although apps share most graphic user interfaces (GUIs) and functionalities …