Understanding design collaboration between designers and artificial intelligence: a systematic literature review
Recent interest in design through the artificial intelligence (AI) lens is rapidly increasing.
Designers, as a special user group interacting with AI, have received more attention in the …
Designers, as a special user group interacting with AI, have received more attention in the …
Never-ending learning of user interfaces
Machine learning models have been trained to predict semantic information about user
interfaces (UIs) to make apps more accessible, easier to test, and to automate. Currently …
interfaces (UIs) to make apps more accessible, easier to test, and to automate. Currently …
ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations
Multimodal Vision-Language Models (VLMs) enable powerful applications from their fused
understanding of images and language, but many perform poorly on UI tasks due to the lack …
understanding of images and language, but many perform poorly on UI tasks due to the lack …
BrushLens: hardware interaction proxies for accessible touchscreen interface actuation
Touchscreen devices, designed with an assumed range of user abilities and interaction
patterns, often present challenges for individuals with diverse abilities to operate …
patterns, often present challenges for individuals with diverse abilities to operate …
AI Assistance for UX: A Literature Review Through Human-Centered AI
Recent advancements in HCI and AI research attempt to support user experience (UX)
practitioners with AI-enabled tools. Despite the potential of emerging models and new …
practitioners with AI-enabled tools. Despite the potential of emerging models and new …
Generating Automatic Feedback on UI Mockups with Large Language Models
Feedback on user interface (UI) mockups is crucial in design. However, human feedback is
not always readily available. We explore the potential of using large language models for …
not always readily available. We explore the potential of using large language models for …
Interactive Flexible Style Transfer for Vector Graphics
J Warner, KW Kim, B Hartmann - Proceedings of the 36th Annual ACM …, 2023 - dl.acm.org
Vector graphics are an industry-standard way to represent and share visual designs.
Designers frequently source and incorporate styles from existing designs into their work …
Designers frequently source and incorporate styles from existing designs into their work …
Learning from unique perspectives: User-aware saliency modeling
S Chen, N Valliappan, S Shen, X Ye… - Proceedings of the …, 2023 - openaccess.thecvf.com
Everyone is unique. Given the same visual stimuli, people's attention is driven by both
salient visual cues and their own inherent preferences. Knowledge of visual preferences not …
salient visual cues and their own inherent preferences. Knowledge of visual preferences not …
UniAR: Unifying Human Attention and Response Prediction on Visual Content
Progress in human behavior modeling involves understanding both implicit, early-stage
perceptual behavior such as human attention and explicit, later-stage behavior such as …
perceptual behavior such as human attention and explicit, later-stage behavior such as …
Unblind Text Inputs: Predicting Hint-text of Text Input in Mobile Apps via LLM
Mobile apps have become indispensable for accessing and participating in various
environments, especially for low-vision users. Users with visual impairments can use screen …
environments, especially for low-vision users. Users with visual impairments can use screen …