Screen recognition: Creating accessibility metadata for mobile applications from pixels
Many accessibility features available on mobile platforms require applications (apps) to
provide complete and accurate metadata describing user interface (UI) components …
provide complete and accurate metadata describing user interface (UI) components …
Webui: A dataset for enhancing visual ui understanding with web semantics
Modeling user interfaces (UIs) from visual information allows systems to make inferences
about the functionality and semantics needed to support use cases in accessibility, app …
about the functionality and semantics needed to support use cases in accessibility, app …
Machine learning-based prototyping of graphical user interfaces for mobile apps
K Moran, C Bernal-Cárdenas, M Curcio… - IEEE Transactions …, 2018 - ieeexplore.ieee.org
It is common practice for developers of user-facing software to transform a mock-up of a
graphical user interface (GUI) into code. This process takes place both at an application's …
graphical user interface (GUI) into code. This process takes place both at an application's …
Reverse engineering mobile application user interfaces with remaui (t)
TA Nguyen, C Csallner - 2015 30th IEEE/ACM International …, 2015 - ieeexplore.ieee.org
When developing the user interface code of a mobile application, in practice a big gap exists
between the digital conceptual drawings of graphic artists and working user interface code …
between the digital conceptual drawings of graphic artists and working user interface code …
Screen parsing: Towards reverse engineering of ui models from screenshots
Automated understanding of user interfaces (UIs) from their pixels can improve accessibility,
enable task automation, and facilitate interface design without relying on developers to …
enable task automation, and facilitate interface design without relying on developers to …
A systematic literature review of applications of the physics of notations
D Van Der Linden, I Hadar - IEEE Transactions on Software …, 2018 - ieeexplore.ieee.org
INTRODUCTION: The Physics of Notations (PoN) is a theory for the design of cognitively
effective visual notations, emphasizing the need for design grounded in objective and …
effective visual notations, emphasizing the need for design grounded in objective and …
It takes two to tango: Combining visual and textual information for detecting duplicate video-based bug reports
N Cooper, C Bernal-Cárdenas… - 2021 IEEE/ACM …, 2021 - ieeexplore.ieee.org
When a bug manifests in a user-facing application, it is likely to be exposed through the
graphical user interface (GUI). Given the importance of visual information to the process of …
graphical user interface (GUI). Given the importance of visual information to the process of …
App store effects on software engineering practices
In this paper, we study the app store as a phenomenon from the developers' perspective to
investigate the extent to which app stores affect software engineering tasks. Through …
investigate the extent to which app stores affect software engineering tasks. Through …
ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations
Multimodal Vision-Language Models (VLMs) enable powerful applications from their fused
understanding of images and language, but many perform poorly on UI tasks due to the lack …
understanding of images and language, but many perform poorly on UI tasks due to the lack …
Who will leave the company?: a large-scale industry study of developer turnover by mining monthly work report
Software developer turnover has become a big challenge for information technology (IT)
companies. The departure of key software developers might cause big loss to an IT …
companies. The departure of key software developers might cause big loss to an IT …