Generative artificial intelligence (AI) has emerged as a powerful technology with numerous
applications in various domains. There is a need to identify the requirements and evaluation …

This paper provides a comprehensive review of the literature concerning the utilization of
Natural Language Processing (NLP) techniques, with a particular focus on transformer …

The task of generating code from a natural language description, or NL2Code, is considered
a pressing and significant challenge in code intelligence. Thanks to the rapid development …

Code review is an essential activity for ensuring the quality and maintainability of software
projects. However, it is a time-consuming and often error-prone task that can significantly …

The availability of large-scale datasets, advanced architectures, and powerful computational
resources have led to effective code models that automate diverse software engineering …

As Large Language Models (LLMs) become popular, there emerged an important trend of
using multimodality to augment the LLMs' generation ability, which enables LLMs to better …

The automation of code review activities, a long-standing pursuit in software engineering,
has been primarily addressed by numerous domain-specific pre-trained models. Despite …

Pretrained language models have been shown to be effective in many software-related
generation tasks; however, they are not well-suited for editing tasks as they are not designed …

Automated program repair (APR) aims to fix software bugs without manual debugging efforts
and plays a crucial role in software development and maintenance. Template-based APR …

In this work we systematically review the recent advancements in code processing with
language models, covering 50+ models, 30+ evaluation tasks, 170+ datasets, and 700 …