In 2006, Geoffrey Hinton proposed the concept of training “Deep Neural Networks (DNNs)”
and an improved model training method to break the bottleneck of neural network …

This paper provides a comprehensive review of the literature concerning the utilization of
Natural Language Processing (NLP) techniques, with a particular focus on transformer …

Recent advances in Large Language Models (LLM) have made automatic code generation
possible for real-world programming tasks in general-purpose programming languages …

Pre-trained models for Natural Languages (NL) like BERT and GPT have been recently
shown to transfer well to Programming Languages (PL) and largely benefit a broad set of …

This paper provides a survey of the emerging area of Large Language Models (LLMs) for
Software Engineering (SE). It also sets out open research challenges for the application of …

Code summarization and generation empower conversion between programming language
(PL) and natural language (NL), while code translation avails the migration of legacy code …

Benchmark datasets have a significant impact on accelerating research in programming
language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster …

Human developers can produce code with cybersecurity bugs. Can emerging 'smart'code
completion tools help repair those bugs? In this work, we examine the use of large language …

Pre-trained models for programming language have achieved dramatic empirical
improvements on a variety of code-related tasks such as code search, code completion …

Automatic program repair (APR) is crucial to improve software reliability. Recently, neural
machine translation (NMT) techniques have been used to automatically fix software bugs …