Evaluating large language models at evaluating instruction following

Z Zeng, J Yu, T Gao, Y Meng, T Goyal… - arXiv preprint arXiv …, 2023 - arxiv.org
As research in large language models (LLMs) continues to accelerate, LLM-based
evaluation has emerged as a scalable and cost-effective alternative to human evaluations …

Robot learning in the era of foundation models: A survey

X Xiao, J Liu, Z Wang, Y Zhou, Y Qi, Q Cheng… - arXiv preprint arXiv …, 2023 - arxiv.org
The proliferation of Large Language Models (LLMs) has s fueled a shift in robot learning
from automation towards general embodied Artificial Intelligence (AI). Adopting foundation …

Branch-solve-merge improves large language model evaluation and generation

S Saha, O Levy, A Celikyilmaz, M Bansal… - arXiv preprint arXiv …, 2023 - arxiv.org
Large Language Models (LLMs) are frequently used for multi-faceted language generation
and evaluation tasks that involve satisfying intricate user constraints or taking into account …

Can Large Language Models Understand Real-World Complex Instructions?

Q He, J Zeng, W Huang, L Chen, J Xiao, Q He… - Proceedings of the …, 2024 - ojs.aaai.org
Large language models (LLMs) can understand human instructions, showing their potential
for pragmatic applications beyond traditional NLP tasks. However, they still struggle with …

From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models

Q He, J Zeng, Q He, J Liang, Y Xiao - arXiv preprint arXiv:2404.15846, 2024 - arxiv.org
It is imperative for Large language models (LLMs) to follow instructions with elaborate
requirements (ie Complex Instructions Following). Yet, it remains under-explored how to …

Controllable Text Generation in the Instruction-Tuning Era

D Ashok, B Poczos - arXiv preprint arXiv:2405.01490, 2024 - arxiv.org
While most research on controllable text generation has focused on steering base
Language Models, the emerging instruction-tuning and prompting paradigm offers an …

Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles

R Louie, A Nandi, W Fang, C Chang, E Brunskill… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent works leverage LLMs to roleplay realistic social scenarios, aiding novices in
practicing their social skills. However, simulating sensitive interactions, such as in mental …

Intertwining CP and NLP: The Generation of Unreasonably Constrained Sentences

A Bonlarron, JC Régin - arXiv preprint arXiv:2406.15473, 2024 - arxiv.org
Constrained text generation remains a challenging task, particularly when dealing with hard
constraints. Traditional Natural Language Processing (NLP) approaches prioritize …

Open Sesame? Open Salami! Personalizing Vocabulary Assessment-Intervention for Children via Pervasive Profiling and Bespoke Storybook Generation

J Lee, S Yoon, K Lee, E Jeong, JE Cho… - Proceedings of the CHI …, 2024 - dl.acm.org
Children acquire language by interacting with their surroundings. Due to the different
language environments each child is exposed to, the words they encounter and need in …

CFBench: A Comprehensive Constraints-Following Benchmark for LLMs

T Zhang, Y Shen, W Luo, Y Zhang, H Liang… - arXiv preprint arXiv …, 2024 - arxiv.org
The adeptness of Large Language Models (LLMs) in comprehending and following natural
language instructions is critical for their deployment in sophisticated real-world applications …