If machine learning models were to achieve superhuman abilities at various reasoning or
decision-making tasks, how would we go about evaluating such models, given that humans …

Pretrained language models can be queried for factual knowledge, with potential
applications in knowledge base acquisition and tasks that require inference. However, for …

While large pretrained language models (PLMs) demonstrate incredible fluency and
performance on many natural language tasks, recent work has shown that well-performing …

Semantic consistency of a language model is broadly defined as the model's ability to
produce semantically-equivalent outputs, given semantically-equivalent inputs. We address …

If a Large Language Model (LLM) answers" yes" to the question" Are mountains tall?" then
does it know what a mountain is? Can you rely on it responding correctly or incorrectly to …

English based datasets are commonly available from Kaggle, GitHub, or recently published
papers. Although benchmark tests with English datasets are sufficient to show off the …

Neural language models (NLMs) are susceptible to producing inconsistent output. This
paper proposes a new diagnosis as well as a novel remedy for NLMs' incoherence. We train …

In order to ensure autonomous vehicles are safe for on-road deployment, simulation-based
testing has become an integral complement to on-road testing. The rise in simulation testing …

Unmanned aerial vehicles (UAVs) have become popular, and accordingly, diagnosing
failure is essential but not easy due to its difficulty and complexity. Online UAV forums can …

Consistency is a fundamental dimension of trustworthiness in Large Language Models
(LLMs). For humans to be able to trust LLM-based applications, their outputs should be …