Multi-layer transformers gradient can be approximated in almost linear time
The computational complexity of the self-attention mechanism in popular transformer
architectures poses significant challenges for training and inference, and becomes the …
architectures poses significant challenges for training and inference, and becomes the …
In-context learning may not elicit trustworthy reasoning: A-not-b errors in pretrained language models
Recent advancements in artificial intelligence have led to the creation of highly capable
large language models (LLMs) that can perform tasks in a human-like manner. However …
large language models (LLMs) that can perform tasks in a human-like manner. However …
Generating action-conditioned prompts for open-vocabulary video action recognition
Exploring open-vocabulary video action recognition is a promising venture, which aims to
recognize previously unseen actions within any arbitrary set of categories. Existing methods …
recognize previously unseen actions within any arbitrary set of categories. Existing methods …
Circuit Complexity Bounds for RoPE-based Transformer Architecture
Characterizing the express power of the Transformer architecture is critical to understanding
its capacity limits and scaling law. Recent works provide the circuit complexity bounds to …
its capacity limits and scaling law. Recent works provide the circuit complexity bounds to …
Towards Friendly AI: A Comprehensive Review and New Perspectives on Human-AI Alignment
As Artificial Intelligence (AI) continues to advance rapidly, Friendly AI (FAI) has been
proposed to advocate for more equitable and fair development of AI. Despite its importance …
proposed to advocate for more equitable and fair development of AI. Despite its importance …
MageBench: Bridging Large Multimodal Models to Agents
LMMs have shown impressive visual understanding capabilities, with the potential to be
applied in agents, which demand strong reasoning and planning abilities. Nevertheless …
applied in agents, which demand strong reasoning and planning abilities. Nevertheless …
VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Values
Large vision language models (VLMs) have demonstrated significant potential for
integration into daily life, making it crucial for them to incorporate human values when …
integration into daily life, making it crucial for them to incorporate human values when …
Beyond the Binary: Capturing Diverse Preferences With Reward Regularization
Large language models (LLMs) are increasingly deployed via public-facing interfaces to
interact with millions of users, each with diverse preferences. Despite this, preference tuning …
interact with millions of users, each with diverse preferences. Despite this, preference tuning …
CityBench: Evaluating the Capabilities of Large Language Model as World Model
Large language models (LLMs) with powerful generalization ability has been widely used in
many domains. A systematic and reliable evaluation of LLMs is a crucial step in their …
many domains. A systematic and reliable evaluation of LLMs is a crucial step in their …
The EPOCH of AI: Human-Machine Complementarities at Work
In this paper, we study the impact of AI and emerging technologies on the American labor
force by exploring AI's potential for substitution and complementarity with human workers …
force by exploring AI's potential for substitution and complementarity with human workers …