We introduce Brain-Inspired Modular Training (BIMT), a method for making neural networks
more modular and interpretable. Inspired by brains, BIMT embeds neurons in a geometric …

This work examines the presence of modularity in pre-trained Transformers, a feature
commonly found in human brains and thought to be vital for general intelligence. In analogy …

Incorporating modular designs into neural networks demonstrates superior out-of-
generalization, learning efficiency, etc. Existing modular neural networks are generally …

There are good arguments to support the claim that deep neural networks (DNNs) capture
better feature representations than the previous hand-crafted feature engineering, which …

Abstract Modular Neural Networks (MNNs) demonstrate various advantages over monolithic
models. Existing MNNs are generally explicit: their modular architectures are pre-defined …

An approach to improve network interpretability is via clusterability, ie, splitting a model into
disjoint clusters that can be studied independently. We find pretrained models to be highly …

Linear classifier probes are frequently utilized to better understand how neural networks
function. Researchers have approached the problem of determining unit importance in …

Natural target functions and tasks typically exhibit hierarchical modularity--they can be
broken down into simpler sub-functions that are organized in a hierarchy. Such sub …

Contributions to a Methodology for the Building of Modular Neural Networks by David
Castillo Bolado Modularity is a powerful concept that has been long leveraged by humanity …

Artificial Neural Networks (ANNs) are a type of machine learning algorithms that are used to
solve problems such as medical diagnosis. In recent times, the amount of data that is …