We discuss two approaches for tackling algebraic systems, one is based on block preconditioning and the other is based on multifrontal and hierarchical matrix methods. First we consider a new preconditioner framework for supporting implicit time integration within an atmospheric climate model. We give an overview of the computational infrastructure used in atmospheric climate studies, address specific challenges of weak-scalability of numerical methods used in these codes, outline a strategy for addressing these challenges, and provide details about the software infrastructure being developed to implement these ideas. In the second part, we present our recent results of employing hierarchically semiseparable low-rank structure in a multifrontal factorization framework. This leads to superfast linear solvers for elliptic PDEs and effective preconditioners for a wider class of sparse linear systems.