Representation learning is a critical ingredient for natural language processing systems.
Recent Transformer language models like BERT learn powerful textual representations, but
these models are targeted towards token-and sentence-level training objectives and do not
leverage information on inter-document relatedness, which limits their document-level
representation power. For applications on scientific documents, such as classification and
recommendation, the embeddings power strong performance on end tasks. We propose …