Advance fee fraud is a significant component of online criminal activity. Fraudsters can often make off with significant sums, and victims will usually find themselves plagued by follow-up scams. Previous studies of how fraudsters persuade their victims have been limited to the initial solicitation emails sent to a broad population of email users. In this paper, we use the lens of scam-baiting -- a vigilante activity whereby members of the public intentionally waste the time of fraudsters -- to move beyond this first contact and examine the persuasive tactics employed by a fraudster once their victim has responded to a scam. We find linguistic patterns in scammer and baiter communications that suggest that the mode of persuasion used by scammers shifts over a conversation, and describe a corresponding stage model of scammer persuasion strategy. We design and evaluate a number of classifiers for identifying scam-baiting conversations amidst regular email, and for separating scammer from baiter messages based on their textual content, achieving high classification accuracy for both tasks. This forms a crucial basis for automated intervention, with a tool for identifying victims and a model for understanding how they are currently being exploited.