The Domain Name System (DNS) is an essential infrastructure service on the internet. It provides a worldwide mapping between easily memorizable domain names and numerical IP addresses. Today, legitimate users and malicious applications use this service to locate content on the internet. Yet botnets increasingly rely on DNS to connect to their command and control servers. A widespread approach to detect bot infections inside corporate networks is to inspect DNS traffic using domain C&C blacklists. These are built using a wide range of techniques including passive DNS analysis, malware sandboxing and web content filtering. Using DNS to detect botnets is still an error-prone process; and current blacklist generation algorithms often add innocuous domains that lead to a large number of false positives during detection.This paper presents a new system called Mentor. It implements a scalable, positive DNS reputation system that automatically removes benign entries within a blacklist of botnet C&C domains. Mentor embeds a crawler system that collects statistical features about a suspect domain name, including both web content and DNS properties. It applies supervised learning to a labeled set of known benign and malicious domain names, using its features set in order to build a DNS pruning model. It further processes domain blacklists using this model in order to skim-off benign domains and keep only true malicious domains for detection. We tested our system against a wide set of public botnet blacklists. Experimental results prove the ability of this system to efficiently detect and remove benign domain names with a very low false positives rate.