The information society is building on data and the software required to collect and analyse these data, which means that the trustworthiness of these data and software systems is crucially important for the development of society as a whole. Efforts to establish the trustworthiness of software typically include parameters, such as security, reliability, maintainability, correctness and robustness.In this paper we explore ways to determine the trustworthiness of software, in particular code where some of the constituent components are externally sourced, e.g. through crowd sourcing and open software systems. We examine different quality parameters that we believe define key quality indicators for trustworthy software and define CodeTrust, which is a content based trust metric for software.We present the design and evaluation of a research prototype that implements the proposed metric, and show the results of preliminary evaluations of CodeTrust using well known open source software projects.