A Collaborative filtering (CF), one of the successful recommendation approaches, makes use of history of user preferences in order to make predictions. Common drawback found in most of the approaches available in the literature is that all users are treated equally. i.e., all users have same importance. But in the real scenario, there are users who rate items, which have similar rating pattern. On the other hand, some users provide diversified ratings. We assign relevance scores to users based on their rating pattern in order to improve the quality of predictions. To do so, we incorporate probability based user relevance scores into the similarity calculations. The improvement of predictions of benchmark item based CF approach with the inclusion of user relevance score is demonstrated in the paper.