Across the United States, higher education institutions increasingly employ a holistic review of prospective students' application materials. In a holistic review process, admissions offices consider a student's personal and academic context when reviewing applications for admission. A key feature of a holistic review is a student's application essay, or personal statement. However, admissions offices rarely standardize their essay review process and very little research exists regarding whether student essays predict successful outcomes in college. This paper summarizes a quality improvement study conducted within the University of Vermont Admissions Office. It examines the extent to which non-cognitive student characteristics present in student admissions essays (e.g., grit, creativity, intrinsic motivation, leadership, community engagement, cultural fluency) are correlated with pre-admission factors and subsequent college outcomes. The study involved developing a new essay scoring rubric and evaluating the usefulness of this rubric by scoring 320 undergraduate admission essays. Findings suggest that the rubric is useful in identifying evidence of non-cognitive factors in student essays, but that overall scores do not strongly correlate with pre-admissions characteristics or first-fall college GPA. The study supports the practice of holistic review and provides insight into how admissions offices can begin to operationalize the review of essays and non-cognitive factors in their admissions processes.