Desempenho de cotistas e não-cotistas em prova de vestibular: uma análise psicométrica

Abstract

Among the most important affirmative action policies of Brazilian society, there is the quota system for entering higher education. Thus, this mechanism needs to be constantly evaluated, in relation to its effectiveness in the issue of social justice and in terms of efficiency in the selection of candidates who enter universities. This is a cross-sectional study that sought to evaluate the performance of quota students and broad competition from high school in an evaluation process (vestibular) of a public university in the State of Alagoas. The data for the year 2015 of all appraisers was considered (N = 3807), 2858 of which were highly competitive and 949 quota holders. It was decided to perform the analyzes only with the candidates of the Bachelor of Medicine course because it has the largest population (N = 2722), being 535 (20%) quota holders and 2187 (80%) non-quota holders, and for presenting the lowest standard error in addition to being the course with the highest scores. The assessment instrument was the entrance exam according to the ENEM model. The R was used and through the mirt package, the three parameters were verified for each competence and item of the test, as well as the DIF (Differential Item Functioning) to check the difference in the probability of an item’s success between the quota holders. and non-quota holders. The three-parameter IRT Model (3LP) proved to be adequate for the analysis of the tests, as well as the Expected a Posteriori method for the proficiency of the candidates. The English test had the lowest average difficulty score of the items (b = -0.98) and the Mathematics test the highest (b = 1.25). The Natural Sciences test presented the best fit between the difficulty of the test and the proficiency of the candidates, while the Mathematics test presented the greatest discrepancy. Through the Mantel-Haenszel method for DIF, 23 items were identified that discriminate between quota holders and non-quota holders, with the highest concentration (10 items) in the Humanities test. The results demonstrate that there is DIF for some items and that this aspect needs to be considered for the composition of the performance score in the test. In addition, the items that are used in entrance exams require at least some previous system of classification and validation of the level of difficulty - such as the Angoff method. The present study highlights the relevance of the differential functioning of the items for understanding the performance of quota students and non-quota students in the entrance exam in the manner of ENEM.