Measuring the academic reputation through citation networks via PageRank

Open access preprint Measuring the academic reputation through citation networks via PageRank, Massucci and Docampo, arXiv (2018).

Abstract:

The objective assessment of the prestige of an academic institution is a difficult and hotly debated task. In the last few years, different types of University Rankings have been proposed to quantify the excellence of different research institutions in the world. Albeit met with criticism in some cases, the relevance of university rankings is being increasingly acknowledged: indeed, rankings are having a major impact on the design of research policies, both at the institutional and governmental level.

Yet, the debate on what rankings are exactly measuring is enduring. Here, we address the issue by measuring a quantitive and reliable proxy of the academic reputation of a given institution and by evaluating its correlation with different university rankings. Specifically, we study citation patterns among universities in five different Web of Science Subject Categories and use the PageRank algorithm on the five resulting citation networks. The rationale behind our work is that scientific citations are driven by the reputation of the reference so that the PageRank algorithm is expected to yield a rank which reflects the reputation of an academic institution in a specific field.

Our results allow to quantifying the prestige of a set of institutions in a certain research field based only on hard bibliometric data. Given the volume of the data analysed, our findings are statistically robust and less prone to bias, at odds with ad–hoc surveys often employed by ranking bodies in order to attain similar results. Because our findings are found to correlate extremely well with the ARWU Subject rankings, the approach we propose in our paper may open the door to new, Academic Ranking methodologies that go beyond current methods by reconciling the qualitative evaluation of Academic Prestige with its quantitative measurements via publication impact.

cross-citations-engineering
The institutional network of cross-citations in the Telecommunication Engineering WoS category. Each node of the network is an academic institution featured both in the Telecommunications ARWU GRAS and as an affiliation in at least one publication of the Telecommunication Engineering WoS category. Edges are citations from a publication produced by an institution to those authored by another one (10% of the total edges are plotted). The node size is proportional to the number of publications.

Implementation of preference ranking organization method for enrichment evaluation on selection system of student’s achievement

Open access Implementation of preference ranking organization method for enrichment evaluation (Promethee) on selection system of student’s achievement, by Karlitasari, Suhartini, and Nurrosikawati, IOP Conference Series: Materials Science and Engineering (2018) Volume 332, conference 1.

Abstract:

Selection of Student Achievement is conducted every year, starting from the level of Study Program, Faculty, to University, which then rank one will be sent to Kopertis level. The criteria made for the selection are Academic and Rich Scientific, Organizational, Personality, and English. In order for the selection of Student Achievement is Objective, then in addition to the presence of the jury is expected to use methods that support the decision to be more optimal in determining the Student Achievement. One method used is the Promethee Method. Preference Ranking Organization Method for Enrichment Evaluation (Promethee) is a method of ranking in Multi Criteria Decision Making (MCDM). PROMETHEE has the advantage that there is a preference type against the criteria that can take into account alternatives with other alternatives on the same criteria. The conjecture of alternate dominance over a criterion used in PROMETHEE is the use of values in the relationships between alternative ranking values. Based on the calculation result, from 7 applicants between Manual and Promethee Matrices, rank 1, 2, and 3, did not change, only 4 to 7 positions were changed. However, after the sensitivity test, almost all criteria experience a high level of sensitivity. Although it does not affect the students who will be sent to the next level, but can bring psychological impact on prospective student’s achievement.

Predicting U.S. News & World Report ranking of regional universities in the South using public data

Predicting U.S. News & World Report ranking of regional universities in the South using public data, Ph.D. dissertation by Angela E. Henderson (2017)

USNWR-best-colleges
Process chart showing steps used by USNWR to calculate the 2016 Best Colleges institutional rankings.

Abstract:

Using correlational analyses and multiple regressions, this study uses U.S. News & World Report’s (USNWR) 2016 college rankings data and data from the National Center for Education Statistics’ (NCES) Integrated Postsecondary Education Data System (IPEDS) to examine variables that explain institutional peer assessment score and rank. This study focused on the 97 institutions included in USNWR’s 2016 Best Regional Universities (South) ranking list.

Analyses in this study addressed four major foci: 1) correlations between USNWR subfactor data values and selected IPEDS proxies, 2) IPEDS variables that explained variance in peer assessment score, 3) IPEDS variables that explained variance in rank, and 4) the extent to which rank could be predicted based on these results.

The results of this study indicated three main findings. First, USNWR subfactors with direct or indirect IPEDS proxies were highly correlated with the identified proxies. Second, more than 85% of variation in peer assessment score could be explained by five or fewer proxy variables, which differ dependent upon institution sector (private or public). Third, more than 85% of variation in institutional ranking could be explained by five proxy variables and without the inclusion of the peer assessment score subfactor. Collectively, findings suggest USNWR rankings are no more than a reflection of institutional outcomes and financial resources.

usnwr-predicted-ranks
Percentage of predicted ranks classified into same decile as actual ranking.

Universities’ Global Ranking Criteria Modification According to the Analysis of Their Websites

Universities’ Global Ranking Criteria Modification According to the Analysis of Their Websites, by Mohammed Al-Hagery, IJCSNS (2017) Vol. 17 No. 12 pp. 67-78

arwu_modified_indicators
A Modified Indicators and Weights for ARWU

Abstract:

Global universities are subject to the academic ranking every year. One of the common ranking types that are applied annually is called the Academic Ranking of World Universities (ARWU). It developed by a team of researchers and experts. The ARWU is composed of a set of common criteria related to academic tasks and it does not include any indication or factor relevant to the recent technology, such as the websites of universities. Actually, there is a lack to find out the relationship between universities’ global ranking and their website features. Therefore, this research aimed at updating the current ranking model by adding a new criterion reflexing the websites’ features related to its contents and structure. This research focuses on universities as two classes ranked and unranked. This process includes extract, analyze websites’ datasets, visualize the initial results, study the relationship and the significant differences between the two classes if found, and modify the ARWU by updating the criteria list & their weights. A special S/W tool applied to analyze websites and to extract the required data. This research contributes to modify and enhance the ARWU model to be more comprehensive than the current one. The involvement of universities’ websites in the ranking process will encourage universities to improve their websites to achieve a higher-ranking level amongst leading universities. Furthermore, it gives a good chance for all universities to participate in the global ranking competition, especially the universities that have excellent outcomes and perfect websites.

arwu_modified
Design of the ARWU Modified Model Components