Back to AAIR Journal Volume 19, No. 1

An Investigation of the Relationship Between University Rankings and Graduate Starting Wages

Published in Volume 19, No. 1, 10 July 2014

Print friendly version

David Carroll
Graduate Careers Australia

Submitted to the Journal of Institutional Research, 14 November 2013, accepted for publication 14 March 2014.

Abstract

The rise of global university rankings has garnered much attention in recent years. Various ranking systems exist, but all are conceptually similar in that universities are evaluated and ranked on the basis of comparable indicators, with a focus on research performance. Although these rankings are widely criticised as over-simplistic and methodologically deficient, there may be advantages to attending a well-ranked university. According to a recent Backgrounder on university rankings published by the Group of Eight (Go8), being well ranked globally can have positive effects on graduate employment and earnings. In this article I investigate this proposition in the context of the Australian graduate labour market. Specifically, I use data from the Graduate Destination Survey to investigate whether bachelor degree graduates from top-ranked Australian universities earn higher starting wages, ceteris paribus, than graduates from other institutions. I do this using a two-step regression approach to control for the potential non-random selection of students into top-ranked universities.

The era of global university rankings began in the first half of the last decade, with the first appearance of several influential rankings, including Asiaweek magazine’s short-lived rankings of institutions in the Asia-Pacific region in 2000, followed by the Shanghai Jiao Tong rankings (later renamed Academic Ranking of World Universities) in 2003, and the Times Higher Education World University Rankings in 2004. In 2010, Quacquarelli Symonds, the previous data provider for the World University Rankings, ended its association with the Times and published its own rankings known as the QS World University Rankings. While each of these rankings differ in terms of methodology, each are similar in that universities are scored and ranked on the basis of comparable indicators, with a focus on research performance.

Global university rankings are widely criticised as having issues with construct validity, measurement precision and reproducibility (Ioannidis et al., 2007); yet rankings are increasingly gaining attention. In Australia, the publication of global rankings is now accompanied by numerous articles in the popular and trade press, with most applauding the performance of specific institutions on the world stage. As such, being well ranked may be of benefit to universities and the students who attend them. Indeed, a recent Backgrounder on global university rankings from the Group of Eight (Go8) puts this explicitly, noting that being placed among the top universities in the world can have positive effects on graduate employment and earnings, along with other benefits, both tangible and intangible (Go8, 2012). This article investigates whether graduates from globally ranked universities do indeed earn higher starting wages, all else equal.

This article falls within a large, primarily American body of literature that analyses the effect of university quality on graduate labour market outcomes. Quality is typically measured by national rankings (e.g., Brewer, Eide, & Ehrenberg, 1999); university-level measures, such as the student–staff ratio, share of teaching staff with a PhD and average entry score, which can either be combined into a composite measure (e.g., Long, 2008) or studied separately (e.g., Betts, Ferrall, & Finnie, 2007); or by university groupings, such as those in Australia (e.g., Birch, Li, & Miller, 2009). I have found no studies that use global university rankings as a quality measure.

Evidence on the relationship between university quality and labour market outcomes is somewhat mixed. In the American literature Brewer et al. (1999) reported a large premium for attending an elite private school, which was also reported by Thomas (2003). Brand and Halaby (2003) found that any premium associated with elite college attendance occurs shortly after graduation and wanes thereafter, while Thomas and Zhang (2005) found the opposite. Long (2008) reported that the quality wage premium was sensitive to estimation methodology, with ordinary least squares (OLS) estimates tending to be positive and significant and non-OLS estimates being insignificant. Dale and Krueger (2002) reported that university quality had non-significant or negative effects after controlling for confounding factors. In the non-American literature, Milla (2012) reported an association between wages and university reputation for Canadian graduates. Investigating university characteristics in their study of Canadian graduates, Betts et al. (2007) reported a positive association between wages and student–staff ratios, but only for males. In Europe, Holmlund (2009) reported only a weak link between university quality and earnings, with a positive effect only for those in full-time work and those in the top of the income distribution. In Australia, Birch et al. (2009) found that broad university categories, which should reflect university quality, have little to no influence on graduate wages, echoing the earlier findings of Miller and Volker (1983). It is likely that the mixed results in the literature might stem, at least in part, from differences in samples, methodologies and university quality measures.

A key empirical challenge in university quality studies is the non-random selection of students into universities of different quality levels. Students who enrol in top-ranked universities may possess different characteristics, both measured and unmeasured, to those students who enrol in lower-ranked universities. If these characteristics are related to wages, the estimates on the effect of attending a top-ranked university will be biased. Most studies assume that including an adequately rich set of explanatory variables will address non-random selection into universities. Heckman and Robb (1985) refer to this as selection on observables. Some studies, however, use selection models to account for selection on unobservables (e.g., Brewer et al., 1999).

In this article, I present empirical results on the association between attending a globally ranked university and graduate starting wages. My analysis is based on data from the 2012 Graduate Destination Survey (GDS), which collected labour market data on Australian graduates who completed their studies in 2011. Given the lack of appropriate individual background variables in the GDS data, I use a two-stage treatment-effects model to account for the possible non-random selection of students into universities. In brief, the results show that there is a small but significant wage premium of around 3% associated with attending a globally ranked university. Moreover, there is evidence that this result is not driven by non-random selection of students into universities. Field of study influenced wages to a greater extent, with 29 percentage points separating the highest- and lowest-earning fields. Wages were also influenced by age, sex and degree type.

The rest of this article is organised as follows. Section 1 describes the data and variables used in this study. Section 2 gives an overview of my empirical methodology. Results are presented and discussed in Section 3, while Section 4 concludes.

1. Data and variables

This study is based on data from the 2012 Graduate Destination Survey (GDS). Since 1972, graduates from Australian higher education institutions have participated in this representative survey of their labour market outcomes around four months after course completion. In 2012, 105,502 Australian resident graduates responded to the GDS, a response rate of 60.7%. Previous studies have established that the GDS data are reliable indicators of the full-time labour market position of the broader graduate population (Guthrie & Johnson, 1997). The cohort of interest is Australian resident bachelor degree graduates aged less than 25 who were employed full-time in Australia at the time of the survey. Focusing on young graduates reduces the likelihood that the results are affected by previous work experience, which is not captured in the GDS data. The dependent variable, log hourly wage, was calculated by dividing graduates’ annual salary by 52 weeks and then dividing by their weekly working hours, and taking the log of the result. Weekly working hours were top coded at 50 hours so that high earners who work long hours are not counted as low paid. Wages above the 99th percentage were removed, as were those below the Australian minimum hourly wage in 2012. This involved removing hourly wages below $15.51 or above $57.69. After exclusions, the final analysis sample consisted of 13,704 graduates.

The university ranking variable of interest in this study is based on the Academic Ranking of World Universities (ARWU). This ranking was chosen because it is well known and because its methodology is objective, stable and transparent. Ultimately, the choice of ranking is moot, since the set of globally ranked Australian universities is similar across rankings (Go8, 2012). To construct this variable, graduates from Australian universities ranked in the top 100 in the world in 2012 according to the ARWU are coded as 1 and 0 otherwise (see Table A1). All other variables are computed from the GDS data. Sample means for all of the variables used in the analysis are shown in Table 1, stratified by the university ranking variable. Focusing on the notable differences between categories, graduates from globally ranked universities are more likely to be from the natural and physical sciences, engineering and related technologies, and society and culture fields; to be completing an honours degree and to be from a capital city. They are less likely to be female, to be from the education and creative arts fields, and to have deferred some or all of their course fees. In all, 21% of the graduates in our sample attended a globally-ranked university.

Table 1: Sample Means

2. Empirical methodology

The task of this study was to investigate whether attending a globally ranked university is associated with higher starting wages, after controlling for individual graduates’ characteristics. As noted earlier, an empirical consideration is the possibility of non-random selection of students into universities of different quality levels. We may suspect that students with a higher ability level will be more likely to attend a globally ranked university and earn higher wages. If this is not addressed, the estimates on the effect of attending a globally ranked university will be biased. To account for this, I use a two-stage treatment-effects model (see Maddala, 1983). The first stage is a probit model to account for the decision to attend a globally ranked university and the second is a regression of log hourly wage on the university ranking indicator and a set of control variables that influence wages. A selection bias control factor (lambda) is computed from the first-stage estimates and added to the second-stage model as an additional control variable. Lambda is computed as ϕ(wi ŷ)/Φ(wi ŷ) for graduates from globally ranked universities and (wi ŷ)/{1-Φ(wi ŷ)} otherwise, where Φ(.) and ϕ(.) are the cumulative and density functions of the standard normal distribution, respectively, wi is a set of variables to explain the decision to attend a globally ranked university and ŷ is an estimated parameter vector. If the estimate on lambda is equal to zero, selection bias is not an issue and the wage equation could be estimated by OLS.

Estimation is more precise if exogenous variables that have no effect on wages are included in the probit and not in the wage model. I included three such variables. The first indicates residence in a capital city (rurban), which is assumed to influence the decision to attend a globally ranked university because all have their main campuses in capital cities. The second indicates part-time study (ptstud). It is assumed that students who elect to study part-time would be less likely to attend more “traditional” universities. The third indicates that a graduate deferred some or all of their course fees (deffee), which I included as a rough proxy for socioeconomic status. It is assumed that young students from a high socioeconomic background are more likely to pay their course fees upfront, and that these students are also more likely to attend a highly ranked university. Exploratory analysis confirmed that none of these were significantly linked to wages, net of other factors in the second-stage model.

3. Results

Table 2 reports the results from the estimation of the first- and second-stage models described in Section 2, along with baseline results from the OLS estimation of a wage model without the selection bias control factor. Considering first the probit estimates, every variable bar one field of study dummy (majord) significantly influenced the decision to attend a globally ranked university. In terms of the three omitted variables, all have the expected signs. Completion of an honours degree is a particularly strong predictor, possibly due to the research focus of the globally ranked universities in this study. The fields of natural and physical sciences; engineering and related technologies; agriculture, environmental and related studies; medicine and related studies; public health; and society and culture were all positively associated with attending a globally ranked university. The opposite was true for the remaining fields, although the estimate for architecture and building is not significant at a conventional level. Females had a lower probability of attending a globally ranked university, which was significant at the 10% level. The age term and its square indicate that there is a curvilinear relationship between age and attending a globally ranked university; however, it must be stressed that the age range in the analysis sample is quite limited (19–24 years).

Turning to the baseline OLS estimates in the first results column of Table 2, it is evident that attending a globally ranked university is associated with an hourly wage premium of 3%. This is marginally higher than the 2% Go8 wage premium reported by Birch et al. (2009) using a similar set of controls; however, their sample also included older graduates and part-time workers, and our globally ranked definition included only five of the Go8 universities. The effect is clearly modest, especially when compared with other wage determinants in Table 2. There was, for example, a 29 percentage point difference in wages across fields. Engineering and related technologies graduates earn 20% more, on average, than graduates from the omitted reference field of management and commerce, while creative arts graduates earn 9% less. Graduates from the fields of natural and physical sciences, information technology, public health and education are also high earners. An honours degree is associated with a wage premium of 6%. As expected, age is positively associated with wages. The squared age term is not significant, suggesting a linear association between age and wages in our sample of young graduates. Females earned 2% less, on average, than males. These results are broadly consistent with similar studies on this population (e.g., Graduate Careers Australia, 2010).

Table 2: Estimates from Baseline OLS and Treatment-Effects Models

Notes. Both models were highly significant (p < 0.0001). Standard errors are in parentheses. * = significant at 10%; ** = significant at 5%; *** = significant at 1%

Perhaps the most notable finding from the second-stage estimates in the third results column is that the selection bias control factor (lambda) is not significantly different from zero, which implies that we cannot reject the baseline OLS model. In other words, the OLS results do not appear to be driven by non-random selection of students into universities. This in itself is notable and echoes Brewer et al. (1999), who reported that correcting for selectivity had little impact on their estimates. The result on the globally ranked university variable in the second-stage model suggests that OLS could in fact understate this effect; however, this result should be interpreted with caution given that the lambda term is not significant. At any rate, the estimated effect in the second-stage model (5%) is of a similar magnitude to that in the baseline OLS model. As a further sensitivity analysis, I estimated these models again, comparing globally ranked universities with non-ranked universities only (listed in Table A1) and obtained similar results, both in terms of the wage effect and the significance of the lambda term.

4. Conclusions

This study investigated whether graduates from globally ranked Australian universities earn higher starting wages than graduates from lower-ranked or non-ranked universities, based on data from the 2012 GDS and an empirical approach that accounts for the potential non-random selection of students into universities. The results show a modest but significant wage effect associated with attending a globally ranked university, which does not appear to be driven by selection. Field of study had a much stronger effect on graduate starting wages, with a difference of 29 percentage points between the highest- and lowest-earning fields, which were engineering and related technologies, and creative arts, respectively. While these results support the proposition that graduates from globally ranked universities do enjoy higher wages, the effect is arguably minor. This is somewhat surprising given the press coverage that accompanies the release of new university rankings. One example is the article “Five Australian universities make global top 100” in the Financial Review (Dodd, 2013), which also described the ARWU as the world’s most influential university rankings. It could be that the Australian labour market does not care much about university rankings. In fact, the wage effect associated with globally ranked universities may not actually be driven by the rankings themselves, perhaps instead reflecting a small Go8 wage premium (see Birch et al., 2009) or other institutional characteristics. Disentangling the specific effect of rankings on starting wages is difficult given the data available, but represents an important avenue for further research. It could also be that young graduates from different universities are fairly homogeneous. The non-significant selection bias control factor in the second-stage model may be some evidence of this. Finally, it could be that the research-heavy rankings overstate the variation between universities in terms of the quality of coursework education provided, which may be more uniform across universities than the rankings suggest.

References

Betts, J., Ferrall, C., & Finnie, R. (2007). The role of university characteristics in determining post-graduation outcomes: Panel evidence from three recent Canadian cohorts (Cat. No. 11F0019MIE, No. 292). Ottawa, ON: Statistics Canada.

Birch, E. R., Li, I., & Miller, P. W. (2009). The influences of institution attended and field of study on graduates’ starting salaries. Australian Economic Review, 42(1), 42–63.

Brand, J. E., & Halaby, C. N. (2003). Regression and matching estimates of the effects of elite college attendance on career outcomes. Paper presented at the 2003 Annual Meeting of the American Sociological Association, Atlanta, GA. Retrieved from http://www.wjh.harvard.edu/~cwinship/cfa_papers/elite_jbrand_rev803.pdf

Brewer, D. J., Eide, E. R., & Ehrenberg, R. G. (1999). Does it pay to attend an elite private college? Cross-cohort evidence on the effects of college type on earnings. Journal of Human Resources, 34(1), 104–123.

Dale, S. B., & Krueger, A. B. (2002), Estimating the payoff to attending a more selective college: An application of selection on observables and unobservables. Quarterly Journal of Economics, 117(4), 1491–1527.

Dodd, T. (2013, August 15). Five Australian universities make global top 100. Financial Review. Retrieved from http://www.afr.com/p/national/ive_australian_universities_make_hRdYWjgGfL1Zrx2xwUOc9I

Graduate Careers Australia. (2010). Graduate Salaries 2009. Melbourne: Author.

Group of Eight. (2012). World university rankings: Ambiguous signals (Go8 Backgrounder 30). Canberra: Author.

Guthrie, B., & Johnson, T. J. (1997). Study of non-response to the 1996 Graduate Destination Survey. Canberra: Department of Employment, Education, Training and Youth Affairs.

Heckman, J., & Robb, R. (1985), Alternative methods for evaluating the impact of interventions. In J. Heckman & B. Singer (Eds.), Longitudinal analysis of labor market data (pp. 156–246). Cambridge: Cambridge University Press.

Holmlund, L. (2009). The effect of college quality on earnings: Evidence from Sweden (Umeå Economic Studies No. 781). Umeå, Sweden: Umeå University.

Ioannidis, J. P. A., Patsopoulos, N. A., Kavvoura, F. K., Tatsioni, A., Evangelou, E., … Liberopoulos, G. (2007). International ranking systems for universities and institutions: A critical appraisal. BMC Medicine, 5(30). doi:10.1186/1741–7015–5-30

Long, M. C. (2008). College quality and early adult outcomes. Economics of Education Review, 27(5), 588–602.

Maddala, G. S. (1983). Limited-dependent and qualitative variables in econometrics. New York: Cambridge University Press.

Milla, J. (2012). University quality and labor market outcomes of Canadian youth (Discussion Paper 2012–12). Guelph, ON: University of Guelph.

Miller, P. W., & Volker, P. (1983). Starting salaries and destinations of graduates in Australia (Conference Paper No. 14). Canberra: Bureau of Labour Market Research.

Shanghai Ranking Consultancy. (2013). Academic Ranking of World Universities 2012. Retrieved from http://www.shanghairanking.com/ARWU2012.html

Thomas, S. L. (2003). Longer-term economic effects of college selectivity and control. Research in Higher Education, 44(3), 263–299.

Thomas, S. L., & Zhang, L. (2005). Post-baccalaureate wage growth within four years of graduation: the effects of college quality and college major. Research in Higher Education, 46(4), 437–459.  

Table A1: Australian Universities on the Academic Ranking of World Universities, 2012

Notes. Adapted from Shanghai Ranking Consultancy (2013). An asterisk denotes a globally ranked university in this study. NR denotes a non-ranked university.

 

This article was first presented at the 23rd Annual AAIR Forum 'Insights from Institutional Research: Exploring New Shores', Perth, Western Australia, 13–15 November 2013.

Correspondence to:

David Carroll
Graduate Careers Australia
PO Box 13222
Law Courts, Melbourne Victoria

E: david.carroll@graduatecareers.edu.au

 The Journal of Institutional Research (JIR) was published between November 1991 and July 2014. The JIR was the publication of the Australasian Association for Institutional Research (AAIR), and remains freely available on the AAIR website. The JIR officially ceased publication in March 2016.


colorswathe   © Australasian Association for Institutional Research Inc. 2010 kindleman