• 
  • Releases
  • Speeches
  • Articles
  • Events
  • Subscribe
  • Learning Link
  • Multimedia Centre
  • Practically Speaking
  • Profiles in Learning
• 
  • Upcoming Events
  • Past Events
  • Minerva Lecture Series
• 
  • Requests for Proposals
  • Funded Research
  • Reviewing Proposals
  • Building Capacity for Research
  • Researcher in Residence
  • Research Events, Presentations & Workshops
  • Student Success
• 
  • Composite Learning Index
  • Health Literacy
  • Post-secondary Education
  • Reading the Future
  • State of Aboriginal Learning in Canada
  • State of E-Learning
  • State of Learning in Canada
  • Redefining success in Aboriginal learning
  • Commissioned Reports
  • Question Scans
  • State-of-the-field Reviews
  • Systematic Reviews
  • Survey of Canadian Attitudes toward Learning
  • Literacy Assessment
  • Lessons in Learning
  • Journal of Applied Research on Learning
  • Data Warehouse
• 
  • What We Do
  • President and CEO
  • Board of Directors
  • Knowledge Exchange
  • 21st Century Learning Initiative (Canada)
  • Knowledge Centres
  • Crosscutting Themes
  • Joint Initiatives
  • Corporate Reports
  • Working at CCL
  • Contact Us

• Composite Learning Index
• Health Literacy
• Post-secondary Education
• Reading the Future
• State of Aboriginal Learning in Canada
• State of E-Learning
• State of Learning in Canada
• Redefining success in Aboriginal learning
• Commissioned Reports
• Question Scans
• State-of-the-field Reviews
• Systematic Reviews
• Review 1: Criteria used to effectively measure research and innovation in PSE environments
• Review 2: Measuring quality in post-secondary education
• Review 3: Accurately forecasting labour market needs
• Review 4: Factors influencing post-secondary enrolment increases and decreases
• Review 5: Policies & practices supporting the Educational achievement and social integration of first and second generation migrants
• Review 6: The Social Consequences of Economic Inequality for Canadian Children: A Review of the Canadian Literature
• Review 7: Adult Literacy: A Synthesis of Evidence
• Review 8: Examining the impact of homework on academic achievement
• Survey of Canadian Attitudes toward Learning
• Literacy Assessment
• Lessons in Learning
• Journal of Applied Research on Learning
• Data Warehouse

Systematic Reviews

What criteria might be used to effectively measure research and innovation in post-secondary environments?

Executive Summary

This paper systematically reviews and evaluates studies of research and innovation (R&I) to determine criteria that might be used to measure R&I in post-secondary environments. This review applies thorough and transparent procedures to the collection and analysis of evidence from research-oriented articles obtained from a variety of academic and fugitive sources disseminated between 1985 and the present.

The field of R&I is a complex one that presents many challenges for those who try to study and understand it. For example, the field lacks clear definitions of “research” and “innovation” and relies heavily on proxies or indicators. To address these challenges, the studies captured by this review are divided into three categories, based on their treatment of innovation indicators. The three categories are descriptive, analytical/critical and empirical. The focus of this review is on the empirical category because the articles included in this category tested the validity and reliability of the innovation indicators using statistical methods. Research trends are identified, based on frequency analysis of indicators. The methodological quality is assessed by an econometrician.

Based on this analysis, conclusions may be drawn regarding the study of R&I, including, but not limited to, the following:

Scientific production is often confused or combined with innovation. This leads to an underlying assumption that if a study measures scientific production, it will measure innovation. As a result, the majority of studies measure innovative production and fail to measure innovative processes (Salazar and Holbrook 2004).

Collaboration seems to be positively associated with innovation (Belderbos, Carree and Lokshin 2004; Gulbrandsen and Smeby 2005; Medda, Piga and Siegel 2005).

• R&I seems to be positively associated with labour productivity and research production (Belderbos, Carree and Lokshin 2004; Gulbrandsen and Smeby 2005; Medda, Piga and Siegel 2005).
• The majority of studies measured innovation using three indicators: research and development (R&D), patents and publications. In fact, 60% of the studies failed to examine indicators other than those included in the research and commercialization category. This result suggests that other aspects of innovation are being under-utilized.
• One aspect of innovation that has received particularly little attention is the innovation environment. This category includes indicators such as tax revenue and internet connectivity. Including this category in the study of innovation may have an effect on the results.
• Eight out of 17 studies were determined to be of good methodological quality, and the remaining nine studies were determined to be of fair quality. Not a single study received a poor rating. As a result, it appears that the overall methodological quality of studies examining R&I is reasonably good.

View the full report  (580 KB) 

 

Top