Bibliometri i finansiering og evaluering av forskning

Transkript

1 2,0 % 1,8 % 1,6 % Sverige 1,4 % 1,2 % Danmark 1,0 % 0,8 % 0,6 % Finland 0,4 % 0,2 % Norge 0,0 % Gunnar Sivertsen: Bibliometri i finansiering og evaluering av forskning

2 Oversikt 1. Hvordan finansieres og evalueres forskning? 2. Hvordan publiseres forskning? 3. Den norske publiseringsindikatoren 4. Evalueringen av indikatoren i Videreutvikling av indikatoren

3 Oversikt 1. Hvordan finansieres og evalueres forskning? Tokanal-systemet Gjennom Forskningsrådet Evaluering av samfunnsrelevans. Altmetrics Direkte finansering av institusjoner Evalueringsbasert og indikatorbasert finansiering Evaluering av samfunnsrelevans: Publiseringsindikatoren

4 The dual funding system (Public sources) Government Research Councils Projects and Programmes Competition Research Evaluation Direct grants Historical, Political Strategic Performance based Research institutions

5

6

7 The REF 2014 (UK) inspired the methodology in a Norwegian evaluation of the humanities in : 169 impact cases Research Excellence Framework (UK, 2014) SIAMPI (Netherlands, ) IMPACT-EV (European Commission, ) STAR METRICS (US National Science Foundation, ) ERC Impact Framework (European Research Council)

8 Oversikt 1. Hvordan finansieres og evalueres forskning? Tokanal-systemet Gjennom Forskningsrådet Evaluering av samfunnsrelevans. Altmetrics Direkte finansering av institusjoner Evalueringsbasert og indikatorbasert finansiering Evaluering av samfunnsrelevans: Publiseringsindikatoren

9 Frameworks for the understanding and evaluation of the societal impact of research in the humanities Gunnar Sivertsen Nordic Institute for Studies in Innovation, Research and Education, Oslo, Norway

10 Outline A typology of societal impact in the humanities Limitations of the impact case methodology Understanding and evaluating normal impact

11 Method (type of taxonomy) A classification of each of the 169 cases with regard to the societal domain or sector with which there was an interaction

12 Cultural sector: The literary institution

13 Cultural sector: Performances

14 Educational sector: Research-based teaching aids

15 Media, software and internet: Grammar control (Norwegian language) for Microsoft Office

16 Public space: Ethical considerations in societal dilemmas and in working life

17 Foreign relations: Understanding other languages and cultures

18 Tourism: Knowing cultural history

19 The memory of society

20 Typical areas of societal impact of the humanities The cultural sector The educational sector Media, software and the internet Public space Foreign relations Tourism The memory of society

21 Outline A typology of societal impact in the humanities Limitations of the impact case methodology Understanding and evaluating normal impact

22 The impact evaluation methodology of the REF Impact case template as Word file (6,975 submissions)

23 REF definition impact An effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia Underpinning research must be excellent (2* quality or above)

24 The linear models of innovation and communication Asking for narratives about: The research that underpinned the impact: This section should outline the key research insights or findings that underpinned the impact, and provide details of what research was undertaken, when, and by whom, The resulting impact: A clear explanation of the process or means through which the research led to, underpinned or made a contribution to the impact (for example, how it was disseminated, how it came to influence users or beneficiaries, or how it came to be exploited, taken up or applied).

25 Challenges with the linear model for impact evaluation Ilkka Turunen (2013), based on OECD (2008) Causality: the relationships between research and innovation inputs, activities, outputs, and impacts are often unclear or nonlinear. Attribution: it is difficult or even impossible to separate the impact of research and innovation from other inputs and activities. Internationality: the impacts of research and innovation are international by nature activities and value chains are global, and both positive and negative spillovers exist. Time scale: the impacts are realized both in the short and long-term. Breakdown of impacts to a particular socio-economic target: how to define e.g. impact on public welfare or culture. Data issues: data on many issues related to science are unobservable or are not systematically collected; fundamental uncertainty around the concepts.

26 Figure 6 : Events Leading to the Development of the Oral Contraceptive The research leading to the contraceptual pill In scientific journals in unconnected fields 1920 Physiology of Reproduction Hormone Research Steroid Chemistry Non-Mission Research Mission-Oriented research Development & Application The Oral Contraceptive

27 Figure 6 : Events Leading to the Development of the Oral Contraceptive The research leading to the contraceptual pill In scientific journals in unconnected fields 1920 Physiology of Reproduction Hormone Research Steroid Chemistry 1930 Also in applied journals and in mission oriented research programmes 1940 Non-Mission Research Mission-Oriented research Development & Application The Oral Contraceptive

28 Figure 6 : Events Leading to the Development of the Oral Contraceptive The research leading to the contraceptual pill In scientific journals in unconnected fields 1920 Physiology of Reproduction Hormone Research Steroid Chemistry 1930 Also in applied journals and in mission oriented research programmes 1940 Non-Mission Research Mission-Oriented research Development & Application 1950 Fields are united, implementation 1960 The Oral Contraceptive

29 Figure 6 : Events Leading to the Development of the Oral Contraceptive Research on the social and cultural effects of the contraceptual pill 1920 Physiology of Reproduction Hormone Research Steroid Chemistry Non-Mission Research Mission-Oriented research Development & Application Cultural and social impact 1950 Research into cultural and social effects 1960 The Oral Contraceptive

30 Presenting the results of the Norwegian research evaluation exercise: Normal contributions are not part of the story Contributes to information technology, bioethics, peace processes Contributes to emergency communication, genetic counselling, ethical regulations Contributes to health care, engineering sciences, climatic research and energy

31 Extraordinary versus normal impact of the humanities

32 Normal impact Is found in responsible relations between academia and other institutions of civilization Is about daily activities and how well they are organized, not about individual incidents of visible impact A medical faculty is part of the health care system, not only of the university

33 Extraordinary impact can also be a violation of normal impact Research misconduct also has societal impact Can be understood as an institutional responsibility in the perspective of normal impact Institutional level evaluation and learning is needed Normal impact is what needs to be secured

34 Normal impact Syrian-Norwegian collaboration on documenting Palmyra (UNESCO world heritage) since 2008 Jørgen Christian Meyer, professor of archaeology, University of Bergen Khaled Mohamad al-asaad, director of Palmyra archaeological site

35 Extraordinary impact The sudden importance of the project after the outbreak of the civil war in 2011 Returned Beheaded by ISIS

36 Understanding and evaluating societal impact Societal impact of research is normal and part of society Normal impact is about daily activities and how well they are organized, not about individual incidents of particularly interesting or impressive impact Normal impact is almost impossible to measure. Interesting evidence can be provided, but this is costly. Societal impact primarily needs to be understood in order to be evaluated. Normal interactions with society can be evaluated as such at the organizational level, taking all inter-actors into consideration

37 Measurement? Measurement: Direct measurement not possible or of limited value (Macroeconomic modelling is possible) Evidence: Possible to provide, will change the culture of research organizations and have a mission in the media The problems with attributing societal impact to its origin in research are still unsolved Understanding: What we cannot measure, can still be understood and valued

38 How do the humanities and social sciences speak to society?

39

40 How do the sciences «speak» to society?

41 How do the sciences «speak» to society?

42 Societal impact? Or just social activity within academia?

43 Oversikt 1. Hvordan finansieres og evalueres forskning? Tokanal-systemet Gjennom Forskningsrådet Evaluering av samfunnsrelevans. Altmetrics Direkte finansering av institusjoner Evalueringsbasert og indikatorbasert finansiering Evaluering av samfunnsrelevans: Publiseringsindikatoren

44 The dual funding system (Public sources) Government Research Councils Projects and Programmes Competition Research Evaluation Direct grants Historical, Political Strategic Performance based Research institutions

45 PRFS before 2003 Red: Evaluation-based, Blue: Indicator-based Geuna & Martin, Minerva 2003

46 PRFS before 2010 Red: Evaluation-based, Blue: Indicator-based Hicks, Research Policy, 2012

47 PRFS before 2016 Red: Evaluation-based, Blue: Indicator-based Jonkers & Zacharewicz, European Commision, 2016

48 Four examples: United Kingdom Red: Evaluation-based; Purposes: Research evaluation and institutional funding Seven major research assessments since Originally, evaluation was the method and funding allocation was the purpose. Now, the method has become an even more important purpose. The Metric Tide report (Wilsdon et al., 2015): Metrics should support, not supplant, expert judgement.

49 Four examples: The Netherlands Purpose: Research evaluation The indicator-based funding model for the universities does NOT include research performance. The research evaluation exercises do NOT influence institutional funding. This country does NOT have a PRFS.

50 Four examples: Sweden Blue: Indicator-based Purpose: Institutional funding PRFS reallocation of a small portion of institutional funding based on two indicators: 1) External revenues 2) Publications and citations in Web of Science

51 Four examples: Sweden in 2014: A report to the government Red: Evaluation-based Purposes: Research evaluation and institutional funding A UK-inspired model was designed by the Research Council and presented to the government

52 Four examples: Sweden in 2016: FOKUS not implemented Blue: Indicator-based Purpose: Institutional funding PRFS reallocation of a small portion of institutional funding based on two indicators: 1) External revenues 2) Publications and citations in Web of Science

53 Four examples: Norway since the 1990 ies Red: Evaluation-based Purpose: Research evaluation, not funding National research assessments inspired by the UK are performed by intervals. Like in the Netherlands, they do not influence funding.

54 Four examples: Norway since 2002 Blue: Indicator-based Purpose: Institutional funding, not evaluation The indicator-based model reallocates a small portion of the funding. It is based on several indicators. Most of them represent education, some represent research. The publication indicator reallocates 1,6 per cent Rammebevilgning fra KD (34,4 milliarder) Basisdel 69 % Resultatbasert del 31 % Åpen ramme 85% Lukket ramme 15% Studiepoeng 6,8 milliard Inntekter fra EU 500 millioner Studentutveksling 134 millioner Vitenskapelig publisering 550 millioner Kandidater 1,6 milliard Inntekter fra NFR og RFF 300 millioner Doktorgradskandidater 550 millioner BOA-inntekter 300 millioner

55 The two purposes of PRFS may have different emphasis: Research evaluation and funding Hicks (2012) defines PRFS as related to both purposes. They are national systems of research output evaluation used to distribute research funding to universities.

56 Oversikt 1. Hvordan finansieres og evalueres forskning? Tokanal-systemet Gjennom Forskningsrådet Evaluering av samfunnsrelevans. Altmetrics Direkte finansering av institusjoner Evalueringsbasert og indikatorbasert finansiering Evaluering av samfunnsrelevans: Publiseringsindikatoren

57 What made the impact? A piece of research? Writing an impact case for NIFU in 2015 Gunnar Sivertsen Nordic Institute for Studies in Innovation, Research and Education, Oslo, Norway

58 The linear models of innovation and communication Asking for narratives about: The research that underpinned the impact: This section should outline the key research insights or findings that underpinned the impact, and provide details of what research was undertaken, when, and by whom, The resulting impact: A clear explanation of the process or means through which the research led to, underpinned or made a contribution to the impact (for example, how it was disseminated, how it came to influence users or beneficiaries, or how it came to be exploited, taken up or applied).

59 The Norwegian template A. Abstract B. Describe the underpinning research C. Describe the societal impacts and how the research contributed to them D. Bibliographic references to the research publications E. References to other sources, reports, news items, etc. F. External contact persons

60 «The Norwegian model» The model has three components: A. A complete representation in a national database of structured, verifiable and validated bibliographical records of the peer-reviewed scholarly literature in all areas of research; B. A publication indicator with a system of weights that makes field-specific publishing traditions comparable across fields in the measurement of Publication points at the level of institutions; C. Institutional funding criteria which reallocate a small proportion of the annual direct institutional funding according the institutions shares in the total of Publication points. Institutional funding criteria

61 Controversial impact

62 Wide impact a funding indicator in three research sectors Hospitals Research institutes HE institutions Akershus universitetssykehus HF Agderforskning Ansgar Teologiske høgskole Betanien Hospital (Telemark) Bygdeforsk - Norsk senter for bygdeforskning Arkitektur- og designhøgskolen i Oslo Betanien Hospital HF CICERO - Senter for klimaforskning Bjørknes høyskole Diakonhjemmet sykehus AS CMI - Chr. Michelsens Institutt Dronning Mauds Minne Høgskole Finnmarkssykehuset - Helse Finnmark HF CMR - Christian Michelsen Research AS Fjellhaug Internasjonale høgskole Frambu senter for sjeldne diagnoser FFI - Forsvarets forskningsinstitutt Forsvaret - Forsvarets høgskole Helgelandssykehuset HF FNI - Fridtjof Nansens Institutt Forsvaret - Forsvarets ingeniørhøgskole Helse Bergen HF - Haukeland Universitetssykehus Folkehelseinstituttet Forsvaret - Krigsskolen Helse Fonna HF Forskningsstiftelsen Fafo Forsvaret - Luftkrigsskolen Helse Førde HF Havforskningsinstituttet Forsvaret - Sjøkrigsskolen Helse Møre og Romsdal HF IFE - Institutt for energiteknikk Handelshøyskolen BI Helse Nord-Trøndelag HF IRIS - International Research Institute of Stavanger Høgskolen i Bergen (Høgskulen på Vestlandet) Helse Stavanger HF - Stavanger universitetssykehus ISF - Institutt for samfunnsforskning Høgskolen i Hedmark (Høgskolen i Innlandet) Kreftregisteret KRUS - Kriminalomsorgens utdanningssenter Høgskolen i Lillehammer (Høgskolen i Innlandet) LHL-klinikkene MARINTEK - Norsk marinteknisk forskningsinstitutt AS Høgskolen i Molde (SINTEF Ocean) Lovisenberg Diakonale Sykehus Meteorologisk Institutt Høgskolen i Oslo og Akershus Martina Hansens Hospital Møreforsking AS Høgskolen i Sørøst-Norge Modum Bad Nasjonalmuseet for arkitektur, kunst og design Høgskolen i Østfold NKVTS (Nasjonalt kunnskapssenter om vold og traumatisk stress) NERSC - Nansen senter for miljø og fjernmåling Høgskolen Stord/Haugesund (Høgskulen på Vestlandet) Nordlandssykehuset HF NGI - Norges Geotekniske institutt Høgskulen for landbruk og bygdeutvikling Oslo universitetssykehus HF NIFES - Nasjonalt institutt for ernærings- og sjømatforskning Høgskulen i Sogn og fjordane (Høgskulen på Vestlandet) Private ideelle i Helse-Vest(Haraldsplass Diakonale NIFU - Nordisk institutt for studier av innovasjon, Høgskulen i Volda sykehus) forskning og utdanning RBUP Øst og Sør NIKU - Norsk institutt for kulturminneforskning Høyskolen Diakonova Revmatismesykehuset AS NILU - Norsk institutt for luftforskning Høyskolen for ledelse og teologi Sjukehusapoteka Vest NINA - Norsk institutt for naturforskning Høyskolen Kristiania St. Olavs hospital HF NIOM - Nordisk institutt for odontologiske materialer Kunsthøgskolen i Oslo Sunnaas sykehus HF NIVA - Norsk institutt for vannforskning Lovisenberg diakonale høgskole Sykehusapotek Nord HF NOFIMA AS Menighetsfakultetet - Det teologiske Menighetsfakultet Sykehusapotekene HF Nordlandsforskning NLA Høgskolen Sykehusapotekene i Midt-Norge HF Norsk institutt for bioøkonomi NMBU Sykehuset i Vestfold HF Norsk Regnesentral Nord universitet Sykehuset Innlandet HF NORUT - Northern Research Institute AS Norges handelshøyskole Sykehuset Telemark HF NORUT - Northern Research Institute Narvik AS Norges idrettshøgskole Sykehuset Østfold HF NTNU Samfunnsforskning AS Norges musikkhøgskole Sørlandet sykehus HF NUPI - Norsk utenrikspolitisk institutt NTNU Universitetssykehuset Nord-Norge HF PRIO - Institutt for fredsforskning Politihøgskolen Valnesfjord Helsesportssenter SINTEF energi Samisk høgskole/sámi allaskuvla Vestre Viken HF SINTEF fiskeri og havbruk AS Steinerhøyskolen

63 Impact also in some other countries

64 Scientific impact For the dimensions coverage, currency, legitimacy and transparency, the Norwegian Model is preferable to WoS-based Field normalized citation rates or Field normalized journal impact.

65 The underpinning research? NAVFs utredningsinstitutt (NIFU) in 1991

66 2017

67 The Norwegian model is not the impact of a piece of research Instead there was: 2002: a. Long-term funding for building bibliometric research at NIFU b. Research-based and independent constructive criticism from NIFU of the of the Norwegian research information systems as a potential data source c. A short paper from NIFU, invited by the Research Council, expressing b. d. A demand from the Government and the institutions (UHR) for a better performance indicator representing research in the funding model e. A large national meeting on November 17 th for all parties representing the interests in a-d, with a decision to follow up NIFU s proposal.

68 The Norwegian model is not the impact of a piece of research Instead there was: 2002: a. Long-term funding for building bibliometric research at NIFU b. Research-based and independent constructive criticism from NIFU of the of the Norwegian research information systems as a potential data source c. A short paper from NIFU, invited by the Research Council, expressing b. d. A demand from the Government and the institutions (UHR) for a better performance indicator representing research in the funding model e. A large national meeting on November 17 th for all parties representing the interests in a-d, with a decision to follow up NIFU s proposal : a. Research-based development and innovation in collaboration and interaction between NIFU, the funding organizations and the funded organizations in the three sectors.

69 The Norwegian model is not the impact of a piece of research Instead there was: 2002: a. Long-term funding for building bibliometric research at NIFU b. Research-based and independent constructive criticism from NIFU of the of the Norwegian research information systems as a potential data source c. A short paper from NIFU, invited by the Research Council, expressing b. d. A demand from the Government and the institutions (UHR) for a better performance indicator representing research in the funding model e. A large national meeting on November 17 th for all parties representing the interests in a-d, with a decision to follow up NIFU s proposal : a. Research-based development and innovation in collaboration and interaction between NIFU, the funding organizations and the funded organizations in the three sectors. Since around 2006: NIFU presents the Norwegian model or studies based on it at international conferences and in international publications NIFU is invited by governments or institutions in other countries to inform and/or as an active advisor in local implementations.

70 The story is about interaction, not about impact on An independent research organization: Responding to demands Engaging in collaboration Participating in innovation Remaining at critical distance when needed Which is normal at NIFU and not only realized in this particular case

71 Understanding and evaluating societal impact Societal impact of research is normal and part of society Normal impact is about daily activities and how well they are organized, not about individual incidents of particularly interesting or impressive impact Normal impact is almost impossible to measure. Interesting evidence can be provided, but this is costly. Societal impact primarily needs to be understood in order to be evaluated. Normal interactions with society can be evaluated as such at the organizational level, taking all inter-actors into consideration

72 The Norwegian template: Need for a redesign? A. Abstract B. Describe the underpinning research C. Describe the societal impacts and how the research contributed to them D. Bibliographic references to the research publications E. References to other sources, reports, news items, etc. F. External contact persons

73 Oversikt 1. Hvordan finansieres og evalueres forskning? 2. Hvordan publiseres forskning? Sideblikk: Åpningen av humsam-tidsskrifter 3. Den norske publiseringsindikatoren 4. Evalueringen av indikatoren i Videreutvikling av indikatoren

74

75 Fordelingen mellom publikasjonsformer pr. fagområde Tidsskriftsartikler Bokartikler Bøker Humaniora Samfunnsfag Medisin og helse Naturvitenskap Teknologi 0 % 20 % 40 % 60 % 80 % 100 %

76 Prosentandel publikasjoner på norsk pr. fagområde 100% 90% Humaniora 80% 70% Samfunnsfag 60% 50% 40% 30% Medisin og helse Naturvitenskap 20% 10% Teknologi 0%

77 Andel publikasjoner på norsk pr. fag: Samfunnsfag Rettsvitenskap Kjønnsforskning Pedagogikk og utdanning Sosiologi Medier og kommunikasjon Økonomisk-administrative fag Statsvitenskap Sosialantropologi Geografi Samfunnsøkonomi Bibl- og informasjonsvit Utviklingsstudier 0% 20% 40% 60% 80% 100%

78 Andeler av tidsskriftene og artiklene som utgis på norsk Basert på artikler i vitenskapelige tidsskrifter, norske U&H % 40% 30% Tidsskrifter Artikler 20% 10% 0% Hum Samf Med Nat Tekn

79 Scholarly journals in the national language (examples)

80 2011 National disciplinary journals have a long tradition of communication with society Now threated by decreasing subscriptions and increasing needs for subsidies Why not instead subsidize them to be completely open, thereby reinforcing their traditional societal role?

81 2011, 2013 A study of 76 national SSH journals: Contents Authors Roles in communication Proposal for a new business model

82 2011, 2013, 2017 Implemented in a governmental white paper on the humanities!

83 The solution A narrow selection of only the central, most respected and most needed SSH journals publishing in Norwegian No author payment Funded partly by the government, partly by a consortium of research institutions that frequently publish in the journals Monitored by a national panel appointed by the deans of the humanities and social sciences

84 Example: Sociology: Level 2 journals in white Number of articles per journal from Norwegian institutions in Norsk sosiologisk tidsskrift Journal of Risk Research Nordic Journal of Working Life Studies Journal of Ethnic and Migration Studies Qualitative Inquiry Sociology of Health and Illness Acta Sociologica Sociologia Ruralis Studies in Critical Social Sciences Sociology Professions and Professionalism European Sociological Review International Journal of Action Research Social Indicators Research Work, Employment and Society European Societies Society & Natural Resources Social Inclusion Ethnic and Racial Studies Social and Critical Theory Human Affairs Sociological Review Child Indicators Research Journal of Youth Studies Leisure Studies 9 other journals on level other journals on level 1 13,4 % of all articles

85 Contents: Not only original research articles These journals are fora for scholarly communication, not just article collections Thematic introduction Original articles Essay for debate on policy implications Book reviews

86 Authors: Not only employed by research institutions These journals represent disciplines that relate to society Government official Working for NGO Emeritus

87 Oversikt 1. Hvordan finansieres og evalueres forskning? 2. Hvordan publiseres forskning? Sideblikk: Åpningen av humsam-tidsskrifter 3. Den norske publiseringsindikatoren 4. Evalueringen av indikatoren i Videreutvikling av indikatoren

88

89 «The Norwegian model» The model has three components: A. A complete representation in a national database of structured, verifiable and validated bibliographical records of the peer-reviewed scholarly literature in all areas of research; B. A publication indicator with a system of weights that makes field-specific publishing traditions comparable across fields in the measurement of Publication points at the level of institutions; C. Institutional funding criteria which reallocates a small proportion of the annual direct institutional funding according the institutions shares in the total of Publication points. Institutional funding criteria

90 «The Norwegian model» The model has three components: A. A complete representation in a national database of structured, verifiable and validated bibliographical records of the peer-reviewed scholarly literature in all areas of research; B. A publication indicator with a system of weights that makes field-specific publishing traditions comparable across fields in the measurement of Publication points at the level of institutions; C. Institutional funding criteria which reallocates a small proportion of the annual direct institutional funding according the institutions shares in the total of Publication points. Institutional funding criteria

91 Publication points in the funding of higher education (HE) institutions National budget for HE institutions Not performancebased 69 % Performance-based 31 % Reallocates 1,6 per cent of total budget Educational activities 85% Research activities 15% Study points EU external funding Student mobility Publication Points Final master degrees National external funding Final PhD Revenues from contract research

92 Resultatindikatorene i grunnbevilgningene til universiteter og høgskoler, forskningsinstitutter og helseforetak Universiteter og høgskoler Tildelte doktorgrader Tildelinger fra EU Tildelinger fra NFR Vitenskapelig publisering Helseforetak og institutter Utførte doktorgrader Tildelinger fra EU Tildelinger fra NFR Vitenskapelig publisering

93 «The Norwegian model» The model has three components: A. A complete representation in a national database of structured, verifiable and validated bibliographical records of the peer-reviewed scholarly literature in all areas of research; B. A publication indicator with a system of weights that makes field-specific publishing traditions comparable across fields in the measurement of Publication points at the level of institutions; C. Institutional funding criteria which reallocates a small proportion of the annual direct institutional funding according the institutions shares in the total of Publication points. Institutional funding criteria

94 Ufullstendig dekning av internasjonale tidsskrifter. Veldig begrenset dekning av vitenskapelige bøker. Tilfeldig eller manglende dekning av vitenskapelig publisering på nasjonalspråkene. Books Indexed journals Journals and series Coverage of 70,500 scholarly publications from the higher education sector in Norway % 90% 80% Scopus Web of Science 70% 60% 50% 40% 30% 20% 10% 0% Health Sciences Natural Sciences Engineering Social Sciences Humanities

95 En felles definisjon av vitenskapelig publikasjon Publikasjonen må: 1. presentere ny innsikt 2. være i en form som gjør resultatene etterprøvbare eller anvendelige i ny forskning 3. være i et språk og ha en distribusjon som gjør den tilgjengelig for de fleste forskere som kan ha interesse av den 4. være i en publiseringskanal (tidsskrift, serie, bokutgiver, nettsted) med rutiner for fagfellevurdering*) *) Lokale kanaler inkluderes ikke (2/3 eller mer av forfatterne er fra samme institusjon)

96

97 Referenser importert fra eb of Science

98 Egenrapporterte referanser

99 Principles behind the use of institutional data in a shared national Current Research Information System Completeness: All scholarly publications and other results from research are included Transparency: Every institution can see and check all other institutions data. The national database is also online and open to society at large. Multiple use of the data: CV s, applications, evaluations, annual reports, internal administration, bibliography for Open Archives, links to full text, etc.

100

101

102 «The Norwegian model» The model has three components: A. A complete representation in a national database of structured, verifiable and validated bibliographical records of the peer-reviewed scholarly literature in all areas of research; B. A publication indicator with a system of weights that makes field-specific publishing traditions comparable across fields in the measurement of Publication points at the level of institutions; C. A performance-based funding model which reallocates a small proportion of the annual direct institutional funding according the institutions shares in the total of Publication points. Institutional funding criteria

103 En felles definisjon av vitenskapelig publikasjon Publikasjonen må: 1. presentere ny innsikt 2. være i en form som gjør resultatene etterprøvbare eller anvendelige i ny forskning 3. være i et språk og ha en distribusjon som gjør den tilgjengelig for de fleste forskere som kan ha interesse av den 4. være i en publiseringskanal (tidsskrift, serie, bokutgiver, nettsted) med rutiner for fagfellevurdering*) *) Lokale kanaler inkluderes ikke (2/3 eller mer av forfatterne er fra samme institusjon)

104 Three main publication types 1) Article in ISSN-title 2) Article in ISBN-title 3) ISBN-title Channel Type E.g. scientific article in Nature or scholarly ISBNtitle on Oxford University Press.

105 Dynamic records of scientific and scholarly ISSN-titles og ISBNpublishers 20,000 ISSN-titles and 1,000 book publishers so far Africa Today African Affairs African American Review African Archaeological Review African Arts African Development Review African Economic History African Entomology African Identities African Journal of Biotechnology African Journal of Crop Science African Journal of Ecology African Journal of Environmental Assessment and Management African Journal of Food, Agriculture, Nutrition African Journal of Neurological Sciences African Literature Today CABI Publishing Cambridge University Press Cappelen Akademisk Forlag Carfax Publishing Carl Heymanns Verlag Carlsson bokförlag Catholic University of America Press Central European University Press Channel View Publications Chinese University Press Christian Ejlers s Forlag

106 All channels can be searched, and new channels can be suggested

107 Publication points for institutional level measurement Publication type Level 1 Level 2 Article in ISSN-title 1 3 Article in ISBN-title 0,7 1 ISBN-title 5 8

108 Publication points for institutional level measurement Publication type Level 1 Level 2 Article in ISSN-title 1 3 Article in ISBN-title 0,7 1 ISBN-title 5 8 How to count publications with more than one author and/or institutional affiliation? Publication Author 1 Author 2 Author 3 Institution A Institution B Institution C Institution C

109 Simulering i NVI-data fra Gjennomsnittlig antall publikasjoner og poeng Fagområde Forskere Antall publikasjoner Nåværende poeng Nye poeng Humaniora ,7 3,9 4,6 Medisin og helsefag ,3 1,5 4,5 Naturvitenskap ,4 1,8 4,7 Samfunnsvitenskap ,5 3,1 4,7 Teknologi ,5 2,1 4,4

110 Publication points for institutional level measurement Publication type Level 1 Level 2 Article in ISSN-title 1 3 Article in ISBN-title 0,7 1 ISBN-title 5 8 Publication points are fractionized using the square root of the institution s fraction Publication Author 1 Author 2 Author 3 Institution A Institution B Institution C Institution C

111 How are the two levels defined? Publication type Level 1 Level 2 Article in ISSN-title 1 3 Article in ISBN-title 0,7 1 ISBN-title 5 8? Publication points are fractionized using the square root of the institution s fraction Publication Author 1 Author 2 Author 3 Institution A Institution B Institution C Institution C

112 Incentives: A balance between internationalization and local language publishing Level 2 (20 %) represents only the most prestigous international journals and book publishers in each field On level 1 (80 %), publication channels in all languages, including Norwegian, are given the same points Level 2: Higher points to 20 % of the publications Level 1: Normal points to 80 % of the publications

113 Example: Gastroenterology Level 2 (20 %) represents only the most prestigous international journals and book publishers in each field On level 1 (80 %), publication channels in all languages, including Norwegian, are given the same points Endoscopy, Gastroenterology, Gut, Inflammatory Bowel Diseases, Hepatology, Journal of Hepatology = 20 per cent of the world s articles 胃肠病学和肝病学杂志, Scandinavian Journal of Gastroenterology, plus all other journals = 80 per cent

114 Decisions on level 1 and 2 Level 2 is decided by national disciplinary councils in collaboration with a National Publishing Board Level 1 is decided by the managing organization of the register in collaboration with the National Publishing Board Level 2: Higher points to 20 % of the publications Level 1: Normal points to 80 % of the publications The Journal Impact Factor is used only as information, not to take the decision Quality control for inclusion, but WoS coverage not necessary

115

116

117 Discussion of the Level 1 and 2 distinction Ahlgren, Colliander & Persson (2012) For the dimensions coverage, currency, legitimacy and transparency, the Norwegian Model is preferable to WoS-based Field normalized citation rates or Field normalized journal impact.

118 Discussion of the Level 1 and 2 distinction Schneider, Aagaard & Bloch (2015) The results show that for the Norwegian case, overall publication activity goes up, impact remains stable, and there is no indication of a deliberate displacement of journal publication activities to the lowest-impact journals.

119

120 Oppslag på toalettene ved Senter for fremragende forskning (lingvistikk) ved Universitet i Tromsø

121 2017

122 Helsides opprop fra 223 professorer i Aftenposten 5. mai 2006 (betalt av Den norske Forleggerforening, Den norske Bokhandlerforening, Norsk faglitterær forfatter- og oversetterforening og Institusjonen Fritt Ord)

123 Oversikt 1. Hvordan finansieres og evalueres forskning? 2. Hvordan publiseres forskning? Sideblikk: Åpningen av humsam-tidsskrifter 3. Den norske publiseringsindikatoren 4. Evalueringen av indikatoren i Videreutvikling av indikatoren

124 Evaluering publisert 16. januar 2014 Utført i 2013 av Dansk Center for Forskningsanalyse, Aarhus Universitet for Universitets- og høgskolerådet finansiert av Kunnskapsdepartementet

125

126 Andel artikler i Web of Science Kilde: Web of Science (Thomson Reuters) hos CWTS, Leiden Universitet 2,0 % 1,8 % Sverige 1,6 % 1,4 % 1,2 % Danmark 1,0 % 0,8 % 0,6 % Finland 0,4 % 0,2 % Norge 0,0 %

127 Overordnete positive virkninger Mer fokus på og prioritering av de ansattes forskningsvilkår Forskning oppfattes i større grad som et felles, ikke bare et individuelt ansvar Nye publikasjoner får intern oppmerksomhet og anerkjennelse, ikke bare ute blant spesialister Diskusjon av vitenskapelige standarder og publiseringsmønstre på tvers av fag- og avdelingsgrenser Bedre, sammenlignbar informasjon om egen virksomhet

128 De tre I ene 1. Informasjon: Oversikt over og synliggjøring av forskningen 2. Indikator: Statistikk, å kunne følge utviklingen 3. Incentiv: Verdsetting av forskning

129 Oversikt 1. Hvordan finansieres og evalueres forskning? 2. Hvordan publiseres forskning? Sideblikk: Åpningen av humsam-tidsskrifter 3. Den norske publiseringsindikatoren 4. Evalueringen av indikatoren i Videreutvikling av indikatoren

130

131 Manglende fagfeltsnøytralitet

132 Balanse er viktig fordi indikatoren bidrar til finanseringen av institusjoner med ulike fagprofiler 100 % 90 % Tekn 80 % 70 % MatNat 60 % 50 % Helse 40 % 30 % Samf 20 % 10 % Hum 0 % NTNU UiO NMBU

133 Den nye indikatoren Publication type Level 1 Level 2 Article in ISSN-title 1 3 Article in ISBN-title 0,7 1 ISBN-title 5 8 Brøkdeling av publikasjonen ved mer enn én institusjon Publication Institusjonens andel erstattes med kvadratroten av institusjonens andel Author 1 Institution A Institution B Author 2 Institution C Author 3 Institution C

134 Konsekvensene En mer fagnøytral publiseringsindikator Økt stimulans til samarbeid om vitenskapelig publisering mellom institusjoner og land En fortsatt motvirkning av et mulig ønske om å føre opp flere forfattere enn rimelig i publikasjonen Større vanskeligheter enn før med å bruke indikatoren på individnivå

135