REPORT eCampus summary

To: Ministry of Education and

Research, universities and

university colleges

From: The eCampus programme

Written by: Ingrid Melve and Thorleif

Hallén

Copy: eCampus prioritization council

Date: 01/02/2017

Concerning: Digitization of higher

education

2

Contents

Preface ............................................................................................................... 6

Summary ............................................................................................................. 7

1 Introduction .................................................................................................... 9

1.1 About the report ......................................................................................... 9

1.2 Structure .................................................................................................. 9

2 eCampus as a tool in higher education ................................................................... 10

2.1 Focus areas .............................................................................................. 10

2.2 Established services .................................................................................... 11

2.3 Working groups and best practice .................................................................... 12

2.4 Joint agreements ....................................................................................... 12

2.5 UNINETT’s role .......................................................................................... 12

2.6 Local organization ...................................................................................... 13

3 Focus area: eCampus video for learning .................................................................. 14

3.1 Videos in higher education ............................................................................ 14

3.2 Video services ........................................................................................... 15

3.2.1 Simple, user-centred functionality: Relay ..................................................... 15

3.2.2 Automated, large-scale needs: Mediasite ..................................................... 16

3.3 Deployment complete, but full potential not yet reached ....................................... 17

4 Focus area: eCampus collaboration services ............................................................. 18

4.1 Areas of application for collaboration services .................................................... 18

4.2 Online collaboration services in higher education ................................................. 18

4.3 Development of and preparation for next generation solutions ................................. 21

5 Focus area: eCampus national ICT infrastructure ....................................................... 22

5.1 Facilitating for implementation ...................................................................... 22

5.2 Joint requirements ..................................................................................... 22

5.3 Development ............................................................................................. 23

6 New focus area: eCampus digital examination .......................................................... 24

6.1 Digital assessment ...................................................................................... 24

6.2 National digital examination project 2014–2015 ................................................... 24

6.2.1 Broad collaboration ............................................................................... 25

6.2.2 Development contracts and market maturation .............................................. 26

6.2.3 Best practice documents ......................................................................... 26

3

6.2.4 Establishment and management of FS integration ........................................... 27

6.3 Procurement and experience sharing ................................................................ 27

7 eCampus strategy: Digital learning environment ........................................................ 29

7.1 From LMS to learning platform ....................................................................... 29

7.2 Standardized interfaces for modular solutions ..................................................... 32

7.3 Open education resources (OER) ..................................................................... 32

7.4 National services for a digital learning environment .............................................. 33

8 Snapshots of higher education in Norway today ......................................................... 35

8.1 Higher education: a functional model ............................................................... 35

8.2 Mapping the current state of affairs ................................................................. 35

8.2.1 Digital State of Affairs 2014 ..................................................................... 35

8.2.2 Student priorities .................................................................................. 36

8.2.3 State of affairs in Northern Norway, eCampus report 2012 ................................ 36

8.2.4 Mapping out the organizational support for digitization in the HE sector ................ 37

8.3 Videos in higher education ............................................................................ 37

8.4 Collaboration (and cloud services) in higher education ........................................... 38

8.5 ICT architecture (and digital examinations) ........................................................ 38

8.6 Discussions at the final eCampus conference ...................................................... 39

9 Lessons from eCampus ....................................................................................... 41

9.1 Work methodology ...................................................................................... 41

9.1.1 Working groups .................................................................................... 41

9.1.2 Best practice ....................................................................................... 41

9.1.3 Collaboration with providers, development agreements .................................... 42

9.1.4 Large/small institutions .......................................................................... 42

9.1.5 Cost/benefit assessments ........................................................................ 43

9.2 Centralized measures, local responsibility .......................................................... 43

9.2.1 Server consolidation .............................................................................. 43

9.2.2 Support portal ...................................................................................... 43

9.2.3 Things that took longer than anticipated ...................................................... 44

9.3 Quality development and a shared understanding of needs ..................................... 44

9.3.1 Different levels of technological maturity .................................................... 44

9.3.2 Timing: hitting the mark ......................................................................... 45

9.3.3 Collaboration with other national agencies ................................................... 45

9.3.4 Different approaches to support function organization ..................................... 46

9.3.5 Local eCampus projects .......................................................................... 46

4

9.4 What do national strategies for ICT in higher education need? .................................. 46

10 Vision ........................................................................................................... 48

10.1 Trends .................................................................................................... 49

10.1.1 International trends ............................................................................... 49

10.1.2 MOOCs to Norway .................................................................................. 49

10.2 Objectives from ICT strategy ......................................................................... 49

10.3 eCampus lessons and vision ........................................................................... 51

10.3.1 Lessons .............................................................................................. 51

10.4 Vision for tools .......................................................................................... 52

10.5 Vision for quality development ....................................................................... 52

10.6 Vision for a joint framework .......................................................................... 52

11 The road ahead ............................................................................................... 54

11.1 Shared vision ............................................................................................. 54

11.2 Digitization prerequisite in quality development .................................................. 54

11.2.1 Objectives .......................................................................................... 54

11.2.2 Digitization arena ................................................................................. 54

11.3 Tools for higher education ............................................................................ 54

11.3.1 Objectives .......................................................................................... 54

11.3.2 Local support and national services ............................................................ 55

11.4 Joint framework for ICT use in higher education .................................................. 55

11.4.1 Objectives .......................................................................................... 55

11.4.2 Better use of data: specifications and progress .............................................. 55

11.5 Ongoing initiatives ...................................................................................... 56

12 References ..................................................................................................... 57

5

Appendix 1: Digitization initiatives ............................................................................. 59

Digitization prerequisite in quality development .......................................................... 59

Local measures ............................................................................................... 59

National measures ............................................................................................ 59

Tools for higher education ..................................................................................... 59

Local measures ............................................................................................... 59

National measures ............................................................................................ 61

Joint framework for ICT use in higher education .......................................................... 61

Local measures ............................................................................................... 61

National measures ............................................................................................ 62

Appendix 2: Working groups ..................................................................................... 63

Appendix 3: eCampus projects .................................................................................. 64

Appendix 4: Campus Best Practice Documents ................................................................ 65

Appendix 5: eCampus prioritization council ................................................................... 67

Appendix 6: eCampus staff ....................................................................................... 69

6

Preface

This report summarizes the status of ICT in higher education, and is based on experiences from the eCampus programme. The eCampus prioritization council wanted the programme’s final report to focus on the future. What remains to be done before ICT implementation in higher education reaches the level we would like to see? Through the eCampus programme, we have had the pleasure of working closely with university and university college personnel. We would like to take this opportunity to thank everyone who participated in working groups, coming together to get to the bottom of problems, document best practices and share the results for the benefit of others. It was a delight to experience the sharing culture and your strong dedication first hand. Throughout the programme period, several hundred people participated in working groups and workshops. The use of ICT in higher education has changed considerably in the years since the programme first started. Contributions from the eCampus programme helped universities and university colleges develop better solutions for the use of ICT in education and learning. We have observed that the use of videos has increased exponentially during this period. Technological developments in society in general have also driven change; tablets, for example, did not yet exist when the programme application was submitted. We have worked closely with the Norwegian Agency for Digital Learning in Higher Education on digitizing higher education. Their approach was innovation in teaching, quality development and accessibility, whereas our approach was more technologically driven. This document is intended to serve as a basis for discussion for both reports and practical work on infrastructure and joint technological solutions for ICT in higher education. We opted to include some information on local projects; prioritizations should be made on the basis of the overall picture, and not be limited to a narrow description of national measures, without seeing these in context with the work that is carried out at each institution.

7

Summary

eCampus developed infrastructure, built national services and established an arena for digitization, where the institutions can elevate the development of digital solutions to a strategic level, defining objectives and measures for the digitization of learning processes. The eCampus programme aimed to contribute to the digitization of higher education in the period from 2012 to 2016. The programme was terminated after five years of intense activity and considerable changes in higher education in Norway and abroad. We achieved a lot, and many things improved. The journey is still the destination, and the destination is ever-changing. In many ways, eCampus surveyed the land and built the foundation, while the surrounding landscape and climate underwent a massive transformation that is still on-going. We are excited to see how the structure will look in the end, and this report makes recommendations for further construction. Most universities and university colleges are well under way to digitizing their education programmes, and the challenge at this point is to go from the project stage to institutionalized practice. Joint services, a shared understanding of needs, and joint infrastructure are all available, but oftentimes not for entire campuses or every discipline. The national boost provided by eCampus helped, but changing an established practice is a massive undertaking that requires unwavering effort. Joint video services have been established, and new infrastructure is developed in step with increased demand and use. Collaboration services for use in teaching have been developed. Use of the services established as a part of eCampus has increased exponentially, but they have yet to reach their full potential. Several institutions have chipped in to keep these services in operation after the programme ends, and sustainable funding has been established. The digital learning environment, in all its forms, is a field that is continually and rapidly evolving. Initially, eCampus was supposed to focus on the three main areas video for learning, interaction, and national architecture. Later, cloud services, open learning, open learning resources and digital assessment were added as key focus areas. Digital examinations soon became a central focus for eCampus, and the concerted effort to develop solutions and a shared understanding of needs culminated in framework agreements with three suppliers toward the end of 2016. Digital examinations were not originally intended to be a major focus for the programme, but soon became a priority, and considerable resources were dedicated to this area. A joint development effort with suppliers in 2014–2015 led to better solutions for Norwegian institutions, which integrated well with administrative solutions. The development of new solutions and the effort to digitize work processes have been parallel processes, and a wide range of local projects have linked up with the national project. Similarly, the programme has established requirements (which led to collaboration on joint agreements) for learning platforms (LMS) and systems for plagiarism detection. The programme also established a number of joint national services with a high (and increasing) level of use among students and staff: high-quality joint services at a low cost. The necessary change was effected through close and regular collaboration between key personnel in the HE sector, through pilots and supplier collaboration, and in working groups coordinated by eCampus. The programme was a joint and communal effort for and by the Norwegian higher education sector, driven by a wide range of

8

dedicated individuals, interest groups, disciplines, and associated key individuals from all over the country. We are in the process of forging a solid technological and professional foundation for learning analytics and adaptive learning in the years to come. The primary focus is shifting, from the implementation of the right technology to the utilization of this technology to improve learning and teaching. This process goes far beyond the IT department; the development extends to the classroom, to teachers, administrators and students—technology should facilitate for better learning in all its facets and nuances. The most important lesson the programme instilled in us, is that we must continue to engage with each other for the purpose of exchanging experiences and learning from each other. The entire Norwegian higher education sector must work together to make sure that we make the right choices and implement the best possible measures. Digital literacy should be promoted at every level, from top management to the hands and heads of individuals. Together, we must continue to develop the national services we have already established, and build new ones in response to changing circumstances. A number of specific recommendations for tools, quality development and a joint framework have been made for both local and national initiatives. Local efforts in each institution will be crucial. The job is done, let the work begin.

9

1 Introduction

1.1 About the report This report summarizes the experiences from the eCampus programme (2012–2016) and offers recommendations for further initiatives related to the use of ICT in higher education. Teaching-related services have been discussed on the basis of experiences from the programme and the guidelines established by the proposed ICT strategy for higher education (Ministry of Education and Research working group for ICT strategy and comprehensive solutions, 2016). The report was first presented as a working draft at the eCampus final conference in Tromsø, 28–29 October 2016. This final conference was organized as a workshop, which yielded a number of suggestions for the report and recommendations for further activities after the end of the programme. This document serves as a basis for discussion and informs concretization initiatives both locally and nationally. Appendix 1 includes a detailed list of measures. An evaluation of the eCampus programme (Tømte, Aanstad, & Løver, 2016) was published as a NIFU report (Nordic Institute for Studies in Innovation, Research and Education (NIFU) 2016:44) based on studies carried out in the autumn of 2016.

1.2 Structure First, this document describes the various eCampus initiatives (chapter 2), followed by a review of the strategic areas for the use of ICT in higher education (chapters 3–7). Then, chapter 8 presents the current status at a national level, along with some reports from local initiatives. Lessons from the eCampus programme point to certain factors to consider in other, similar initiatives (chapter 9). This is followed by a presentation of the vision for the field in light of experiences gleaned from the eCampus programme (chapter 10). Finally, recommendations for future initiatives are established (chapter 11). Appendix 1 lists recommendations for a specific list of measures, including both measures intended for nation-wide implementation and measures to be implemented by the individual institutions. Appendix 2 lists working groups affiliated with the eCampus programme. Appendix 3 includes a list of projects completed as part of the eCampus programme. Appendix 4 focuses on best practices and discipline specifications developed as part of the programme. Appendix 5 presents the prioritization council’s mandate and composition. Appendix 6 includes a presentation of eCampus associates.

10

2 eCampus as a tool in higher education The eCampus programme was the Ministry of Education and Research’s signature programme for ICT in higher education in the period from 2012 to 2016. The programme was executed by UNINETT, and Ingrid Melve served as the programme director.

2.1 Focus areas The eCampus programme application from 2009 described three main focus areas, based on discussions from a working group comprising resource personnel from the HE sector.

• Video for learning: Implementation of video and lecture recording • Interaction: Collaborative tools and video conferencing • National architecture: Mobile solutions and ICT architecture

The first two areas have been subject to various initiatives throughout the programme period, whereas focus in the last area gradually shifted away from mobile solutions toward digital examinations.

Figure 1: Primary focus areas in the eCampus programme, including preparatory phases the eCampus programme focused on the green area in Figure 2, which focuses on large-scale implementation of technology in education. Visionaries quickly recognize the utility value, and pragmatists are essential for taking a project from pilot to systematic implementation. Sceptics who fail to implement the new technology, and explorers who are always moving on to the next big thing, are not part of the target audience for the programme.

2009 •Programme application

2011 •Head start: Interaction and video

2012 •Start-up Northern Norway: Interaction and video

2013 •Full programme launch: Video, interaction and architecture

2014 •Video, interaction and digital examination

2015 •Video, interaction and digital examination

2016 •Digital learning environment, video and interaction

11

Figure 2: Deployment curve for the implementation of new technology, based on Everett’s “Diffusion of Innovation”

2.2 Established services Surveys carried out at start-up clearly indicated that a sector-wide basic package including solutions for video broadcasting, interaction and sharing was requested across the board. This well-defined feedback made it possible to get these services in place relatively quickly. The survey process included a report from SINTEF (Finne & Hatling, 2012) and interviews carried out at all universities and university colleges in Northern Norway. The SINTEF report was a discussion memo, reviewing the various aspects of ICT use in higher education. One of its conclusions was that “[i]n the long run, we must develop strategies at an institutional level, wherein the teaching communities lead the way in collaboration with the IT communities. In order to move forward, we have to shift our focus away from a strategy that relies on the deployment of standardized solutions and implementation of best practices, toward a strategy based on user-driven innovation in teaching and learning, supported by a stable infrastructure for IT and IT support.” In cases where the needs were clearly defined and solutions were available either from external suppliers or suppliers affiliated with the HE sector, it was possible to establish services right off the bat when the programme launched in 2013, based on start-up initiatives and preparations from 2012. In cases where needs were less distinct, or where the market were not ready, working groups were established to consider available options, as described in chapters 3–6. Access for everyone also entails that the services must be available at all times. In practice, this means that the services must be designed for a high degree of self service and have a user-friendly interface. User support is handled through collaboration with the support services of the respective institutions and the establishment of a self-service portal. The portal (UNINETT, 2013-2016) is tailored to each user based on his or her institutional affiliation. In addition, second-line support is available to technical support personnel at the respective institutions. The portal has considerably reduced the number of e-mail and phone support requests to the eCampus team, but it has also given rise to some confusion concerning the difference between first- and second-line support. Self service is a powerful streamlining tool, as it frees up resources that would otherwise have had to be

12

dedicated to dealing with all support requests, regardless of topic. These resources are better allocated elsewhere, e.g. in discipline-specific ICT use.

2.3 Working groups and best practice Working groups with open participation from the HE sector were chosen as the preferred collaboration model for identifying best practices and establishing solution requirements. Several hundred individuals have participated in these working groups over the course of the programme period. The digital examination initiative, in particular, has involved a high number of people, as it required expertise and input from examination offices, IT, user support, student administration, students, and teachers. Working group results have been documented in best practice documents (cf. Appendix 4). The working groups also served as networks of competence for participants.

2.4 Joint agreements Joint agreements have been concluded for services and the procurement of solutions for the respective disciplines. Through joint agreements, the eCampus programme has been able to establish a shared understanding of needs, reduce additional costs for each institution in the procurement process, establish joint operations, and increase the market power of Norwegian universities and university colleges. Physical adaptations on a campus to update the institution’s approach to teaching in response to new needs and to implement new technology are costly and time-consuming. In order to systematize this process, joint specifications and a joint procurement agreement have been established. At the beginning of the programme period, teaching room specifications were a priority, but with the emergence of digital examinations it has become necessary to document specifications for examination rooms as well. We have seen that collaboration and active learning give rise to new requirements for on-campus and virtual-campus spaces. Internet access is now a pre-requisite for any space intended for serious work or study. Society in general has made access to resource material, writing tools and collaboration solutions readily available to a much greater degree than ever before, which influences the students’ habits and expectations. Server consolidation for joint services generated considerable savings and made it possible to provide easy access without initiating projects to develop solutions and competence at each institution. This has freed up local resources, and made them available for user support and development projects focused on relevant and discipline-specific applications.

2.5 UNINETT’s role The current discourse on structure in the Norwegian HE sector means there are some uncertainties concerning the organization of joint IT services. The recommendations for future initiatives in appendix 1 do not take organizational structure into account, but outlines areas for which joint solutions ought to be found. UNINETT is one of the sector’s own internal suppliers of ICT infrastructure and joint solutions in support of, inter alia, education and research. UNINETT’s role is therefore limited to considering quality in education from an ICT perspective. The White Paper on Structure (White Paper no. 18 (2014–2015) Concentration for Quality (Norwegian Government, 2014)) points to ICT as an enabling

13

technology for change and quality improvement in the sector. We support this conclusion, but find that the sector struggles considerably when it comes to utilizing this technology. Several national ICT organizations want to work with ICT departments and other supporting communities to support the sector’s primary functions: research and education. The sector has come a long way in reaching many of its stated objectives, but increased demands for efficiency, digitization, and quality improvement, and, not least, new technological developments in general, mean the sector is expected to do even more. There is a pressing need for uniform control, as described in the White Paper on Structure, and the report Kunnskapssektoren sett utenfra (Gjedrem & Fagernæs, 2016) further reinforces this understanding of the situation.

UNINETT coordinates the effort to document best practices for ICT solutions. Best practices are based on work carried out at individual institutions, who have established good practices on campus, and collect experiences across several universities and university colleges. The effort is normally organized as a working group, where UNINETT serves as the secretariat and encourages written documentation. Best practices are collected in best practice documents, the majority of which have been translated and shared internationally. Best practice documents are used as documentation in procurement processes, and contribute to a shared understanding, and effective dissemination, of knowledge. The eCampus programme has organized a number of working groups and workshops. The working groups primarily focused on specific tasks, and their collaboration largely took place online. Physical workshops were primarily used to share experiences, and as a forum for discussion and the presentation of results. Experience sharing has been crucial for reaching a wide audience, and the participation from universities and university colleges has been good. UNINETT currently oversees several teaching-related national services. These services are organized as a cooperative effort, where participating institutions share the cost of procurement, administration and operation. The services are available through joint procurement, as locally operated services, as services operated by UNINETT, as cloud services, or as services developed by the Nordic HE sector.

2.6 Local organization The organization of support for ICT implementation in teaching and learning varies in universities and university colleges. Institutions all offer a minimum package of services, but the composition of this package varies from institution to institution. There are considerable differences between institutions in terms of digital competence, available services, and the organization of support for ICT implementation. There are also considerable cultural differences between different communities within one institution in terms of which services are implemented. The dearth of any systematic approaches to ICT-supported teaching was discussed in the Norwegian Agency for Digital Learning in Higher Education’s report entitled Kunsten å ile langsomt (The art of hurrying slowly) (Norwegian Agency for Digital Learning in Higher Education, 2010). Most institutions have initiated projects looking into the digitization of education, but few have developed a broad, systematic strategy. Several institutions engaged in local eCampus initiatives for a time, focused either on video implementation or digital examinations. We see, however, that the institutions that have applied a systematic and long-term strategy, with a structured and well-ordered administration of the digitization effort, play in a league far above the institutions where this effort is reliant on dedicated individuals or stand-alone projects.

14

Study programme coordinators have played a key role in the systematic digitization effort, in addition to the extensive development work driven by support units and dedicated individuals from various disciplines. The support units, where available, provided valuable feedback for the work carried out at national level, and they actively contributed to the greater good. A systematic and long-term approach is required for a successful transition to an active implementation of ICT and online resources in an effort to improve education quality. Sporadic initiatives are far less successful than a systematic approach, and this is true for lecture recording and other video applications as well. On campuses where recording functionality has been made available in all teaching rooms, and where users have been provided with proper training to ensure digital literacy, use of the services is much more widespread than on campuses where no such efforts have been made. Automation also promotes use, in addition to reducing costs. Emphasizing self service has reduced user support costs.

3 Focus area: eCampus video for learning Through the eCampus programme, a joint video infrastructure for higher education has been developed, with several national services. Equipment specifications have been documented, and procurement agreements have been concluded and renewed. The number of students with access to recorded lectures has increased considerably.

3.1 Videos in higher education The use of videos in higher education serves several purposes, and it is not an Internet-driven phenomenon. The Norwegian Institute of Technology had its own TV studio in the 1960s, where they recorded lectures that were later marketed to industry clients. Patterns of use developed in step with technology, and the cost of production and distribution gradually decreased. As a result, videos from Norwegian universities became more and more accessible. Naturally, developments in IT over the past 20 years have promoted more widespread use. The Internet streamlines distribution, and more affordable, better and faster technology contributes to increased use, increased production, and better quality. Education-related videos is by far the most commonly produced type of video within higher education. Since the millennium, a considerable percentage of these videos have been recordings of physical lectures; cameras have been placed in the auditorium, the lecturer has worn a microphone, and, in many cases, a separate recording has been made of the images on the projector in the room. This practice has, particularly in recent years, received its fair share of criticism, as recording a physical lecture does not provide any added educational value. This approach achieves nothing beyond retelling what happened in the auditorium, in the exact way it happened, at best. For students, however, this type of video still has merit. Studies (Witthaus & Robinson, 2015) show that students extensively use this type of video when they are studying for an exam, and when they have missed a lecture, or failed to grasp a concept presented during the lecture. For students, therefore, this might have a perceived value in terms of freedom and flexibility. The contrast to lecture recordings is short clips recorded away from the auditorium. These have garnered a lot of attention through the emergence of the “flipped classroom” (https://en.wikipedia.org/wiki/Flipped_classroom), where videos are used to present material to the students before the lecture, whereupon the lecture time is used for discussion. These videos

15

contributed to increased educational value for the students; they get an added value from the lecture as a result. All HE institutions need to inform others about, manage and distribute knowledge from time to time, and video is a suitable format. Typical examples include conference applications, popular science dissemination, and researchers using video to present their findings. Conference and seminar recordings are offered both as a service to those who are unable to attend, and as a collective reminder for those who did attend; they can go back and view something a second time. Popular science lectures are common, and the institutions are hoping these can reach a wider audience. Also, many institutions use video as a tool to educate and inform students and staff about a specific topic, or as an educational tool in ICT training. In other words, video recordings are useful whenever the objective is to inform or present facts.

3.2 Video services In order to meet both simple, user-centred needs and automated large-scale needs, two separate video services were developed in parallel. Developers worked closely with HE organizations and carried out extensive testing, and several solutions and concepts were tried out. Nation-wide services were developed for two services, selected by the working group for video, with an existing user mass in the Norwegian HE sector. Consolidating operations to free up local resources for improved user support and training proved highly cost-effective. Implementation of these services increased throughout the programme period, and is still increasing, both in terms of the number of institutions implementing them, and the quantity of videos produced and downloaded at each institution.

3.2.1 Simple, user-centred functionality: Relay The pilot for Relay was intended as a trial phase with 50–100 users, which could easily be scaled up to include more users if the service became popular. The user mass quickly increased to 1000 users. When the service was put into production, the increase in user mass continued, and we also saw a spike in the number of videos produced—from 10,000 in 2013 to 34,000 in 2016. Figure 3 shows the increase in videos produced per year, and the number of users producing video through the Relay service. The small increase from 2013 to 2014 is attributable to the transition from pilot in 2013 to production from 2014 onward. The service made it simple to record displays and audio, and it was used by lecturers to record lectures and by students/teachers to present a wide range of information on a screen. Feedback from users indicates that simplicity and a strong focus on the presented material are the two most important criteria for success.

16

Figure 3: Video production in higher education (Relay)

3.2.2 Automated, large-scale needs: Mediasite Mediasite is a service that offers automated recordings, which links the recordings to rooms (often auditoriums), and not to the individual teacher. Mediasite had been implemented by several of the participating institutions before the programme launched, and the production data therefore reflect already established local procedures and existing equipment in auditoriums at start-up. Figure 4 shows the number of videos produced per year and the number of viewings per year. As we can see, the figures triple in four years, from less than 20,000 videos produced in 2013, to 67,000 videos produced in 2016. The trend for viewings shows a similar development, from 400,000 viewings in 2013, to 1,400,000 viewings in 2016.

Figure 4: Video production in higher education (Mediasite)

0

500

1000

1500

2000

2500

3000

3500

0

5 000

10 000

15 000

20 000

25 000

30 000

35 000

2013 2014 2015 2016

Use

rs p

rodu

cing

vid

eo

Prod

uced

vid

eos

Relay: Video production

Produced video Users producing video

0

10000

20000

30000

40000

50000

60000

70000

80000

0

200000

400000

600000

800000

1000000

1200000

1400000

1600000

2013 2014 2015 2016

Vide

os p

rodu

ced

View

ed v

ideo

s

Mediasite: views and produced videos

Viewed videos Videos produced

17

3.3 Deployment complete, but full potential not yet reached The eCampus programme eliminated certain technological hurdles, and made it simple to use video as a learning and teaching tool. This means that any remaining hurdles to the implementation of video in education are primarily pedagogical or organizational in nature. Digital literacy among teachers has been emphasized as a major hurdle to all aspects of ICT implementation in education.

18

4 Focus area: eCampus collaboration services In this context, the term collaboration is applied to solutions for learning, teaching, working, studying, reading, exploring or testing together online. Seeing as there are many different collaboration needs, there are also many different solutions and collaboration tools that could potentially be relevant for use in learning and education in the HE sector. The eCampus programme established several joint national services for collaboration, tailored specifically to teaching and learning. At the same time, individual institutions have made institutional collaboration services a priority, which means that the area has seen a boost, both locally and nationally.

4.1 Areas of application for collaboration services Studies on online meetings (Meijer & Melve, 2010) have identified three areas of application for collaboration services:

• Meetings where participants need to: communicate, see the person who is talking, send summons, send and read minutes, chat, share files and share screens

• Teaching situations where participants need to: see the person who is talking, see each other, chat, share files, access learning resources, share in small groups and share screens

• Collaborations where participants need to: communicate, see each other, chat, share files, share work spaces and share screens

The common denominators for all scenarios are that participants need to be able to communicate (audio and chat), see the person who is talking, share files and share screens. For meetings, support for sending and reading summonses and minutes is required, and it is not always necessary for the participants to be able to see all of the other participants. In teaching situations, it is highly beneficial to be able to see all contributors, it is necessary to have access to various resources, and it is useful to be able to split the group into smaller groups for shorter projects. When several people come together to collaborate online, it may sometimes be enough just to be able to share the work space/screen, but usually, support for chat/audio is preferable. Collaborations are the scenarios whose needs differ the most, because the needs may be discipline-specific, or they may vary with the type of collaboration and the habits of the individuals working together. This distinction between meetings, teaching and collaborations provided a good foundation for assessing various collaboration solutions. Many providers have solutions that work well for either the minimum specifications, shared by all scenarios, or the meeting scenario, but very few offer support for teaching-oriented solutions (all participants can see each other, learning resource sharing, and meeting rooms that are always available, with the necessary learning resources).

4.2 Online collaboration services in higher education The first collaboration service established in the eCampus programme was FileSender (or CloudStor, which the service was originally called). At the time, it was extremely difficult to share large files over the Internet, but FileSender made it possible to upload large files and notify the recipient, who then would download the file. Users were suddenly able to easily send videos and collaborate on data. The service required at least one of the parties to log in with a Feide ID, in order to verify users and limit use of the service to the HE sector. Figure 5 shows that the use of FileSender

19

increased steadily until 2015, when better solutions for small files emerged, leading to a reduction in volume. Log analyses show that the service is no longer being used to share small files, whereas the number of large files being shared is still increasing.

Figure 5: Use of FileSender, from pilot in 2010 to production in 2012–2016 The next service in high demand was online meetings. This type of solution heavily overlap with personal video conferencing and unified communications. Unified communications span the entire organization, with extensive tool integration. Personal solutions often rely on the use of several parallel tools, and are usually not coordinated with the organization’s list of users. An assessment based on the three scenarios of application described above concluded that support for teaching was inadequate in the meeting solutions, and a call for tenders was put out for joint procurement of an online meeting solution for education. Adobe Connect won the bid and the solution was implemented in several online courses. Norway was the first country in the world to give students access to setting up their own meeting rooms, and in the first two years, approx. one third of the rooms had students as administrators. The option to collaborate without having to ask teaching staff to set something up for them made it easier for both on-campus students and students taking online courses to work together. Traffic data also show that the service is used for collaborations at night (and on weekends), after all teaching activities at the respective institutions are finished. As shown in Figure 6 below, use per meeting room has increased, even after other services (e.g. Skype for Business) were introduced and implemented for meetings from 2014 onward, and the number of Adobe Connect users has stabilized.

0

500

1 000

1 500

2 000

2 500

3 000

3 500

0

5 000

10 000

15 000

20 000

25 000

30 000

2010 2011 2012 2013 2014 2015 2016

Num

ver

of u

niqu

e up

load

ing

user

s

Upl

oade

d fi

els

for

shar

ing

Filesender

Uploaded files for sharing Unique uploading users

20

Figure 6: Use of Adobe Connect, key figures for use in the HE sector Video-conferencing was a hotchpotch of local agreements, joint procurement of some equipment, some common components and a strong need for a video catalogue. Common components include video bridges and conference call bridges, which are capable of connecting several participants. The video catalogue for the HE sector was updated and maintained by two Norwegian providers. During the programme period, we have seen a strong increase in the use of personal video conferencing through implementation of Skype for Business (formerly known as Lync). Joint national initiatives in the sector focusing on realtime solutions yielded better solutions for interconnecting several meeting rooms, and through the new service Videobro, which launched in late 2016, it is now possible to link both individuals and rooms to the same video conference. The proliferation of tablets and cloud services, increased use of mobile devices, and shadow IT were the backdrop for the final collaboration service established as part of the programme. The need to access files across devices and to share these files with others over the Internet grew exponentially. A joint Nordic call for tenders resulted in a contract with Box for a cloud service for personal file sharing. This service cannot be used for sensitive personal data, classified information or health data, but it has proved useful for common types of information. ICT departments requested a Norwegian data processing agreement. Use is increasing, but not rapidly. Traffic analyses from the research and education network have shown that there is considerable traffic to Dropbox and other, similar services for which no data processing agreement is in place. There are other alternatives, e.g. OneDrive, which offers a Norwegian data processing agreement.

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

0

500

1000

1500

2000

2500

3000

3500

4000

4500

2012 2013 2014 2015 2016

Møt

etim

er

Num

ber

of m

eeti

ng h

osts

and

roo

ms

Web meetings: Adobe Connect

Meeting hosts Meeting rooms Meeting hours

21

Figure 7: Box service, user data 2015–2016

4.3 Development of and preparation for next generation solutions

Development initiatives in this area have been carried out in collaboration with the EU project GÉANT4. Studies into WebRTC technology show that we are on the brink of a massive breakthrough in collaboration solutions (Otto, 2016), and that several of the expected changes are relevant for education

• Services become more user-friendly—available in a browser from the cloud • Easy to add the services as part of a work space, easy to include the services as part of

online services, easy to move data to/from the services • Cloud services are available on a large scale, and they include both specialized

collaboration tools and entire software suites, e.g. Office365 The next generation of solutions should be more user-friendly for the end user, easy to manage for the educational institution, and realized as cloud services. Development initiatives have focused on WebRTC, as this is deemed to be the most likely candidate to make the technological leap required for new solutions.

0

1000

2000

3000

4000

5000

6000

jan.

15

feb.

15

mar

.15

apr.

15

mai

.15

jun.

15

jul.

15

aug.

15

sep.

15

okt.

15

nov.

15

des.

15

jan.

16

feb.

16

mar

.16

apr.

16

mai

.16

jun.

16

jul.

16

aug.

16

sep.

16

okt.

16

nov.

16

des.

16

Box personal storage/sharing: lisenses used

22

5 Focus area: eCampus national ICT infrastructure The development of best practices for ICT implementation in higher education requires clarification of how systems are to be integrated, and which specifications solutions must comply with. Through workshops and working groups, the eCampus programme has developed a shared understanding and documented joint requirements. Joint procurement agreements are key in the standardization of solutions and developing a shared competence for the long term.

5.1 Facilitating for implementation Procurement agreements ensure that equipment and solutions are available. A joint procurement agreement has been established for equipment installed in teaching rooms (AV equipment such as microphones, cameras, video-conferencing equipment, screens, etc.). The AV procurement agreement makes it easier to ensure that joint specifications contribute to cost-efficiency and standardization. In combining documentation of joint requirements, standardized equipment, systematic procurement, and presenting a unified front in communicating with providers, it is possible to achieve far better results in terms of long-term development and maintenance. In designing new buildings, it has been a priority to ensure that new standards for teaching and examination rooms are taken into account. Procurement agreements for various services ensure that joint requirements are met, be they cloud services, joint services or local solutions. Best practices for teaching room equipment have been documented in a series of best practice documents known as UFS:

• Functional description of AV equipment for classrooms and meeting rooms (Støfringsdal, Funksjonsbeskrivelse AV-utstyr for undervisnings- og møterom (UFS116), 2013)

• Technical and functional system requirements for AV equipment (Støfringsdal, Teknisk og funksjonelle systemkrav for AV-utstyr (UFS119), 2013)

• Operational support systems and audiovisual transmission (Støfringsdal, Driftstøttesystem og overføring av lyd og bilde (UFS120), 2011)

In addition to ordinary procurements, development agreements with two providers were established for digital examinations. Development agreements are a relatively new strategy for digitization measures in higher education, and our experiences with both provider collaboration and agreement format have been useful.

5.2 Joint requirements Best practices have been documented for infrastructure and support for the implementation of ICT in education. The UFS series is continually updated and expanded as new procurement agreements require new specifications. In connection with digital examinations, a series of joint specifications have been established for examination rooms, ICT architecture, integration, client device/student computer specifications, and supervision. In addition, it has been imperative to establish clear specifications for the use of cloud services, where it has been necessary to do several rounds with different parties to clarify terms of use, user expectations and relevant regulations. The use of cloud services is increasing, but in term of procurement, uncertainties concerning regulations and terms of use remain an obstacle, as described in the guide to cloud services (UNINETT, 2016). The Box service was used as a case study in the procurement of cloud services,

23

and with flexible providers and help from the information security community, it was possible to broker an agreement. Procurement regulations for services can be quite resource-intensive, which makes it tempting for users to resort to shadow IT, making it easier to implement services. Shadow IT is characterized by users entering the cloud on their own, using individual solutions in their organizational context. Current regulations do not encourage collaboration for the purpose of ensuring compliance with regulations or for customers in the public sector to present a unified front. In connection with the digital examination effort, we found that many of the prospective providers used cloud services, and it was therefore imperative that we establish a best practice for cloud service terms of use. These specifications can, to a considerable degree, be repurposed for other types of services, and have been included in the Ministry of Education and Research’s strategy for a higher education cloud. Joint specifications have been applied both locally and nationally to a much greater extent than the investment roadmap for eCampus services. (UNINETT, 2013)

5.3 Development Digital examinations require digitization of both the work setting for students during the examination itself, and of all other processes related to holding, grading and following up on the examination. eCampus initiatives have mapped these processes—documenting existing practices and establishing joint objectives for digital processes. The overall ICT architecture for digital examinations has been established, and will be taken into account in future procurements. This initiative has attracted considerable international attention, and the documentation of best practice, both for architecture and other examination specifications, has been translated into English. The cost of translation, as well as some development costs, is covered by the EU through the GÉANT project, which ensures implementation of these practices in other European research and education network as well. In the development of a system for digital examinations, developers made sure to establish a joint point of integration to ease integration between the Common Student System (FS) and solutions for digital assessment and ensure a good flow of data. This work was carried out in collaboration with FSAT (now a part of Ceres). The proliferation of tablets and apps also made it possible to easily implement video and collaboration services in everyday activities. Several of the established collaboration services (Box for filesharing, Adobe Connect for online meetings) have dedicated apps for mobile devices. Other services rely on browser user interfaces to ensure compatibility with different types of ICT equipment.

24

6 New focus area: eCampus digital examination One of the biggest changes in the eCampus programme was to elevate the digitization of examinations to the position of a separate focus area. When participants shared experiences related to the implementation of ICT in education, we found that many institutions were working in parallel to digitize examinations. After consulting with the Ministry and the prioritization council, some eCampus resources were allocated to a national digital examination project. The objective for this national project has been to clarify constraints, develop resources and document best practices to prepare for the procurement of solutions.

6.1 Digital assessment An expert panel appointed by the Norwegian Association of Higher Education Institutions (UHR) and the Norwegian Agency for Digital Learning in Higher Education started working on digital assessment in 2013. Their report on regulations and the legal framework for digital examinations (Expert group for digital assessment and examinations (Norwegian Association of Higher Education Institutions og Norwegian Agency for Digital Learning in Higher Education), 2014) identified constraints and relevant regulations for digital examinations. The expert panel revealed the need for technical groundwork, and the eCampus programme changed course to ensure that a national project could effect the necessary change in 2014–2015. The challenge of digital assessment is three-pronged:

1. Let students use word processing and other digital tools during the examination 2. Digitize the examination process, from registration to announcement of grades 3. Ensure that good formats for digital assessment are available

Pen and transfer paper have been the traditional format for all Norwegian students taking written exams under the supervision of invigilators. In take-home examinations, submission has usually be handled by existing LMS platforms. At the beginning of the eCampus programme, there were some pilot projects with a limited scope, looking into new forms or assessment or the possibility of giving students access to word processing tools or specific applications, such as programming environments. Students were also lobbying for access to writing and submitting examination assignments in a digital format. Digital written examinations had top priority. At the end of the programme period, considerable effort had gone into local projects at the respective participating institutions, in addition to the national project. The number of students whose examinations are now digitized varies from institution to institution; in some institutions it may be as few as 10–50 candidates, whereas others may have digitized more than half of all its written examinations. Focused strategies and institution-wide effort yielded great results for some institutions, but the wait-and-see approach adopted by others robbed their students of the opportunity to use good tools for exams.

6.2 National digital examination project 2014–2015 A national project for digital examinations under the direction of the eCampus programme was established as a clear priority in the National Budget of 2014. The objective was to give universities and university colleges access to joint, state-of-the-art digital solutions and tools, and to promote academic and administrative collaboration at home and abroad, both between institutions and between the educational sector and industry.

25

By working together on digital solutions and learning resources, the sector can get better results than if each institution has to fend for itself. It is particularly important to work together on good solutions and frameworks for digital examinations, as this is a new field, where many institutions will have to make the same clarifications at the same time.

6.2.1 Broad collaboration The response from institutions invited to participate in the project was very good, and during the project period a total of 31 institutions participated. The project covered the following main areas:

• Knowledge sharing • Market development and & pilots • Identification of best practice • Infrastructure and administration • Risk and sensitivity analysis • Preparations for procurement

The project was overseen by UNINETT, and participants included all Norwegian universities and most university colleges, as well as FS/FSAT, the Norwegian Agency for Digital Learning in Higher Education, BIBSYS, Universell (the national coordinator for universal design and inclusive learning environments in higher education), and the National Union of Students in Norway (NSO).

Figure 8: Organizational structure for the national digital examination project In total, the project has included approx. 70 volunteers, in various groups and areas of responsibility. The majority of participants were administrative and technical staff, but academic staff were also represented. Semi-annual workshops drew anywhere from 60 to 150 attendees.

26

Figure 9: Log-ins to examination solutions, total, not limited to written examinations

6.2.2 Development contracts and market maturation Over the two-year project period, we have seen a marked maturation in the understanding and knowledge of digitization of the examination process, and what this might entail in terms of scope, complexity and possibilities. We have also experienced a great demand from institutions for information and knowledge sharing. One of the project’s most important and most requested roles has been to serve as a facilitator of knowledge sharing at various levels—between institutions, between local projects, with different providers, or with other stakeholders. The project has attracted a lot of interest internationally, particularly from other Nordic countries and the UK, but also from Slovenia, Spain and the Netherlands. Our experience seminars have had attendees from several Nordic countries, and we also organized a European workshop in collaboration with the British organization JISC. While approaches may vary, the main challenges associated with digital examinations are the same in other countries as well. All countries therefore have a mutual interest in and stand to benefit from knowledge sharing. As part of the project, UNINETT has brokered development agreements with various providers, and several institutions have contributed heavily to testing, experience sharing and solution development. Providers also benefited from this collaboration; the relatively immature market of providers has acquired valuable experience that can be applied in the development of their own solutions. In an effort to initiate testing, the project granted seed funding to providers to develop solutions with FS and Feide integration, requiring recipients to partner with a university or university college in a pilot project.

6.2.3 Best practice documents The project dedicated considerable time and effort to identify specifications and formalize these in UNINETT best practice documents (UFS). These specifications are based on experience and best practices from pilot projects in the sector throughout the project period. The purpose of these best practice documents is to serve as tools in the planning and execution of digital examinations, and the documents cover several areas, such as

• infrastructure and examination rooms UFS145 (UNINETT e. a., 2014) • client administration and computer requirements UFS146 (Strømdal, et al., 2015) • Integration UFS147 (UNINETT, eCampus working group, 2016) • ICT architecture and work processes UFS148 (Melve & Smilden, 2015)

0

100 000

200 000

300 000

400 000

500 000

600 000

700 000

2014 2015 2016

Digital Assessment logins

Spring semester Autumn semester

27

• Logging and monitoring UFS149 (Strømdal & Melve, Krav til logging og overvåkning ved digital vurdering (UFS149), 2015)

• Cloud service specifications UFS150 (National secretariat for information security in the academic sector, UNINETT, 2016).

The best practice documents are intended for different target audiences in the field: managers, technical personnel, and consultants. The best practice documents have been translated into English with support from the GÉANT project for campus best practices, thus making the active best practice documents within the EU (GÉANT Campus Best Practice).

6.2.4 Establishment and management of FS integration In order to facilitate for a flow of information between examination solutions and other systems, a point of integration for digital examinations was established in the spring of 2015. Initially, it was possible to retrieve data from the Common Student System (FS), and the integration has since been expanded to include several user interfaces, making it possible to export grades and other data back to FS. FS has been quite enthusiastic about these developments, and a solid collaboration has been forged. Traffic through the point of integration has increased in step with the increased implementation of digital examinations in the HE sector. Figure 10 below presents the current situation. Please note that these figures represent the number of requests between systems, and not the number of examinations. In that examinations are time-critical, it was necessary to establish high-availability operation in the pilot period, which was continued during regular operation from 01/01/2016. There have been no incidents or down time that affected examinations.

Figure 10: Traffic through point of integration for digital examinations autumn 2016

6.3 Procurement and experience sharing Available best practice documents were applied in the procurement and conversion processes. A joint procurement process involving 20 institutions has developed specifications based on best practice documents for large-scale digital examination solutions. Contracts were signed in the autumn of 2016. These procurement processes take place outside of the eCampus programme and have a time frame of 3–6 years, as considerable developments are expected, in both the market and the technological solutions themselves.

0

20000

40000

60000

80000

100000

120000

140000

160000

Sep Oct Nov Dec

Transactions on integration service 2016

28

Workshops and the exchange of experiences have always been integral to the eCampus programme, and these mechanisms have proven very useful in a phase where many institutions were grappling with the same challenges at the same time. Attendance at experience workshops for digital examinations has increased, from 30 in the pilot project stage, to 60 at launch, and 150 at the final workshop.

29

7 eCampus strategy: Digital learning environment A digital learning environment is comprised of many different components, the most important of which, according to students, is a work space that collects all of the data, services, and tools they need to work on their studies in a practical and efficient manner. This work space has traditionally been provided in the form of an LMS service, with a portal for each class (or subject/course), where the student can find all the information and services he or she needs. A personal learning environment (PLE) is tailored to each student’s needs for information, services, data and ICT solutions (Ministry of Education and Research working group for ICT strategy and comprehensive solutions, 2016). From the institution’s perspective, the digital learning environment is a combination of physical adaptation, services and data. From the point of view of the teacher, the digital learning environment includes the administrative adaptation for the class, content services related to the teaching material, and collaboration solutions for collaborations within the class. The report “Digital State of Affairs 2014” (Norwegian Agency for Digital Learning in Higher Education, 2015) described how the LMS often is widely used for administrative purposes (send messages, find/provide information about the reading list, submit assignments, etc.), but rarely used to share learning material beyond simple lecture notes. Collaboration services have largely been replaced by cloud services or so-called unified communications, integrated with other solutions for collaboration. A report—“Digital learning environment of the future”— (Digital learning environment working group, UNINETT, 2015) written by one of the eCampus working groups documents a shared understanding of needs and outlines how the digital learning environment should develop. This chapter summarizes the philosophies and arguments on which the changes currently being implemented through the joint procurement of new LMS solutions are based.

7.1 From LMS to learning platform In Norway, learning management systems were introduced in higher education in the wake of the Qualify Reform in 2003, and the deployment was quite effective. In just a few short years, most lecturers and students had implemented a LMS in their everyday activities (Li, 2006). Each class had a portal, where they could post information and organize their studies. Even so: learning management systems have been criticized from day one. Some of this criticism can likely be attributed to their widespread implementation in the respective institutions; they have been used by virtually every student and most of the administrative and academic staff in the higher education sector. With this many different user groups, with varied and often intersecting needs, it is extremely challenging to design a system that ticks every box. The systems have been criticized for shortcomings in functionality, closedness, integration, lack of educational benefit, etc. Time and again the death of learning management systems has been proclaimed. One drawback of LMSes is that they have traditionally come with a package of “relevant” technologies. Only a limited number of these technologies can be said to have been truly LMS specific, such as dividing a class into groups, assignment approval, etc., whereas the rest are generic technologies that have been deemed relevant in a learning or teaching perspective. Examples of such technologies include video, collaboration and chat functionality. The learning

30

management systems themselves have been composed of the sum of LMS-specific technologies and the more generic technologies. The crux of the challenges facing LMS providers is that the landscape that is relevant technologies has seen a breakneck development over the past 15 years, while what constitutes relevant technologies could vary from field to field, and from institution to institution. Providers have had to relate to a highly dynamic landscape, and it has been difficult to keep up with the demands of users and customers. The result, in part, has been dissatisfied customers. One might also question whether the technologies offered by LMS providers are the technologies the sector needs, or whether they simply are the technologies the sector is forced to accept, in that they come as part of the package. For example, support for eCampus services has been virtually non-existent in the LMSes used in the Norwegian HE sector, but these tools have been widely used throughout the sector. This has created a “gap” between the LMS and the other systems used in the digital learning environment. The LMSes have been used to post messages and simple files, as well as for some two-way communication, but other key functionality, such as online meetings and video lectures have, at best, been available as a link inside the LMS. Another example of the irrelevance of the tools offered as “part of the package” is the high number of third-party tools used by lecturers and students, including free tools, such as Google Docs or YouTube. These tools have to a very limited degree been integrated in the digital learning environment.

In the spring of 2015, the American association Educause published a report entitled The Next Generation Digital Learning Environment (EDUCAUSE, 2015). Where previous generations of learning management systems have been generic and built on the same rigid format, regardless of application or use by the various organizations implementing them, focus has now shifted, to flexibility, personalization and openness. Educause argues for a future where organizations, departments and individuals build their own digital learning environment, in line with the discourse on personal learning environments (PLEs) in Norwegian ICT strategies, based on the “Lego principle”, where standards allow LMSes and third-party tools to easily integrate in a work space where flexibility, personalization and openness is the norm. The whole point of Legos is that the blocks fit together. They are built around a system that ensures that a Lego block (almost) always will fit together with any other Lego block. If the goal is to connect and disconnect applications and functionality in a similar way, the blocks have to fit together. When they do, it will be possible to build a system, or a platform, that is not just built to control and manage, but that facilitates for better and more flexible learning, a learning platform. In practical terms, one could imagine that an institution has a number of systems in place to facilitate learning processes. These could be video systems or collaboration systems. They may be any digital tool that is relevant for a student or lecturer in what they set out to do. The learning

File sharing

Video

LMS

Online meetings

URL

URL

URL

Figure 11: Current integration between LMS and services

31

platform will offer these tools in a uniform interface for its users. The tools are integrated with the platform through established standards, and presents as part of the platform. Data and rights flow between the learning platform and the tools, in order to give the user as seamless an experience as possible. Whenever the user’s needs change, or the institution modifies the range of tools it is offering, other, new tools can easily be plugged in to the digital learning environment. Whenever tools become obsolete or fall out of use, they can easily be plugged out. Data can be stored in such a way that they, too, can be plugged in or out in the same vein. There are at least two major benefits to building a digital learning environment based on this model. First, it is possible to offer a dynamic digital learning environment with relevant tools available at any time without having to initiate costly procurement processes. Second, the learning platform, or LMS, which is a key part of the digital learning environment, can also be replaced, without having to disrupt other available functionality or tools. It is as simple as swapping one Lego block for another. In the autumn of 2015, UNINETT invited the Norwegian higher education sector to participate in several joint procurements, including an LMS. The specifications of this procurement were largely inspired by Educause’s report, and its parallel, the Norwegian eCampus report “Digital learning environment of the future”, (Digital learning environment working group, UNINETT, 2015) from the spring of 2015. By way of this procurement, the majority of the HE sector in Norway (21 institutions) will move to an architecture where the digital learning environment offered to students and lecturers is flexible and dynamic, and where the system can change in response to new requirements, needs, technologies or pedagogic principles.

Digital learning environment

Video Online meetings

File sharing

Calendar Chat File sharing

New tool

Figure 12: Integration of new tools in modular solutions

32

7.2 Standardized interfaces for modular solutions

We see the digital learning environment as a dynamic work space, where functionality can be removed or added in response to changing organizational requirements and personal needs. This is achieved through Learning Tools Interoperability (LTI) (IMS Global). LTI is a standard used to connect third-party tools to other systems, such as learning platforms or digital examinations systems. If both the learning platform and the third-party tool support LTI, the third-party tool can be connected to the learning platform with minimal configuration work. The nation-wide learning platform procurement process heavily emphasized LTI. In the future, one should strive to ensure that any third-party tools implemented also support LTI, thus moving closer to a “Lego-like” reality, where personalization and learning needs take pride of place.

7.3 Open education resources (OER) Ever since MIT introduced Open CourseWare in 2003, (MIT) open learning resources have been on the agenda in higher education. Open CourseWare (OCW) was, much like today’s MOOCs, a way to make lectures readily available on the Internet. Before this, universities—and especially American universities, which are much more commercial than their European and Norwegian counterparts— generally considered their lectures and learning resources akin to family jewels. MIT’s Open CourseWare caused a tidal wave of openness to wash over higher education. This wave started in the United States, where other universities recognized that OCW, despite MIT’s claims that the system had been developed for entirely altruistic reasons, brought considerable financial gain for MIT. This sparked many similar initiatives across the United States, and the wave eventually travelled across the globe. Differences in university funding between the United States and Europe manifested a different type of solutions, but the focus on openness and sharing took hold. Open CourseWare was never about making teaching freely available, but rather about making the digital learning objects making up the teaching freely available. In MIT’s case, they emphasized that using OCW would not make you an MIT student. This would still require acceptance to MIT, studying with fellow MIT students, and having an MIT professor put the learning objects into context for you. Technological developments and changing demands meant that the sector eventually transitioned from OCW to MOOCs (massive open online courses) (Orr, Rimini, & Damme, 2015). MOOCs are characterized by two-way communication between the institution and students, and between students. The first MOOCs to attract widespread interest were given by Stanford University in 2011

Digital learning environment

Video Online meetings

File sharing

Calendar Chat BIBSYS DLR

LTI

LTI

Figure 13: Integration between digital learning environment and services through LTI

33

(https://en.wikipedia.org/wiki/Massive_open_online_course), which garnered more than 100,000 signups each. MOOCs spread quickly, and Norway was no exception; NTNU and Arne Krokan offered Norway’s first MOOC in 2013. The Official Norwegian Report MOOCs for Norway (NOU 2014:5, 2014) sees MOOCs as one of several instruments to digitizing higher education. Higher education is diversifying, and the interaction between online courses, workshops and ICT implementation on campus means that the digital learning environment is part of the process. Regardless of whether the respective institutions offer MOOCs, open education resources hold considerable potential for the Norwegian HE sector. There are degrees of openness, and there are degrees of sharing. Thousands of digital learning resources are produced in Norway every day. At every single university and university college, lecturers, researchers and students are producing digital resources for others to use as part of their learning process. These learning resources hold considerable potential if they are made searchable, and if they are made available across courses, MOOCs, faculties and institutions. Lecturers and students should have access to relevant learning resources they can use in their courses, regardless of where they are produced, and lecturers and students should be encouraged to share any resources they produce. In order to achieve this goal, systems must be put in place that facilitate for the sharing and reuse of learning resources, such as BIBYS DLR (BIBSYS, 2016). Second, political and organizational policies should promote not only the sharing of any resources we produce, but also the reuse of resources produced by others.

7.4 National services for a digital learning environment As part of the eCampus programme, several national services have been established that fit together with the digital learning environment, much like separate building blocks. These services are primarily directed at eCampus’ primary focus areas: video and collaboration. These services are organized as a cooperative effort, where users share the costs associated with the operation and development of the services. Large-scale operation means it is possible to offer a higher-quality service and to concentrate the knowledge of the systems on which the services are built, which, in itself, promotes quality. The establishment of these services was subsidized by eCampus, which helped reduce procurement costs, but all the services are now part of UNINETT’s standard service offering, financed exclusively by the organizations who use them. By establishing national services, we were also able to establish national standards in the areas each service is intended to cover. We have succeeded in bringing the majority of the HE sector over to a limited number of platforms. Procurement agreements for LMSes and systems for digital

Digital learning environment

Video Online meetings

File sharing

Calendar Chat File sharing

BIBSYS DLR

Figure 14: Digital learning environment based on the interaction of several interchangeable modules, with examples of relevant modules

34

examinations and plagiarism detection will contribute to further consolidation. The outcome will be increased interoperability and easier integration between the various modular systems.

35

8 Snapshots of higher education in Norway today The digitization of higher education is well under way, and through various eCampus initiatives, we have identified several areas where further measures are needed. Developments promoting quality in education, more efficient solutions and sound use of ICT will likely never be completed, but become an ongoing process at the respective institutions.

8.1 Higher education: a functional model In its efforts to map the needs of and ICT use in higher education, the eCampus programme applied a model that has been used by several Norwegian universities and university colleges.

Figure 15 Functional model of higher education This distinction between information, presentation of material, reflection, guidance, and assessment has worked well in terms of highlighting which functions matter to students and academic and administrative staff.

8.2 Mapping the current state of affairs

8.2.1 Digital State of Affairs 2014 Respondents in the report Digital State of Affairs 2014 (Norwegian Agency for Digital Learning in Higher Education, 2015) identify three factors that are crucial for quality:

• Digital tools must be an integrated part of the teaching strategy, and they must feel relevant

• The tools must be accessible and user-friendly

Information

Forum/LMS

Notes, news

Website

Text documents

E-mail

Presentation of material,

lectures

Text documents

Textbooks

Podcast: Audio, MediaSite/Relay,

PowerPoint w/audio

Links

Reflection

Social media

Forum/LMS

Reading room, kitchen table

Online meetings: Skype,

AdobeConnect, video-conferencing

Guidance

Online meetings: Skype,

AdobeConnect,

Forum/LMS

Text

Podcast

Assessment

Examination: written, LMS

Assignments

Survey

Projects

2014

design by Freepik.com

BLOGGwww.ecampus.no

DIGITAL EKSAMENPROSJEKT

TECHSMITH RELAYFORELESNINGSOPPTAK

21 abonnenter

1570 brukere

11000 presentasjoner

449230 visninger

FILESENDERFILOVERFØRING

39 abonnenter

2154 opplastere

24578 delte filer

234GB største fil

BOXPERSONLIG FILDELING & LAGRING

15 abonnenter

2100 lisenser

ADOBE CONNECTWEBMØTER

23 abonnenter

13153 brukere

122271 møtetimer

4238 møterom

MEDIASITEFORELESNINGSOPPTAK

20 abonnenter

9899 presentasjoner

687306 visninger

SUPPORTsupport.ecampus.no

2732 brukere (Feide)

516 tickets via support@ecampus.no

24 læresteder

5 arbeidsgrupper

11 leverandører i dialog

Unike besøkende på support.ecam

pus.no

115

230

0Jan Des

Unike besøkende på w

ww

.ecampus.no

90

180

0Jan Des

106 innlegg

36

• Both students and academic staff must be sufficiently proficient in using the technology The institutions largely rely on dedicated individuals, and the institutionalization of best practices varies considerably. The respective institutions’ management have partly incorporated ICT as a tool in their strategies, but the day-to-day follow-up varies. Initiatives focused on ICT in education are currently undergoing a paradigm shift, from “what” to “why”, whereby tool-specific knowledge is no longer enough; digital literacy now requires knowledge of discipline-specific ICT. Increased demands for quality and efficiency will require a combination of streamlining and innovation. Standardization promotes efficiency. Innovation can take the form of gradual improvements or developmental leaps; gradual improvements can take place on the same platform, whereas developmental leaps often require flexible solutions. Large-scale quality development is a challenge when development initiatives make the move from small pilots to common practice. The report discusses three central topics:

• When pedagogy and teaching take the lead, we can effect change in both teaching and learning practices.

• How can we facilitate for increased flexibility in education from a student perspective? • Joint learning platforms: predictability for students, efficiency for lecturer, predictability

for ICT support. The report’s conclusions establish clear recommendations for UNINETT and eCampus:

UNINETT and eCampus must continue the establishment and development of infrastructure and joint solutions for learning technology in the sector. This work is essential if we are to succeed with the long-term objective of effecting broad digitization in pursuit of educational innovation, quality improvement and accessibility.

Infrastructure and joint solutions for learning technology have been central in eCampus initiatives, and the report gives a clear recommendation to continue these efforts beyond the programme period.

8.2.2 Student priorities The eCampus prioritization council also included students, represented by NSO. The students were very clear on their priorities of areas that needed digitization:

1. Digital examinations 2. Video used in teaching, especially lecture recordings

The students have also suggested various measures to improve their digital learning environment, but their priorities are less consistent in this area. The eCampus programme strived to improve the technological foundation for the digital learning environment by establishing national services, and by creating forums where stakeholders can exchange experiences and establish best practices.

8.2.3 State of affairs in Northern Norway, eCampus report 2012 At the start of the eCampus programme, SINTEF surveyed the digital state of affairs for universities and university colleges in Northern Norway (Finne & Hatling, 2012). The report concluded that critical success criteria were in place for comprehensive promotion of ICT in education, but that it would require considerable practical effort. This practical effort began by mapping out the sector’s

37

needs in the region, conducting interviews at all institutions and building a network of support personnel. This served as the wellspring for the national services to come.

8.2.4 Mapping out the organizational support for digitization in the HE sector In collaboration with eCampus, the Norwegian Agency for Digital Learning in Higher Education appointed the Eastern Norway Research Institute to map the organizations’ support functions for digitization in the HE sector (Kårstein, Stokke, Olsvik, Lundhaug, & Willumsen, 2017). Their report points out that support functions are changing, and the absence of local support may delay the digitization effort. Top management and department management in many HE institutions answered “don’t know” to many of the questions, which indicates that they either do not know enough or interpret things differently, or that things are changing as a result of sector restructuring. The overall impression is that support functions are experiencing considerable uncertainty, and while there is firm support from management, the transition from project to practice change can be challenging. There was considerable uncertainty concerning responsibility for digital competence enhancement, and concerning which groups that should be responsible for the development of digital competence. Digital competence is the key to quality in the implementation of ICT. The digitization effort is still dominated by tool strategies, and quality development seems to have taken a back seat.

8.3 Videos in higher education A video infrastructure has been developed, and use seems to be increasing rapidly. At the start of the programme, eCampus inherited a joint procurement agreement for AV equipment, along with best practice documentation. Best practices, terms and conditions, and agreements have been amended as needed. Investments in AV equipment have increased in the programme period. Figure 16 below shows a model for the use of video in education, and the green area is what falls under the scope of the eCampus programme.

A central technological foundation has been established. Universities and university colleges have expanded their local production, both in teaching rooms and studios, as well as in terms of the number of individuals producing material. Distribution takes place in several different ways, depending on student needs. The respective institutions each manage their own produced material, but on the basis of a joint best practice. Two national services for the production, management and sharing

of video material were established early on in the process. One was a basic service for recording a podcast with screen images and audio, and the other was a solution for automated lecture recordings with full video functionality. To set up the first service, eCampus collected active server licenses in the HE sector and consolidated the operation and administration (and later expansion with additional licenses). For the second service, an agreement for joint server operation was

Media production Distribution

Digital

Organizational foundation

DigitalTechnological foundation

Ownership and usage

competence competence

Lecturer

mobilemeta data markupeditingcapture

auditorium,video conference

and retrieval

conversionstorage and archivelicensingsupport for search

LMS, VLEpodcast, RSSstreaming

Access everywhere

●

●

●

●

●

●

●

●

●

●

●

●

Learning environmentPedagogic support

Classes, courses, student groups

Students

Standards, data formats, meta data formatsEquipment, work flow

Media management

● iTunes, YouTube

Figure 16: Model for the use of video in education. The green technology box only accounts for a small part of the learning environment.

38

established, and existing operation solutions were phased in over time. Both agreements make use of the AV procurement agreement in procuring production equipment for the individual institutions.

8.4 Collaboration (and cloud services) in higher education Students and staff should have the same access to data, services and infrastructure. In order to integrate the students in the academic community, both students and staff should have access to high-quality collaboration tools.

The programme has established solutions for several types of collaboration.

• File sharing, particularly large files (video, measurement data, documents, etc.). This service is also useful for researchers, who need to transfer large quantities of data.

• Online meetings for teaching, meetings, and communication. • Support for conventional video-conferencing, in collaboration with several Norwegian

organizations (list of video rooms, contacts, etc.). • Cloud services for sharing across devices and individuals.

The programme has also worked on establishing an overall architecture for collaboration and cloud services. It has proven quite challenging to define the terms of use for cloud services at a time of considerable uncertainty as to what these terms are and what they should be. In 2015, the European Court of Justice ruled that the regulatory framework of cloud services would have to be replaced due to uncertainties concerning the terms for the international use of cloud services, which illustrates how unsettled the area is, on several levels.

Real-time services is an area where so-called unified communications, which integrates many different forms of communication for an organizational unit, clash with person-centric solutions over which is the best collaboration tool. Both types of systems have advantages, but the systems do not interact well. The field of collaboration is expected to keep changing for at least another technological generation.

8.5 ICT architecture (and digital examinations) In the time that passed between the programme application and the programme’s launch, tablets took the world by storm. Tablets not only represented a new user interface, but also changed people’s expectations; users now expected to find wireless Internet access everywhere. In combination with YouTube, and later also Snapchat, we saw an explosion in the use of video as a form of communication. In the early years of the eCampus programme, digital examinations became a bigger priority, and the programme was redirected to include a national project to coordinate digital examination initiatives. The architectural effort was refocused to solve specific challenges related to the digitization of examinations, and the programme forged a strong collaboration with student administration departments and FS/FSAT. Various solutions for digital examinations were tested and developed in collaboration with providers, and best practices have been documented. The programme identified the need to look at the architecture for digital learning environments as a whole, instead of focusing solely on summative evaluation through digital examinations. The next generation digital learning environment, NGDLE (EDUCAUSE, 2015), has been mentioned, and there is agreement in the sector that we are heading for a generational shift where the technological

39

foundation will be replaced with modular, data-centric solutions. This generational shift will require considerable architectural modifications. This is most evident in terms of the implementation of integrations, but more work is also needed in the documentation of best practices. Long-term projects, such as converting rooms for learning and investing in updates for teaching rooms, depend on established best practices for investments in and use of AV equipment. Changing physical parameters takes time, but much progress has been made in the field over the course of the programme period.

8.6 Discussions at the final eCampus conference At the final eCampus conference, the consensus seemed to be that consistent and systematic quality development is needed. ICT and teaching professionals need to establish better forums for collaboration, and they have to communicate better on how ICT can/should be used. This discourse needs to take place at several levels—at programme level involving teachers, at institution-level, and, in some areas, even at a national level. National collaborations are relevant in areas with similar needs, in areas where specifications must be defined across institutional borders, or in areas where cross-institutional networks of competence add quality. As the eCampus programme is wrapping up at a time of considerable organizational changes in ICT initiatives for the HE sector, it will be necessary to follow up on relevant forums for topics where national collaboration is necessary. One topic that was subject to much debate at the final conference was the interest in a closer collaboration between support personnel and teaching communities in the HE sector. It was argued that teaching staff at universities and university colleges do not communicate well enough with ICT on how ICT can (or should) be used, and what their needs are. Specific topics of discussion included:

• Digital learning environment: How should the digital learning environment be designed: how do we handle the generational shift, how do we work to continually improve the system, how do we best facilitate, what are the needs?

• Various forms of assessment must be evaluated from a pedagogic perspective, and this is related to ICT implementation for digital examinations and learning platforms/LMS. Various forms of assessment and digital assessment: further discourse based on quality development is needed in this area. Neither a recreation of paper-based written examinations nor technical limitations should be allowed to curb this discourse. ICT expertise should be included as one of several stakeholders in any evaluation of the area.

• Mapping the digital competence of staff is necessary to fully utilize carefully planned, facilitated opportunities for good digital learning environments that promote quality in education. Which types of digital competence are needed? Quality development of digital competence will likely require a review of criteria as well as an evaluation of the current state of affairs. To a much greater degree, we need to map which kinds of digital competence the institution’s staff have and need in order to fully utilize carefully planned, facilitated opportunities for good digital learning environments that promote quality in education.

• Cloud application packages: What are the criteria of a good learning environment? How can systems like Office 365 and/or G Suite be used for educational purposes? Co-writing, word processing, spreadsheets, pictures, etc.

• Criteria for integration of solutions: what are the criteria for quality from a teaching and learning perspective, and how should we approach the implementation of integrations?

40

What are the criteria for solutions to work well together? What is important for learning and teaching, and how to we combine components to build a good joint learning environment?

The following national initiatives exerting some influence on quality development have been carried out as part of the eCampus programme, and participants at the final conference in Tromsø prepared recommendations for these national efforts moving forward:

• Documentation of best practice: a comprehensive collection of best practice documents for digital examinations has been developed and should be maintained. The same applies to procurement agreements for AV equipment.

• Experience workshops for digital examinations, the use of videos in teaching, collaboration services, AV equipment specifications.

• Working groups that have looked into specific needs should present their work at experience workshops or as best practices, in order to ensure broad feedback from the entire sector.

• Shared definitions of security, usability, opportunities and needs. • Joint procurement, with joint preparation of specifications, joint evaluation of bids and

joint acceptance testing. • National services have been established, and will be included in UNINETT’s portfolio of

services.

41

9 Lessons from eCampus The Nordic Institute for Studies in Innovation, Research and Education (NIFU) conducted an external evaluation of the eCampus programme, and their results are presented in a report (Tømte, Aanstad, & Løver, 2016). The following is the programme administration’s reflections from the programme’s execution, including important lessons that should be taken into account in any similar efforts in the future.

9.1 Work methodology The eCampus programme opted to use physical workshops, online meetings, blogs and regular dialogue with HE institutions to build a joint forum for digitization. The programme strongly emphasized openness and experience sharing, which made it possible to continually improve the services and build competence.

9.1.1 Working groups The working groups have been the engine driving the eCampus programme forward, and participants report that working with these groups has been useful (Tømte, Aanstad, & Løver, 2016), particularly in terms of knowledge sharing and network building. The working groups brought “like-minded people” together—individuals who fill the same roles and who struggled with the same problems. They created an arena where it was possible to share both problems and successes, where one could get help in solving a problem or let others benefit from one’s success. The challenge with working groups is to get the right individuals to take part. A connection with the HE sector is needed in many more places beyond the communities that the eCampus programme typically have been involved with. A broader participation would strengthen this connection, which will lead to better results through “more accurate services” and more relevant knowledge within the groups. The working groups were open to participants from the entire HE sector, which, in practice, led to the groups being divided in two. The core, specifically those who did most of the work, varied in size, from 3 to 15 people (but usually around 4–7), whereas the “onlookers” used the working groups as a means to observe the process and did not actively participate in the preparation of documents. The core of hard workers is central to progress and knowledge development. The observers are important for quality assurance, competence dissemination and a broad experience base. One downside to working groups is that they may be too non-committal, and in situations where dialogue including institution management is required, working groups are not always a good fit. Such groups are, however, a good tool for operative, practical and technical clarifications.

9.1.2 Best practice Best practices for infrastructure development in the Norwegian HE sector have been formalized as best practice documents from UNINETT, where the quality of the content is assured through working groups and consultations. The document is normally prepared in a working group where institutions that are at this stage of development participate, whereas the consultation round is sector-wide to ensure broad assessment and feedback. The eCampus programme inherited three best practice documents on specifications for AV equipment for use in teaching rooms, which meant that the ground had already been prepared for a joint procurement agreement. In this context, the best practice documents where used both as specifications and as documentation for use locally at each institution. The documents have been updated several times throughout the programme period.

42

A new series of best practice documents were prepared for digital examinations. The document establishing best practices for examination rooms has been especially important for progress, as this is an area where investment costs are quite high. Best practice documents in other areas have proven useful for establishing a shared understanding, serving as a de facto standardization of digitization expectations.

9.1.3 Collaboration with providers, development agreements Collaboration with providers has taken several different forms throughout the programme:

• Community of use for services from major providers, where we as customers have no influence on the design of the service: In this situation it has been imperative to establish a Nordic collaboration in order to become big enough for the providers to want to take error reports and any development requests.

• Joint procurement, with joint specifications based on a shared understanding of needs and best practices in the HE sector.

• Development agreements, both to work iteratively with our understanding of our needs, and to promote maturation of the market: For digital examinations we had development agreements with two providers, where one also included Innovation Norway.

Provider collaboration in combination with development agreements turned out to be the chosen strategy for digital examinations. This is the first time development agreements have been used in the HE sector between many institutions in collaboration with providers. The understanding of how development agreements work varied considerably. Some independent institutions made procurement choices that conflicted with the Confederation of Norwegian Enterprise (NHO)/Agency for Public Management and eGovernment (Difi)’s programme for innovative procurements in the public sector. Through the development agreements and dialogue workshops with providers, we were able to develop functionality that aligned with Norwegian needs and establish integration with the Common Student System (FS). Support for digital work processes was developed in close collaboration with providers. Some institutions did not believe that development agreements could be used to hold actual examinations, and therefore chose to run their own procurement processes over using the development agreements. Close communication with the procurement communities in the HE sector is a critical criterion for success in major ICT investment processes. Balancing close collaboration, innovation and stability can be challenging in a development agreement setting. It takes time to build a shared understanding of needs between HE institutions and providers, which either requires good dialogue over time, or ready-made, standard solutions. When the products on the market are not mature, or when new needs or digitization changes the parameters, it is essential that new forms of collaboration be established. In our experience, development agreements may be a good approach to collaboration, but if development contracts are to be used in joint procurement processes, a shared understanding of the procurement framework is essential.

9.1.4 Large/small institutions Small institutions have short decision-making loops and a straightforward local organization. This makes it easier to decide on the implementation of eCampus solutions. At the same time, small institutions are heavily reliant on individuals, and will not be able to carry out quality assurance for all technological choices made. Over the course of the programme period, and particularly towards the end of the programme, the large institutions have changed their organization of support services for academic implementation

43

of ICT. These large institutions, the four oldest universities in particular, have IT departments with considerable expertise and the drive to tailor solutions to fit local needs. Simplified solutions, joint operation and server consolidation have therefore not always been at the top of the agenda for these IT departments. However, large universities have been key collaborative partners for eCampus by virtue of their expertise and participation.

9.1.5 Cost/benefit assessments Server consolidation has been found to be immediately cost-effective. In some cases, coming together on solutions nationally or internationally gives us the heft to change business models, making joint operation the better choice, such as UNINETT developing support services (for Relay), organizing joint operation nation-wide (for Mediasite), or securing better terms of use for cloud services (for Box). Establishing joint specifications carries an additional cost, and this must be weighed against the overall cost. For cloud services especially, we saw that the cost of establishing joint specifications in areas where few or no such specifications existed was quite high. Joint procurements at a scale of the eCampus NORDUnet procurement had not previously been carried out in this country or anywhere else. The HE cloud strategy builds on these experiences, as does the European collaboration project GÉANT, which facilitates for the exchange of experiences between European research networks.

9.2 Centralized measures, local responsibility

9.2.1 Server consolidation Experiences from eCampus show that considerable economies of scale can be achieved through server consolidation. Savings can be especially great in infrastructure-demanding applications, such as video. Benefits to server consolidation include:

o Operating costs shared by all participants o Limited resources, such as processor time, can be allocated across a larger part of

the day. o Local personnel will have more time to spend on other tasks. o Expertise can be limited to a smaller number of people.

Server consolidation also entails consolidation of procurement processes and evaluations, e.g. of legal aspects or security. At the same time, however, there is no doubt that consolidation limits choice. If we are to succeed in achieving economies of scale, or in maximizing such economies, we have to get as many as possible to agree on a joint solution. This solution will likely not meet everybody’s needs, but it must at least meet a minimum of needs all parties share.

9.2.2 Support portal Early on in the eCampus programme, we recognized that user support for national services could pose a challenge. Some of the services, such as Relay, do not have an “administrative layer”. The end user is normally using the service on his or her own, and there are many end users. When thousands of users potentially need assistance, we have potential scalability issues. The solution to this problem was to establish a support portal to reduce the number of requests, both to UNINETT and to local support resources. The portal makes use of a large number of integrations to offer the right information to the right user (operating status, local support

44

resources, service access, etc.), and soon established itself as a hub for eCampus-related services. The portal was used to make training resources (for use of the tools) available, and to refer end users to their respective support organizations. Initially, the portal also included a forum function, where the idea was to share knowledge across organizations, similar to the approach we saw in the working groups. As is the case for many of these forums, however, only the most dedicated people used them, which meant the activity there was limited. Therefore, the decision was made to close the forum.

9.2.3 Things that took longer than anticipated Rolling out new technology is easy, provided you have enough money and people. Getting people to use new technology is not easy, especially in academic organizations, where autonomy reigns. For eCampus, this mechanic meant that the deployment of services to the respective institutions was a quick and relatively straightforward process. The majority of HE institutions subscribe to the services, but the organizational effort to implement the services in learning and teaching processes has been a much slower process. The tools have either not found their way into the hands of end users, or the end users have made the choice not to use them. Direct contact with academic staff was far outside the scope of the eCampus programme. The programme has communicated mostly with technical, administrative or academic support personnel. Consequently, we have not been in a position to influence end users to any significant degree; we have had to rely on the ability and drive of local communities to get their respective organizations to implement the services. The deployment of video services is a good example. These services quickly got a large number of subscribers (cf. Figure 3 and Figure 4), which would indicate that the services were a success. While the services were popular, statistics show that local potential is heavily under-utilized. Organizational implementation was much greater where users received active local support. The tools did not find their way into the hands of end users on their own. Use of these services has increased considerably recently, however, seemingly without any specific initiatives on the part of eCampus. The likely explanation for this is that local politics and local needs have “caught up” with the technological solutions offered.

9.3 Quality development and a shared understanding of needs All changes in practice for ICT implementation takes place locally at universities and university colleges, whereas central measures facilitate. Quality development is a local responsibility, but central measures can offer support. In eCampus, a conscious decision was made early on to focus on deployment to a majority of users, without paying much attention to explorers or sceptics, as defined by the deployment model in Figure 2. Students and staff would get access to the same services.

9.3.1 Different levels of technological maturity The three main focus areas of the eCampus programme have different levels of technological maturity. Video infrastructure is relatively well understood, and there are several different solutions to choose from. The primary challenge with video is how to use it to improve learning outcomes. Collaboration services are a bit more challenging, as the field is in the middle of a generational shift, but in this area, too, the primary challenge is to facilitate for collaboration in a way that promotes better learning outcomes. In both these areas, we have to find a way to make better use of existing possibilities. As for digital examinations, there was surprisingly little debate

45

about what assessment should be like, and a surprisingly great willingness to recreate written examinations in a digital context, without considering which forms of assessment are the best. Learning platforms have traditionally been silos, as discussed in chapter 7, and the transition to the next generation of solutions means we have to consider which functions to support. Establishing a shared understanding of needs is no easy feat when teaching practices are so person-centric, and each teacher is relatively autonomous. We see that systematic efforts at institution-level, both in terms of study programme development and in terms of digitization, affect the demands for technology as well as the practical applications of said technology.

9.3.2 Timing: hitting the mark Universities and university colleges often make different choices in their digitization strategies, as well as in the areas they choose to focus on. If an institution is heavily invested in video implementation, its progress in terms of digital examinations may be poor, or vice versa. Joint solutions are often based on a certain minimum of needs all participants share, sometimes with additional options for advanced users. For institutions that are ahead of the game, joint solutions may therefore be too little, too late. For some institutions, we also see that the functionality offered by a joint solution will be insufficient for their specific needs. Through the Nordic partnership in NORDUnet, we have been able to offer several different types of video solutions, even if the national solution did not have the full functionality some participants wanted. When many parties work together to establish specification it takes time to arrive at a conclusion, and some institutions prefer to move at a faster pace. In the area of digital examinations, this discussion surfaced, combined with procurement-related arguments, in connection with the use of development agreements. Lacking a joint basis for decision-making, the programme has consulted with the prioritization council on all decisions. This approach of including a prioritization council makes it difficult to make binding decisions, and specifications must be formalized in separate agreements. We should define forms of collaboration in digitization development agreements, so that institutions are not forced into disadvantageous types of agreements before the necessary development and specifications are in place.

9.3.3 Collaboration with other national agencies The eCampus programme has both enjoyed and benefited from working with the Norwegian Agency for Digital Learning in Higher Education, which has a long history of supporting development initiatives and pilots related to ICT in higher education. Their network of connections and meeting grounds have been invaluable for engaging the visionaries in dialogue, as well as for identifying development initiatives that deserve to be developed into national services. The Agency’s focus on teaching also complemented UNINETT’s technological expertise. On the technological side, the programme benefited from working with FSAT’s FS team and BIBSYS. BIBSYS has been a strong partner in infrastructure operation and development initiatives related to data management in education. In the digital examination area, it has been critical to maintain a good collaboration with FSAT on the use of FS data, to make sure that student administrative needs can be met within FS, eliminating the need to build parallel systems. These three agencies approached the eCampus programme with goodwill, expertise and dedication which ensured a successful execution of the programme.

46

9.3.4 Different approaches to support function organization The Eastern Norway Research Institute mapped and analyzed the organizational approach of HE institutions to digitization in education, and their report (Kårstein, Stokke, Olsvik, Lundhaug, & Willumsen, 2017) shows that the organization of support functions varies considerably. The Institute’s findings indicate that the management of digitization in education is fragmented and opaque. These findings correspond well with experiences from the eCampus programme, as well as findings from the evaluation report (Tømte, Aanstad, & Løver, 2016). In the beginning, we assumed that centralized measures would be received by a local team at each institution. We soon learned that the organization of ICT support in education varies considerably from institution to institution. During the initial stage in Northern Norway, which involved 7 institutions, the Association of Higher Education Institutions in Northern Norway took action and established a dedicated ICT working group for education, which included representatives from all institutions. This eCampus North group was a big help in terms of communication with institutions and connecting with the communities. When it came time to include the whole country, the complexity of local organizational structures impeded collaborations and the establishment of specifications. It will be interesting to see whether the recent mergers in the sector will have any levelling effect on this terrain at all. The evaluation report (Tømte, Aanstad, & Løver, 2016) recommends mapping all support functions and active digitization projects once the mergers are completed.

9.3.5 Local eCampus projects The eCampus programme has built national services, but the respective institutions have been responsible for local deployment of these services. Some institutions established local eCampus projects to promote digitization in education. These initiatives were valuable in terms of collaboration, but they were, however, highly reliant on individuals given their small size and varying follow-up from local management. Systematic efforts, collaboration with the line organization and dedicated focus from the rector/pro-rector are critical factors for success. Many institutions have relied on dedicated individuals and their burning passion, as described in the Digital State of Affairs reports (Norwegian Agency for Digital Learning in Higher Education, 2015), without taking steps to effect broad organizational change. The programme has needed local initiatives to go the extra mile to ensure that the services find their way into the hands of teachers and students. When local initiatives vary, the rate of implementation for services deployed to the institutions also varies.

9.4 What do national strategies for ICT in higher education need?

If we were to summarize the lessons from the eCampus programme in terms of execution, we would see that similar initiatives would benefit greatly from:

• Taking the time to establish a shared understanding of the undertaking with the largest institutions, and then select a smaller group to present recommendations for the rest of the sector.

• Establish a strong link to quality development, both to ensure that the right problems are solved at the right time, and to ensure continuous development.

47

• Initiate targeted projects to execute the development of infrastructure, and facilitate for a good collaboration with providers. Infrastructure development requires considerable technological expertise.

• Establish an arena for digitization, with both physical and online meeting grounds, including both institutions and other national agencies.

• Open working groups with a secretariat to monitor the preparation of written documentation.

• Documentation of best practice in joint documents, preferably from working groups. The most important lesson is that any initiative of a similar nature should include and involve the HE sector, and not be established as a project parallel to or outside the sector. Collaboration with all those who participated in working groups and in workshops was invaluable and led to a permanent competence change.

48

10 Vision The proposed ICT strategy for higher education establishes that A comprehensive digitization of education places exacting demands on institutional management, the development of new educational competence, and facilitation of user-friendly tools, among other things. Norway must also invest in a world-class national infrastructure for education. eCampus facilitated tools and developed infrastructure, and to some degree engaged in building competence through the documentation of best practices and network building. The programme’s approach to the field is illustrated in Figure 17, where a joint framework has been emphasized as an agent for infrastructure development across many independent organizations. Quality development requires both digital competence and discipline-specific competence. Procurement requires both a dedication to quality development and knowledge about tools. Integration between various tools in the infrastructure is necessary, and must be based on a clear understanding of the architecture. Coordination requires an ability to see the big picture, both for the individual elements and the overall objective that digitization should simplify, improve and revitalize.

Figure 17: Vision for joint strategy for the use of ICT in higher education

Tools•Video infrastructure•Collaboration•Learning environment•Assessment

Framework•Technical

organization•Modular•Standards

Quality•Digital skills•Human capital•Network of

people

Coordination

Procurement

Integration

Best practice

49

10.1 Trends

10.1.1 International trends There is a strong digitization trend in higher education across the globe. Two central trends have been the discourse on online education through MOOCs (NOU 2014:5, 2014) and streamlining driven by cost reduction. In terms of digital examinations, the Norwegian HE sector overall is reasonably well placed, but some communities in other countries are ahead of the pack. Nordic, British, Dutch and American communities are considered most relevant for collaboration initiatives. Market, cloud service and quality developments in connection with the next generation of digital learning environments (EDUCAUSE, 2015) point to the need for a generational shift in digital learning environments. At the same time, there is increased demand for automated support processes, more self service, and student active learning. Adaptive learning (NMC Horizon, 2016) and learning analytics are new technologies that may improve study quality, and these are under-utilized in Norway, even though SLATE (Centre for the Science of Learning & Technology) (UiB) was established in early 2016. The use of video is increasing rapidly among young people, from 18 minutes in 2010 to 47 minutes in 2015 for the 16–19 age group (MedieNorge, 2016), but this increase has not been as marked in higher education. We are expecting to see a development in video use over the next few years. As a result of video becoming a commonplace tool, we expect the use to increase in an educational setting as well. At this point, the greatest impediment to increased use seems to be digital competence, in Norway as well as in other countries (NMC Horizon, 2016).

10.1.2 MOOCs to Norway The MOOC report (NOU 2014:5, 2014) recommended a wide range of measures. The majority of these were related to quality development: digital competence, knowledge development, better incentives for education, and testing in specific areas, such as learning analytics and digital assessment. One of these measures was related to tools, recommending the establishment of a joint portal for Norwegian MOOCs. Several measures also related to a joint framework for digital assessment, internationalization, industry collaboration, open education resources, and universal design of a national MOOC portal. What all of these recommendations had in common was that on-campus learning melds with online resources and activities, increasing demands on accessibility and flexibility. Digitization is a tool that contributes to the change emphasized by the MOOC report.

10.2 Objectives from ICT strategy Given the vision for an ICT strategy for the HE sector (Dæhlen, 2016) and experiences from the eCampus programme, we propose the following objectives.

1) Students and staff shall have the same access to data, services and infrastructure. Students should be included in the academic community to a much greater degree, and this influences the IT solutions offered. Every teacher and student should be able to access his or her data and services regardless of time or place. They should be free to choose the services they want and the context for when to use them. They should be able to easily share data with their class or group.

50

It is necessary to clarify terms, licences, technical possibilities and the use of cloud services, and this should be a central focus of any joint quality development initiative. Documented best practices should be applied to selected solutions. International collaboration must be possible, given increasing demands of internationalization in higher education.

2) Teachers and students need an “eCampus” that supports activating and varied teaching methods. Locally, the eCampus, much like the physical structures on campus, should be part of the available infrastructure and offer a high-quality digital learning environment. Varied teaching methods require further development of ICT support for video, collaboration solutions and assessment. Interaction with business and industry and internship follow-up should be better utilized through digital solutions. Shared labs and research communities should be made available online. Open education resources should be shared and reused, and it must be easy to publish such open education resources for both teachers and students. Joint procurement of solutions ensures equal access across institutions and study programmes. Joint administration and development of video infrastructure ensure full access to video services for everyone. Joint administration and development of collaboration services ensure full access to everyone, at any time and from anywhere. Documentation of best practices establishes usage policies. Sector-wide solutions for sharing and reuse of open education resources will promote increased use, for both consumers and producers of such resources.

3) A quality culture requires systematic effort and a successful transition from free-standing projects whose progress depend on dedicated individuals, to everyday practice supported by management. Digital assessment is an area that is currently making the transition from project to practice, but success still requires further development of tools, quality and architecture. Transitioning from the current siloed learning management systems to modular solutions for digital learning environments is another area with a similar need for joint solutions on the tool side and unified quality parameters for quality development initiatives. Experience workshops, joint procurement agreements and best practice documents all contribute to a systematic approach and constitute a sound foundation for decision-making in local initiatives at each institution.

4) Cross-institutional collaboration should be possible, both nationally and internationally. Nomenclature standardization is essential for effective cross-institutional collaboration, and it requires that we work on the information architecture. Standardization should be carried out wherever it is relevant. We must establish decision-making structures that ensure effective execution. We need to identify common data sources, in line with the work already carried out for FS/archives/item pools for digital examinations, and establish/manage joint integrations. System silos that are only available to one organization impede collaboration and should be avoided.

5) Steps to better utilization of data: learning analytics, adaptive learning and open education resources.

51

The ICT strategy points to three areas where initiatives are needed, and all three of these are related to data and the use of data in education. Learning analytics is about collecting and analyzing data about the learning environment. Adaptive learning is about utilizing learning analytics to tailor the education to the needs of the individual student. Open education resources have the potential for greater transparency in both content and learning processes, making it possible to better utilize the available data.

10.3 eCampus lessons and vision At the launch of the eCampus programme, its strategic objectives were defined as follows:

• Make simple and good ICT solutions that support learning available on a large scale. eCampus will promote user-driven innovation through good examples, and will make it possible to make lectures available on the Internet on a national scale.

• Contribute to the development of digital competence in academic communities and institution management, and promote good practices for ICT use in education and research. ICT architecture must be followed by digital competence throughout the organization.

• Contribute to making ICT in education commonplace. eCampus will promote the use of tools and put these in context, in interaction with educational and organizational processes.

• Take definitive action nation-wide to promote quality in and collaboration for ICT implementation. National solutions must work with local ICT support.

10.3.1 Lessons Simple and good ICT solutions have been established for video, collaboration and digital examinations. Solutions and tools should continually be simplified, improved and renewed. This work must continue even though the development phase is completed. Digital competence is an area that is entirely dependent on local initiatives. In the area of digital examinations, the work we started should be developed further to also include digitization of formative assessment. Experience workshops and the work to document best practices should continue. The generalization of ICT in education is a necessary consequence of the need for change in educational processes and technological changes in the wake of the proliferation of cloud services. The Norwegian Cloud Initiative for Higher Education (UNINETT) follows up on the technological needs. The need for change in educational processes was discussed in the proposed ICT strategy for education (Ministry of Education and Research working group for ICT strategy and comprehensive solutions, 2016). National progress has been made in several areas, including video infrastructure and digital examinations. Interaction between solutions is another area where important progress has been made through the establishment of joint best practices, but this is also an area where a lot of work still remains. Needs related to modularization, adaptive learning and learning analytics means more work is needed in areas like architecture and integrations.

52

10.4 Vision for tools A digital learning environment requires tools in a number of areas. As discussed in the Digital State of Affairs 2014 (Norwegian Agency for Digital Learning in Higher Education, 2015), a big part of it is facilitating for greater flexibility in education from a student perspective. The basic package for teaching-related tools should include

• Good work spaces for students and teachers • Solutions for video and video implementation • Solutions for sharing and collaboration • Access to data and infrastructure

In cases where the needs are the same across the sector, national services and solutions work well. In addition, it is important to better utilize the potential of cloud services and other services that simplify the process of implementing discipline-specific ICT. Joint specifications for cloud service terms of use are a central first step. Further work on discipline-specific ICT use should be linked to digital competence initiatives. We must develop mechanisms that handle quality development and register changed needs, and procurement processes that meet shared needs across the sector.

10.5 Vision for quality development When pedagogy and teaching take the lead, we can effect change in both teaching and learning practices. Quality development is a critical requirement for any ICT solution that is going to be used in education. Joint quality development is beneficial for more effective competence development, in the establishment of best practices, and for streamlining both local and sector-wide practices. Quality development initiatives for ICT should give the HE sector

• The knowledge to choose (renew) • The competence to implement (simplify) • The competence to change (improve)

We do not want stagnation, we want continuous progress—always looking for ways to improve. In this endeavour, ICT is critical, both for basic digitization needs and for the everyday improvements to new learning processes. ICT use in higher education must relate to and support local and national quality criteria. Today, the exchange of experiences is largely informal, and best practices should be formalized.

10.6 Vision for a joint framework A joint framework for ICT use in higher education would include technological organization, architecture and technological consequences of comprehensive guidelines. Technological organization is about how ICT solutions interact, and covers ICT architecture, information flow, types of data, collaboration interfaces, modularization and deciding when standardization would be beneficial. The ICT architecture should be transparent enough to ensure that good choices are made in the implementation, use and change of ICT solutions. We need to map dataflows, the market situation for technology, and integration needs. Requirements for joint learning platforms include predictability for students, efficiency for lecturers, and predictability for ICT support.

53

Areas that will require particular attention: • Reduce complexity by uniform administration, controlling the system portfolio, harmonizing

similar processes and uniform/documented information sources. • The right information at the right time in the right process: define the need for shared

concepts, define these shared concepts and how they relate to each other, make data available

• Transition to more modular systems—increased needs for integrations related to digital learning environments and digital assessment

• Cloud services and a different approach to ICT solution delivery • Standardization of data and processes to increase support for automation, reuse and

personalized flexibility. • Open controlled interfaces for data access, so that data can reside with their master data

source and be reused in various learning environments • Adaptations in the three areas identified as strategic areas in the ICT strategy for higher

education o tools for adaptive learning o solutions for digital assessment o open education resources

54

11 The road ahead

11.1 Shared vision The measures recommended to help make the vision a reality have been categorized according to each of the eCampus strategic objectives:

• Accessible, simple and good ICT solutions (continue development of established solutions) • Digital competence at all levels of the organization • Distinct national progress followed up by local initiatives • Generalizing ICT in education: change teaching processes and increase education quality

11.2 Digitization prerequisite in quality development

11.2.1 Objectives Guidelines for quality development and ICT use:

• A quality culture requires systematic effort and a successful transition from free-standing projects whose progress depend on dedicated individuals, to everyday practice supported by management.

• Steps to better utilization of data: learning analytics, adaptive learning and open education resources.

• Digital competence at all levels of the organization: Both students and academic and administrative staff are digitally literate

• Arenas for exchanging experiences that document both best practices and achievements, with close ties to quality development in higher education

11.2.2 Digitization arena eCampus established a digitization arena for ICT in higher education. These networks should be maintained through working groups and workshops. Our focus should shift, from the development of infrastructure to improving, simplifying and renewing for quality development. This entails more local initiatives at each institution and close follow-up from institution and faculty management. Digital illiteracy has been identified as the biggest, and most difficult to overcome, impediment to better digital education, both in this country (Norwegian Agency for Digital Learning in Higher Education, 2015) and in other countries (NMC Horizon, 2016). In Digital Agenda (Norwegian Government, 2016), the Government promotes digital management competence, and this initiative is transferable to education. Local quality development initiatives in the HE sector must have a clear ICT profile, including productive meeting grounds for academic staff and ICT support personnel. The respective institutions must use active digital management and good incentives to take ICT from the project stage to institutionalized practice.

11.3 Tools for higher education

11.3.1 Objectives Guidelines for tools and ICT use:

• Students and staff shall have the same access to data, services and infrastructure. • Steps to better utilization of data: learning analytics, adaptive learning and open

education resources.

55

• Teachers and students need an “eCampus” that supports activating and varied teaching methods.

o Joint specifications for AV equipment intended for teaching o Video infrastructure o Collaboration tools o Digital assessment o Digital learning environment

11.3.2 Local support and national services The national services developed as part of the eCampus programme continue as a cooperative effort, where participating institutions share the cost. Initiatives involving joint procurements of learning platforms and digital examination solutions will serve as natural hubs for joint efforts into good work spaces and technological solutions for teaching-related services. In order to make better use of data, we need developments in the areas of learning analytics, adaptive learning, modularization and sharing of education resources. We recommend using development contracts with providers, preferably in combination with a national arena for development of education-related ICT. Large-scale practical applications at institution-level requires local support and integration with quality development work for education. In order to ensure transparency and good cross-institutional collaboration, we should map the organizational approach of participating institutions now that the structural changes, including mergers, in the sector are completed.

11.4 Joint framework for ICT use in higher education

11.4.1 Objectives Guidelines for technological organization and ICT use:

• Steps to better utilization of data: learning analytics, adaptive learning and open education resources.

• Cross-institutional collaboration should be possible, both nationally and internationally. • Students and staff shall have the same access to data, services and infrastructure. • Teachers and students need an “eCampus” that supports activating and varied teaching

methods, promotes academic flexibility, and supports organizational changes. • A joint description and understanding of ICT solution architecture. • Facilitate for automated processes through standardized interfaces and simplified

functionality. • Modular solutions where components are combined to form a uniform digital learning

environment.

11.4.2 Better use of data: specifications and progress A joint basis for decision-making in national initiatives should be based on the work carried out in connection with the establishment of a joint ICT strategy. Today there is no section-wide basis for decision-making in the HE sector. Financing for the joint architecture initiative is not yet in place. The transition to modular solutions should be coordinated by a unit with insight into the solutions currently in use, and control of data flowing between the different modules.

56

The recommendations from the proposed ICT strategy for higher education related to digital assessment and adaptive learning must be actively addressed to ensure that they are handled well. The level of maturity for both digital assessment and learning platforms should be increased, and joint development initiatives should be evaluated and tied to a joint decision-making structure. ICT architecture is a developing field, and the HE sector’s competence varies, as does its investments. Several large institutions (BOTT partnership, BI Norwegian Business School) are spearheading developments, whereas others are simply consumers of results of the joint efforts. ICT architecture is not limited to students and academic staff; it is necessary to see the big picture. Work in this area is critical for handling successful transitions for solutions for digital assessment, digital learning environments, cloud services and large-scale video use.

11.5 Ongoing initiatives A lot of good initiatives have been launched, and new infrastructure has been developed. We have established an arena for digitization that serves as an excellent starting point for further efforts to improve the use of ICT in higher education. Quality development locally will require improved implementation of ICT for education at every institution, every faculty, every programme of study and in every course. Digital competence is a key factor for good progress in this area. Joint specifications for infrastructure, guidelines and best practices are critical for success. Collaborations to establish best practices is an effective tool to ensure progress and a shared understanding of the challenges at hand. The joint effort to promote ICT in higher education has started, and the work continues at several levels. Let’s continue the good work and pull together.

57

12 References Arbeidsgruppa for digitalt læringsmiljø, UNINETT. (2015). Framtidens digitale læringsmiljø.

Retrieved from https://uninett.box.com/s/q6jag9s1ue9y8zbujymxzjxb0de2ww5s BIBSYS. (2016). DLR tjeneste. Retrieved from http://www.bibsys.no/produkter-

tjenester/produkter/digitale-laeringsressurser-dlr/ Dæhlen, M. (2016). IKT-strategi og helhetlige løsninger i norsk universitets- og høgskolesektor.

Arbeidsgruppen for IKT-strategi og helhetlige løsninger. EDUCAUSE. (2015). The Next Generation Digital Learning Environment. Retrieved from

https://net.educause.edu/ir/library/pdf/eli3035.pdf Ekspertgruppen for Digital vurdering og eksamen (UHR og Norgesuniversitetet). (2014). Digital

vurdering og eksamen - en juridisk vurdering. Retrieved from https://norgesuniversitetet.no/files/attachment/2830/digital-vurdering-eksamen-juridisk-versjon1.pdf

Finne, H., & Hatling, M. (2012). Hva kan bidra til bedre IT-støtte for fleksibel undervisning? Retrieved from https://www.ecampus.no/wp-uploads/2013/12/sintef-refleksjonsnotat1.pdf

GÉANT Campus Best Practice. (n.d.). Retrieved from http://services.geant.net/cbp/Pages/Home.aspx

Gjedrem, & Fagernæs. (2016). Kunnskapssektoren sett utenfra. Retrieved from https://www.regjeringen.no/no/dokumenter/kunnskapssektoren-sett-utenfra/id2469493/

IMS Global. (n.d.). Learning Tols Interoperability. Retrieved 2016, from https://www.imsglobal.org/activity/learning-tools-interoperability

Kårstein, A., Stokke, M., Olsvik, V., Lundhaug, K., & Willumsen, T. (2017). Kartlegging og analyse av UH-institusjonenes organisering for digitalisering av utdanningene. Østlandsforskning.

KDs arbeidsgruppe for IKT-strategi og helhetlige løsninger. (2016). Forslag til IKT-strategi for utdanning. KDs arbeidsgruppe for IKT-strategi og helhetlige løsninger.

Li, J. (2006). Læringsteknologi i norsk høgre utdanning. Norgesuniversitetet. MedieNorge. (2016). Internettbruk etter type bruk. Retrieved from Norsk Mediebarometer:

http://www.medienorge.uib.no/statistikk/aspekt/tilgang-og-bruk/329 Meijer, J., & Melve, I. (2010). Web conference discussion memo. Retrieved from eCampus

webmøter: https://openwiki.uninett.no/_media/ecampusnorge:ecampus-webconferencing-memo-09.pdf

Melve, I., & Smilden, B. (2015). IKT-arkitektur for digital vurdering (UFS148). Retrieved from https://www.uninett.no/sites/default/files/ufs148-digg-utkast091.pdf

MIT. (n.d.). MIT OpenCourseWare. Retrieved from http://ocw.mit.edu/index.htm NMC Horizon. (2016). NMC Horizon Report > 2016 Higher Education Edition. Retrieved from

http://www.nmc.org/publication/nmc-horizon-report-2016-higher-education-edition/ Norgesuniversitetet. (2010). Kunsten å ile langsomt. Retrieved from

https://norgesuniversitetet.no/skriftserie/2-2010-kunsten-a-ile-langsomt-ikt Norgesuniversitetet. (2015). Digital tilstand 2014. Retrieved from

https://norgesuniversitetet.no/file/3658/download?token=pU0rwnQK NOU 2014:5. (2014). MOOC til Norge— Nye digitale læringsformer i høyere utdanning. Orr, D., Rimini, M., & Damme, D. v. (2015). Open Educational Resources: A Catalyst for Innovation,

Educational Research and Innovation. Retrieved from http://www.oecd.org/innovation/open-educational-resources-9789264247543-en.htm

Otto, S. (2016). WebRTC in UC - an Architecture Overview.

58

Regjeringen. (2014). Strukturmeldingen, Meld. St. 18 (2014-2015) Konsentrasjon for kvalitet – Strukturreform i universitets- og høgskolesektoren. Retrieved from https://www.regjeringen.no/no/dokumenter/meld.-st.-18-2014-2015/id2402377/

Regjeringen. (2016). Digital Agenda: NOU 27 (2015-2016). Retrieved from Regjeringen: https://www.regjeringen.no/no/dokumenter/meld.-st.-27-20152016/id2483795/

Sekretariat for informasjonssikkerhet i UH-sektoren, UNINETT. (2016). Krav til bruk av skytjenester (UFS150). Retrieved from https://www.uninett.no/sites/default/files/ufs-150-krav-til-til-bruk-av-skytjenester_oppdatert_juni_2016.pdf

Støfringsdal, B. (2011). Driftstøttesystem og overføring av lyd og bilde (UFS120). Retrieved from https://www.uninett.no/sites/drupal.uninett.no.uninett/files/webfm/_Produkter%20og%20tjenester/campustjenester/@campus/UFS/pdf/ufs120.pdf

Støfringsdal, B. (2013). Funksjonsbeskrivelse AV-utstyr for undervisnings- og møterom (UFS116). Retrieved from https://www.uninett.no/webfm_send/779

Støfringsdal, B. (2013). Teknisk og funksjonelle systemkrav for AV-utstyr (UFS119). Retrieved from https://www.uninett.no/webfm_send/780

Strømdal, M., & Melve, I. (2015). Krav til logging og overvåkning ved digital vurdering (UFS149). Retrieved from https://www.uninett.no/sites/default/files/portal_docs/ufs-149-digital-eksamen-logging-og-overvakning-v1.0.pdf

Strømdal, M., Holst, K., Johansen, V., Langfeldt, O., Melve, I., & Knarlag, K. (2015). Klienter for digital eksamen (UFS146). Retrieved from https://www.uninett.no/sites/default/files/UFS-146-klienter-for-digital-eksamen-v1.0.pdf

Tømte, C., Aanstad, S., & Løver, N. (2016). Evaluering av eCampus-programmet (NIFU rapport 2016:44). NIFU. Oslo: NIFU.

UiB. (n.d.). SLATE. Retrieved from SLATE, a global research centre for the advancement of the Learning Sciences: http://www.slate.uib.no/

UNINETT. (2013). eCampus - investeringsveikart for universiteter og høgskoler. UNINETT. (2013). eCampus strategi 2012-2016. UNINETT. (2013-2016). eCampus support-portal. Retrieved from eCampus support-portal:

https://support.ecampus.no UNINETT. (2016). Retrieved from Juridisk veileder for skytjenester:

https://www.uninett.no/skytjenester/juridisk-veileder-skytjenester UNINETT. (n.d.). UH-sky. Retrieved 2016, from https://www.uninett.no/skytjenester UNINETT, e. a. (2014). Fysisk infrastruktur for digital eksamen (UFS145). Retrieved from

https://www.uninett.no/sites/default/files/UFS_fysisk_infrastruktur_digital_eksamen.pdf UNINETT, eCampus arbeidsgruppe. (2016). Integrasjoner digital eksamen (UFS147). Retrieved from

https://www.uninett.no/sites/default/files/portal_docs/ufs-147-digital-eksamen-integrasjon.pdf

Witthaus, G., & Robinson, C. (2015). Lecture Capture Literature Review. Centre for Academic Practice, Loughborough University.

59

Appendix 1: Digitization initiatives

This appendix describes recommendations for specific initiatives to promote digitization in higher education.

Digitization prerequisite in quality development

Local measures As described in chapter 11, digital illiteracy has been identified as the biggest, and most important, impediment to better digital education. We recommend the following measures:

• The current state of affairs in digital competence must be mapped • Quality criteria for digital competence must be developed • The digital competence of academic staff must be strengthened • The digital competence of students must be strengthened • Digital competence must be strengthened in education management, so that the respective

institutions’ management are able to operate in compliance with the vision. In order to successfully achieve these changes, it will be necessary to map the current state of affairs for digital competence in participating organizations, and to establish unified quality criteria. Without a clear idea of the current status or any way to measure competence, it will not be possible to evaluate whether we have successfully effected the change. Once first-versjon quality criteria have been established, institutions should implement specific measures, followed by an evaluation.

National measures We recommend the following measures:

• Develop quality criteria for digital competence. • Map local organization after completion of sector restructuring. • Establish arenas for exchanging experiences that document both best practices and

achievements, with close ties to quality development in higher education. Specific meeting grounds for HE academic staff and ICT support for education.

• Better use of data: Establish a national collaboration for establish best practices for the use of learning data.

Establish dialogue with all Centres for Excellence in Education to get them more involved in the digitization of higher education. Their role in the digitization of higher education should be defined, preferably in conjunction with defining the scope of initiatives under the strategy for ICT in education.

Tools for higher education

Local measures We recommend the following changes:

• Institutions must facilitate for large-scale production of academic videos in accordance with the best practice in the field. Institutions that have established such production, should evaluate their use and adjust their practices for optimal use in their respective fields.

• Institutions must offer collaboration tools that allow students to collaborate anywhere, at any time, in accordance with the best practice in the field.

60

• Institutions must work actively to amend their forms of assessment to take full advantage of the benefits offered by digital assessment, in accordance with the best practice in the field.

• Institutions should actively evaluate their digital learning environments and contribute to the establishment of a best practice in the area.

• Recommendations from the ICT strategy for higher education must be followed up: o Digital assessment: from examination to formative assessment o Adaptive learning should be actively explored in collaboration with SLATE

We recommend the following measures:

• Joint specifications for AV equipment intended for teaching o Institutions make use of the central procurement agreement for AV equipment to

ensure standardization, investments and maintenance. o Documentation of local best practices is developed at each institution, based on

equivalent national documentation, where available. o Best practices, both national and local, shall be applied in the design and

maintenance of teaching rooms. • Video infrastructure

o Institutions use national services for simple and cost-effective deployment. o All institutions participate in national and local networks for experience exchange. o Documentation of local best practices is developed at each institution, based on

equivalent national documentation, where available. o Best practices, both national and local, shall serve as guidelines in the selection of

solutions, development of facilities and adaptations for end users. • Collaboration tools

o Institutions use national collaboration service wherever available. o All institutions participate in national and local networks for experience exchange. o Documentation of local best practices is developed at each institution, based on

equivalent national documentation, where available. o Best practices, both national and local, shall serve as guidelines in the selection of

solutions, development of facilities and adaptations for end users. • Digital assessment

o Institutions should use joint national solutions for digital assessment. o All institutions participate in national and local networks for experience exchange. o Documentation of local best practices is developed at each institution, based on

equivalent national documentation, where available. o Best practices, both national and local, shall serve as guidelines in the selection of

solutions, development of facilities and adaptations for end users. • Digital learning environment

o Institutions should use joint national solutions for digital learning environments. o All institutions participate in national and local networks for experience exchange. o Documentation of local best practices is developed at each institution, based on

equivalent national documentation, where available. o Best practices, both national and local, shall serve as guidelines in the selection of

solutions, development of facilities and adaptations for end users. These measures have been recommended because the changes that are coming will require effective utilization of joint solutions, strong local guidelines, cross-institutional experience sharing and clear incentives to exercise best practice. These factors were identified in experience workshops and can only be implemented with systematic local effort at each individual institution.

61

National measures We recommend the following measures:

• Video infrastructure established under eCampus is continued and developed further to ensure that the HE sector has access to high-quality, cost-effective video solutions.

• Continue to actively share experiences and document best practices for video use even after the end of the eCampus programme.

• The HE sector continues to work together on joint collaboration solutions, while documenting best practices in the application of these solutions.

• The HE sector continues to work on solutions for digital assessment even after the national procurement process in the area has been completed. This effort should be focused on new forms of assessment, joint specifications for these, and a collective understanding of needs.

• The HE sector must continue to work on experience sharing and documentation of best practices in digital assessment.

• We must develop mechanisms that enables collaboration on joint procurement and development related to digital learning environments.

• The HE sector must continue to work on experience sharing and documentation of best practices in digital learning environments.

• The best practice for AV equipment in the HE sector must be maintained, along with the related framework agreement for the procurement for such equipment, in order to ensure good equipment quality in teaching rooms and to keep costs low in connection with upgrades and the construction of new teaching rooms.

The current financing model is: Financing is linked to procurement agreements for AV equipment, digital assessment and digital learning environments. For video infrastructure and collaboration tools, financing is linked to national services, where a surcharge is added to cover experience sharing and documentation of best practices. UNINETT’s prioritization council for teaching-related services defines the scope of the activities.

Joint framework for ICT use in higher education

Local measures We recommend the following changes:

• The respective institutions must use active digital management and good incentives to take ICT from the project stage to institutionalized practice.

• The institutions must also transition to modular solutions for learning environments and implement data-centric solutions with a strong emphasis on user needs.

• Follow-up of recommendations from the ICT strategy for higher education o Digital assessment: map need for digitization of exams and formative assessment. o Adaptive learning should be actively explored in collaboration with the Centre for

the Science of Learning & Technology. o Implementation of a learning object repository (LOR) for open education resources

(and other learning resources). These measures have been recommended because these areas have been emphasized time and time again by participants at eCampus experience seminars. These are measures that have to be

62

implemented at each institution, and they are critical for the implementation of large-scale use and sound execution of change processes.

National measures We recommend the following measures:

• A three-year project working to establish the technological foundation for a joint ICT architecture for the HE sector. This work should be seen in context with data-centric modular solutions, and be available to initiatives looking into adaptive learning, open education resources and digital assessment.

o Standardization o Follow-up of integrations o Follow-up of best practice

• A decision-making structure should be established for national initiatives related to ICT use in higher education, and this should follow up on ICT strategies and structural changes.

• The transition to modular solutions should be coordinated by a unit with insight into the solutions currently in use, and control of data flowing between the different modules.

• The recommendations from the proposed ICT strategy for higher education related to digital assessment and adaptive learning must be actively addressed to ensure that they are handled well. The level of maturity for both digital assessment and learning platforms should be increased, and joint development initiatives should be evaluated and tied to a joint decision-making structure.

63

Appendix 2: Working groups

Working group Description eCampus North Commissioned by the Association of Higher Education Institutions in

Northern Norway, eCampus had observer status Main project results: inter-institutional collaboration and establishment of a shared understanding of needs

Recording Testing video recording solutions Main project results: network for support personnel, experience exchange related to practical applications and local implementation

Recording: Opencast Testing open source code solution for lecture recordings Main project results: pilot of Matterhorn Opencast

Online meetings Worked on solutions for online meetings and video-based collaboration tools Main project results: joint specifications for procurements of online meeting solutions, experience exchange related to practical applications and local implementation

AV Responsible for AV procurement agreement for teaching room equipment Main project results: joint specifications for framework agreements for procurement of AV/media equipment, follow-up of providers and needs comprehension

Digital examinations: client equipment for digital examinations

Worked on students using their own devices and the institution’s devices during digital written examinations. Main project results: best practice document with specifications for the use of end user equipment during digital examinations

Digital examinations: examination room infrastructure

Worked on specifications for physical infrastructure in digital written exams Main project results: best practice document with specifications for examination rooms

Digital examinations: workflow

Analyzed work processes related to digital written examinations Main project results: report establishing a shared understanding of requirements for digital workflows in digital exam execution, part of best practice document

Digital examinations: ICT architecture

Worked on ICT architecture for digital writtene examinations Main project results: best practice document for ICT architecture in digital examinations

Digital examinations: integrations

Worked on specifying, developing and testing joint integrations for digital examinations Main project results: best practice document and operational collaboration on the use of integrations

Personal storage Worked on specifications for and collaboration on joint procurement of a cloud solution for personal storage and file sharing Main project results: joint specifications for personal storage and file sharing

Digital learning environment of the future

Define joint overall requirements for provider-independent learning platforms Main project results: report detailing joint requirements for digital learning environments, recommendations for learning platform specifications

64

Appendix 3: eCampus projects

Grants from the Ministry of Education and Research to the eCampus programme total NOK 70 million. 2011

(launch) 2012 2013 2014 2015 2016

Ministry grant (NOK)

3.5 million 12 million 15 million 15 million 15 million 9.5 million

Project Description Time frame eCampus recording

Project will test, document and facilitate for recording solutions in the HE sector. Specific hypotheses will be tested in close collaboration with universities and university colleges.

10/12/2010 – 22/12/2016

eCampus collaboration

The project will contribute to establish a cohesive collaboration infrastructure (including real-time communication) that is available for as many users as possible in the Norwegian HE sector within 2016. This includes making sure that as many users as possible have the digital competence required to fully utilize the potential of said infrastructure.

10/12/2010 – 22/12/2016

eCampus digital examinations

This development project shall ensure that students get access to digital examinations, and that workflows related to exam execution in higher education are digitized.

15/02/2014 – 27/02/2015

eCampus digital learning environment

This project shall document and develop a shared understanding of digital learning environments in higher education. This shared understanding shall be applied to • support local initiatives involving digital learning environments in participating institutions • create a roadmap for digital learning environments • oversee a generational shift of solutions for digital learning environments

29/01/2016 – 09/12/2016

eCampus online meetings

Joint online meeting solution for teaching, meetings and real-time collaboration.

31/01/2011 – 15/12/2013

eCampusFileSender

Develop and deply a simple and secure solution for sharing large files over the Internet for all students, researchers, lecturers and other staff in the Norwegian HE sector. Includes the coordination of an international consortium for the FileSender solution.

03/01/2011 – 30/11/2013

Until the end of 2013, service delivery was organized as part of the projects. National joint services were organized as activities under the programme from 2014: - eCampus deployment (joint) - eCampus support (joint) - Simple solution for video recording: Techsmith Relay - Automated solution for video recording: Mediasite - Online meetings: Adobe Connect - File sharing, large files: FileSender - Personal storage and file sharing: Box - Video conferencing: video catalogues and video bridge A number of small pilots testing specific functionality have also been carried out.

65

Appendix 4: Campus Best Practice Documents The eCampus programme has updated and prepared a number of best practice documents.

Specifications for AV equipment for teaching rooms This range of best practice documents existed before the programme, but they have been updated and expanded as part of the eCampus programme:

• Functional description of AV equipment for classrooms and meeting rooms (UFS 116) • Technical and functional system requirements for AV equipment (UFS 119) • Operational support systems and audiovisual transmission (UFS 120)

UFS 116: FUNKSJONSBESKRIVELSE AV-UTSTYR FOR UNDERVISNINGS- OG MØTEROM

(Published in English as CBP: Functional description AV equipment for classrooms and meeting rooms) UFS 116 specifies functional requirements for recommended solutions for AV equipment in auditoriums, seminar rooms, classrooms, meeting rooms and group rooms. It is one of three documents detailing specifications for AV equipment to be used in the higher education sector. The primary target audience is technical personnel in charge of AV equipment.

UFS 119: TEKNISKE OG FUNKSJONELLE SYSTEMKRAV FOR AV-UTSTYR (Published in English as CBP: Technical and Functional System Requirements for AV equipment) UFS 119 is a shared supporting document for UFS 116 and UFS 120. These documents specify functional requirements for recommended solutions for AV equipment in the higher education sector. UFS 119 specifies technical and functional system requirements for various components included in the functional descriptions. The primary target audience is technical personnel in charge of AV equipment.

UFS 120: DRIFTSTØTTESYSTEM OG OVERFØRING AV LYD OG BILDE (Published in English as CBP: Operational Support Systems and Audiovisual Transmission) UFS 120 is a supporting document for UFS 116. This document deals with shared resources when a number of rooms are used in connection with operational support and external and internal audio and video transmission (monitoring and remote control of AV installations, broadcasting/streaming of lectures, multi-party video conferences, infosystems, etc.). The primary target audience is technical personnel in charge of AV equipment.

Requirements for digital examinations This range of best practice documents was prepared by working groups and projects under the eCampus programme.

• Physical infrastructure for digital assessment (UFS 145) • Clients for digital assessment (UFS 146) • Integration for digital assessment (UFS 147) • Architecture for digital assessment (UFS 148) • Logging and monitoring of digital assessment (UFS 149) • Requirements for the use of cloud services (UFS 150)

66

UFS 145: INFRASTRUKTUR FOR DIGITAL EKSAMEN

(Published in English as CBP 42: Physical infrastructure for digital assessment.) UFS 145 contains recommendations for physical infrastructure in permanent and temporary halls used for holding digital exams. Standards for physical wiring of networks and set-up of wireless networks have been described in previous documents (UFS 102, UFS 112 and UFS 127). The primary target audience is technical personnel in charge of planning and executing digital examinations.

UFS 146: KLIENTER FOR DIGITAL EKSAMEN (Published in English as CBP 43: Clients for digital assessment.) UFS 146 looks at relevant clients for digital examinations, both BYOD and the institution’s own equipment. The document does not look at software solutions for the execution of digital examinations, but focuses on benefits and drawbacks of existing client solutions. The primary target audience is technical personnel and advisors in charge of planning and executing digital examinations.

UFS 147: INTEGRASJON AV LØSNINGER FOR DIGITAL EKSAMEN (Published in English as CBP 44: Integration for digital assessment.) Data in existing systems must be reused in the examination solution. UFS 147 identifies six existing systems with master data sources, and describes the standard for the exchange of data to and from the examination solution. The primary target audience is developers, systems integrators and on-campus technical personnel.

UFS 148: IKT-ARKITEKTUR FOR DIGITAL VURDERING (Published in English as CBP 44: ICT architecture for digital assessment.) This best practice document describes the recommended best practice for digital workflows in connection with the holding of exams. The overall presentation provides a common description, across institutional boundaries in the HE sector, of the elements required in order to specify a digital exam solution for higher education. The primary target audience is institution management and advisers in charge of planning and holding digital examinations.

UFS 149: LOGGING OG OVERVÅKING (Published in English as CBP 45: Logging and monitoring of digital assessment.) UFS 149 contains specifications for logging and monitoring in connection with digital examinations, and recommends how logging and monitoring should be done. It furthermore describes how to categorize logged information, and how to handle the logs. The primary target audience is institution management and technical personnel in charge of planning and holding digital examinations.

UFS 150: SPECIFICATIONS FOR USE OF CLOUD SERVICES UFS 150 addresses the legal requirements for the use of cloud services in connection with digital examinations, and details the progress of legally establishing a digital examination solution in the cloud. This best practice document is general in nature, and can also be applied in the establishment of other types of cloud services. The primary target audience is institution management and technical and security personnel in charge of establishing services in the cloud.

67

Appendix 5: eCampus prioritization council

Prioritization council mandate

eCampus shall strive to promote better ICT infrastructure within higher education in Norway, for the benefit of both research and education. The prioritization council shall ensure that eCampus develops a technical infrastructure through collaboration with universities and university colleges by

• working with ICT architecture and common components that support education, research and dissemination

• offer simple and good ICT tools and ICT services for education and research

• facilitate for high-quality ICT support in education and research

• promote initiatives to make lectures available on the Internet on a national scale

Good interaction with local priorities is critical for achieving the best possible utility value from eCampus initiatives. Universities and university colleges are autonomous institutions, where prioritization is a local process, limited to each respective institution. The prioritization council is composed of individuals who, by virtue of their expertise, can advise on joint efforts and strategic prioritization.

The prioritization council is tasked with

• Offering strategic advice on the long-term plans for eCampus. Promoting HE sector coordination by joint prioritization of eCampus activities. Prioritizing eCampus responsibilities, activity content, new priorities and other changes.

• Forging collaborative relationships with similar initiatives in the HE sector, so that eCampus can gain valuable insight into how best to utilize the available competence and resources, and how best to contribute its expertise.

• Giving recommendations for the organization of eCampus initiatives.

All projects financed or partially financed through eCampus should comply with UNINETT’s standard contracts and standard project procedures. UNINETT is responsible for programme follow-up and budget, but the prioritization council has a quality assurance responsibility, which entails

• Following up on the quality of project results.

• Following up on the work carried out in eCampus: progress, results, offer advice on non-conformities and other challenges.

The members of the eCampus prioritization council are appointed by UNINETT’s board. The prioritization council’s members shall include representatives of both students and staff. The council shall also have representatives of both education and IT, and both universities and university colleges shall be represented.

The prioritization council shall meet no less than three times a year. Beyond this, the council decides on the type of information flow they would prefer and how to communicate within the council and with the educational sector at large. Meetings may be held electronically, and minutes shall be kept.

68

Prioritization council members

• Petter Kongshaug, UNINETT (2012–2016)

• Oddrun Samdal, University of Bergen (2014–2016)

• Berit Kjeldstad, NTNU (2012–2016)

• Petter Aasen, Vestfold University College, Buskerud and Vestfold University College, University College of Southeast Norway (2012–2016)

• Bjørn-Anders Hind, National Union of Students in Norway (2012–2013)

• Endre Olsvik Elvestad, National Union of Students in Norway (2012–2014)

• Amanda Schei (2014–2015)

• Håvar Uhre Halvorsen, National Union of Students in Norway (2014–2016)

• Jan Atle Toska, University of Nordland (2012–2016)

• Geir Tutturen, Norwegian University of Life Sciences (2012–2013)

• Trond Akerbæk, Østfold University College (2013–2016)

• Gunnar Stave, Volda University College (2012–2014)

• Jon Lanestedt, University of Oslo (2012–2014)

• Lars Oftedal, University of Oslo (2015–2016)

• Håkon Solberg, Norwegian School of Sport Sciences (2015–2016)

• Eva Gjerdrum, Norwegian Agency for Digital Learning in Higher Education (observer) (2012–2016)

• Tove Lyngra, Ministry of Education and Research (observer) (2012)

• Borghild Abusland, Ministry of Education and Research (observer) (2013–2016)

69

Appendix 6: eCampus staff

Figure 18: eCampus team in 2015 (Photo: Mattis Daae) Back row, from left: Freddy Barstad (on loan from NTNU as project manager for digital examinations), Alice Sporstøl, Thorleif Hallén, Lars Kviteng and Jan Meijer. Centre: Ingrid Melve and Snorre Løvås. Front row, from left: Monica Gjørv Steneng, Renate Langeland and Magnus Strømdal. Simon Skrødal and Lars Fuglevaag were not present when the picture was taken. Other eCampus staff has included Bernt Skjemstad, Hege Ossletten, Armaz Mellati, Christine Johnsen, Otto Wittner, Jim Flaten, Jardar Leira, Heidi Bergh-Hoff, Stefan Otto, Håvar Aambø Fosstveit, Mattis Daae, Niels Chamara Myrdal and Faisal Naseri.