The paper explores the challenges of designing personalised learning paths in SPOCs (Small Private Online Courses). It opens with a discussion on different approaches to tailoring teaching to individual needs and moves on to introduce a SPOC that was developed for Continuing Professional Development (CPD) for primary and lower-secondary teachers in Denmark. The SPOC, which performs adaptation using a recommendation system, allows for students to create a personalised learning path on the basis of three components: a learner profile, a content model and an adaptation model. Using the three components as a starting point, the SPOC is analysed in order to identify differences between the intended design (what the SPOC set out to do), the implemented design (how the SPOC is used by its users) and the attained design (the outcome of the SPOC). The analysis draws on data from a series of semi-structured interviews with SPOC students and their lecturers. We find that the implemented design deviates from the intended design in several respects, most notably in relation to how the personalised learning paths are created and how decisions as to curriculum contents are made. Moreover, it is suggested that differences between the intended design and the implemented design are rooted in differences in the learning perspectives of the students, the lecturers and the educational designers of the SPOC. Despite the fact that the implemented design deviates from the intended design, the attained design is nevertheless successful in that a high percentage of the students enrolled succeeded in passing their examinations and thus obtained the formal qualifications in the subjects they teach. It is concluded that further research in adaptive learning designs for online platforms such as MOOCs (Massive Open Online Course) and SPOCs is needed to minimise the gap between intended designs and implemented designs in order to create a more personalised learning experience for the students involved.
In formal education, the curriculum plays a crucial role, because it outlines the planned programme of objectives, content, learning experiences, resources and assessment offered by an education institution or an education programme. The word curriculum stems from Latin and translates into ‘a race’ or ‘the course of a race’ (which in turn derives from the verb ‘currere’ meaning ‘to run’ or ‘to proceed’). In formal education throughout the world, students are often expected to complete ‘the course of a race’, i.e. their studies, at the same pace by studying the same contents in much the same manner to reach the same goals in the form of a number of shared learning objectives and a final examination. Paradoxically, it is also a well-known fact that students learn in different ways and that their personal prior knowledge is a highly influential factor in the learning process (Glaser, 1984).
Consequently, different approaches to tailoring teaching to individual needs have emerged. Most notably the concept of differentiation has been introduced, where a variety of teaching methods are used to meet the individual learning needs and preferences of students who share the same learning goals. Likewise, the concept of individualization addresses students with the same learning goals, but this approach differs from differentiation in that students can move forward at their own pace and relate to specific contents or activities in different ways (the U.S. Department of Education, 2010). But what if students do not share the same learning goals and bring with them radically different prior academic knowledge? This is often the case in Continuing Professional Development (CPD) and here personalisation offers promising opportunities. Personalisation refers to ‘instruction that is paced to learning needs, tailored to learning preferences, and tailored to the specific interests of different learners. In an environment that is fully personalized, the learning objectives and content as well as the method and pace may all vary’ (ibid.: 12). In this way, personalisation encompasses both differentiation and individualization, and it relies on the use of learning management systems, which can perform or support personalised adaptation. In a literature review on adaptive educational hypermedia by Akbulut and Cardak (2012), it is found that adaptive and personalised learning designs tend to have a positive effect on students’ learning outcome.
One of the ways in which personalisation can be performed is through e.g. a Small Private Online Course (SPOC), which serves as the empirical context of the study presented here. A SPOC is a type of MOOC (Massive Open Online Course), but while MOOCs are both massive and open to (potentially) anyone, a SPOC is small and private, and is used as a supplement to classroom teaching rather than being a replacement for it (Fox, 2013). In effect, a SPOC is based on blended learning and only a limited number of students have access to the course. The empirical data analysed here stem from a SPOC developed within a Design-Based Research project to meet the needs for Continuing Professional Development (CPD) prompted by a national education reform, which requires that all teachers in the Danish primary and lower-secondary school hold a formal exam in the subjects they teach by 2020 or, alternatively, hold a positive validation of non-formal and informal learning. Thus, it is required that the solution, i.e. the SPOC, is flexible in relation to the students’ working life as they are often employed as full-time teachers and that it takes into consideration that the students are experienced teachers who possess both professional knowledge and skills.
In this paper, we look into the challenges of designing a Small Private Online Course (SPOC) that provides the learner with a personalised learner pathway on the basis of a recommendation system. In a discussion on how to classify MOOCs, Conole (2013, 2014) proposes a classification framework consisting of twelve dimensions that are central for describing the nature of MOOCs and similar formats. One of the twelve dimensions mentioned in Conole’s discussion is the dimension of learner pathways, which covers a continuum ranging from learner-centred to teacher-centred and highly structured.
Learner pathways are a particularly central feature to adaptive learning systems, which ‘tend not to deliver flat, linear structured knowledge but leaning experiences driven by back-end algorithms’ and ‘[…] analytics are also used to change and improve the course in the future’ (Clark, 2013). Adaptive learning systems, such as the SPOC under study here, can be based on two different adaptation strategies: recommendation systems or guided navigation (Khribi et al., 2015). In a recommendation system, a range of possibilities is identified by the system based on a model of the learner or the learner’s performance in the system. This means that the learner is free to choose whether or not to follow the recommendations provided by the system. In a guided navigation system, the contents that are irrelevant to the learner are hidden by the system, either because they do not comply with the model of the learner or because they do not match the learner’s performance in the system (Gynther, 2016a; Gynther, 2016b). The adaptive SPOC under study here makes use of a recommendation system to create personalised learner pathways.
According to McKenney and others (McKenney et al., 2006; McKenney and Reeves, 2012), a three-level classification of the intended, implemented and attained designs can serve as a useful analytical tool for evaluating and redesigning educational designs. The intended design represents the original design intentions, the implemented design describes how lecturers and students actually use the design and the attained design includes the specific outcomes of the educational design in terms of e.g. students satisfaction, completion rates and grade scores (McKenney and Reeves, 2012). In the present study, the users’ experiences with the SPOC are discussed in relation to the three design levels in order to answer the following research question: What are the experiences of students and lecturers in a personalised SPOC environment, and what differences can be identified between the intended, implemented and attained designs of the platform? With this question we aim at not only shedding light on how users of the SPOC experience this type of learning environment, but also on how design processes from initial idea through implementation and all the way to the reaction of the users can be viewed and analytically dissected.
In the following, we explain the methodological background of the project from which the SPOC was developed and illustrate how personalised learner pathways are identified in the SPOC on the basis of a recommendation system. Finally, we discuss the differences between the intended design (what the SPOC set out to do), the implemented design (how the SPOC is used by its users) and the attained design (the outcome of the SPOC).
The empirical data analysed in the paper derive from a Design-Based Research (DBR) (Amiel and Reeves, 2008) project in which a SPOC for continuing professional development for experienced teachers was developed. DBR is an approach to educational research that aims at developing solutions to real-life educational challenges in collaboration with practitioners through different types of interventions (Anderson and Shattuck, 2012).
The intervention under study here is the aforementioned SPOC, and the empirical data referred to stem from a series of semi-structured interviews with SPOC lecturers and focus group interviews with SPOC students. Approximately 20 interviews with 30 students and 10 SPOC lecturers were conducted throughout a 2-year period. The interview data were transcribed and thematically coded using key terms from the intended design and subsequently categorised as being in concurrence with or deviating from the intention of the SPOC. Examples of codes were: personalisation (in concurrence with/deviating from), competency profile, non-progressional and personalised recommendation. Another string of codes was related to the different parts of the SPOC i.e. the learner, content and adaptation model. Finally, different expressions of learner experiences were coded as they appeared through the interviews (i.e. concern, surprise, lack of motivation, confusion, etc.). The researchers involved met on a regular basis to discuss and analyse the learner experience categories, which were continuously bundled or split up, as a result of this work.
The SPOC was the first of its kind both within the organization, but also in the field of professional teacher development in Denmark. Therefore, the project was in the initial phases regarded as a development study. This meant that the digital department spent all their time assisting the production of the content for the SPOC and not until recently provided tools for excerpting quantitative data material from the platform. In the early stages, few plugins were implemented as part of the MOODLE platform and the research team did not gain access to any of this data.
The lecturers were involved in the project as part of their work portfolio and the students interviewed were selected through a process were the researchers asked for informants during ongoing courses. The group of students to be interviewed was selected in order to ensure diversity of subjects represented among the volunteers. The students were studying the subjects Danish, Mathematics or English. To ensure anonymity, students are referred to as student A, student B, etc., and where applicable gendered pronouns are omitted.
In his book, Out of Our Minds, Robinson discusses a distinct and in his view problematic principle in curriculum design throughout educational systems around the world: the manufacturing principle of linearity. It refers to the educational process as consisting of sequential stages, which are meant to logically build on one another, and the idea that the final outcomes can be predicted with reasonable reliability (Robinson, 2011). Embedded in this principle is the idea that education is essentially a preparation for something that will happen later on. Consequently, this approach to education focuses primarily on standardisation and conformity, and less on what the individual student is interested in. Like an assembly line, students are brought from room to room to be taught by different teachers specialised in separate disciplines. Students within academic systems are taught broadly the same contents, and they are assessed against common scales of achievement, with relatively few opportunities for choice or deviation (ibid.).
To some extent, the underlying principles of formal education described by Robinson can also be found in approaches to learning that are based on acquisition. This argument is further supported by Laurillard (2012) who claims that approaches based on acquisition are still among the most widely used approaches in formal education. Within this perspective, the teacher uses a transmission approach where the students play a relatively passive role by reading books, exploring websites, listening to podcasts, watching videos, etc. Language is the dominant form of presentation, whether the students acquire new knowledge through listening or reading. The strength of the approach lies in the exposure of expert ways of thinking and in building on the progressive development of ideas (ibid.). Furthermore, listening and reading can foster more active participation among a student group, such as engaging in discussions with lecturers or peers or working on assignments either in groups or individually. In this way, other approaches, such as learning through discussion or collaboration as described by Laurillard (2012), can take up the dominant position. Although the approaches of learning through discussion and collaboration are often aligned, the latter poses the challenge of creating a joint reference, a product, to move to further exploration. Both, however, rely on the social aspect of learning as the most important part of learning activities as exemplified through discussion or small group sessions (ibid.). More importantly in relation to the challenges of linear thinking in curricula as discussed above, the social aspect of learning brings forth the preconceptions and prior knowledge that students bring with them to the learning situation, and it seeks to enable students to take control of their own learning by, for instance, letting them set their own learning goals.
On the one hand, using a one-size-fits-all curriculum that has been pre-designed for all students entering a course and which requires the students to passively acquire knowledge through expert presentations, is less than ideal. On the other hand, many would argue that factual knowledge is indeed required to scaffold students to, for instance, engage in collaborative learning activities and to design their own personalised learning paths. These seemingly opposing positions are further discussed below as the SPOC is analysed at three different design levels, namely at the level of the intended design, the implemented design and the attained design.
In a discussion on how curricula can be represented in different ways, van den Akker distinguishes between three curriculum levels referred to as the intended, implemented and attained curriculum (van den Akker, 2003). This three-level classification is interesting in relation to the focus of the present paper because, as McKenney and others argue (McKenney et al., 2006; McKenney and Reeves, 2012), it can also serve as a useful analytical tool for evaluating and redesigning educational designs. Thus, the intended design represents the design principles on the basis of which the design was originally developed, the implemented design describes how lecturers and students actually use the design and the attained design includes the specific outcomes of the educational design in terms of e.g. students satisfaction, completion rates and grade scores (McKenney and Reeves, 2012). Following this, evaluation of an educational design must test both the intended, implemented and attained designs, and the gaps that may exist between the three design levels.
In the following, we look into the three design levels of the SPOC in relation to three interrelated components that allow for adaptation within the design: a learner profile, a content model and an adaptation model.
In accordance with the intended design, the first public version of the SPOC comprises three components: a learner model, a content model and an adaptation model. The overall purpose of the SPOC was to provide to the students a personalised learning experience in order to increase their learning outcome, to decrease dropout rates and to deliver a highly flexible learning environment without increasing the costs related to teaching.
The first component, the learner profile, refers to the identification of the student’s competencies visualised in a competency profile. The purpose of the competency profile is to identify the parts of the curriculum that are already covered by the student’s prior knowledge as well as the ones that are not. The learner profile and a recommended learning path for the student are created on the basis of an algorithm in order to support the student in identifying relevant course contents to study in the SPOC. The learner profile, which is illustrated in Figure 1 below, is based on a digital self-assessment test carried out by the student, which is adjusted on the basis of a face-to-face self-assessment interview between the student and a lecturer. All the learning objectives of the course are covered in the digital self-assessment tool and in the face-to-face interview, and the learning objectives and the corresponding course contents are categorised into three categories: fully competent (percentage of the competencies met by the student), partially competent (percentage of the competencies that are partially met by the student) and not competent (percentage of the competencies that are not met) (see Figure 1).
The learner profile illustrated in Figure 1 shows that the student is fully competent in relation to 46% of the curriculum (as measured against the learning objectives), partially competent in relation to 23% of the curriculum and insufficiently competent in relation to 31% of the curriculum. In sum, the student’s studies are estimated to comprise of a total of 12 ECTS out of 30 ECTS.
The student is also provided with a more elaborated and detailed overview of the learning objectives that she is not able to meet. The overview also shows which academic contents the student should study in the SPOC in order to become fully competent in relation to the learning objectives that she is unable to meet.
The second component, the content model, refers to the structuring of learning objectives, topics to be studied and assignments to be solved. It divides the subject into smaller units, which are linked to different types of learning resources (e.g. videos, texts, quizzes, etc.). As a consequence, the academic contents of the various 30 ECTS subjects offered in the SPOC had to be deconstructed into smaller units so that a personalised match between the students’ learner profiles and specific academic contents was possible. Thus, the idea of academic progression between the modules within a specific subject was abandoned to allow for personalised learning paths and, in principle, there is no shared starting point in the SPOC for the students.
The third and final component, the adaptation model, refers to the manner in which the learner profile and the academic contents are matched. The adaptive model is based on a recommendation system, which identifies the contents that are relevant to the student on the basis of her learner profile (the learner model). As opposed to adaptation based on guided navigation, the SPOC student is free to choose whether or not to follow the recommendations provided by the system as the contents rated as irrelevant by the system are still available to the student, even though they do not match her learner profile. In this way, the student is ultimately the one who controls the adaptation process. At the same time this means that there are no classes or larger study groups to facilitate discussion or collaboration. In order to face this challenge every student is placed in a study group, preferably formed by a lecturer, with students working at the same school and studying the same subject.
To sum up, the intended design of the SPOC aims at providing a personalised curriculum for each student. The personalised curriculum is identified by comparing the prior knowledge of the student to the learning objectives of a given subject as described in the national curriculum. The learner profile (exemplified in Figure 1) provides an overview of the student’s competency gap in relation to the learning objectives and, moreover, it shows the estimated workload for the student. As such, the SPOC is based on personalisation rather than individualisation as the students are likely to pursue different learning objectives and engage with different academic contents.
We now turn our attention to the implemented design of the SPOC, that is, how the SPOC is actually used by the students and lecturers, and the possible deviations between the intended and the implemented designs in relation to the three components: the learner model, the content model and the adaptation model.
Firstly, we look into how the learner profiles were created and how the students used the self-assessment tool and the supplementary self-assessment interview. Next, we discuss the content model and how the lecturers reacted to designing an online learning platform that is not based on curricular progression between modules and, lastly, we discuss the deviations between the intended and the implemented designs in relation to the adaptation model from the perspectives of the students and the lecturers.
In general, the self-assessment test and the follow-up interview with a lecturer were well received by the students. While the self-assessment test provided the students with an insight into the full spectrum of the curriculum, the interview helped iron out the feelings of insecurity expressed by some of the students in relation to e.g. understanding the learning objectives correctly, solidifying the value of prior experience and, in general, the interview helped the students in understanding the relationship between their learner profiles and the academic contents in the SPOC. One student states:
“The self-assessment test also provided a valid picture of what you are supposed to learn”. (student A)
However, some students, especially those with less experience as teachers or with only limited knowledge of the subject, expressed concerns regarding the massive workload presented to them:
“Let’s say you end up feeling really good once you have done the self-assessment test. I just have to do themes number 3 and 4 or something, then obviously it is not stressful, but what if this is not the case? In that way, there are many different starting points for entering the course. Some might need a lot of time for studying, while others don’t. I certainly need a lot!” (student B)
Another student supports this perspective and adds that factors other than teaching experience are important:
“Yes, and it’s not necessarily a matter of how much teaching experience you have. I’ve taught this subject for twenty years, but my profile ended up being quite red [the student is referring to the percentage of learning objectives that she was not able to meet according to her learner profile, see the illustration in Figure 1]. I might be competent as to all the practical stuff but all the theories, the fancy words, they need to be studied and this takes time. When I saw the results of my learner profile, I actually turned pale because my principal thought I would skate right through the course due to my extensive experience.” (student C)
Because many of the students were anxious about not studying the entire curriculum, but only parts of it, they would have liked to have had the opportunity to attend what they refer to as ‘introductory lectures’ that cover large units of the curriculum, as exemplified by two interviewees:
‘What we need is an overall basic lecture on the essential contents for this module. Period!’ (student D)
“Yes, that would be great!” (student E)
However, the basic lectures that the students are asking for do not conform with the intended design, which is based on the premise that the curriculum is personalised in accordance with each student’s prior knowledge and, for this reason, identifying academic contents that are essential to all students would be impossible. This discrepancy in learning perspective between the intended design and the needs expressed by some students is further considered in the discussion below.
In an evaluation of the SPOC in which the lecturers were interviewed, the manager of the research project argues that:
‘The greatest legitimacy problem of the design is related to the breakdown of a subject into competency units without progression. This design element conflicts with the teachers’ common conceptions of curriculum design’.
In one of the subjects in particular, the lecturers expressed a concern regarding this issue. Leaving decisions on what to study and in which order to the students alone did not correspond with the lecturers’ perception of the subject. As a consequence, they introduced a physical course book covering most of the themes of the subject and quite a few students left the self-assessment interview with the conviction they had to complete the course in a chronological order.
The group interviews with the students show that the students learning paths in the SPOC are not necessarily based on their individual learner profiles, but rather on 1) collective decisions made by study groups and 2) a fear of missing out.
There is a tendency for students to negotiate a shared learning path in their study groups, which means that their learning paths are not, as intended, based on their personal learner profiles. Instead, they are the result of collective negotiation within the study groups. As one student puts it:
‘It usually goes like this: ‘Ok, so what do we feel like studying today’. That is, if we haven’t already agreed on this the last time we met […]. It’s very difficult [deciding on what to study in the MOOC]’. (student F)
Another student explains that the topics, i.e. the academic contents, her study group is studying are not always relevant to her:
‘If I am the only one who would like to focus on a specific topic and the other four [members of the study group] say “but we really like this topic”, then I’m like “Ok, then we’ll focus on that”.’ (student G)
When asked by the interviewer if this approach is counterproductive in relation to the students’ individual learner profiles and, in effect, their learning needs, the students agree. The students explain that their study groups have been formed according to geography, so that students who live within close proximity of each other are in the same group, rather than according to their learner profiles or academic interests, which, they argue, would perhaps have been a more relevant criterion for group formation.
Some students simply do not follow the personalised learning paths advised by their learner profiles; instead, they study the entire curriculum and all the resources that are available in the SPOC. As one student explains:
‘I started out completely chronologically this year, having lots of energy and knowing exactly what to do. I printed the whole lot and I’ve been reading and reading ever since…’ (student H)
This approach clearly clashes with the intended design in that the student 1) assumes that the SPOC builds on chronological progression and 2) assumes that she must study the entire curriculum even though her learner profile tells her otherwise. It is likely that the student is suffering from what we refer to as academic fear of missing out (inspired by the popular acronym FOMO) to the extent that she is unable to follow a personalised learning path. However, it also becomes clear that the student does not fully understand the basic design of the SPOC:
‘I’m not very good at using the internet, which is probably why I printed everything. I simply need physical copies. My youngest son is 19 and he can help me. My lack of IT competences is a huge challenge. I had, in fact, read a lot of texts before I realised: ‘My God, everything is actually systematised in the SPOC!’. I’d just printed all the texts in the order they appeared in, and it really made no sense reading the texts in chronological order, and suddenly it occurred to me: ‘Wauw, this [a collection of resources] makes up a topic!’ (student H)
This serves to show that the design of the SPOC may be incompatible with (at least some) students’ ideas of how a course should be structured and, at the same time, it becomes clear that the SPOC is less successful in addressing some types of students, i.e. students who are not competent IT users or experienced in studying in an online learning environment.
As mentioned previously, the attained design includes the specific outcomes of the educational design in terms of e.g. student satisfaction, completion rates and grade scores (McKenney and Reeves, 2012). A total of 586 students have been enrolled as SPOC students since the SPOC was first launched in 2016, and 515 students have completed the courses they signed up for, that is, they have passed the formal written and/or oral examinations related to the courses in which they were enrolled. The students who have passed one or more formal exams have, generally speaking, received good results. For instance, the SPOC students (19 in total) enrolled in the course English (30 ECTS) received the following results: A = 2 students, B = 4 students, C = 11 students, D = 2. In the course Danish (40 ECTS), another group of students (12 in total) received similar results: A = 3 students, B = 6 students, C = 3 students.
Despite the fact that the implemented design deviates from the intended design, the SPOC is nevertheless successful in delivering what it was set out to do, i.e. a very high percentage of the students enrolled succeed in passing the formal examinations that they have signed up for and, ultimately, they succeeded in obtaining the formal qualifications in the subjects they teach, as required by the national educational reform.
In this section, we discuss the findings presented above, with particular emphasis on the differences in learning perspectives as observed in the intended design and the implemented design.
The intended design is based on a highly individualised learning perspective. The SPOC targets students who are experienced teachers in possession of relevant prior knowledge in relation to the subjects they are studying. For this reason, the intended SPOC design allows for the students to create a learner profile and on the basis of this, the system recommends a personalised learning path for each student. In the SPOC, the students are to work on different types of resources, e.g. texts, videos and quizzes, but they are free to decide which particular resources they want to focus on. Even though the resources are multimodal, oral and written language is nevertheless the he dominant form of presentation and the students are invited to follow rather than initiate. Thus, the student targeted by the intended design is both a competent and highly reflexive person who is able to assess her academic level in relation to a given subject and, at the same time, she is expected to assume the role of a relatively passive receiver of information.
A transmission approach is somewhat replicated by the students, who express the need for lectures on essential academic contents within a subject and are happy to leave decisions as to which contents to include to the lecturer. On the other hand, the students also opt for a more discussion-based approach as they collectively negotiate a shared learning path in their study groups as an alternative to having personalised learner paths based on their individual learner profiles. Thus, the students find they gain more from engaging in discussions with their peers than from travelling through a personalised path on their own.
In the case of the lecturers, the breakdown of a subject into smaller units without progression conflicts with their conceptions of curriculum design.
Basically, what the students are asking for is a one-size-fits-all curriculum, or, one-size-fits-all lectures and the lecturers, on the other hand, are challenged in designing a course without progression. This, we would argue, echoes the manufacturing principle of linearity (Robinson, 2011) and may account for some of the differences identified between the intended design and the implemented design of the SPOC.
In the paper, we have explored the challenges of designing personalised learning paths in a SPOC, which performs adaptation on the basis of a recommendation system, and we have discussed the three interrelated components that allow for adaptation within the SPOC, namely the learner profile, the content model and the adaptation model, in order to identify differences between the intended design (what the SPOC set out to do), the implemented design (how the SPOC is used by its users) and the attained design (the outcome of the SPOC).
It is found that the implemented design deviates from the intended design in several respects, for instance, the students collectively negotiate a shared learning path rather than following the path recommended by their learner models, and both the students and the lecturers find it hard to abandon the notion of a fixed curriculum, which leads to a less personalised course.
Moreover, it is concluded that the intended design of the SPOC is based on a highly individualised learning perspective and that it targets students who are competent and highly reflexive but, at the same time, it positions the course participants as relatively passive receivers of information. To some extent, a learning perspective based on transmission is reproduced by the students, who wish for joint lectures for all students, regardless of whether the contents match their individual learner profiles, and they seem content to leave contents-related decisions to the lecturer. Yet, the students seem to employ a more discussion-based approach in their study groups as they collectively negotiate a shared learning path for all study group members.
Further research in adaptive learning designs for online platforms such as MOOCs and SPOCs is needed to minimise the gap between the intended and implemented designs in order to create a more efficient and targeted learning experience for students in Continuing Professional Development. A place to start would be to explore the methods of participatory design or meta-design as a means for developing dynamic design solutions in education.
The authors have no competing interests to declare.
Akbulut, Y., & Cardak, C. S. (2012). Adaptive educational hypermedia accommodating learning styles: a content analysis of publications from 2000 to 2011. Computers & Education, 58(2), 835–842. DOI: https://doi.org/10.1016/j.compedu.2011.10.008
Anderson, T., & Shattuck, J. (2012). Design-based research: a decade of progress in education research? Educational Researcher, 41(1), 16–25. DOI: https://doi.org/10.3102/0013189X11428813
Clark, D. (2013). MOOCs: taxonomy of 8 types of MOOCs. Retrieved January 31, 2018, from http://donaldclarkplanb.blogspot.dk/2013/04/moocs-taxonomy-of-8-types-of-mooc.html
Conole, G. (2013). MOOCs as disruptive technologies: strategies for enhancing the learner experience and quality of MOOCs. Revista de Educación a Distancia, 39(15), 1–17. Retrieved December 2, 2018, from http://www.um.es/ead/red/39/conole.pdf
Conole, G. (2014). A new classification schema for MOOCs. The International Journal for Innovation and Quality in Learning, 2(3), 65–77. Retrieved December 2, 2018, from http://innoqual.efquel.org/current-issue/
Fox, A. (2013). From MOOCs to SPOCs. Communications of the ACM. 56(12), 38–41. DOI: https://doi.org/10.1145/2535918
Glaser, R. (1984). Education and thinking: the role of knowledge. American Psychologist, 39, 93–104. DOI: https://doi.org/10.1037/0003-066X.39.2.93
Gynther, K. (2016a). Adaptive designs for learning based on MOOCs: a design framework for personalized learning in teacher professional development. In A. M. Nortvig, B. H. Sørensen, M. Misfeldt, R. Ørngren, B. Allsopp, B. Henningsen, and H. Hautopp (Eds.), Proceedings for the 5th international conference on designs for learning: designing new learning ecologies (pp. 46–63). Aalborg: Aalborg Universitetsforlag.
Gynther, K. (2016b). Design framework for an adaptive MOOC enhanced by blended learning: supplementary training and personalized learning for teacher professional development. The Electronic Journal of eLearning, 14(1), 15–30.
Khribi, M. K., Jemni, M., & Nasraoui, O. (2015). Recommendation systems for personalized technology-enhanced learning. In R. H. Kinshuk (Eds.), Ubiquitous learning environments and technologies (pp. 159–180). Berlin: Springer. DOI: https://doi.org/10.1007/978-3-662-44659-1_9
Laurillard, D. (2012). Teaching as a design science: building pedagogical patterns for learning and technology. New York: Routledge Taylor and Francis Group. DOI: https://doi.org/10.4324/9780203125083
McKenney, S., Nieveen, N., & van den Akker, J. (2006). Design research from a curriculum perspective. In J. van den Akker, K. Gravemeijer, S. Mckenney, and N. Nieveen (Eds.), Educational design research (pp. 67–90). London and New York, Routledge.
Robinson, K. (2011). Out of our minds – learning to be creative. Oxford: Capstone. DOI: https://doi.org/10.1002/9780857086549
The U.S. Department of Education, Office of Educational Technology. (2010). Transforming American education – learning powered by technology. National Education Technology Plan. Retrieved February 1, 2018, from https://www.ed.gov/sites/default/files/NETP-2010-final-report.pdf
van den Akker, J. (2003). Curriculum perspectives: an introduction. In J. van den Akker, W. Kuiper, and U. Hameyer (Eds.), Curriculum landscapes and trends (pp. 1–10). Dordrecht: Kluwer Academic Publishers. DOI: https://doi.org/10.1007/978-94-017-1205-7_1