Active learning analytics in mobile: visions from PhD students

Eda Atasoy (Department of Distance Education, Anadolu University, Eskişehir, Turkey)
Harun Bozna (Department of Distance Education, Anadolu University, Eskişehir, Turkey)
Abdulvahap Sönmez (Department of Distance Education, Anadolu University, Eskişehir, Turkey)
Ayşe Aydın Akkurt (Department of Distance Education, Anadolu University, Eskişehir, Turkey)
Gamze Tuna Büyükköse (Department of Distance Education, Anadolu University, Eskişehir, Turkey)
Mehmet Fırat (Department of Distance Education, Anadolu University, Eskişehir, Turkey)

Asian Association of Open Universities Journal

ISSN: 2414-6994

Article publication date: 7 May 2020

Issue publication date: 19 October 2020

1676

Abstract

Purpose

This study aims to investigate the futuristic visions of PhD students at Distance Education department of Anadolu University on the use of learning analytics (LA) and mobile technologies together.

Design/methodology/approach

This qualitative research study, designed in the single cross-section model, aimed to reveal futuristic visions of PhD students on the use of LA in mobile learning. In this respect, SCAMPER method, which is also known as a focused brainstorming technique, was used to collect data.

Findings

The findings of the study revealed that the use of LA in mobile can solve everyday problems ranging from health to education, enable personalized learning for each learner, offer a new type of evaluation and assessment and allow continuous feedback and feedforwards; yet this situation can also arise some ethical concerns since the big data collected can threaten the learners by interfering with their privacy, reaching their subconscious and manipulating them as well as the whole society by wars, mind games, political games, dictation and loss of humanity.

Research limitations/implications

The research is limited with the views of six participants. Also, the sample of the study is homogeneous in terms of their backgrounds – their age range, their departments as PhD students and their fields of expertise.

Practical implications

The positive perceptions of PhD students provide a ground for the active use of LA in mobile. Further, big data collected through LA can help educators and system makers to identify patterns which will enable tailored education for all. Also, use of LA in mobile learning may stimulate the development of a new education system including a new type of evaluation and assessment and continuous feedback and feedforwards.

Originality/value

The widespread use of mobile technologies opens new possibilities for LA in the future. The originality of this research comes from its focus on this critical point.

Keywords

Citation

Atasoy, E., Bozna, H., Sönmez, A., Aydın Akkurt, A., Tuna Büyükköse, G. and Fırat, M. (2020), "Active learning analytics in mobile: visions from PhD students", Asian Association of Open Universities Journal, Vol. 15 No. 2, pp. 145-166. https://doi.org/10.1108/AAOUJ-11-2019-0055

Publisher

:

Emerald Publishing Limited

Copyright © 2019, Eda Atasoy, Harun Bozna, Abdulvahap Sönmez, Ayşe Aydın Akkurt, Gamze Tuna Büyükköse and Mehmet Fırat

License

Published in the Asian Association of Open Universities Journal. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) license. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this license may be seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

Learning and technology have always been two influential factors in human life since the usage of a piece of sharp stone. With the advance in human beings' ability to process materials and shape them for their daily use, technical aspects have become vital. Improvements in technology have led changes in education and changes in education have led an inquiry for new technological means. Each day the relationship between technology and education has become more reciprocal and the means of advanced technology such as computers and the internet have contributed not only to the reformation of classical face-to-face learning environments but also the rise of open and distance learning as an alternative for traditional learning. Notwithstanding, so far the most breath-taking invention has been mobile technologies, since they enable the learner to get whatever he/she has inquiry about at anytime and anywhere.

“Mobile learning is seen as both a process and a product of sociotechnical construction, which involves and considers the individual's social environment” (Viberg, 2015, p. 17). Thus, the term technology literacy has a significant importance for the individual who wants to keep up with the modern flow of information and update himself/herself for the task he/she is going through without falling in the trap of technological determinism. The accessibility of information without any handicap is crucial for this information age. Since there are all kinds of information around us, it is not so primary to get the information. To get right information easily and immediately is now the main task of technological devices. Fortunately, mobile devices, that are wireless ICTs, can serve the needs of information societies by providing non-stop connectivity independent of place and time, extra capabilities such as information processing as well as contributiveness to each individual user (Grantham and Tsekouras, 2004).

The advent of mobile learning opportunities (m-learning) as an advanced means of technology has made learning both easier for those who are already learning in face-to-face environments and accessible for those who cannot attend the learning processes because of some difficulties of them like being a full-time worker or disabled. Thus, m-learning has the potency to change the current situation of both face-to-face and distance learning. “In our ever-changing marketplace, today's learners are being educated, largely, for jobs that have not been invented” (McQuiggan, 2015, p. 1). When this is taken into consideration, the vital role of mobile learning and its function in distance education become more apparent because mobile learning enables learners to learn and experience in a more meaningful way, which can eventually bring the end of boring old-fashioned learning styles and environments. Thus, m-learning especially in open and distance learning may revolutionize the learning and teaching practices.

Another promising concept concerning the learning and teaching practices in open and distance learning is learning analytics (LA). It is a new term in education. LA tries to implement new tools to improve learning and teaching activities. LA is the measurement, collection, analysis and reporting of big data about learners in order to understand and optimize learning and learning environments (Aljohani and Davis, 2012a; Becker et al., 2018; Siemens, 2011).

The systematic approach of LA stems from a process of five steps (Baker, 2007). The first step is the collection of raw data. The interpretation of data leads to the creation of information. Knowledge is the third step of the process where information is analyzed and synthesized. The last step of the process is using the knowledge for a certain goal.

LA has four components that should interact with one another for an effective LA: Computers, people, theory and organization. Computers collect and store data gathered from users and can retrieve them whenever needed. Computers can also visualize data processing in two ways: dashboard and social network maps. Both computer system and outputs of LA can be meaningful if they are based on a sound theory. Although technological means have improved significantly, the processing of data and its visualization in a meaningful context still need humans' cognitive skills. People, computer and theory have sense if they constitute an organizational structure. The application of LA has a five-step process: Capture, report, predict, act and refine.

Figure 1 shows the five-step application process of LA. The overall aim of these steps is to have a prediction about the next step of the topic/ person. Predictive LA predicts the future outcomes of learning activities by analyzing educational data. The application of predictive LA takes places in the middle of the process. Predictive LA requires intervention either passively or actively by humans or automated systems (Alhadad et al., 2015). Thus, LA in education is not just a systematic data gathering; it is the intervention constructed according to the data analysis.

LA serves as a field that allows to search massive and multi-dimensional data scientifically (Fırat and Yüzer, 2016, p. 6). Such big data virtually covers every type of human interaction with the environment and helps to reveal not only perspectives but also future visions in both social and academic contexts since the big data collected can be considered as real-time representations of actual events which can be used for living the moment and later for planning the future (Rizzi, 2015). LA has an aim of improving the quality of learning within education borders by using data of education such as learner demographics, electronic grade data, grade point average, attendance data and LMS log data. This assessment process requires “investment in data collection and storage, development of context-sensitive algorithms and staff development” (Ferguson and Clow, 2017, p. 3). Although LA is a new assessment and evaluation system in education, it has already had connections with many other “fields of study including business intelligence, web analytics, academic analytics, educational data mining and action analytics” (Elias, 2011). Therefore, the integration of such an multidisciplinary tool into learning and teaching practices can lead to beneficial feedback and feedforwards. For example, the use of LA in education can enable instructors and academic advisors to evaluate the data gathered and improve the success of learners accordingly in a short time by gathering data about learners' interaction with other learners, their total time in online learning activities, the frequency and time of logging in.

Mobile devices can serve the needs of information societies by providing non-stop connectivity independent of place and time, extra capabilities such as information processing as well as contributiveness to each individual user as Grantham and Tsekouras (2004) suggested. In this context, mobile devices can also serve LA which focuses on the systematic collection, analysis and reporting of data about learners in order to optimize learning (Aljohani and Davis, 2012a; Becker et al., 2018; Siemens, 2011). However, the big data gathered from mobile learners has not been adequately examined in the literature and only a limited scope of research has been conducted to address this issue so far (Aljohani and Davis, 2012b; Fulantelli et al., 2013; Gaftandzhieva et al., 2018; Petropoulou et al., 2014; Shorfuzzaman et al., 2019).

It is obvious that a better understanding of the learner; that is knowing about his/her demographics, grades, attendance, log data, interaction, the time he/she spends in online learning activities, and his/her response to the interventions as well as learning designs would benefit learners and thus the educational institution's retention and success rate (Slade and Prinsloo, 2013). On the other hand, the ethical and privacy issues derived from LA have not been properly addressed (Pardo and Siemens, 2014). Actually, systematic collection of all data belongs to a specific learner and use of it may bring forward some ethical concerns such as privacy, violation of individual rights, informed consent; location and interpretation of the data; as well as classification and management of the data (Slade and Prinsloo, 2013). Being aware of ethics in LA is important for (1) designers to guide the implementation of LA platforms, (2) for learners to know about the procedures and their rights and (3) for instructors to have a more detailed account of the issues to address when adopting LA techniques (Pardo and Siemens, 2014).

Considering the limited number of research in the literature on the use of LA in mobile devices and on the ethics while applying LA, this paper tries to shed light into the visions of PhD students on the use of LA in mobile technologies and possible consequences of using them together. Five PhD students and one professor from Distance Education department of Anadolu University participated in this study. SCAMPER method, a focused brainstorming technique was used to collect data from these participants. The findings of this study are important since they may inform instructional designers, instructors and learners on both the good sides and the dark sides of the LA in mobile so that they can shape their future actions accordingly.

1.1 Literature review

There are various studies related to LA in literature. These studies often focus on improving learning environments by taking advantage of LA, as well as ways to be followed in this process. In addition, in the twenty-first century, also known as the digital age; the place of technology, its adaptation to learning environments, the use of mobile learning and LA in higher education are among the popular studies. In line with these, in this study the databases were scanned with the keywords “LA” and “mobile application”, and the current 14 studies published in 2012–2019 related to the focus topic were selected. These studies were reviewed and summarized within the scope of this research.

Aljohani and Davis (2012a) researched the potential advantages of using LA techniques to increase and facilitate learning in mobile and ubiquitous learning environments. The research was conducted using the Mobile and Ubiquitous Learning Analysis Model (MULAM) to analyze data from mobile learners based on the five-step analytical learning model of Campbell and Oblinger's at Southampton University, UK and King Abdulaziz University. The research focused on two structures, Mobile Learning Analytics (MLA) and Ubiquitous Learning Analytics (ULA), and was proposed as a model for future studies. In terms of MLA, the main source of data was mobile learners, while the ULA analyzed the knowledge of mobile learners. Then, ULA went one step further by analyzing contextual data gathered from mobile learners. It is possible to say that the structure and model proposed in this research is able to give direction to future researches within the scope of LA.

Aljohani and Davis (2012b) also enlightened the benefits of using LA to enrich mobile learning and extended learning environments. The study introduced the concepts of MLA and pervasive LA and resulted in instantaneous feedback of LA' greatest contribution to mobile and distributed learning environments. At the end of the study, it was seen that instant feedback provided to learners using the LA functioned effectively.

Kruse and Pongsajapan (2012) stated that LA is an essential element in education. According to the researchers, the problem is that designing oriented LA, which may even reveal hidden life at universities, may have negative consequences. Therefore, learners should benefit from LA, but this should be done individually. Learners should see the results of their own analysis and develop strategies. In summary, in this research the authors emphasized the importance of LA but recommended analytics to be used by learners.

Fulantelli et al. (2013) developed a new approach to the analysis of learners' activities in mobile learning environments, based on the semantic relationships posed in the Semantic Web paradigm and the Linked Open Data cloud. In this study, MeLOD was presented as a mobile learning environment for learning with explicit data linking and the potentials and difficulties of this approach were analyzed for both teachers and learners.

Petropoulou et al. (2014) introduced a new cloud-based assessment tool called LA Enhanced Rubric (LAe-R), which was developed as a Moodle plugin (version 2.2+). Teachers can use this tool to evaluate a learner's various basic skills and competencies using an improved version of an existing classical criterion. Again, thanks to LAe-R, teachers can add new criteria types in addition to traditional performance measures (e.g. the criteria associated with project outputs); collaborations, assessment reviews, as well as using learning and interaction analysis indicators.

Vallance et al. (2014) did their research to develop an iOS application called iFUN. In the research, there were nine participants; a technology specialist and field specialists at Hakodate University in Japan. Their focus was to develop university curriculum and to investigate the process of using it in multidisciplinary fields. With this practice, course designers can identify the strengths, gaps, mistakes and repetitions of the curriculum. iFUN is applied to predict and evaluate current and future curricula and contributes to learning processes of learners in various disciplines. For the application, course contents were transferred from Excel and Word documents and social network connections were added. By this way, learners were able to access local information, peer feedback, necessary web links and follow classes instantly. iFUN can suggest “ways of learning” individually and offers academic support. In addition, with its cost-effective and “cloud in-data” approach, the curriculum data can now update itself in real-time without being have to manually update the application.

Tabuenca et al. (2015) developed an application in which learners were tracked on how much time they spend using mobile learning tools. 36 learners using this application, taking three different online courses, had been observed for four months. Data was collected through two questionnaires in the process. The results of the study showed that “tracking time” had positive effects on time management. The researchers also investigated the effects of channel diversity and mobile notifications and content on promoting reflection applications. Findings shed light on how stimuli should be organized for learners in online lessons.

Arrigo et al. (2015) also pointed out the benefits of mobile learning and the complexities and challenges associated with mobile learning that accompany these benefits. Researchers stated that overcoming the complexity of mobile learning would provide better learning environments if LA were applied in this context. Therefore, it can be said that LA can facilitate mobile learning when it is used purposefully.

Chiappe and Rodriguez (2017) stated that teaching and learning in the twenty-first century moved to a higher dimension with the development of technology. In this study, researchers found that technology; especially use of mobile learning technologies in education determines the shape of the learning and teaching. In this context, 1,300 articles and nearly 100 related readings on the application of LA were reviewed. Accordingly, researchers have stated that LA is an indispensable form of evaluation of the data.

Wong (2017) explored how LA was used in higher education and the expected benefits of them for higher education institutions. Experimental research and case studies in the field of LA were conducted and the research details were classified. The results showed that LA helps institutions use their current data effectively in decision-making. Based on this information, it is possible to mention that the LA may contribute to the improvement and evaluation of learning pedagogies and instructional designs.

Kuhnel et al. (2017) tested the availability of the application called MyLA (My Learning Analytics), prepared by The Cooperative State University Mannheim and the University of Mannheim with the participation of two German universities. The researchers focused on supporting personalized and self-regulated learning. MyLA collected the data of learning behaviors, strategies and personality traits. 105 learners from two universities participated in the study. In addition to the quantitative usability survey, the application's navigation and design were evaluated through an eye tracking research with seven participants. The findings show that the use of the prototype MyLA was easy; however, some modifications to the design of the application were required. In addition, it was necessary to make arrangements especially on the color selection. It can be said that the prepared prototype was user friendly for the target group. The results were generally positive, suggesting that the combination of LA and mobile learning had a positive effect on the learning process.

Similar to other researches, as a result of his work Williams (2017) stated that university leaders will face important strategic decisions on how to implement and apply LA effectively. University leaders apply restrictive management and supervision and/or increase the educational and personal development of learners if they wish. If the academy is seen primarily as a place of personal and professional development through learning, then these applications of social LA are very comprehensive and transformative with the potential to support them. Social LA can be used effectively to support and evaluate collaborative learning of learners in realistic contexts. Following such a strategy means recognizing and agreeing with the changing environment in which the universities operate in the twenty-first century and the new professional environment of the graduates.

Also, in their study Gaftandzhieva et al. (2018) tried to provide solutions for some problems related to the technological considerations in LA including data collection, data storage, data processing and data reporting with the purpose of helping learners to meet their learning goals and to improve their success. Eventually, they purposed using the mobile application named Mobile LAP by which learners can keep track on their own activities, success rates, adherence to the learning schedules.

Shorfuzzaman et al. (2019) conducted a study in which they discussed limited processing capability of the mobile devices and to this end suggested to use a cloud-based mobile learning framework utilizing big data analytics technique to extract values from a big amount of mobile learners' data. To them, in the near future big data platforms such as Hadoop will be important to process both structured and unstructured data and mobile learning big data is going to use their proposed cloud framework to meet limited processing capability of the mobile devices.

Figure 2 summarizes the research studies conducted on LA and/or mobile learning between the years of 2012 and 2019. When the relevant literature is examined, researchers are likely to find a number of studies on LA and mobile learning. In these research studies, it has been observed that LA is often used to understand and improve learning environments. Johnson et al. (2011) stated that the most important contribution of LA is the enhancement of higher education. Universities will be able to define learners' learning needs more clearly and adapt instructions appropriately.

Although there is a strong relation between LA and mobile technologies, related literature is limited to explain this connection. More importantly, widespread use of mobile technologies opens new possibilities for LA in the future. The originality of this research comes from its focus on this critical point. This research aims to investigate the futuristic visions of PhD students on the use of LA and mobile technologies together.

To sum up, LA can make learning easier and learners can follow their persistence in learning. In addition, LA enables to predict learners' performance, identify undesirable learning behaviors and emotional states, ascertain and monitor learners at risk and provide appropriate help for learners. It can also stipulate learners with learning features that will make their learning experience more personal and engaging, encourage reflection and development and stronger descriptions of patterns (Wong, 2017).

1.2 Purpose

The purpose of this study is to generate future visions on the use of LA and mobile technologies together. Departing from this, a seven-stage vision research has been conducted through SCAMPER method.

2. Methodology

In this study, survey research design was adopted. This qualitative research study, designed in the single cross-section model, aimed to reveal futuristic visions of PhD students on the use of LA in mobile learning. In this respect, SCAMPER method, which is also known as a focused brainstorming technique, was used to collect data.

SCAMPER method is a brainstorming technique used for the production of new solutions or the transformation of the prior ideas into the new and creative ones. Even though this technique was first introduced by Alex Osborne in 1953, it was mostly improved by Bob Eberle. SCAMPER, also known as the guided brainstorming, helps revealing “intuitive, fictitious and creative potentials” (Yağcı, 2012). SCAMPER leads to the creation of useful and efficient ideas by the pursuit of predetermined stages. SCAMPER is acronym for seven stages: (S) Substitute, (C) Combine, (A) Adapt, (M) Modify/Minify/Magnify, (P) Put to other uses, (E) Eliminate/Elaborate, (R) Reverse/Rearrange. By the follow-up of these stages, this guided brainstorming process turns into a “creative, practical, and fun brainstorming technique” (Karataş and Tonga, 2016).

2.1 Participants

The participants of this study are five PhD students taking “Learning Analytics” course and the professor giving that course at Distance Education department of Anadolu University, Turkey in 2017–2018 academic terms. The professor is considered as one of the participants of this study because he/she acted as both facilitator and participant during the brainstorming. As the sampling method, purposeful sampling was applied to decide the target participants of the study for it is one of the most efficient sampling methods that can be used for qualitative research studies (Creswell and Creswell, 2005). In purposeful sampling, participants are chosen in line with the hypothesis and purpose of the study, and with the participants who will best serve the research, the study is conducted closely (Patton, 2002). Departing from this definition, field expertise, prior knowledge on and readiness for LA and mobile technologies and academic level were set as the criteria of purposeful sampling that will enable to conduct this study.

When it comes to demographics, the age of the participants ranges from 30 to 40. Three of the participants are male while three of them are female. All of the participants are working as instructors. Two of them are teaching at K12 while four of them are teaching at university. Even though all of them are PhD students at Distance Education department, their expertise fields as an instructor are different: Two of them are teaching at Distance Education Department at University, two of them are teaching at School of Foreign Languages at University, one of them is teaching English at K12, and one of them is teaching information technologies at K12.

2.2 Data collection

The research study was carried out in the seventh week of the “Learning Analytics” course. The reason behind this timing is to give participants enough time to generate visions since the relevant themes and studies on LA and mobile learning were just covered in the previous weeks. The data was collected through focused group interview which lasted for 2 h in total. The interview was conducted in participants' native language; that is Turkish. During the data collection period, SCAMPER method was used. The following questions were asked in each stage of SCAMPER:

  • (S) Substitute: Can we substitute LA and mobile with something else? Any examples come to your mind?

  • (C) Combine: Can we combine LA with mobile? If we combine these two, what can be the possible consequences?

  • (A) Adapt: With the combination of LA and mobile, what kind of products will appear? In the field of education, health, etc. …?

  • (M) Modify/Minify/Magnify: How can LA in mobile serve in horizontal and vertical plane?

  • (P) Put to other uses: For what other purposes can we use LA in mobile?

  • (E) Eliminate/Elaborate: Which of these can be eliminated or elaborated?

  • (R) Reverse/Rearrange: Reverse all your thoughts that you have shared so far. Suppose that using LA and mobile devices caused a lot of damages and did not work at all, what else can we use as an alternative to LA and mobile?

Through the brainstorming, the professor both acted as the facilitator and the participant. As a facilitator, he/she asked the aforementioned questions as well as some follow-up questions departing from the ideas presented by the students and as a participant, he/she also presented his own ideas and visions on the use of LA in mobile.

2.3 Data analysis

In the current study, Nvivo 11 package program was used to analyze the qualitative data collected through the focused group interview. This package program was chosen since it enables managing data, managing ideas, query data, visualizing data as well as reporting from the data all at once (Bazeley and Jackson, 2013). In the analysis process, the data was organized as themes, codes and nodes respectively and to visually represent the relationships among these items, concept maps were created. To keep the privacy of the participants in the study, each participant was given a code name and they were named after the cities around the world: Amsterdam, Bangkok, Budapest, Prague, Rome and Vancouver.

3. Results

Depending on the analysis of the qualitative data collected from the participants via the SCAMPER method; that is also known as a focused brainstorming meeting, in Nvivo 11, seven categories were brought up under the main theme of “visions for use of LA in mobile”.

As it can be seen in Figure 3, all nodes and codes are grouped under seven categories which are entitled as substitute, combine, adapt, modify–magnify–minify, put to other uses, eliminate–elaborate and reverse–rearrange. When looked closely, it will be seen that these seven categories also represent each stage of SCAMPER method. Following the analysis of the codes, four nodes came out under the first stage of SCAMPER method; that is “substitute”.

During this stage, participants were asked to suggest anything to substitute the concepts of “mobile” and “LA”. As it can be seen in Figure 4, participants put forward wearable technologies and Internet of things (IOTs) as the substitutes of mobile and big data as the substitute of LA.

Rome: Internet of things. There is an internet connection in every device. With internet connection, I mean the web. The possibility of accessing to data everywhere. If there were an internet or a web in every device, would we still need mobile?

On the other hand, when it comes to “big data”, participants made a warning considering the breadth and width of the content in LA.

Amsterdam: LA, as a concept, is so broad that even the big data is a part of it.

At the second stage of SCAMPER method, participants were asked whether they can combine LA and mobile and how they can combine both of them to find creative solutions. Seven nodes emerged out of the stage “combine”.

First, the stage of “combination of mobile and LA” was discussed in terms of big data from mobile, bringing data together under the same roof, as well as synchronous data analysis. As it is shown in Figure 5, during the brainstorming, participants suggested that it is a good idea to combine mobile and LA for; (1) nowadays we can get the deepest data from mobile devices, (2) scattered data makes it hard to access data and use it productively to solve problems and (3) it makes synchronous data analysis possible. Later, it was concluded that combination of mobile and LA may lead to creative solutions on three areas: life needs, personal learning and content and health problems.

Prague: Learning arises from the needs of humans. Real life needs. To full my stomach, I should learn how to cook. I need it. Via a mobile application, I can trace when I come home from work, what I need, how much energy I spend, what I need to compensate the energy I lost, and which education I should take on these aforementioned issues.

Rome: Personal learning and content will be encouraged when LA and mobile are combined together with learning.

Most of the creative solutions were proposed on the area of health. Participants made suggestions on combining LA and mobile for tracking heart beats and sleeping habits, sending notifications on trainings for a healthy diet, as well as recording every data on one's health to give feedback and feedforwards to users on their possible health problems.

Amsterdam: It is already recorded where we go via GPS. A training notification on how to make a salad may be sent to a person who goes to McDonald's.

Rome: … We have a serious disease but we don't know about it because data on our health is not recorded and analyzed continuously. Creative ideas such as an application which sends notifications to the people on their diseases, gives education to them on these diseases, tells them which health centers to go, where to find the best doctors, which medicines to take, and other examples of their cases, as well as organizes a meeting to them with a person with cancer and his family may be put into use through the analytics and big data.

In addition, participants drew attention on the importance of and priority in data matching. They stated that the content delivered to the users of the application can be personalized on the basis of some variables such as age, gender, location, economic background, health background and educational background.

Amsterdam: Consuming fat excessively has some effects on humans' health. The same statement can be expressed differently for the individuals graduated from primary school and high school. A notification like “Have you read this article?” can be sent to a PhD graduate.

In the third stage of SCAMPER method, the participants tried to provide examples of products which can come out of the combination of LA and mobile discussed in the previous stage. Examples from similar studies, cases and achievements were provided. As a result, for this stage, “adapt”, eight nodes emerged.

As it can be seen in Figure 6, a new certification system, a new education system, a new support system, new educational institutions and personal learning environments (PLEs) were proposed as the products of combination of LA and mobile.

Prague: Certification system may change. Instead of giving degrees on fields such as teaching and engineering, it can be said that this person has these credentials so he/she can work in these fields.

Amsterdam: In time student support systems will change. Not only in Open and distance learning but also in face-to-face traditional learning support, mentoring, orientation and consulting will need to change for learners. We talk about personalized education but still give the same support for every learner.

For the participants, the products from the combination of LA and mobile can also be used for determining educational needs of the learners and discovering their hidden interests.

Rome: Not only asking the person directly or seeing his/her family but also using the factual quantitative data it can be detected what the person really needs and what type of education he/she is suitable for.

Rome: When we reach sufficient data, we can support learning processes even learners don't know about.

In the fourth stage of SCAMPER method; that is modify, magnify, and minify, participants were asked to magnify and minify the main theme, the use of LA in mobile. They were also asked to go deeper and discuss the theme on the horizontal and vertical limits. Ten nodes emerged out of the discussions.

As Figure 7 shows, direct learning through chips, device integration into neural system, microdevices in blood and smart dust were the ideas introduced in vertical plane.

Budapest: Smart dust is a good idea. Think about a person swallowing a tiny particle of dust. When it enters this person's body, it can detect all actions including this person's hormonal and mental states.

In the horizontal plane, continuous recording, data collection from the setting and people around the individual, data coming from things apart from human (nonhuman data), uncovering hidden potentials and tendencies and uncovering subconscious were suggestions put forward in the process of magnifying the visions for the use of LA in mobile. At this stage, it was asserted that if a person's data is recorded continuously from the childhood on, we can know everything about this person, even his/her hidden potentials, tendencies and subconscious.

Rome: … We can collect data from the childhood on and use them on behalf of detecting learners' needs and tendencies. When we reach sufficient data, maybe we can support learning processes, even the individual doesn't know himself.

Budapest: If the application is developed more, it is a bit frightening but like the psychologists who uncover subconscious by talking to their patients, we can also reveal learners' subconscious by making queries and orienting them with the help of applications such as Siri.

The fifth stage of SCAMPER method; that is put to other uses, aimed to find an answer for the question “For what other purposes can we use LA in mobile?” During the creative meeting, five nodes emerged for the stage “put to other uses”.

As it can be seen in Figure 8, participants stated that we can use LA in mobile for (1) the resolution of societal cases such as terrorist attacks in Turkey, (2) connection with the people who use the same application with you in a particular circle such as a workplace, (3) predicative learning, to take a precaution before something happens, (4) the introduction of best ways to follow to handle a problem and (5) learning at any time any place like learning with family, friends or while walking.

Vancouver: Predicative learning … Depending on the analysis, all the possible consequences may be listed before taking a step for an action.

In the sixth stage of SCAMPER method, eliminate and elaborate, participants were asked to decide on which is necessary and which is not. For example, they were asked whether they need big data to conduct all these analyses or to what extent they need evaluation and assessment or a teaching design. In this stage, five nodes emerged.

As Figure 9 points out, at this stage, participants proposed: (1) a new generation social media in which people socialize via the networks, not via a specific web-page or web-application as an alternative for regular social media, (2) web 2.0 or social media as an alternative for data mining, (3) LA or big data as an alternative for evaluation and assessment, as well as (4) personalized learning philosophies and theories as an alternative for regular pedagogy or andragogy.

Rome: … Profile will like profile. Our free profile which is registered by the government. This profile will both record the real people you come across and his/her likes. It is transforming something far beyond social media, isn't it?

Budapest: There will be no need for a defacto theory because personalized learning philosophies will emerge. There will be personalized pedagogies.

However, some participants believed that in the future we will still need a pedagogy or andragogy. Maybe the current pedagogy or andragogy will change and evolve or new ones will emerge. But anyway, they will sustain their existence.

Amsterdam: Pedagogy and andragogy will evolve but still they will be in use as long as humans exist and learn.

At the last stage of brainstorming meeting, participants tried to reverse and rearrange what they have talked about so far. They stated ideas on what may be the negative consequences of the use of LA in mobile. Nineteen nodes emerged out of the stage, reverse and rearrange.

Participants of the study were extremely cautious about the use of LA in mobile as it can be seen from Figure 10. First, they had some concerns on the control of the big data. They stated that the recorded data may be used by the powerful people to dictate the powerless.

Prague: … Who will control the recorded data? Will he/she use it for his own benefit or for humans' benefit? The power leads people to use it for their own benefits.

Furthermore, participants asserted that the big data collected in LA may be used to access to the personal and private data on people and this may end up with interference into individuals. For them, LA may be used for the manipulation of perception and mind.

Budapest: We expose to people as long as we like. We have an option to spend time at home alone when we don't want to spend time with people. But what would happen if there were consistent notifications coming from the electronic devices I had when I was alone? I don't see a person. The end! But what about electronic devices? They will constantly send notifications. I may ignore the first notification, the second, but what about the third?

Prague: In the elections, they manipulate people and their perceptions. For example; big five … They can take something you have by force.

Also, participants were a bit concerned on the ethical issues. They thought that the use of LA in mobile may result in the loss of humanity.

Vancouver: Humans will lose their humanity. They will be robots because everything we do will be public. We will no longer have secrets.

On the other hand, participants were also optimistic. They believed that people learn from their past experiences and use them to head forward for a better future. For them, chaos may end up with a new beginning.

Rome: We know that not only humans but also societies learn. Departing from a negative experience we can learn something and step into the next level.

Budapest: … We need to have societal chaos so that we can start something new from it. Every chaos brings a new beginning …

All in all, a variety of nodes came out in each stage of SCAMPER brainstorming process.

As it can be seen in Table 1, Stage 7, that is reverse/rearrange, has been the stage in which participants shared the most visions by far. At this stage, the participants made some prophecies on what would happen if something went wrong. They shared a lot of disaster scenarios all of which show the necessity of considering ethics while using LA in mobile.

Finally, each node was analyzed in terms of number of references it took.

As it can be seen in Figure 11, importance and priority in data mining, interference into individual, a new certification system, continuous recording, uncovering hidden interests, new generation social media, feedback-feedforward and solutions on health have been the mostly discussed issues among participants. Departing from these references, it can be concluded that participants focused on benefits of the use of LA in mobile rather than the dangers it brings with it.

4. Discussion

Studies in the related literature showed that LA has multidisciplinary qualities (Siemens, 2013) and can be used in mobile applications and devices (Aljohani and Davis, 2012a; Arrigo et al., 2015; Fulantelli et al., 2013; Tabuenca et al., 2015). In this respect, the present study aimed to explore the visions on the potential use of LA in mobile. According to the findings from SCAMPER method, participants believe that the use of LA in mobile has a lot of benefits, which correlates with the findings from two studies conducted by Aljohani and Davis (2012) examining the possible benefits of using LA techniques to enhance mobile and pervasive learning environments, as well as discussing potential benefits of using LA techniques.

The participants of this study proposed LA as a key element to initiate a proper and ideal twenty-first century education as stated by Chiappe and Rodriguez (2017). For them, the use of LA in mobile is a result of and must for today's information societies. The reason behind this idea can be their beliefs on the rise of a new certification system, a new student support system and a new education system owing to the use of LA in mobile, which will eventually lead to the emergence and development of new educational institutions like the ones delivering open and distance learning. Additionally, in this new generation educational systems, personal learning and content and PLEs will gain importance. To the place of a specific pedagogy or andragogy, there will be personalized theories and philosophies that fit each learner and application of a student-centric, inquiry-based model of analytics will put the tools and premises of analytics into the hands of learners and empower them as metacognitive agents of their own learning (Kruse and Pongsajapan, 2012). The findings of this study also foresee that there will be a change in evaluation and assessment system. Evaluation and assessment can be considered as a component of LA. In other words, LA naturally involves evaluation/assessment within itself, so it allows simultaneous data collection and assessment. This finding correlates Williams (2017)'s study, which suggests that social LA provides an opportunity for authentic assessment and therefore authentic learning. Authentic assessment means evaluating learners' collaborative learning in realistic contexts, so a mobile application involving LA may be the very thing to evaluate the big data collected from the learners continuously and to provide them feedback on the learning processes they have undergone so far and at the end to feedforward the experts, instructors, learners and the application itself on what actions or steps to take next. Thanks to this feedforward, the learning process will be time, money and effort efficient, three of which are among the main philosophies open and distance learning is based on.

Moreover, the findings of the present study show that LA can be used in mobile to meet life needs, health needs and educational needs. Learning arises out of the needs of humans. A mobile application involving LA can detect the everyday problems such as traffic jam, pollution, global warming and waste of natural resources. This application can also inform people on their health by recording them continuously. In addition, this application may provide suggestions on education by finding out the fields and subjects a person should work on, proposing the best ways that are the techniques and strategies to access information in these subjects and providing personalized experiences during these learning processes. The big data is the thing which makes application of aforementioned ideas possible. In this study, during the brainstorming session big data was suggested as a replacement for LA by the participants. In the related literature, it is emphasized that both fields, that are big data and LA, require multidisciplinary components (Chatti et al., 2012). Even though LA has emerged with the rise of big data (Mishra et al., 2015), it is believed that in terms of its depth and breadth LA is a more comprehensive field even embracing the big data.

On the other hand, the findings of this research study also revealed the dark sides of the use of LA in mobile. For the participants, the use of LA in mobile is threatening the privacy of individuals by accessing into personal and private data, recording it continuously, uncovering subconscious and even worse intervening into individuals and manipulating their perceptions and minds. In a way, it is wise to handle LA cautiously and critically and to keep in mind the extract “Just because it is accessible does not make it ethical” (Boyd and Crawford, 2011, p. 10).

When it comes to ethics, the participants of this study also questioned the possible damage LA can give to the human nature. LA is based on numbers and algorithms that can extract and inform us on massive patterns in human behavior. Therefore, as a result of the use of LA in mobile applications, learners may face the danger of losing their humanity and may turn into a kind of robots.

Another concern came out in this study on the use of LA in mobile is “Who will control all these big data collected from learners for a large scope of time?” Such high-tech innovation, that is the use of LA in mobile, is an inquiry of values and power including instrumental, cultural and even symbolic (Slaughter, 2018). In the world, there has always been class-based differences such as the powerful and the powerless, the rich and the poor, the royal or the peasant. And the current ecosystem around big data – LA creates a new divide: the big data rich and the big data poor (Boyd and Crawford, 2011). Manovich (2011) states that there are three classes of people in the realm of big data: those who create data (both consciously and by leaving digital footprints), those who have the means to collect it and those who have expertise to analyze it' (as cited in Boyd and Crawford, 2011). It is clear that in today's society the last class is the most powerful one. Actually, this class represents LA since it has the right not only to collect the data but also to analyze and use it. In other words, this class is the one which can control the data alone and even use it to have a control on the other classes or in this context, the learners. However; departing from the philosophy “there is good in every bad”, the participants of the present study also put forward positive visions on the use of LA in mobile and stated that owing to LA, we can collect big data on the prior negative experiences, analyze them to learn deeply about these experiences and use them to head forward more strongly.

5. Conclusion

The era of LA has begun. In this particular era, the definition of learning is changing. Learning has become a nonlinear, complex and dynamic system, so to record big data on learners and analyze them gained more importance. The record and analysis of the data to this huge extent can only be achieved by means of mobile technologies such as iPads, laptops, tablet PCs, PDAs and smartphones with wireless networks since they enable mobility and mobile learning and allow teaching and learning to extend to spaces beyond the traditional classroom (Educause, 2018); that is to open and distance learning. Departing from these, the present study aimed to explore the futuristic visions of five PhD students and a professor in the department of Distance Education on the active use of LA in mobile. Several conclusions were drawn from this study.

This study provides a strong ground for the active use of LA in mobile, for the use of LA in mobile can solve everyday problems ranging from health to education. The data on health, education and environmental problems may be recorded consistently and used as a remedy for these problems. Also, the collection of large amounts of data, big data, can help educators and system makers to identify patterns which will enable tailored education for each individual. By this way, pedagogy and andragogy can break their chains; become free from “one-size-fits-all” principles. Rather than trying to fit each learner into a ready-made learning philosophy or theory, an individualized philosophy can be fit into the learner. Further, the use of LA in mobile may encourage a new type of evaluation and assessment. LA in mobile is equipped with all necessary elements (e.g. big data, continuous recording, statistics) to assess a learner's performance frequently and to make adjustments accordingly. LA in mobile not only allows the assessment of the learners but also the system itself by providing feedback and feedforwards. Considering all, LA in mobile can be proposed as the very thing to serve new generation educational institutions in the twenty-first century digital societies.

However, the findings of the present study should be treated with caution, as the use of LA in mobile can threaten the privacy of the learners. The paradox here is that the big data collected and analyzed can be used for the benefits of the learners but at the same time they can be used by the big data rich to interfere into the individual, reach their subconscious and even manipulate them. In this case, while using LA in mobile, the ethical issues should also be kept in mind and there should always be a back-up plan in case anything goes wrong since new technologies are never free; they always come with a price such as weaknesses and costs, winners and losers (Slaughter, 2018). Knowing about the ethics is important, for it may increase the awareness of the designers, instructors and learners on the implementation of LA in mobile and provide visions on what can go wrong before it is too late. Therefore, ethical issues regarding the use of LA in mobile can specifically be addressed in the future research. Another reason to treat the findings of this study with caution is the limited number of the participants. There are only six participants of this study which makes it harder to draw general conclusions from it. Also, the sample of the study is homogeneous in terms of their backgrounds – their age range, their departments as PhD students and their fields of expertise, so a further study can be conducted on a wider scope of sample since it may inform us on different perspectives, ideas, thoughts and visions on the use of LA in mobile.

Figures

The five-step application process of LA

Figure 1

The five-step application process of LA

The research studies conducted on LA and/or mobile

Figure 2

The research studies conducted on LA and/or mobile

Seven categories of “visions for use of LA in mobile”

Figure 3

Seven categories of “visions for use of LA in mobile”

The nodes that came out in substitute stage

Figure 4

The nodes that came out in substitute stage

The nodes that came out in combine stage

Figure 5

The nodes that came out in combine stage

The nodes that came out in adapt stage

Figure 6

The nodes that came out in adapt stage

The nodes that came out in modify/magnify/minify stage

Figure 7

The nodes that came out in modify/magnify/minify stage

The nodes that came out in Pt to other uses stage

Figure 8

The nodes that came out in Pt to other uses stage

The nodes that came out in eliminate/elaborate stage

Figure 9

The nodes that came out in eliminate/elaborate stage

The nodes that came out in reverse/rearrange stage

Figure 10

The nodes that came out in reverse/rearrange stage

The most discussed issues among participants

Figure 11

The most discussed issues among participants

A summary of the nodes that came out in each stage

Visions for use of LA in mobile
SubstituteCombineAdaptModify/Magnify/MinifyPut to other usesEliminate/ElaborateReverse/Rearrange
Wearable technologiesBig data from mobileA new certification systemContinuous recordingAnalysis of societal casesHolding current pedagogy and andragogyPolitical games
IOTBringing data together under the same roofA new education systemData collection from the setting and people around the individualCreation of a social networkNew generation social mediaRules and boundaries
Big dataImportance and priority in data matchingA new support systemData coming from things apart from human (nonhuman data)Feedback and feedforwardTo the place of data miningTransformation into a consumer society
Breadth and width of contentLife needsEducational needsDirect learning through chipsGuidance and counsellingTo the place of evaluation and assessmentUnstable human nature
Personal learning and contentLearning materials on the basis of sleeping habitsIntegration into neural systemLearning at any time any placeTo the place of pedagogy and andragogyWinning a war/ earning money
Solutions on healthNew educational institutionsMicrodevices in blood A new beginning from chaos
Synchronous data analysisPLEProduction Access into personal and private data
Uncovering hidden interestsSmart dust Change in human
Uncovering hidden potentials and tendencies Cognitive learning
Uncovering subconscious Control of data only by one power
Control of the powerless by the powerful
Exploitation of the application
Humans losing their humanity
Impossibility of control of the data only by one power
Impossibility of undoing destruction
Interference into individual
Learning from past and heading forward
Manipulation of perception and mind
Overlap of reality and virtual

References

Alhadad, S., Arnold, K., Baron, J., Bayer, I., Brooks, C., Little, R., Rocchio, R., Shehata, S. and Whitmer, J. (2015), “The predicative learning analytics revolution: leveraging learning data for student success”, available at: https://library.educause.edu/∼/media/files/library/2015/10/ewg1510-pdf.pdf (accessed 9 May 2018).

Aljohani, N.R. and Davis, H.C. (2012a), “Learning analytics in mobile and ubiquitous learning environments”, available at: https://eprints.soton.ac.uk/342971/1/mlearn2012_Full_Paper-_Aljohani_and_Davis.pdf (accessed 12 May 2018).

Aljohani, N.R. and Davis, H.C. (2012b), “Significance of learning analytics in enhancing the mobile and pervasive learning environments”, 2012 6th International Conference on Next Generation Mobile Applications, Services and Technologies, IEEE, Paris, pp. 70-74.

Arrigo, M., Fulantelli, G. and Taibi, D. (2015), “Challenges of using learning analytics techniques to support mobile learning”, 11th International Conference on Mobile Learning 2015, International Association for Development of the Information Society, Madeira, pp. 147-149.

Baker, B.M. (2007), “A conceptual framework for making knowledge actionable through capital formation”, Doctoral dissertation, University of Maryland University College, College Park, MD.

Bazeley, P. and Jackson, K. (2013), Qualitative Data Analysis with NVivo, Sage Publications, New York, NY.

Becker, S.A., Brown, M., Dahlstrom, E., Davis, A., DePaul, K., Diaz, V. and Pomerantz, J. (2018), NMC Horizon Report: 2018 Higher Education Edition, EDUCAUSE, Louisville, CO.

Boyd, D. and Crawford, K. (2011), “Six provocations for big data”, available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1926431 (accessed 12 May 2018).

Chatti, M.A., Dyckhoff, A.L., Schroeder, U. and Thüs, H. (2012), “A reference model for learning analytics”, International Journal of Technology Enhanced Learning, Vol. 4 No. 6, pp. 318-331.

Chiappe, A. and Rodríguez, L.P. (2017), “Learning Analytics in 21st century education: a review”, Ensaio: Avaliação e Políticas Públicas em Educação, Vol. 25 No. 97, pp. 971-991.

Creswell, J.W. and Creswell, J.D. (2005), “Mixed methods research: developments, debates, and dilemmas”, Research in Organizations: Foundations and Methods of Inquiry, Vol. 1, pp. 315-326.

Educause. (2018), “Mobile learning”, available at: https://library.educause.edu/topics/teaching-and-learning/mobile-learning (accessed 13 May 2018).

Elias, T. (2011), “Learning analytics: definitions, processes and potential”, available at: http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf (accessed 13 May 2018).

Ferguson, R. and Clow, D. (2017), “Learning analytics: avoiding failure, EDUCAUSE review”, available at: https://er.educause.edu/articles/2017/7/learning-analytics-avoiding-failure (accessed 15 May 2018).

Fulantelli, G., Taibi, D. and Arrigo, M. (2013), “A semantic approach to mobile learning analytics”, Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality, Association for Computing Machinery, New York, pp. 287-292.

Fırat, M. and Yüzer, V. (2016), “Learning analytics: mass of big data in distance education”, International Journal on New Trends in Education and Their Implications, Vol. 7 No. 2, pp. 51-63.

Gaftandzhieva, S., Doneva, R., Petrov, S. and Totkov, G. (2018), “Mobile learning analytics application: using students' big data to improve student success”, International Journal on Information Technologies and Security, Vol. 10 No. 3, pp. 53-64.

Grantham, A. and Tsekouras, G. (2004), “Information society: wireless ICTs' transformative potential”, Futures, Vol. 36 No. 3, pp. 359-377.

Johnson, L., Smith, R., Willis, H., Levine, A. and Haywood, K. (2011), The 2011 Horizon Report, The New Media Consortium, Austin, TX.

Karataş, S. and Tonga, E.S. (2016), “Scamper (yönlendirilmiş beyin fırtınası) tekniğinin kullanımına yönelik öğretmen adaylarının görüşleri”, Eğitim ve Öğretim Araştırmaları Dergisi, Vol. 4 No. 1, pp. 57-68.

Kruse, A.N.N.A. and Pongsajapan, R. (2012), “Student-centered learning analytics”, available at: https://cndls.georgetown.edu/m/documents/thoughtpaper-krusepongsajapan.pdf (accessed 2 May 2018).

Kuhnel, M., Seiler, L., Honal, A. and Ifenthaler, D. (2017), “Mobile learning analytics in higher education: usability testing and evaluation of an APP prototype”, Interactive Technology and Smart Education, Vol. 15 No. 4, pp. 332-347.

Manovich, L. (2011), “Trending: the promises and the challenges of big social data”, Debates in the Digital Humanities, Vol. 2 No. 1, pp. 460-475.

McQuiggan, S., McQuiggan, J., Kosturko, L. and Sabourin, J. (2015), Mobile Learning: A Handbook for Developers, Editors and Learners, SAS Institute, North Carolina, John Wiley & Sons.

Mishra, P., Koehler, M.J. and Greenhow, C. (2015), Handbook of Educational Psychology, 3rd ed., Taylor and Francis, New York, NY.

Pardo, A. and Siemens, G. (2014), “Ethical and privacy principles for learning analytics”, British Journal of Educational Technology, Vol. 45 No. 3, pp. 438-450.

Patton, M.Q. (2002), “Qualitative research and evaluation methods in: Suri H. (2011) purposeful sampling in qualitative research synthesis”, Qualitative Research Journal, Vol. 11 No. 2, pp. 63-75.

Petropoulou, O., Kasimatis, K., Dimopoulos, I. and Retalis, S. (2014), “LAe-R: a new learning analytics tool in moodle for assessing students' performance”, Bulletin of the IEEE Technical Committee on Learning Technology, Vol. 16 No. 1, p. 1.

Rizzi, F. (2015), “Introduction: sustainability in a data rich world”, Futures, Vol. 74 No. 1, pp. 78-79.

Shorfuzzaman, M., Hossain, M.S., Nazir, A., Muhammad, G. and Alamri, A. (2019), “Harnessing the power of big data analytics in the cloud to support learning analytics in mobile learning environment”, Computers in Human Behavior, Vol. 92 No. 1, pp. 578-588.

Siemens, G. (2011), “Learning and academic analytics”, available at: http://www.learninganalytics.net/?p=131 (accessed 5 August 2011).

Siemens, G. (2013), “Learning analytics: the emergence of a discipline”, American Behavioral Scientist, Vol. 57 No. 10, pp. 1380-1400.

Slade, S. and Prinsloo, P. (2013), “Learning analytics: ethical issues and dilemmas”, American Behavioral Scientist, Vol. 57 No. 10, pp. 1510-1529.

Slaughter, R.A. (2018), “The IT revolution reassessed part three: framing solutions”, Futures, Vol. 100 No. 1, pp. 1-19.

Tabuenca, B., Kalz, M., Drachsler, H. and Specht, M. (2015), “Time will tell: the role of mobile learning analytics in self-regulated learning”, Computers & Education, Vol. 89 No. 1, pp. 53-74.

Vallance, M., Murayama, H. and Sperling, D. (2014), “A multi-disciplinary approach developing a mobile curriculum mapping app called iFUN”, Bulletin of the IEEE Technical Committee on Learning Technology, Vol. 16 No. 1, pp. 10-13.

Viberg, O. (2015), “Design and use of mobile technology in distance language education: matching learning practices with technologies-in-practice”, Doctoral dissertation, Repro, Örebro universitet, Örebro.

Williams, P. (2017), “Assessing collaborative learning: big data, analytics and university futures”, Assessment & Evaluation in Higher Education, Vol. 42 No. 6, pp. 978-989.

Wong, B.T.M. (2017), “Learning analytics in higher education: an analysis of case studies”, Asian Association of Open Universities Journal, Vol. 12 No. 1, pp. 21-40.

Yağcı, E. (2012), “Yönlendirilmiş beyin fırtınası tekniği: scamper konusunda veli görüşleri üzerine bir çalışma”, Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, Vol. 43, pp. 485-494.

Corresponding author

Mehmet Fırat can be contacted at: mfirat@anadolu.edu.tr

Related articles