Designing mobile learning with education outside the classroom to enhance marine ecological literacy

1. Introduction

The aim of this study was to explore how purposeful educational design using mobile learning might enable integration of classroom and outside of classroom teaching and learning. The context for the study was marine conservation with a goal to enhance the ecological literacy (ecoliteracy) of primary school students and their parents, and by extension to promote sustainable communities.

There is growing evidence that Information and Communication Technology (ICT) has the potential to enhance learning, increase knowledge, and promote transformative changes in the attitudes and behaviour of both individuals and the broader community (Aguayo, 2014; Becta, 2009; Somekh, 2007). Within ICT, today’s mobile learning technologies (e.g., smartphones, tablets) have multiple potential positive impacts for teaching and learning. Most importantly, they allow for learning processes to occur practically anywhere in collaboration with anyone (Cochrane et al., 2013; Pachler, Bachmair, & Cook, 2010). They also promote innovative (Parsons, 2013), inclusive (Traxler, 2010), and transformative (Lindsay, 2015) types of learning that challenge traditional teaching and learning approaches (Merchant, 2012; Cochrane, 2014). The content can be shaped to fit individual characteristics and needs (Aguayo, 2016), through self-determined and real-life learning, and within user/ learner-generated content and contexts, an approach known as heutagogy (Hase & Kenyon, 2013; Luckin et al., 2010; Narayan & Herrington, 2014).

Recent mobile learning research has emphasised new patterns of connected social learning, and the designing of research around the transformative possibilities of digital tools (Cook & Santos, 2016). We recognise that learning in the 21st century can occur practically anywhere at any time, in school and out, between students, teachers, non-formal educators, and parents/adult learners. Twenty-first century learning outcomes also focus on enhancing access to knowledge and promoting organic and distributed social learning throughout the community, with an awareness that learning can be influenced by technological innovations and affordances (Aguayo, 2014; Aguayo, 2016; Bull, Petts, & Evans, 2008; Pachler et al., 2010). We also recognise that the sociocultural context provided by educational settings and those within them can build on, disrupt, and challenge the personal constructs learners bring to those settings (Aguayo, 2016; Rennie & Johnston, 2004). A heutagogical approach mediated by mobile technologies could then act to create connections and reinforce learning between classroom and outside classroom experiences.

Outside the classroom educational settings, such as zoos, aquariums, museums, and visitor centres, offer unique learning experiences for relevant, context-based education (Ballantyne & Packer, 2011; Boyer & Roth, 2005). Learning in these “free-choice” settings can be described as either non-formal, in which a structured, guided educational programme is provided in which learners choose to participate, or informal, in which unstructured, unguided learning opportunities are provided. The learning then may be governed by individuals’ or groups’ needs and motivations (Brookfield, 1986; Falk & Dierking, 2002). These settings also provide important experiential learning opportunities that can complement learning within formal (i.e., school) contexts (Falk, 2005). Contexts such as nature-based visitor centres can offer the opportunity to complement and reinforce outdoor experiences by promoting reflection and meaning-making processes around socio-ecological issues (Ballantyne & Packer, 2005, 2011). This education outside the classroom (EOTC) can meet objectives of The New Zealand Curriculum (Ministry of Education, 2007), in this study around science and sustainability within the local environment, that can develop student ecoliteracy (Ministry of Education, 2016; Warner, Eames, & Irving, 2014).

An ecologically literate citizenry is fundamental to addressing the profound and complex global challenges and social transformations taking place worldwide (Hackmann & St Clair, 2012; United Nations Educational, Scientific and Cultural Organization [UNESCO], 2006). For example, our marine environment is under threat from overfishing, food web changes, pollution, land use changes, and climate change (Royal Society of New Zealand, 2016). Education is critical in creating a more ecoliterate citizenry (Orr, 2004; United Nations, 2012). Ecoliteracy requires more than just knowledge development (Hollweg et al., 2011; Kollmuss & Agyeman, 2002); it also includes experiences that can lead to learning and attitudinal development, and both knowledge and attitudes are intrinsically linked to the action required to bring about change for sustainability (Jensen & Schnack, 1997).

Significant life events research has indicated the importance of time spent in nature in influencing proenvironmental behaviour in later life (Liddicoat & Krasny, 2013), and place-based research has illustrated the importance of (re)-connecting people to places to support adoption of sustainable ways of living (Gruenewald, 2003; Penetito, 2009; Smith, 2007). However, as noted by Ballantyne and Packer (2011), research also indicates that although visitors often leave a “free-choice” learning setting with enhanced knowledge and an intention to adopt more environmentally-friendly behaviour, only a small number of visitors actually do so. Ballantyne and Packer (2005, 2011) stress the need for further research to:

• develop empirically-based principles for the design of learning experiences in non-formal settings that optimise learning for sustainability
• develop ways of complementing and reinforcing the free-choice learning experience with post-visit action resources.

Recent studies into the use of post-visit action resources have illustrated that the use of mobile technologies can stimulate students to remain engaged with an inquiry begun through a museum visit (Delen & Krajcik, 2017) and nature tourism visitors to remain engaged through social media with the State Park they visited (e.g., Wheaton et al., 2015). These studies recommend further research into the use of digital technologies for educating and supporting visitors during their experiences and to encourage sustained commitment to proenvironmental behaviour.

Developments in digital learning technologies that emerged during this project led us to choose Mixed Reality (MR) as an approach to enhance learning. MR is seen as an immersion continuum between experiences in the real environment and the digital environment (Aguayo, Cochrane, & Narayan, 2017; Akçayır & Akçayır, 2017; Milgram & Kishino, 1994). We sought to design a framework for using an MR approach in the classroom, outside the classroom, and between the two settings, to enhance teaching and learning to promote development of student ecoliteracy.

This study was based on a marine reserves learning experience at a primary school which connected with a marine science and conservation visitor centre. Our partnership brought together a “bring your own device” (BYOD) primary school classroom teacher and a class of her senior students (Years 5–8), educators from the marine reserve visitor centre, a marine scientist, a learning design/technology team, and researchers experienced in science and sustainability education. This partnership constituted the REEF (Research into Ecoliteracy Enhancement Forum). Our aim was to co-create and trial a mobile learning intervention that could enhance ecoliteracy for students and their parents.

The study addressed the question:

How can mobile learning be designed with education outside the classroom to enhance the ecological literacy of students and their parents?

This question was addressed through four specific sub-questions:

1. What are the ecoliteracy outcomes for students in response to a teaching unit on marine reserves incorporating EOTC?
2. How does the REEF work together to co-construct a framework for mobile learning in EOTC?
3. What elements and design principles does the REEF recommend for a framework for mobile learning to complement a teaching unit incorporating EOTC?
4. How does an intervention informed by a mobile learning framework contribute to ecoliteracy and other outcomes for students and their parents?

2. Research design

This 2-year study used a design-based research (DBR) methodology (Amiel & Reeves, 2008; The Design-Based Research Collective, 2003) to produce design principles for mobile learning integrated with EOTC, and to test technological solutions in practice. This allowed us to incorporate the unpredictable, dynamic, and innovative nature of learning technologies along with the emergence of new options such as MR, while accounting for learning processes based on a marine reserves unit incorporating an outside the classroom visit. It involved one primary school class learning about marine reserves, which included a visit to the Goat Island Marine Reserve in North Auckland and the associated Goat Island Marine Discovery Centre (the “Discovery Centre”) hosted by the University of Auckland.

In Phase 1 in early 2017, the teacher planned and implemented a marine reserves unit to her class of 20 students (age range 8–12). The unit included a visit to the Goat Island Marine Reserve and a visit to the Discovery Centre. Data collection included pre-unit and post-unit student questionnaires exploring ecoliteracy (knowledge about marine ecology and marine reserves, attitudes towards marine reserves, and intentions to act in relation to marine reserves), and attitudes towards, and use of, mobile learning technology (if any) during the unit; observations (either in person or video) of the Discovery Centre educator, teacher, student, and parent behaviour during the visit; post-visit focus groups with students and interview with the Discovery Centre educators; and post-unit focus groups with students and interviews with the teacher and parent helpers. Data analysis and review by the REEF team responded to sub-question 1 and provided a baseline to inform development of a mobile learning framework.

In Phase 2, the REEF team then worked collaboratively face to face, and online using a Google+ community, to design a mobile learning framework. This framework drew on the data evidence from the marine reserves unit and a number of theoretical positions:

1. theories from environmental and sustainability education, which recognise the importance of constructivist and place-based knowledge and attitudes, motivation to act for a sustainable future, and post-visit learning reinforcement
2. theories of mobile learning, which recognise the inclusive and transformative potential of mobile devices and their affordances within self-determined learning environments (i.e., student-generated contents and contexts, or heutagogy), the strategic importance of BYOD settings and use of freely available and/or low cost development of mobile learning technologies, and the emergence of collective and distributed learning through mobile-based technologies.

Data were gathered through observations at two face-to-face meetings, and the online posts, and were thematically analysed to respond to sub-question 2 about how the REEF worked together to co-construct the framework and sub-question 3 about the elements of the framework.

In Phase 3, this co-constructed mobile learning framework was then used to inform the design of an MR intervention for the same unit to be taught by the same teacher with her 2018 class, including a visit to the marine reserve and Discovery Centre. This design and development involved a continuous loop between: (1) collaborative refinement of practical problems; (2) development of solutions informed by existing theory and technological innovation; (3) iterative cycles of testing and implementing solutions in practice; and (4) reflection to produce design principles and enhance solution implementation (Amiel & Reeves, 2008) (see Figure 1). This process involved user-informed design that drew on the ideas of the students, the needs of the teacher and the Discovery Centre educators, and the infrastructure available at the visit setting.

In Phase 4 in early 2018, the teacher planned and implemented a marine reserves unit again to her class of 28 students (age range 8–12), this time incorporating an MR-based intervention (see 3.3 below for details). As this was a multi-age class, approximately 50% of the students had been in the same class the year before and had experienced the marine reserves unit. The unit again included a visit to the Goat Island Marine Reserve and a visit to the Discovery Centre. Data collection again included pre-unit and post-unit student questionnaires exploring ecoliteracy, and attitudes towards, and use of, mobile learning technology during the visit and the wider unit; observations (either in person or video) of the Discovery Centre educator, teacher, student, and parent behaviour during the visit; post-visit focus groups with students and interview with the Discovery Centre educators; and post-unit focus groups with students and interviews with the teacher and parent helpers. Data analysis and review by the REEF team responded to sub-question 4 regarding outcomes of the MR intervention.

Data analysis was structured around a sociocultural activity theory analytical framework, as described by Engeström (1987), and Krasny and Roth (2010), and as previously used by the researchers (Aguayo, 2014, 2016) in the context of using digital learning technologies for community education for sustainability. This involved the identification and description of the overarching elements constituting the activity systems in the study. These elements included the tool that was mediating the system, the subjects of the system and what was trying to be achieved, the sociocultural and historical background of the target community, the roles of actors within the activity system, and the rules that affected the activity system (Aguayo, 2016; Leadbetter, 2005). Each part of the study was recognised as a novel activity system, which in turn interacted with the following activity system.

Within this framework, data were analysed for their deductive coherence with the theoretical positions described above, and inductive emergent properties. Quantitative data (from the pre- and post-unit questionnaires) were processed using Microsoft Excel software to produce descriptive statistics. Qualitative data (from the observations and interviews) were processed following Braun and Clarke’s (2006) suggested method for qualitative data thematic analysis.

Figure 1 below represents the different phases of the research in relation to the four design-based research stages, with individual activity systems representing the analytical lens used for each phase following activity theory as the analytical framework.

FIGURE 1. Research phases represented in relation to design-based research and activity theory

Primary data analysis was conducted by the co-principal researchers (Eames and Aguayo) and all aggregated findings were shared with the REEF team and debated to inform all steps in the study. A final meeting was held with members of the REEF to gather reflections, evaluate the outcomes of the whole research project, and to collectively address the research questions based on findings from the data gathering phases, and outcomes from the iterative design-based research process.

3. Findings

The findings of this study are now presented in response to each of the four research sub-questions. The lens of Activity Theory is applied to more deeply examine the interactions within the learning system at each point.

3.1 Ecoliteracy outcomes in a marine reserves unit

An inquiry unit on marine reserves was implemented by the teacher from late February to early April 2017. The Goat Island visit included a 1-hour snorkelling event supported by a commercial company in the morning and a 90-minute visit to the Discovery Centre in the afternoon. As Figure 2 below shows, this unit acted as the tool that could mediate the learning of students and their parent helpers. The students were from a small rural school not far from the coast. One class of 20 students was involved in the study, with the students being a mix of Years 5–8 students. Six parent helpers attended the visit.

FIGURE 2. Phase 1 activity system

The marine reserves unit fitted with The New Zealand Curriculum (Ministry of Education, 2007) through the Science Learning area Living World strand at Levels 3–4. This learning context provided excellent opportunities for the use of sustainability education content knowledge and the pedagogical approach of education about, in, and for the environment (Barker & Rogers, 2004).

The community element in the above activity system included that the school involved in the study had a policy of fostering EOTC and conducted trips outside the classroom at least twice per term to connect classroom and outside the classroom learning. The teacher described these experiences as “phenomenal”, explaining that “the things you really remember from school are all of those activities that you learnt through doing something outside the classroom”. Parent helpers interviewed in this study also strongly supported this policy. The Discovery Centre educators were clearly supportive as well, and for the combination of the reserve and the visitor centre, one said “I think the Centre’s really good to reinforce what they might have seen out there.”

The school also had a BYOD policy in place so all students had access to some digital device, mostly iPads. Students indicated that they enjoyed using their devices for learning but that they tended to do that mostly at school. They weren’t certain whether the devices were helping their learning but thought they might be, with one student commenting “There is a game that is fun but I didn’t know it helped me learn maths.” The teacher held mixed views about learning with mobile technology tools. While she could see their potential, she felt that, although “some students could articulate that they were on a particular website or app that was improving their learning”, others thought that they were just playing a game. The teacher felt that her own knowledge and experience in guiding learning through mobile tools needed improvement. Parents expressed a range of views about using mobile devices for learning. These ranged from a feeling that they are being over-used, leading to disconnection from the real world, to an acceptance that they are a good tool “not just for engagement but to learn in different ways”, to a view of their being “crucial to give kids those skills for the future”.

In considering how mobile learning tools could be used within an EOTC experience, the teacher could see how they could add value in the Discovery Centre, but was adamant that they should not be used in the natural environment “because it’s a distraction”. All parent helpers accepted that mobile learning tools held potential to, for example, capture images or find information, but some expressed reservations about their use getting in the way of real experience of the environment, and wanted a “balance between hands on learning and opportunities that mobiles provide”. The Discovery Centre educators were fully supportive of introducing more use of technology in the Centre, to provide interactive experiences with the environment that can’t easily be achieved by direct interactions.

From the activity system in Figure 2, other community elements not yet described included the Marine Reserve and the Discovery Centre. In 2017, the Discovery Centre featured interactive elements such as a touch tank in which animals could be handled, a Big Eye microscope which provided magnified views of live marine animals to explore aspects such as tentacles and suckers in close-up, many jars of preserved specimens, drawers of shells, and many static reading panels containing information. The Goat Island Marine Reserve was the first marine reserve gazetted in New Zealand in 1975. It is relatively small in size and contains a mix of reefs, rocky shores, and sandy bottoms.

The division of labour in this activity system was spread across the teacher and her students, parent helpers on the visit to Goat Island, and two Discovery Centre educators. The teacher planned an inquiry unit in which brief pre-visit activities were designed to sensitise the students to the context of marine reserves and Goat Island in particular. Students were expected to immerse themselves in the visit experience and then to return to class and follow inquiries of their own interest connected to the visit. However, this did not happen. As the teacher explained, “I was trying to use the visit as a provocation, but actually what it did was, all this buzz and excitement, and there was no way that I could live up to any of that for them back in class. We had lots and lots of conversations about it, but they felt we learnt everything at the Discovery Centre.” Curiously, at the end of the unit, the students in a focus group reflected on their learning after completing the post-unit questionnaire we set for them and realised there were things that they would have liked to have learnt more about.

In 2017, the visit to Goat Island included a snorkelling tour in the morning. The snorkelling tour was organised and run by a commercial company that provided equipment and safety features. All but two students snorkelled for 45 minutes with several parent helpers also in the water, whilst the researchers (Eames and Aguayo) observed in the water. This observation was somewhat challenging, as capturing student conversations and actions relied on the researchers being able to focus on what was happening below the water and above the surface at the same time. Nevertheless, it was very clear that the students were excited and highly engaged in what they were doing, and participated in aspects of social learning, as experiences were shared between peers and with parent helpers and snorkel company staff. Back in the classroom, focus group discussions with the students indicated that the snorkelling was the activity enjoyed the most by those who undertook it, and students were able to recall memories of the snorkelling clearly 2 months later at a subsequent focus group.

In the Discovery Centre, educators delivered their usual programme in which one educator provided a PowerPoint-assisted presentation with questions and answers for 40 minutes, followed by parent helper– mediated group explorations of the Centre. Two formal sessions were organised around the touch tank and the Big Eye microscope, and then the students and their parent helper were free to explore the other exhibits in the Centre. Parent helpers tended to be unengaged once the formal presentations were complete, and left the students to explore the Centre by themselves. Students appeared engaged in the presentation and were able to provide answers to many of the questions. They were highly engaged in the touch tank, with particular students returning to the touch tank during the free exploration time. They were also quite engaged with the Big Eye microscope and were able to describe what they saw in later focus groups, although details of the animals and the features they had were less clear to them. Few students in the later focus groups mentioned other features of the Discovery Centre being memorable for them.

As shown in Figure 2 above, the object of the activity system was development of ecological literacy for students and their parent helpers (including knowledge, attitudes, and behaviour). In response to the unit and specifically the site visit, students showed slight knowledge gains, and very slight shifts in attitudes and behaviour towards environmentally-friendly positions in relation to marine environments. Students indicated good knowledge about marine reserves prior to the trip, reflecting the teacher’s preparation and the environmental awareness of a number of their parents (as they expressed in their interviews). This knowledge developed slightly after the trip and persisted at the end of the unit, with 23 out of 30 students reporting that marine reserves helped protect marine life in an open question in the post-unit survey, and one student commenting that marine reserves were “trying to make the Earth a better place”. Students did appear to make greater connections to ideas of interdependence and systems thinking after the visit, but these connections were not always clear. Students’ attitudes towards marine reserves remained largely unchanged after the site visit. There was evidence of good sustainability thinking and striking the balance between environmental protection and human lifestyles, as one student said about marine reserves in a focus group that “there should be more but not too many so people can still fish, as some people need fish to eat”.

There was little clear evidence of change in reported student environmentally-friendly behaviour at home or at school. They had been involved in rubbish clean-ups and tree planting previously but didn’t clearly connect those experiences to helping marine reserves. Parent helpers corroborated these findings in their post-unit interviews, indicating that, while their children had really enjoyed the visit, and wanted to go back, they had not observed any change in their environmentally-friendly behaviour. The parent helpers agreed that they themselves had learnt during the visit to the Discovery Centre, but felt that the trip had supported their proenvironment beliefs rather than significantly changed them, as one said: “We felt that our views were reinforced by the visit.”

In summary, Phase 1 showed an engaging visit to Goat Island was not followed by productive learning back in the classroom. Whilst ideal conditions existed for combining EOTC with mobile learning due to sound opportunities and policy positions in the school, there was little complementarity achieved for learning. Learning outcomes were also not clearly evident.

3.2 The REEF collaboration

Phase 2 involved the implementation of a collaborative online research forum for the research team (i.e., the REEF, involving the school teacher and the Discovery Centre educators as the practitioner partners of the research, a marine scientist as adviser, a mobile learning designer, and the researchers as facilitators). A general objective of the REEF was to carry out the design-based research agenda (Figure 1) for the duration of the intervention. The REEF met twice face to face, in July 2017 at the start of Phase 2, and in November 2017 at the end of Phase 2 (see Figure 3). Ongoing collaboration occurred in between these two events via a Google+ Community online forum space mediated by the researchers. As shown in Figure 4, the REEF forum acted as the tool mediating the development of a mobile learning framework (as the object) with the members of the REEF acting as subjects of the activity system.

FIGURE 3. REEF workshop July 2017 (left) and workshop November 2017 (right)

The activity system also considered digital mobile affordances, heutagogy (self-determined learning), and various theories of mobile learning and of science and sustainability education as guiding theoretical considerations acting as the rules. These theoretical platforms acted as the conceptual starting points of the DBR methodology (second stage in Figure 1) of the development of the mobile learning framework. The REEF functioned well as a team, with members posing questions for others on issues outside their domain and contributing expertise within their domain.

During the REEF Workshop One in July 2017, four key areas informing the framework were defined from current theory and knowledge and data from Phase 1: marine science and conservation; development of ecological literacy; teaching and learning (including EOTC as a research domain); and mobile learning opportunities. Following the first workshop, each key area was explored and discussed online in relation to the framework, moderated by one or two REEF members at a time, for a period of 2–3 weeks. Members of the forum contributed with ideas and examples of practice, which were also then discussed in relation to pragmatic aspects to be considered when applying and testing the framework during the second visit to the Discovery Centre by the class in March 2018.

FIGURE 4. Phase 2 activity system

At the second REEF workshop in November 2017, emerging ideas from the online interactions were reviewed, discussed, condensed, and organised in relation to the different theoretical areas informing the research, and the emerging approach of MR. A mobile learning framework was agreed by the REEF which then underpinned the testing and refinement of an MR intervention to be implemented in the context of the Discovery Centre for the visit by the 2018 class. The framework is presented next.

3.3 Elements and design principles of a mobile learning framework

The work of the REEF team led to a set of key messages for each one of the four main areas of the framework. These key messages summarise an extended list of principles and ideas discussed by the REEF, yet they cover the key aspects to be considered when using mobile learning to enhance the development of marine ecological literacy within an EOTC context. Table 1 below shows these key messages.

TABLE 1. Mobile learning framework key messages

1. Marine science and conservation (the context)

Marine reserves conserve the marine environment through protection, research, and education. Marine reserves protect biodiversity (e.g., range of species within the reserve). Biodiversity is crucial for interdependence (e.g., food webs snapper/kina). Pollution threatens marine environments (e.g., sedimentation runoff, plastics). Global warming threatens marine environments (e.g., ocean temperature change and acidification).

2. Development of ecological literacy

To develop ecological literacy about marine reserves, learners need to: gain knowledge of marine ecology and the impact of marine reserves (environmentally, socioculturally, and economically) through authentic inquiry and meaningful experiences consider their own and others’ attitudes and values towards marine reserves iii. be motivated and able to think critically, plan, and take action in respect of marine reserves. Holistic approaches that view marine reserves as systems that connect the natural environment with society and its culture and economy are important.

3. Teaching and learning

The placement of the visit within a teaching unit is pedagogically important. Visit structure is pedagogically and logistically important. Pre-visit resources can help to sensitise learners and initiate connections to place. The visit should allow freedom to experience but also have some focus to scaffold learning, and to promote discussions between learners (social learning). Interactive learning options increase learner autonomy and engagement. Learning needs to be reinforced post-visit to deepen knowledge, clarify attitudes, and support actiontaking.

4. Mobile learning opportunities

Mobile learning implementation and use must focus on BYOD, in part to leverage the cost of technology. Mobile and real experiences should complement each other. Mobile learning affordances should focus on self-determined learning to occur, with the learning experience unfolding in unique ways for each learner based on their own motivations and needs. User-informed design and co-creation should guide the design of affordances to promote meaningful learning. An authentic, integrated, and scaffolded experience is critical for the success of mobile learning. Access to technology (e.g., WIFI connectivity and IT infrastructure) and staff professional development are critical for the success of mobile learning initiatives at the institutional level.

These key messages are generic in many ways but flexible and adaptable enough to be locally developed according to the specific characteristics and conditions of each educational context. In this project, Auckland University of Technology’s AppLab, led by Dr Aguayo and with support from Conical Studios, enacted the framework, developing a set of mobile learning digital experiences using an MR approach. In this context, and following the original definition from Milgram and Kishino (1994), MR is understood as the merging of real and virtual worlds with different degrees of immersion within a digital continuum. The MR intervention at the Goat Island Marine Discovery Centre is shown in Figure 5. The kelp forest was an interactive touch-based installation done as part of a Master’s Thesis in Digital Design by James Smith at Auckland University of Technology, cosupervised by Dr Aguayo.

FIGURE 5. The MR approach developed following the mobile learning framework

Figure 6 below presents the activity system defining the development of the MR approach. Members of the digital development team acted as the subjects of the activity system, engaging with the mobile learning framework which mediated their activity towards enacting it by developing a set of digital learning experiences in the form of an MR intervention. The digital development period was from December 2017 to February 2018. The design of the MR intervention was impacted by the “Rules”, “Community elements”, and “Division of labour” as shown below.

FIGURE 6. MR intervention development activity system (Phase 3)

From the perspective of the design-based research methodology, the mobile learning framework, and its MR enactment by the digital development team, represents the development of solutions within the theoretical framework stage (stage 2 in Figure 1), and the testing phase with the class at the Discovery Centre represents the testing of solutions in practice stage (stage 3 in Figure 1). The community provided feedback on the iterative development of the MR, with input from the teacher and the Discovery Centre educators during the testing of solutions in the DBR model.

FIGURE 7. Graphic elements portraying aspects of Pipi’s World Augmented Reality app, including Pipi the snapper

The digital elements of the MR intervention were themed around “Pipi’s World”, an App based on Pipi the snapper (see Figure 7), a young female character that was inspired by original designs from the school students in a co-creative design process. Pipi’s World offered users different engaging digital learning experiences through Augmented Reality (AR) via the mobile app (freely available on Google Play or App Store). There was also a computer-generated Virtual Reality (VR) experience accessible through different types of VR headsets, and a range of other complementary experiences accessible through different entry points (e.g., QR codes, online VR viewer, complementary array of apps, and other already existing content). The narrative was centred around Pipi the snapper who could show users her world—the Goat Island Marine Reserve, the positive impacts of marine conservation inside the reserve over time, and the fishing pressures she and other marine life get exposed to outside the boundaries of the reserve.

When regarded from the perspective of the DBR methodology, the enactment of the mobile learning framework through a digital development process corresponds to the third stage of DBR depicted in Figure 1, where theoretical solutions (the framework) are tested in practice. In this case, tested and evaluated in the context of the Discovery Centre by the teacher and her class during their second visit to the marine reserve area in March 2018. Findings from this event are presented next.

3.4 Mobile learning intervention impacts on marine ecoliteracy outcomes

A unit on marine reserves was again taught by the same teacher from late February to early May 2018. In 2018, the class included 28 students, with the students again being a mix of Years 5–8 students. Half of these students had been in the teacher’s class in 2017 and had therefore been involved in the unit and the visit in that year. The unit included one day at Goat Island with a 2-hour visit to the Discovery Centre, this time in the morning, and a 1-hour snorkelling event in the afternoon supported by the same commercial company.

In this second year of the study, the teacher felt that the unit had been more effective than the previous year. She said: “… it’s been excellent actually. The engagement from the students.” She noted from the visit that the students loved the snorkelling and students reported that they enjoyed the snorkelling most of all, with 15 out of 19 student respondents rating it as their favourite activity. The teacher also noted that the students spent much longer in the Discovery Centre than in 2017.

As Figure 8 below shows, the MR intervention served as a tool that could mediate the learning of students and their parent helpers.

FIGURE 8. Phase 4 activity system showing an expansive process into a second activity system derived from the first one

In comparing the MR intervention with the initial 2017 class experience, two new “rules” are apparent in this activity system—digital affordances and heutagogy.

The digital affordances were deployed at the Discovery Centre and back in the classroom only, with the teacher and parent helpers adamant that they did not want them to intrude on the students’ sensory experiences at the marine reserve. The teacher noted in her interview that “we talked at the beginning about the balance of not having devices on the beach, I still prefer not to do that” and one parent agreed that “I wouldn’t have them on the beach, they would distract from kids using their senses, and gaining a 3D appreciation of what is around them.”

The affordances of the mobile technologies were the subject of both affirmation and caution in the findings. The mobile learning opportunities were seen to highly engage the students, to enhance learning in some cases, and to promote social learning between student peers, and with parent helpers on site, and between students and family members at home. Most students (20 out of 27) liked how the character Pipi looked, indicating the iterative design process had captured their ideas, but some felt that Pipi looked too cartoonish, as one student noted “She is definitely appropriate for kids. The thing I don’t understand is she has too many human features, she’s got eyelashes, snapper don’t have eyelashes, and where are all of her teeth?” These findings led the design team to revise Pipi on the app for future use.

All data indicated high engagement of the students with the VR opportunity. In the post-unit survey, 19 out of 27 students agreed that they liked learning through VR on the trip, although only 12 out of 27 agreed that they could learn just as well through VR as having real marine experiences such as by snorkelling. Students rated the VR as their most favourite mobile experience, with one mature student commenting that “I really think the VR has so much potential, it could become something really cool if you had missions and things like that, save dolphins from getting caught in nets.” However, some students indicated that they found the experience less exceptional, as one noted “The VR was basically just like watching TV”, and “I kind of liked the whole VR with Pipi, but I’d rather be snorkelling and see.” The teacher agreed that the VR appeared to have had the most impact on the students, noting that “they were so wowed by it, you know, you saw one of them on it, and it was such a new thing for them”, but she also felt that the students had learnt little from the experience. A second issue that emerged in the use of VR was instances of reported motion sickness by some students, with one saying “I felt sick after it, I hadn’t done it before.” These issues with learning through VR point to a need for sensitising learners to the nature of the experience before meaningful learning can occur, a point that has recently been made elsewhere (Southgate et al., 2019).

A number of opportunities to engage with AR were presented to the students at the Discovery Centre. Students appeared variably engaged in these experiences, in part due to a restricted number of tablets and smartphones being available with which to engage with them. Observations indicated that students did operate in their allocated groups for these activities, which provided good examples of social learning on site, as students worked together with each other and their parent helper to undertake the activities. Just over half (15 out of 26) of the students agreed that they learnt using AR at the Discovery Centre. Students in the focus groups were able to recall the lobster march (“I remember the lobsters going to the reef to have their babies so they wouldn’t get eaten by the fish”) and the food web (“It was how to keep a healthy environment and what different animals need to keep it balanced. The snapper eats the kina and the kina eats the seaweed.”).

There were also a number of opportunities to engage with QR codes that showcased videos of the area and other information about marine reserves. Engaging with these was dependent on the availability of a mobileconnected device and at times a shortage of these appeared to frustrate students in their access to the QR codes. The final element of the MR intervention was a simulated kelp forest, which was supported by marine life paper cut-outs that students could colour in and types of plastic rubbish (bottles, wrappers) that get washed/ blown into the sea, all velcroed to the strips of “kelp” hanging down from a frame. Students were encouraged to interact with this exhibit by removing the cut-outs and colouring them in and refastening them, and by removing the plastic rubbish into a recycling bin. The students enjoyed interacting physically with the “kelp” forest and younger students in particular engaged in the colouring, but students also enjoyed helping the forest, as one said at the end of the unit “I really liked it, it’s basically telling me if you see plastic in the sea or on land you should probably pick it up and throw it away. I spent quite a lot of time there cleaning the forest, it was fun.”

A key issue that arose during the mobile learning intervention was Wifi connectivity. This was a key principle identified in the mobile learning framework developed by the REEF team. Despite the best efforts of the educators at the Discovery Centre to ensure access prior to the visit, on the day there were significant problems with connecting numerous devices. Eventually, enough devices were connected for the intervention to proceed. Interestingly, few students raised this as a concern after the trip, with some mentioning briefly that some AR features and QR code scanning didn’t work due to the Wifi issues. The teacher was understanding of the problem, the Discovery Centre educators recognised the “organised chaos” that ensued and dealt with it as well as they could, and a number of the parent helpers mentioned the Wifi issues, with one saying “there were challenges with getting connected. That was a bit off-putting.”

Despite the high engagement with the digital experiences on the trip, after the visit most students (20 out of 27) strongly agreed that snorkelling to see fish was preferable to seeing them on screen. Observational data generated during the snorkelling supported the high engagement of students, with two girls agreeing with each other after 20 minutes in the water “that this is the most great thing I have done”, and sharing experiences with peers and parent helpers, and there was evidence that learning experiences from the Discovery Centre were being transferred into their water experiences, as one student called out “I can see a kelp forest”. Focus group comments provided further illumination of these views, with snorkelling supporters noting that they liked “Real life—you can see things properly. See things that are real, not fakes”, while others were more equivocal, seeing value in “mobiles—you can see things you wouldn’t normally be able to see” but also that they “can make you think things are real”.

The principle of learner-centred activity fostered by mobile technologies (heutagogy) was adopted for the MR intervention. Firstly, the teacher deliberately planned the visit to the Discovery Centre to enable students to somewhat follow their own learning interests. The students were placed in groups with a parent helper as with the previous year. However, unlike in the previous year, where each parent accompanied their students around organised “stations”, students were able to lead by choosing where they wanted to learn. As a result, there was a great deal of activity as the students used their mobile devices to access the AR and QR code options, visited and revisited the kelp forest, and spent time at the Big Eye microscope and touch tank. Parent reactions to this heutagogical approach were mixed, with one saying that “I think giving parents directions sort of helped but I found it hard to do the tasks in the order given, as there was some congestion at some points, and kids followed their own interests. I did write some responses down but not as much as the teacher expected I think.” This latter concern emphasises a focus on completing a task set by adults, which is somewhat at odds with a heutagogical approach. The Discovery Centre educators also experienced some challenges with the approach taken, as one said “As to my role, I just didn’t know how to kind of … if I should be teaching, if I shouldn’t, should I be standing off, you know.” But one observation he made indicated the power of the heutagogical approach, as he noted “I saw them using [the Big Eye microscope] in ways that I didn’t think of, they were more like focusing on patterns, like getting amazed by the actual patterns within the animals rather than focusing on the function, so it’s completely different seeing them to what we see.” These findings clearly indicate to us that for both the parents and the Discovery Centre educators there was a greater need for briefing them about the heutagogical approach prior to the visit.

The mobile learning intervention appeared to stimulate learning opportunities back in the classroom. The teacher was clear that, unlike the previous year when interest in further learning died once back in the classroom, in this case further learning was largely driven by student interest. She said “they enjoyed spending time finding new things. And then side-tracking onto rubbish and things like that.” The teacher indicated that students chose what further inquiry they wished to engage in. One group decided to construct their own kelp forest, which drew in the rest of the class and led to in-depth work around marine plastic pollution. This issue had been in the media a lot at the time, with graphic images of seabirds and turtles with stomachs full of plastic. It appeared that the experience in the kelp forest at the Discovery Centre had provided a trigger for further learning about this. The class decided to create “fish” out of plastic bags and to fill each one with the plastic rubbish in their lunchboxes. This made for a very graphic display which was highly memorable, with students readily commenting in the focus groups that “we bring too much plastic in our lunchboxes and that we should start putting things in more containers and use less plastic bags” and “I try not to bring too much plastic and things in my lunchboxes.”

There was some re-engagement with the AR markers and QR codes back in the classroom which led to reinforcement of learning between the DC and the classroom. But one of the aspects in the mobile learning framework that wasn’t fulfilled was the plan to use social media to keep a connection between the class and the Discovery Centre educators. This was envisaged as a question and answer forum that would be mediated by the teacher through a Google+ platform. For a variety of reasons, this didn’t come together, as the Discovery Centre educators and teacher struggled to engage in this type of post-visit reinforcement.

Finally, the object of this activity system was to develop the ecological literacy (including knowledge, attitudes, and behaviour) of students and parent helpers. In response to the unit and specifically the visit, there was a slight increase in student knowledge about marine reserves and all students were able to respond appropriately in the negative to the statement “People are not allowed to fish in a marine reserve but can take shellfish and crabs”. In the focus groups, all (28 out of 28) students expressed support for marine reserves. The students showed evidence of increased knowledge about interdependence of snapper, kina, and seaweed (which was the focus of one of the AR games), and demonstrated an ability to transfer understanding about systems thinking from street pollution to soil runoff impacting the ocean.

The students expressed enhanced positive attitudes after the unit, with 66% (20 out of 30) strongly agreeing with the statement “I think marine reserves are good for fish” increasing to 87% (20 out of 23), and an increase from 20% to 93% in students believing that “we should have more marine reserves”. When asked prior to the unit if they wanted to do more to help the environment, 29 out of 30 students agreed that they did. After the trip, when asked if they actually had done more since learning about marine reserves, 12 out of 19 students indicated that they had. The students noted their concerns about marine life and the oceans as being oil spills, overfishing, and plastic pollution. In particular, they demonstrated knowledge of the impacts of plastic in the ocean, with one student saying “Crayfish can think a plastic bag is a jellyfish so they eat it.” The following focus group comments were insightful: “I would like to help but I’m not sure how I would be able to, which is the problem” and “Since we are only 11 and 12, there’s not much we can do, except to try our best not to use plastic. One of the biggest problems with plastic is poverty, because no matter what, you’re going to buy plastic because it’s the cheapest thing for them, and they’ll do what they can to survive and I can’t blame them for that.”

Despite these challenges, there was evidence that students’ experiences and developing ecological literacy were also impacting on home life. Parents who had accompanied the class on the trip commented on their own learning, with one who had been on both 2017 and 2018 trips saying “I learnt more this year.” They noted that post-trip conversations revolved around many aspects, with one parent explaining that her daughter talked “about the big eye microscope, the crayfish march, and the role of kina. She also talked about plastics in the oceans killing birds.” Most parents reported that their child had exhibited some aspect of environmentallyfriendly behaviour at home since the trip. In some cases, this was strong and overt (“She is very hot on not having plastic in her lunchbox. She is driving that”, and “He has been telling us off for using plastic bags and straws, whereas he always used to use them happily”) to an acceptance of family norms (“We were already very environmentally friendly by reducing plastic use. But I think the kids are more OK with that now and don’t see it as weird.”).

4. Discussion and implications

4.1 Discussion

4.1.1 Ecoliteracy development in a marine reserves unit

Findings from Phase 1 of the study indicated that conditions for learning were present in the design of the marine reserves unit incorporating a trip to Goat Island. The unit fitted with the objectives of The New Zealand Curriculum (Ministry of Education, 2007), the visit offered opportunities for learning both through immersion in the environment (snorkelling) and through an educator-led programme in the Discovery Centre, and follow-up activities were planned for the classroom,

Despite these conditions, it appeared that learning outcomes were not significantly achieved, with a lack of connection between the learning outside the classroom and in the classroom (Falk, 2005). Students appeared to be receptive and ready for learning, showing eagerness for the trip and topic, with some existing knowledge and attitudes towards marine reserves. While the students exhibited very moderate gains in knowledge and understanding and some shifts in attitudes towards environmentally-friendly positions, they neither expressed intentions nor took actions towards resolving environmental issues. It appears that the stimulation of the visit wore off quickly once back in the classroom and, without post-visit action resources, no learning reinforcement appeared to occur (Ballantyne & Packer, 2005). It appeared that an opportunity existed to design and trial a mobile learning intervention, and this is discussed next.

4.1.2 The REEF process

During Phase 2, the REEF developed a mobile learning framework through collaborating face-to-face and online using a Google+ Community page. Adopting a systems thinking approach to the REEF process and management (i.e., active facilitation, promoting diversity, a decentralised control structure, and means for members to affect each other (Aguayo, 2014, 2016)), enhanced the collaborative environment.

The DBR methodology (Amiel & Reeves, 2008) was important in directing the work of the REEF towards generating practical solutions in the form of the final framework. DBR also permitted the quick uptake of new technologies and strategies (e.g., the MR, new literature, and emerging ideas from the REEF).

4.1.3 A mobile learning framework

The mobile learning framework and DBR methodology enabled the development of a collection of digital experiences in a collaborative, co-creative, dynamic, and iterative way. The mobile learning framework can be seen as general guidelines intended to be flexible and adaptable to local conditions and emerging technologies. The collaborative and co-creation design and development process (e.g., in developing and subsequently refining the Pipi the snapper character) appears to have increased the meaningfulness of mobile learning experiences (Aguayo & Eames, 2017; Facer & Williamson, 2004). The focus of the DBR approach on dynamic and iterative design and development led us to incorporate an MR approach which wasn’t even imagined at the outset of the 2-year project. Such digital dynamism and rapid adaptation to new technologies may be a key strength in the potential of mobile learning (Aguayo et al., 2017; Pachler et al., 2010).

The impact that the MR intervention had on students and their parents is discussed next.

4.1.4 MR-enhanced marine ecoliteracy

The MR intervention in the marine reserves unit appeared to heighten student engagement and learning at the Discovery Centre, and promote post-visit learning (Ballantyne & Packer, 2005). All elements of the MR intervention appeared to contribute. It is noteworthy that the activities that engaged students the most are located at either end of the MR immersion continuum, leading to speculation that the continuum might actually exist as a circle rather than a line, in which VR becomes juxtaposed with immersion in the real world. This has implications for deeper connections between virtual and real worlds that can engender learning, and that learning can be socially shared and provide a bridge between outside the classroom, the classroom, and the home.

Further research is needed into digital literacy, including what we have termed VR literacy, the use of the tools to maximise learning, from each of the designers’, educators’, and students’ perspectives. The costs involved with creating and implementing computer-based VR are considerable but there are emerging examples of ready-made VR experiences being available in cost-effective ways for schools (see Sir Peter Blake Trust, 2019), and mobile accessible apps and QR codes provide other feasible learning pathways. Mobile connectivity is critical to gain the most from these tools, particularly when outside the classroom.

Adopting a heutagogical approach appeared to assist in engaging the students in learning both at the

Discovery Centre and back in the classroom, but presented challenges to the parent helpers and Discovery Centre educators, indicating a need for better preparation for these roles under this approach. A post-visit reinforcement strategy emerged, not using social media as initially envisaged, but through re-engagement with the MR resources in the classroom. The ongoing availability of the AR markers and QR codes for the students, with access to mobile devices and the freely available Pipi’s World app, facilitated seamless and timeindependent integration of the experiences had during the EOTC visit with subsequent classroom work.

Evidence suggests that the MR intervention assisted some knowledge and attitude development, and there was adoption of environmentally-friendly behaviour related to plastic pollution that appeared to be connected to learning through the MR intervention. In summary, the adoption of an MR intervention incorporating an EOTC visit, in which mobile-less immersion in a real environment combined with mobile-enhanced immersion in the Discovery Centre, complemented by a heutagogical approach with post-visit reinforcement for teaching and learning, appears to have had educational impact. This suggests some implications for practice.

4.2 Implications for practice

The following implications can be drawn from this study:

• Opportunities for EOTC and use of educational digital technology exist in schools and are supported by government guidelines (Ministry of Education, 2016, 2018). There are no policy barriers to combining EOTC and MR.
• EOTC can be done anywhere and cost-effectively but there can be logistical challenges to consider such as parental support.
• A mobile learning framework provides a solid foundation for infusing MR into EOTC in any learning context and development of the framework is enhanced through collaboration, between diverse partners. Embracing a systems thinking paradigm and following a design-based research methodology can further enhance this process.
• MR can provide a continuum from real to digitally immersive experiences. Offering a range of experiences along this continuum can provide effective opportunities for diverse learners. Attention to the local context in designing these experiences is achieved through co-creation of content with partners in ways that are meaningful to the target audience.
• MR can also provide a continuum between EOTC activities and classroom activities, reinforcing the learning from outside the classroom into the classroom. A heutagogical learning approach is recommended to assist this.
• There is a cost associated with creation of digital experiences, but, increasingly, “off the shelf” digital experiences are becoming available at low cost, and the challenge is now how to integrate these ready-made tools into classroom practice for effective learning.
• Educators and students are as yet quite naive in their use of digital tools such as VR, and there needs to be support for development of skills to enable them to be used most effectively for teaching and learning. Skills in adopting a heutagogical approach when using these tools also need development.
• Development of ecological literacy in connection to local contexts through complementary EOTC and MR is possible, and given the dire need for this development, is worthy of further investigation.
• Ecological literacy development in connection to non-local contexts where EOTC is not easily achieved is also critical and may still be fostered through the digital end of the MR continuum. Further research to determine the effectiveness of this latter approach is important.

Acknowledgements

We gratefully acknowledge the partnership in this study with Ahuroa School with Jessica Hanlon and her students, the Goat Island Marine Discovery Centre with Tim Haggitt and Angela Rapson, the University of Auckland with Dr Nick Shears, and Auckland University of Technology with Dr Thom Cochrane. We also acknowledge the contribution made by Conical Studios.

Authors

Chris Eames (Contact: chris.eames@waikato.ac.nz)

TEMS Education Research Centre, University of Waikato

Claudio Aguayo

Centre for Learning and Teaching, Auckland University of Technology

References

Aguayo, C. (2014). The use of education for sustainability websites for community education in Chile. Doctoral (PhD) thesis, University of Waikato, Hamilton, New Zealand.

Aguayo, C. (2016). Activity theory and community education for sustainability: When systems meet reality. In D. Gedera & J. Williams (Eds.), Activity theory in education: Research and practice (pp. 139–151). Rotterdam: Sense.

Aguayo, C., Cochrane, T., & Narayan, V. (2017). Key themes in mobile learning: Prospects for learner-generated learning through AR and VR. Australasian Journal of Educational Technology, 33(6), 27–40.

Aguayo, C., & Eames, C. (2017). Promoting community socio-ecological sustainability through technology: A case study from Chile. International Review of Education, 63(6), 871–895. https://doi.org/10.1007/s11159-017-9685-7

Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20, 1–11.

Amiel, T., & Reeves, T. C. (2008). Design-based research and educational technology: Rethinking technology and the research agenda. Educational Technology & Society, 11(4), 29–40.

Ballantyne, R., & Packer, J. (2005). Promoting environmentally sustainable attitudes and behaviour through free-choice learning experiences: What is the state of the game? Environmental Education Research, 11(3), 281–295.

Ballantyne, R., & Packer, J. (2011). Using tourism free-choice learning experiences to promote environmentally sustainable behaviour: The role of post-visit ‘action resources’. Environmental Education Research, 17(2), 201–215.

Barker, M. and Rogers, L. (2004). “In, about and for”: Exploring the foundations of environmental education. set 2, 15-18.

Becta. (2009). Becta: Our work, its impact. Coventry, England: Author.

Boyer, L., & Roth, W. M. (2005). Individual | collective dialectic of free-choice learning in a community-based mapping project. Environmental Education Research, 11(3), 335–351.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. http://dx.doi. org/10.1191/1478088706qp063oa

Brookfield, S. (1986). Understanding and facilitating adult learning: A comprehensive analysis of principles and effective practices. Milton Keynes, England: Open University Press.

Bull, R., Petts, J., & Evans, J. (2008). Social learning from public engagement: Dreaming the impossible? Journal of Environmental Planning and Management, 51(5), 701–716.

Cochrane, T. (2014). Critical success factors for transforming pedagogy with mobile Web 2.0. British Journal of Educational Technology, 45(1), 65–82.

Cochrane, T., Buchem, I., Camacho, M., Cronin, C., Gordon, A., & Keegan, H. (2013). Building global learning communities. Research in Learning Technology, 21, 21955. Retrieved from http://dx.doi.org/10.3402/rlt.v21i0.21955

Cook, J., & Santos, P. (2016). Three phases of mobile learning state of the art and case of mobile help seeking tool for the health care sector. In D. Churchill, J. Lu, T. K. F. Chiu, & B. Fox (Eds.), Mobile learning design (pp. 315–333). Singapore: Springer.

Delen, I., & Krajcik, J. (2017). Using mobile devices to connect teachers and mobile devices. Research in Science Education, 47(3), 473– 496.

Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research. Helsinki, Finland: OrientaKonsultit.

Facer, K., & Williamson, B. (2004). Designing educational technologies with users. Bristol, UK: Futurelab.

Falk, J. H. (2005). Free-choice environmental learning: Framing the discussion. Environmental Education Research, 11(3), 265–280. Falk, J. H., & Dierking, L. D. (2002). Lessons without limit. Boston, MA: AltaMira Press.

Gruenewald, D. A. (2003). The best of both worlds: A critical pedagogy of place. Educational Researcher, 32(3), 3–12.

Hackmann, H., & St Clair, A. (2012). Transformative cornerstones of social science research for global change. International Social Science Council, Paris: ISSC.

Hase, S., & Kenyon, C. (2013). Self-determined learning: Heutagogy in action. Sydney: Bloomsbury Academic.

Hollweg, K. S., Taylor, J. R., Bybee, R. W., Marcinkowski, T. J., Mcbeth, W. C., & Zoido, P. (2011). Developing a framework for assessing environmental literacy. Retrieved from https://naaee.org/our-work/programs/environmental-literacy-framework

Jensen, B. B., & Schnack, K. (1997). The action competence approach in environmental education. Environmental Education Research, 3(2), 163–179.

Kollmuss, A., & Agyeman, J. (2002). Mind the gap: Why do people act environmentally and what are the barriers to pro-environmental behaviour? Environmental Education Research, 8(3), 239–260.

Krasny, M. E., & Roth, W.-M. (2010). Environmental education for social ecological system resilience: A perspective from activity theory. Environmental Education Research, 16(5), 545–558.

Leadbetter, J. (2005). Activity theory as a conceptual framework and analytical tool within the practice of educational psychology. Educational & Child Psychology, 22(1), 18–28.

Liddicoat, K., & Krasny, M. E. (2013). Research on the long-term impacts of environmental education. In R. B. Stevenson, M. Brody, J.

Dillon, & A. E. J. Wals (Eds.), International handbook of environmental education (pp. 289–297). New York, NY: Routledge.

Lindsay, L. (2015). Transformation of teacher practice using mobile technology with one-to-one classes: M-learning pedagogical approaches. British Journal of Educational Technology, 47(5), 883–892. https://onlinelibrary.wiley.com/doi/pdf/10.1111/bjet.12265

Luckin, R., Clark, W., Garnett, F., Whitworth, A., Akass, J., Cook, J., … Robertson, J. (2010). Learner-generated contexts: A framework to support the effective use of technology for learning. In M. Lee & C. McLoughlin (Eds.), Web 2.0-based e-learning: Applying social informatics for tertiary teaching (pp. 70–84). Hershey, PA: IGI Global.

Merchant, G. (2012). Mobile practices in everyday life: Popular digital technologies and schooling revisited. British Journal of Educational Technology, 43(5), 770–782. https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-8535.2012.01352.x

Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), 1321–1329.

Ministry of Education. (2007). The New Zealand curriculum. Wellington: Learning Media.

Ministry of Education. (2016). Bringing the curriculum alive: EOTC guidelines 2016. Wellington: Author.

Ministry of Education (2018). Statement of intent 2018–2023. Retrieved from https://www.education.govt.nz/assets/Documents/ Ministry/Publications/Statements-of-intent/Statement-of-Intent-2018-2023-web.pdf

Narayan, V., & Herrington, J. (2014). Towards a theoretical mobile heutagogy framework. In B. Hegarty, J. McDonald, & S.-K. Loke (Eds.), Rhetoric and reality: Critical perspectives on educational technology (pp. 150–160).

Orr, D. (2004). Earth in mind: On education, environment, and the human prospect (10th anniversary ed.). Washington, DC: Island Press.

Pachler, N., Bachmair, B., & Cook, J. (2010). Mobile learning: Structures, agency, practices. Boston, MA: Springer US. Retrieved from http://link.springer.com/10.1007/978-1-4419-0585-7

Parsons, D. (Ed.). (2013). Innovations in mobile educational technologies and applications. Hershey, PA: IGI Global.

Penetito, W. (2009). Place-based education: Catering for curriculum, culture and community. New Zealand Annual Review of Education, 18, 5–29.

Rennie, L., & Johnston, D. J. (2004). The nature of learning and its implications for research on learning from museums. Science Education, 88(1), S4–S16.

Royal Society of New Zealand. (2016). Climate change implications for New Zealand. Retrieved from http://www.royalsociety.org.nz/ expert-advice/papers/yr2016/climate-change-implications-for-new-zealand/

Sir Peter Blake Trust. (2019). New Zealand Virtual Reality project. Retrieved from https://sirpeterblaketrust.org/nz-vr

Smith, G. A. (2007). Place-based education: Breaking through the constraining regularities of public school. Environmental Education Research, 13(2), 189–207.

Somekh, B. (2007). Pedagogy and learning with ICT: Researching the art of innovation. New York, NY: Routledge.

Southgate, E., Smith, S. P., Cividino, C., Saxby, S., Kilham, J., Eather, G., Scevak, J., Summerville, D., Buchanan, R., & Bergin, C. (2019). Embedding immersive virtual reality in classrooms: Ethical, organisational and educational lessons in bridging research and practice. International Journal of Child-Computer Interaction, 19, 19–29.

The Design-Based Research Collective. (2003). Design-based research : An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://journals.sagepub.com/doi/abs/10.3102/0013189X032001005

Traxler, J. (2010). Will student devices deliver innovation, inclusion, and transformation? Journal of the Research Center for Educational Technology (RCET), 6(1), 3–15.

United Nations. (2012). Resilient people, resilient planet: A future worth choosing. New York, NY: Author.

United Nations Educational, Scientific and Cultural Organization (UNESCO). (2006). Education for sustainable development: Memo bureau of public information. Retrieved from http://www.unesco.org/bpi/pdf/memobpi39_sustainabledvpt_en.pdf

Warner, A., Eames, C., & Irving, R. (2014). Using social media to reinforce environmental learning and action-taking for school students. International Electronic Journal of Environmental Education, 4(2), 83–96.

Wheaton, M., Ardoin, N. M., Hunt, C., Schuh, J. S., Kresse, M., Menke, C., & Durham, W. (2015). Using web and mobile technology to motivate pro-environmental action after a nature-based tourism experience. Journal of Sustainable Tourism, 24(4), 594–615. .