Citizen scientists in the classroom: Investigating the role of online citizen science in primary school science education

1. Introduction

This project aimed to assess the impact of online citizen science (OCS) participation on the science education of primary school children in New Zealand. Four exploratory cases were used to examine the nature and impact of embedding OCS projects that use web-based online tools within primary classroom environments. Findings provide key insights into primary teachers’ perception of OCS. They offer initial insights into how teachers embed OCS in a classroom environment, and why this improves science learning aptitudes, inquisitiveness, and capabilities in primary school children. They also show how successfully embedding OCS projects in education is affected by the project context, human computer interaction (HCI), and dissemination of results, among other factors. Recommendations are made for further research that provides comprehensive coverage beyond this exploratory study. It is important that support is provided on how OCS can be utilised by schoolteachers in science education. We take a step towards achieving this by releasing a digital resource on the Science Learning Hub,^[1] available to all teachers in New Zealand, that provides an inventory of OCS projects along with descriptions about how these can be used in alignment with The New Zealand Curriculum (Ministry of Education, 2007) (NZC).

2. Research background and rationale

The New Zealand Curriculum has, at the core of its science learning area, an aim to build students’ ability to engage with science and scientific issues of interest to them through its “Participating and contributing” sub strand. Overall, the intention is to ensure that “students explore how both the natural physical world and science itself work so that they can participate as critical, informed, and responsible citizens in a society in which science plays a significant role” (Ministry of Education, 2007, p. 17).

However, student engagement with science has been declining globally and New Zealand is not spared (EARU & NZCER,^[2] 2012). New Zealand research suggests both a decline in engagement with school science as students move through the education system, and that this decline is happening at younger ages than previously observed (Bolstad & Hipkins, 2008; Tytler, Symington, Kirkwood, & Malcolm, 2008; Caygill, Hanlar, & Harris-Miller, 2016; EARU & NZCER, 2012). The proportion of students who indicate that they enjoy science learning in New Zealand is lower than in many other countries (Caygill et al., 2016). Such attitudinal data is significant, because students’ achievement in science is related to their attitude towards science (EARU & NZCER, 2012).

Given these trends, the problem of how to improve scientific engagement and at the same time develop scientific literacy is a significant one. One way of doing so may be through citizen science. Citizen science projects are participatory studies that, in various ways, involve members of the public in scientific or technological research (Hassman, Mugar, Østerlund, & Jackson, 2013). Online citizen science is the subfield of such participatory science that makes use of systems on the World Wide Web to let volunteers contribute to real and wide-ranging scientific endeavours. Some existing OCS initiatives invite volunteers to tag images and videos of wild animals as part of bigger conservation projects,^[3] for example tagging and classifying images of penguins from the Antarctic Peninsula^[4] to help understand their lives and environment. Other projects require participants to classify images of galaxies,^[5] to transcribe non-digital artefacts,^[6] to create 3D atlases of the human brain,^[7] or trace gene sequences.^[8] These projects expose participants to diverse user experiences (UX), user interfaces (UI), and task designs within the online platforms used. Sometimes the actual scientific endeavour is hidden behind a sophisticated game narrative and participants contribute to data collection without taking notice. In other projects the full complexity of the actual challenge may be directly displayed to participants, who have to acquire specific skills (e.g., distinguishing patterns or counting objects in images) in order to contribute.

Research suggests that participants in OCS projects learn about scientific challenges and develop scientific skills (Hassman et al., 2013; Tinati, Luczak-Roesch, Simperl, & Hall, 2016). However, current knowledge of science capability learning using OCS is limited to informal settings where individual participation is driven by personal interests and circumstances. How OCS might fit within the formal setting of science education in primary and secondary schools has received little attention to date.

OCS projects have great potential to help increase children’s science capabilities at different ages, while also supporting the development of basic skills such as numeracy and reading in addition to advancing digital literacy. The drive by the Ministry of Education to develop digitally fluent teachers and learners, and innovative classroom spaces, with the associated requirement for pedagogical approaches that provide for individual student choice and learning needs (Imms, Cleveland, & Fisher, 2016; Ministry of Education, 2015) suggests that introducing relevant OCS projects may be a means of engaging students in science. There is a diverse range of science projects available online from which schools could choose. Participating in such projects provides opportunities to experience the nature of scientific investigation in fields that are not normally available to primary students, such as astronomy.

Importantly, OCS has potential to advance the expectations and intentions of NZC regarding science. The overarching and compulsory strand of the science learning area in NZC is the Nature of Science strand (Ministry of Education, 2007, p. 28). The need to develop functional understanding about the nature of science has been emphasised by the Ministry of Education through the development of objectives and resources to enhance Science Capabilities for Citizenship.^[9] This site provides teachers with resources to address the Nature of Science strand in a way that enables students to develop capabilities useful for engaging with science as citizens. The five capabilities are:

1. Gather and interpret data
2. Use evidence
3. Critique evidence
4. Interpret representations
5. Engage with science.

Empirical research suggests that students need to develop these functional capabilities for effective science literacy, rather than learning abstract scientific tenets (Allchin, 2011; Bull, 2015). To develop such capabilities, students need to participate at school in the kinds of tasks required of citizens confronting and addressing socio-scientific issues (Allchin, 2011). In doing so, they can develop a “grasp of practice” through participating in science investigations (Ford, 2008, p. 147), or “a junior version of the full game of science” (Bull, 2012, p. 27).

While OCS research in New Zealand schools is limited, anecdotal work suggests that there are potential benefits. Much of the science education literature regarding citizen science projects concerns place-based conservation and ecology projects (e.g., Newman et al., 2017). Participation by a New Zealand primary class in one such community-based monitoring project was reported to improve students’ attitudes to science and their engagement with the topic (Chen & Cowie, 2013). An important aspect noted in the study was the development of students’ identity as “scientists”—a positive change in the way in which they saw themselves in relation to science. Previous TLRI projects found that online tools enabled greater student control, ownership and responsibility for their learning; and that learning through science investigation was more successful when teachers focused explicitly in their planning and teaching on a specific scientific practice or capability (Moeed, Anderson, Rofe, & Bartholomew, 2016). Other New Zealand studies in primary science education have shown how participating in authentic scientific investigations provides opportunities for children to learn about how science works and to see themselves as scientists (Anderson, 2014; Anderson & Baskerville, 2016). The findings from empirical research outlined above highlight the importance of actions taken by the classroom teacher in making scientific behaviours and values explicit and scaffolding students as they undertake and explore scientific practices. Moreover, these findings also suggest that teachers have a key role to play in the learning that occurs through the introduction and facilitation of citizen science projects in classrooms.

Despite existing research highlighting the potentials of OCS for learning, little is known regarding how to systematically and effectively integrate this participatory science approach into the classroom environment. To address this knowledge gap, we asked the following research questions (RQs):

• RQ1: Which existing OCS projects best align with the New Zealand science curriculum and are most suited for use in the classroom?
• RQ2: What learning benefits derive from embedding OCS projects in the classroom?
• RQ3: How might participation in OCS specifically support development of science capabilities for citizenship?

3. Research methodology

3.1 Co-constructive partnership

Our research project was designed as a co-constructive partnership between three researchers at Victoria University of Wellington and four primary teachers from the wider Wellington region who are advocates of science education. These teachers were invited to be collaborators in the project because of their grounding in the Science Capabilities for Citizenship through their participation in the Science Teaching Leadership Programme (SLTP) administered by the Royal Society Te Apārangi.^[10] It is important to note that the researchers did not study the teachers, rather researchers and teachers together investigated the role of OCS projects in science education. The ethos of this partnership was that teachers and researchers were actively involved in all parts of the project, including the classification and selection of OCS projects, the adaptation of unit plans, and the development of a practitioners’ website.^[11]

3.2 Multiple case-study design

An exploratory multiple case-study design was adopted for this study. Each of the four primary school teachers and their classes who embedded an OCS project into their teaching units was treated as a case (see Table 1). The specific learning intended in each case was determined by the teacher’s specific teaching unit, which was in turn determined by a wider school plan. Teachers reviewed and selected relevant OCS projects that could be linked with the unit’s topic and goals. They implemented their units over the course of a term, introducing and embedding their selected OCS project(s). Below we provide a description of the four cases.

Table 1: Case summaries

Case	Teacher	Topic Focus	OCS Used	OCS Task	No. of Students
Case 1	A	Animal identification	Identify New Zealand Animals	Identifying animals in photos taken by motion-activated cameras	25
Case 2	B	Pollution in streams	The Plastic Tide	Locating plastic items in drone photos of beaches	62
Case 3	C	Light & sound	Agent Exoplanet & Planet Hunters	Finding exoplanets by analysing data from telescopes	44
Case 4	D	Power of light	Globe at Night	Measure & submitting local night sky brightness observations	21

3.2.1 Case 1: Teacher A

The first case consisted of 25 students between the ages of 9 and 11, where the focus of the unit was around local animal identification and behavior. It ran for more than 3 weeks. Within this unit, Teacher A identified several intended learning outcomes: making observations and collecting data about pests; understanding how scientists collect data and make observations from it; appreciating how technology can assist in making these observations; and developing an understanding of kaitiakitanga.^[12] To help achieve these outcomes, Teacher A selected an OCS project called “Identify New Zealand Animals”. The project aims to understand the impact that introduced mammalian predators, such as rats and stoats, have on urban environments. Citizen scientists were required to detect and classify animals in photos taken by motion-activated cameras around different New Zealand cities. The students had the added benefit of the professional scientist who set up this project coming into the classroom to discuss it with them and explain how the photos were generated and the aims of the project before they started (Photos 1–2). The unit ran for six weeks. Students also used the project tutorial to understand what they were required to do, and then used the OCS website to classify the animals in the photos provided there.

Photos 1–2

A researcher visits the school to introduce his OCS project to the students and explain how it can be used

3.2.2 Case 2: Teacher B

The second case consisted of 62 students between the ages of 6 and 10, where the focus of the unit was on the impact of plastic in waterways. (This was part of a larger project focusing on caring for their local stream.) It ran over 20 weeks. With this unit, Teacher B identified several intended learning outcomes: understanding how scientists use observations to help answer questions about our world; identifying and understanding how humans have changed the stream habitat and water quality; and identifying and reflecting on actions to help protect waterways. To help achieve these outcomes, Teacher B chose an OCS project called” The Plastic Tide”. The project’s aim is to build an open source machine-learning algorithm that can detect plastic washed up on beaches using images captured by drones. Citizen scientists are involved in training this algorithm by classifying the plastic and litter they see in the photos provided. Teacher B initially discussed with the students where plastic comes from and the impact plastic litter has on the environment. The teacher then introduced sand samples she had collected from a local beach, containing a range of plastic items. Students identified and sorted these plastics into groups of different types. Following this activity, students were introduced to the OCS project, where they were asked to tag and classify the types of litter in the photos of beaches around the world (Photo 3).

Photo 3: Students using OCS

3.2.3 Case 3: Teacher C

The third case consisted of 44 students between the ages of 11 and 13, where the focus of the unit was light and sound. It ran for 10 weeks. In this unit, the learning outcomes identified by Teacher C included how exploration and investigation enable studying the behaviour of light and sound, understanding of these behaviours and features, and a variety of applications of light and sound. To achieve these learning outcomes, the students undertook several practical activities aimed at developing their understanding of the properties of light and sound. Teacher C then chose two OCS projects— “Agent Exoplanet” and” Planet Hunters”— to teach about light properties. The aim of both projects is to identify exoplanets. Agent Exoplanet provides citizen scientists with images taken by telescopes to classify, while Planet Hunters provides graphs of stars’ light curves to classify. Teacher C initially introduced students to exoplanet types and used models to show how the brightness of stars can be measured by gauging the dip in light intensity that occurs when a planet passes between the star and the receiving telescope (Photo 4). Following these activities, students used support material provided by the teacher, and the two OCS projects, to understand the task they had to complete. When familiar with each project, they participated in interpreting the graphs and images.

Photo 4: A teacher showing the model of a star and an exoplanet

3.2.4 Case 4: Teacher D

The fourth case consisted of 21 students between the ages of 8 and 10, where the focus of the unit was the behaviour of light. It ran for 9 weeks. In this unit, Teacher D identified such learning outcomes as understanding the physics of light, how light pollution impacts the living and physical world, and how to critically evaluate the reliability of data. In order to achieve these understandings, Teacher D selected an OCS project called “Globe at Night”. The project aims to raise public awareness of the impact of light pollution. Citizen scientists were required to collect and submit data by observing their night skies, providing information about date, time, and location, and how they compared to one of seven magnitude charts. Teacher D spent the initial 4 weeks building students’ understanding of light, covering topics such as properties of light, methods for measuring light, sources of light, and light pollution. Following this, the OCS project was introduced, where Teacher D created a “treasure hunt” on the OCS website to familiarise students with the site. They then used some of the support material provided by Globe at Night, including videos and documentation. While the initial plan was for students to contribute to the OCS project by taking pictures of their night skies, this was not possible due to weather conditions. However, Teacher D was able to take some pictures that they could use together with magnitude charts to categorise the degree of light pollution. Students were also asked to examine data already available from the Globe at Night website to identify and suggest explanations for any patterns and trends, and then compare the data with images available from satellite maps, considering the reliability of each dataset.

Photo 5: The observations, inferences, and wonderings of Year 5 and 6 students after exploring “Globe at Night”, one of the OCS projects

3.3 Fieldwork summary

Our partnership with teachers enabled us to embed guided interviews at three different stages of the study. First, the pre-intervention stage that occurred in April 2018 sought to answer how existing OCS projects best align with the NZ science curriculum from teachers’ perspectives. It consisted of the teachers creating their teaching units to embed OCS. It also included a detailed analysis of the Ministry of Education’s Science Capabilities for Citizenship website,^[13] amongst other resources^[14] for levels 1–2 and 3–4. At the end of this stage, each of the teachers completed a questionnaire. Second, the mid-intervention stage which occurred in May 2018 sought to capture teachers’ experiences and the processes involved in embedding OCS in learning environments. Researchers took part in this stage by making observations of the activities in each of the four cases. The researchers sought to observe what was working well, what difficulties were encountered, and what actions were taken to support student learning with respect to each embedded OCS project. Third, the postintervention stage occurred in July 2018 and sought to capture teachers’ experiences having completed the implementation of their teaching unit. These questions asked teachers to explain how they helped students to engage with the OCS project, what successes and challenges they experienced in using the OCS, and what information they think teachers require when considering an OCS project as part of a science teaching unit. A final workshop was run to discuss their experiences in greater detail.

3.4 Data sources and analysis

Data sources included notes from observations, and audio recordings and notes from researcher/teacher focus-group interviews pre-, during, and post-intervention with emphasis on successes, difficulties, and key actions to support student learning and engagement. Other sources included unit plans, personal reflections by teachers, and student work examples. This wide range of sources enabled us to compare findings from multiple sources, and this triangulation strengthened the trustworthiness of findings. To answer our first research question, we also carried out an anonymous survey of teachers who have participated in the Science Teaching Leadership Programme.

Analysis of the data from the four cases was conducted using NVivo software. It consisted of a thematic analysis, where a deductive coding approach was adopted, allowing themes to emerge from the data. One of the investigators coded the unit plans and questionnaires, and the coding was then reviewed independently by two other investigators for confirmation of the emerging themes. All three agreed on the themes that had emerged from the collected data. Results are presented in the following section.

4. Research findings and discussion

The findings are structured according to the research questions that guided the study. First, we present findings regarding the alignment of OCS projects with the NZ science curriculum. Second are findings about the benefits for learning derived from embedding OCS in classroom environments. Third are findings that use experiences from the research to show how participation in OCS projects by primary school students may specifically support their development of Science Capabilities for Citizenship.

4.1 Which existing OCS projects best align with the NZ science curriculum and are most suited for use in the classroom?

4.1.1 The OCS landscape through the lens of NZC

Having had an experience in embedding an OCS in their teaching, we asked our partner teachers to consider what information they looked for from an OCS project in order for them to select an appropriate one for use in their classroom. We discussed the current landscape of OCS projects through the lens of the NZ science curriculum. We augmented data from these discussions with an anonymous survey that targeted teachers that are part of the Science Teacher Leadership Programme (SLTP) of the Royal Society Te Apārangi.

From this data we deduced eight criteria that are relevant when assessing whether OCS projects are suitable for New Zealand primary school classrooms (see Table 2). We then had cycles of iterative feedback meetings with the teachers, discussing the criteria, to confirm and refine them. Table 2 presents these criteria alongside an explanation that was derived from the practitioner data we collected and provides guidance for teachers when browsing the wide variety of available OCS projects as part of their planning.

Table 2: Criteria when selecting OCS for New Zealand primary school classrooms

Criterion	Explanation
1. Purpose of the project	Provide a general overview of the project, explaining the purpose, and what it wants to achieve, including the area of focus.
2. Project timeframe	State when the project will start, how long it will run for, at what stages participants will be required to complete their tasks (including when they will be expected to do so), and when and how results will be disseminated.
3. Requirements	Technical requirements, such as: devices that can be used app(s) and software needed accessibility (is an individual account required, or will a group one do?) Skill requirements, such as: numeracy skills literacy skills technical/computer skills
4. Support provided	Specify what support will be provided for the participants for the different stages of the project. This includes materials that are available to gain an understanding of the project, and what’s expected of the participants, such as videos and/or documentation.
5. Age level	Indicate what the minimum suitable age level for participation is.
6. Relevance to students’ learning	Explain the area of focus for the project, what aspects of this area will be touched on during the project, and how students will be exposed to it by their participation
7. Geographic scope	Indicate if the OCS project is of local, national, or international interest.
8. Data availability	Indicate if the data that is gathered or used in the project is available for access outside of the project itself for further manipulation, such as being able to collaboratively compare it with other classes/schools locally, nationally, and/or internationally

4.1.2 The role of digital devices for OCS in the classroom

OCS selection is also a technical question that goes beyond matching capability needs to incorporate human– computer interaction (HCI) aspects. Teachers therefore would benefit from understanding HCI factors that have a positive or negative effect on student learning. To assess this, we observed how device use, device features, and related technical aspects of OCS use affected student learning within our four cases.

There is a common concern among educators that more devices, ideally one device per each student, will lead to better learning outcomes. However, we observed that “one device per student” affected engagement and interaction with others in group activities. There were many cases of “face-to-screen” communication, where students communicated with each other without looking away from their screen. In groups with shared devices, more face-to-face engagement was possible. Group sizes differed from two students per device to up to six per device in some cases. In Case 1 students were assigned to teacher-selected groups designed to facilitate cooperation between students who otherwise might not have much interaction. This became a good Universal Design for Learning (UDL) opportunity because all students were able to participate and engage, including those with special needs. In Case 2 the teacher assigned students with different age and computer skills to different groups, to promote a tuakana–teina relationship where an older or more expert student (tuakana) helps and guides a younger or less expert child (teina). Additionally, the type of device played an important role for the effectiveness of group work with shared devices. The most effective cooperation observed by researchers was when students were sharing a PC with a large touchscreen. Because the nature of many OCS tasks includes aim-and-click or drag-and-draw (e.g., tagging the litter on the beach or identifying an animal on a photo), the touch screen allowed several students to actively participate in the process and not be hindered by the control of input devices (physical mouse and keyboard). Some problems observed with shared devices included fighting for device control; giving up because another student was in control; persistent nagging to gain control; and, at times, competition between groups to complete tasks, which often meant that full engagement with tasks was reduced.

We also identified a difference between the use of school devices and student devices. School devices are usually customised for school use, with limited options and restricted access, while the latter had higher likelihood to distract students and trigger exit actions. Exit actions are on-device actions students performed that led them to leave the actual OCS project and do something else instead, such as watching a video on YouTube.

While we observed on various occasions that OCS project interfaces were not straightforward and intuitive for primary school children, one issue stood out. Many children did not correctly assess the size of objects presented in photos for classification because they were not familiar with how very small artefacts had been zoomed in on. Additionally, some OCS interfaces did not have interactive features for the children to easily play with objects in order to fully examine and understand them.

As the OCS landscape is growing, there are hundreds of projects available online at various levels of completion, implemented at varying degrees of professionalism, and covering widely different subject matters. Here, we have provided some insights on what teachers can look for to align with the NZ science curriculum. We have also hinted on the emerging HCI issues to look out for in order to make the learning experience interactive and productive.

4.2 What are the learning benefits derived from embedding OCS projects in the classroom?

We found that at primary school level, OCS provides unique opportunities for teachers and their students to engage with professional scientists in learning environments, to stimulate interest and engagement with science in young learners, and to support the learning and acquiring of critical skills and capabilities. Key areas include student engagement, learning from participation, contextual learning, and learning about science itself. These are presented here.

4.2.1 Student engagement

All four teachers highlighted student engagement as a strongly positive outcome of using the OCS. Teachers A and B connected students’ engagement to participation in scientific work. Participation gave students “a view of the bigger picture much more than I expected.” They saw “real life science at work and the students felt the importance of their data gathering” (Teacher A). Teacher B noted: “There was very real global connection to a local problem and the children easily related to this. They felt empowered by contributing to a project that was being used by scientists and which might make a difference to how people treat Papatūānuku.”

For Teachers C and D, the OCS engaged students by extending the learning contexts and applications available to them; it provided “the ability to connect to an authentic context for their learning about the physics of light, which normally would be completely inaccessible for them” (Teacher C). Teacher D also felt that the OCS added usefully to the learning students did in the rest of the unit: “Students were interested in the context and using the OCS helped make links to the learning we had done around the physical properties of light and light pollution and how it impacts on humans, the natural world, and scientists.”

Student engagement appeared to increase when, as in Cases 1 and 4, there was a strong connection with the scientist responsible for the OCS. A common comment from focus group students in Case 1 was that they liked helping the scientist by identifying the animals in the photos. In Case 4 observations, when the teacher referred to a local scientist who had talked with the class about his involvement in creating a dark sky zone in a nearby location, the students became visibly and verbally excited. Involvement in a local related issue as in Cases 1, 2, and 4 also supported students’ engagement: “The children have seen a bigger picture. We are not the only people who are gathering and interpreting data about plastic pollution. This increased the importance of the science we are doing in their eyes and also their self-belief in seeing themselves as scientists” (Teacher B). Teacher A noted how much students enjoyed seeing the completion rates grow on the OCS website: “When we started this (6 weeks ago) the project was just over 50% complete. A couple of boys raced up to me this morning, telling me that today it is on 99%! We contacted [scientist] about this and he is keen to share the data with us once the analysis is completed.” She observed that some students used the OCS at home and that a “Healthy competition between peers, siblings, and friends stemmed from this too—-discussions about what they observed outside of class time when they met up first thing each day.”

Students generally felt positive about using OCS in their learning. On average, 72% indicated they liked learning using OCS, of which 40% liked it a little whilst 32% liked it a lot. On average, 29% were less positive, of which 19% did not like it much and 9% not at all. A detailed presentation that shows distribution of student responses across cases in shown in Figure 1.

Figure 1: “How much did you like using the OCS project?” Number of students per response across cases

4.2.2 Learning from participation in OCS

On a rating scale between 1 (not at all) to 4 (a lot), students mostly indicated that they found OCS helpful in their learning (see Figure 2). We further investigated the areas of learning that students identified as mostly enhanced by the use of OCS. Figure 3 shows that many students indicated that participating in OCS improved their science capabilities, such as learning to observe carefully and getting better at interpreting graphs. Many students also identified OCS as helpful in context-related learning about the topic (Figure 3). For instance, a Case 1 student said they learnt that a stoat is a mustelid from the OCS; students in Case 3 commented that they had learnt what a transit of a planet was and how it created a dip in light intensity; many in Case 4 described learning about the effects of light pollution from using the OCS site. A few students indicated learning about the nature of scientific work, e.g., that perseverance and patience are necessary. Quite a few students identified learning that was directly observational, (i.e., what the OCS participation showed them). For example, in Case 2, some students commented there was lots of rubbish on the beach. Some students, especially those from Case 2, described their learning as value judgements or actions they could take in response their experience with the

OCS: “There is too much rubbish on the beach” and “We need to put rubbish in bins.”

Figure 2: “How much did OCS help you learn?” Number of students per response across cases

Figure 3: Students’ perceptions of the areas of learning improved by engaging in an OCS project across cases

4.2.3  Contextual learning

The teachers all noted learning that addresses the contextual strands of the science learning area of NZC as an outcome from using the OCS. In Case 1, Teacher A identified that students learned about “the behaviour of animals—recognising characteristics and habitats”. Teacher B identified that “students made very rapid links to their prior learning about marine environments and to their current learning about sustainability, plastics in the environment, and litter in our community which gets into our storm water.” In Case 3, Teacher C felt that the OCS “helped to support their learning about the physics of light, related to light’s properties of travelling in a straight line, the nature of shadows, and the amplitude of light.” Teacher D commented that “students can discuss properties of light, light pollution, its impact on humans, the natural world, and astronomical scientists.” In her case the OCS website was helpful in supporting student inquiries about the effect of light pollution: “The Globe at Night project provided in-depth information and interactives about light pollution and links to other sites that provided further educational resources that were helpful” (Teacher D).

It is important to note that students more often attributed their contextual learning to means other than the OCS. Questionnaires asked them to identify their most important learning from the unit and how this learning occurred. Contextual learning was the most common response to this question. While the OCS was sometimes described as agentic in their contextual learning, particularly in Case 4 where they used the OCS as a source of information, most often the teacher was seen as the source. In Case 3, practical work during the unit also played an important role in students’ development of concepts about light. These findings suggest that it is important that an OCS is not the seen as the whole science unit”.

4.2.4 Learning about science itself

Benefits for learning about the nature of science from OCS were apparent through observations and in comments from teachers and students. Students were able to work with data that they could not have otherwise accessed. The opportunity to work with real data created opportunities, difficult to reproduce in normal classroom situations. The OCS tasks were largely data-interpretation tasks, which meant students experienced directly the difficulties and issues that can occur in making sense of data. In Case 1 students were identifying animals in black and white images taken by motion-activated cameras around different New Zealand cities. Images were often indistinct, blurred, and problems of scale commonly arose: i.e., was it a large animal like a cat further away, or a smaller animal like a hedgehog close to the camera? Students were concerned about making correct identifications and were observed discussing and debating decisions. In Case 2, the plastic objects in images of the beach were difficult to see and it was hard to tell without experience if they were actually plastic, or a piece of seaweed, for example. In Case 3, students were required to do multiple calibrations of star brightness across a range of sky backgrounds as part of identifying possible exoplanets. In preparing to gather and categorise their own images of the night sky, students in Case 4 practised matching images with magnitude criteria provided by the OCS website. As Teacher D noted, “Students made their own judgements and then shared them with the class and recorded them. It became apparent that even using the magnitude guide provided by the Globe at Night project there was significant disagreement about each photo.”

The teachers recognised that their students’ uncertainty and concerns about making accurate interpretations were opportunities to promote productive class discussions about reliability of data and its interpretation in science: “We talked about our school values—especially integrity and how as citizen scientists we had to show integrity with our observations” (Teacher A); “The project gave me a real context to discuss scientific thinking, accuracy and repetition of experiments” (Teacher B); “the Globe at Night provided a good platform to explore and discuss data collection and the reliability of evidence” (Teacher D).

Student comments in questionnaires and focus groups indicated that learning about uncertainty and the role of human interpretation in science occurred from these opportunities: “It’s sort of like a lot of people look at it. If a lot of people go ‘Yeah, I think this is a planet,’ then a scientist will look at it and then another scientist will look at it and it might be a planet after enough people agree” (Focus group, Case 3); “People don’t really trust one person when they make a claim that they found a planet or something. You need lots of people claiming that they found a planet” (Focus group, Case 3); “I think that scientists … they don’t just try it once. They try it many times to see if they get the same answer or a different answer” (Focus group, Case 4). Younger students in Case 2 did not articulate their ideas as clearly, but their responses indicated ideas about the need for rigour. They learned about “searching thoroughly” and that classification can be difficult because “some things look like a thing, but it is not” (Focus group, Case 2). Students participating in these OCS projects had first-hand experience that science involves interpretation and human sense-making.

4.3 How might participation in OCS specifically support development of science capabilities for citizenship?

Figure 3 above shows that improved science capability was the most commonly reported learning that students attributed to the OCS project. It appeared that the experiential and participatory nature of OCS, and the challenging judgements required, made capability-related learning explicit for young learners. Many also associated capabilityrelated learning specifically with science. When asked to identify the scientific things they learned from the science unit, many students, particularly in Cases 1 and 2, again identified capability-related learning, indicating that they got better at observing or identifying. This was interesting because existing literature tends to suggest the most common response to be an aspect of contextual learning (Khishfe & Lederman, 2007; Anderson, 2012; Moeed, Anderson, Rofe, & Bartholomew, 2016). Students most often notice topic-related learning that is new, whereas their developing scientific capability, or growing experience of the science discipline, is not as immediately apparent to them. The high degree of student reporting of capability-related learning appeared significant.

We therefore sought to understand the specific capabilities that were the focus of the learning opportunities created by the planned incorporation of OCS, and to describe the ways in which teachers maximised OCS-related opportunities for capability development. Table 3 shows coding frequencies for the capabilities in each case. As can be seen, Gather and interpret data was the most common capability identified in all cases. This was expected, considering the nature of the OCS embedded in the learning units for all the cases. In Cases 1 and 2 students were required to interpret photographic images. In Case 3 one of the OCS projects required calibration and then identification of possible dips in star intensity from images, so again the focus was on interpreting data. In Case 3 students were required to interpret graphs of light intensity. In Case 4 students were required to gather and interpret their own data, but also used data available on the website in a teacher-created task that required students to look for patterns in reported light pollution data over several years. Thus, the coding frequencies reflect the emphasis on both gathering and interpreting data and on critiquing evidence. Other capabilities were not directly connected to the use of the OCS but were more a result of the way the teacher embedded the OCS within the overall unit.

Table 3: Overall coding frequencies for science capabilities across cases (for all data)

	Gathering and interpret data	Use evidence		Critique evidence	Interpret representations	Engage with science	Total
Case 1	45	7		4	7	11	74
Case 2	34	14		2	5	11	66
Case 3	51	5		4	46	11	117
Case 4	56	6		41	8	15	126
Total	186	32		51	66	48	383

KEY (gradient scale: “green-yellow-red”: green color shows the most coded, red- the least)
	40+ codes
	30-39 codes
	10-29 codes
	06-10 codes
	05 and below

Figures 4 and 5 show that patterns in frequency of students’ perceptions of learning mostly reflected the emphasis in teachers’ planning and perceptions concerning learning for each capability. Those that received most emphasis in teachers’ planning and actions were those that appeared most often in students’ responses across questionnaires, focus groups, and during observed lessons—although learning to critique evidence was not as explicitly evident to students as some of the other capabilities. Students also mentioned using evidence more than teachers. Data sources included in the “Teacher’s planning and actions” category are their plans, reflections before during and after the unit, and the observations of their teaching using the OCS. Teachers’ perceptions of learning are again from teacher reflections. Data sources for the “Students’ response and perceptions” category are student questionnaires, focus group interviews, and classroom observations.

Figure 4: Teachers’ perceptions regarding science capabilities enabled by use of OCS projects

Figure 5: Students’ perceptions regarding science capabilities learnt from engaging in OCS projects

Each teacher readily identified the potential for science capability development inherent in their selected OCS and ways in which they could use it to support other capabilities (as shown in Table 4). Their post-intervention comments show that they could talk with some justification about their students’ capability development and the impact of the OCS on this learning. As described in Table 4 below they were able to recognise the potential for such learning regarding the OCS they selected and used.

Table 4: Teachers’ perceptions of potential and actual opportunities for science capability development associated with the OCS

Teacher	Perceptions about science capability development opportunities prior to implementation	Perceptions about science capability development opportunities post- implementation
Teacher A	Gather & interpret data: students will be making observations and recording data along with other online citizen scientists for gathering evidence about pests in the local area. The photos that they observe and identify will hopefully be engaging and they will be encouraged and see how important it is to make careful observations as citizen scientists. Wigand, Steinfield & Markus (2005) students will use the data gathered to make decisions about where to trap in our school. Critique evidence: students will see how others’ observations are also considered and used in making accurate evaluations. (If 6 people make an observation of a “rat” and 1 as a “bird”, it is more likely that the sample is a rat.) Engage with science: students will understand the importance of what we are doing as online citizen scientists and how we are contributing to a real-life data-gathering project.	Lots of interpreting data. Making connections with our own pest trapping at school—shifted traps with purpose and understanding after using OCS instead of moving them randomly. Much care was taken with observations— justifying what they thought and having the option “uncertain” where necessary. Encouraged discussions, sharing and support. Connections were made with our own school environment and where our pest tracks were found.
Teacher B	Gather & interpret data and Interpret representations: The Plastic Tide supports the use of Capability 4, as the children are using photographs to identify plastic waste. Engage with science: The Plastic Tide will help them link our local issues with the global problems that plastic waste is creating.	The project has added another level to our use and understanding of the five science capabilities. My evidence was the improvement in their own gathering and interpreting of data. Evidence includes the physical improvement I have seen in careful and accurate observations and recording of data. The children gained an awareness of the use of photographs to gather evidence / data sets. They had to differentiate between a variety of types of plastics and make judgements about what they were seeing. Engage with Science: This one was huge. It gave the children a global appreciation of a local problem and has encouraged them to pursue their local study and present their findings to our local community.
Teacher C	Gather & interpret data: these OCS projects allow me to have students understand how these graphs are created, and then skip the painstaking, laborious, and in the case of exoplanets impossible process of gathering the data. They can go right on to interpreting the graphs and be fully engaged and excited by the fact that they are using REAL data and that their interpretations are used by REAL scientists for a REAL purpose. Interpret representations: they will also be interpreting representations as they use a model, create graphs and interpret graphs as part of the OCS.	Students practised “Interpreting representations” extensively. I think the students were quite successful and spoke knowledgeably about what they were doing.
Teacher D	Gather & interpret data: students will make their own observations of the night sky and of light pollution in their local neighbourhood. We will use this data to interpret the level of light pollution in different parts of the Hutt Valley. Use evidence: we will use our evidence to discuss the level of light pollution in the Hutt Valley and use the data gathered on the OCS site to draw conclusions about global levels of light pollution. Critique evidence: support students to critique data-collection methods and the reliability of the evidence/data collected. Support students to see the need to for multiple points of data collection. Interpret representations: students will create a map of light pollution in their local area (using Google Maps to support). We will contribute our data to the global set and will discuss and explore how it is represented on the OCS website, thinking about how it is represented, how do we read it, what does it tell us, and if there are other ways we could represent data. Engage with science: the OCS has suggestions for how to take action to limit pollution. Students will use these ideas to take action in our local community.	Students developed their ability to critically evaluate evidence. They developed their ability to identify factors that affect the reliability of the data collected through the Globe at Night project Using a second light pollution data set (from satellite measurements) supported students to see gaps in the data and to think critically about what factors can account for these gaps.

Teachers helped their students to understand what was involved in the OCS task, noticed opportunities to support capability development, and created further opportunities for students. A summary of key teacher practices associated with science capability development from the use of the OCS, as observed by researchers, are summarised in Table 5.

Table 5: Teacher practices that supported science capability development using OCS

Planning:		Incidental teaching:
Selection of a relevant OCS project	Teachers reviewed OCS projects and selected one that fitted the intended focus of their planned unit, but also that they felt would provide engaging opportunities that would extend their students’ experiences.	Teacher noticing of opportunities for capability development	Teachers actively noticed opportunities for capability development. For instance, Teacher A noticed students’ debate and concern about accurately identifying the images. She discussed integrity of evidence and highlighted how to record the degree of certainty of the classification on the website.
Identifying a specific science capability pertinent to the OCS project	Each teacher developed specific learning goals for  capability development that related closely to the opportunities within the OCS. An important component was teachers’ ability to recognise these opportunities for capability development.	Use of science capability focused questioning	Teachers regularly used questions from the capability website, e.g., “What do you notice?” and “What makes you think that?” Teacher C asked students to critique the exoplanet models he used, by asking them what they did not show as well as what they showed, and how they may differ from the reality. These ideas are suggested on the science capability website section supporting the Interpret representations capability.
Making the OCS task relevant and real for students.	For example: choosing a locally relevant project connecting to a scientist using models to build understanding of the exoplanet data sorting through sand to find and categorise plastic rubbish. Making the OCS project relevant seemed to provide student motivation that facilitated capability development, e.g., making sure they were being accurate and careful, or persevering with repetitive tasks, enabling further practice	Making to connection to science explicit	All four teachers regularly connected student behaviours and practices with science. For example, Teacher A asked students to think about why it was important for the scientist to have reliable identifications. She connected the school value of “integrity” with this science behaviour. Teacher B often asked children to think like a scientist and highlighted student examples of scientific thinking.
Explicit teaching of the capability	Teachers planned ways to support development of the OCSrelated capability. For example, Teacher D used questions like those on the science capabilities website in a scaffolded sequence supporting critique of evidence: “What patterns can you see in the data?” “What data is missing?” “What do you think could explain the missing data?” “How much do you trust the data? Why?”

These practices highlight the importance of teachers understanding the intention of the curriculum and ways to support science capability development if OCS projects are to be used advantageously.

5. Limitations

There were some limitations to our research project. First, we acknowledge the limitation in scope with only four cases and purposefully selected teachers with considerable professional development in NZ science education. We have ongoing endeavours to expand this scope with a bigger national project. We also look forward to grounding our assessments in the NZ curriculum and science capabilities and intend to expand our assessment and evaluation of the benefits of OCS along their guidelines. Overall, the provoking insights emerging from this exploratory study are difficult to ignore and can be a start in building towards successful OCS projects from both the perspectives of professional scientists, and of teachers and learners in NZ educational environments.

6. Conclusion

This project was an initial exploratory step to understand the intersection of science education, citizen science participation, and digital technologies. We started with limited knowledge about how OCS projects may facilitate formal science education, which interventions involving OCS participation may work in classrooms, and which aspects of learning and teaching should be measured to assess the usefulness of OCS participation in science education. Our work was a unique interdisciplinary endeavour into this matter that brought together academic perspectives from education as well as human–computer interaction. Hence, the results have significant value for both of these academic communities and can be summarised as follows.

First, we identified a gap in the way citizen science in general and OCS in particular is conceptualised. The current definitions are not well aligned with the goals of a curriculum such as the one in New Zealand. This is particularly evident in the observation that citizen involvement is mostly limited to crowdsourcing data and participation in forum discussions, and that participants cannot access the collected data for their own analyses.

Second, our observations reveal that participation in OCS as part of science education is useful educationally, particularly for expanding students’ appreciation of science as a discipline and developing Science Capabilities for Citizenship. However, the embedding of OCS is crucial for its effectiveness. While participation in the OCS provided potential for science learning, each teacher used their knowledge of the capabilities and science to firstly recognise these opportunities, and then to shape and add to them to make them relevant and accessible to their students to promote productive learning.

Third, we provide lessons learned about the role of digital devices in classrooms and modern learning environments. The differences we observed in face-to-face and face-to-screen communication when students were working in groups with shared devices (face-to-face communication) compared to groups where students had individual devices (face-to-screen communication) are particularly important in this regard. They convey the message to schools (especially those that self-identify as under-resourced) that a smaller number of devices utilised for group work may be desirable in promoting collaborative learning.

7. Implications and recommendations

From our research results we deduce implications and recommendations for future research as well as for teaching and learning practice.

7.1 For science education research

• Longitudinal research is needed to fully understand the impact of interventions like the ones described here. Questions to answer include “Does the observed student engagement have sustained positive effects on (i) their attitudes towards the sciences in general; (ii) their performance in science subjects at school; and (iii) their likelihood to move on to a professional career pathway in the sciences after they leave school?”
• It needs to be investigated how role-model teachers, such as the ones in our project, can multiply their motivation and knowledge to other teachers within and beyond their schools.

7.2 For science education practice

• Teachers need to be able to recognise opportunities for science capability development and learning about science itself if OCS is to be of benefit in addressing the intentions of NZC.
• Linking the OCS to a local issue or context appears useful in engaging students.
• Having limited digital devices is not a barrier for purposeful engagement with OCS projects in class as meaningful group activities that involve shared devices can be developed.
• General purpose devices and applications are not suited for students at all ages to engage purposefully with OCS. While the flexibility of general-purpose technology may bear the potential for meaningful research trajectories in and out of OCS platforms (e.g., critique of research design by comparing projects), this flexibility increases the potential that students get distracted and lose focus, specifically at primary school age.

7.3 For human–computer interaction research

• OCS platforms should follow principles of the open science movement, making all data collected available not only to other researchers on request but also the citizen scientists themselves. This may stimulate unanticipated insights into the data and create new research opportunities into the practices of citizen scientists as well as the kinds of research questions they may ask and how these differ (or are similar) to the those asked by professional scientists.
• Emphasis on R & D is needed into platforms that allow for citizens to engage with science at all stages of the scientific life cycle.
• As step towards meeting the outcomes of this project, we released a digital resource guide on how OCS can be utilised in science education practice on the Science Learning Hub.^[13] This resource is available for all teachers in New Zealand and provides an inventory of OCS projects along with descriptions about how these can be used in alignment with NZC. It includes small video small clips in which our partner teachers describe experiences and best practices around using OCS in classroom activities, and material that was successfully used in class and that can be adapted in similar settings.

 

Footnotes

1. 1. https://www.sciencelearn.org.nz/citizen_science ↑
   2. Educational Assessment Research Unit & New Zealand Council for Educational Research ↑
   3. Instant Wild: https://instantwild.zsl.org/ ↑
   4. Penguin Watch: https://www.penguinwatch.org/ ↑
   5. Galaxy Zoo: https://www.galaxyzoo.org/ ↑
   6. Operation War Diary: https://www.operationwardiary.org/ ↑
   7. Eyewire: http://eyewire.org/ ↑
   8. Eterna: http://www.eternagame.org/ ↑
   9. http://scienceonline.tki.org.nz/Science-capabilities-for-citizenship/ ↑
   10. https://royalsociety.org.nz/what-we-do/funds-and-opportunities/science-teaching-leadership-programme ↑
   11. https://www.sciencelearn.org.nz/citizen_science ↑
   12. The Māori concept of guardianship, for the sky, the sea, the land, and living things ↑
   13. http://scienceonline.tki.org.nz/Science-capabilities-for-citizenship/Introducing-five-science-capabilities/↑
   14. For instance, the aims and objectives of the NZC science learning area. ↑

 

Acknowledgements

We would like to acknowledge the support and advice throughout the project of Associate Professor Azra Moeed and Dr Craig Rofe from the Faculty of Education, Victoria University of Wellington, Dr Cathy Buntting from the University of Waikato, and Dr Rosemary Hipkins from the New Zealand Council for Educational Research.

References

Allchin, D. (2011). Evaluating knowledge of the nature of (whole) science. Science Education, 95(3), 518–542.

Anderson, D. (2012). Teacher knowledges, classroom realities: Implementing sociocultural science in New Zealand Year 7 and 8 classrooms. Unpublished doctoral dissertation, Victoria University of Wellington, New Zealand.

Anderson, D. (2014). What makes it science? Primary teacher practices that support learning about science. Set: Research Information for Teachers, 2014(2), 9–17.

Anderson, D., & Baskerville, D. (2016). Developing science capabilities through drama: Learning about the nature of science through a guided drama–science inquiry process. Set: Research Information for Teachers, 2016(1), 17–23.

Bolstad, R., & Hipkins, R. (2008). Seeing yourself in science. Wellington, New Zealand: New Zealand Council for Educational Research.

Bull, A. (2012). Making the most of science learning opportunities in the primary school. Set: Research Information for Teachers, 2012(1), 26–28.

Bull, A. (2015). Capabilities for living and lifelong learning: What’s science got to do with it? NZCER, Wellington. Retrieved from: http://www. nzcer.org.nz/research/-publications/capabilities-living-and-lifelong-learning-whats-science-got-do-it.

Caygill, R., Hanlar V., & Harris-Miller, C. (2016). Student attitudes to maths and science: what we know from New Zealand’s TIMSS 2014/15 results for Year 5 and Year 9. Retrieved from: http://www.educationcounts.govt.nz/publications/series/2571.

Chen, J., & Cowie, B. (2013). Developing ‘butterfly warriors’: a case study of science for citizenship. Research in Science Education, 43(6), 2153–2177.

Curious Minds. (2019). What is Curious Minds? Retrieved from: https://www.curiousminds.nz/about/

EARU & NZCER. (2012). Wānangatia te Putanga Tauira National Monitoring Study of Student Achievement Science 2012. Retrieved from: https://nmssa.otago.ac.nz/reports/2012_Science_ONLINE.pdf

Ford, M. (2008). ‘Grasp of practice’ as a reasoning resource for inquiry and nature of science understanding. Science & Education, 17(2), 147–177.

Hassman, K. D., Mugar, G., Østerlund, C., and Jackson, C. (2013). Learning at the seafloor, looking at the sky: The relationship between individual tasks and collaborative engagement in two citizen science projects. In Proceedings for 10th International Conference on Computer Supported Collaborative Learning.

Imms, W., Cleveland, B., & Fisher, K. (Eds.) (2016). Evaluating learning environments: Snapshots of emerging issues, methods and knowledge. SpringerLink (Online service).

Khishfe, R., & Lederman, N. (2007). Relationship between instructional context and views of nature of science. International Journal of Science Education, 29(8), 939–961.

Lederman, N. G. (2004). Syntax of nature of science within inquiry and science instruction. In L. B. Flick & N. G. Lederman (Eds.), Scientific inquiry and nature of science (pp. 301–317). Dordrecht: Kluwer Academic.

Lederman, N. G., & Lederman, J. S. (2014). Research on teaching and learning of nature of science. In N. G. Lederman & S. K. Abell (Eds.), Handbook of research on science education (Volume II) pp. 600–620. Abingdon: Routledge.

Leveraging the power of place in citizen science for effective conservation decision making. Biological Conservation, 208, 55–64.

Ministry of Education. (2007). The New Zealand curriculum. Wellington: Learning Media. Retrieved from:http://nzcurriculum.tki.org.nz/ The-New-Zealand-Curriculum

Ministry of Education. (2015). Towards digital fluency. Retrieved from: https://www.education.govt.nz/assets/Uploads/Towards-DigitalFluency.pdf

Moeed, A., Anderson, D., Rofe, C., & Bartholomew, R. (2016). Beyond play: learning through science investigation. Retrieved from: http:// www.tlri.org.nz/tlri-research/research-completed/school-sector/beyond-play-learning-through-science-investigation

Newman, G., Chandler, M., Clyde, M., McGreavy, B., Haklay, M., Ballard, H., … & Gallo, J. (2017). Leveraging the power of place in citizen science for effective conservation decision making. Biological Conservation, 208, 55-64.

Shadbolt, N., & Burton, M. (1995). Knowledge elicitation: A systematic approach. Evaluation of Human Work: A Practical Ergonomics Methodology, pp. 406–440.

Tinati, R., Luczak-Roesch, M., Simperl, E., & Hall, W. (2016). Because science is awesome: Studying participation in a citizen science game. In Proceedings of the 8th ACM Conference on Web Science, pp. 45–54, ACM.

Tytler, R., Symington, D., Kirkwood, V., & Malcolm, C. (2008). Engaging students in authentic science through school–community links: Learning from the rural experience. Teaching Science: The Journal of the Australian Science Teachers Association, 54(3), 13–18.

Research team

Markus Luczak-Roesch markus.luczak-roesch@vuw.ac.nz
Associate Professor, School of Information Management, Victoria University of Wellington

Dayle Anderson Dayle.Anderson@vuw.ac.nz
Senior Lecturer, School of Education, Victoria University of Wellington

Brigitte Glasson Brigitte.science@gmail.com
Science Education Consultant

Cathal Doyle Cathal.Doyle@vuw.ac.nz
Lecturer, School of Information Management, Victoria University of Wellington

Rodreck David Rodreck.David@vuw.ac.nz
Research Assistant, School of Information Management, Victoria University of Wellington

Yevgeniya Li Jane.Li@vuw.ac.nz
Research Assistant, School of Information Management, Victoria University of Wellington

Cameron Pierson Cameron.Pierson@vuw.ac.nz
Research Assistant, School of Information Management, Victoria University of Wellington