 |
Loki Jörgenson1
Nathalie Sinclair2 3
loki@cecm.sfu.ca
nathsinc@educ.queensu.ca
Simon Fraser University
Queen's University
Burnaby, BC CANADA V5A 1S6
Kingston, ON CANADA K7L 3N6
Keywords: participatory design, children, interactive learning environments, mathematics
Traditionally, researchers have evaluated software for learners by observing the meanings and understanding that children derive from interaction with the software. More recently, researchers have used the iterative design process in which a certain tool converges to a final design through successive observations and modifications of an initial tool. In both cases, researchers have concentrated on defining and constraining interface features for the learner with the hope that given interactions and representations will guide and motivate the learner in the right direction. However, children are rarely included in the actual development of the software. In this paper, we describe a project in which children were invited to participate at the formative stage in the research design of component-based mathematics education technology. We outline the research methodologies used to facilitate this process and recommend several strategies for enhancing collaboration between researchers and children.
Children's input into the design process of new learning technologies can be critical for successful development [Cypher and Smith1995,Druin et al.1998,Oosterholt et al.1996], The value of involving users in the design process has long been recognized by the human-computer interface (HCI) community. More recently the participatory design (PD) approach has shown to be effective, supporting users and designers in collaboration as partners in the design process [Scaife et al.1996]. However, using PD with children in developing learning technologies presents two challenges:
The notion that students can be effective researchers is not a new one, and indeed has been found in other studies [Druin et al.1997]. Not only is it empowering for students to work within such a framework, but it affirms that students are capable of identifying features and functionalities which are the most motivating and engaging for themselves. However, as children cannot typically develop their own learning goals, Scaife et al [Scaife et al.1996] argue that children should be viewed as "native informants'' in the design process. This position emphasizes the existence of certain constraints that researchers may introduce with respect to aspects of learning/teaching practices.
A common technique for facilitating collaborative design is to use low-tech tools that allow users and developers to collaboratively construct informal prototypes. One such example of this technique which has been applied successfully is called PICTIVE [Nielsen et al.1992]. In using familiar materials and easy-to-learn low tech methods, this technique is particularly well suited for use with children. However, as the ultimate use of the new technology is computer-based, it is important for low-tech prototyping to be informed by some familiarity with computational environments.
We have been pursuing a component-based approach which exploits the rapidly growing potential of network technologies such as Java, JavaBeans, MathML, OpenMath, etc. The toolkit under development offers discrete, inter-connectable pieces called OpenMath JavaBeans that are manipulated via intuitive visual environments. While their functionalities are still relatively crude, they already make it possible to simplify the programming process to the point where non-programmers with limited computer literacy can ``write'' their own programs. Eventually it should be possible for students to rapidly assemble powerful tools and resources into learning contexts.
Before this goal can be realized, many questions need to be answered. First among them include:
Can we create a toolkit that a typical middle school teacher can use to build interactive applications?We are concerned that the planned technology be able to meet the minimum requirements for building useful resources. Another important question is:
Will children be able to learn and apply the requisite mental framework in order to assemble their own learning resources?The construction of resources from components requires an understanding of event-driven processes and an ability to visualize the flow of information. This may be unreasonably complex for middle school students.
The intention was to engage a group of middle school students in the design and assessment of software in the early stages of development. They were to be included in the process as conscious participants, aware of their role and active in their contribution to it. Although the topic under review was mathematics, this activity was incorporated within the information technology curriculum.
The goal of the third phase was to identify the degree to which students could adopt the design methodology required by the technology. Would it be possible for the students themselves to create their own learning resources? In order to explore this possibility, a low-tech design tool dubbed simCHET7 was used. Based on the visual programming environment of Java Studio8, it offers a facsimile environment based on paper, string and markers that supports group learning and interaction, hands-on construction and heuristic component design. Three 2 hour simCHET trials were undertaken within a period of one month.
Data collection occurred at a variety of levels. To support and maintain an atmosphere of collaboration, various techniques were used to support students' active contribution to the data collection.
The following types of data were collected over the period of the study.
An ethnographic approach to the study was taken to maximize the interaction between researchers, students and the school context. The ATiC lab provided the requisite expertise and data gathering/analysis services.
A broad range of observations were made, culled from the various sources mentioned. Those bearing directly on the participatory process are reported here9:
Although the students made significant contributions to the development of the technology, they were not fully cognizant of them. Their experiences transpired within the familiar framework of their regular classroom activities and subsequently the students tended to maintain their traditional roles, vis-à-vis the teacher. In essence, the students were slow to take ownership of their part in the research.
When queried, the students commented that having their regular teacher lead the project engendered an inauthentic working environment. And they often felt overshadowed by the teacher's mathematical and technological expertise. Indeed, they responded much better when working with the other members of the PDG.
The students also suggested that oral de-briefing sessions were more effective than journal writing. They found it easier to communicate orally and were often inspired through the comments of their peers. They also felt that the journal writing was more ``school-like'' and were less motivated to contribute through this means.
One particular oral-debriefing session was conducted in order to seek advice from the students on how the collaborative process could have been better achieved. Several suggestions came out including:
Although the students' distance from their researcher role was prevalent throughout the project, the students became more aware of it as the project unfolded. They were certainly encouraged when they saw implementations or modifications that they had suggested. However, it was during the final simCHET implementation that the students felt most legitimized in the collaboration. At this stage, the students were given open license to conceive their own applets. With full support from the members of the PDG, they worked with familiar low-tech materials and the focus shifted from a teacher/school orientation to that of player/community.
It was during the final phase that the researchers gained the most from the collaboration with the students. Not only were answers found to the study's motivating questions, the PDG found itself reviewing its own assumptions with regards to higher-level issues, ranging from pedagogy to technical implementations. Most importantly, the students gave valuable feedback on the actual composition of the toolkit they should be given, the most intuitive ways for the students to use the toolkit, and the things which would be possible and desirable for them to build.
The students were most motivated when playing their ``initiator'' roles during simCHET. We had planned to provide extra incentives should the students be at a loss of ideas or unmotivated but this was not necessary. It became apparent that students wanted to build games, and they especially wanted to build collaborative ones wherein more than one person played together, possibly at different physical locations.
Although the idea of networked interaction had been in development at the CECM, it had not yet been introduced to the students. It was both suprising and gratifying to see that they could see the potential of the network and the possibility in the technology they were using. In addition to a 'Bean for collaboration, there were other toolkit components required that had not been anticipated by the developers; these included 'Beans for object collision, coloring, and random number generation.
We did not find the students restrained by pre-conceived notions; this could be due to the fact that they were building highly authentic artifacts with which they had had no previous experience. During the simCHET stage of the project, the students were involved in more deep and subtle design concepts. We would argue that they were mature enough to grasp these concepts, well enough to be able to apply their abundant imagination and enthusiasm to the generation of original designs.
For any subsequent use of collaborative design in a middle school context, we would make several recommendations over our original implementation. Some of these are adapted from other studies with different goals and age-groups; some are derived from the students' suggestions; and some arise out of our own observations:
Druin et al [Druin et al.1997] suggest that students and adults should work together in teams and that there should be more than one of each per team. The objectives are on the one hand to prevent the students from feeling overshadowed by adults and, on the other, to diminish the dominance of the ``school-like'' dynamic. A one-to-one ratio would probably be ideal but at least one adult to two students should be sufficient. Due to the small number of adults available during this project, it was difficult to achieve this goal. Druin also suggests that adult-to-adult interaction is important and that adults should be informal and playful in their interactions both with other adults and with students.
Klawe and Phillips [Klawe and Phillips1995] emphasize the importance of carefully explaining the role of a researcher and the kinds of activities that a researcher might undertake. This includes encouraging students to formulate research questions and to pay attention to rigor and detail while conducting research. Other strategies might include inviting researchers to speak about their work and having students conduct a preliminary research project of their own which might or might not be related to the final project. As well, students' own opinions on what they need to feel included might be solicited.
The students themselves made the suggestions that their roles as researchers would have felt more genuine had they had the opportunity to visit the PDG lab and to interact more often with members of the PDG team. Evidence to support the potential positive impact of this strategy emerged when the students were shown a screenshot of the applet building environment used by the researchers, Java Studio. They were immmediately captivated by the original and visual interface.
They began asking questions about the different components of the interface and how the teacher worked with it. They were able to compare their own activities with simCHET with the environment simCHET was emulating. Had the students been able to see the actual physical environment in which the PDG members worked, had they been able to sit at the computers and explore the interface, the expectation is that they would have developed a stronger personal connection to the project.
Clearly emphasis needs to be placed on creating an authentic collaborative experience. In our case, having the teacher lead the project at the school detracted from their sense of participation. Although the presence of a familiar person helps create a comfortable environment for the students, we propose that the teacher take on a more secondary role in any research collaboration.
As there are many different personalities within a student body, it is difficult to accomodate everyone's participation. For example, while oral debriefing was found to be most effective for eliciting useful feedback, there was a concern that weaker, less verbose students' potential contributions would be left out. A strategy which would address this concern would be to hold smaller group de-briefings in addition to full class ones. Another hybrid format might be to have smaller discussion groups in which one student was responsible for recording the comments and suggestions of the group members.
We were able to observe that the students were more motivated to collaborate upon seeing their suggestions implemented. A continuous flow of information between the students and the developers is conducive to creating an authentic collaborative environment. This might include trying several suggestions regardless of any judgements held by the developers. This would demonstrate the value of their input early and also show them how even unsuccessful suggestions contribute to the design process. In any case, it is not unusual for students to propose ideas which never occur to developers.
Druin et al [Druin et al.1997] argued that children ages 7 to 10 years old make the most effective design partners. This is mainly because they are mature enough to discuss what they are thinking yet young enough not to be too heavily burdened with preconceived notions of the way things ``are supposed to be''. In our case, the students were 12 to 14 years of age; we found them to be easy to work with, partly due to the existing sense of community among them.
In summary, our recommendations for future work with middle school students in guided collaboration projects are:
Participatory design techniques support users' direct involvement in the design and formative evaluation of software prototypes. The potential for improving the resulting designs has been shown by others to be significant. In the case of educational technology, this means involving relatively young students and poses numerous challenges.
We experienced moderate success in a guided collaboration effort at a private middle school around a component-based technology for constructing mathematics resources; we were able to unambiguously answer questions about the viability of the technological approach (these results are discussed in another article). And we were able to support an enjoyable and beneficial collaboration between university researchers and middle school students.
Further, we derived considerable direct experience with the participatory design process in the middle school context. As a consequence, we have improved on our original methodology and anticipate applying it again in future projects. We believe that this kind of design process offers considerable value to educational software designers, especially in the very early stages of conception and development.
We will derive the most from students when we trust them to fill authentic research roles; this means inviting the students into the very early stages of design. Once the groundwork has been laid, the students will be more engaged during the more detailed design work which follows.
We would also like to thank principal Ted Spear of the Island Pacific School, all the students who participated, and the very supportive community of Bowen Island.
We would also like to thank BC Tel Advanced Communications for donation of an ISDN line to Bowen Island, Zentra Computers for donations of PC hardware, and Innovative Computing Solutions and Redesign for their networking support.
This work has been in part supported by research and equipment grants from the TeleLearning - National Centre of Excellence (TL-NCE) and the Pacific Institute for the Mathematical Sciences.
This document was generated using the LaTeX2HTML translator Version 98.1p1 release (March 2nd, 1998)
Copyright © 1993, 1994, 1995, 1996, 1997, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
The command line arguments were:
latex2html -split 0 -dir Involving/FULL paper.tex.
The translation was initiated by Loki Jorgenson on 1999-06-14
