Problem-Solving

---

Evaluation

• Introduction
  
  
• Results
  
  
• An Intensive Case Study Evaluation
  • Details
  • Results
    
    
• The National Problem-Solving Project: Summary and Conclusions

Results

The Attitude Questionnaire

On the Attitude Questionnaire, a number of the components showed very little significant change (a not uncommon finding, since pupils have no benchmark against which to judge their pre-test responses). Those that did change mainly related to improved self-esteem and relationships in class. There was a 10% rise in answering Yes to item 1: "I enjoy maths lessons", although the overall positive response to that item remained the same. Item 4 showed a significant shift away from "Keeping ideas to myself". No-one thought maths games were boring (item 5) and everyone tried their hardest at maths by post-test (item 6). At pre-test, 80% thought that maths was hard all or some of the time, but post-project that figure had gone down to 47%. There was a small rise in the percentage that "Can now understand maths words" (item 12). In item 13 "Other children don't like to work with me" there was a significant rise in the percentage of No responses, from 32% to 68%.

Group Discussions with Pupils

In general, participant pupils were enthusiastic about the project. Some children reported finding the instructions for games too long and therefore hard to understand on a first reading, but when they read them 2-3 times they became clearer. One participant commented that this helped him a lot in his homework, because he had to read over what it said two or three times, something he didn't mind doing now. Most participants thought that the shorter instructions were easier to follow, but they were able to ask for help if required. Some felt that they could now follow instructions more easily. On first reading, they used to always think that instructions were hard, but now they felt that they could "read them a lot better". When invited to share what they regarded as the positive aspects of the project, the majority of children reported that they found it enjoyable and interesting. Some reported that they made new friendships and enjoyed working together playing the games. Others commented that they would like to work with someone from a different class "to build up a friendship". This was because they have experience of monitoring younger classes but never get a chance to "bond with them". They also reported that it would be better for the younger ones, because it would make maths "easier for them by playing the games".

Participants appreciated learning new games and felt they could adapt them for future use. They reported that the gaming sessions were more successful if played twice a week for 30 minutes, rather than three 20-minute sessions. The 20-minute sessions were too short by the time "you got the game out and read the instructions". When asked about their favourite games box, the majority of participants chose the Strategy or Puzzle boxes. One pupil commented that this was "because you had to use your brain a lot". Another stated that it was good because "you had time to think of what moves you could make", and you had to watch your partner's moves too. Most participants felt that the Strategy games made them "think - think a lot!" When asked if it was hard to choose games, some reported that it was hard because there were so many good games.

Parent Feedback

The parents' reactions were very positive and suggested that they had found it very easy to take part in the project. The game instructions were generally not difficult to follow. The parents reported that the children were enthusiastic to play the games and reminded the parent if the parent forgot. Only occasionally did either parents or children become bored with the session. Many of the parents said that they would continue to play maths games with their children regularly and that they would probably recommend Paired Maths to other parents.

In discussion with the teacher/researcher the majority of parents reported that both they and the children had found that "maths can be fun", even interesting sometimes. Some noticed an increase in confidence in maths homework and mental calculations. One parent commented that the repetition of the game improved strategies and confidence. Time spent in the daily sessions was also valuable - "we could learn together and from each other". One parent felt their child had learned a deeper understanding of number in everyday life, although he was initially resistant to the project. Parents thought that more commercially produced games might be perceived by children as less school orientated, but in general they thought there was a good variety of games. All parents reported that it would be helpful to reduce the number of project days at home to 5 per week (i.e. exclude weekends), to retain enthusiasm and to fit in with daily routine. Other things happened at weekends and it was more inconvenient to fit in maths games.

Some parents felt that letters, rather than attendance at a meeting, could have conveyed more information about the project, because some parents could not attend the initial meeting. More flexibility with game exchange days might be useful, as some parents were unable to change on a particular day owing to unforeseen circumstances. Card games were the most popular game and the one most likely to be continued after the project. It was reported that a repeat of the project would be beneficial, but maybe for a shorter spell. All parents thought that it was a good exercise to encourage children to participate in practical games and discuss strategies. However some parents expressed some anxiety about their child refusing to play the game. One child apparently displayed a very negative attitude at times, although he did recognise the benefits to be gained. The project had motivated some parents to continue to find opportunities to do similar activities.

Teacher Perceptions and Observations

At the end of the project the teacher and the classroom assistant of the experimental class completed an evaluation questionnaire. In general, responses were very positive. Involving the pupils in the project was thought to have been enjoyable and interesting. Pupils had shown increased independence in their ability to organise themselves in the selection, playing and returning of the games. It was noted that there was enhancement in motivation and self-esteem in almost all of the participating pupils. One of the negative aspects of the project was the time required to gather the material and the organisation prior to the project itself. However, it was found that once these materials were established, it was easy to organise, and the children themselves were good at keeping the games in order.

By the end of the project the pupils were more effective at taking responsibility for their own learning. There was clearly a more positive attitude towards mathematics - the teacher noted that the children were quick to point out when it was Maths Games day. The children showed high engagement with the games. This promoted greater concentration and persistence as well as decision making and problem solving. There was evidence of enhanced team co-operation in learning and use of interpersonal skills such as discussion and turn taking. Before the project began, children identified themselves according to which primary year they came from, either 6 or 7. There appeared to be a lack of cohesion in the group. For the duration of the project the pairs were matched randomly and this resulted in some children working in mixed pairs, i.e. across both gender and primary year. The teacher reported enhancement of a feeling of bonding within the maths set, which was not there prior to the project. The children acknowledged that they had made friends within the set and that these friendships continued in the playground and other situations outwith the maths class.

The teacher also noted the choices the children made of games and where to play them. Given the freedom of the room some children chose to play their games on the floor, while others chose the more conventional table and chairs. It was noted that in the latter stages of the project there were signs of enhanced self-confidence in the selection of games and where to play them. The teacher also observed a general enhancement in mathematical confidence, although this did not come across in the children's responses to the attitude questionnaire. The children became more willing to tackle problems and puzzles in formal maths lessons. The teacher also noted that there was evidence of peer collaboration rather than peer tutoring. It was felt that more training could have and perhaps should have been give to the pupils with regard to the notional tutoring aspect of the project. However some participants spontaneously expressed a wish to "tutor" younger children in the games which they were now familiar.

Meeting with parents on an informal basis was another gain from the project. It appeared that parents became more comfortable about coming into the classroom and discussing the next game with the teacher. It was felt that parents and teacher were working together. Most parents were motivated to continue the project when they could see improvement in their child's achievement. The project also highlighted for the teacher the different learning styles of the pupils. Many pupils had benefited from working in pairs or small groups and the teacher encouraged this to continue in traditional maths lessons.

Mathematical Tests

The focus of this project was problem solving. It was felt that traditional standardised maths tests would not be sensitive to the project focus or to short-term gains. However it was decided to use NFER-Nelson 9/10 tests at the end of the project, as the school normally used these tests. The intervention and comparison groups took NFER-Nelson 9/10 mathematics tests in June 2001 and at the end of the project in March 2002. It was felt that there was an issue with comparability between Tests 9 and 10. Therefore it was decided to use the same test as before for each pupil rather than going on to the next test. Cockcroft (1982) emphasised that tests "should enable candidates to demonstrate what they know and should not undermine the confidence of those who attempt them". As one of the main aims of this project was to enhance the self-esteem and self-confidence of the participants, the teacher felt that they should not be put into a situation which threatened feelings of failure. Therefore the pupils took the same level of test as they had done the year before. The comparison group had equivalent experience in every respect apart from the involvement in the project, each pupil also taking the same test as they had previously.

There were forty-five items in the maths test and the highest possible score was 45. The chronological age of some pupils at post-test did not allow the generation of standardised scores - they had exceeded the age ceiling for the norms of the test. Consequently, raw scores were compared for all pupils. As these were uniformly available for both experimental and comparison groups (albeit from different pre-test baselines), this was felt to offer the best compromise. The raw scores of both the intervention group and the comparison group increased from pre-test to post-test, as would be expected from natural maturation. However, the percentage improvement for the experimental group was substantially larger than that for the comparison group on both Test 9 and Test 10 (see Figure 1 below).

At pre-test the mean scores of the comparison group were of course higher than those for the intervention group. At post-test the difference between experimental and comparison mean scores had narrowed significantly.

Discussion

The six-week intervention might have been considered too short for any significant enhancement of the pupils' self-confidence in maths to be likely to be observable or measurable. The majority of the pupils thought that the games were fun and interesting and wished for the games to continue as before. Therefore the project could have had an eight or even ten-week intervention, depending on the duration of the school term, and the results might have been even better. The parents noted this short duration and said they would be happy to take part in this project twice in a school year. However they felt that it would be more convenient to do the game sessions at home for only fives times per week, Monday to Friday.

It was felt that the inclusion of some commercially produced three dimensional games would have enhanced the project. According to pupil evaluation, the best games were considered to be the puzzle and strategy games (reproduced from Topping & Bamford, 1998a). However according to parent evaluation the best games were the number games. Parents felt the reinforcement of mental maths skills was the most important thing that their child learned from the project. The majority of the rules required a good standard of reading to interpret them, even though the researcher had simplified them. However it was felt that the children benefited from reading and re-reading the rules. It improved their reading, comprehension and problem-solving skills. Reading rules was somewhat problematic for the poorer readers, but a classroom assistant was always on hand to help with instructions. Some children reported that they now read problems in maths two or three times to understand it better.

The parent's games could have been managed better. It was more difficult to keep track of these games. Not all parents changed games on the same day. Sometimes the teacher was engaged in conversation with one parent while the others chose their games and left. Some parents were unable to come to school during school hours and relied on their child to choose a game.

The attitude questionnaire pre-post scores were quite similar. This might be taken to suggest a high degree of test-retest reliability, although it conflicted with the teacher's and parents' naturalistic observations of change. However, at post-test the questionnaire did indicate children were significantly more willing to share their ideas with a partner.

The mathematics test was a written rather than oral test, and did not align with the project activities in various other ways. It gave an assessment of overall mathematical attainment, but did include some questions on the topics of Number, Measure, Shape & Space and Handling Information. Two participants showed the same result in both assessments and two participants regressed. But others showed a significantly higher score, possibly mainly owing to increased self-confidence in mathematics.

Conclusion

The value of the whole project in terms of raising self-confidence and self-esteem as mathematicians was obvious to the teacher/researcher during the games sessions. The participants clearly enjoyed taking part in the games and parents were keen to be involved a second time. Some gains were observed in motivation and enthusiasm for maths in and out of school. Parents commented that maths can be fun and they had enjoyed playing the games with their children. They also enjoyed having quality time with children, not always common in the upper primary ages. Some pupils continued to play some of the games at home after the project officially ended. Levels of concentration on the games increased as the project progressed. Reading and interpretation skills were enhanced as the children examined the rules. Self-esteem was raised as participants found their own answers to some of the puzzle games and achieved success in the card solitaires. They were highly motivated because they were actively participating in the games. The games consolidated class work and encouraged discussion. They were fun for both child and parents too.

Onward Action

The incorporation and embedding of Problem Solving into the annual cycle of parental involvement is now the aim of the teacher/researcher. She found that this project combined two important elements in her teaching: the enhancement of self-esteem and self-confidence through co-operative and active learning situations. In previous years she had been responsible for developing the Personal and Social Education programme in the school, setting up a whole school policy and programme of work using Circle Time. The development of the Problem Solving project has provided her with the enthusiasm and motivation to change the way maths is tackled in the school. Other teachers in the school are keen to use the Problem Solving project games, and in the future staff collaboration will match pupil collaboration, with the aim of developing parental involvement. The support from the parents was arguably the strongest evaluative indicator for such a programme to continue. After being involved in the project, the teacher/researcher's own practices have been more clearly defined. There is clear value in providing children with active and co-operative learning situations. She now ensures that she makes time for such activities.

Many Scottish Schools, including the project school, have traditionally devoted a high number of hours to mathematics, using mainly worksheets and textbooks, but have arguable failed to use homework effectively, while showing a tendency to blame poor results on low pupil ability. This project has encouraged the teacher involved, because she has convincingly demonstrated that teachers can have some effect on children's attitude towards themselves as mathematicians. The pupils' enthusiasm and motivation can be raised through an effective problem solving project, but probably not through constant worksheets and textbooks. Parents appear to be a powerful resource, not merely to reinforce learning, but also to enhance it by creating situations where learning maths can be fun for both parent and child. The teacher/researcher is building bridges between theory and practice, and taking effective action forward.

Recommendations for Future Research

It would be interesting to investigate gender variation in different approaches to learning mathematics. Do boys prosper in a more active approach to problem solving and learning? It would also be interesting to investigate the effect of peer collaboration or tutoring in same gender pairings, with other age groups, and with greater age differentials. Other ways of measuring gains in attitudes, self-esteem and self-confidence might also be explored.

Recommendations for Future Practice

In the future the practitioners may want to trial a longer project. They should consider including mathematical games and puzzles purchased from stores and educational catalogues, so that more children do not perceive the games as homework. Practitioners should not under-estimate the time taken to familiarise themselves with the games and the simplification of any rules. They should have clear objectives regarding the mathematical language and strategies to be used.

Practitioners might also benefit from allocating more time than in this project to the initial training of peers. This should include practice in tutoring skills. Time should be given throughout the project to revise these principles. There is also a case for giving the tutors time prior to the project to become familiar with the games, rather than encounter them for the first time with the tutees. It was felt that because of a lack of effective tutor training, the project became peer collaboration rather than peer tutoring. However members of the intervention group spontaneously expressed a wish to be involved with another class, showing them the games. There might be a follow-up project involving cross-age peer tutoring. In this case more emphasis would be put on training and revision of skills for the tutors.

Practitioners may also want to revise the allocation of time given to parents' game sessions at home. Many parents felt that it was difficult to continue the game session at the weekend because of different routines. Timing of the launch meeting should also take into consideration those parents who are working. The researcher found that some parents were interested but unable to attend that meeting. They still took part in the project but they would have liked more written information about it in the form of a letter.

(Thanks to Terry Kerr, Maryburgh Primary School, for this intensive evaluation report)