complete report

The focus of this project is the Higher and Int 2 Information Systems class at Braidhurst High School. As the school is relatively small (610 pupils) the class is always split level and as the only teacher qualified to teach the subject I take the class every year.

As my previous school Dalbeattie was also very small (only 350 pupils) I’ve always been the lead teacher for Information Systems and Computing. This has been both a blessing and a curse, but after six years of having sole responsibility for delivering the Higher I am in a strong position with a sound understanding of what is required for pupils to succeed.

Since I started at Braidhurst there have been some significant changes in the numbers and make up of the classes as detailed in the table below.

Year	No. of Higher Pupils	No. of Int2 Pupils
2008-2009	2 (2 boys)	4 (4 boys)
2009-2010	3 (2 boys)	8 (4 boys)
2010-2011	15 (10 boys)	4 (3 boys)
2011-2012	16 (14 boys)	1 (1 boy)

As can be seen there has been a huge growth in the number of higher pupils and I have had to alter the way I teach the course to accommodate this. The subject is now much more important at a whole school level with only English producing more Higher passes.

This project focuses on the Using Information Unit, which I have always taught first between August and November. There are a number of reasons for this. Firstly, it is the part of the prelim paper that pupils perform most poorly at and requires the retention of a lot of facts, as opposed to the other two units Relational Databases and The Internet which focus more on technique type material.

Secondly, I have been of the view that in previous years pupils lacked of exam focus even though all the material had been covered. The very tight timescales meant that I have had precious little time, usually two weeks, immediately before exam leave, to dedicate to past paper question practice.  Last year, I became increasingly frustrated with the pupils’ lack of ability to decipher what was being looked for in exam questions.

Thirdly, this will form a legacy where I can use what I have learned from this project to enhance and improve the way I deliver this and other courses in the future. Finally, since the start of my teaching career I have been very interested in formative assessment and how it can improve pupil motivation and attainment. Therefore, I have been a member of the formative assessment committee in the past and for the last two years was a member of the schools TLC. I now lead the new TLC which started in August 2011.

Aims and Objectives

To improve overall exam performance and pupil motivation by altering the way the Using Information unit of Higher Information Systems is taught by focusing on a planned sequence of peer assessment tools coupled with structured revision skills.

Context

Last year I planned my classes on a Sunday for the following week and would at that point decide whether to incorporate an AiFL tool like jigsawing or peer teaching. As can be seen from the lesson plans in Appendix 2 I taught using a more traditional approach but with some AiFL activities thrown in. I wanted to see how this would compare with a sequence of lessons which were fully planned and based almost entirely on peer assessment.

Justification of the topic and Analysis and justification of methods used

Of all the material I have read since becoming a teacher the theories of Dylan Wiliam have had the biggest impact on me. It is Wiliam who stated that building in formative assessment tools can make the biggest single difference to the performance of a teacher.

Wiliam also believes that teachers would be more successful if they focused on one of the tools and became expert at that. Therefore, my focus is peer assessment as I have the most experience with this and it is a tool that would work with this particular pupil group.

Structure of report

The report is structured with an initial focus on the Methodology, how I planned to implement the project followed by the Development process, how the project panned out including any changes and improvements made.

I will then discuss the findings based on both empirical and quantitative evidence before reaching a conclusion on the basis of everything I have learned.

Literature Review

Introduction

Having studied formative assessment at University while training to be a teacher and ­been involved in it since starting my teaching career I am keen to find out just how much fully integrating it into a course can improve pupil performance. This has lead me to reread many of the key documents on AiFL in addition to look into the field of action research as only by applying some scientific rigour will I be able to quantify the outcomes. 

Formative Assessment

Dylan Wiliam is widely recognised as the principal authority on AiFL in the UK. He defines is as

“An assessment functions formatively to the extent that evidence about student achievement elicited by the assessment is interpreted and used to make decisions about the next steps in instruction that are likely to be better or better founded, than the decisions that would have been taken in the absence of that evidence.”

web.me.com/dylanwiliam/Dylan_Wiliams.../NEEC%20breakout.ppt

From ten classrooms to ten thousand: heuristics for scaling up formative assessment

There are a number of conclusions to draw from this. Firstly, as a classroom teacher, it is my job to create an environment where pupils can obtain good quality feedback to improve their learning. Secondly, both the pupils and I must know what direction their learning is headed, with the use of learning outcomes, so that the feedback is as valuable as possible. Thirdly, I will ensure that there is ample opportunity for evidence to be collected so that I can make changes in the future.

In addition, I am conscious that the programme must be planned meticulously as it is easy to let the formative assessment elements of the lesson slip and merely concentrate on the content. Again, in the words of Dylan Wiliam

“The hard thing is to say you get more learning by getting the students to do more of the work. You can’t do anybody else’s learning for them. We believe that in our heads, but we don’t believe it in our hearts, because, when the pressure is on we revert to telling.” Assessment for Learning: why, what and how
With senior classes, in particular, as there are very tight timescales, I always fell into the trap and “reverted to telling.”

He continues

“ What we do know is that when you invest in teachers using formative assessment you get between two and three times the effect of class size reduction at about on-tenth the cost.” Assessment for Learning: why, what and how, Dylan Wiliam

He identified five key strategies to promote learning

“Questioning – Engineering effective class discussions, questions and learning tasks that elicit evidence of learning.

Feedback – Providing feedback that moves learners forward.

Sharing Learning Intentions – Clarifying and sharing learning intentions and criteria for success.

Self Assessment – Activating students as owners of their own learning.

Peer Assessment – Activating students as instructional resources for one another.”

Improving the learning of numeracy through formative assessment, Dylan Wiliam, National numeracy Conference, March 2009
In addition to Wiliam I also investigated what Learning Teaching Scotland had to say on the subject. The following quote is from their website.
 “Research has shown that learners learn best when ...

• they understand clearly what they are trying to learn, and what is expected of them.
• they are given feedback about the quality of their work and what they can do to make it better.
• they are given advice about how to go about making improvements.
• they are fully involved in deciding what needs to be done next, and who can give them help if they need it.”

http://www.ltscotland.org.uk/learningteachingandassessment/assessment/progressandachievement/howweassess/learningteachingandassessment/supportinglearning.asp

Pupils must be given clear learning objectives, regularly told how well they are doing and how to make improvements in addition to being an integral part of the decision making process.

My research also took in the work of The Assessment Review Group which was an excellent source of information on Formative Assessment. I found the Assessment for Learning 10 principles, research-based principles to guide classroom practice, detailed in Appendix 1, particularly useful when I was creating my plan for my Work Based Project as it provides practical advice on specifically how to implement AiFL tools.

A novel concept was raised by Mark Potts in his paper “How can I live out my democratic values in practice more fully by using formative assessment techniques to influence my own learning and the learning of others?”

www.actionresearch.net/writings/module/mpeeform.doc

Potts concludes that a teacher can improve pupil motivation by behaving in a democratic fashion empowering learners to drive their own progress.

This is a similar point to Dylan Wiliam’s about teachers can’t do the learning for the pupils. The learning process can no longer be about one dictator (the teacher) impressing their view on the pupils but a collective working for the benefit of everyone.  

My reading of current issues also highlighted the concern that although schools appear to have bought into the notion of Formative Assessment the cultural change to fully embrace it has not taken place. The document produced by the Assessment Reform Group sums it up below.

“Many schools give the impression of having implemented AIFL when in reality the change in pedagogy that it requires has not taken place.”

http://www.assessment-reform-group.org/publications.html

I now fully understand why Teaching and Learning Communities (TLC) were established.

Teaching and Learning Community

In the last two academic years I was a member of the TLC which led to me becoming leader of the current group. Therefore, I have either used or observed the use, of a wide variety of AIFL techniques. Having given it some thought the ones, and of course basing it on my review, I felt would be most appropriate for this course and pupil group are detailed below.

·         Carousels – students in a group assess each others work

·         Best Composite Answer – students work in a small group to build the best composite answer based on their individual work.

·         Check Peer work against pre-flight checklist – students compare each others work against a list of required components.

·         Students check peers’ work with marking schemes

·         Traffic lighting peers’ work.

However, in her book Formative Assessment in the Secondary Classroom Shirley Clarke makes the point that the use of traffic lights should be approached with caution as they tend to come at the end of a task which is often too late for the teacher to intervene.

Peer Assessment

I was surprised to read that Dylan Wiliam’s view was that teachers are better concentrating on the types of formative assessment tools they are already experienced in and attempting to hone their skills in those rather than trying to introduce new ones. Therefore, I have decided to focus on peer assessment because I feel this is the area where I can add most value to the pupils learning.

I researched the use of peer assessment and found one particularly useful example of the successful use of peer assessment from a project at Kirkwall Grammar school in Orkney where teachers worked with S5 and S6 pupils to involve pupils more in their own learning. The following paragraph is taken from the LTS website

“The project allowed pupils to participate more actively in the assessment of their own work and pupil motivation increased noticeably as a result. Staff found that they were able to give pupils feedback in a more focused and specific way because of the greater emphasis on learning intentions and success criteria.”

http://www.ltscotland.org.uk/learningteachingandassessment/sharingpractice/index.asp?bFilter=1&strSearchText=

The key points I take from this is that pupil motivation is increased if a scenario can be created where the feedback given is highly specific and the learning intentions are made clear from the outset.

E-Assessment

I intend to use technology for a number of reasons; firstly, as it is an Information Systems class there is a requirement upon and expectation of the pupils to be using technology as much as possible. Secondly, feedback can be instantaneous and easily viewed. Thirdly, pupils can easily refer to prior feedback both to help identify where they are in their learning.
Finally, and very importantly, it provides anonymity. The focus group I held with last year’s class indicated that when asked to assess their peers’ work pupils tended to be more generous to pupils they know well and even if the name is covered they all recognise each others hand writing.
An interesting case study is to be found at http://www.jisc.ac.uk/media/documents/projects/scopingfinalreport.pdf where groups of trainee teachers worked together to create a wiki based on a collective effort. It is explained in more detail below.
“A set of ideas was built up on the wiki by one of the students using a Tablet-pc linked to a data projector so that the developing list could be viewed by the whole group.”
I took this idea and developed it to create, not a wiki, but an interactive website at braidhurstis.co.cc where the pupils could answer questions then review each others work. I created it so that upon submission the answers automatically appear in the pupil answers section of the site where another pupil, selected at random, will mark their answers providing feedback. I created this site from scratch using HTML and PHP.
Action Research in the Classroom
Obviously, at the conclusion of the project, I must find out whether there has been a notable improvement in pupil performance and motivation. Therefore, I read a number of papers and documents on Action Research and have found Jean McNiff’s definition the most useful. She describes it as being
“identifying a problematic issue, imagining a possible solution, trying it out, evaluating it and changing practice in the light of the evaluation.”
http://jeanmcniff.com/userfiles/file/Publications/AR%20Booklet.doc
She goes on to make a further important point that the researcher is also the subject, therefore, it involves self reflection as professionals should be capable of learning and developing new concepts using their own initiative.
To quote McNiff. 

“We modify what we are doing in the light of what we have found, and continue working in this new way and monitor what we do reviewing and evaluating the modified action.”

As this is a significant piece of work, which will, hopefully, have a positive impact on my teaching in the future I must ensure what I am doing is “systematic and rigorous” or there will be little to be gained from the conclusions drawn.

Conclusion

All my research reflects my own personal view that formative assessment improves pupil learning but I must be able to prove that using empirical evidence. I am also keen to ensure that everything must be planned with a high level of detail but without losing the flexibility to be able to make alterations if required.

In addition, the pupils must have prior practice of peer assessment for the project to be as effective as possible to help ensure I have created an environment where pupils are free to take responsibility for their own learning.

Methodology/Development Process

Background

Due to recent increases in pupils choosing the subject I have evaluated the way I teach it to ensure I am maximising the pupils’ learning. With the group who sat the exam in last year (2011) I used a few formative assessment tools but it tended to be on an ad hoc basis. I also, as would be expected, focused on exam technique immediately prior to the commencement of exam leave but I often felt that it was “too little, too late” by this point. In addition, I “reverted to telling” too often which lead to the pupils being passive recipients of information rather than active learners.

This approach was met with some success with the relative STAC values a positive 0.43 and the progression value a positive 0.41. These are measures which assess how well pupils perform in my class against how they performed in their other subjects. A STAC value of 1 would signify an improvement of one band.  However, I still think there is room for improvement and thought the class could really benefit from a structured and pre-planned programme of lessons building in formative assessment combined with a specific and early focus on interpreting exam questions.

The actual formative assessment techniques I chose to use

1. Peer Marking of Exam Style Questions

These were created either by myself to reflect different levels of challenge or by the pupils themselves. In response to pupil feedback I ensured that the process was anonymous with each pupil being issued with a confidential unique reference number which they used whenever they were answering questions.

The pupils answered their questions using the website I created at braidhurstis.co.cc. There are three main benefits of using this technique. Firstly pupils are answering exam style questions, which became progressively more difficult, without referring to notes and in a limited timeframe so they are under pressure as they will be in the final exam.

Secondly, the pupils are given the responsibility for marking another individual’s work. Pupils tend to take this more seriously than if they marked their own work often agonising over the marking scheme for lengthy periods before making a decision.

Thirdly, the marker has to give some thought to how well the individual questions have been answered and provide feedback. This ensures that they study the answers and marking scheme and explain the gap that exists between the two. This really makes the pupils focus on the marking scheme. I see this as a double benefit.

2. Creating exam style questions for peers to answer 

The second technique I implemented was giving pairs of pupils responsibility for creating exam questions, pitched at increasing levels of difficulty, with an answer scheme. The best set of questions were selected and issued to the rest of the class in a future lesson as a mini assessment with the pupils who set the test acting as invigilators and markers.

The benefits of using this technique are as follows, firstly, as the pupils create the questions they have to be familiar with how the past paper questions look and are structured. Secondly, they will actively have to read the notes and slides to find both questions and answers. As this is a competitive task they will scour the information looking for the most challenging topics to cover.

Thirdly, as I selected the best set of questions to form the mini assessment, there is a motivation to the pupils to complete the work to a high standard and on time. Finally, the pupils who have the job of marking have to read a wide variety of answers and use a marking scheme and then provide feedback to the pupils.

3. The Hot Seat

The third technique is “The Hot Seat”, which is a combination of different existing AiFL techniques. I use a small computer program; The Hat which randomly selects pupils names as if from a hat. The first pupil nominated sits in the hot seat at the front; they are then asked a question by the pupil out the hat next. The pupil in the hot seat stays there until they get a question wrong. If so, then another pupil is selected at random and so it continues.

There are a number of benefits to this approach. Firstly, the pupils prepare questions with solutions in advance. This involves them reading through the course notes to find questions and answers. Secondly, nobody knows who will be selected next either to go onto the hot seat or when they will be required to ask a question. They will, therefore, not be able to switch off at any point.

Thirdly and very importantly, it allows me assess where there are misconceptions or misunderstandings. If it is a relatively minor point I will interrupt the hot seat and instigate a discussion on it. However, if it is more major I will deal with it in the next lesson by covering the material in a different way.

4. Exit Tickets

For the last 5-10 minutes the pupils are required to answer a small number of questions, only 5 marks worth or so, on their way out on the material covered that day. The pupils are not allowed to leave until they have answered the questions. This is an excellent way for me to assess how well the learning objectives of the lesson have been met.

What I actually did and how

I worked out a detailed plan beforehand focusing on three or four periods a week. The plan is attached in appendix 2 with the main points detailed below.

Week 1

The pupils made up their own lessons in groups of three and then delivered them to the rest of class on Management Information systems. I deliberately made the scope of the lessons very limited so that they would only last a maximum of five minutes. All the groups with the exception of one worked well and had their tasks completed within the expected timeframe. They also created a number of questions based on their topics. In the end I didn’t use them as they were too variable in quality. I realised more work was required to be done on this part of the project and the pupils would require more support.  

Verbal feedback afterwards suggested that certain pupils didn’t take it that seriously as it was their peers who were delivering the content. This is another misconception I dealt with later in the course.

To assess how much of the information had penetrated I carried out a peer assessment task where the pupils marked each others answers to a test created by me, at a basic C level. This was reasonably well done which gave me encouragement that the peer assessment activities would work successfully.

Week 2

This was the first time the pupils created their own questions and marking schemes and I chose the “best one” for the rest of the class to answer. At this point all the sets of questions were flawed but I went with one set.

Much to my surprise, the marking process took significantly longer than completing the answers. Obviously, this is helping to develop the pupils’ knowledge and understanding as they have to be actively engaged in marking rather than being passive receptors of information.

The actual results of the test were really mixed with a group of pupils excelling but certain others not giving their full effort and scoring very poorly. I took solace by recognising that the value of this type of task is not only in the sitting of the test but in the preparation by reading through notes to create exam questions, to which everyone contributed.

In the penultimate lesson of the week I introduced the topic we would be covering in the next two weeks, the five strategies. The pupils who had passed Int 2 the previous year, who make up roughly half the class, were already very familiar with these concepts so I paired each of them up with a pupil who was crashing the course to teach them. I tested how well this had worked by using an Exit ticket where all the pupils had to write down the 5 strategies and a phrase summarising what they had learned. Obviously, I was focused on the performance of the pupils who were new and generally they had remembered most of the key points.

The following day, I used an Entrance Ticket with the pupils having to write a very brief summary of what they had learned the day before. Disappointingly, there were significant variations in the quality of the responses.

Week 3

This week I asked the pupils in pairs to create a 12-15 mark test, with a marking scheme on all the material covered so far. The winning group sat out the test and then marked everyone else’s answers.

It was specified that the questions should be level C KU from section 1, the most straightforward part of the paper. In the previous lesson, I had briefed the pupils on the structure of the paper and what level C type questions looked like. The pupils doing the marking provided individual feedback to the pupils in written and verbal form. This was a breakthrough with all the pupils contributing a good set of questions and answers.

Week 4

This week I used my peer assessment website for the first time. For the first 20 minutes of the double period the pupils answer specified questions. They were then allocated the number of the pupils work to mark and completed a form detailing the marks and feedback for each question.

It was based on level B type questions from section 1 of the paper but with a straightforward section 2 question worth 20 marks in total. There were a couple of problems. The first one, which was fairly easy to amend, was with the website, where all fields had to be completed but if a pupil completed some of the fields then tried to submit they would lose their work.

The second problem was more profound and is at the core of the Project. Initially the level of feedback given was very poor. In retrospect this was to be expected as, even though these pupils are in S5 or S6, they have had very little training or experience in giving feedback to one another. The initial feedback featured responses like “good”, “not very good” and so on. I realised very quickly that the pupils would need to be coached in how to give valuable feedback.

Later in the lesson, the pupils created questions which they would use for the hot seat. When we started the hot seat I could immediately see that many of the pupils lacked any depth of knowledge. This was excellent feedback for me and I knew that I was going to have to revisit the topics covered.

Week 5

The Hot Seat the following week was much more successful with the pupils responding well to the hat which I used for the first time.  Following this the pupils worked in pairs to make up exam type questions based on the strategies.

They were to create questions worth 20 marks of mainly straightforward questions but with about 4-5 marks worth more of more challenging questions at the end.  My strategy was to continue using the peer exam style questions tasks but to gradually make what was required of the pupils increasingly complex. Pupils then sat the best test from the two options which meant that no pupil was sitting out altogether.

By this point I could see clear improvements in the type of questions being posed by the pupils and the range of material tested. Some were of such good quality they could have been written by a teacher.

As was the norm by now I wrapped up week by using the Hot Seat on Friday to cover all topics up until now. By this point this process was operating much more smoothly with my interventions becoming increasingly infrequent.

Week 6

This week the pupils sat two of the tests created last week, on different topics. The format was the same in the previous weeks with the pupils whose test had been used marking everyone else’s at the end. This worked well with the marks being generally positive. I then used the hat to revise the strategies as we still need to fine tune these. Again, by this point this approach was working well.

Week 7

The first task was for the pupils to create their own lessons on separate topics in pairs consisting of someone who has the Int2 already working with a pupil who is crashing the higher.

This task went reasonably well but with the usual problems of the pairs not working fully in tandem with one pupil doing more of the work. I created a blank PowerPoint using the ideas from the pupils to make the main points on the board. This worked well and we would only move onto the next point once the class had reached agreement.

On the Wednesday the pupils made up past paper style questions worth 14-15 marks with solutions based on social and economic implications pitched at a B level. I tightened the timescales with everyone completing their work in a single period.

I was being observed during this lesson as part of the TLC by David Gardiner, PT of PE, who was very positive and now plans to use this technique in the future. I am now pleased with the way these lessons are working as the pupils are completing the tasks within the tight timescales.

Week 8

On the Monday, I used the Hot Seat to cover legal implications having asked the pupils to prepare in advance. The hot seat is now much more sleek with the pupils all knowing their required roles. Pupils have increased autonomy and will decide as group whether to award the mark.

On the Wednesday, the pupils used the website to answer past paper questions 2009 then use the marking scheme to provide feedback to each other. Pupils all worked well on this task and took, on average, just under the allocated 45 minutes. It was all done anonymously although, frustratingly, the pupils continue to ask each others their numbers.

The marking sequence during the second period went well with the pupils all being focused and working within the timescale. However, the level of feedback offered by the pupils continues to be very patchy with some comments being “The rest of the Stuff” or “The whole unit in general”, “Well done, you’re a good guy” or much more effective ones like

“study software strategy” or “ identities on the internet, software and network strategies.” One solution is to alter the layout of the questions to be grouped by topic so that the pupils would receive a comment for each one.

On the Thursday, as the unit is almost complete, I asked the pupils to read through a slideshow which covers the entire unit while making a note of any topic they weren’t sure of so that I could go over them again. Information Management systems, the strategies and the implications were the most popular so that gives me useful feedback for revision purposes.

Week 9

This week I used the Hat in a different way with the selected pupil explaining a point they picked out from the list created the previous week. Once a correct answer was given the pupil would select a different topic for the next pupil out of the Hat to explain. The pupils displaying sound subject knowledge and I only had to intervene a couple of times as, in general, the pupils knew the subject matter.

I used the rest of the week to do the now commonplace, peer assessment task. The quality of the answers was by now of a far higher standard.

Week 10

On the Monday, the pupils again used the website to answer past paper questions. This was a successful task with the quality of the answers being much better and most of the grades are either As or Bs.

After the NAB on Wednesday I asked the pupils to create questions for the hot seat which I was using to help revise for the upcoming end of unit test. The questions were to be challenging enough to ensure even the most difficult parts of the course were being covered. Unfortunately, some of the questions asked were way too challenging but when expert systems came up there was a consensus in the room that they would like to go over that again, therefore, I went over that on for the Friday lesson. This was really satisfying as this is the reason I’m using the hot seat, to pick out areas of misunderstanding.

Other than the problem with exceptionally difficult questions the hot seat was a real success with the pupils all now knowing how it works and what their role should be.

Week 11

As the Monday class was the last one before the end of unit test I used it to go over the homework and revise some final points in preparation. Again I used peer assessment to mark the homework. We are now at a point where the pupils fully understand what is required of them and will approach these tasks with relish using the form to give topic specific feedback.

I used the rest of the class to ask individual pupils to talk to the class about some of the areas they had wanted to go over. Again, this worked remarkably well, with the pupils having the confidence to stand in front of the class and explain their particular topics.

Findings

Gathering, interpreting and presenting the data

Assessing the success of the project is obviously very important which is why I have used every method possible including comparisons of end of unit tests and NABs from the previous year, pupil focus groups and questionnaires which are detailed below.

Justification of means to assess the success of the project.

1.      Empirical method by comparing the pupils results in from this year in the end of unit test to the pupils who sat the test this year. The differences in the abilities of the pupil groups have been taken into account.

2.      Focus groups where pupil volunteers can speak candidly and at length on the ways they have learned.

3.      Questionnaires where all the pupils can use an anonymous format to answer questions relating to how they have been taught and their own preferences for learning.

4.      Peer Assessment, as the honest views of respected and experienced colleagues are invaluable.  

These are explained in more detail below

End of Unit Test

The main empirical method I used to analyse the results was to compare how this group of pupils performed against the class from the previous year in the end of unit test. The test was identical to ensure that the results were comparable.

Analysis 1

To help make the comparison as worthwhile as possible I studied how the pupils in both year groups had performed in their standard grades in S4. The group who sat the higher in the last academic year (2010-2011) didn’t perform as well in averaging a 2.18 grade across all their exams compared with 1.82 for the group sitting the higher this year (2011-2012),  a difference of 16.5%.

The average score in the Using Information end of unit test for last years group was 52.92%. If I increase this by 16.5% to take account of the difference in the pupil group then I decided on the following to measure success.

Average mark in end of unit test 2010-2011 (100)	Average mark in end of unit test 2011-2012 (100)	Difference as a percentage (%) using 53.92 as the base	Success / Failure
53.92	<62	<16.5	Failure
53.92	>=62 and <65	>=16.5 and <21.5	Neither
53.92	>=65 and <69	>=21.5 and <29	Success
53.92	>=69	>=29	Outstanding success

Analysis Method 2

1. As mentioned earlier the class of 2010-11 averaged 2.18 in their standard grades while the class of 2011-12 averaged 1.82. A difference of 0.36.

2. To gain a 2 in a credit paper the student must score 50%, a 1 requires 66.66% Therefore, we can work on the basis that there is a difference of at least 16.66% between the average 2 pass and the average 1 pass.

3. If we multiply 0.36 by 16.66 then we will have the amount of marks difference in the end of unit test out of 100 between the two classes. This figure is 6.

The chart below shows the actual marks for the end of unit test against the predictions

Even if we choose the higher of the two predicted grades, 62.81% it is evident that the project has been a clear success with the improvement being 7.20% greater than expected.

Had I operated similar teaching approach with my S5/6 class of last year and each pupil had scored 7.20% higher in the final exam then 8 out of 15 of them would have improved their grade either than from a C to a B or from a B to an A. This is a profound difference and I am genuinely shocked, pleasantly so, that this project has had such an impact.          

Using Information NABs

Using a similar analysis to the one above for the NABs produced the following results. The average score of those in last years class was 16. Again using the 16.5% difference to take account of the different pupil group I was looking for the following

Average mark in NAB 2010-2011 (20)	Average mark in NAB 2011-2012  (20)	Difference as a percentage %	Success / Failure
16.07	18.7	<16.5%	Failure
16.07	>=18.7 and <19.5	>=16.5 and <21.5	Neither
16.07	>=19.5	>=21.5	Success

The bar is far higher here as the pupils in last years higher class performed much better in the NAB, averaging over 80% compared with the end of unit test where they averaged 53%.

There are a number of reasons for this;  the NABs are pitched at a C level and are, therefore, less demanding than the end of unit test which was based on past paper questions. In addition, the NAB is only 20 multiple choice questions while the test was out of 84 and required paragraph based exam style questions which is the skill that is required in the final exam.

For the 2011-12 class the average mark in the NAB was 18.4 which is such a marginal difference that it is almost negligible.  Without being able to read much into the NAB results I used the comparison of the end of unit tests as a far more accurate measure of the success of the project.

Focus Groups

I conducted focus groups with 6 pupils from each of the classes to gain feedback on which of the formative assessment tools were the most helpful. In the last academic year, I used a number of formative assessment tools in a less structured way and less frequently.

Pupils in the 2010-11 class

The focus group from last year highlighted the following points. They all thought “teacher talk” was the “Best way [to learn] because you can take in all the information” although another pupil did mention “If it is for too long it can get boring. Slides need to be interesting and 20 minutes is the limit.”

These comments struck me as I wasn’t expecting the pupils to have such a high opinion of teacher talk.

All of them found marking each others answers to past paper questions as beneficial with it being summed up by the comment. “Beneficial as then we know what the examiner is looking for” although they would be biased in favour of their friends with one comment summing up the view of the group “I’d be less harsh on my friends”.

They were less positive about some of the other tools including jigsawing; “Bad, as you only learn certain bits and it depends on the quality of your colleagues”, mark your own past papers “not as strict with yourself so it’s false” and team and individual quizzes “I don’t like it as much as it depends on the quality of your colleagues. I prefer it if the teacher asks the questions”

When I asked them about my proposals for the following year with internet based past paper questions they were reasonably positive saying “It would be good as you would not be biased.”

Pupils in the 2011-12 class

Before starting the focus group my expectation was that the pupils would say they liked some of the AiFL tools but in smaller doses and would prefer more teacher talk, in common with the views of last years class.

However, I was wrong. They all said that they much preferred learning using the AiFl tools rather than being “told”. Some of the quotes included

“It get’s boring sitting listening and taking notes.”

“You learn from your mistakes more [by using AiFL].”

“Taking notes you lose concentration and your mind wanders”

“A double period of writing is too much to take in.”

They also explained that it helped them understand how well they are doing and, interestingly, how their peers were doing, which they feel helps keep them motivated.

I explored the strategy of teachers giving the pupils notes in their jotters for them to refer to later, something I used to do. They said that it didn’t help as they would never read the notes again and that they were too busy taking the notes during the class that they wouldn’t understand what they were learning.

They all agree that they looked forward to the Information Systems class much more as there is more variety. The issue of feedback came up repeatedly with the pupils saying in other subjects they were expected to

sit and listen and work without any feedback. Which lead to them being “cautious about asking for help.”

I then moved the discussion onto the individual tools used. There was less agreement in the group about the different techniques. A number preferred the hot seat explaining

“When it is fun you remember more about it.”

I was curious to find out whether they retained focus during the hot seat even when they were in the role of observers, but they all agreed that you had to retain concentration as

“you might be the next person to ask a question.”

They all also enjoyed the competitive element of it.

“makes you pay attention as you might be asked the same question.” and

“information sticks in your head more than normal.”

I then asked them about creating past paper style questions with answer schemes which turned out to be the most popular. Some of their views included

“more motivated as there is a prize for the best one.”

And they

“read through the notes with a purpose.” and

“you have to know the answers as you create the questions.”

All the pupils agreed that they tried to make their questions as difficult as possible and that they retain far more information after they had finished.

I was curious to know how they felt about a classmate marking their work as opposed to a teacher. Again this was something they were positive about.

“It makes you try harder as you don’t want to be embarrassed by a peer.”

Their positive feedback continued when I moved the conversation onto a different AiFL tool - teach other parts of the course to other pupils. Some of the quotes included

“It’s good as you have to understand it yourself.”

“When teaching it helps you know your own weaknesses and where to improve.”

It makes you think like a teacher by knowing you have helped someone get it right.”

So, again, we are seeing the pupils feeling they are receiving more effective feedback and they are more motivated.

I then explored how they viewed Information Systems compared to their other subjects. Most of the group look forward to it more as they are more motivated and think they will do better in this subject. For example

“I’ve not done this before and it is one of my best subjects.”

“This is my favourite class.”

and

“I learn much better when I’m active and doing activities.”

I then explored how I should teach the next two units; they said they would be horrified if I reverted to telling and want the course to continue to be taught in the same way. They went on to say that they don’t get feedback in other classes so they don’t know how well they are doing. One of the S5 pupils is even going to ask if his other teachers could use AiFL techniques.

Originally, I thought the pupils were enjoying the subject but their feedback had totally exceeded my expectations. In terms of meeting the objective of improving pupil motivation I don’t think there is any doubt that this has been achieved.

Questionnaires

I also used web based questionnaires in conjunction with the focus groups to obtain a complete picture of the pupils views. The benefit of using the questionnaire is that it took in the viewpoints of entire class. However, the responses are much more limited and not so in depth as the focus group.

Pupils in the 2010-11 class

At the start of the course 33% thought they would get an A, another 33% thought they would get a B with16% thinking they would get a C and another 16% felt they would get a D.

By the time a second survey was conducted in April 2011 these figures had altered with the pupils feeling much more positive about the course. 58% thinking they would get an A, 33% a B and 8% a fail.

As is turned out the results were as follows 27% gained an A, 27% a B, 40% a C and 6% a fail. So the pupils’ opinions at the start of the course of where they would finish were fairly accurate. A self fulfilling prophecy perhaps?

By April the pupils were slightly overly optimistic about how they would perform but at Braidhurst I’ve never seen this as a problem as in general the pupil group tend to be very dismissive of their own abilities and lacking in self belief.

This group also rated direct teaching with the teacher explaining using slides as the most effective way to learn. A highly significant 83% thought this was an either Excellent or Quite Good way to learn. Some comments included

“I find it easy to listen and take in information from a PowerPoint as it allows me to read the information and take it in at my own pace” and “prefer this method to the others”. 

However, only 43% were positive about jigsawing; pupils teaching each other different parts of the course. Typical comments included “This is not a good option as we may learn incorrect information and/or end up in an [irrelevant] conversation.” And “I personally don’t find group work as helpful as plain old past papers and direct teaching. Nevertheless, it’s good to have a variety of teaching methods.”

The other techniques covered by the questionnaire; Answer past paper questions then mark it myself, answer past paper questions then another pupil marks it and create past paper type questions with an answer scheme were met with a fairly mixed response with the cohort being split roughly 50:50 on these techniques.

Pupils in the 2011-12 class

Survey 1 – August 2011 start of the unit

I conducted one brief survey at the very start of the course in August which I asked the pupils how they felt they would get on and which teaching techniques had worked for them in the past.

I was encouraged to find out that 71% felt they would get an A and 29% would get a B. Nobody felt they would do any worse than a B, so this group couldn’t be accused of negative thinking.

Again in terms of teaching techniques they had found helpful in the past a massive 93% thought that Direct Teaching was either Excellent or Quite Good. Only 20% were positive about jigsawing where the pupils taught each other different parts of the course.

Interestingly 57% preferred tasks where they are working on their own and 43% preferred to work in groups of 3 or 4. Nobody liked working pairs according to this.

Survey 2 – November 2011 end of the unit

At the end of the project I also conducted a similar survey on the same group asking their view on the techniques now that they had actually been taught using them. This made for some very interesting reading with the general mood being very positive for example 83% now believe they will obtain an A or B in the final exam which is very encouraging as it shows the pupils are feeling very positive about the subject.

I was keen to find out, as this is the most intense AiFL sequence of lessons I have ever delivered, their overall opinion. I was pleasantly surprised by some of the comments which included “I much preferred the teaching method of this course as it actively includes you and you aren’t required just to sit and listen, you have to take part” and “I quite enjoy the way I’m getting taught in this course. It is also better as I’m doing much better compared to my other subjects.”

This comment encouraged me even more, “I have enjoyed this much more as you can co-operate with others and also have time to work individually, its better to work like this as it doesn’t make is as boring and you are always doing something rather than sitting in one seat writing and taking notes. It gets people more involved and it helps me to learn much more.” I couldn’t have put it better myself.

Looking into the detail of the strategies used the pupils were very positive about all of them but especially create past paper style questions with an answer scheme with 92% of the class rating it as either excellent or quite good. Some of the comments included

“It’s good as it helps me get to know past paper questions and also find the answers to them, which will remain in my head.”

And

“This is another good technique as the questions we make up are challenging and the answer scheme helps me to understand how to answer the question.”

The next most popular AiFl tool according to the pupils was the Hot Seat with 75% viewing it positively. Some of the positive thoughts included.

“This is good as the questions are hard and it gives you an insight to how you would cope with these types of question while under pressure.” I was pleased to note that a couple of the pupils observed that the Hot Seat is not just about the time a pupil is in the hot seat but about being alert while a peer is being questioned. For example “We find out the mistakes others make and what we get wrong we can work on.” I was really pleased that this point was made as it accurately identifies one of the main points of the hot seat, to help identify areas of weakness allowing pupils to work on them.

However 75% also said that they thought direct teaching where the teacher

explains using slides was a good way to learn. However the comments didn’t back this up with pupils saying

“It’s good for the first ten minutes then I find myself falling asleep”, “It’s okay for a short while but in long doses concentration is lost.” Although there were some positive thoughts for example “using slides is good as there is no unnecessary information and focuses on the main points of that area.”

Peer Feedback for Principal Teacher of PE

As part of the TLC I was observed twice by D Gardiner the PT of PE during the WBP. His comments were as follows

“I really enjoyed observing the class where they created their own past paper style questions, they set about the task with a clear purpose and having spoken to a few of them they were highly motivated and enjoying the subject.”

And

“This was the first time I have seen the hot seat in action and it really seemed to work with this class as there was a tension and healthy competitive spirit in the air. This is something that will work well in my own subject area, PE, and I look forward to feeding back what I have seen to my own department.”

What have I learned from planning, implementing and evaluating the project.

Lessons learned from Planning

As I planned this sequence of lessons well in advance prior to the start of the academic year this has allowed me a valuable amount of time to think about how to best implement the project. However, it has been important to retain some flexibility, as things don’t always go to plan and I have altered and changed my plan based on the feedback from pupils. Surprisingly, I have managed to get through this unit more quickly than usual.

Implementation

The implementation has gone smoothly with the pupils enthusiastically undertaking the tasks. Initially, one issue was the quality of feedback given in the peer assessment tasks. However, there has been a marked improvement in the quality of feedback given towards the end of the unit.

As can be imagined there are a number of things I would have do differently when I implement the same sequence of lessons next year.

1.      I will dedicate one of my lessons very early in the course to peer assessment and learning. Even though it is something which is an essential skill for the pupils to gain this group appear to have had precious little experience of it.

2.      I will ensure any technical hitches with the website are resolved.

Following the positive pupil feedback I will use the same AiFL tools with the remaining two units of the course.

Evaluating

In terms of evaluation I took two distinct approaches. Firstly, immediately after delivering the lessons I would evaluate how I felt the class had gone. I noted this in my lesson planning sheet attached in the appendix.

Secondly, over the course of the project I was always evaluating how it was going in general basing my view of the pupil knowledge compared to how much they needed to know for the exam.

The lessons I have learned in evaluating the project are the following

1.      I really benefitted from making evaluation notes after the lessons as my opinion afterwards was generally accurate. As a classroom teacher I am always reflecting on my practice but it was helpful to note my thoughts formally. I will use from now on.

2.      As discussed earlier I used a wide variety of tools to assess the success of the project. I have learned that none of these on its own will give a full picture of the pupils’ progress. I intend to use focus groups, which can be time consuming, but very valuable in terms of the quality of the information given again. There are also big advantages to using questionnaire to assess the views of the pupils as they are quick and easy to implement and the whole class can be included. 

Outcomes

How far did the project succeed in achieving its aims?

There were two main criteria I used to judge the success of the project, firstly improvement in results and performance and secondly improvement in pupil enjoyment and motivation.

The result and performance are by far the easiest to measure and having analysed the results in the same test across the two pupil groups there was at least  7.20% improvement in expected results. In my view this is a significant enough improvement to consider the project a success.

Secondly, did the project have a positive impact on pupil enjoyment and motivation? The results of the final questionnaire and focus group indicate clearly that the pupils really enjoyed learning this way and that they would be “horrified” if I reverted back to more traditional teaching methods.

Finally, it is my view that this group of pupils took the work seriously, were well motivated and ultimately the vast majority will be successful in gaining a very good grade in the Higher exam.

Conclusion

To sum up what I did, having taught Higher Information Systems for a number of years I have always been keen to improve attainment and pupil motivation so I made these my key success criteria.

I then planned a sequence of lessons to deliver the first unit Using Information building in a variety of AiFL tools which were used in almost every period. In addition, there was a focus on exam technique throughout this sequence.

Over an eleven week period I implemented my plan, taking time to evaluate each lesson. I used a website I created to allow anonymity for the peer assessment tasks.

Upon completion I used a range of tools including questionnaires, focus groups, test and NAB results to assess the success of the project. From this I came to the conclusion that, firstly, the use of AIFL and structured revision skills helps to improve pupil performance in tests by a significant margin which if replicated across the whole course could improve the final grade of over half the class.

Secondly, the use of AIFL tools significantly increases pupil motivation

allowing them to become more responsible for their own learning and the

learning of their peers. In addition, they liked the variety of the tasks and the

fact that they knew exactly where they were in their learning thanks to the

constant feedback they were receiving.

My next step will be to develop resources and lessons for the delivery of the next two units, Relational Databases and The Internet using AiFL. This approach is clearly working with this group of pupils and I think I’d have a mutiny on my hands if I ever dared to “revert to telling” with them.

I have kept my departmental and TLC colleagues updated throughout this project and there has been a lot of interest in what I have been trying with many colleagues promising to embed AiFL tools into their teaching.

In the long run, I will definitely use all the material and resources with my Higher class next year and work on introducing these techniques with my classes further down the school. With the imminent introduction of the Curriculum for Excellence courses I can make sure that any development work carried out in the department has AiFL tools embedded.  

This has been a very positive experience for me as I have thoroughly enjoyed teaching my Higher class this year and it is very satisfying to receive such positive feedback.

Recently one of the pupils said to me “Thanks for teaching the course this way” I replied “Don’t thank me, thank Dylan Wiliam.” He looked perplexed.