I
will then discuss the findings based on both empirical and quantitative
evidence before reaching a conclusion on the basis of everything I have
learned.
Literature Review
Introduction
Having
studied formative assessment at University while training to be a teacher and been
involved in it since starting my teaching career I am keen to find out just how
much fully integrating it into a course can improve pupil performance. This has
lead me to reread many of the key documents on AiFL in addition to look into
the field of action research as only by applying some scientific rigour will I
be able to quantify the outcomes.
Formative Assessment
Dylan
Wiliam is widely recognised as the principal authority on AiFL in the UK. He defines
is as
“An assessment functions
formatively to the extent that evidence about student achievement elicited by
the assessment is interpreted and used to make decisions about the next steps
in instruction that are likely to be better or better founded, than the
decisions that would have been taken in the absence of that evidence.”
web.me.com/dylanwiliam/Dylan_Wiliams.../NEEC%20breakout.ppt
From ten classrooms to ten thousand: heuristics for
scaling up formative assessment
There
are a number of conclusions to draw from this. Firstly, as a classroom teacher,
it is my job to create an environment where pupils can obtain good quality
feedback to improve their learning. Secondly, both the pupils and I must know
what direction their learning is headed, with the use of learning outcomes, so
that the feedback is as valuable as possible. Thirdly, I will ensure that there
is ample opportunity for evidence to be collected so that I can make changes in
the future.
In
addition, I am conscious that the programme must be planned meticulously as it
is easy to let the formative assessment elements of the lesson slip and merely
concentrate on the content. Again, in the words of Dylan Wiliam
“The hard thing
is to say you get more learning by getting the students to do more of the work.
You can’t do anybody else’s learning for them. We believe that in our heads,
but we don’t believe it in our hearts, because, when the pressure is on we
revert to telling.” Assessment for
Learning: why, what and how
With senior
classes, in particular, as there are very tight timescales, I always fell into
the trap and “reverted to telling.”
He
continues
“ What we do know is that when you invest in
teachers using formative assessment you get between two and three times the
effect of class size reduction at about on-tenth the cost.” Assessment for Learning: why, what and how,
Dylan Wiliam
He
identified five key strategies to promote learning
“Questioning – Engineering effective class
discussions, questions and learning tasks that elicit evidence of learning.
Feedback – Providing feedback that moves learners
forward.
Sharing Learning Intentions – Clarifying and sharing
learning intentions and criteria for success.
Self Assessment – Activating students as owners of
their own learning.
Peer
Assessment – Activating students as instructional resources for one another.”
Improving the learning of numeracy through formative
assessment, Dylan Wiliam, National numeracy Conference, March 2009
In addition to
Wiliam I also investigated what Learning Teaching Scotland had to say on the
subject. The following quote is from their website.
“Research has shown that learners learn best
when ...
- they understand clearly what they are trying to learn, and what is
expected of them.
- they are given feedback about the quality of their work and what
they can do to make it better.
- they are given advice about how to go about making improvements.
- they are fully involved in deciding what needs to be done next,
and who can give them help if they need it.”
http://www.ltscotland.org.uk/learningteachingandassessment/assessment/progressandachievement/howweassess/learningteachingandassessment/supportinglearning.asp
Pupils must be
given clear learning objectives, regularly told how well they are doing and how
to make improvements in addition to being an integral part of the decision
making process.
My
research also took in the work of The Assessment Review Group which was an
excellent source of information on Formative Assessment. I found the Assessment for Learning 10 principles,
research-based principles to guide classroom practice, detailed in Appendix
1, particularly useful when I was creating my plan for my Work Based Project as
it provides practical advice on specifically how to implement AiFL tools.
A
novel concept was raised by Mark Potts in his paper “How can I live out my
democratic values in practice more fully by using formative assessment
techniques to influence my own learning and the learning of others?”
www.actionresearch.net/writings/module/mpeeform.doc
Potts
concludes that a teacher can improve pupil motivation by behaving in a
democratic fashion empowering learners to drive their own progress.
This
is a similar point to Dylan Wiliam’s about teachers can’t do the learning for the
pupils. The learning process can no longer be about one dictator (the teacher)
impressing their view on the pupils but a collective working for the benefit of
everyone.
My reading of current
issues also highlighted the concern that although schools appear to have bought
into the notion of Formative Assessment the cultural change to fully embrace it
has not taken place. The document produced by the Assessment Reform Group sums
it up below.
“Many schools give the
impression of having implemented AIFL when in reality the change in pedagogy
that it requires has not taken place.”
http://www.assessment-reform-group.org/publications.html
I now fully understand why
Teaching and Learning Communities (TLC) were established.
Teaching and Learning Community
In
the last two academic years I was a member of the TLC which led to me becoming
leader of the current group. Therefore, I have either used or observed the use,
of a wide variety of AIFL techniques. Having given it some thought the ones,
and of course basing it on my review, I felt would be most appropriate for this
course and pupil group are detailed below.
·
Carousels
– students in a group assess each others work
·
Best Composite Answer – students work in a small group to build the best
composite answer based on their individual work.
·
Check Peer work against pre-flight checklist –
students compare each others work against a list of required components.
·
Students check peers’ work with marking schemes
·
Traffic lighting peers’ work.
However,
in her book Formative Assessment in the Secondary Classroom Shirley Clarke
makes the point that the use of traffic lights should be approached with
caution as they tend to come at the end of a task which is often too late for
the teacher to intervene.
Peer Assessment
I
was surprised to read that Dylan Wiliam’s view was that teachers are better
concentrating on the types of formative assessment tools they are already
experienced in and attempting to hone their skills in those rather than trying
to introduce new ones. Therefore, I have decided to focus on peer assessment
because I feel this is the area where I can add most value to the pupils
learning.
I
researched the use of peer assessment and found one particularly useful example
of the successful use of peer assessment from a project at Kirkwall Grammar
school in Orkney where teachers worked with S5 and S6 pupils to involve pupils
more in their own learning. The following paragraph is taken from the LTS website
“The
project allowed pupils to participate more actively in the assessment of their
own work and pupil motivation increased noticeably as a result. Staff found
that they were able to give pupils feedback in a more focused and specific way
because of the greater emphasis on learning intentions and success criteria.”
http://www.ltscotland.org.uk/learningteachingandassessment/sharingpractice/index.asp?bFilter=1&strSearchText=
The
key points I take from this is that pupil motivation is increased if a scenario
can be created where the feedback given is highly specific and the learning
intentions are made clear from the outset.
E-Assessment
I intend to use
technology for a number of reasons; firstly, as it is an Information Systems
class there is a requirement upon and expectation of the pupils to be using
technology as much as possible. Secondly, feedback can be instantaneous and
easily viewed. Thirdly, pupils can easily refer to prior feedback both to help
identify where they are in their learning.
Finally, and very
importantly, it provides anonymity. The focus group I held with last year’s class
indicated that when asked to assess their peers’ work pupils tended to be more
generous to pupils they know well and even if the name is covered they all
recognise each others hand writing.
An interesting case study is to be found at http://www.jisc.ac.uk/media/documents/projects/scopingfinalreport.pdf
where groups of trainee teachers worked together to create a wiki based on a
collective effort. It is explained in more detail below.
“A set of ideas
was built up on the wiki by one of the students using a Tablet-pc linked to a
data projector so that the developing list could be viewed by the whole group.”
I took this idea
and developed it to create, not a wiki, but an interactive website at
braidhurstis.co.cc where the pupils could answer questions then review each
others work. I created it so that upon submission the answers automatically
appear in the pupil answers section of the site where another pupil, selected
at random, will mark their answers providing feedback. I created this site from
scratch using HTML and PHP.
Action Research in the Classroom
Obviously, at the
conclusion of the project, I must find out whether there has been a notable
improvement in pupil performance and motivation. Therefore, I read a number of
papers and documents on Action Research and have found Jean McNiff’s definition
the most useful. She describes it as being
“identifying a
problematic issue, imagining a possible solution, trying it out, evaluating it and
changing practice in the light of the evaluation.”
http://jeanmcniff.com/userfiles/file/Publications/AR%20Booklet.doc
She goes on to
make a further important point that the researcher is also the subject, therefore,
it involves self reflection as professionals should be capable of learning and
developing new concepts using their own initiative.
To quote McNiff.
“We
modify what we are doing in the light of what we have found, and continue
working in this new way and monitor what we do reviewing and evaluating the
modified action.”
As
this is a significant piece of work, which will, hopefully, have a positive
impact on my teaching in the future I must ensure what I am doing is
“systematic and rigorous” or there will be little to be gained from the
conclusions drawn.
Conclusion
All
my research reflects my own personal view that formative assessment improves
pupil learning but I must be able to prove that using empirical evidence. I am
also keen to ensure that everything must be planned with a high level of detail
but without losing the flexibility to be able to make alterations if required.
In
addition, the pupils must have prior practice of peer assessment for the
project to be as effective as possible to help ensure I have created an
environment where pupils are free to take responsibility for their own
learning.
Methodology/Development
Process
Background
Due
to recent increases in pupils choosing the subject I have evaluated the way I
teach it to ensure I am maximising the pupils’ learning. With the group who sat
the exam in last year (2011) I used a few formative assessment tools but it
tended to be on an ad hoc basis. I also, as would be expected, focused on exam
technique immediately prior to the commencement of exam leave but I often felt
that it was “too little, too late” by this point. In addition, I “reverted to
telling” too often which lead to the pupils being passive recipients of
information rather than active learners.
This
approach was met with some success with the relative STAC values a positive
0.43 and the progression value a positive 0.41. These are measures which assess
how well pupils perform in my class against how they performed in their other
subjects. A STAC value of 1 would signify an improvement of one band. However, I still think there is room for
improvement and thought the class could really benefit from a structured and
pre-planned programme of lessons building in formative assessment combined with
a specific and early focus on interpreting exam questions.
The
actual formative assessment techniques I chose to use
1. Peer Marking of Exam Style Questions
These
were created either by myself to reflect different levels of challenge or by
the pupils themselves. In response to pupil feedback I ensured that the process
was anonymous with each pupil being issued with a confidential unique reference
number which they used whenever they were answering questions.
The
pupils answered their questions using the website I created at braidhurstis.co.cc.
There are three main benefits of using this technique. Firstly pupils are
answering exam style questions, which became progressively more difficult,
without referring to notes and in a limited timeframe so they are under
pressure as they will be in the final exam.
Secondly,
the pupils are given the responsibility for marking another individual’s work.
Pupils tend to take this more seriously than if they marked their own work
often agonising over the marking scheme for lengthy periods before making a
decision.
Thirdly,
the marker has to give some thought to how well the individual questions have
been answered and provide feedback. This ensures that they study the answers
and marking scheme and explain the gap that exists between the two. This really
makes the pupils focus on the marking scheme. I see this as a double benefit.
2. Creating exam style questions for
peers to answer
The
second technique I implemented was giving pairs of pupils responsibility for
creating exam questions, pitched at increasing levels of difficulty, with an
answer scheme. The best set of questions were selected and issued to the rest
of the class in a future lesson as a mini assessment with the pupils who set
the test acting as invigilators and markers.
The
benefits of using this technique are as follows, firstly, as the pupils create
the questions they have to be familiar with how the past paper questions look
and are structured. Secondly, they will actively have to read the notes and
slides to find both questions and answers. As this is a competitive task they
will scour the information looking for the most challenging topics to cover.
Thirdly,
as I selected the best set of questions to form the mini assessment, there is a
motivation to the pupils to complete the work to a high standard and on time. Finally,
the pupils who have the job of marking have to read a wide variety of answers
and use a marking scheme and then provide feedback to the pupils.
3. The Hot Seat
The
third technique is “The Hot Seat”, which is a combination of different existing
AiFL techniques. I use a small computer program; The Hat which randomly selects
pupils names as if from a hat. The first pupil nominated sits in the hot seat at the front; they are then
asked a question by the pupil out the hat next. The pupil in the hot seat stays
there until they get a question wrong. If so, then another pupil is selected at
random and so it continues.
There
are a number of benefits to this approach. Firstly, the pupils prepare
questions with solutions in advance. This involves them reading through the
course notes to find questions and answers. Secondly, nobody knows who will be
selected next either to go onto the hot seat or when they will be required to
ask a question. They will, therefore, not be able to switch off at any point.
Thirdly
and very importantly, it allows me assess where there are misconceptions or
misunderstandings. If it is a relatively minor point I will interrupt the hot
seat and instigate a discussion on it. However, if it is more major I will deal
with it in the next lesson by covering the material in a different way.
4. Exit Tickets
For
the last 5-10 minutes the pupils are required to answer a small number of
questions, only 5 marks worth or so, on their way out on the material covered
that day. The pupils are not allowed to leave until they have answered the
questions. This is an excellent way for me to assess how well the learning
objectives of the lesson have been met.
What I actually did and how
I
worked out a detailed plan beforehand focusing on three or four periods a week.
The plan is attached in appendix 2 with the main points detailed below.
Week 1
The
pupils made up their own lessons in groups of three and then delivered them to
the rest of class on Management Information systems. I deliberately made the
scope of the lessons very limited so that they would only last a maximum of
five minutes. All the groups with the exception of one worked well and had
their tasks completed within the expected timeframe. They also created a number
of questions based on their topics. In the end I didn’t use them as they were
too variable in quality. I realised more work was required to be done on this
part of the project and the pupils would require more support.
Verbal
feedback afterwards suggested that certain pupils didn’t take it that seriously
as it was their peers who were delivering the content. This is another
misconception I dealt with later in the course.
To
assess how much of the information had penetrated I carried out a peer
assessment task where the pupils marked each others answers to a test created
by me, at a basic C level. This was reasonably well done which gave me
encouragement that the peer assessment activities would work successfully.
Week 2
This
was the first time the pupils created their own questions and marking schemes
and I chose the “best one” for the rest of the class to answer. At this point
all the sets of questions were flawed but I went with one set.
Much
to my surprise, the marking process took significantly longer than completing
the answers. Obviously, this is helping to develop the pupils’ knowledge and
understanding as they have to be actively engaged in marking rather than being
passive receptors of information.
The
actual results of the test were really mixed with a group of pupils excelling
but certain others not giving their full effort and scoring very poorly. I took
solace by recognising that the value of this type of task is not only in the
sitting of the test but in the preparation by reading through notes to create
exam questions, to which everyone contributed.
In
the penultimate lesson of the week I introduced the topic we would be covering
in the next two weeks, the five strategies. The pupils who had passed Int 2 the
previous year, who make up roughly half the class, were already very familiar
with these concepts so I paired each of them up with a pupil who was crashing
the course to teach them. I tested how well this had worked by using an Exit
ticket where all the pupils had to write down the 5 strategies and a phrase
summarising what they had learned. Obviously, I was focused on the performance
of the pupils who were new and generally they had remembered most of the key
points.
The
following day, I used an Entrance Ticket with the pupils having to write a very
brief summary of what they had learned the day before. Disappointingly, there
were significant variations in the quality of the responses.
Week 3
This
week I asked the pupils in pairs to create a 12-15 mark test, with a marking
scheme on all the material covered so far. The winning group sat out the test
and then marked everyone else’s answers.
It
was specified that the questions should be level C KU from section 1, the most
straightforward part of the paper. In the previous lesson, I had briefed the
pupils on the structure of the paper and what level C type questions looked
like. The pupils doing the marking provided individual feedback to the pupils
in written and verbal form. This was a breakthrough with all the pupils contributing
a good set of questions and answers.
Week 4
This
week I used my peer assessment website for the first time. For the first 20
minutes of the double period the pupils answer specified questions. They were then
allocated the number of the pupils work to mark and completed a form detailing
the marks and feedback for each question.
It
was based on level B type questions from section 1 of the paper but with a
straightforward section 2 question worth 20 marks in total. There were a couple
of problems. The first one, which was fairly easy to amend, was with the
website, where all fields had to be completed but if a pupil completed some of
the fields then tried to submit they would lose their work.
The
second problem was more profound and is at the core of the Project. Initially
the level of feedback given was very poor. In retrospect this was to be
expected as, even though these pupils are in S5 or S6, they have had very
little training or experience in giving feedback to one another. The initial
feedback featured responses like “good”, “not very good” and so on. I realised very
quickly that the pupils would need to be coached in how to give valuable
feedback.
Later
in the lesson, the pupils created questions which they would use for the hot
seat. When we started the hot seat I could immediately see that many of the
pupils lacked any depth of knowledge. This was excellent feedback for me and I
knew that I was going to have to revisit the topics covered.
Week 5
The
Hot Seat the following week was much more successful with the pupils responding
well to the hat which I used for the first time. Following this the pupils worked
in pairs to make up exam type questions based on the strategies.
They
were to create questions worth 20 marks of mainly straightforward questions but
with about 4-5 marks worth more of more challenging questions at the end. My strategy was to continue using the peer
exam style questions tasks but to gradually make what was required of the
pupils increasingly complex. Pupils then sat the best test from the two options
which meant that no pupil was sitting out altogether.
By
this point I could see clear improvements in the type of questions being posed
by the pupils and the range of material tested. Some were of such good quality
they could have been written by a teacher.
As
was the norm by now I wrapped up week by using the Hot Seat on Friday to cover
all topics up until now. By this point this process was operating much more
smoothly with my interventions becoming increasingly infrequent.
Week 6
This
week the pupils sat two of the tests created last week, on different topics.
The format was the same in the previous weeks with the pupils whose test had
been used marking everyone else’s at the end. This worked well with the marks
being generally positive. I then used the hat to revise the strategies as we
still need to fine tune these. Again, by this point this approach was working
well.
Week 7
The
first task was for the pupils to create their own lessons on separate topics in
pairs consisting of someone who has the Int2 already working with a pupil who
is crashing the higher.
This
task went reasonably well but with the usual problems of the pairs not working
fully in tandem with one pupil doing more of the work. I created a blank PowerPoint
using the ideas from the pupils to make the main points on the board. This worked
well and we would only move onto the next point once the class had reached
agreement.
On
the Wednesday the pupils made up past
paper style questions worth 14-15 marks with solutions based on social and
economic implications pitched at a B level. I tightened the timescales with
everyone completing their work in a single period.
I
was being observed during this lesson as part of the TLC by David Gardiner, PT
of PE, who was very positive and now plans to use this technique in the future.
I am now pleased with the way these lessons are working as the pupils are
completing the tasks within the tight timescales.
Week 8
On
the Monday, I used the Hot Seat to cover legal implications having asked the
pupils to prepare in advance. The hot seat is now much more sleek with the
pupils all knowing their required roles. Pupils have increased autonomy and
will decide as group whether to award the mark.
On
the Wednesday, the pupils used the website to answer past paper questions 2009
then use the marking scheme to provide feedback to each other. Pupils all
worked well on this task and took, on average, just under the allocated 45
minutes. It was all done anonymously although, frustratingly, the pupils
continue to ask each others their numbers.
The
marking sequence during the second period went well with the pupils all being
focused and working within the timescale. However, the level of feedback
offered by the pupils continues to be very patchy with some comments being “The
rest of the Stuff” or “The whole unit in general”, “Well done, you’re a good
guy” or much more effective ones like
“study
software strategy” or “ identities on the internet, software and network
strategies.” One solution is to alter the layout of the questions to be grouped
by topic so that the pupils would receive a comment for each one.
On
the Thursday, as the unit is almost complete, I asked the pupils to read
through a slideshow which covers the entire unit while making a note of any
topic they weren’t sure of so that I could go over them again. Information
Management systems, the strategies and the implications were the most popular so
that gives me useful feedback for revision purposes.
Week 9
This week I used the Hat in a different way with
the selected pupil explaining a point they picked out from the list created the
previous week. Once a correct answer was given the pupil would select a
different topic for the next pupil out of the Hat to explain. The pupils
displaying sound subject knowledge and I only had to intervene a couple of
times as, in general, the pupils knew the subject matter.
I used the rest of the week to do the now commonplace,
peer assessment task. The quality of the answers was by now of a far higher
standard.
Week 10
On
the Monday, the pupils again used the website to answer past paper questions. This
was a successful task with the quality of the answers being much better and
most of the grades are either As or Bs.
After the NAB on Wednesday I asked the pupils to
create questions for the hot seat which I was using to help revise for the
upcoming end of unit test. The questions were to be challenging enough to
ensure even the most difficult parts of the course were being covered. Unfortunately,
some of the questions asked were way too challenging but when expert systems came
up there was a consensus in the room that they would like to go over that
again, therefore, I went over that on for the Friday lesson. This was really
satisfying as this is the reason I’m using the hot seat, to pick out areas of
misunderstanding.
Other than the problem with exceptionally difficult
questions the hot seat was a real success with the pupils all now knowing how
it works and what their role should be.
Week 11
As the Monday class was the last one before the end
of unit test I used it to go over the homework and revise some final points in
preparation. Again I used peer assessment to mark the homework. We are now at a
point where the pupils fully understand what is required of them and will
approach these tasks with relish using the form to give topic specific
feedback.
I used the rest of the class to ask individual pupils
to talk to the class about some of the areas they had wanted to go over. Again,
this worked remarkably well, with the pupils having the confidence to stand in
front of the class and explain their particular topics.
Findings
Gathering, interpreting and presenting
the data
Assessing
the success of the project is obviously very important which is why I have used
every method possible including comparisons of end of unit tests and NABs from
the previous year, pupil focus groups and questionnaires which are detailed
below.
Justification of means to assess the success
of the project.
1.
Empirical method
by comparing the pupils results in from this year in the end of unit test to
the pupils who sat the test this year. The differences in the abilities of the
pupil groups have been taken into account.
2.
Focus groups where
pupil volunteers can speak candidly and at length on the ways they have
learned.
3.
Questionnaires
where all the pupils can use an anonymous format to answer questions relating
to how they have been taught and their own preferences for learning.
4.
Peer Assessment,
as the honest views of respected and experienced colleagues are invaluable.
These
are explained in more detail below
End of Unit Test
The
main empirical method I used to analyse the results was to compare how this
group of pupils performed against the class from the previous year in the end
of unit test. The test was identical to ensure that the results were
comparable.
Analysis 1
To
help make the comparison as worthwhile as possible I studied how the pupils in
both year groups had performed in their standard grades in S4. The group who
sat the higher in the last academic year (2010-2011) didn’t perform as well in
averaging a 2.18 grade across all their exams compared with 1.82 for the group
sitting the higher this year (2011-2012), a difference of 16.5%.
The
average score in the Using Information end of unit test for last years group
was 52.92%. If I increase this by 16.5% to take account of the difference in
the pupil group then I decided on the following to measure success.
Average mark in end
of unit test 2010-2011 (100)
|
Average mark in end
of unit test 2011-2012 (100)
|
Difference as a
percentage (%) using 53.92 as the base
|
Success / Failure
|
53.92
|
<62
|
<16.5
|
Failure
|
53.92
|
>=62 and <65
|
>=16.5 and <21.5
|
Neither
|
53.92
|
>=65 and <69
|
>=21.5 and <29
|
Success
|
53.92
|
>=69
|
>=29
|
Outstanding success
|
Analysis Method 2
1.
As mentioned earlier the class of 2010-11 averaged 2.18 in their standard grades
while the class of 2011-12 averaged 1.82. A difference of 0.36.
2.
To gain a 2 in a credit paper the student must score 50%, a 1 requires 66.66% Therefore,
we can work on the basis that there is a difference of at least 16.66% between
the average 2 pass and the average 1 pass.
3.
If we multiply 0.36 by 16.66 then we will have the amount of marks difference
in the end of unit test out of 100 between the two classes. This figure is 6.
The
chart below shows the actual marks for the end of unit test against the
predictions
Even
if we choose the higher of the two predicted grades, 62.81% it is evident that
the project has been a clear success with the improvement being 7.20% greater
than expected.
Had
I operated similar teaching approach with my S5/6 class of last year and each
pupil had scored 7.20% higher in the final exam then 8 out of 15 of them would
have improved their grade either than from a C to a B or from a B to an A. This
is a profound difference and I am genuinely shocked, pleasantly so, that this
project has had such an impact.
Using Information NABs
Using
a similar analysis to the one above for the NABs produced the following results.
The average score of those in last years class was 16. Again using the 16.5%
difference to take account of the different pupil group I was looking for the
following
Average mark in NAB
2010-2011 (20)
|
Average mark in NAB
2011-2012 (20)
|
Difference as a
percentage %
|
Success / Failure
|
16.07
|
18.7
|
<16.5%
|
Failure
|
16.07
|
>=18.7 and <19.5
|
>=16.5 and <21.5
|
Neither
|
16.07
|
>=19.5
|
>=21.5
|
Success
|
The
bar is far higher here as the pupils in last years higher class performed much
better in the NAB, averaging over 80% compared with the end of unit test where
they averaged 53%.
There
are a number of reasons for this; the
NABs are pitched at a C level and are, therefore, less demanding than the end
of unit test which was based on past paper questions. In addition, the NAB is
only 20 multiple choice questions while the test was out of 84 and required
paragraph based exam style questions which is the skill that is required in the
final exam.
For
the 2011-12 class the average mark in the NAB was 18.4 which is such a marginal
difference that it is almost negligible.
Without being able to read much into the NAB results I used the
comparison of the end of unit tests as a far more accurate measure of the
success of the project.
Focus Groups
I
conducted focus groups with 6 pupils from each of the classes to gain feedback
on which of the formative assessment tools were the most helpful. In the last
academic year, I used a number of formative assessment tools in a less
structured way and less frequently.
Pupils in the 2010-11 class
The
focus group from last year highlighted the following points. They all thought
“teacher talk” was the “Best way [to learn] because you can take in all the
information” although another pupil did mention “If it is for too long it can
get boring. Slides need to be interesting and 20 minutes is the limit.”
These
comments struck me as I wasn’t expecting the pupils to have such a high opinion
of teacher talk.
All
of them found marking each others answers to past paper questions as beneficial
with it being summed up by the comment. “Beneficial as then we know what the
examiner is looking for” although they would be biased in favour of their
friends with one comment summing up the view of the group “I’d be less harsh on
my friends”.
They
were less positive about some of the other tools including jigsawing; “Bad, as
you only learn certain bits and it depends on the quality of your colleagues”,
mark your own past papers “not as strict with yourself so it’s false” and team
and individual quizzes “I don’t like it as much as it depends on the quality of
your colleagues. I prefer it if the teacher asks the questions”
When
I asked them about my proposals for the following year with internet based past
paper questions they were reasonably positive saying “It would be good as you
would not be biased.”
Pupils in the 2011-12 class
Before
starting the focus group my expectation was that the pupils would say they liked
some of the AiFL tools but in smaller doses and would prefer more teacher talk,
in common with the views of last years class.
However,
I was wrong. They all said that they much preferred learning using the AiFl
tools rather than being “told”. Some of the quotes included
“It
get’s boring sitting listening and taking notes.”
“You
learn from your mistakes more [by using AiFL].”
“Taking
notes you lose concentration and your mind wanders”
“A
double period of writing is too much to take in.”
They
also explained that it helped them understand how well they are doing and,
interestingly, how their peers were doing, which they feel helps keep them
motivated.
I
explored the strategy of teachers giving the pupils notes in their jotters for
them to refer to later, something I used to do. They said that it didn’t help
as they would never read the notes again and that they were too busy taking the
notes during the class that they wouldn’t understand what they were learning.
They
all agree that they looked forward to the Information Systems class much more
as there is more variety. The issue of feedback came up repeatedly with the
pupils saying in other subjects they were expected to
sit
and listen and work without any feedback. Which lead to them being “cautious
about asking for help.”
I
then moved the discussion onto the individual tools used. There was less
agreement in the group about the different techniques. A number preferred the
hot seat explaining
“When
it is fun you remember more about it.”
I
was curious to find out whether they retained focus during the hot seat even
when they were in the role of observers, but they all agreed that you had to
retain concentration as
“you
might be the next person to ask a question.”
They
all also enjoyed the competitive element of it.
“makes
you pay attention as you might be asked the same question.” and
“information
sticks in your head more than normal.”
I
then asked them about creating past paper style questions with answer schemes
which turned out to be the most popular. Some of their views included
“more
motivated as there is a prize for the best one.”
And
they
“read
through the notes with a purpose.” and
“you
have to know the answers as you create the questions.”
All
the pupils agreed that they tried to make their questions as difficult as
possible and that they retain far more information after they had finished.
I
was curious to know how they felt about a classmate marking their work as
opposed to a teacher. Again this was something they were positive about.
“It
makes you try harder as you don’t want to be embarrassed by a peer.”
Their
positive feedback continued when I moved the conversation onto a different AiFL
tool - teach other parts of the course to other pupils. Some of the quotes included
“It’s
good as you have to understand it yourself.”
“When
teaching it helps you know your own weaknesses and where to improve.”
It
makes you think like a teacher by knowing you have helped someone get it
right.”
So,
again, we are seeing the pupils feeling they are receiving more effective
feedback and they are more motivated.
I
then explored how they viewed Information Systems compared to their other subjects.
Most of the group look forward to it more as they are more motivated and think
they will do better in this subject. For example
“I’ve
not done this before and it is one of my best subjects.”
“This
is my favourite class.”
and
“I
learn much better when I’m active and doing activities.”
I
then explored how I should teach the next two units; they said they would be
horrified if I reverted to telling and want the course to continue to be taught
in the same way. They went on to say that they don’t get feedback in other
classes so they don’t know how well they are doing. One of the S5 pupils is even
going to ask if his other teachers could use AiFL techniques.
Originally,
I thought the pupils were enjoying the subject but their feedback had totally
exceeded my expectations. In terms of meeting the objective of improving pupil
motivation I don’t think there is any doubt that this has been achieved.
Questionnaires
I
also used web based questionnaires in conjunction with the focus groups to
obtain a complete picture of the pupils views. The benefit of using the
questionnaire is that it took in the viewpoints of entire class. However, the
responses are much more limited and not so in depth as the focus group.
Pupils in the 2010-11 class
At
the start of the course 33% thought they would get an A, another 33% thought
they would get a B with16% thinking they would get a C and another 16% felt
they would get a D.
By
the time a second survey was conducted in April 2011 these figures had altered
with the pupils feeling much more positive about the course. 58% thinking they
would get an A, 33% a B and 8% a fail.
As
is turned out the results were as follows 27% gained an A, 27% a B, 40% a C and
6% a fail. So the pupils’ opinions at the start of the course of where they
would finish were fairly accurate. A self fulfilling prophecy perhaps?
By
April the pupils were slightly overly optimistic about how they would perform
but at Braidhurst I’ve never seen this as a problem as in general the pupil
group tend to be very dismissive of their own abilities and lacking in self
belief.
This
group also rated direct teaching with the teacher explaining using slides as
the most effective way to learn. A highly significant 83% thought this was an either
Excellent or Quite Good way to learn. Some comments included
“I
find it easy to listen and take in information from a PowerPoint as it allows
me to read the information and take it in at my own pace” and “prefer this
method to the others”.
However,
only 43% were positive about jigsawing; pupils teaching each other different
parts of the course. Typical comments included “This is not a good option as we
may learn incorrect information and/or end up in an [irrelevant] conversation.”
And “I personally don’t find group work as helpful as plain old past papers and
direct teaching. Nevertheless, it’s good to have a variety of teaching
methods.”
The
other techniques covered by the questionnaire; Answer past paper questions then
mark it myself, answer past paper questions then another pupil marks it and
create past paper type questions with an answer scheme were met with a fairly
mixed response with the cohort being split roughly 50:50 on these techniques.
Pupils in the 2011-12 class
Survey 1 – August 2011 start of the
unit
I
conducted one brief survey at the very start of the course in August which I asked
the pupils how they felt they would get on and which teaching techniques had
worked for them in the past.
I
was encouraged to find out that 71% felt they would get an A and 29% would get
a B. Nobody felt they would do any worse than a B, so this group couldn’t be
accused of negative thinking.
Again
in terms of teaching techniques they had found helpful in the past a massive
93% thought that Direct Teaching was either Excellent or Quite Good. Only 20%
were positive about jigsawing where the pupils taught each other different
parts of the course.
Interestingly
57% preferred tasks where they are working on their own and 43% preferred to
work in groups of 3 or 4. Nobody liked working pairs according to this.
Survey 2 – November 2011 end of the
unit
At
the end of the project I also conducted a similar survey on the same group
asking their view on the techniques now that they had actually been taught
using them. This made for some very interesting reading with the general mood
being very positive for example 83% now believe they will obtain an A or B in
the final exam which is very encouraging as it shows the pupils are feeling
very positive about the subject.
I
was keen to find out, as this is the most intense AiFL sequence of lessons I
have ever delivered, their overall opinion. I was pleasantly surprised by some
of the comments which included “I much preferred the teaching method of this
course as it actively includes you and you aren’t required just to sit and
listen, you have to take part” and “I quite enjoy the way I’m getting taught in
this course. It is also better as I’m doing much better compared to my other
subjects.”
This
comment encouraged me even more, “I have enjoyed this much more as you can
co-operate with others and also have time to work individually, its better to
work like this as it doesn’t make is as boring and you are always doing
something rather than sitting in one seat writing and taking notes. It gets
people more involved and it helps me to learn much more.” I couldn’t have put
it better myself.
Looking
into the detail of the strategies used the pupils were very positive about all
of them but especially create past paper style questions with an answer scheme
with 92% of the class rating it as either excellent or quite good. Some of the
comments included
“It’s
good as it helps me get to know past paper questions and also find the answers
to them, which will remain in my head.”
And
“This
is another good technique as the questions we make up are challenging and the
answer scheme helps me to understand how to answer the question.”
The
next most popular AiFl tool according to the pupils was the Hot Seat with 75%
viewing it positively. Some of the positive thoughts included.
“This
is good as the questions are hard and it gives you an insight to how you would
cope with these types of question while under pressure.” I was pleased to note
that a couple of the pupils observed that the Hot Seat is not just about the
time a pupil is in the hot seat but about being alert while a peer is being
questioned. For example “We find out the mistakes others make and what we get
wrong we can work on.” I was really pleased that this point was made as it
accurately identifies one of the main points of the hot seat, to help identify
areas of weakness allowing pupils to work on them.
However
75% also said that they thought direct teaching where the teacher
explains
using slides was a good way to learn. However the comments didn’t back this up
with pupils saying
“It’s
good for the first ten minutes then I find myself falling asleep”, “It’s okay
for a short while but in long doses concentration is lost.” Although there were
some positive thoughts for example “using slides is good as there is no
unnecessary information and focuses on the main points of that area.”
Peer Feedback for Principal Teacher of
PE
As
part of the TLC I was observed twice by D Gardiner the PT of PE during the WBP.
His comments were as follows
“I
really enjoyed observing the class where they created their own past paper
style questions, they set about the task with a clear purpose and having spoken
to a few of them they were highly motivated and enjoying the subject.”
And
“This
was the first time I have seen the hot seat in action and it really seemed to
work with this class as there was a tension and healthy competitive spirit in
the air. This is something that will work well in my own subject area, PE, and I
look forward to feeding back what I have seen to my own department.”
What have I learned from planning,
implementing and evaluating the project.
Lessons learned from Planning
As
I planned this sequence of lessons well in advance prior to the start of the
academic year this has allowed me a valuable amount of time to think about how
to best implement the project. However, it has been important to retain some
flexibility, as things don’t always go to plan and I have altered and changed
my plan based on the feedback from pupils. Surprisingly, I have managed to get
through this unit more quickly than usual.
Implementation
The
implementation has gone smoothly with the pupils enthusiastically undertaking
the tasks. Initially, one issue was the quality of feedback given in the peer
assessment tasks. However, there has been a marked improvement in the quality
of feedback given towards the end of the unit.
As
can be imagined there are a number of things I would have do differently when I
implement the same sequence of lessons next year.
1.
I will dedicate
one of my lessons very early in the course to peer assessment and learning.
Even though it is something which is an essential skill for the pupils to gain
this group appear to have had precious little experience of it.
2.
I will ensure
any technical hitches with the website are resolved.
Following
the positive pupil feedback I will use the same AiFL tools with the remaining
two units of the course.
Evaluating
In
terms of evaluation I took two distinct approaches. Firstly, immediately after
delivering the lessons I would evaluate how I felt the class had gone. I noted
this in my lesson planning sheet attached in the appendix.
Secondly,
over the course of the project I was always evaluating how it was going in
general basing my view of the pupil knowledge compared to how much they needed
to know for the exam.
The
lessons I have learned in evaluating the project are the following
1.
I really
benefitted from making evaluation notes after the lessons as my opinion
afterwards was generally accurate. As a classroom teacher I am always
reflecting on my practice but it was helpful to note my thoughts formally. I
will use from now on.
2.
As discussed
earlier I used a wide variety of tools to assess the success of the project. I
have learned that none of these on its own will give a full picture of the
pupils’ progress. I intend to use focus groups, which can be time consuming,
but very valuable in terms of the quality of the information given again. There
are also big advantages to using questionnaire to assess the views of the
pupils as they are quick and easy to implement and the whole class can be
included.
Outcomes
How far did the project succeed in achieving
its aims?
There
were two main criteria I used to judge the success of the project, firstly improvement
in results and performance and secondly improvement in pupil enjoyment and
motivation.
The
result and performance are by far the easiest to measure and having analysed
the results in the same test across the two pupil groups there was at least 7.20% improvement in expected results. In my
view this is a significant enough improvement to consider the project a
success.
Secondly,
did the project have a positive impact on pupil enjoyment and motivation? The results
of the final questionnaire and focus group indicate clearly that the pupils
really enjoyed learning this way and that they would be “horrified” if I
reverted back to more traditional teaching methods.
Finally,
it is my view that this group of pupils took the work seriously, were well
motivated and ultimately the vast majority will be successful in gaining a very
good grade in the Higher exam.
Conclusion
To
sum up what I did, having taught Higher Information Systems for a number of
years I have always been keen to improve attainment and pupil motivation so I
made these my key success criteria.
I
then planned a sequence of lessons to deliver the first unit Using Information
building in a variety of AiFL tools which were used in almost every period. In
addition, there was a focus on exam technique throughout this sequence.
Over
an eleven week period I implemented my plan, taking time to evaluate each
lesson. I used a website I created to allow anonymity for the peer assessment
tasks.
Upon
completion I used a range of tools including questionnaires, focus groups, test
and NAB results to assess the success of the project. From this I came to the
conclusion that, firstly, the use of AIFL and structured revision skills helps
to improve pupil performance in tests by a significant margin which if replicated
across the whole course could improve the final grade of over half the class.
Secondly, the use of AIFL tools significantly
increases pupil motivation
allowing them to become more responsible for their
own learning and the
learning of their peers. In addition, they liked the
variety of the tasks and the
fact that they knew exactly where they were in their
learning thanks to the
constant feedback they were receiving.
My
next step will be to develop resources and lessons for the delivery of the next
two units, Relational Databases and The Internet using AiFL. This approach is
clearly working with this group of pupils and I think I’d have a mutiny on my
hands if I ever dared to “revert to telling” with them.
I
have kept my departmental and TLC colleagues updated throughout this project
and there has been a lot of interest in what I have been trying with many
colleagues promising to embed AiFL tools into their teaching.
In
the long run, I will definitely use all the material and resources with my
Higher class next year and work on introducing these techniques with my classes
further down the school. With the imminent introduction of the Curriculum for Excellence
courses I can make sure that any development work carried out in the department
has AiFL tools embedded.
This
has been a very positive experience for me as I have thoroughly enjoyed teaching
my Higher class this year and it is very satisfying to receive such positive
feedback.
Recently
one of the pupils said to me “Thanks for teaching the course this way” I
replied “Don’t thank me, thank Dylan Wiliam.” He looked perplexed.
