This post details my use of a student survey to evaluate my performance on the PEC course at MMU. It shows some work I have carried out under dimension A5 of the UK PSF, as I am evaluating my professional practices with a view to improve pedagogy. I believe it also also demonstrates dimension K5 (methods for evaluating the effectiveness of teaching), and K6 (quality assurance and quality enhancement for academic and professional practice).
As this process constitutes a simple data collection to feed into a reflection on my teaching rather than a formal evaluation much of the available literature is something of an ‘overkill’. However, Gravestock & Gregor-Greenleaf provide an excellent summary of the benefits and pitfalls of student evaluation, and I have endeavoured to follow some of the principles they outline, including educating students on the use of the data, and avoiding a “poor presentation and contextualisation of evaluation data” (2008:6).
Sharp (1990) advises an ‘illuminative’ style of evaluation, with varied data collection and an explicit involvement from students. In this case I shall be collecting simple numerical ‘Lickert-type’ data (Likert, 1932), but also encouraging students to provide a more detailed commentary in text box for each rating. Such quantitative data is important as the low number of student on the course gives a ‘small N’, making entirely quantitative judgements invalid (Boysen et al., 2014).
Students scored each course activity on a scale from “1: of no use and/or uninteresting” to “5: extremely useful and/or very interesting”. Each numerical option was given a comments box, and students were encouraged to add information to explain their choice here. They were also advised that the evaluation was entirely anonymous, and will only be used to improve the course for forthcoming years.
Scriven (1995) highlights a range of common errors related to the use of course ratings data, including the use of scores without regard to distribution. The mean score for activities given below, and I have included the range of scores for each to give some idea of the distribution.
- Physics sessions: 4.8 (4 – 5)
- Biology sessions: 4.3 (2 – 5)
- Chemistry sessions: 2.5 (1 – 3)
- Maths sessions: 2.8 (1 – 4)
- Moodle resources: 4.1 (3 – 5)
- Mid-Course Online Assessments: 4.3 (4 – 5)
- Personal project: 4.3 (3 – 5)
- Portfolio completion: 4.6 (4 – 5)
- Your project presentation: 4.6 (4 – 5)
- Observation of others’ presentations: 4.7 (4 – 5)
- Educational visits: 3.8 (3 – 5)
- External sessions with school pupils: 3.5 (3 – 5)
Overall I was very pleased with the high marks for university physics sessions; they were mainly 5s, with entirely positive comments including “well-structured”, “high-quality”, “absolutely essential”, and “so much information and really well-delivered”. It was also interesting to see students picking up on the variety of pedagogical approaches; “many useful tips that I can take into my teaching career”, “useful to observe [tutors’] different teaching styles”, and “…a great help in understanding the physics principles and the practical skills required for teaching.” No suggestions for improvement were received from students at this point. Whilst not precluding changes being made, it does suggest that physics instruction may be a strength of the course.
In order to further confirm students positive reporting of physics sessions I observed a colleague teaching on the course. The content delivered was at an appropriate level, and used a range of suitable strategies (presentations, animations, demonstrations, independent research, and ‘hands-on’ practical work). It was interesting to note that he adopted a very similar manner with students to my own approach, being quite informal and conversational rather than didactic. Whilst this is perfect for such a small group as it allows them to be more comfortable asking questions and seeking help, it would not be appropriate for larger groups or lectures. The tutor I observed had a different approach to practical work, in that rather than demonstrate an activity for students to replicate he frequently had a sample method and additional activities printed on a single side of A4. This is something I have since used in my own practice, and is a more efficient use of time that promotes an independent approach to practical work. He was also very proactive in making connections between the theory being studied and the wider world through the use of recent news articles and interesting examples. This added a lovely colourful dimension to the session, and is something that I have tried to emulate.
There is a variability in students’ responses to other subject sessions, a small number of which were included to either prepare students for qualification as a general science teacher, or to help embed mathematical skills associated with the subject.
Biology was generally well-received, with comments such as “enjoyable content and appropriate pace”, “excellent… clear and concise”, and “nice to have additional information”. This tutor had explicitly structured sessions to include PEC students in an existing course, which was not the case in chemistry. This may be reflected in the low feedback scores and somewhat negative comments; “not great joining people already on the course”, “part of an ongoing course so wasn’t targeted at out needs”, “not enough chemistry as was part of [existing course] not aimed at us”. This suggests a need to work on the integration of the chemistry portion, and an opportunity for the biology tutor to share good practice.
The student experience was again different in maths sessions, with a larger range of scores reflecting written feedback; “maths was atrocious”, “not specific enough to us”, “my maths is strong but if it wasn’t these sessions wouldn’t have helped improve it”, “good for one session, but became very ’samey’ “. I know that one aim of the maths sessions was to develop individual thinking and problem-solving skills, and highlight how these can be promoted in schools. Perhaps insufficient communication of these aims with physics students devalued the experience somewhat.
I was pleased with the Moodle resources and online assessment scores, especially given the efforts expended in improving this aspect; some concerns were raised around the speed at which materials were shared with students, but otherwise feedback was very positive. The variety of assessments was appreciated, and seen as useful in portfolio building.
The portfolio was deemed to be highly beneficial, and many students produced excellent materials. Two wrote their own version of a physics textbook, and several others filled three lever arch files full of quality material that will be transferable to their own teaching. This reflects the positive outcomes suggested in an earlier post (see section ‘Assessments and Misconceptions’).
Whilst trying not to read too much into small differences, feedback regarding the research projects was generally good. Independence was celebrated, as was the chance to apply learning; “good to have lots of freedom”, “great to have an objective to help keep focused”, and “gave me a chance to explore physics and open my mind”. The chance to practice presentation skills was also appreciated; “helped to prepare [me] for teaching, and offered a chance to find and show a passion for what you’ve learnt!”, “nice to see everybody’s progress from when we first began, also a good practice for teaching in front of a group of people”.
The visits were self-organised, so I imagine the students’ experienced depended upon the venue selected. There were some suggestions for improvement around paperwork and connections to wider aspects of physics which will be addressed.
Experiences interacting with school pupils were seen quite negatively, and I think this is due to poor organisation on our part, especially with the deployment of PEC students in such situations. Feedback echos this, as these experiences were “Interesting, but feel it would be good for PEC students to have more of an input” and “didn’t feel like we were needed”. Whilst further involvement with school pupils may be difficult given health and safety aspects of interactions, I take on board requirement for more involvement.
- Maintain integrity of physics instruction – this is the main body of the course so remains a key priority
- Increase the range of resources and assessments available on Moodle, and ensure resources are shared sufficiently quickly
- Ensure all additional subject input is integrated with the physics student experience, and relevance to the course is regularly emphasised
- Provide more structure and a flexible word count for writing-up educational visits
- Consider more active deployment of students at events involving school pupils
Models and Trends
The MA Education course at MMU is a part-time taught programme designed for education practitioners to interrogate their practice. As the online course profile states, it is…
…founded on a philosophical principle that educational work is an intellectual activity, and, as such, educators are entitled to an autonomous academic voice. Much of our activity focuses on enriching that voice, and supporting it so that it might operate in a more assertive and substantiated way.
Students on the course attend conference weekends for each four-month, 30 credit unit, which is assessed through a written assignment that is usually grounded in their practice. The course finishes with a 12-14,000 dissertation.
The course attracts a range of students, including teachers, lecturers and academics, youth workers, and many others in the field.
A key part of the redevelopment of the Physics Enhancement Course was to improve use of the Moodle virtual learning environment to give a more successful ‘blended learning’ student experience.
This process took three parts;
- An investigation into suitable Moodle tools
- Devising strategies for resource-sharing, including digitisation of existing resources
- Development of more interactive, supportive online resources, activities, and assessments
This post details the development of my skills in using Moodle, and records some observations from its usage. It evidences an engagement with UK PSF dimension A4 – developing effective learning environments and approaches to student support, and incorporates dimension K4 (appropriate learning technologies).
The online element of the course was extremely important; given the range of backgrounds, ages, and personal circumstances of PEC students a variety of avenues by which to access learning was essential. Barriers such as childcare and unavoidable additional employment impact upon attendance perhaps more than other courses, and a degree of flexibility promotes participation and increases equality of opportunity. This fits with dimension V2 of the UK PSF.
Investigation and consultation
Already having a background in the Moodle basics I first sought to improve my skills by enrolling upon the Learn Moodle MOOC. This gave me a better understanding in the types of resources that might be available, but was not specific to my institution.
In order to promote effective use of the Moodle MMU employ several Technology Enhanced Learning Advisers (TELA). Given that one day per week of the PEC timetable was scheduled for consolidating learning and portfolio building, effective use of the online platform was critical to student development. A meeting with the Education TELA highlighted several existing Moodle functions that could be applied in our context;
- File organisation is easier with folders
- The quiz module has much flexibility for questioning and surveys
- The assignment function allows a wide variety of submissions
- Embedded media and animations are handled fairly well
- Links to external sites allow additional tools to be used
Sharing lecture and tutorial resources (such as presentations and handouts) is a common use of Moodle, and many people are familiar with the upload process and organising files in folders. Students on the PEC found this particularly useful, as can be seen in their course evaluations. Below I concentrate on the remaining four uses of Moodle listed above.
Using the quiz functionality
At the start of the course all students complete a subject knowledge audit. This self-assessment allows tutors to judge current levels of understanding and confidence to ensure learning sessions are pitched correctly and tailored to students’ needs.
The audit has traditionally been completed on paper, or as an Excel document. However, this has prevented effective analysis as data is distributed between sources and in a variety of formats. To centralise this data, and make the audit process more simple for students, I used the Moodle quiz module to put the audit on the VLE, as can be seen below.
In addition to simplifying the audit process, it became an easy exercise to repeat the audit at the end of the course and see what progress students thought they had made. I used downloads of data from Moodle and a Microsoft Word mail merge to create a documentary record of individual student’s perceived progress, which was used in both assessment and at the exam board.
Using assignments flexibly
The assignments function in Moodle allows students to upload any text or document for assessment. Rather than be limited to essays or exam questions in this case I had students use an online tool to produce a crossword then upload it as an assignment, as seen below.
The thinking behind this was to provide the words and force students to seek out appropriate definitions, therefore not only learning standard meanings but also experiencing what it would be like for school pupils who were set a similar task in dealing with a plurality of definitions. This ‘thinking like a pupil’ is an important part of developing effective pedagogical practices, and as such an important skill for trainee teachers.
Sporcle provides a range of trivia quizzes, some of which can be used as a tool to inform or engage students. For most quizzes there is a ‘share’ button on the page which allows code to be downloaded and inserted directly into Moodle.
On the PEC we used the above quiz on naming elements as a session starter and discussion point. When it transpired that students wanted to practice in their own time I uploaded it to Moodle. This not only gave access to the quiz, but also forced students to use Moodle, thus displaying other resources an assessments stored there.
The below example again shows a use of external sites for assessment, engagement, and submission through the assignment module. Timeglider allows the creation of timelines including notes, images, sounds, and videos. This was used to assess students’ ability to describe the history of our understanding of the atom in an engaging way – an important skill given that the Dalton, Thomson, and Rutherford models are all used in GCSE Physics.
This use of engaging, appropriate Assessment for Learning (Black & Wiliam, 1990) was certainly successful; all students submitted all assignments, and it gave valuable information on their abilities mid-course that informed the shape and focus of sessions in the second half. Student evaluations were universally positive. It is also important to try new approaches and exemplify good (or at least different) practice, especially as the PEC course is 20% subject pedagogy.
This post describes my teaching of the course planned in an earlier post, and as such demonstrates an example of my meeting dimension A2 of the UK PSF. It also shows my understanding of how students learn in this context (K3) and an ability to design appropriate methods for teaching postgraduate pre-ITT students about DC Electricity (K2).
Teaching began with a group discussion on the topic “What is electricity?”, the purpose being to ascertain prior understanding to enable personalised instruction and differentiated teaching later in the unit. This approach has had positive responses from students in previous studies (e.g. Windschitl, 1999). The discussion lead into a more didactic ‘lecture-style’ presentation of foundation concepts using slides of notes, images, and visual animations, alongside practical demonstrations (such as the use of a van der Graaff generator to support understanding of the ‘threshold concept’ of electrical charge).
The content-led didactic portions of sessions were interspersed with simple activities to increase student engagement. An example of this was ‘code-cracking’ to determine the value of a resistor from the coloured bands printed upon it. Whilst this may seem trivial it emphasised the range of values and tolerances available, familiarised students with the resistor as an electrical component, gave a chance to demonstrate problem-solving skills, and the subsequent experimental confirmation allowed experience of hands-on testing of components.
The importance of practical work in studying science subjects has been emphasised in many studies, (see Woodley, 2009; Dillon, 2008; and Abrahams & Millar, 2008). Whilst some practical activities included in this unit of the PEC were somewhat basic, differentiation was used to ensure all students were engaged and learning. An example of this was testing materials to see if they are electrical conductors or insulators. Although this is a primary school level activity, some less experienced students were given a method to follow and the aim of familiarising themselves with equipment, whilst students with more of a background in the subject were encouraged to take a problem-based learning approach, and use patterns, measurements, and additional independent research to attempt to relate their findings to the atomic structure of the materials in question.
In other practical activities students took the lead on learning. A particularly successful example of this approach was making fruit batteries. The initial activity was to see which fruit produced the greatest potential difference, but students set themselves the extension task of seeing how many grapes would be required to charge a mobile phone. At this point they allocated group roles depending upon expertise; those with more mathematical confidence carried out calculations and made predictions, whilst those still getting to grips with equipment put mathematical predictions to the test. This not only shows a level of ownership but also embodies a sort of active learning that has shown positive results in studies (see for example the metasynthesis of Prince, 2004).
Each day finished with some small-group examination-style questions to strengthen connections between the theory covered in the session and the prescribed content, which has been shown to greatly aid retention, far more than standard ‘revision’ (Karpicke & Roediger, 2008). Content was re-visited at the start of the following day through a series of ‘quick-fire’ activities involving anagrams, quizzes, and puzzles based around definitions.
A significant portion of the unit (and indeed Physics) is mathematical in nature, so it is essential to embed numerical skills development (the importance of mathematical skills in HE has been recognised by the HEA – see Croft & Grove, 2006).
Familiarity with manipulating equations was promoted by the use of a range of questions of increasing difficulty. Students were encouraged to get as far as possible in a fixed time, with a use of classroom timers to give a sense of pace. This is one use of use of Assessment for Learning (Black & Wiliam, 1990) to give a ‘baseline’ level of mathematical ability which could be subsequently used to inform future activities and groupings in later sessions.
The approach to embedding mathematical skills during practical work was usually twofold. First, students made experimental measurements and plotted a graph. Clear success criteria for plotting were given in advance, and one-to-one support provided. Practical results were then algebraically checked against theory. Students worked in pairs or small groups, and I ensured that those most familiar with mathematical techniques were paired with students without such a strong background. This type of peer support has been shown to be effective by studies such as Parkinson (2009).
Subject pedagogical knowledge
Discussing the pedagogy of DC Electricity as it applies to high-school teaching is easy to embed in the PEC; an approach that asks questions along the lines of “we are carrying out some learning activities that are appropriate for the topic. How might you implement them in a school setting? What difficulties might you face? What might school pupils find difficult? Or interesting?” serves as a suitable starting point.
The van der Graaff demonstration is an excellent example of this, as not only can one emphasise the variety of learning opportunities in the demonstration, but also highlight the health and safety implications of such equipment in schools, referencing not only practitioner literature (e.g. SSERC, 2007), but also personal anecdotes from my experiences of teaching this topic.
Assessment and Misconceptions
Alongside the taught portion of the DC Electricity unit students were given time to construct a portfolio which A) demonstrated and documented what they had learnt, and B) would allow them to easily access this learning on subsequent courses. Part A received significant continuous formative feedback during session contact time, and was made up of results of experiments, exam questions, and class-based written tasks. Part B was assessed summatively, and was more like a personalised textbook of notes and teaching materials.
Given that role of the PEC is to provide a background knowledge for students to take into their initial teacher training courses, it seems logical that a large proportion of the formal assessment is by portfolio. Segers et al. (2008) also suggest that adopting ‘assessment tasks as an integrated part of the learning environment… resulted in enhanced performance’ (p. 43).
A major strength of such continuous assessment was the prompt identification of misconceptions. One example of this is understanding the ‘threshold concept’ of current and voltage, ideas that are frequently confused. After an initial discussion of the topic highlighted some misconceptions, I employed the use of two analogies; a some school buses, and a series of water pipes. As a group we discussed the strengths an shortcomings of analogy, and students were encouraged to think of own – a task that not only shows understanding, but also serves and aid to memory (Halpern et al., 1990). As such students were better able to articulate the different concepts of current and voltage.
Outcomes and Evaluation
Whilst it is beyond the scope of this post to formally evaluate the unit, and results of a end-of-course student survey are presented in another post, some brief observations can be made.
The first is the excellent work produced for portfolios, which I believe show a suitable assessment model and a high level of engagement. The mixed and differentiated approach to learning also seemed successful, with students informally reporting that they found the sessions informative, useful, and enjoyable.
I was, however, surprised by the by level of preference for ‘hands-on’ practical work; the demonstration with the van der Graaff generator was not so well received, especially in comparison with playing with fruit batteries. It seems that even postgraduates enjoy getting their hands dirty!
The final observation I would make was that three continuous days of relentless DC electricity was perhaps a little exhausting (for students and tutors alike!). There was also a little replication of learning with some practical tasks being too similar. Next year I will cut the content into more discrete chunks, and trim down the quantity of practical exercises.
Wordcount: 1301 words
The Physics Enhancement Course (PEC) has been running at MMU for a number of years, but resources have not traditionally been shared or stored centrally. As a new member of staff this meant developing resources for my taught units from a standing start, the first being the unit on DC Electricity.
This post describes my use of an evidence-informed approach (UK PSF dimension V3) to plan the unit, utilising knowledge of subject material (K1) and an understanding of how students learn (K3), which covers dimension A1. A further post considers the teaching methods employed in this unit.
Course and Unit Structure
Teaching national curriculum, school-level science to children is a very mature area of research, but strategies for performing the same to postgraduates in an HE environment are less well-documented. Given that DC electricity and circuits are a fundamental part of any physics course, however, it is no surprise that much has been written on the teaching of the subject.
The rationale for the PEC is such that
Participants will be able to develop their subject knowledge where physics is a non-specialist subject or requires strengthening after a period of absence from the topic. There will also be a focus on how the pedagogy of physics is related to a deeper understanding of how science works and ultimately how it underpins our fundamental understanding of the Universe. (from the course handbook)
This represents something of an interesting nexus; school-level physics content being taught to postgraduates prior to their initial teacher training later in the year; it is a subject-refresher, a professional course in pedagogical skills, and an introduction to basic laboratory work in physics. For this unit the question then lay in looking at how to distribute the subject content knowledge (“learning the subject”), subject pedagogical knowledge (“learning to teach the subject”), and ongoing assessments.
Content knowledge must take primacy, as it represents the key purpose of the course. Results of national surveys express a preferential split of roughly 80% content knowledge and 20% subject-specific pedagogy (Gibson et al., 2013:54). The learning activities involved should be varied and at different levels, as “It is possible to bring non-specialists up to speed with subject knowledge, but highly differentiated teaching approaches must be used…” (Shepherd, 2008:48).
The outcomes stated in the course handbook suggest that the DC Electricity unit should
…develop an understanding of static electricity and its uses before moving onto series and parallel circuits and concepts of measuring D.C. electricity… Students in this part of the course will get the opportunity to engage in extensive practical work which is designed to help with their understanding of the topic. (from course handbook)
In addition to the material in the course handbook, subject content was taken from the National Curriculum for KS3 and 4 (DfE, 2013), and AQA GCSE (AQA, 2016) and A level (AQA, 2015) specification documents. This provided a means to ensure that all relevant content was covered. For the purposes of planning this unit I took much from the metasynthsis of Guisasola (2014), particularly section 5.1.3 which highlights the misconceptions held by physics undergraduates when learning this topic, and section 5.4 which provides ’guidelines for designing teaching-learning sequences’ in this area. Based upon the ideas of Land et al. (2005) I focused on the sequence of content and paying attention to threshold concepts of charge, current, potential difference, and resistance.
The importance of both subject and pedagogical knowledge is highlighted in Angell et al. (2005). To ensure I included appropriate teaching strategies and highlighted potential misconceptions of schools pupils I consulted Mulhall et al. (2001) and Wellington (2000), who also provided some guidance on potential practical work. Having taught this subject at a secondary-school level for 9 years I was able to draw upon my own experiences and existing resources to support my teaching.
Three sessions of 6 hours each were calendared for this topic. Through combining pedagogical theory and resources listed above it seemed that placing an emphasis on the following aspects would allow students to effectively meet the stated aims.
- A differentiated range of theoretical, practical, and investigative activities to cover subject content
- Modelling and signposting of teaching approaches and key points of subject pedagogical knowledge
- Some formative assessment, including identifying and unpicking potential misconceptions
Once the priorities for teaching of the unit were established the next step was to plan and teach individual sessions, the specifics of which are in another post.
Wordcount: 722 words
Science teachers in secondary schools are frequently required to teach outside of their subject specialism; Physics graduates teach Biology classes, Biologists teach Chemistry and so on. Whilst many of the practical, pedagogic, and classroom-management skills are transferable, there is always the possibility of gaps in subject content knowledge. Furthermore some applicants onto secondary science ITT courses may not have a science degree, frequently coming instead from an engineering or sport & exercise background. In an attempt to mitigate any subject knowledge gaps whilst addressing the vast shortage of science teachers (Osborne & Dillon, 2008, p. 24) the Department for Education introduced university-led Subject Knowledge Enhancement courses (SKE), the successful completion of which allows postgraduates to teach in areas outside of their existing subject specialism.
At MMU we offer two versions of the Physics SKE; the 24-week Physics Enhancement Course (PEC), and an 8-week Physics SKE ‘booster’. Both courses have a heavy practical focus, significant face-to-face teaching, and a ‘blended learning’ element of independent study. I have been employed by the university since January 2016 to administer and deliver both of these courses, but in this post will be focussing on the 24-week PEC.
As the first course started 2 weeks after my appointment there was limited time to carry out any changes, so the 2016 PEC ran primarily using existing course structures and materials. The target was to run the course through once in 2016, carry out an evaluation then update the course to begin again in 2017. The only exception was that I chose to increase the use of the Moodle virtual learning environment to give a more rich blended-learning experience for students beyond the basic file-sharing capabilities of the existing Microsoft OneDrive system.
I hope to use this portfolio to document the process of evaluating and modifying the PEC for 2017. In order to demonstrate that my practice is aligned with the UKPSF I will tie this into UK Professional Standards Framework published by the HEA.