Miscue Analysis - Running Records: What does the research show us?

This is the hub of the site and the place to post queries, start discussions and join in the conversation!
Post Reply
User avatar
Debbie_Hepplewhite
Posts: 2499
Joined: Sat May 23, 2015 4:42 pm

Miscue Analysis - Running Records: What does the research show us?

Post by Debbie_Hepplewhite »

Thank you once again to Dr Kerry Hempenstall (member of IFERI's Advisory Group) for the provision of summaries and links to research looking at the validity and reliability of the form of assessment known as miscue analysis or running records:
“This article has raised a number of concerns about the use of running records. These concerns include: (1) a lack of clarity in the guidelines about whether running records are appropriate for beginning and fluent readers, (2) problems with the comparability of running records taken on different texts, and the lack of assessment of comprehension, (3) the absence of evidence to support the use of self-corrections as an indicator of effective reading, and (4) erroneous interpretation of the meaning of oral reading errors.” (p. 16-17). … “The problems associated with the analysis of errors in running records mean that this practice cannot be recommended. … It is time to reappraise the widespread use of running records as the main assessment of children’s reading in the first years at school. There is a need for other assessments that link more directly to what it is that children require for fluent reading. Teachers need to have access to carefully constructed tests of reading accuracy and comprehension, as well as measures of grapheme-phoneme correspondences, phonological awareness, and word recognition. The use of such tests will provide teachers with crucial information about what children currently know and what they still need to learn to become successful independent readers.” (p.18)

Blaiklock, K. (2004). A critique of running records of children’s oral reading. New Zealand Journal of Educational Studies, 39, 241-252. Retrieved from https://www.google.com.au/#q=%22lack+of ... ds+are+%22
“Running records have traditionally been viewed as producing accurate assessment results because they provide an approximation of authentic school and home reading. However, reliability data have not been conclusive regarding the use of running records (Ross, 2004). … Running records have traditionally been viewed as producing accurate assessment results because they provide an approximation of authentic school and home reading. However, reliability data have not been conclusive regarding the use of running records (Ross, 2004).” (p.114). … “Hoffman, Roser, and Salas (2001) found that teachers using the Fountas and Pennell leveling structure can reliably level text. However, when texts leveled in that manner assess student reading performance, they produce highly unreliable results. Running record scores that are acquired from a single-leveled text reading would not necessarily represent a student’s true reading level. … Making absolute decisions with a running record requires the teacher to average student scores on at least three passages with at least two raters. Our results indicate that the most limiting factor in rendering students’ running record scores reliable is the number of passages used. That finding supports the contention of Ross (2004) that passage might exhibit a sizable source of error variance when scoring running records. Using a single score obtained from reading a single passage to portray that student’s universe score would be highly questionable.” (p. 123). ... “Teachers should recognize that traditional text-leveling procedures do not fully account for all factors that affect the difficulty of a text. Even when controlling for text level, type, and structure, there are still naturally occurring topical variations between texts that will render one more difficult than another”.(p. 125)

Fawson, P.C., Ludlow, B.C., Reutzel, D.R., Sudweeks, R., & Smith, J.A. (2006) Examining the reliability of running records: Attaining generalizable result. The Journal of Educational Research, 100(2), 113-126. Retrieved from https://llmotivation.wikispaces.com/fil ... ecords.pdf
“In summary, there has long been a significant difference of attitude toward miscue analysis and running records between the (mainly) whole language fraternity and reading researchers. The underlying theory has been thoroughly discredited (Gough, 1993; Perfetti, 1985; Rieben & Perfetti, 1991; Nicholson, 1991; Stanovich, 1986, 2000; Turner& Hoover, 1993), and the method of assessing the miscues has been shown to be unreliable (Allington, 1984; Leu, 1982). One would imagine that miscue analysis would have slipped quietly into education’s long history of intuitively derived concepts that proved incorrect when the light of empirical study was played upon them. However, consistent with the long standing disconnect between evidence and practice in education, miscue analysis and running records remain evident in many education settings (Beatty & Care, 2009), no doubt partly due to the continued acceptance among educators of the 3 cueing process.”

Hempenstall, K. (2013). Miscue mischief. April 11, 2013. Retrieved from http://www.nifdi.org/resources/news/hem ... e-mischief
“This study revealed that children who were reading at or above grade expectations were more likely to make use of graphic and phonic cues when they approached an unfamiliar word in a text. As has been suggested (e.g. De Lemos, 2002; Adams, 1990) the ability to decode graphic and phonic information is a reliable determinant of higher reading ability. Therefore, these findings suggest that Goodman and colleagues may have underestimated the role of the grapho-phonic cueing system and overestimated the role of the semantic and syntactic cueing systems in relation to differentiating reading ability.”

Beatty, L., & Care, E. (2009). Learning from their miscues: Differences across reading ability and text difficulty. Australian Journal of Language and Literacy, Oct 2009. Retrieved from http://connection.ebscohost.com/c/artic ... difficulty
Back to Debbie Hepplewhite again:

Following all the objective observation of children's reading above, I'm just going to add a personal anecdote - because this was one of many aspects of my teaching experience that alerted me to the fact we can teach reading badly, or wrongly, with poor consequences for children's reading habits:

I am a practitioner with a primary teaching background (in England) and was trained to use a form of running record and miscue analysis. Many years ago I deliberately focused on infant teaching for a while because I couldn't understand why, teaching juniors, so many of them struggled with reading and writing so I wanted to investigate. I noted that most of my struggling readers' mistakes ('miscues') were based on the very reading strategies that I was also being told as a (then) infant teacher to teach the children! How could this be? In effect, the errors were invariably based on word-guessing from various cues and very poor, or no, application of phonics decoding (this is before the official promotion of systematic synthetic phonics in England). So, as teachers we were being trained in the multi-cueing reading strategies which, in effect, caused the children's reading errors! We were trained that it was not good if children relied on their phonics and that they SHOULD be using a range of strategies (guess from the picture, the context, the initial letter). 'Phonics' was the very poor relation and used more 'to check' if the word that had been guessed matched the first letter, or letters, of the printed word.

Yes - THAT bad!

Not only that, I also noticed that my very able, advanced readers - very articulate and with great comprehension - had this reading habit of making up all the little words and even great chunks of sentences - in effect, matching how they spoke or predicting what they thought the sentence would be. They reflected worrying inaccuracy word-by-word - but they had great comprehension to all intents and purposes. They also read with the much promoted 'fluency and expression' - and yet they were so inaccurate and seemed oblivious to these inaccuracies. Very interesting, and worrying, too - was that their level of writing did not match their reading level and comprehension level - it was much, much weaker and they were often reluctant writers. It's not that they could not express themselves orally, it was that they were not good spellers and not fluent with handwriting. Again, this was before the uptake of systematic synthetic phonics in England - the teaching principles of which include teaching not only print-to-sound decoding (reading), but also the reverse sound-to-print for encoding (spelling).

We have progressed a great deal in England thanks to the dedication of some individual politicians and others to the point where we have the systematic synthetic phonics teaching principles embedded in the statutory national curriculum for English for Key Stages One and Two (primary) - but we still have problems with 'mixed methods' and even 'mixed messages' - as you can see from the various threads on the IFERI message forum!

I am now a teacher-trainer and part of my training includes some discussion about running records and miscue analysis (still the basis of the Reading Recovery intervention programme!). I point out that we can all note exactly the same features of a child reading a text (that is, end up with the identical running record of the child's reading) - but that our 'professional understanding' will heavily influence how we interpret the 'miscues' or errors.

As a systematic synthetic phonics teacher, I would be concerned by a child guessing at the words, not reading all the words accurately, not noticing that the sentences as read by the child did not make sense because of wrongly-guessed words, stumbling because the words included letter/s-sound correspondences that the child did not know (one or two, fine, but lots, not fine). I can recognise by the way a child reads a text how the child has been taught to read in the first place by the range of such miscues. I can recognise when a book is not suitable for a child to be asked to read independently because the child has to guess so much just to 'get through' the book. This is a very stressful situation which many children continue to be put through because of lack of adult's knowledge and understanding about reading instruction and reading practice - and the continued practice of Book Bands levelling or Levelled Readers in other countries.

In other words, children are mistaught.

A mixed methods or whole language teacher, however, would think it is just fine for a reader to take calculated word-guesses from various 'cues'. After all, aren't they teaching multi-cueing in the first place?

I must add, however, that during all these years of me on my personal voyage of discovery of 'why' so many children were not strong in their foundational literacy skills (even when they were highly articulate), the published reading book schemes were designed on the basis of repetitive and predictable texts - not cumulative, decodable phonics books. I personally approached various publishers at that time - urging them to produce cumulative, decodable phonics books that would match systematic phonics teaching. Even any existing phonics programmes at that time did not have any matched, decodable reading books.

In England, we have made a lot of progress and now we have a number of good quality cumulative, decodable reading book schemes which are SO important for beginner readers to be able to practise their phonics code knowledge and decoding skills - building up their reading fluency and confidence. Unfortunately, however, we also have a Book Bands cataloguing system which continues to include the kind of early reading books which, in effect, children cannot get through without resorting to guessing. Many teachers still provide a mixture of these different types of books - which is potentially very damaging for the weaker readers in particular - and which undermine any good explicit phonics teaching.

The story continues.....
Post Reply