What's Wrong with Rubrics

[scrivener]

Alfie Kohn, author of one of my favorite books about education, Punished by Reward, recently published in The English Journal (a publication of NCTE) a great article called "The Trouble with Rubrics."

I have been a rubric devotee for most of my ten years in the classroom. Some of my colleagues call me "The Rubricator" (which is actually a word, believe it or not!) because I even make rubrics for math tests, which you'd think would be objective enough! However, lately, I've been having second thoughts about the whole rubric thing, and this article really articulates some of my feelings.

It begins:
Once upon a time I vaguely thought of assessment in dichotomous terms: the old approach, which consisted mostly of letter grades, was crude and uninformative, while the new approach, which included things like portfolios and rubrics, was detailed and authentic. Only much later did I look more carefully at the individual floats rolling by in the alternative assessment parade -- and stop cheering.

and goes on to say:
What all this means is that improving the design of rubrics, or inventing our own, won’t solve the problem because the problem is inherent to the very idea of rubrics and the goals they serve.

Kohn is a great writer with a clear voice, so the article's worth a read even if only for that reason, but I think he brings up some very good points. Since I'm inclined to agree with him, I'd love to hear other opinions, if you've got 'em!

Check out the full article here (http://www.alfiekohn.org/teaching/rubrics.htm) and tell us what you think!

[Krafty]

Thank you for the link. While I have heard the term rubric, I had never bothered to discover the meaning. (I keep stating this for those that don't know, I am just starting to train in teaching. I have not taken college course in education. I'm using this board as one of my sources for information. I should just put it in my signature. :D )

I can certainly see, from a standardized testing perspective, why a rubric would be attractive. Writing is subject to the interpretation of the person grading it and this would give a more precise ruler for measurement. On the other hand, just because a piece is accurate, answers the question and is technically correct doesn't make it "good". "Good" is also a feeling that you get from the piece you are reading, and no matrix can measure that feeling.

If we could put writing standards in a strict formula, we could in theory develop a computer program to do all writing for us,from essays to poems to plays. Plug in a topic and let the program run. But there would be no sense of style, no subtle humor or insightful comparisons. As Kohn seems to indicate, you can write an essay that scores well but is still lacking in many regards.

I found this particular quote of interest:
The fatal flaw in this logic is revealed by a line of research in educational psychology showing that students whose attention is relentlessly focused on how well they’re doing often become less engaged with what they're doing. There’s a big difference between thinking about the content of a story you’re reading (for example, trying to puzzle out why a character made a certain decision), and thinking about your own proficiency at reading. “Only extraordinary education is concerned with learning,” the writer Marilyn French once observed, whereas “most is concerned with achieving: and for young minds, these two are very nearly opposites.”

One of my favorite quotes is from Albert Einstein who said "education is that which remains when one has forgotten everything learned in school". I feel Einstein's quote is along the same lines of Ms. French's thinking, though I am sure at the time it was not referring to rubrics. It makes sense that if students know they will be graded strictly on the rubric matrix and have it in hand ahead of time, they will write to the rubric standard only and not in a way that shows the ability to write creatively or think beyond it. This is the standard; I will only get a good grade doing it this way so I will conform. So what have they actually learned? It would seem that many are only learning how to get a good grade, not how to think, discover and continue learning.

Unfortunately, standardized tests are a reality. For good or bad, they exist and a measured standard for grading is necessary and a rubric may be unavoidable. However, it would seem that while a rubric may be a good tool for developing a base grade in writing (at least for the technical aspects of spelling, grammar, punctuation, etc), it should not be the basis for the entire grade on the classroom level. Use of analogies, humor, unique approaches, etc are not measurable by a strict ruler. Students need feedback, not only on the technical aspects of their writing, but on the intangibles as well.

This certainly gives me some insight on the methodology that may lie behind the Praxis II test I will have to take that requires 4 essays. I spoke with a teacher friend today that seemed to think that the essays were easy enough to crack if you made sure to address each question and "regurgitate" some of the buzz words. Sort of sounds like a rubric to me.

[AAA Teacher Forum]

I can certainly see, from a standardized testing perspective, why a rubric would be attractive. Writing is subject to the interpretation of the person grading it and this would give a more precise ruler for measurement. On the other hand, just because a piece is accurate, answers the question and is technically correct doesn't make it "good". "Good" is also a feeling that you get from the piece you are reading, and no matrix can measure that feeling.

Krafty,
I understand your point of not wanting to make the art of writing too standardized or lose it's emotional qualities, but at the same time I would have to disagree that a rubric can't be used on all aspects of a piece.

I know that you are just beginning your teacher training, so you may want to visit the North West Regional Educational Laboratory (NWREL) and take a look at the 6+1 Trait writing research they have done. You will also find the various rubrics they use when assessing writing. http://www.nwrel.org/assessment/department.php?d=1

From this research, they have found that there are 6-7 traits that make up every writing piece. Here is what they have to say about "Voice," which is what I hear you referring to in your post.

Voice
The Voice is the writer coming through the words, the sense that a real person is speaking to us and cares about the message. It is the heart and soul of the writing, the magic, the wit, the feeling, the life and breath. When the writer is engaged personally with the topic, he/she imparts a personal tone and flavor to the piece that is unmistakably his/hers alone. And it is that individual something–different from the mark of all other writers–that we call voice.

I whole-heartedly feel that you can never eliminate the "human" aspect of writing. I believe this is why in Colorado, our standardized writing test follows a rubric, similar to the 6+1 Trait, and is scored by humans. It's tricky...you have to be able to fairly assess students (and report out to parents/community) while also respecting the craft of writing and human creativity.

Jennifer

[javamomma]

The only reason I like to grade using a rubric is there are some students papers that if I glance I think "no way". Usually this student has not used punctuation, or capitals, ect. But when I actually disect the the writing I realize there is some good content there. They dont get an "A" but not an "F" either.

[Bruce Gremmler]

I think the weakness of rubrics, as described in the article, is that they're tied to letter or numeric grades.

The most recent example I've seen of an effective rubric was last winter when my family took a ski vacation. Our four-year-old daughter spent her days in ski school while her mom and I skiied. At the end of each day, we received a "report card."

The daily report was nothing more than a set of objectives, presented in the sequence that they were taught. Every objective that had been taught and assessed on that day or the days previous were labled "mastered" or "working."

When we received that card every day, we knew exactly what we could and couldn't do with our little girl during the hour we spent skiing with her at the end of each day. The skills she had mastered were reinforced and the skills that still needed work were remediated (homework?) by her mom and me.

If students are using rubrics as an justification for doing the bare minimum of effort to get the desired mark, what does that say about the rigor of our curricula?

[scrivener]

Excellent response. Thanks.

I guess the way to look at a rubric is as a useful tool. But you need the right tool for the job, and you have to be careful to use the tool correctly and not to let the tool dictate the project. A hammer is a lovely instrument, but it won't staple a stack of papers, and if wielded a certain way, can be threatening and dangerous.

I like Javamomma's suggestion, too, that a rubric can remind a teacher of what it is we're looking for. My English partner and I used to sit down after papers were turned in but before we'd read any of them, and we're evaluate ourselves: What did we spend considerable time discussing in class? What did we make a point of emphasizing during peer-editing and during our mini-lessons on grammar? We would list the things and give each of them a certain weight, and that would basically be our rubric. As always, grades are negotiable anyway, so if a student looked at how he or she had been evaluated and disagreed with the number at the top of the page, we were happy to discuss the situation.

It might be fun someday to get a little collection here of the rubrics we've used.

[AAA Teacher Forum]

Another aspect I feel rubrics offer is the chance for students to see what they need to do to go "above and beyond." Our school district in Colorado uses a 4, 3, 2, 1 scale in the elementary schools...NO letter grades & percents...I LOVE it!! On our progress reports (report cards) are standards based.

A student who earns a "4" is consistently performing above grade level. Providing students with a rubric allows them to see what they need to do in order to reach those higher expectations.

Jennifer

[dolmansaxlil]

I use rubrics a lot for writing. We, too, are on the 4 point grading scale (though we have to transfer those four points to letter grades up to grade 6, then pegged percentages in grade 7/8, which is ridiculous, but that's another post). I introduce the rubric to the kids when I introduce the project. They know exactly what I'm looking for, and what is being marked on that particular writing piece. I'll also show them samples of a level 3 and a level 4. I love it because sometimes a kid will really struggle with proofreading/spelling/grammar, but the content is amazing, so I can give them a level 4 in one area but a level 1 or 2 in another. I record these seperately in my gradebook rather than giving an overall level for the whole piece. When I'm doing my report cards, I can look at where their low marks were, and make comments based on that.

[AndreaB]

The Kohn quote in Krafty's earlier post:

"The fatal flaw in this logic is revealed by a line of research in educational psychology showing that students whose attention is relentlessly focused on how well they’re doing often become less engaged with what they're doing."

seems to refer to the issue of performance goals vs. learning/mastery goals, which has become a prominent concept in educational psychology. Students with learning goals are focused on the task at hand, and want to master it, while students with performance goals focus on whether or not others (such as the teacher) think they're competent at a task (such as writing). Students with learning goals tend to be more engaged in the task itself than students with performance goals. Because rubrics emphasize what a good "performance" looks like, they might tend to encourage performance goals.
But there's another way of looking at it. Recent research suggests that the best students have both performance AND learning goals, so rubrics wouldn't interfere with that. And perhaps rubrics let students with learning goals understand when they are in fact mastering the task--they serve as a kind of feedback. The burden is then on teachers to make sure that rubrics represent, to the best possible extent, the richness of what a good performance on the task looks like.

[landreth2007]

Deep thoughts. Like everything else in education there is a place for rubrics. They are especially helpful for grading writing, and our district uses the 4, 3, 2, 1 method of scoring. The rubric helps keep me focused on specifics and keeps me from grading on my "feelings". Writing is difficult enough to grade.

[teacher333]

The only reason we use rubrics at the upper grade levels, esp. 5th, is for the parents - when they feel their child has "worked very hard" on a project and deserves an A and not a C, the rubric helps to show them where the project was lacking!

[EricDNA]

Rubrics should be shown to students before a task is started. Then they know what needs to be done. Once done, parts should be redone (iteration) til it passes the rubric. Points can be used and also words to indicate the level of success. Don't use alone, use maybe checklists or other kinds of assessment measures that show a student's growth. That is the point of using a rubric in the first place. Progressive learning.

[Boxcar]

I like rubrics for the upper grades. I think they help students see what is expected and can act as a checklist when the student is proofreading and whatnot.

Of course, this means giving out the rubric ahead of time, which I think is a must.

Rubrics also break down the grade, so the student can see exactly what went wrong.

That is my intial opinion. I'll go read the article now.

[mopar]

I can't say that I disagree with the article. By giving students the rubric, we do take the creativity out of their writing. We tell them what we are looking for. However, for many teachers that do not excel in the area of writing, it is important to tell the students what we are looking for, especially if we are to grade them.
If we could nix grading students writing, we could then stop giving rubrics. However, as long as I have to assign a grade to a paper, I believe that the fairest way to assess a student is to give them what I will be looking for. And I've found the rubric to be helpful in that aspect.
However, I do think that the rubric is leading us to give students what we want and thereby stifling creativity. Students will only produce what we ask, not go above and beyond that. Therefore, we are limiting our students. Not sure how to assess without limiting them because we still must assign grades to their work.

[Chef Dave]

Since I teach culinary arts, most of my grades are product oriented. If I didn't use a rubric to assess performance, my grade would be subjective.

As it is, the rubric helps break down the student product into various component areas:

1) Production of product
2) Product presentation and taste
3) Cleanliness of the work station area
4) Student hygiene and use of safe food handling techniques
5) Use of appropriate communication skills in creating the product and interacting with the chef instructor

[RibbityRibbit]

I work in a school that does not give grades, use rubrics, etc. A very "Alfie Kohn" sort of school, but I'm sure he'd find flaws in it as well... though we are mostly all devotee's.

In any case, I don't use a rubric, but I use a checklist for editing and revising. Those are skills which need to be reinforced and practiced. Capital letters, punctuation, spelling, indenting, neatness, etc. all need practice and there has to be some way of doing that. I can see a rubric would work well for conventions. In my class, they simply check off when they think they have done certain things. I look over it with them (conference) and double check. They also fill out a self-evaluation. I fill out a teacher evaluation.

At my last school, we used a rubric (NY ELA standards) and it really annoyed me that for ideas, the main thing they looked for was a beginning, middle and end and whether or not it stuck to the topic. When I wrote a comment about a child having a strong voice, on the paper, the principal made me re-do my comments because they did not relate specifically to the rubric. Things can be taken too far.

My students started a new writing project last week, and one child asked me for a grade (never had grades or report cards before.) This happens sometimes. They want that feedback. I am now trying to figure out how to grade their work for this one piece.. again, conventions and neatness are gradeable, ideas- not so much. How can you convey to a child that some ideas are strong, and their's might not be as strong? It's so objective. I don't want them to focus on getting a grade and not on the work itself. It will be a very interesting experiment for me.

There are definitely flaws in all systems... probably the best way to get feedback is by giving a group of people the writing piece, all using the rubric, then use their combined feelings on the piece to average a grade. My BA was in creative writing, and we would all read and comment on every student's writing. There was no rubric. We simply commented, then got together and critiqued. Ah-ha! I think I will do that with my kids!!

[mopar]

I like the idea of having the students critic. Then you can give a student improvements without grades. Nothing is perfect and students can always improve.
I think it would be nice not to have to grade students or assess their writing. However, most schools are not as fortunate.

[kingrichie]

I don't particularly care for them. They're especially difficult to calibrate. But, because scoring writing is so subjective, it's the closest scoring method we can use.

[gsslmt2]

kingrichie, I'm with you on this one, I'm in So. Cal. too, wonder if that has anything to do with it. Glad to see you here!

[busbus]

As the article rightly suggests, Rubrics do have flaws. However, in the long run, they are tools to help us to grade objectively. For this reason, I like using them. I also like to review the rubrics with my students because they should know what is expected in their writing. I do not think that creativity is stifled by the use of rubrics, unless Kohn is just criticizing only the standardized rubrics. To me, these are for scoring standardized tests.

Although I create my own rubrics for class assignments, I always leave room for my students' input. I won't say it's 50/50; but, I include some of what they want and sometimes adjust my weights to some degree if they can convince me with valid arguments. In this way, I think that the students feel that the rubric is as much theirs as mine for grading purposes.

In my opinion, using rubrics for grading purposes keeps subjectivity out of the equation.

[anyalee]

In an interview last year, the interviewers were crazy about rubrics. They asked me a question which they wanted me to magically come up with the answer "rubrics" for. Since I just graduated, I had rubrics beaten into my head. I assumed that everybody used rubrics for grading, but apparently not! Needless to say...I didn't get that job and looking back I'm happy because it had 5 preps! ICKY!

[Miss T]

I use rubrics to show my students step-by-step what they need to do to earn their points. That way, when they get points off, they can see where and why, and there's little to no room for argument.

[busbus]

I think that rubrics are great. They help to keep subjectivity out of the equation when grading. They help to bring students into the grading process so that there are no surprises on their part. In addition, oftentimes, teachers will allow students to help in creating some of the rubrics for projects.

Grading should not be a mystery for our students. They need to know just what is being required for project grades - from the high grade to the low grade. They also need to know the rubric for overall grading - how many points for each part that is used to determine the grade for the report card period.

[Chef Dave]

An English teacher at my high school recently told me that although she likes using rubrics, she has noticed a disturbing tendency among some students to ignore the assessment criterion so as to "earn" the highest possible marks.

What did she do about it?

She told her students that if there was more than a 10 point discrepancy between her score and their score, she would subtract 20 points from the final grade.

[Boxcar]

I don't have an issue with rubrics. I think it is good to know what a teacher is looking for.