“What we have described above is a set of predispositions and experiences with which teachers approach the task of constructing their knowledge of students.”

This quote made me think of my own practice with regards to evaluating students, and how the mark is in some ways a construct of my own knowledge of a student in relation to his or her mathematical knowledge. As a result, because I am always limited in the information I am privy to, that evaluation will always be lacking.

With regards to the first study mentioned in the Morgan and Watson article, I think it would be interesting and unnerving to have someone sit in the back of my class and observe the class on an ongoing basis – it would probably feel like teacher’s college all over again. I would imagine, like the teachers in the study, there would be behaviours that I would fail to notice. I would like to believe that I do not mix my assessment with my impressions of a student, but there are undoubtedly times that I likely do. I think a third party would be better able to observe when I do this. Related to this thought is the quote:

“On the contrary, the study suggests that informal assessments, including those that furnish information for summative assessments of performance, are inevitably and unavoidably influenced by a variety of factors that may have little to do with mathematical achievement.”

This illustrates the problem I have with informal assessment – how do I do it without being biased? I know that this is next to impossible and as a result I don’t think it is fair to count towards a grade. If need be, in parent teacher interviews or discussing marks, I would rather give my impression of what I think is going on and make it known that I only have a limited snapshot of the situation pertaining to a student’s work or achievement.

Using Research to Improve Large Scale Assessments

This weeks papers made me realise how easy it is to become complacent with one’s assessment practices. While I have worked very hard to make my math classes more accessible to students I don’t t really challenge my assessment practices, even after being part of a pilot for a reform assessment program (all subjects). I changed my practices, but don’t routinely question my assessment practices. I am fairly comfortable with other teachers watching my teaching, but I would be very uncomfortable with having others examine my assessment practices.

In assessing, I will now be looking for systematic biases (do I assume students with IEPs know less for the same written content?) and want to look at recorded biases against specific groups. Also I recently had a long conversation with a mother of smart boys who get discouraged by unclear evaluations. My own daughter (grade 11 functions) just got discouraged by a rubric marking rather than simply right or wrong. The rubric was bang on, but my daughter had truely never been graded on communication and thinking before.

Unlike some in the class, I didn’t see the articles as complete critiques of the EQAO and teacher assessment practices, rather a call for further improvements. There have been changes to EQAO since this report. I have noticed that recent EQAOs do not have questions broken into parts see question above or http://www.eqao.com/Educators/Secondary/09/BookletsandGuides.aspx?Lang=E&gr=09&yr=10

And the students get one mark that encompasses all categories. So I wonder if EQAO now has a clearer matrix as to which topics are covered to ensure more consistency to the curriculum and year to year.

Our board has been using moderated marking with grade 9 teachers for 3-4 years to help teachers prepare for the test. It has helped change teacher attitudes about what is important to teach. I think our students are seeing more rich problems and understanding what good answers look like (with flexibility). We have now extended it to all college and Workplace math teachers to push their assessments and learning activities from short calculations to richer problems. So even if the current EQAO isn’t promoting investigations it is promoting rich tasks.

I appreciated the Morgan & Watson article because it provided a vision of how assessment of investigations could occur- using moderated marking to develop more consistent assessment practices and then have teachers assess in class investigations with an consistent evaluation rubric and possible invigilation. The current Ontario “summative project” with little guidance and only an exemplar is not pushing more investigations in the classroom. If this is a important goal for the Ontario curriculum, Morgan & Watson provide some guidelines to creating an somewhat equitable assessment of investigations. As Suurtamm Lawson and Koch argue, assessments “defin[e] what is worth knowing, hence what is worth focusing on in classrooms” ( p.41).

Morgan & Watson: Assessment of students'...Issues of Equity

I approached the reading of this article with hopes of discovering information to help improve my assessment practices. I have felt, like others have mentioned, as if I will have had to explain my assessment/teaching practices one day since I have chosen not to “teach to the test”, but allow for creativity and logical understanding. This has left me on my own in my department and open to much comparison this year. Since reading the articles (Suurtamm as well) my conclusion is that I can only continue what I am doing, unless I find colleagues to do moderated marking with, and to keep a continual awareness that there are "possible alternative perceptions and interpretations" (p.102) and that there is not only one correct way of assessing my students’ work. If I am able to get the opportunity to dialogue and incorporate moderated marking with colleagues that would be an improvement, but until then I can only keep in mind, and possibly quote, “reliability is not a concept that can be applied simplistically to assessments of students’ mathematics. …tighter specification of criteria will lead to stereotyped responses from both teachers and students –in opposition to the value ascribed to creativity, openness and authenticity within ‘reform’ discourse” (p.103). With this in mind I can have improved confidence that what I am doing is the best I can do if I am going to continue my attempt at reform pedagogy.

Exercising professional judgment when it comes to assessing

As I was reading Devika's post this quotation stood out for me:

"Ultimately, without clear direction, teachers make their own decisions [...]" (Reys and Reys 1998, p. 237)

This quotation that Devika posted reminded me of a discussion we had a while back in this course about what it means to be a professional in the field of education. I remember Indigo pointing out that unlike doctors who ultimately have the same goal of saving a patient's life, the goals of teachers are often dissimilar because educators have different views on what content should be learned and how learning should take place. Part of this may lead to teachers doing amazing things in the classroom and encouraging students to become critical thinkers, while another part of this lack of consistency may lead to teachers acting on a misguided view of what professional judgment means. However, without a clear direction from the Ministry, teachers are truly left on their own to figure out what they should teach, how they should assess but at the same time be able to take responsibility (hence, the accountability piece) if ever a student, a parent, a colleague, or an administrator takes issue with how the teacher is doing his/her job.

My question is, how does this belief (that a teacher should exercise his/her professional judgment in what specific content/topics are taught and how assessments are carried out) align with practices like EQAO? It seems like a much easier and safer path to follow for teachers to "teach to the test" in the case of something like the EQAO if for no other reason than to not have to justify one's professional reasoning skills when, for example, an administrator or a parent asks why another teacher's class spent weeks preparing for the test while this teacher's class carried on with regular lessons not tailored to EQAO. Are there any ways to get around this issue, I wonder? Even if we had more holistically-assessed inquiry tasks as suggested in the Suurtaam (2008) article, would it just bring up more questions about teachers' professional judgment when it comes to assessing a piece of work without a clearly defined marking scheme?

Connections (in response to Praboda's post)

My comment in response to your post got too long so I decided to make a separate post:


I completely agree with you: we're walking in circles and having the same conversation, over and over again!

You brought up several key points in your post. There are two that specifically reminded me of some previous readings:

1."Although reform-oriented mathematics education seeks to de-emphasize procedural knowledge, we certainly cannot dismiss it entirely."

I agree and so do Hiebert et al. (1997).  They had this to say about teaching skills (procedural understanding) and conceptual understanding:

  

In spite of our belief that understanding and skills can and should develop together, we must make it clear that we assume the primary goal of mathematics instruction is conceptual understanding.  But we must also make it clear that setting conceptual understanding as the primary goal does not mean ignoring computation skills (p. 6).

2."If we are to ‘authentically’ promote the notion of interconnected-ness in mathematics, then can we have a curriculum that communicates a high degree of specificity in terms of individual expectations?"

Alan Schoenfeld (2002) referred to this exact point in terms of the Curriculum and Evaluation Standards for School Mathematics from NCTM in 1989:

..the Standards were long on direction and short on detail - it was a vision statement rather than a blueprint.  In hindsight, that was a good thing, for both political and intellectual reasons (p. 15)

He goes on to argue 'no set of curricular recommendations would have been politically viable" and that we would have missed out on 'significantly enriched curricular discourse" among the mathematical community.

Interestingly, Reys and Reys (1998) disagreed with Schoenfeld years earlier when they wrote this:

Ultimately, without clear direction, teachers make their own decisions on the basis of the mixed messages received from the collective forces of parents. fellow teachers, standardized assessments, curriculum materials, backgrounds that their students bring to the learning environment, and their own beliefs about how children learn (p. 237)

Surprisingly, NCTM did get specific by releasing Curriculum Focal Points:

NCTM's Curriculum Focal Points

Personally, I adopted a philosophy a while back to take my cues from my students.  They told me which parts on the curriculum to focus on.  My circle walking days are not nearly over though :-)

References:

Hiebert, L., Carpenter, T.P. & al. (1997) Making Sense: Teaching and Learning Mathematics with Understanding.  Portsmouth, NJ: Hieneman. Preface & Chapter 1, p. 1-15

Reys, B.J., & Reys, R.E. (1998) Computation in the Elementary Curriculum: Shifting the Emphasis.  Teaching Children Mathematics, 5(4), p. 236-241

Schoenfeld, A. (2002) Making Mathematics Work for All Children: Issues of Standards, Testing, and Equity.  Educational Researcher, 31(1), p. 13-25

The (Antithesis of Problem Solving in Large-Scale Assessment)^2

I was struck by some of the points mentioned in Suurtamm (et al) article regarding large-scale assessment. The authors did well to illustrate how large-scale assessments, such as the EQAO Grade 9 mathematics assessment, may ironically conflict and contradict the tenants of reform-oriented curriculum they were originally designed to support.

Reform-oriented mathematics encourage learners to develop a deeper understanding of mathematics and promote “problem solving approaches to teaching mathematics” rather than memorizing procedural knowledge (p. 32). Using the content strands and the achievement chart (process) categories from Ontario’s Grade 9 math curriculum, the EQAO assessment divides each of the multiple choice items (questions) and short answer items into one content strand and one process category. The extended response tasks, on the other hand, are filtered (or “mapped”) into 1-2 content strands. These tasks, which are designed to promote and demonstrate critical thinking, are broken down into scaffolded questions in order to cover all process categories (knowledge, application, problem solving and communication). Once this is accomplished, all EQAO assessment items are then paired with the specific curriculum expectation(s) from the Ontario Ministry documents.

As the authors have pointed out, the structure for developing the Grade 9 EQAO assessment does not maintain the integrity of reform mathematics. By breaking down extended response tasks into smaller, scaffolded questions, the authors argue that the EQAO assessment “artificially isolates the processes” and once again, “problems are presented as a series of smaller steps”. Ironically, the EQAO assessment seems to value procedural knowledge and certain algorithms of problem solving. Also, the authors also note that assigning EQAO items with process categories can be extremely ambiguous and subjective as these categories are “neither easily separated, distinct, nor distinguishable” (p. 38). This lead to several inconsistencies and once again, EQAO assessment structures imply that these process components can be compartmentalized and segregated into mathematical tasks. Suurtamm et al. also attack the Grade 9 Ontario mathematics curriculum stating that it “gives no clear indication of the important ideas that should be focused on and hence assessed” (p. 41). Ultimately, a curriculum that perpetuates an “insufficient degree of specificity” leaves large-scale assessment incomplete.

After reading this article, I couldn’t help but wonder what exactly is so wrong about problem-solving tasks being broken up and scaffolded into smaller questions. Although reform-oriented mathematics education seeks to de-emphasize procedural knowledge, we certainly cannot dismiss it entirely. (After all, we’re still using “process” categories to measure and map assessment items that seek to shift attention away from procedural tasks – hmm). Considering elements of universal design, I think all students, regardless of mathematics ability, can benefit from scaffolded questions (like those found on the EQAO assessment) during problem solving tasks. Of course, many students may have different methods to solve a task and scaffolded questions may be somewhat limiting, but I don’t think this format would completely discourage a learner’s thought process. If we want to engage students into thinking in “multiple” ways and through different mediums, then perhaps we can consider these scaffolded questions as an example of “a way of thought”.

To add more complexity, I thought there was (dare I say it) a potential paradox with the arguments that the author(s) brought. Suurtamm et al. pointed out the lack of specificity with the curriculum but at the same time, they also value the broad and interconnected strands of mathematical problem solving and inquiry. If we are to ‘authentically’ promote the notion of interconnected-ness in mathematics, then can we have a curriculum that communicates a high degree of specificity in terms of individual expectations? A curriculum that is highly specified may also come across to many educators & stakeholders as being prescriptive and narrowing as well. Also, with the subjectivities that the authors pointed out in their article, I would also wonder who exactly gets to design and choose expectations for a focused curriculum? Again, I feel like we are walking in circles.

Large scale assessments are used to measure student achievement but they continue to provoke intense debate. Educators, teachers and parents are increasingly looking into the purposes of these tests such as student’s learning, teacher’s goals and accountability of education system. The concept of large scale assessment is not new, it has been a norm in many countries for a long time. I have taken these tests myself in my home country but I guess reform mathematics is changing what was being initially expected from these large scale assessments.

It is important to point out that Canadian teachers are responsible for the development and grading of provincial and territorial large-scale criterion- referenced assessments. The national SAIP instruments are also developed and graded by classroom teachers. These activities are coordinated with respective ministries or departments of education. This is in stark contrast to the United States, where large-scale assessments are typically norm-referenced and are run primarily by commercial organizations outside of the education system. This distinction is important and may account for the fact that Canadian testing programs tend to account for greater linkages with classroom practice than their American counterparts (Gambell & Hunter, 2004).

The reliance on criterion- referenced testing also suggests that Canadian testing programs tend to be more aligned with mandated curricula, and as our Ontario curriculum prompts to take an investigative approach to learning mathematics then its not fair with the students and teachers that our assessments do not go far enough in addressing investigative component.

The article of Suurtaam (2008) states that problems in EQAO are the scaffolded version with multiple steps and suggests that problems should be rich and open ended, but I was wondering, are the inherent issues of grading such open ended questions and the excessive time that might be involved in solving such problems, the restraining factors keeping us from achieving this goal?