Schema-based assessment

Schema-based assessment has the same focus as schema-based instruction but a different goal. The goal of the instruction is to facilitate the creation of schemas by students who experience the instruction. The goal of the assessment is to examine the extent to which learners have developed and can use the schemas that guided instruction. Schema-based assessment requires us to specify a priori the structure of the schemas to be assessed so that items evaluating those schemas can be constructed. In assessment, to a much greater extent than in instruction, the structure of the schema itself becomes a crucial factor because it influences the way that the test items are developed and interpreted. 

Schema assessment presents some difficulties that do not arise in other types of assessment. A central one stems from the fact that schemas are highly individualized. Schemas are created and tailored by individuals to reflect their own experiences and understandings of the world. No two individuals will form identical schemas because no two individuals can experience the world identically, but both of them may have well-formed and useful schemas. The assessment question is How can we obtain a true snapshot of schema development for these different individuals  

Consider for a moment a case in which individualization is not a factor, namely, the acquisition of a particular fact by several individuals as a result of some instruction. There is no ambiguity whatsoever in what these individuals know They either have memorized the fact or they have not. Suppose the fact of interest is 

The impact of individualization is change in the scope of assessment, making it much broader. Whereas an individual may not have a specific piece of information stored in memory as part of a schema, that individual may nonetheless have sufficient schema knowledge to use the schema well. A test of a specific piece of information will not be a fair test of schema development, unless 

Students know full well that the above-mentioned premises underlie most of the tests that they take. We have only to look at their study habits for confirmation. They memorize dates, figures, isolated facts, lists, and as many trivial details as they can. They expect to be quizzed on these things. They do not expect to be asked to demonstrate their reasoning, their ability to apply their understanding in a meaningful way to a new situation, or to create anything. In fact, they may become aggrieved when asked to do so, because it disrupts their expectations (Marshall, in press). Have we not all heard complaints that some tests were unfair because students were not told in advance that they would be asked a particular type of question Tests have become routine look-up or recall exercises. Students memorize some material and 

---

Instruction and learning from instruction have been the topics of the previous two parts. Now, the focus shifts to issues of assessment. To some degree, the ideas offered here have already been introduced, because it has been impossible to discuss the evaluation of learning from schema-based assessment without delving into some of the assessment questions and techniques. These issues are elaborated more fully here. 

The two chapters of part IV describe the theory and practice of schema-based assessment. Chapter 10 looks at the theoretical issues that accompany schema-based assessment. Chapter 11 describes assessment as it is performed in SPS and PSE, pointing out how the theory can be put into practice and examining some of the unique features of the evaluation in these systems. 

Therefore, as the above paragraphs suggest, schema-based assessment requires a reconsideration of the basic principles behind test construction and use. The issues just mentioned outline some of the differences between schema-based tests and traditional tests. The remainder of this chapter examines some of the major implications of schema theory for test development and test interpretation. Some of the implications are far-reaching and must await new methodologies. Others are candidates for immediate implementation. 

The most obvious consequences of schema-based assessment show up in the content of test items. Innovative items are needed for the simple reason that traditional items will not suffice. Different questions are to be posed different answers are to be sought. The innovations may derive from a modification in the nature of instruction, or they may result from a shift in test objectives. Both of these reasons are operative when we consider schema-based tests. 

Perhaps the single most important outcome of schema-based assessment is that it leads to a reexamination of the test s place in the instructional process and subsequently to a new interpretation of test results. At present, it is no secret that many educators feel that tests drive the curriculum and that teachers feel compelled to teach to the test (Shepard, 1991 M. L. Smith, 1991). The tests... 

An approach that may be well suited to interpreting schema-based assessment is to use expert knowledge found in expert s responses to test items and to develop a generalized scoring scheme from them. One would have the opportunity here to see how experts use schema knowledge and to look for similar implementations by students. Exact specification of schemas would seem to be critical for this approach. 

Schema-based assessment puts the what and the why of assessment first, and it does so by supplying an integrated picture of memory, learning, and assessment. The schema comprises multiple knowledge components, each made up potentially of many different pieces of information. Any test item for a schema will call for some subset of one or more of these knowledge components. The task of the test developer is to create items to test various subsets and thereby estimate efficiently the completeness of an individual s schema knowledge.

---