Welcome to Training Session 5, Assessment and Evaluation. This section is on assessment versus evaluation. Educators use two distinct processes to help students build lifelong learning skills. These are assessment and evaluation. These two processes are complementary and necessary in education. This module draws important distinctions between assessment and evaluation, underscoring the need for both processes to occur at separate times and places, and ideally through different roles. At the end of this session, the participants will be able to, first, define the concepts assessment and evaluation, and second, identify situations where the use of assessment or evaluation is pertinent. In the last 15 years, much has been written about assessment and evaluation, but the terms have not always had distinct meanings. As accrediting agencies have become increasingly interested in improvement, it has become imperative to have a word that describes feedback for improvement that is distinct from one that describes the determination of quality. So let's start by defining assessment and evaluation. Assessment is a process of collecting, reviewing, and using data for the purpose of improvement of future performance. In the literature of the last several years, assessment has usually been used to indicate that at least some hint of improvement is expected in the assessment process. Evaluation, on the other hand, is described as an act of passing judgment on the basis of a set of standards. Evaluation is usually used to indicate that some sort of judgment of quality will be made. Improvement is a term used to look at how the level of quality of a performance or outcome could be improved in the future. It includes strengths that should be sustained as well as high priority areas for improvement. The assessment process is not concerned with the level of quality, only with how to improve the level of quality. Evaluation on the other hand is the term used to describe the determination of the level of quality. The evaluation process focuses only on the actual level of quality with no interest in why that level was attained. Assessment and evaluation both have their purposes and, when used correctly, both can add significant value to teaching and learning. The relationship between the people involved is different in the assessment and evaluation process. In both cases, the learner requests the process, whether it be an evaluation or assessment. The person that benefits in the assessment process is the learner, while in the evaluation process the locus of control rests with the observer or external stakeholders and decision makers. However, there can be detrimental effects when the people involved have not agreed whether the process is evaluation or assessment, or when the assessment methodology gets confused with the evaluation methodology. Although assessment and evaluation are used for different reasons, they do have some similar steps. Both involve specifying criteria to observe in a performance or outcome. Both require the collection of data and other evidence by observing the performance or by looking at the outcome or product. Both require a performer and a person who collects information about the performance. Both processes also conclude with the report of the findings, which include all the similarities and at least as many differences. In the assessment process, the report includes information about why the performance was as strong as it was and describes what could be done to improve future performances. In assessment, there is no mention of the actual quality of the performance, only how to make the next performance stronger. There is no language indicating the level of quality, such as good, terrible, or horrible. Conversely, in the evaluative report, only information regarding the actual quality of the performance is given. This might be in the form of a grade or a score or an evaluative comment, such as great work. The purpose of the evaluative report is to report the level of quality and possibly any consequences based on the determined level of quality. It is not used to suggest improvements in future performances. The chart that is shown clarifies the differences and similarities between the two processes. The case vignettes are examples of the use of the assessment process or evaluation process. This section addresses ways that evaluation and assessment can become confused. In the first example, the attending physician believes he is assessing, but the resident, who is the learner, perceives the feedback as evaluative because the learner has not worked with the attending physician to set up criteria and valuable feedback. The patient is recovering well from his PCI and he had a drug-eluting stent to his LAD. He's back on a normal diet, his vital signs look good, and I think he's ready to be discharged today. He didn't have any complications. Good. He does have a remote history of pulmonary embolism. Is there anything you want to do about that? Well, he's going to be on aspirin and Plavix, so shouldn't that be enough? Well, it turns out that in that circumstance, you probably do need to add a true anticoagulant as well. So we'll talk to pharmacy about what's the safest choice that we have for this patient. To prevent this confusion, both parties must determine performance criteria early in the process for which the learner needs to obtain feedback. Performance improvement will only be attained if the learner wants to apply the knowledge gained through feedback. In the second example, a supervisor serves as a mentor and provides assessment feedback as well as evaluative judgments. I've always found that adding color to your notes about medical cases really helps you make your studying more efficient. That's what I used to do when I was in medical school. Okay. That's great advice. Thank you. I actually always use Evernote to categorize and tag my notes, but I'm afraid that if I don't do what he wants to do, do you think maybe it's going to be reflected poorly on my evaluation? I don't know, but I think you should listen to his advice. Okay. It might be difficult for the learner to feel in control of the use and application of feedback as a supervisor will be evaluating performance at some point. This problem can be reduced by selecting different criteria for each case. Often, when one is used to giving feedback on the level of quality only, someone can feel uncomfortable giving critical feedback to a learner, feeling that pointing out areas to improve is the same as criticizing the performance. This can cause even more problems when the learner also perceives the feedback as evaluative as in the first case example. Analysis and building trust help the situation the most, but it can also help if the learner imagines what feedback he or she would have wanted if he or she had been the evaluator. It is important for the learner to send the message that he or she would like to have the feedback from someone that he or she trusts. In conclusion, this module is intended to strengthen outcomes from assessment and evaluation in teaching and learning situations. Assessment is a process used to improve a performance or outcome. Evaluation is a process used to determine the quality of a performance or outcome and to make decisions based on the quality. Both processes can be formative, undertaken while an educational process is ongoing, or summative, taken at the conclusion of an educational process. Before starting either assessment or evaluation, it is essential for instructors to clarify the purposes of the process. It is then critical to communicate this purpose to everyone involved and to establish whether this will be conducted as assessment or evaluation. Finally, one should be cautious when assessment is initiated without buy-in from the learner.

Assessment

Evaluation

Education

Data

Standards