Using Rating Scales in Your Practice

CCPR: Dr. Collett, you have written many reviews of rating scales. How are rating scales useful for clinicians?

Dr. Collett: For one, clinicians can use rating scales to make an initial assessment and document that a child really has a problem relative to other kids his or her age. For instance, it is important to know that the level of hyperactivity that a preschooler in your office is showing is different than what we would expect preschoolers to show. it is really striking how often a perfectly normal preschooler refuses, and argues, and demonstrates a lot of behaviors that could be considered part of the pathology of adhd. rating scales provide a big normative sample that you can use for comparison, allowing clinicians to say more definitively that a behavior is different than what you would expect from another kid the same age.

CCPR: So a rating scale can answer the question, “Is this normal compared to other kids of this age group or developmental stage?”

Dr. Collett: a rating scale gives you data to back up your clinical judgment. it also gives you an efficient system to gather data from multiple people, such as teachers, the parents, and the children themselves. With a questionnaire, they all respond to the same questions. it gives you information from different people’s perspectives and reports behavior across different settings, and it is usually more efficient to send out a rating scale rather than make phone calls to different people to get that information.

CCPR: How do you think rating scales should be used? Should clinicians ask all new patients and their families to fill them out? Or should they use ratings scales before diagnosis, after diagnosis, and after treatment to track progress?

Dr. Collett: There are two categories of rating scales: broad- and narrow-band.Broad-band rating scales, such as the child Behavior checklist and Basc [Behaviorassessment system for children], can help you assess a wide range of presentingproblems at the outset. it makes sense to get comfortable using one of those scales toget a thorough patient evaluation. Narrow-band scales are used to assess more specificproblems. For instance, clinicians can use Masc (Multidimensional anxiety scalechildren) to assess anxiety or some of the conners’ rating scale short forms that aretailored to assess adhd. clinicians can use these scales for a more refined assessmentor even for repeated assessment—before and after you started treatment. The scalescan help a psychiatrist monitor the effect of a medication or some other treatment.

CCPR: What factors should a clinician consider in choosing a rating scale?

Dr. Collett: you want to think through what the function is going to be for this test.is it going to be an initial part of your diagnostic assessment or is it going to be a part of your ongoing monitoring? if it is part of the initial assessment, it is okay to use a slightly longer rating scale because respondents will only fill it out once. you really wantit to have good coverage and you want it to distinguish clinical from nonclinical groups. if you are choosing a scale for ongoing assessment, length matters a lot; you don’t want parents to have to sit and fill out a 200-item scale every time they come to see you. you want them to be able to do a more focused assessment and you want it to be sensitive to treatment gains.

CCPR: Sometimes you can have tests that are sensitive to treatment but not necessarily good diagnostic instruments. How does this apply to ratings scales?

Dr. Collett: again, you need to consider what you are using the scale for. obviously, if you are using the scale to justify your initial diagnosis you want really good sensitivity in terms of distinguishing clinical groups from nonclinical groups. if you are using a scale to document treatment effect, then you really want to know that it has sensitivity to detect a change.

CCPR: The Individuals with Disabilities Education Act (IDEA) requires quantifiable testing for diagnosing a condition that impairs education. Do rating scales count when it comes to producing quantifiable results?

Dr. Collett: They do. rating scale results would be a very reasonable source of data for that kind of decision. in fact, that is one of the tools that school systems use internally when they are doing their eligibility evaluations.

CCPR: How do we know that rating scales are accurate?

Dr. Collett: in general, the manual for a good rating scale provides information about its reliability, such as how consistentthe scores are over time and across different raters. however, sometimes rating scales get disseminated without having a lot ofthat data. The next step in developing a scale is to collect validation data showing that kids who have a diagnosis differ in some predictable way from kids who don’t have a diagnosis. Finding validation data sometimes takes more digging, and is usually harder for test developers to establish. some tests might actually not have as much validation data as you would want.

CCPR: Is that worth looking for?

Dr. Collett: definitely. you may need to do a literature review or actually look at a manual for the scale that you are going touse. Then the other kind of validity data would be sensitivity—for example, knowing that it is sensitive to a treatment effect or sensitive to other variations over time. you want to know that if your treatment produced a change in a child that you should see a corresponding change in the rating scale.

CCPR: What are some particular problems using rating scales with children and adolescents?

Dr. Collett: parents will often say, “Well, i could make my child ‘look’ however i wanted to on this rating scale. if i really wanted him to have adhd i could endorse all of the twos, and if i didn’t, i could endorse all of the zeroes,” or a teacher could do the same if they have their own agenda.

CCPR: So how is the best way to manage that?

Dr. Collett: it is absolutely true that people could skew findings on a rating scale in one way or another, and it is worth thinking about some of the variables that might affect responding. For example, sometimes a parent’s or teacher’s level of distress or concern can influence ratings. so you want to have multiple different reporters, then look at the differences among them, and think through how this might all make sense. it is also important not to make a diagnosis just because a child scores high on a rating scale. of course, you want to consider how the score compares to your clinical judgment about the patient and what you are seeing in the office.

CCPR: Do rating scales present different problems for young children than they do for older kids?

Dr. Collett: a child under age five or six really cannot provide a great self-report on a rating scale. you will need to rely mostly on reports from the parents and teachers for kids this young.

CCPR: When we are dealing with teenagers, do you think that the patient’s self-report should have more weight than the other individuals’ reports?

Dr. Collett: Not necessarily. Kids tend to underreport externalizing (behavior) problems overall. so if you are asking teens about the deviant things that they do, they tend overall to report less than their parents and their teachers do, which isn’t so surprising, really. on the flip side, parents and teachers tend to underreport internalizing problems and subjective distress like anxiety or depression. Which one has more impact on the treatment depends on some of the other presenting facts that you know about the teen.

CCPR: Do you think that there are any inherent risks in using rating scales?

Dr. Collett: one risk is assuming that just because something comes out of a scoring program or algorithm that it is “the truth.” That is dangerous because it should be viewed as just one source of data, weighed against all of the other data that you have about a child and your clinical judgments and impressions. all of those things don’t go out the window just because you have this one instrument.

CCPR: How would you recommend reconciling clinical judgment with a rating scale?

Dr. Collett: in private practice a psychiatrist can keep the mental status exam and their own clinical impressions in the room and then compare that to what they get from scale and think about what the differences are. Keep in mind that a child’s behavior can vary in different settings. in my work with very young children under five years old, they often are fine when they come to my office. however, i do believe their parents’ reports of tantrums at home and other behavior problems. They just don’t show up in our setting because it is novel and the children are usually getting a fair amount of attention. it is not that the rating scales are right and i’m wrong; it is just that behavior varies across different settings. it is important to use all of the data that you get.

CCPR: How should a clinician go about choosing a rating scale?

Dr. Collett: reviewing the literature is a good start. see what has been reported in the peer-reviewed research and look for a rating scale that suits your needs. so if you are working mostly with inner city kids in chicago, you want to be sure that the rating scale reflects that kind of population, the “normative group” as it is called. The group that the scale was developed with should reflect the population that you are working with.

CCPR: So it should be as close as possible to your own patient group?

Dr. Collett: yes, and that can be hard sometimes for people in unique settings. in general it should be close or you should atleast know how it differs from the population that you work with, which you can find out from the test manual or from published research.

CCPR: In practical terms, some of the scales are free and others are decidedly not. Some require complicated scoring and some do not. Is there an easy way to sort those out?

Dr. Collett: There are some good free scales. For example, the Vanderbilt scales for adhd have a really good set of evidence behind them. They have norms and good data on reliability and validity. There are other scales you can find free online that still have pretty good data. sometimes it takes a little more investigating to be sure that the one you choose does have that kind of data. published (ie, those you have to pay for) instruments usually have at least reasonably good norms and evidence of reliability. They won’t always have as much validity data as we would like, but a clinician who uses a published scale might not have to dig as they would for some of the free scales to get that information.

CCPR: Thank you, Dr. Collett.

Child Psychiatry