{"id":1886,"date":"2021-05-01T04:44:04","date_gmt":"2021-05-01T09:44:04","guid":{"rendered":"https:\/\/curriculumsolutions.net\/?p=1886"},"modified":"2021-03-31T13:46:18","modified_gmt":"2021-03-31T18:46:18","slug":"the-problem-with-proficient","status":"publish","type":"post","link":"https:\/\/curriculumsolutions.net\/blog\/2021\/05\/01\/the-problem-with-proficient\/","title":{"rendered":"The Problem with &#8216;Proficient&#8217;"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"alignright wp-image-1887\" src=\"https:\/\/curriculumsolutions.net\/wp-content\/uploads\/testing0306.jpg\" alt=\"\" width=\"425\" height=\"283\" \/><\/p>\n<p>Back in 2016, <a href=\"https:\/\/www.edweek.org\/teaching-learning\/what-does-proficient-on-the-naep-test-really-mean\/2016\/06#:~:text=Here's%20how%20the%20National%20Center,situations%2C%20and%20analytical%20skills%20appropriate\">there was a little kerfuffle about how the term \u2018Proficient\u2019 as a measure of student ability on standardized tests should be interpreted.<\/a>\u00a0 In a nutshell, the education news site <em>The 74<\/em> noted that \u201c2 out of 3 8th graders in this country cannot read or do math at grade level,&#8221; an assertion based on the 2016 NAEP scores for 8th graders and the percentage scoring &#8216;Proficient.&#8217;\u00a0 That sparked some argument from the Brookings Institute, which tweeted\u00a0 the NAEP definition of proficiency, which they felt <em>The 74<\/em> had misinterpreted. You can click the link above for all the back and forth.<\/p>\n<p>But <em>The 74<\/em> should perhaps be excused for its interpretation of NAEP scores because &#8216;Proficient&#8217; means different things on different instruments and even different things on the <em>same<\/em> instrument.\u00a0 Check this out:<\/p>\n<ul>\n<li>NAEP definition of &#8216;Proficient&#8217;: \u201cStudents have demonstrated competence over challenging subject matter.\u201d According to the NAEP, this is not the same as working at grade level, BUT they qualify \u2018Basic&#8217; (the level below \u2018Proficient\u2019) as \u201cpartial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade.\u201d Sooooo\u2026.\u2019Basic\u2019 is partial mastery of knowledge required for grade level work but \u2018Proficient\u2019 is not grade level work\u2026.What now?<\/li>\n<li>On the <a href=\"https:\/\/itp.education.uiowa.edu\/ia\/documents\/Standard_Score_Ranges_for_Achievement_Levels.pdf\">Iowa Assessment<\/a>, \u2018Proficiency\u2019 is measured by cut scores, but the relationship to the student\u2019s actual number of correct answers is not explicit. The band for \u2018Proficient\u2019 is by far the largest band and when placed in context with \u2018Not Proficient\u2019 and \u2018Advanced\u2019 strongly resembles a bell curve. Since this is a normed test, we can probably take this to mean that \u2018Proficient\u2019 is going to encompass 50% of all the students taking the test and is therefore something of a moving target from year to year. Some years, proficiency might be lower in terms of actual correct answers than in others.<\/li>\n<li>The <a href=\"https:\/\/tea.texas.gov\/sites\/default\/files\/STAAR_Performance_Labels_and_Policy_Definitions.pdf\">STAAR test<\/a> (Texas) doesn\u2019t use \u2018Proficient,\u2019 but rather uses &#8216;Meets Grade Level.\u2019 This would appear to be a clearer measure of a child\u2019s ability but the actual definition for this is: \u201c\u2026students have a high likelihood of success in the next grade or course but may still need some short-term, targeted academic intervention. Students in this category generally demonstrate the ability to think critically and apply the assessed knowledge and skills in familiar contexts.\u201d This seems a little bizarre \u2013 short term intervention would imply some degree of shakiness on the part of the student with regard to concepts and content. It must also be remembered that the STAAR is a test which requires little in the way of higher order thinking so the bar here is already fairly low. So \u2018Meets Grade Level\u2019 does not mean a kid is 100% ready for the next grade level, even though that is what it implies.<\/li>\n<li><a href=\"https:\/\/www.nj.gov\/education\/archive\/assessment\/parcc\/scores\/Spring16ScoreReportInterpretationGuide.pdf\">PARCC <\/a>also uses different designations. &#8216;Met Expectations&#8217; means that students &#8220;have demonstrated readiness for the next grade level\/course and, ultimately, are on track for college and careers.&#8221; Like the Iowa Assessment, though, the designations are represented by cut scores that don&#8217;t clearly relate to the actual number of correct answers.\u00a0 The level below &#8216;Met Expectations&#8217; &#8212; &#8216;Approached Expectations&#8217; &#8212; is defined as &#8220;likely need[ing] academic support to engage successfully in further studies in the subclaim content area&#8221; which sounds a lot like the STAAR&#8217;s description of &#8216;Meets Grade Level.&#8217;<\/li>\n<li>And of course, now that many states have jumped ship on national tests like PARCC, many individual states have their own tests and their own designations for &#8216;Proficient.&#8217;<\/li>\n<\/ul>\n<p>It&#8217;s a <em>situation.<\/em><\/p>\n<p>None of this even addresses the larger issue which is the alignment of such tests &#8212; <em>any<\/em> such tests &#8212; with what&#8217;s being taught in classrooms.\u00a0 Kids may be getting a fine education, but if it doesn&#8217;t align with the content or contexts of the test, they may not perform well and look less competent than they really are.\u00a0 Or the test may be so poorly aligned that what it really measures is socio-economic status and the teachers look like rock stars even though they&#8217;re not teaching anything on the test.\u00a0 Yet these &#8216;performance indicators&#8217; get picked up by states and the media and applied to measures of supposed competence.\u00a0 Does this mean that competence is underestimated? Over-estimated?\u00a0 I don&#8217;t think we can really know for sure because the answer to that is going to be highly dependent on the individual districts in question &#8212; even individual schools and classrooms if there&#8217;s not good coordination across grade levels and teachers with regard to objectives and expectations for mastery. What, out of curiosity, does &#8220;challenging subject matter&#8221; (NAEP) actually entail?\u00a0 That&#8217;s an entirely different definition that we&#8217;d need to fully unpack for clarity.<\/p>\n<p>Here are my three takeaways from this:<\/p>\n<ol>\n<li>Be aware of these semantic issues and the underlying confusion and lack of consistency they\u00a0 embody.\u00a0 Take test results with a least a small grain of salt and remember that they don&#8217;t in any meaningful way measure the total humanity, dignity, personality, and potential of your students. They are not the last word.<\/li>\n<li>Give students time to practice the content that will be on the test in the same ways (contexts) it appears on the test.\u00a0 And by time, I mean 9 months or so.\u00a0 Now, before everyone gets all wound up about &#8220;teaching to the test&#8221; and how it&#8217;s destroying American education, the third takeaway is this:<\/li>\n<li>Instruction in the classroom must go <em>beyond<\/em> the test in content, context, and cognitive demand.\u00a0 Students taught to &#8220;go high&#8221; &#8212; meaning, studying concepts at depth and using higher-order thinking and more complex and engaging contexts that they can apply in hands-on, real-world situations &#8212; will always be able to &#8220;go low&#8221; &#8212; meaning, do well on a typical, low level multiple choice instrument.\u00a0 <em>Do not make the mistake of making the floor your ceiling.<\/em> Investing in a lot of pre-packaged test prep material is not a good idea and is not going to produce the kind of flexible, critical thinkers capable of complex problem solving that virtually every district says is their end-goal for students.<\/li>\n<\/ol>\n<p>CMSi can help your district develop high quality curriculum that both aligns to state standards and prioritizes and refines those standards for local use.\u00a0 \u00a0Our <a href=\"https:\/\/curriculumsolutions.net\/trainings\/district-and-school-solutions\/\">Curriculum Writing<\/a> Training walks you through the process of curriculum development in a way that is designed to promote teacher buy-in.\u00a0 Your end result will be a curriculum that promotes higher order thinking and hands-on, real-world activity for students and supports good pedagogy for your teachers so that students are getting the best possible delivery of the content available.\u00a0 <a href=\"https:\/\/curriculumsolutions.net\/contact-us\/\">Contact us<\/a> for information &#8211; we would love to help your district start building a high quality curriculum today!<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Back in 2016, there was a little kerfuffle about how the term \u2018Proficient\u2019 as a measure of student ability on standardized tests should be interpreted.\u00a0 In a nutshell, the education news site The 74 noted that \u201c2 out of 3 8th graders in this country cannot read or do math at grade level,&#8221; an assertion [&hellip;]<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/posts\/1886"}],"collection":[{"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/comments?post=1886"}],"version-history":[{"count":3,"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/posts\/1886\/revisions"}],"predecessor-version":[{"id":1902,"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/posts\/1886\/revisions\/1902"}],"wp:attachment":[{"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/media?parent=1886"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/categories?post=1886"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/curriculumsolutions.net\/wp-json\/wp\/v2\/tags?post=1886"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}