I also just posted a brief research review of this too. Corbin et al usefully clarify the problem ... but a deeper wicked problem remains: how to train educators to apply the "permissions" in practice, i.e., to help them design assessments that balance integrity and authenticity, and to empower them to determine degree of validity rather than defaulting to policing.
The wicked problem inside the wicked problem:
• Layer 1: AI destabilizes assessment itself.
• Layer 2: Even if we accept the “3 permissions” (compromise, diverge, iterate), there is still the unsolved challenge of training educators to use them skillfully.
Formulated more clearly, the wicked problem of "assessment training" is how can educators be prepared to:
1. Exercise the three permissions confidently in real settings (compromise, diverge, iterate),
2. Design assessments that balance workload, integrity, and authenticity, and
3. Judge and maintain validity rather than defaulting to policing?
Not common knowledge, and not an easy task.
Solving the first-order wicked problem of AI and assessment will require solving the second-order wicked problem of educator capability.
I agree. I am increasingly seeing teacher training as the major bottleneck. I am working on some different strategies right now. Teachers need both general knowledge and disciplinary specific applications. That is my general framework. The disciplinary specific work could be the space where teachers feel that they have an established knowledge base thus open up space for more authentic engagement with the permission.
I think, humanity should understand that we entered into a new era where you need to entirely transform interms of understanding the new ground rules, what you felt was complex and intelligent is no more called complex and intelligent. Initially we got a help from industries and machinaries made it possible which are physically impossible by a human kind and we are into an era where AI gonna make it possible which are mentally impossible by a human kind. So, clean your slate, rewrite, redefine everything from the beginning and better use of the available resources and focus on what else mind state instead of thinking about the one you are going to loose sooner.
What if AI isn’t the source but the reveal of the wicked problem? What if the wicked problem has been with us all along? What if the wicked problem isn’t the bot by the concept of assessment as it has fossilized in a university model that hasn’t faced the ideological epistemic power asymmetry in “assessment” imposed on learning since the Middle Ages? “I don’t know” means “I can see no other model of assessment than endless tinkering with the normal curve and the isolated individual and the teacher’s authoritative presence? There are other ways to and purposes for assessment. Having been a university assessment coordinator working with professors from seven different disciplinary colleges, I saw the whole construct of assessment as a wicked, wicked problem. The hand wringing is old and stale and it’s not for AI but for revisioning what assessment can do.
I also just posted a brief research review of this too. Corbin et al usefully clarify the problem ... but a deeper wicked problem remains: how to train educators to apply the "permissions" in practice, i.e., to help them design assessments that balance integrity and authenticity, and to empower them to determine degree of validity rather than defaulting to policing.
The wicked problem inside the wicked problem:
• Layer 1: AI destabilizes assessment itself.
• Layer 2: Even if we accept the “3 permissions” (compromise, diverge, iterate), there is still the unsolved challenge of training educators to use them skillfully.
Formulated more clearly, the wicked problem of "assessment training" is how can educators be prepared to:
1. Exercise the three permissions confidently in real settings (compromise, diverge, iterate),
2. Design assessments that balance workload, integrity, and authenticity, and
3. Judge and maintain validity rather than defaulting to policing?
Not common knowledge, and not an easy task.
Solving the first-order wicked problem of AI and assessment will require solving the second-order wicked problem of educator capability.
I agree. I am increasingly seeing teacher training as the major bottleneck. I am working on some different strategies right now. Teachers need both general knowledge and disciplinary specific applications. That is my general framework. The disciplinary specific work could be the space where teachers feel that they have an established knowledge base thus open up space for more authentic engagement with the permission.
I think, humanity should understand that we entered into a new era where you need to entirely transform interms of understanding the new ground rules, what you felt was complex and intelligent is no more called complex and intelligent. Initially we got a help from industries and machinaries made it possible which are physically impossible by a human kind and we are into an era where AI gonna make it possible which are mentally impossible by a human kind. So, clean your slate, rewrite, redefine everything from the beginning and better use of the available resources and focus on what else mind state instead of thinking about the one you are going to loose sooner.
There can be many wicked problems is my sense of the ontology.
What if AI isn’t the source but the reveal of the wicked problem? What if the wicked problem has been with us all along? What if the wicked problem isn’t the bot by the concept of assessment as it has fossilized in a university model that hasn’t faced the ideological epistemic power asymmetry in “assessment” imposed on learning since the Middle Ages? “I don’t know” means “I can see no other model of assessment than endless tinkering with the normal curve and the isolated individual and the teacher’s authoritative presence? There are other ways to and purposes for assessment. Having been a university assessment coordinator working with professors from seven different disciplinary colleges, I saw the whole construct of assessment as a wicked, wicked problem. The hand wringing is old and stale and it’s not for AI but for revisioning what assessment can do.