How should we test our kids?
Every year kids take a long list of standardized tests to measure learning. This takes significant time and energy away from other teaching and learning opportunities and gives an incomplete picture of student performance. To get good educational outcomes, we need to balance the trade-offs. The Meta-Problem Method can help.
Complex problems are often vague and have many possible solutions. The Meta-Problem Method may lead you far away from the dilemma that started your quest. That’s because the method forces you to clarify what you really want and what you are willing to give up. It enables you to compare objectively the possible pathways and their trade offs. It prevents you locking into solutions mode too early and then doubling down on solving a low-yield problem that does not serve your goals as well as the alternatives. At the end of this process, you will have a better understanding of your priorities and how to achieve them.
Steps in the Meta-Problem Method
Dilemma
The high-level issue you are trying to address
Accurately assess the effectiveness of education for every student.
Goal
The changes you want to make to address the dilemma. There are usually many options.
Supporting Goals
- Administrators can objectively know how students are doing.
- Students get maximum learning.
Other goals could include students enjoying school, students developing fluency with the concepts, and minimizing the cost of delivering education.
Problem Space
The set of problems you could chose to solve to advance your goals, plus the constraints that hold you back.
Example problems
- How can administrators objectively know how students are doing? Maybe the problem to solve is “What assessments are immune to teacher and school bias or manipulation?”
- How can students learning be maximized? Maybe the problem to solve is “How can teachers identify learning gaps for each student?”
There are many other potential problems to solve related to assessing student learning. Each goal has many possible problems we could link to it. Are there other problems linked to these first two goals? Which options come to mind for the other goals?
High-Yield Problems
Sometimes solving one problem helps make progress towards several goals. In this step, we identify these “two-for-the-price-of-one” problems.
Which options will advance more than one goal?
- Comprehensive standardized testing provides an objective measure of student achievement on the tests, and costs little to deliver across all students. However, these tests short-change many students, time spent on tests takes away from learning, and the testing process can put substantial strain on student engagement.
- Alternative methods like performance-based assessments allow more students to demonstrate their knowledge accurately, can help students enjoy school more, improve fluency, and improve student learning. However, there is more risk of variability across teachers and schools, and grading the assessments may add additional work to already busy teachers.
There are many potential solutions that will have varying effects on the set of goals. Which alternatives improve the most important goals? How might the unknown change the right path forward? What other possible solutions are there to address the dilemma?
Problem Selection
Which of the many possible options in the high-yield problem step is the best set to address the dilemma?
- Which solutions make the most sense to upgrade education?
- Which solutions will best address the dilemma?
- Which solutions will deliver the best outcome for the least amount of time, effort and money?
Implement, Learn and Adapt
Check continuously that you are still solving the best problem, as new information emerges.
Observe and learn as you go. As new information reveals itself, check continuously that you’re still solving the right problem.