04 July 2014

The Education Funnel

About the difficulty of national tests, and pass rates.

In Romania, children aged 14–15 went through national tests. There was an outcry that the test on language and literature was too hard. The pass rate was $71\%$. Last year, there was an outcry that the same test was too easy. The pass rate was $76\%$. It is very difficult to believe that both outcries are justified.

I would like to discuss two issues that are seldom made explicit:

  1. Why do we classify children into two categories, pass and fail?
  2. If the pass rate is low, does it mean the test was too hard?

My comments apply not just to Romania, but in general.

Pass and Fail. We kick children out of the education system for economic reasons. To put it bluntly, economy would collapse if everybody would be in school. Somebody needs to be out of school, doing work. To ensure this, we could organize the education system in different ways.

One way it to stop education for everyone at the same time.

Another way is to educate only an elite, until death.

Yet another way is the funnel.

All of these have downsides. The age related system denies education to those who want more. In the elite system, most people don't even know what they are missing. Finally, in the funnel system, people are regularly kicked out. The funnel system is what we have, and is the best of the three possibilities.

Yes, being kicked out is annoying, even if you wanted to leave anyway. Those who fail are often those who dislike school and want to leave. But, they are still annoyed by being formally labelled ‘failures’. Who wouldn't?

As I see it, the villain here is not the funnel system. The villain is the label ‘fail’.

Pass Rate and Difficulty. Often, ‘pass’ is defined to mean that the grade exceeds a threshold. For example, Romania's baccalaureate is passed if the grade is at least 6. Such a definition links the pass rate to the difficulty of the test: The more difficult the test, the lower the pass rate.

Which should come first: the pass rate or the difficulty? The conventional view is that the process works as follows: (1) choose questions of the right difficulty, (2) administer the test, and (3) measure the pass rate. What is the ‘right difficulty’, though? The recent outcries give a hint. The right difficulty is whatever leads to desirable pass rates. Thus, the process seems to have two rules: (a) if last year the pass rate was lower than desired, decrease difficulty; (b) if last year the pass rate was higher than desired, increase difficulty. In reality, pass rates determine difficulty: The higher the pass rate, the more difficult the test next year.

Good or bad, the fact is that pass rates determine difficulty. It is not inevitable, though. Consider an alternative system of deciding who passes and who fails. Before the test, we decree that the pass rate must be $75\%$. Then we administer the test. Then we sort all children by grade. Then … guess what? … we decree that the top $75\%$ pass. With this system, pass rates and difficulty are completely decoupled. We would be free to (1) pick pass rates based on economic considerations, and (2) choose the difficulty of tests based on other criteria. This system is sometimes applied under the guise of ‘scaling grades’. For some reason, most people find it psychologically difficult to ditch grade thresholds entirely.

No comments:

Post a Comment

Note: (1) You need to have third-party cookies enabled in order to comment on Blogger. (2) Better to copy your comment before hitting publish/preview. Blogger sometimes eats comments on the first try, but the second works. Crazy Blogger.