Friday, December 2, 2011

How Reliable Are Intelligence Tests?



From my dictionary: re·li·a·ble, adj- that may be relied on; dependable in achievement, accuracy, honesty.  This article looks at IQ tests in terms of reliability.

I'll start with definition.  Reliability as used by test makers has very little to do with our understanding of the dictionary definition of reliability.  So what DO they mean?
When a test is deemed to be reliable, it simply means that the test will give the same results when given to the same person at two different times.  Testing of reliability is done in one of three ways:
1.     Administer the same test at two different times.
2.    Administer half the test each at two different times.
3.    Administer two 'forms' of the test at two different times.

A reliable test must give the same outcome each time.  (Test results do, of course, tend to 'improve' with practice, but that can be taken into consideration fairly easily.) A great deal of attention is given to the design of tests that can be shown, by one of the above methods, to be reliable. Why is this important? This kind of test reliability bears little resemblance to the sort of things one usually associates with being 'dependable'.  It says nothing about the test being 'useful' for any purpose at all.  So one wonders why it's something that is measured and tested so carefully.
Simply put, test designers will tell us that if a test has any chance whatsoever of being useful, it must first be reliable.  Their logic is inescapable.  There is simply no way we can expect to rely on test scores to indicate ANYTHING meaningful if the test gives significantly different results for the same person on different attempts.
But what about the question of a test being the 'dependable' kind of 'reliable'?  This is the point of testing to begin with, right?  Unfortunately, dependability is a moving target that is very hard to HIT, so test designers are forced to settle for what they call "Validity". There are actually three different measures of validity: 
1.     Criterion validity – the test is accurate when measured against some specific scale, such as a list of desired mathematical skills.
2.    Content validity  – the test contains all of the material it is designed to cover.
3.    Predictive validity –the test results can be used to make accurate predictions of future performance.

“Ah!” you say!  Now we are getting somewhere!  Yes, in terms of testing, validity best approximates what WE think of as reliability.  Some tests, such as those taken for admission to secondary and post-secondary schools, have been shown to be 'valid' when the results are used to predict student success, but this still does not mean that the test is a good measure of INTELLIGENCE.  It may seem strange, but the validity of an intelligence test is often measured by how closely its results mirror those obtained from ANOTHER intelligence test.  What, precisely, either test measures is difficult to define, and it is an undeniable fact that no IQ test yet devised can predict future success with any accuracy. It is reasonable to say that a battery of several IQ tests can give a picture of innate abilities in an individual, but no score, not even a set of scores, can actually quantify intelligence. 
In view of all of this, my advice would be to go ahead and take intelligence tests (or have your children take them). There is no harm in this kind of testing as long as you realize that you should not take the scores VERY seriously.

Jo Karabasz
Managing Director
www.overlooktutorialacademy.net

No comments:

Post a Comment