Why 40 questions, not 10 or 150
The length of a test decides how much it errs and how much it tires you out. IQNorth's 40 questions are the point where both are acceptable at the same time.
In this article
A trade-off between error and fatigue
Each question adds a little information about your reasoning. The more questions, the smaller the measurement error. The gain, however, shrinks with every addition. A formula known since 1910, the Spearman-Brown formula, shows this: a test with a reliability of 0.80 rises to about 0.89 when its length is doubled, and drops to about 0.67 when it is cut in half.
On the other side is the person taking it. Attention and willingness have limits, and a test that goes past them starts measuring fatigue. Choosing the length means finding the balance point between these two costs.
When questions are scored with Item Response Theory, what matters is not only how many questions there are but also where they fall on the scale. Forty questions spread from easy to hard tell more about your level than sixty bunched at the same difficulty. The test was built to cover the range where most people are, and to accept a larger error at the ends.
Why not 10
With 10 questions, a single lucky guess or a careless mistake moves the result too much. The confidence range would be so wide that the number would say almost nothing.
There is also a coverage problem. A good test needs easy, medium and hard questions to be able to separate people across the whole scale. With 10 questions, you're left with one or two per level, and several areas of reasoning would have no representative at all.
Why not 150
Full batteries administered by psychologists usually take more than an hour, with breaks and someone overseeing them. Online, alone, in front of a screen, few people keep the same level of attention for that long. Toward the end of a long test, impulsive answers start to appear, given just to finish, and they add error instead of information.
That behavior, in fact, is one of the signals that gets a result flagged as unreliable on IQNorth: answers too fast to have been read. A shorter test, taken attentively from start to finish, tells more than a long one rushed through.
The 37 that count
Of the 40 questions, 37 go into the estimate: 34 reasoning questions and 3 memory recalls. The other 3 are about you. Age changes the comparison, because the result is calculated relative to people in your age group; education and self-rating don't change the number. The test takes about 20 minutes.
| Area | Questions that count |
|---|---|
| Matrices | 10 |
| Number and letter series | 5 |
| Figure rotation | 5 |
| Logic | 5 |
| Verbal reasoning | 5 |
| Memory (digits and recall) | 7 |
Matrices carry more weight because, for decades, they have been one of the tasks that comes closest to general reasoning. The structure is inspired by ICAR, a public-domain battery administered online to nearly 97,000 people (Condon and Revelle, 2014); IQNorth's questions are original.
Why we don't give an IQ by area
Five questions measure an area very poorly. Applied to such a short block, the same Spearman-Brown logic gives a low reliability, and a logic IQ or a verbal reasoning IQ would have an enormous margin of error. Showing that number would mean selling a precision that doesn't exist.
The report does something else: it shows how many questions you got right in each area and whether that area came out above, at or below your overall result. For each area, it includes a question of the same type solved step by step, which is not one of the ones you answered.
From easy to hard, with no clock per question
The difficulty rises gradually. Starting easy lets you understand each format before facing its more demanding versions, and keeps people from giving up right at the start when faced with a tough question. It's the same logic as Raven's Progressive Matrices, published in 1938.
There is no time limit per question. The timer shows only the total time and doesn't change the score: the test aims to measure how you reason, not how fast you go. Time is used only to detect answers that are too fast.
See also: How long it takes.
Read next: Free or paid IQ test: what changes?
Sources
- Spearman, C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 3.
- Brown, W. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 3.
- Raven, J. C. (1938). Progressive Matrices. London: H. K. Lewis.
- Condon, D. M., & Revelle, W. (2014). The International Cognitive Ability Resource: Development and initial validation of a public-domain measure. Intelligence, 43, 52–64.
- Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. Mahwah, NJ: Erlbaum.
