Comparing Measures of Individual Differences in Conditional Reasoning
Gloria Seoane , Mª Dolores Valiña, , Mª José Ferraces & Montserrat Martín
University of Santiago de Compostela (Spain)
The study of thinking and reasoning is a topic of central interest for economists, anthropologists, logicians, pedagogues and of course for psychologists. A central problem in the experimental investigation in Psychology is to describe how people think and reason deductively and inductively.
There are three fundamental theoretical approaches to deductive reasoning in the Cognitive Psychology: mental logic, mental models and pragmatic schemas There are several proponents of a universal mental logic (Inhelder & Piaget, 1958) or natural logics (Braine, 1978, 1990, 1994; Braine & O‘Brien, 1991; Braine & Rumain, 1983; Osherson, 1974, 1975; Rips, 1983, 1990, 1994). Other authors propose that reasoning is based on construction and evaluation of mental models (Johnson-Laird, 1983; Johnson-Laird & Byrne, 1991). A third approach asserts that reasoning is not based on general inference rules and assumes that people have domain-specific reasoning mechanisms such as pragmatic reasoning schemas inductively acquired (Cheng & Holyoak, 1985, 1989; Cheng, Holyoak, Nisbett & Oliver, 1986; Holyoak & Cheng, 1995) or innates procedures for identify potential deviations from social contracts (Cosmides 1985, 1989; Cosmides & Tooby, 1992)
Psychometrics studies the thinking from a different perspective. The central interest for the researchers in Psychometrics is not the understanding of underlying cognitive processes and mental representations but the study of the individual differences in these mental processes.
However, despite the differences between these two approaches to the study of human reasoning the categorical syllogisms and linear syllogisms were included on early intelligence tests (Burt, 1919, 1921; Thurstone, 1938; Guilford, 1959). Moreover, in the past decades there is a novel and comparatively neglected field: the study of qualitative and quantitative differences in reasoning. Roberts (1993, p. 575) suggested that:
"The problem of individual differences is as follows: if a theory of reasoning is being proposed that is intended to describe the processes used by all people for all reasoning tasks, then what is the status of this theory if it is subsequently found that not all people are using the same processes?."
Galotti, Baron & Sabini (1986) examined the correlates of reasoning ability on a syllogistic reasoning task. They found evidence for the use of both models and rules of reasoning. In a previous work Sternberg and Weil (1980) found individual differences in reasoning strategies (a mental model strategy, a deduction rule strategy and a mixture of both) in the resolution of experimental tasks that involve linear syllogisms.
Alternatively, Sternberg and Gastel (1989) investigated information processing during the solution of inductive reasoning problems (analogies, classifications and series completions) and also administred five psychometric tests to each subject. They show correlations between experimental tasks and psychometric tests. These correlations address two principal questions. First, are scores on the experimental tasks related to scores on the psychometric tests?. Second, do the correlations with the reasoning tests differ from those with verbal/perceptual factor?. It was found that the correlations of the experimental task with the reasoning tasks are higher than those with verbal/perceptual tasks. Thus, "the experimental tasks do appear to tap abilities related to those tapped by the psychometric tests" (p. 8).
Despite the importance of conditional reasoning in daily life, the study of qualitative individual differences has not become a central focus in cognitive or psychometric studies. There is no nearly previous experimental research about this issue. We found in a previous study (Valiña, Seoane, Ferraces & Martín, 1995) a considerably better performance in the Wason’s selection task in the higher verbal group (DAT-VR) but there were no differences between subjects with high and low scores on the PMA-E psychometric test. In the present experiment we explore the relation among different measures in psychometric ability tests (verbal comprehension and reasoning) and the performance of this experimental conditional reasoning task.
The experimental task
The Wason’s selection task is one paradigm widely used for studying conditional reasoning. The original problem was elaborated by Wason (1966,1968). He presented a conditional rule ’every card that has a vowel on one side has an even number on the other’ and four cards: E, K, 4 and 7. The subjects’ task is to decide which cards should be turned over to test the conditional rule.
Frequently, the subjects only selected the E card (p) or the E and 4 cards ( p and q). The correct response is the selection of the E and 7 cards (p and not q), but only 5-10% of the subjects chose these cards. The subjects selected a case for which the rule is true, but it is a negative instance which provides a violating case and can prove the truth or the falsity of the rule.
We selected this task in part because has long been of interest to experimental psychologist (see Evans, 1982, 1989; Wason, 1983; Evans, Newstead & Byrne, 1993; Newstead & Evans, 1995, for revisions) and moreover because it is one of the most widely used paradigm for studying the importance of factors related to the role of pragmatic knowledge in reasoning (Waso & Shapiro, 1971; Johnson-Laird, Legrenzi & Legrenzi, 1972; Griggs, 1983, 1989; Wason, 1983; Chrostowski & Griggs, 1985; Yachanin, 1986; Pollard & Evans, 1987; Girotto & cols., 1989; Valiña, Seoane, Ferraces & Martín, 1995, 1996).
Concretely, in this paper we examined the following questions: (1) the relation among different measures in psychometric ability tests (verbal comprehension and reasoning), computerised measure of comprehension skills and the subjects’s performance in the experimental task of conditional reasoning, (2) whether or not good and poor comprehenders sistematically differ in their performance in Wason’s selection task (Wason, 1966, 1968) and (3) the differential influence of rule content and instructions on the subject’s performance in the selection task.
One hundred and fifty-four undergraduates (20 males, 134 females; mean age 21 years), studying Psychology at the University of Santiago de Compostela (Spain) participated in this study. The students participated as partial fulfillment of a course requirement. They had not participated in similar experiment and none had any prior training in formal logic.
Data from 18 participants were not used because they failed to follow the experimental instructions or they had not completed all the task.
Materials and apparatus
1) Psychometric tests
The participants completed three spanish versions of three psychometric ability tests: DAT-VR, PMA-V and PMA-R
2) Gernsbacher’s Battery Comprehension
The spanish version of the Battery was presented on a DX-486 computer using a computer programme elaborated by Manuel de Vega, of the University of La Laguna (Spain). The programme presents 4 narrative texts with times of exposition on the screen of 3,5 seconds for each sentence. Subjects read sentences that were presented sentence-by-sentence on the computer monitor. After the last sentence of each story disappeared, a test of five alternativequestions about each experimental text appeared. The times of presentation of each question about the story were of 20 seconds. The subjects’ task was to select as rapidly and accurately as possible the correct alternative that had occurred in the text they had just finished reading. Finally, the programme presented the next text 15 seconds after the final response of the subject.
The programme registered both the correct responses and the reaction times of the participants to the questions about the stories.
3) Selection task
Each subject received three rules, with the following types of content: abstract, thematic-permission and thematic-obligation. Half of the subjects received true-false instructions of the rule and the other half violation instructions. The test booklets were used in previous investigations (Martín, 1996; Valiña, Seoane, Ferraces & Martín, 1996). The information for each of the three tasks was as follows:
a) Abstract selection task. "If a Wasit card has an A on one side, then it must have a 3 on the other". The four cards presented to the subjects were: "A", "K", "3" and "7".
b) Thematic-permission. In this rule a law was expressed; therefore it is similar to permission. The rule was: "If a person is more than 18 years old, then he has the right to vote". The four cards said: "20 years old", "16 years old", "you have the right to vote" and "you do not have the right to vote".
c) Thematic- obligation. The rule expressed a traffic regulation: "If a person rides a motorbike, then they must wear a helmet". The four cards that were represented were: "motorbike", "car", "helmet" and "cap".
The instructions were used previously (Chrotowski & Griggs, 1985; Yachanin, 1986; Valiña, Seoane, Ferraces & Martín, 1995, 1996). In the true-false version, the instructions were:
"Your task consists of selecting cards and only those that must be turned over to decide if the rule is true or false (select those cards which you consider necessary to turn over to check if the person carrying out the experiment has lied or not in relation to the composition of the rule".
For the violation version, the instructions were:
"Your task consists of selecting only those cards that must be turned over in order to decide if the rule is being violated or not".
Two different versions were made for each of the types of booklets. In one of these the thematic versions were at the beginning, followed by the abstract rule and in the other the abstract version was included at the beginning. Additionally, the order of presentation of the two thematic versions was counterbalanced.
Participants met in groups of up to 12 with two experimenters over 2 days. On the 1st day they received in the laboratory both the psychometric tests with conventional instructions and the spanish version of the Gernsbacher’s Battery Comprehension. Subjects were tested with each interacting on a separate microcomputer in the same laboratory.
On the 2nd day of the experiment, participants were assigned at random to one of two experimental groups: (1) true-false instructions and (2) violation instructions. Subjects were tested in groups of 12. Each subject received a booklet with instructions on the first page, followed by three selection tasks (an abstract one and two thematic selection tasks). The instructions were read to the subjects and questions were solicited to ensure that they understood the instructions. Finally, they were instructed to work at their own rhytm, without a time limit.
The analysis were carried out with the data from the 136 subjects, once those who had not completed the task had been eliminated.
First we performed ANOVAs to test the differential influence of rule content and instruction on the subjects’ performance in the selection task (Wason, 1966, 1968). Other ANOVAs were performed for checking whether or not good and poor comprehenders sistematically differ in their performance in Wason’s selection task, and finally we performed analysis in order to provide a test of the relation among different measures in psychometric ability tests (verbal comprehension and reasoning), the computerised measure of comprehension skills and the subjects’ performance in the experimental task of conditional reasoning.
The logical and matching indices were calculated for each of the three tasks. Both indices vary between +2 and -2, according to Pollard and Evans (1987). In the logical index the p or not-q selection gave a mark of +1 and the not-p or q selection gave -1. In the matching index the p or q selection gave a mark of +1 and with -1 the not-p or not-q selection. ANOVAS 2 x 3 ( instructions x content) were made for each type of index, with the data from the 136 participants.
a) Logical index
For the logical index the principal effects of the content (F(1.82; 244.52)= 21.61; p < .0001; e = .912) and the instructions (F(1, 134) = 6.59; p < .011) were registered. In the thematic-obligation higher logical indices were obtained (M = .765), followed by the abstract version of the task (M = .449) and the thematic-permission (M = .154). Similarly, the logical indices were higher in those subjects who received violation instructions (M = .87) compared with those who received true-false instructions (M = .31).
Significant interactive effects have also been registered for instructions x content (F(1.82; 244.52) = 6.32; p < .003; e =.912). In the thematic-obligation task higher indices were obtained in those subjects who received violation instructions
b) Matching index
A significant effect was obtained with the instructions (F (1, 134) = 11.31; p < .001). Participants obtained higher matching index with instructions for checking the rule (M = 1.042) compared to those who received violation instructions (M = .58).
Similarly, significant effects were registered in the content (F (2, 268) = 25.64; p <.0001). Especifically, the highest matching index was obtained with abstract content (M = 1.11), followed by the thematic-obligation (M = .897) and the thematic-permission (M = .375). Abstract content differs significantly from the other groups (F(1,134) = 24.42; p < .0001) and similarly thematic-permission differs significantly from the thematic-obligation (F(1, 134) = 26.97; p < .0001) by orthogonal tests.
2) Three-way mixed ANOVAS
Because the central questions being addressed involve group differences, we performed three-way mixed analyses of variance, with group (good vs. poor verbal comprehenders / good vs. poor reasoners), instructions (violation vs. true-false) and content (abstract, thematic-permission and thematic-obligation) as factors, with repeated measures on the last factor.
In terms of differential analyses there were no differences in the logical and matching indices among good and poor verbal comprehenders (PMA-V & Gernsbacher’s Battery Comprehension) or subjects with high and low scores in the PMA-R, but there were significant differences among good and poor reasoners (DAT-VR).
The logical index was considerably better (M = 1.131) in the higher reasoning-verbal group vs. the group with low scores in the DAT-VR (M = .386). The differences were significant (F(1, 70) = 8.52; p < .005).
Similarly, for the matching index the interaction between group and instructions was significant (F(1,70)= 5.02; p < .028. There were no differences in the matching index of good reasoners in function of the experimental instructions, but differences in the group with low scores in the DAT-VR were found. Particularly, the poor reasoners obtained highest matching indices with instructions for checking the rule
3) CORRELATIONS OF THE EXPERIMENTAL TASK WITH THE PSYCHOMETRIC TESTS SCORES
We performed another analysis in order to provide a test of the relation among different measures in verbal and comprehension psychometric tests, the computerised measures of comprehension skills and the subjects’ performance in the experimental task with both logical and matching indices. The analysis were carried out (a) with the data from the total of 136 subjects and (b) with de data from the two experimental groups (true-false instructions and violation instructions).
The results show for the total sample (N = 136) that: (a) the scores of the DAT-VR are related with the performance in Wason’s selection task with the abstract content (r = .317;p = .0001) and with the thematic-permission (r = .2656 ; p = .002) in terms of logical index and (b) there is a significant relation between scores in the DAT-VR psychometric test and the computerised measures of Gernsbacher’s Battery Comprehension (r = .1663; p = .05).
With the true-false intructions (N = 63) there was found a significant relation between scores in the DAT-VR and the logical index (r = .3524; p = .005) and the matching index (r = - .3779; p = .002) with the abstract content. There was also a significant relation between measures in the Gernsbacher’s Battery Comprehension and the logical index for this abstract content (r = .2862; p = .023).
With the violation instructions (N = 73) there was a significant relation between measures in the DAT-VR and the logical index for the abstract content (r = .2939; p = .012) and the thematic-permission content ( r = .4316; p = .0001).
From this study we draw the following conclusions:
1. A good predictor for performance in Wason´s selection task was the DAT-VR psychometric test
2. There are differences in performance among subjeects with high and low scores in DAT-VR. The logical index was considerably better in the higher reasoning verbal-group.
3. The matching index are higher in those subjects who received instructions for checking the rule in the lower reasoning verbal group. There are not differences for the subjects with high scores in DAT-VR
4. There are no differences in the selection task performance between groups with scores high and low in the PMA-V, PMA-R psychometric tests and the Gernsbacher´s Comprehension Battery
5. The highest logical index was registered with thematic-obligation content, followed by the abstract content and the thematic-permission content
6. The highest matching index was obtained with abstract content, followed by the thematic-obligation and finally the lowest matching index was obtained with the thematic permission content.
7. The logical indices are higher in those subjects who received violation instructions with the thematic-obligation content.
We especially thank Manuel de Vega for providing us with his Spanish computerised version of Gernsbacher’s Battery Comprehension
Braine, M.D.S. (1978). On the relation between the natural logic of reasoning and standard logic. Psychological Review, 85, 1-21.
Braine, M.D.S. (1990). The "natural logic" approach to reasoning. In W.F. Overton (Ed.), Reasoning, neccesity and logic: Developmental perspectives. Hillsdale, NJ: Erlbaum.
Braine, M.D.S. (1994). Mental logic and how to discover. In J. Macnamara & G.E. Reyes (Eds.), The logical foundations of cognition. Oxford, England: Oxford University Press.
Braine, M.D.S., & O’Brien, D.P. (1991). A theory of If: A lexical entry, reasoning program, and pragmatic principles. Psychological Review, 98, 182-203.
Braine, M.D.S., & Rumain, B. (1981). Development of comprehension of "or". Evidence for a sequence of competencies. Journal of Experimental Child Psychology, 31, 46-78
Braine, M.D.S., & Rumain, B. (1983). Logical reasoning. In J.H. Flavell & E.M. Markman (Eds.), Handbook of child Psychology: Vol.3. Cognitive development. New York: Wiley.
Burt, C. (1919). The development of reasoning in school children. Journal of Experimental Pedagogy, 5, 68-77, 121-127.
Burt, C. (1921). Mental and scholastic tests. London: Kinf.
Cheng, P.W., & Holyoak, K.J. (1985). Pragmatic reasoning schemas. Cognitive Psychology, 17, 391-416.
Cheng, P.W., & Holyoak, K.J. (1989). On the natural selection of reasoning theories. Cognition, 33, 285-313.
Cheng, P.W., Holyoak, K.J., Nisbett, R.E., & Oliver, L.M. (1986). Pragmatic versus syntactic approaches to training deductive reasoning. Cognitive Psychology, 18, 293-328.
Chrostowski, J.J., & Griggs, R.A. (1985). The effects of problem content, instructions and verbalization procedures on Wason’s selection task. Current Psychological Research and Reviews, 4, 99-107.
Cosmides, L. (1985). Deduction or darwinian algorithms?: An explanation of the "elusive" content effect on the Wason selection task. Doctoral Dissertation, Harvard University. University Microfilms 86-02206.
Cosmides, L. (1989). The logic of social exchange: Has natural selection shaped how humans reason?. Studies with the Wason selection task. Cognition, 31, 187-276.
Cosmides, L., & Tooby, J. (1992). Cognitive adaptations for social exchange. In J.H. Barkow, L. Cosmides & J. Tooby (Eds.), The adapted mind. Oxford: Oxford University Press.
Evans, J.St.B.T. (1982). The psychology of deductive reasoning. London: Routledge and Kegan Paul.
Evans, J.St.B.T. (1984). Heuristic and analitic processes in reasoning. British Journal of Psychology, 75, 451-468.
Evans, J.St.B.T. (1989). Bias in human reasoning: Causes and consequences. Hove, UK: Lawrence Erlbaum Associates Ltd.
Evans, J.St.B.T. (1991). Theories of human reasoning: The fragmented state of the art. Theory and Psychology, 1, 83-105.
Evans, J.St.B.T., Newstead, S.E., & Byrne, R.M.J. (1993). Human reasoning: The Psychology of deduction. Hove, UK: Lawrence Erlbaum Associates Ltd.
Evans, J.St.B.T., Over, D.E., & Manktelow, K.I. (1993). Reasoning, decision making, and rationality. Cognition, 49, 165-187.
Galotti, K.M., Baron, J., & Sabini, J. (1986). Individual differences in syllogistic reasoning: Deduction rules or mental models?. Journal of Experimental Psychology: General, 115,16-25.
Girotto, V., Gilly, M., Blaye, A., & Light, P. (1989). Children´s performance in the selection task: Plausibility and familiarity. British Journal of Psychology, 80, 79-85.
Griggs, R.A. (1983). The role of problem content in the selection task and THOG problem. In J. St.B.T. Evans (Ed.), Thinking and Reasoning: Psychological Approaches. London: Routledge and Kegan Paul.
Griggs, R.A. (1989). To "see" or not to "see": That is the selection task. The Quarterly Journal of Experimental Psychology, 41A, 517-529.
Guilford, J.P. (1959). Three faces of intellect. American Psychologist, 14, 469-479.
Holyoak, K.J., & Cheng, P.W. (1995). Pragmatic reasoning about human voluntary action: Evidence from Wason´s selection task. In S.E. Newstead y J.St.B.T. Evans (Eds.), Perspectives on thinking and reasoning. Essays in honour of Peter Wason. Hove: Lawrence Erlbaum Associates, Inc.
Johnson-Laird, P.N. (1983). Mental models. Towards a cognitive science of language, inference and consciousness. Cambridge, England: Cambridge University Press.
Johnson-Laird, P.N., & Byrne, R.M.J. (1991). Deduction. Hove, U.K.: Lawrence Erlbaum Associates
Johnson-Laird, P.N., & Byrne, R.M.J. (1992). Modal reasoning, models, and Manktelow and Over. Cognition, 43, 173-182.
Johnson-Laird, P.N., Legrenzi, P., & Legrenzi, S. (1972). Reasoning and a sense of reality. British Journal of Psychology, 63, 336-400.
Manktelow, K.I., & Over, D.E. (1991). Social roles and utilities in reasoning with deontic conditionals. Cognition, 39, 85-105.
Martín, M. (1996). Una exploración del razonamiento cotidiano: Importancia del conocimiento en inferencia condicional. Unpublished doctoral dissertation. University of Santiago de Compostela, Spain.
Newstead, S.E., & Evans, J.St.B.T. (1995). Perspectives on thinking and reasoning. Essays in honour of Peter Wason. LEA. Hillsdale.
Osherson, D.N. (1974). Logical abilities in children: Vol.2. Logical inference: Underlying operations. Potomac, MD: Lawrence Erlbaum Associates Inc.
Osherson, D.N. (1975). Logical abilities in children. Vol.3. Reasoning in adolescence: Deductive inference. Hillsdadle, NJ: Lawrence Erlbaum Associates Inc.
Platt , R.D., & Griggs, R.A. (1993). Facilitation in the abstract selection task: The effects of attentional and instructional factors. The Quarterly Journal of Experimental Psychology, 46A, 591-613.
Pollard, P., & Evans, J.St.B.T. (1987). On the relationship between content and context effects in reasoning. American Journal of Psychology, 100, 41-60.
Rips, L.J. (1983). Cognitive processes in propositional reasoning. Psychological Review, 90, 38-71.
Rips, L.J. (1990). Reasoning. Annual Review of Psychology, 41, 321-353.
Rips, L.J. (1994). The psychology of proof: Deductive reasoning in human thinking. Cambridge, MA: MIT Press.
Roberts, M.J. (1993). Human Reasoning: Deduction Rules or Mental Models, or Both?. The Quarterly Journal of Experimental Psychology, 46A(4), 569-589.
Sternberg, R.J., & Gastel, J. (1989). If dancers ate their shoes: Inductive reasoning, with factual and counterfactual premises. Memory & Cognition, 17 (1), 1-10.
Sternberg, R.J., & Weil, E.M. (1980). An aptitude x strategy interaction in linear syllogistic reasoning. Journal of Educational Psychology, 72(2), 226-239.
Thurstone, L.L. (1938). Prymary mental abilities. Psychometric Monographs, 1. (Adpt. in Tea, 1972).
Valiña, Mª D., Seoane, G., Ferraces, Mª J., & Martín, M. (1995). Tarea de selección de Wason: Un estudio de las diferencias individuales. Psicothema, 7(3), 641-653.
Valiña, Mª D., Seoane, G., Ferraces, Mª J., & Martín, M. (1996, August). Wason’s selection task: Content effect, instruction effect or both?. In Proceedings of The Third International Conference on Thinking. British Psychological Society. Cognitive Psychology Section. University College London.
Valiña, Mª D., Seoane, G., Gehring, S., Ferraces, Mª.J., & Fernández-Rey, J. (1992, September). Conditional reasoning: Scenario or context effects?. Paper presented at The Fifth Conference of the European Society for Cognitive Psychology, Paris.
Valiña, Mª D., Seoane, G., Martín, M., Fernández-Rey, J., & Ferraces, Mª.J. (1992, September). The role of content and context in pragmatic reasoning. Paper presented at The Fifth Conference of the European Society for Cognitive Psychology, Paris.
Wason, P.C. (1966). Reasoning. In B.M. Foss (Ed.), New horizonts in Psychology. Harmondsworth: Penguin.
Wason, P.C. (1968). Reasoning about a rule. The Quarterly Journal of Experimental Psychology, 20, 273-281.
Wason, P.C. (1983). Realism and rationality in the selection task. In J.St.B.T. Evans (Ed.), Thinking and reasoning: Psychological Approaches. London: Routledge and Kegan Paul.
Wason, P.C., & Shapiro, D. (1971). Natural and contrived experience in a reasoning problem. The Quarterly Journal of Experimental Psychology, 23, 63-71.
Yachanin, S.A. (1986). Facilitation in Wason’s selection task: Content and instructions. Current Psychological Research and Reviews, 5, 20-29.