DANCEST 805 Lecture Notes - Lecture 15: Confirmation Bias, Term Logic, Statistical Hypothesis Testing
Task 4 – Reasoning
Introduction to reasoning
Hypothesis testing
• Popper argued that there is an important distinction between confirmation and falsification
→ According to him falsification is necessary for science → however, scientists often seek confirmatory rather than disconfirmatory
evidence when testing their hypotheses
• Positive tests – numbers you produce are instance of hypothesis, only confirmatory if you believe hypothesis is correct
→ negative tests – numbers you produce do not conform to hypothesis, which confirms hypothesis
• 2-4-6 task (Wason) – participants told that three numbers 2-4-6 conformed to a single relational rule; task was to generate sets of three
numbers, and to provide reasons for each choice; after each choice, experimenter said whether the set of numbers conformed to the rule;
task was to discover the rule
→ Rule – three numbers in ascending order of magnitude
→ 21% were correct in first attempt, 28% never discovered rule at all
→ they show confirmation bias: try to generate numbers confirming their original hypothesis & failed to attempt hypothesis
disconfirmation
however, more likely to falsify the same hypothesis if told it was someone else’s
→ However, positive testing is more likely than negative to lead to falsification of negative hypotheses – only if numbers of instances
confirming hypothesis are equal to those conforming to rule
→ Task penalizes positive testing – target rule is extremely general
Works better in narrow hypotheses → more possibilities to be wrong and so few to be right
→ Easy to change the nature of initial hypothesis offered by participants – by changing numbers presented
→ If you give participants two rules, they can identify one rule by confirming the other → do not have to focus on disconfirmation of
hypotheses
→ Performance on the 2-4-6 task involves separable processes of hypothesis generation and hypothesis testing
→ Participants try to preserve as much of the information contained in the example as possible in their initial hypothesis
Hypothesis is typically much more specific than the correct rule → prevents them from discovering the correct rule
→ Less frequent feedback helps → led participants to think more flexibly about the possible nature of the rule
→ Research on this task has shed much light on the strengths and limitations on human inductive reasoning
Limitations
• The 2-4-6 task differs from real-life hypothesis testing
→ Feedback is more informative in real-life
→ It is delayed in time and may not be accurate
• correct rule or hypothesis in the 2-4 -6 task (three numbers in ascending order or magnitude) is very general in that it applies to a fairly high
proportion or set or three numbers
→ most rules or hypotheses apply lo only a smallish proportion of possible objects or events
Positive testing works poorly on the 2-4 6 ta k but not with most other forms of hypothesis testing
• Less evidence of confirmation bias if the hypothesis tested is someone else’s
→ Consistent with scientists’ behavior
Hypothesis testing: simulated and real research environments
• Most experts argue that Popper' views are partially correct but somewhat simplistic
→ Also impractical
• Stereotype that scientific discovery is the result of genius, inspiration and sudden insight
→ Scientists use weak methods which are very general and can be applied to almost any scientific problem
→ Several weak methods are also used in everyday problem solving
• Scientists make extensive use of the unusualness heuristic or rule of thumb
Inductive Reasoning
Deductive Reasoning
= Involves making generalized conclusion from premises referring to a
particular instance.
• Conclusions are probably, but not necessarily, true – generalization
provides no certainty that premise will be true at a later time
• Scientist often make use of it
• Analogical reasoning – individual tries to solve current problem by
retrieving information about a similar problem that was solved in the past
• Hypothesis testing – there is a difference between confirmation (attempt
to obtain evidence that will confirm correctness of hypothesis) and
falsification (attempt to falsify hypotheses by experimental tests)
= Involves drawing conclusions that are definitely valid
provided other statements are assumed to be true.
• Related to problem-solving – deductive reasoning has a
definitive goal, but solution is not obvious
• Based on formal logic – however, most people do not
use traditional logic to solve problems
• Conditional reasoning – based on conditional
propositions that have the form "if p, then q"
• Syllogistic reasoning – involves arriving at a conclusion
based on two or more propositions that are assumed to
be true
→ Unusualness heuristic – a rule of thumb by scientists in which unexpected findings are used to develop new hypotheses and lines
of research
• Other heuristics: challenge conventional wisdom, adopt a step-by-step approach, carry out many experiments on a trial-and-error basis
• Scientists make much use of “what if” reasoning in which they work out what would happen in various imaginary circumstances
• Findings in simulated research environments have various limitations:
→ The commitment motivating real scientists to defend their own theories and disprove those of other scientists is lacking
→ real scientists typically work in teams, whereas participants in simulated research environment sometimes work on their own
→ real research is typically influenced by major motivational and social factors largely absent from simulated research environment
Deductive reasoning
Conditional reasoning
= a form of deductive reasoning based on “if…then” propositions
• logical operator: or, and, if…then, if and only if
• symbols stand for sentences and logical operators are applied to them to reach conclusions
• P stands for proposition, and is the antecedent (e.g. it is raining) and Q is the consequent (e.g. Nancy gets wet); a logical operator “if…then”
is used to link p & q: if p, then q
→ Propositions can only have two truth values – true or false; no uncertainty about truth of p
→ Confusion can arise because in daily language “If A, then B” means “if and only if A, then B,”
→ Strongly influenced by availability of background knowledge & contextual information
→ Propositional logic does not admit any uncertainty about the truth of P
• Nearly everyone makes the valid modus ponens inference, but far fewer make the valid modus tollens inference
• In valid inferences (denial of the consequent, affirmation of the consequent) are accepted much of the time, the former typically more
often
• Experiment with counterexamples:
→ number of counterexamples had a major impact on participants' willingness to accept valid inferences (modus ponens, modus
tollens) which is contrary to traditional logic
→ reasoning performance of participants high in working memory capacity was better than those low in working memory capacity
• reasoners draw inferences when presented with conditional reasoning problems
→ Such inferences arc derived from folk axioms (e.g., people engage in self-interested behaviour) and often lead reasoners to accept
invalid conclusions
→ such inferences are not included within classical logic, which takes no account of people's knowledge and experience
• Limitation: it focused on disinterested reasoning (reasoning in which goals and preferences are irrelevant and which therefore contrasts
greatly with everyday life)
• Two strategies people can use with invalid conclusions:
1. Statistical strategy: intuitive; involves estimating the probability that the conclusion is true given what the individual knows about the
world
2. Counterexample strategy: more cognitively demanding (if participants had unlimited time, this strategy was used); involves trying to
think of a counterexample contradicting the conclusion and accepting the conclusion if no counterexample comes to mind
reasoner using this strategy will correctly reject invalid conclusions with affirmation or the consequent problems provided a
counterexample is identified
→ the counterexample strategy was used on 49% or trials with unlimited time but on only 1.7% of trials with limited time. In contrast, the
less demanding statistical strategy was used on 55% of limited-time trials but only 19% of unlimited-time trials
Rule of inference
Explanation
Modus Ponens
(Implication
Elimination)
• Valid logical reasoning
• If “P, then Q” and “P” is given, we can validly infer Q
• 100% of people made valid modus ponens inference
• Influence of context – additional arguments (e.g. information that is added
to original premises, for example in brackets) lead to reduction in performance
Modus Tollens
(Denying the
Consequent)
• Valid logical reasoning, but often regarded as invalid by individuals
• If “P, then Q” and “Q is false” the conclusion “P is false” necessarily follows
• Less than 60% of people made valid modus tollens inference
• Influence of context – additional arguments lead to reduction in
performance
Denial of
Antecedent (Inverse
Error or Fallacy of
the Inverse)
• Invalid logical reasoning
• Formal fallacy of inferring the inverse from the original statement
• If P, then Q If not P, then not Q
Affirmation of
Consequent
(Converse Error or
Fallacy of the
Inverse)
• Invalid logical reasoning – confusion of necessity and sufficiency
• Formal fallacy of taking a conditional statement (if lamp were broken, the
room would be dark) and invalidly inferring its converse (the room is dark,
so the lamp must be broken)
• Arises when consequent (Q) has one or more other (P) antecedents (lamp
is not plugged in, or lamp is not in working order)
• A little more commonly accepted than denial of antecedent
• People rarely think logically on conditional reasoning tasks as is shown by their poor performance on arguments such as affirmation of the
consequent and denial of the antecedent
• Reasoners use their knowledge of other people’s goals and preferences to draw inferences
→ Also use their general knowledge of causal factors
→ Using one’s available knowledge makes good sense in the real world but produces allegedly “illogical thinking”
• Performance on conditional reasoning tasks depends in part on individuals' ability and preference for using relatively demanding and
effortful forms of processing
• also depends on the time available for reasoning
• much conditional reasoning is closer to decision making than to classical logic
Wason selection task
= task involving hypothesis testing using a conditional rule
• Involves hypothesis testing using a conditional rule; is not purely a deductive-reasoning task
• Standard task – 4 cards lying on table, each card has a letter on one side & a number on the other; participant is told that a rule applies to
the four cards; participant must select only those cards that would need to be turned over to decide whether or not rule is correct
→ Involves indicative rule – if there is a p, then there is a q
→ R, G, 2, 7 & rule is that if there is an R on one side of the cards, there is a 2 on the other side
Most participants choose either R card or R & 2 cards – they try only to confirm the rule: 2 card is irrelevant because if there
is an R on the other side, this only tells us the rule might be correct; if there is another letter on the other side, then we
discover nothing about validity of the rule
Matching bias – (heuristic) tendency for participants to select cards matching items named in rule regardless of whether
matched items are correct
Correct answer – R & &; trying to see whether any of the cards fail to obey the rule: 7 is chosen because it would disprove
the rule if it has an R on the other side
→ It is possible to use a denotic rule – detection of rule violation; easier for people to understand because underlying structure of
problem is more explicit (disproving rule)
Social contract theory – people have rules maximizing their ability to achieve their goals in social situations; people possess
“cheat-detecting algorithm” allowing them to identify cases of cheating;
People perform much better on the task when rule is phrased in a way where showing rule is false involves detecting
cheaters
More motivated to disprove the rule when they are personally affected by the rule
• small percentage of individuals (mostly of high intelligence) use deductive reasoning and provide the correct answer on the standard
Wason election tasks
→ great majority produce incorrect answers because they use simple strategies such as matching bias and/or because they do not
understand fully what the task involves
• Performance is substantially better with deontic rules than indicative ones because the former rules direct people's attention to the
importance of disproving the rule
→ Performance also improves if individuals are personally motivated to disprove the rule
Syllogistic reasoning
• Syllogism – consists of two premises or statements followed by a conclusion
→ All A are B; all B are C; therefore, all A are C
→ All children are obedient, all girl guides are children; conclusion: all girl guides are obedient
→ validity depends only on whether it follows logically from the premises & not on the truth or falsity of the
conclusion in the real world
→ Effect of language – “some” means “at least one and possibly all” in formal logic: contrasts everyday
language where “some” means “some but not all”
→ Belief bias – tendency to accept invalid conclusions if they are believable, or to reject valid conclusions
when they are unbelievable
• performance was influenced by the perceived probability of syllogisms being valid
• belief-by-logic interaction
→ Performance on syllogisms with valid conclusions was better when the conclusions were believable,
whereas performance on syllogisms with invalid conclusions was worse when the conclusions were believable
• people took longer to process unbelievable premises than believable ones
→ participants experienced a conflict between their beliefs and what they were asked to assume, and resolving this conflict was time
consuming
• more likely to accept conclusion that matched the premises in surface features than those not matching
• syllogistic reasoning can be influenced by non-logical factors that do not depend on relevant knowledge and experience
• knowledge of the real world influenced reasoning
