CMNS 260 Lecture Notes - Lecture 3: Guttman Scale, Criterion Validity, Teleology
8 May 2018
School
Department
Course
Professor
1
Introduction to Measurement, Reliability, Validity
Recall: Thinking about causality
• A causal explaatio is a stateet i soial theor aout h eets our that is epressed in
terms of causes and effects. They correspond to associations in the empirial orld. p.32)
• A causal explanation requires:
o Temporal order (cause before effect
o Association (variables must be logically related)
o Elimination of plausible alternatives
• Types of Causal Relationships
o Direct relationship (positive correlation)
o Indirect relationship (negative correlation)
o Null hypothesis
▪ Predicts there is no relationship
▪ If evidence support null hypothesis then what will we find?
Ideas about different types of ausalit i tetook
• Nomothetic explanation
▪ Seeks to explain a class of situations or events rather than a single one. …using
only one or just a few explanatory factors. (p.22)
• Idiographic explanation
o Idio in this context means unique, separate, peuliar, or distit, … [to] fully understand
the causes of what happened in this particular instance. (p.22)
Variables and Causality: Necessary & Sufficient conditions
• Variable identified as indepedet or ause eeds to e related to dependent variable
effets. Tetook eaple about relationship between biological sex and pregnancy
o Distinguish between:
▪ Necessary condition (one that must be present for a specific outcome to occur)?
▪ Sufficient condition (one that, when present, produces a specific outcome) p.89
Pay attention to Units of Analysis when studying the logic of other peoples researh desigs
• A unit of aalsis desigates the kid of epirical case or unit that a researcher observes,
easures, ad aalzes i a stud.
• Can be people or things (artifacts)
Tautology & Teleology
• Tautology: error of explanation that rests on circular reasoning.
o E.g. Obesity is the result of being overweight.
• Teleology: a cause is described as fulfilling some kind of ultimate purpose.
o E.g. Women get paid less than men are because that is the way it is supposed to be.
find more resources at oneclass.com
find more resources at oneclass.com
2
Errors of Explanation-logical errors, incorrect use of data (such as drawing conclusions based on data
using wrong unit of analysis)
Type of Error
Short Definition
Example
Ecological Fallacy
The empirical observations are
at too high a level for the causal
relationship that is stated.
Toronto has a high crime rate.
Joan lives in Toronto. Therefore,
she probably stole my watch.
Reductionism
The empirical observations are
at too low a level for the causal
relationship that is stated.
Because Steven lost his job and
did not buy a new car, the
country entered a long
economic recession.
Spuriousness
An unseen third variable is the
actual cause of both the
independent and dependent
variables.
Hair length is associated with TV
programs. People with short
hair prefer watching football;
people with long hair prefer
romance stories. (Unseen:
Gender)
Ecological Fallacy: Example of the Hung Jury
• Thinking about a hung jury at two levels.
1. As a group, a hung jury is an indecisive jury, is unable to decide the guilt or innocence of
the accused.
2. However, attributing that characteristic (inability to make a decision) to the individual
members of the jury would be incorrect.
o The jury may be hung because the individual members (or a majority) are very decisive –
not indecisive.
o So, they may be indecisive at one level, decisive at another.
• Error of trying to deduce individual behaviour from observations of group behaviour.
Criteria for Evaluating Relationships
• Correlation Association
o There must be an actual, observed relationship – or correlation – between two
variables. P. 85
• Time Order Sequence
o The independent (causal) variable precedes the effect (dependent variable) in time. P.85
• Non-spurious relationships are genuine or authentic. P.85
Measurement in empirical research
• Systematic observation
• Can be replicated (done by someone else)
• Variable must be clearly related to:
o Concepts (constructs)
o Operational definitions: used to create measurement instruments/tools for gathering
information (observing the empirical world)
o Must be able to recognize concept in observations (that is in the empirical measures)
find more resources at oneclass.com
find more resources at oneclass.com
3
Creating Measures of Variables
• Categories of attributes must be:
o Mutually exclusive
▪ Possible observations must only fit in one category
o Exhaustive
▪ Categories must cover all possibilities
o Composite measures must also be:
▪ Uni-dimensional: indicators should fit together coherently and relate to the
same concept
Practice evaluating whether measures propose categories that are exhaustive & mutually exclusive
• Amount of time spent using the computer today rounded to the closest whole number: 1-3
hours, 4-9 hours, 9-12 hours.
o Is this list exhaustive? No.
▪ What would a person answer who had not used a computer today?
▪ What about a person who used computer more than 12 hours?
o Is it mutually exclusive? No, 4-9 and 9-12 overlap
Reliability & Validity (keywords)
• Ways of analyzing how well the empirical measures (observations) fit the conceptual definitions
o ‘eliailit: the dependability or consistency of the easure of a ariale. Esurig that
e repeatedl get the sae alues fro the easureet of the sae thig.
▪ Reliability: dependability -is the indicator consistent? Same result every time?
o Validit: truth that a e applied to the logical tightness of experimental design, the
ability to generalize findings outside a study, the quality of measurement, and the
proper use of proedures.
▪ Are you measuring what you think you are measuring?
Types of measurement validity
• Fae alidit: ake sese as a easure of a ostrut i the judget of others.
• Content validity: a measure represents all aspects of the conceptual definition of a construct.
• Criterion validity: relies on some independent, outside, external verification.
• Predictive validity: relies on the occurrence of a future event or behaviour that is logically
consistent to verify the indicator of a construct.
• Concurrent validity: relies on pre-existing and already accepted measure to verify the indicator
of a construct.
Face & Expert Panel Validity
• Judgement by group or scientific community that indicator measures the construct (conceptual
def.)
o E.g. Socio-eooi status eduatio, ioe…
o E.g. Digital divide (differences in access to computers, iteret, roadad…
find more resources at oneclass.com
find more resources at oneclass.com
Document Summary
Recall: thinking about causality: a causal expla(cid:374)atio(cid:374) is (cid:859)a state(cid:373)e(cid:374)t i(cid:374) so(cid:272)ial theor(cid:455) a(cid:271)out (cid:449)h(cid:455) e(cid:448)e(cid:374)ts o(cid:272)(cid:272)ur that is e(cid:454)pressed in terms of causes and effects. They correspond to associations in the empiri(cid:272)al (cid:449)orld. (cid:859) (cid:894)p. 32: a causal explanation requires, temporal order (cause before effect, association (variables must be logically related, elimination of plausible alternatives, types of causal relationships, direct relationship (positive correlation) Indirect relationship (negative correlation: null hypothesis, predicts there is no relationship. Ideas about different types of (cid:858)(cid:272)ausalit(cid:455)(cid:859) i(cid:374) te(cid:454)t(cid:271)ook: nomothetic explanation, seeks to explain a class of situations or events rather than a single one. Using only one or just a few explanatory factors. (p. 22) Idio in this context means unique, separate, pe(cid:272)uliar, or disti(cid:374)(cid:272)t, [to] fully understand the causes of what happened in this particular instance. (p. 22) Variables and causality: necessary & sufficient conditions: variable identified as indepe(cid:374)de(cid:374)t (cid:894)or (cid:858)(cid:272)ause(cid:859)(cid:895) (cid:374)eeds to (cid:271)e related to dependent variable (cid:894)(cid:858)effe(cid:272)ts(cid:859)(cid:895).
