The null-hypothesis significance-test procedure is still warranted

Chow, Siu L. (1998) The null-hypothesis significance-test procedure is still warranted. [Journal (Paginated)]

Full text available as:



Abstract: Entertaining diverse assumptions about empirical research, commentators give a wide range of verdicts on the NHSTP defence in Statistical Significance. The null-hypothesis significance-test procedure (NHSTP) is defended in a framework in which deductive and inductive rules are deployed in theory corroboration in the spirit of Popper's Conjectures and Refutations (1968b). The defensible hypothetico-deductive structure of the framework is used to make explicit the distinctions between (1) substantive and statistical hypotheses, (2) statistical alternative and conceptual alternative hypotheses, and (3) making statistical decisions and drawing theoretical conclusions. These distinctions make it easier to show that (1) H0 can he true,, (2) the effect size is irrelevant to theory corroboration, and (3) "strong" hypotheses make no difference to NHSTP. Reservations about statistical power, meta-analysis, and the Bayesian approach are still warranted.

Item Type:Journal (Paginated)
Keywords:Statistical signficance, statistical hypothesis testing, corroboration of substantive hypothesis, effect size, statistical power, Bayesianism, statistical null hypothesis, statistcal alternative hypothesis, alternative substantive hypothesis
Subjects:Psychology > Cognitive Psychology
JOURNALS > Behavioral & Brain Sciences
ID Code:837
Deposited By:Chow, Dr. Siu L.
Deposited On:17 Nov 1999
Last Modified:11 Mar 2011 08:54

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-37.

Boring, E. G. (1954). The nature and history of experimental control. American Journal of Psychology, 67, 573-89.

Boring, E. G. (1969). Perspective: Artifact and control. In R. Rosenthal, & R. L. Rosnow (Eds.), Artifacts in behavioral research (pp. 1-11). New York: Academic Press.

Campbell, D. T. (1969). Prospective: Artifact and control. In R. Rosenthal & R. L. Rosnow (eds.), Artifact in behavioral research (pp. 351-382). New York: Academic Press.

Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.

Chomsky, N. (1957). Syntactic structures. The Hague: Mouton.

Chow, S. L. (1987a). Experimental Psychology: Rationale, procedures and issues. Calgary: Detselig.

Chow, S. L. (1987b). Some reflections on Harris and Rosenthal's thirty-one meta-analyses. Journal of Psychology, 121, 95-100.

Chow, S. L. (1987c). Meta-analysis of pragmatic and theoretical research: A critique. Journal of Psychology, 121, 259-71.

Chow, S. L. (1988). Significance test or effect size? Psychological Bulletin, 103, 105-10.

Chow, S. L. (1989). Significance tests and deduction: Reply to Folger (1989). Psychological Bulletin, 106, 161-5.

Chow, S. L. (1991a). Conceptual rigor versus practical impact. Theory & Psychology, 1, 337-60.

Chow, S. L. (1991b). Rigor and logic: A response to Comments on "Conceptual Rigor." Theory & Psychology, 1, 389-400.

Chow, S. L. (1991c). Some reservations about statistical power, American Psychologist, 46, 1088-9.

Chow, S. L. (1992). Research methods in psychology: A primer. Calgary: Detselig.

Cohen, J. (1965). Some statistical issues in psychological research. In B. B. Wolman (Ed.), Handbook of clinical psychology (pp. 95-121). New York: McGraw-Hill.

Cohen, J. (1987). Statistical power analysis for the behavioral sciences (Revised edition). New York: Academic Press.

Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-12.

Cohen, J. (1992a). Statistical power analysis. Current Directions in Psychological Science, 1, 98-105.

Cohen, J. (1992b). A power primer. Psychological Bulletin, 112, 155-9.

Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003.

Cohen, M. R., & Nagel, E. (1934). An introduction to logic and scientific method. London: Routledge & Kegan Paul.

Coltheart, M. (1980). Iconic memory and visible persistence. Perception & Psychophysics, 27, 183-228.

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.

Cook, T. D., & Leviton, L. C. (1980). Reviewing the literature: A comparison of traditional methods with meta-analysis. Journal of Personality, 48, 449-72.

Cooper, H. M. (1979). Statistically combining independent studies: A meta-analysis of sex differences in conformity research. Journal of Personality and Social Psychology, 37, 131-46.

Cooper, H. M., & Rosenthal, R. (1980). Statistical versus traditional procedures for summarizing research findings. Psychological Bulletin, 87, 442-9.

Copi, I. (1982). Symbolic logic (6th edition). New York: MacMillan.

Danziger, K. (1990). Constructing the subject: Historical origins of psychological research. Cambridge: Cambridge University Press.

Darlington, R. B., & Carlson, P. M. (1987). Behavioral statistics: Logic and methods. New York: Collier Macmillan Publishers.

Earman, J. (1992). Bayes or bust? A critical examination of Bayesian confirmation theory. Cambridge, Mass.: The MIT Press.

Eysenck, H. J. (1978). An exercise in mega-silliness. American Psychologist, 33, 517.

Falk, R., & Greenbaum, C. W. (1995). Significance tests die hard: The amazing persistence of a probabilistic misconception. Theory & Psychology, 5, 75-98.

Fillmore, C. J. (1968). The case for case. In E. Bach, & R. T. Harms (Eds.), Universals in linguistic theory (pp. 1-90). New York: Holt, Rinehart, and Winston.

Fisher, R. A. (1959). Statistical methods and scientific inference (2nd edition). New York: Hafner Publishing Co.

Gallo, P. S., Jr. (1978). Meta-analysis--A mixed meta-phor? American Psychologist, 33, 515-7.

Garner, W. R., Hake, H. W., & Eriksen, C. W. (1956). Operationalism and the concept of perception. Psychological Review, 63, 149-59.

Gergen, K. J. (1991). Emerging challenges for theory and psychology. Theory & Psychology, 1, 13-35.

Gigerenzer, G. (1993). The superego, the ego, and the id in statistical reasoning. In G. Keren, & C. Lewis (Eds.), A handbook for data analysis in the behavioral sciences: Methodological issues (pp. 311-39). Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Glass, G. V. (1976). Primary, secondary and meta-analysis of research. Educational Researcher, 5, 3-8.

Glass, G. V. (1978). Integrating findings: The meta-analysis of research. Review of Research in Education, 5, 351-79.

Glass, G. V., & Kliegl, R. M. (1983). An apology for research integration in the study of psychotherapy. Journal of Consulting and Clinical Psychology, 51, 28-41.

Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.

Haber, R. N. (1983). The impending demise of the icon: A critique of the concept of iconic storage in visual information processing. Behavioral and Brain Sciences, 6, 1-11.

Hagen, R. L. (1997). In praise of the null hypothesis statistical test. American Psychologist, 52, 15-24.

Harris, M. J., & Rosenthal, R. (1985). Mediation of interpersonal expectancy effects: 31 meta-analyses. Psychological Bulletin, 97, 363-86.

Hogben, L. (1957). Statistical theory: The relationship of probability, credibility ad error. New York: W. W. Norton.

Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newburry Park, California: SAGE Publications.

Keppel, G., Underwood, B. J. (1962). Proactive inhibition in short-term retention of single items. Journal of Verbal Learning and Verbal Behavior, 1, 153-161.

Kirk, R. E. (1984). Basic statistics (2nd edition). Pacific Grove, CA: Brooks/Cole.

Kraemer, H. C., & Thiemann, S. (1987). How many subjects? Statistical power analysis in research. Newbury Park, CA: Sage.

Leviton, L. C., & Cook, T. D. (1981). What differentiates meta-analysis from other forms of review. Journal of Personality, 49, 231-6.

Manicas, P. T., & Secord, P. F. (1983). Implications for psychology of the new philosophy of science. American Psychologist, 38, 399-413.

Meehl, P. E. (1967). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103-15.

Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 429-71.

Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1, 108-41.

Mill, J. S. (1973). A system of logic: Ratiocinative and inductive. Toronto: University of Toronto Press.

Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81-97.

Miller, G. A. (1962). Some psychological studies of grammar. American Psychologist, 17, 748-62.

Mintz, J. (1983). Integrating research evidence: A commentary on meta-analysis. Journal of Consulting and Clinical Psychology, 51, 71-5.

Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38, 379-87.

Morrison, D. E, & Henkel, R. E. (Eds.). (1970). The significant test controversy: A reader. Chicago: Aldine.

Mosteller, F., & Bush, R. R. (1954). Selected quantitative techniques. In G. Lindzey (Ed.), Handbook of social psychology: Volume 1 - Theory and method (pp. 289-334). Reading, Mass.: Addison-Wesley.

Neisser, U. (1967). Cognitive psychology. New York: Appleton-Century-Croft.

Neyman, J., & Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inferences (Part I). Biometrika, 20A, 175-240.

Oakes, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. Chichester: John Wiley & Sons.

Phillips, L. D. (1973). Bayesian statistics for social scientists. London: Nelson.

Popper, K. R. (1968a). The logic of scientific discovery (originally published in 1959). New York: Harper & Row.

Popper, K. R. (1968b). Conjectures and refutations (originally published in 1962). New York: Harper & Row.

Presby, S. (1978). Overly broad categories obscure important differences between therapies. American Psychologist, 33, 524-515.

Rachman, S., & Wilson, G. T. (1980). The effects of psychological therapy. Oxford: Pergaman.

Rosenthal, R. (1983). Assessing the statistical and social importance of the effects of psychotherapy. Journal of Consulting and Clinical Psychology, 51, 4-13.

Rosenthal, R. (1984). Meta-analytic procedures for social research. Beverly Hills, CA: Sage.

Rosenthal, R., & Rubin, D. B. (1979). A note on percent variance explained as a measure of the importance of effects. Journal of Applied Social Psychology, 9, 395-6.

Rosenthal, R., & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74, 166-9.

Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-84.

Rozeboom, W. W. (1960). The fallacy of the null-hypothesis significance-test. Psychological Bulletin, 57, 416-28.

Savin, H. B., & Perchonock, E. (1965). Grammatical structure and the immediate recall of English sentences. Journal of Verbal Learning and Verbal Behavior, 4, 348-53.

Schmidt, F. L. (1992). What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. American Psychologist, 47, 1173-81.

Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers. Psychological Methods, 1, 115-29.

Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information processing: I. Detection, search, and attention. Psychological Review, 84, 1-66.

Siegel, S. (1956). Non-parametric statistics for the behavioral sciences. New York: McGraw-Hill.

Sohn, D. (1980). Critique of Cooper's meta-analytic assessment of the findings of sex differences in conformity behavior. Journal of Personality and Social Psychology, 39, 1215-21.

Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs, 74,(11, Whole No. 498).

Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25, 26-30.

Tukey, J. W. (1960). Conclusions vs. decisions. Technometrics, 2, 1-11.

Wilson, G. T., & Rachman, s. J. (1983). Meta-analysis and the evaluation of psychotherapy outcome: Limitations and liabilities. Journal of Consulting and Clinical Psychology, 51, 54-64.

Yngve, V. (1960). A model and an hypothesis for language structure. Proceedings of the American Philosophical Society, 104, 444-66.


Repository Staff Only: item control page