An exploration of decision consistency indices for one form tests Hagen, Randi
dc.contributor.department Psychology 2018-08-16T23:20:52.000 2020-07-02T06:07:08Z 2020-07-02T06:07:08Z Sat Jan 01 00:00:00 UTC 1983 1983
dc.description.abstract <p>Two studies compared Huynh (1976) and Subkoviak (1976) estimation procedures for indices of classification consistency, rho and kappa, to be used when only one form of a test is administered. Rho indicates the degree of consistency and kappa indicates the degree of consistency beyond chance. Both studies manipulated test length (25, 50, 75 items), distribution shape (normal and two left-skewed) and passing standards (70%, 80%, 90% of items correct). Study I used simulated data; Study II used data from actual test administrations;Both Huynh and Subkoviak procedures yielded rho estimates of similar magnitude and acceptably high as reliability coefficients. Subkoviak kappa estimates were consistently higher than Huynh kappa estimates. No differences in behavior of coefficients was seen between Studies I and II. Test length affected all estimates, increasing in magnitude as test length increased. Magnitudes of rho coefficients decreased as skewness increased; kappa was at its lowest with less skewed distributions, but no consistent pattern of magnitude and skewness interaction was apparent. The effect of distribution shape was a function of the proximity between standard and distribution mode: rho was at its lowest value and kappa at its highest value when the standard neared the mode;Difficulties with interpretation of kappa coefficients are discussed. Suggestions regarding the practical use of and future research directions for one-form rho and kappa estimates are made;References;Huynh, H. On the reliability of decisions in domain-referenced testing. Journal of Educational Measurement, 1976, 13, 253-264. Subkoviak, M. J. Estimating reliability from a single administration of a criterion-referenced test. Journal of Educational Measurement, 1976, 13, 265-276.</p>
dc.format.mimetype application/pdf
dc.identifier archive/
dc.identifier.articleid 9473
dc.identifier.contextkey 6335109
dc.identifier.s3bucket isulib-bepress-aws-west
dc.identifier.submissionpath rtd/8474
dc.language.iso en
dc.source.bitstream archive/|||Sat Jan 15 02:11:56 UTC 2022
dc.subject.disciplines Experimental Analysis of Behavior
dc.subject.disciplines Psychiatry and Psychology
dc.subject.disciplines Psychology
dc.subject.keywords Psychology
dc.title An exploration of decision consistency indices for one form tests
dc.type article
dc.type.genre dissertation
dspace.entity.type Publication
relation.isOrgUnitOfPublication 796236b3-85a0-4cde-b154-31da9e94ed42 dissertation Doctor of Philosophy
Original bundle
Now showing 1 - 1 of 1
1.48 MB
Adobe Portable Document Format