An exploration of decision consistency indices for one form tests

dc.contributor.author Hagen, Randi
dc.contributor.department Psychology
dc.date 2018-08-16T23:20:52.000
dc.date.accessioned 2020-07-02T06:07:08Z
dc.date.available 2020-07-02T06:07:08Z
dc.date.copyright Sat Jan 01 00:00:00 UTC 1983
dc.date.issued 1983
dc.description.abstract <p>Two studies compared Huynh (1976) and Subkoviak (1976) estimation procedures for indices of classification consistency, rho and kappa, to be used when only one form of a test is administered. Rho indicates the degree of consistency and kappa indicates the degree of consistency beyond chance. Both studies manipulated test length (25, 50, 75 items), distribution shape (normal and two left-skewed) and passing standards (70%, 80%, 90% of items correct). Study I used simulated data; Study II used data from actual test administrations;Both Huynh and Subkoviak procedures yielded rho estimates of similar magnitude and acceptably high as reliability coefficients. Subkoviak kappa estimates were consistently higher than Huynh kappa estimates. No differences in behavior of coefficients was seen between Studies I and II. Test length affected all estimates, increasing in magnitude as test length increased. Magnitudes of rho coefficients decreased as skewness increased; kappa was at its lowest with less skewed distributions, but no consistent pattern of magnitude and skewness interaction was apparent. The effect of distribution shape was a function of the proximity between standard and distribution mode: rho was at its lowest value and kappa at its highest value when the standard neared the mode;Difficulties with interpretation of kappa coefficients are discussed. Suggestions regarding the practical use of and future research directions for one-form rho and kappa estimates are made;References;Huynh, H. On the reliability of decisions in domain-referenced testing. Journal of Educational Measurement, 1976, 13, 253-264. Subkoviak, M. J. Estimating reliability from a single administration of a criterion-referenced test. Journal of Educational Measurement, 1976, 13, 265-276.</p>
dc.format.mimetype application/pdf
dc.identifier archive/lib.dr.iastate.edu/rtd/8474/
dc.identifier.articleid 9473
dc.identifier.contextkey 6335109
dc.identifier.doi https://doi.org/10.31274/rtd-180813-8572
dc.identifier.s3bucket isulib-bepress-aws-west
dc.identifier.submissionpath rtd/8474
dc.identifier.uri https://dr.lib.iastate.edu/handle/20.500.12876/81466
dc.language.iso en
dc.source.bitstream archive/lib.dr.iastate.edu/rtd/8474/r_8407073.pdf|||Sat Jan 15 02:11:56 UTC 2022
dc.subject.disciplines Experimental Analysis of Behavior
dc.subject.disciplines Psychiatry and Psychology
dc.subject.disciplines Psychology
dc.subject.keywords Psychology
dc.title An exploration of decision consistency indices for one form tests
dc.type article
dc.type.genre dissertation
dspace.entity.type Publication
relation.isOrgUnitOfPublication 796236b3-85a0-4cde-b154-31da9e94ed42
thesis.degree.level dissertation
thesis.degree.name Doctor of Philosophy
File
Original bundle
Now showing 1 - 1 of 1
Name:
r_8407073.pdf
Size:
1.48 MB
Format:
Adobe Portable Document Format
Description: