Data analytic methods for latent partially ordered classification models
Address for correspondence: Curtis Tatsuoka, Department of Statistics, George Washington University, Washington DC 20052, USA.
E‐mail: tatsuoka@gwu.edu
Abstract
Summary. A general framework is presented for data analysis of latent finite partially ordered classification models. When the latent models are complex, data analytic validation of model fits and of the analysis of the statistical properties of the experiments is essential for obtaining reliable and accurate results. Empirical results are analysed from an application to cognitive modelling in educational testing. It is demonstrated that sequential analytic methods can dramatically reduce the amount of testing that is needed to make accurate classifications.
Citing Literature
Number of times cited according to CrossRef: 78
- Peter Gallagher, Neuropsychology of Bipolar Disorder, , 10.1007/7854_2020_148, (2020).
- Wenyi Wang, Lihong Song, Teng Wang, Peng Gao, Jian Xiong, A Note on the Relationship of the Shannon Entropy Procedure and the Jensen–Shannon Divergence in Cognitive Diagnostic Computerized Adaptive Testing, SAGE Open, 10.1177/2158244019899046, 10, 1, (215824401989904), (2020).
- Xiaopeng Wu, Rongxiu Wu, Hua-Hua Chang, Qiping Kong, Yi Zhang, International Comparative Study on PISA Mathematics Achievement Test Based on Cognitive Diagnostic Models, Frontiers in Psychology, 10.3389/fpsyg.2020.02230, 11, (2020).
- Zhaoyuan Zhang, Jiwei Zhang, Jing Lu, Jian Tao, Bayesian Estimation of the DINA Model With Pólya-Gamma Gibbs Sampling, Frontiers in Psychology, 10.3389/fpsyg.2020.00384, 11, (2020).
- Yinyin Chen, Steven Culpepper, Feng Liang, A Sparse Latent Class Model for Cognitive Diagnosis, Psychometrika, 10.1007/s11336-019-09693-2, (2020).
- Chen-Wei Liu, Björn Andersson, Anders Skrondal, A Constrained Metropolis–Hastings Robbins–Monro Algorithm for Q Matrix Estimation in DINA Models, Psychometrika, 10.1007/s11336-020-09707-4, (2020).
- Xin Xu, Jimmy de la Torre, Jiwei Zhang, Jinxin Guo, Ningzhong Shi, Estimating CDMs Using the Slice-Within-Gibbs Sampler, Frontiers in Psychology, 10.3389/fpsyg.2020.02260, 11, (2020).
- Mengta Chung, A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model, Journal of Mathematical Psychology, 10.1016/j.jmp.2019.07.002, 93, (102275), (2019).
- Meghan Fager, Jesse Pace, Jonathan L. Templin, Using Mplus to Estimate the Log-Linear Cognitive Diagnosis Model, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_28, (581-591), (2019).
- Li Cai, Carrie R. Houts, Diagnostic Classification Modeling with flexMIRT, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_27, (573-579), (2019).
- Xiaofeng Yu, Ying Cheng, Hua-Hua Chang, Recent Developments in Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT): A Comprehensive Review, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_15, (307-331), (2019).
- Lei Guo, Chanjin Zheng, Termination Rules for Variable-Length CD-CAT From the Information Theory Perspective, Frontiers in Psychology, 10.3389/fpsyg.2019.01122, 10, (2019).
- Xuliang Gao, Daxun Wang, Yan Cai, Dongbo Tu, Cognitive Diagnostic Computerized Adaptive Testing for Polytomously Scored Items, Journal of Classification, 10.1007/s00357-019-09357-x, (2019).
- Douglas L. Steinley, M. J. Brusco, Using an Iterative Reallocation Partitioning Algorithm to Verify Test Multidimensionality, Journal of Classification, 10.1007/s00357-019-09347-z, (2019).
- Matthew S. Johnson, Sandip Sinharay, The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models, Journal of Educational and Behavioral Statistics, 10.3102/1076998619864550, (107699861986455), (2019).
- Mehmet Kaplan, Jimmy de la Torre, A Blocked-CAT Procedure for CD-CAT, Applied Psychological Measurement, 10.1177/0146621619835500, (014662161983550), (2019).
- Sarah J.A. Carr, Judith Jaeger, Shijia Bian, Ping He, Nancy Maserejian, Wenting Wang, Paul Maruff, Ahmed Enayetallah, Yanming Wang, Zhengyi Chen, Alan Lerner, Curtis Tatsuoka, Associating Cognition With Amyloid Status Using Partially Ordered Set Analysis, Frontiers in Neurology, 10.3389/fneur.2019.00976, 10, (2019).
- Shenghai Dai, Dubravka Svetina, Cong Chen, Investigation of Missing Responses in Q-Matrix Validation, Applied Psychological Measurement, 10.1177/0146621618762742, 42, 8, (660-676), (2018).
- Lihong Song, Wenyi Wang, Haiqi Dai, Shuliang Ding, Bayesian Network for Modeling Uncertainty in Attribute Hierarchy, Quantitative Psychology, 10.1007/978-3-319-77249-3_26, (305-317), (2018).
- Wenyi Wang, Lihong Song, Shuliang Ding, Yaru Meng, Canxi Cao, Yongjing Jie, An EM-Based Method for Q-Matrix Validation, Applied Psychological Measurement, 10.1177/0146621617752991, 42, 6, (446-459), (2018).
- Gongjun Xu, Zhuoran Shang, Identifying Latent Structures in Restricted Latent Class Models, Journal of the American Statistical Association, 10.1080/01621459.2017.1340889, 113, 523, (1284-1295), (2018).
- Steven Andrew Culpepper, Estimating the Cognitive Diagnosis $$\varvec{Q}$$Q Matrix with Expert Knowledge: Application to the Fraction-Subtraction Dataset, Psychometrika, 10.1007/s11336-018-9643-8, (2018).
- Yuan-Pei Chang, Chia-Yi Chiu, Rung-Ching Tsai, Nonparametric CAT for CD in Educational Settings With Small Samples, Applied Psychological Measurement, 10.1177/0146621618813113, (014662161881311), (2018).
- Dubravka Svetina, Yanan Feng, Justin Paulsen, Montserrat Valdivia, Arturo Valdivia, Shenghai Dai, Examining DIF in the Context of CDMs When the Q-Matrix Is Misspecified, Frontiers in Psychology, 10.3389/fpsyg.2018.00696, 9, (2018).
- Yinghan Chen, Steven Andrew Culpepper, Yuguo Chen, Jeffrey Douglas, Bayesian Estimation of the DINA Q matrix, Psychometrika, 10.1007/s11336-017-9579-4, 83, 1, (89-108), (2017).
- Steven Andrew Culpepper, Aaron Hudson, An Improved Strategy for Bayesian Estimation of the Reduced Reparameterized Unified Model, Applied Psychological Measurement, 10.1177/0146621617707511, 42, 2, (99-115), (2017).
- Jinsong Chen, A Residual-Based Approach to Validate Q-Matrix Specifications, Applied Psychological Measurement, 10.1177/0146621616686021, 41, 4, (277-293), (2017).
- Chanjin Zheng, Chun Wang, Application of Binary Searching for Item Exposure Control in Cognitive Diagnostic Computerized Adaptive Testing, Applied Psychological Measurement, 10.1177/0146621617707509, 41, 7, (561-576), (2017).
- Luca Stefanutti, Debora de Chiusole, On the assessment of learning in competence based knowledge space theory, Journal of Mathematical Psychology, 10.1016/j.jmp.2017.08.003, 80, (22-32), (2017).
- Yunxiao Chen, Xiaoou Li, Jingchen Liu, Zhiliang Ying, Regularized Latent Class Analysis with Application in Cognitive Diagnosis, Psychometrika, 10.1007/s11336-016-9545-6, 82, 3, (660-692), (2016).
- Hans-Friedrich Köhn, Chia-Yi Chiu, A Procedure for Assessing the Completeness of the Q-Matrices of Cognitively Diagnostic Tests, Psychometrika, 10.1007/s11336-016-9536-7, 82, 1, (112-132), (2016).
- Chanjin Zheng, Hua-Hua Chang, High-Efficiency Response Distribution–Based Item Selection Algorithms for Short-Length Cognitive Diagnostic Computerized Adaptive Testing, Applied Psychological Measurement, 10.1177/0146621616665196, 40, 8, (608-624), (2016).
- Pasquale Anselmi, Egidio Robusto, Luca Stefanutti, Debora de Chiusole, An Upgrading Procedure for Adaptive Assessment of Knowledge, Psychometrika, 10.1007/s11336-016-9498-9, 81, 2, (461-482), (2016).
- Buyun Dai, Minqiang Zhang, Guangming Li, Exploration of Item Selection in Dual-Purpose Cognitive Diagnostic Computerized Adaptive Testing, Applied Psychological Measurement, 10.1177/0146621616666008, 40, 8, (625-640), (2016).
- Hamdollah Ravand, Application of a Cognitive Diagnostic Model to a High-Stakes Reading Comprehension Test, Journal of Psychoeducational Assessment, 10.1177/0734282915623053, 34, 8, (782-799), (2016).
- Robert Henson, Jeff Douglas, Test Construction for Cognitive Diagnosis, Applied Psychological Measurement, 10.1177/0146621604272623, 29, 4, (262-277), (2016).
- Kikumi K. Tatsuoka, James E. Corter, Curtis Tatsuoka, Patterns of Diagnosed Mathematical Content and Process Skills in TIMSS-R Across a Sample of 20 Countries, American Educational Research Journal, 10.3102/00028312041004901, 41, 4, (901-926), (2016).
- Chia-Yi Chiu, Hans-Friedrich Köhn, Consistency of Cluster Analysis for Cognitive Diagnosis, Applied Psychological Measurement, 10.1177/0146621615577087, 39, 6, (465-479), (2015).
- Yunxiao Chen, Jingchen Liu, Gongjun Xu, Zhiliang Ying, Statistical Analysis of Q -Matrix Based Diagnostic Classification Models , Journal of the American Statistical Association, 10.1080/01621459.2014.934827, 110, 510, (850-866), (2015).
- Wenyi Wang, Shuliang Ding, Lihong Song, New Item-Selection Methods for Balancing Test Efficiency Against Item-Bank Usage Efficiency in CD-CAT, Quantitative Psychology Research, 10.1007/978-3-319-07503-7_8, (133-151), (2015).
- Jürgen Heller, Luca Stefanutti, Pasquale Anselmi, Egidio Robusto, On the Link between Cognitive Diagnostic Models and Knowledge Space Theory, Psychometrika, 10.1007/s11336-015-9457-x, 80, 4, (995-1019), (2015).
- Yuan Sun, Shiwei Ye, Yi Sun, Tsunehiko Kameda, undefined, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), 10.1109/DSAA.2015.7344813, (1-10), (2015).
- Edward H. Ip, Shyh-Huei Chen, Sara A. Quandt, Analysis of Multiple Partially Ordered Responses to Belief Items with Don’t Know Option, Psychometrika, 10.1007/s11336-014-9432-y, 81, 2, (483-505), (2014).
- Yunxiao Chen, Jingchen Liu, Zhiliang Ying, Online Item Calibration for Q-Matrix in CD-CAT, Applied Psychological Measurement, 10.1177/0146621613513065, 39, 1, (5-15), (2014).
- Hyokyoung Hong, Chun Wang, Youn Seon Lim, Jeff Douglas, Efficient Models for Cognitive Diagnosis With Continuous and Mixed-Type Latent Variables, Applied Psychological Measurement, 10.1177/0146621614524981, 39, 1, (31-43), (2014).
- Ying Cui, Johnson Li, Evaluating Person Fit for Cognitive Diagnostic Assessment, Applied Psychological Measurement, 10.1177/0146621614557272, 39, 3, (223-238), (2014).
- Yan Huo, Jimmy de la Torre, Estimating a Cognitive Diagnostic Model for Multiple Strategies via the EM Algorithm, Applied Psychological Measurement, 10.1177/0146621614533986, 38, 6, (464-485), (2014).
- Yuan Sun, Shiwei Ye, Huiyang Shi, Haobo Wang, Yi Sun, undefined, 2014 International Conference on Behavioral, Economic, and Socio-Cultural Computing (BESC2014), 10.1109/BESC.2014.7059511, (1-6), (2014).
- Jonathan Templin, Laine Bradshaw, The Use and Misuse of Psychometric Models, Psychometrika, 10.1007/s11336-013-9364-y, 79, 2, (347-354), (2014).
- Curtis Tatsuoka, Huiyun Tseng, Judith Jaeger, Ferenc Varadi, Mark A Smith, Tomoko Yamada, Kathleen A Smyth, Alan J Lerner, Modeling the heterogeneity in risk of progression to Alzheimer's disease across cognitive profiles in mild cognitive impairment, Alzheimer's Research & Therapy, 10.1186/alzrt168, 5, 2, (14), (2013).
- Chia-Ling Hsu, Wen-Chung Wang, Shu-Ying Chen, Variable-Length Computerized Adaptive Testing Based on Cognitive Diagnosis Models, Applied Psychological Measurement, 10.1177/0146621613488642, 37, 7, (563-582), (2013).
- Debora de Chiusole, Luca Stefanutti, Pasquale Anselmi, Egidio Robusto, Assessing Parameter Invariance in the BLIM: Bipartition Models, Psychometrika, 10.1007/s11336-013-9325-5, 78, 4, (710-724), (2013).
- Hong-Yun Liu, Xiao-Feng You, Wen-Yi Wang, Shu-Liang Ding, Hua-Hua Chang, The Development of Computerized Adaptive Testing with Cognitive Diagnosis for an English Achievement Test in China, Journal of Classification, 10.1007/s00357-013-9128-5, 30, 2, (152-172), (2013).
- Chun Wang, Mutual Information Item Selection Method in Cognitive Diagnostic Computerized Adaptive Testing With Short Test Length, Educational and Psychological Measurement, 10.1177/0013164413498256, 73, 6, (1017-1035), (2013).
- Patti West, Daisy Wise Rutstein, Robert J. Mislevy, Junhui Liu, Roy Levy, Kristen E. Dicerbo, Aaron Crawford, Younyoung Choi, Kristina Chapple, John T. Behrens, A Bayesian Network Approach To Modeling Learning Progressions, Learning Progressions in Science, 10.1007/978-94-6091-824-7, (257-292), (2012).
- Ping Chen, Tao Xin, Chun Wang, Hua-Hua Chang, Online Calibration Methods for the DINA Model with Independent Attributes in CD-CAT, Psychometrika, 10.1007/s11336-012-9255-7, 77, 2, (201-222), (2012).
- Li-hong Song, Wen-yi Wang, Hai-qi Dai, Shu-liang Ding, undefined, 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, 10.1109/FSKD.2012.6233829, (1572-1576), (2012).
- Lawrence T. DeCarlo, Recognizing Uncertainty in the Q-Matrix via a Bayesian Extension of the DINA Model, Applied Psychological Measurement, 10.1177/0146621612449069, 36, 6, (447-468), (2012).
- Dimiter M. Dimitrov, Dimitar V. Atanasov, Conjunctive and Disjunctive Extensions of the Least Squares Distance Model of Cognitive Diagnosis, Educational and Psychological Measurement, 10.1177/0013164411402324, 72, 1, (120-138), (2011).
- Young-Sun Lee, Jimmy de la Torre, Yoon Soo Park, Relationships between cognitive diagnosis, CTT, and IRT indices: an empirical investigation, Asia Pacific Education Review, 10.1007/s12564-011-9196-3, 13, 2, (333-345), (2011).
- Chun Wang, Hua-Hua Chang, Jeffery Douglas, Combining CAT with cognitive diagnosis: A weighted item selection approach, Behavior Research Methods, 10.3758/s13428-011-0143-3, 44, 1, (95-109), (2011).
- Jimmy de la Torre, The Generalized DINA Model Framework, Psychometrika, 10.1007/s11336-011-9207-7, 76, 2, (179-199), (2011).
- Young-Sun Lee, Yoon Soo Park, Didem Taylan, A Cognitive Diagnostic Modeling of Attribute Mastery in Massachusetts, Minnesota, and the U.S. National Sample Using the TIMSS 2007, International Journal of Testing, 10.1080/15305058.2010.534571, 11, 2, (144-177), (2011).
- Seongah Im, James E. Corter, Statistical Consequences of Attribute Misspecification in the Rule Space Method, Educational and Psychological Measurement, 10.1177/0013164410384855, 71, 4, (712-731), (2011).
- Lawrence T. DeCarlo, On the Analysis of Fraction Subtraction Data: The DINA Model, Classification, Latent Class Sizes, and the Q-Matrix, Applied Psychological Measurement, 10.1177/0146621610377081, 35, 1, (8-26), (2010).
- Jimmy de la Torre, A Cognitive Diagnosis Model for Cognitively Based Multiple-Choice Options, Applied Psychological Measurement, 10.1177/0146621608320523, 33, 3, (163-183), (2009).
- Seongah Im, Yue Yin, Diagnosing skills of statistical hypothesis testing using the Rule Space Method, Studies in Educational Evaluation, 10.1016/j.stueduc.2009.12.004, 35, 4, (193-199), (2009).
- Jimmy de la Torre, DINA Model and Parameter Estimation: A Didactic, Journal of Educational and Behavioral Statistics, 10.3102/1076998607309474, 34, 1, (115-130), (2009).
- Curtis Tatsuoka, Diagnostic Models as Partially Ordered Sets, Measurement: Interdisciplinary Research & Perspective, 10.1080/15366360802715510, 7, 1, (49-53), (2009).
- Ying Cheng, When Cognitive Diagnosis Meets Computerized Adaptive Testing: CD-CAT, Psychometrika, 10.1007/s11336-009-9123-2, 74, 4, (619-632), (2009).
- Jimmy de la Torre, Jeffrey A. Douglas, Model Evaluation and Multiple Strategies in Cognitive Diagnosis: An Analysis of Fraction Subtraction Data, Psychometrika, 10.1007/s11336-008-9063-2, 73, 4, (595-624), (2008).
- André A. Rupp, Jonathan L. Templin, Unique Characteristics of Diagnostic Classification Models: A Comprehensive Review of the Current State-of-the-Art, Measurement: Interdisciplinary Research & Perspective, 10.1080/15366360802490866, 6, 4, (219-262), (2008).
- Judith Jaeger, Curtis Tatsuoka, Stefanie M. Berns, Ferenc Varadi, Distinguishing Neurocognitive Functions in Schizophrenia Using Partially Ordered Classification Models, Schizophrenia Bulletin, 10.1093/schbul/sbj038, 32, 4, (679-691), (2006).
- Xueli Xu, Jeff Douglas, Computerized adaptive testing under nonparametric IRT models, Psychometrika, 10.1007/s11336-003-1154-5, 71, 1, (121-137), (2006).
- Jonathan L. Templin, Robert A. Henson, Measurement of psychological disorders using cognitive diagnosis models., Psychological Methods, 10.1037/1082-989X.11.3.287, 11, 3, (287-305), (2006).
- Jimmy de la Torre, Jeffrey A. Douglas, Higher-order latent trait models for cognitive diagnosis, Psychometrika, 10.1007/BF02295640, 69, 3, (333-353), (2004).
- Curtis Tatsuoka, Thomas Ferguson, Sequential classification on partially ordered sets, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 10.1111/1467-9868.00377, 65, 1, (143-157), (2003).
- Ralitza Gueorguieva, Eugenia Buta, Meghan Morean, Suchitra Krishnan‐Sarin, Two‐part models for repeatedly measured ordinal data with “don't know” category, Statistics in Medicine, 10.1002/sim.8739, 0, 0, (undefined).
