Differences

This shows you the differences between two versions of the page.

--- knowledge_assessment:q-matrix [2012/07/11 10:30]
jpetrovic [How does a q-matrix look like?]
+++ knowledge_assessment:q-matrix [2023/06/19 18:03]
@@ Line 1: / Line 1: @@
-==== What is a q-matrix? ====
-Q-matrix is a matrix describing relations of questions and concepts required for their understanding. It is a domain-independent model of knowledge represented by a binary matrix showing the relationship between test items and latent or underlying attributes, or concepts.
-==== How does a q-matrix look like? ====
-Q-matrix is a M//x//N matrix, where M equals the number of questions in an assessment, and N equals the total number of concepts required for understanding all questions. The matrix element A[i,j] equals 1 if the i-th concept is required for correctly answering j-th question and 0 if the i-th concept is NOT required for correctly answering j-th question. Alternatively, matrix values can be not just {0,1}, but real numbers from the interval [0,1], describing the probability that a student who knows i-th concept will correctly answer j-th question.
-An example of a q-matrix is shown in the following image.
-{{  :knowledge_assessment:qm.jpg  }}
-From the matrix, one can read that knowledge of concept 1 is required in order to answer correctly questions 3, 4 and 5. One can also read that questions 1 and 2 test only the knowledge of concept 2.
-Furthermore, one could also say that the //ideal response// of a student taking the test formed of those 5 questions who knows only concept 1 should be "00001". This is so since he does not know concept 2 which is required for questions 1 and 2 (therefore the leading "00"). Yet this concept is also required for correct ansqering of questions 3 and 4 so he can not answer those questions neither (therefore the following "00"). Finally, 5th question requires only knowledge about the concept 1 so the student can answer this question correctly (therefore the ending "1", forming all together "00001").
-==== What can I do with a q-matrix? ====
-==== How do I create a q-matrx? ====
-   * "//method, which examines the inputs of many students to automatically extract relationships between questions and underlying concepts, and then uses those relationships in diagnosing and correcting student misconceptions.//"
-   * domain-independent knowledge model
-   * originally a binary matrix showing the relationship between test items and latent or underlying attributes, or concepts
-   * To build the q-matrix, experts constructed a relationship between test questions and concepts (referred to as attributes) and students taking the test were assigned knowledge states based on their test answers and the constructed q-matrix ((see Ham85 for a discussion of item-response theory))
-The goal of q-matrix construction is to extract underlying, or latent, variables, which account for studentsí differential performance on questions.
-Approaches:
-   * Hand construction of the q-matrix by experts' assigning concepts to questions and then comparing student answers to closest matrix responses. Problems: a q-matrix is a much more abstract measure of the relationships of questions to concepts. We might assume that the questions designed to test students are a more accurate reflection of the teaching objectives than an abstract construct which relates questions to underlying concepts.
-   * The alternative to this strategy is to design a method to extract a q-matrix, which explains student behavior, and reveals the underlying relationships between questions. Experts can examine the resulting q-matrix 25 to ensure that the extracted relationships seem to be valid, and then use that q-matrix to guide the generation of new problems.
-Factor analysis:
-How to automatically determine concepts? Using covariance matrix. Number of concepts should be smaller than number of questions. Still, this methos has proven to be less fault tollerant.
-==== Q-matrix method ====
-The q-matrix method is a simple hill-climbing algorithm that creates a matrix
-representing relationships between concepts and questions directly. The algorithm varies
-c, the number of concepts, and the values in the q-matrix, minimizing the total error for
-all students for a given set of n questions. To avoid of local minima, each hill-climbing
-search is seeded with different random Q-matrices and the best of these is kept.
-When forming a correlation matrix, we lose individual student data in favor of calculating average relationships between questions. The q-matrix method is optimized to assign each student the most appropriate knowledge state, using all available response data for each student.
-As Sellers found in her research, the results obtained through
-q-matrix analysis seem to describe relationships among variables in interpretable ways.
-Factor analysis and principal components analysis, on the other hand, do not readily offer
-interpretable results.
-Later researchers found that, although the q-matrix model was a good way to compare student data to a concept model, expert-constructed q-matrices did not correspond to student data any better than random q-matrices did.
-the findings in Brewer's previous research, which found that the factor analysis method performed poorly in comparison with the q-matrix
-method when fewer observations were available.

Learning Theories

User Tools

Site Tools

Differences

Page Tools