This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
knowledge_assessment:q-matrix [2012/07/05 13:02] jpetrovic |
knowledge_assessment:q-matrix [2012/07/11 10:15] jpetrovic [How does a q-matrix look like?] |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | Q-matrix is a | + | ==== What is a q-matrix? ==== |
+ | |||
+ | Q-matrix is a matrix describing relations of questions and concepts required for their understanding. It is a domain-independent model of knowledge represented by a binary matrix showing the relationship between test items and latent or underlying attributes, or concepts. | ||
+ | |||
+ | ==== How does a q-matrix look like? ==== | ||
+ | |||
+ | Q-matrix is a M//x//N matrix, where M equals the number of questions in an assessment, and N equals the total number of concepts required for understanding all questions. The matrix element A[i,j] equals 1 if the i-th concept is required for correctly answering j-th question and 0 if the i-th concept is NOT required for correctly answering j-th question. Alternatively, matrix values can be not just {0,1}, but real numbers from the interval [0,1], describing the probability that a student who knows i-th concept will correctly answer j-th question. | ||
+ | |||
+ | An example of a q-matrix is shown in the following image. | ||
+ | |||
+ | {{:knowledge_assessment:qm.jpg}} | ||
+ | |||
+ | ==== What can I do with a q-matrix? ==== | ||
+ | |||
+ | ==== How do I create a q-matrx? ==== | ||
+ | |||
+ | |||
* "//method, which examines the inputs of many students to automatically extract relationships between questions and underlying concepts, and then uses those relationships in diagnosing and correcting student misconceptions.//" | * "//method, which examines the inputs of many students to automatically extract relationships between questions and underlying concepts, and then uses those relationships in diagnosing and correcting student misconceptions.//" | ||
* domain-independent knowledge model | * domain-independent knowledge model | ||
Line 5: | Line 22: | ||
* To build the q-matrix, experts constructed a relationship between test questions and concepts (referred to as attributes) and students taking the test were assigned knowledge states based on their test answers and the constructed q-matrix ((see Ham85 for a discussion of item-response theory)) | * To build the q-matrix, experts constructed a relationship between test questions and concepts (referred to as attributes) and students taking the test were assigned knowledge states based on their test answers and the constructed q-matrix ((see Ham85 for a discussion of item-response theory)) | ||
- | {{:knowledge_assessment:qm.jpg}} | + | |
The goal of q-matrix construction is to extract underlying, or latent, variables, which account for studentsí differential performance on questions. | The goal of q-matrix construction is to extract underlying, or latent, variables, which account for studentsí differential performance on questions. | ||
Line 15: | Line 32: | ||
Factor analysis: | Factor analysis: | ||
How to automatically determine concepts? Using covariance matrix. Number of concepts should be smaller than number of questions. Still, this methos has proven to be less fault tollerant. | How to automatically determine concepts? Using covariance matrix. Number of concepts should be smaller than number of questions. Still, this methos has proven to be less fault tollerant. | ||
+ | |||
+ | ==== Q-matrix method ==== | ||
+ | |||
+ | The q-matrix method is a simple hill-climbing algorithm that creates a matrix | ||
+ | representing relationships between concepts and questions directly. The algorithm varies | ||
+ | c, the number of concepts, and the values in the q-matrix, minimizing the total error for | ||
+ | all students for a given set of n questions. To avoid of local minima, each hill-climbing | ||
+ | search is seeded with different random Q-matrices and the best of these is kept. | ||
+ | |||
+ | When forming a correlation matrix, we lose individual student data in favor of calculating average relationships between questions. The q-matrix method is optimized to assign each student the most appropriate knowledge state, using all available response data for each student. | ||
+ | |||
+ | As Sellers found in her research, the results obtained through | ||
+ | q-matrix analysis seem to describe relationships among variables in interpretable ways. | ||
+ | Factor analysis and principal components analysis, on the other hand, do not readily offer | ||
+ | interpretable results. | ||
+ | |||
+ | Later researchers found that, although the q-matrix model was a good way to compare student data to a concept model, expert-constructed q-matrices did not correspond to student data any better than random q-matrices did. | ||
+ | |||
+ | the findings in Brewer's previous research, which found that the factor analysis method performed poorly in comparison with the q-matrix | ||
+ | method when fewer observations were available. |