knowledge_assessment:q-matrix
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
knowledge_assessment:q-matrix [2012/07/11 10:31] jpetrovic [How does a q-matrix look like?] |
knowledge_assessment:q-matrix [2012/07/11 11:08] jpetrovic [How do I create a q-matrx?] |
||
|---|---|---|---|
| Line 15: | Line 15: | ||
| {{ :knowledge_assessment:qm.jpg }} | {{ :knowledge_assessment:qm.jpg }} | ||
| - | == Understanding the matrix values and implications == | + | == Understanding matrix values and implications == |
| From the matrix, one can read that knowledge of concept 1 is required in order to answer correctly questions 3, 4 and 5. One can also read that questions 1 and 2 test only the knowledge of concept 2. | From the matrix, one can read that knowledge of concept 1 is required in order to answer correctly questions 3, 4 and 5. One can also read that questions 1 and 2 test only the knowledge of concept 2. | ||
| Line 22: | Line 22: | ||
| ==== What can I do with a q-matrix? ==== | ==== What can I do with a q-matrix? ==== | ||
| + | Q-matrix can be used for understanding students' performance. Due to various knowledge and assessment characteristics, students' responses rarely match //ideal responses// generated from the matrix. Still, by assigning the closest //ideal response// to a student's response vector, it can be assumed which concepts the student does, and which he does not know. This information can be used in order to direct him in further learning. | ||
| ==== How do I create a q-matrx? ==== | ==== How do I create a q-matrx? ==== | ||
| + | The goal of q-matrix construction is to extract underlying, or latent, variables, which account for students' differential performance on questions. A q-matrix can be created in two ways: | ||
| + | * by having the domain experts analyze assessment items (questions) and define the concept (often referred to as attribute) corresponding to each item and thereby construct the q-matrix. In this case all the concepts in the q-matrix are exactly defined and labeled as elements of the domain. Q_matrix and student's knowledge is therefore easily interpretable. Problems: "//a q-matrix is a much more abstract measure of the relationships of questions to concepts. We might assume that the questions designed to test students are a more accurate reflection of the teaching objectives than an abstract construct which relates questions to underlying concepts.//" | ||
| + | * by automatically extracting the matrix directly from obtaines students' performance on the test items. in this case, the number of concepts is determined by the algorithm (and is usually much smaller than the number of assessment items) and their labels are automatically assigned (and do not reffer to exact domain concepts). Experts can examine the resulting q-matrix to ensure that the extracted relationships seem to be valid, and then use that q-matrix to guide the generation of new problems. | ||
| - | * "//method, which examines the inputs of many students to automatically extract relationships between questions and underlying concepts, and then uses those relationships in diagnosing and correcting student misconceptions.//" | + | Factor analysis can be considered as an alternative to q-matrices. Concepts are in that case automatically determined using covariance matrix. Number of concepts should be smaller than number of questions. Still, this methos has proven to be less fault tollerant. |
| - | * domain-independent knowledge model | + | |
| - | * originally a binary matrix showing the relationship between test items and latent or underlying attributes, or concepts | + | |
| - | * To build the q-matrix, experts constructed a relationship between test questions and concepts (referred to as attributes) and students taking the test were assigned knowledge states based on their test answers and the constructed q-matrix ((see Ham85 for a discussion of item-response theory)) | + | |
| - | The goal of q-matrix construction is to extract underlying, or latent, variables, which account for studentsí differential performance on questions. | + | ((see Ham85 for a discussion of item-response theory)) |
| - | Approaches: | ||
| - | * Hand construction of the q-matrix by experts' assigning concepts to questions and then comparing student answers to closest matrix responses. Problems: a q-matrix is a much more abstract measure of the relationships of questions to concepts. We might assume that the questions designed to test students are a more accurate reflection of the teaching objectives than an abstract construct which relates questions to underlying concepts. | ||
| - | * The alternative to this strategy is to design a method to extract a q-matrix, which explains student behavior, and reveals the underlying relationships between questions. Experts can examine the resulting q-matrix 25 to ensure that the extracted relationships seem to be valid, and then use that q-matrix to guide the generation of new problems. | ||
| - | Factor analysis: | ||
| - | How to automatically determine concepts? Using covariance matrix. Number of concepts should be smaller than number of questions. Still, this methos has proven to be less fault tollerant. | ||
| ==== Q-matrix method ==== | ==== Q-matrix method ==== | ||
knowledge_assessment/q-matrix.txt · Last modified: 2023/06/19 18:03 (external edit)
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
