Skip to content
HeuriSight home xResearch

Research record

Research record

BKT and relational-model prior-art boundary note

A boundary note on Bayesian Knowledge Tracing and association-edge estimation

Published
Status
Public scholarly note accompanying HS-WP-2026-09
Evidence status
Literature comparison; the exact human–AI construct remains unevaluated
Scope
Public construct only; implementation details are neither needed nor disclosed
Competing interest
The author is associated with HeuriSight, which is developing the mechanism discussed. No product effect or construct-validity result is claimed.
Sections of this document
  1. Bottom line
  2. Exactly what Bayesian Knowledge Tracing does
  3. Where standard BKT and the proposed construct differ
  4. Bayesian estimation of relations among knowledge elements
  5. The untested integration—and what would test it
  6. References

Bottom line

Established methods cover the broad methodological components; the exact human–AI reasoning-process construct was not located in this review.

Bayesian Knowledge Tracing. BKT has maintained a person-specific, longitudinal Bayesian representation of latent learning from observed performance since Corbett and Anderson's 1994/1995 work. An expert model supplies production rules or skills; each observed opportunity changes the estimated probability that the relevant rule is learned. An expert-defined model, learner-specific overlay, sequential behavioral evidence, Bayesian accumulation, and changing representation of learning are therefore established precedents.

The proposed construct differs from standard BKT in both estimand and evidence. Standard BKT ordinarily estimates mastery of one skill at a time from correctness on tagged opportunities. The proposed graph represents relations among expert heuristics from decisions observed during human–AI work. Co-use is the observation substrate; the intended object is the changing organization of reasoning-in-use. A smoothed event rate is not itself an association unless opportunity and marginal frequencies are addressed. The difference is real, while the broad principle of Bayesian learner modelling is well established.

Relations among knowledge elements. Prior work estimates these relations, including with Bayesian methods. Käser and colleagues model a supplied skill topology with dynamic Bayesian networks. COMMAND jointly learns a population skill-prerequisite graph and student models from response data. Han, Yoon, and Yoo use Bayesian estimation and nested model comparison to evaluate prerequisite edges. U-INVITE is closer still at the generic cognitive level: it Bayesianly infers an individual's concept–concept network from observed retrieval sequences. Epistemic Network Analysis estimates changing connections among coded knowledge, skills, values, and practices from discourse and action, although it is not Bayesian. Ji and colleagues (2026) learn a weighted knowledge-component adjacency matrix from response sequences in a graph-based knowledge-tracing model and simulate heuristic effects by modifying selected weights and node representations. Most directly on the educational-process construct, Bernholt, Lossjew, and Gombert (2026) build individual, longitudinal knowledge-element networks from what students enact in classroom tasks and relate network development to a separately scored, immediate end-of-unit posttest, although their edges are co-occurrences rather than Bayesian posteriors.

No peer-reviewed study located in this search combines all of the following: an individual learner's graph of heuristic-to-heuristic associations; repeated updating during human–AI work; an available-versus-enacted contrast; and probabilistic estimation of changing person-specific relations. That integration may remain untested. Every major component around it already has antecedents.

The resulting research question is therefore narrow:

Does a learner-specific, longitudinal probabilistic graph of expert-heuristic relations add valid information about changing reasoning processes beyond exposure, use counts, node models, and non-Bayesian networks—and does that process representation later predict independent learning outcomes?

Neither part has yet been answered. The first concerns learning-in-process; the second concerns acquired capacity, transfer, and durability.

Exactly what Bayesian Knowledge Tracing does

Corbett and Anderson's ACT Programming Tutor used an expert production-rule model as an “ideal student model.” Model tracing compared a student's action with the rules that could generate a correct step. Knowledge tracing added a probabilistic overlay for the individual learner. For each rule or skill, canonical BKT assumes:

  • a binary hidden state: learned or unlearned;
  • an initial probability of mastery;
  • a probability of transition from unlearned to learned after a practice opportunity;
  • a probability of a correct guess while unlearned; and
  • a probability of a slip while learned.

Standard BKT does not allow forgetting. After a correct or incorrect response, Bayes' rule revises the probability that the skill was already learned; the acquisition transition then adds the modeled possibility that the opportunity produced learning. The resulting state predicts the next response and can support a mastery decision. “Bayesian” here usually refers to filtering the hidden mastery state conditional on parameters that have been estimated from data; it need not mean that the system maintains a full Bayesian posterior over those parameters.

This distinction matters. An observation changes belief about what the learner knew before the response. The transition parameter separately encodes a chance that practice caused acquisition. The hidden state is called “learned,” but neither the state nor its transition is directly observed.

The original paper evaluated four programming-tutor studies. Its evidence primarily concerned calibration, prediction, and mastery-based practice. In Experiment 2, internal expected-versus-observed goal accuracy correlated at r = .75, with mean absolute error .07; a refit to the same sample reached r = .90 and was explicitly treated as an upper bound. In Experiment 4, internal prediction across 214 goals reached r = .71 with mean absolute error .06. On the three external tests, predicted–observed correlations were .24, .36, and .66; only the last was reported significant. Fifty-six per cent of the knowledge-tracing group reached a 90% test criterion, compared with 24% of a comparison group (z = 2.21, p < .05), but the knowledge-tracing group completed about 76% more tutor exercises. The design therefore does not isolate the validity of the latent state from the additional practice that state-based tutoring assigned.

Later BKT work altered emissions, parameters, and skill structure. Baker, Corbett, and Aleven's contextual estimates of slip and guess improved A′ from .66 to .75 and response correlation from .29 to .43 in 171,987 first-step actions from 232 middle-school students. Yudelson, Koedinger, and Gordon's individualized BKT produced small improvements in next-response prediction when the learning parameter varied by learner. These are useful performance results. They do not turn the hidden state into a direct observation of retained or transferable competence.

The identifiability literature also requires precision. Beck and Chang (2007) showed that different parameterizations could fit the same aggregate learning curve while implying different hidden knowledge, and reported AUC .620 ± .002 with Dirichlet priors versus .614 ± .002 without them. Doroudi and Brunskill (2017) later showed that this was not a proof that the full BKT model is structurally non-identifiable: under mild, non-degenerate conditions, the joint distribution of response sequences identifies the standard two-state model. The careful synthesis is that canonical BKT is generically identifiable, while finite samples, local optima, poor skill tags, and semantically degenerate solutions still threaten interpretation. Statistical identification does not establish that the hidden state is learning.

Where standard BKT and the proposed construct differ

DimensionStandard BKTProposed construct at the publishable levelEvidentiary relationship
Expert representationSkills or production rulesInspectable expert heuristicsDifferent granularity; established principle
Learner stateProbability that each skill is masteredStrength and uncertainty of relations in observed reasoning-in-useDistinct process object
ObservationUsually correct/incorrect performance on a tagged opportunityHeuristics available, selected, and enacted in interactionPotentially distinct evidence model
UpdateHidden-state Bayes filter plus modeled acquisition transitionProbabilistic evidence about a relation accumulated over repeated co-use, with older evidence weighted lessDifferent estimator and target; Bayesian accumulation is established
Relations among elementsAbsent from canonical one-skill BKTCentral object of the representationNot answered by canonical BKT; answered in part by later work
Meaning of movementIncreased modeled probability of masteryChanged pattern of heuristic selection and coordinationProcess change is testable; acquired capacity is a further inference

The candidate-versus-enacted contrast also has a precursor inside cognitive tutors. Model tracing begins with rules applicable in the current problem state and credits the rule that matches the student's action. Canonical BKT does not count unchosen applicable rules as failures or estimate pairwise association edges among them. Even so, “available rules versus the rule evidenced by an action” is not a new measurement idea.

Bayesian estimation of relations among knowledge elements

Bayesian and educational antecedents

Käser et al. (2014) explicitly framed their work as going “beyond knowledge tracing.” Their dynamic Bayesian networks model several latent skills jointly and attach conditional parameters to a supplied skill topology. Across five datasets, illustrative improvements included AUC .5975 to .7093 for number representation and .5996 to .6916 for subtraction. This is Bayesian modeling on a fixed domain skill graph, not discovery of an online learner-specific graph from heuristic enactment.

Chen, González-Brenes, and Tian's COMMAND (2016) goes further. Structural expectation–maximization learns one population Bayesian-network structure among latent skills while estimating student mastery from item responses. In 10-fold cross-validation, COMMAND reached AUC .803 (reported 95% CI, ± .008) versus .791 (± .007) for a fully connected network on a 1,720-student, six-skill mathematics dataset (p = .0022), and .775 (± .007) versus .765 (± .008) on a 1,245-student, seven-skill dataset (p = .01). The tests followed instruction in the relevant skills, analyses used filtered complete responses, and reversible edges required a substantive assumption for prerequisite orientation. The improvement was modest, but the comparison is clear: a domain skill-dependency graph and student model were jointly inferred from performance data.

Han, Yoon, and Yoo (2017) used MCMC estimation and nested comparisons to discover skill-prerequisite edges. Across five four-skill simulations with balanced Q-matrices, 1,000 students, and slip and guess drawn from Uniform(0, .05), whole-structure recovery ranged from .816 to .926 and true-edge recovery from .937 to .962. In real data from 936 eighth-graders answering 16 mathematics items, the model reproduced the expert-proposed edges and proposed one additional edge. This is direct Bayesian estimation of a cohort/domain prerequisite structure from correctness, not a changing personal graph from observed heuristic use.

Individual association graphs from retrieval

Zemla and Austerweil's U-INVITE (2018) is the closest generic statistical antecedent. It treats concepts as nodes and semantic associations as edges, uses a psychologically specified retrieval process, and infers group and individual networks from semantic-fluency sequences. Fifty participants supplied three lists in each of three domains, and 101 separate raters judged similarity. The method combines the likelihood of observed lists under a censored random walk with a prior over binary, symmetric, undirected networks. Validity is conditional on that retrieval model and multiple lists per individual.

U-INVITE establishes two relevant precedents: Bayesian inference of an individual association graph and inference of such a graph from observed retrieval behavior. It is nevertheless cross-sectional semantic-memory estimation, not a model of expert heuristics, human–AI interaction, available-versus-enacted evidence, longitudinal education, or durable learning.

Dynamic knowledge networks from enacted elements

Epistemic Network Analysis (Shaffer, Collier, & Ruis, 2016) identifies and quantifies connections among coded elements in discourse and action. Its networks were designed to represent associations among knowledge, skills, values, habits of mind, and other aspects of complex thinking, and can quantify change over time. ENA is not Bayesian, and its edge is a windowed co-occurrence rather than a posterior belief that one heuristic feeds another. It nevertheless establishes the general precedent that patterns of enacted cognitive elements can be represented as a changing learner network.

Bernholt, Lossjew, and Gombert (2026) are closer to the representational claim. They automatically scored responses and artifacts from 300 students in grades 11–13 across 15 classes during a 10–12-week chemistry unit and built individual networks from 17 knowledge elements enacted across 86 tasks. Against a separately scored, immediate end-of-unit posttest, a model combining network features reported R² = .51 (adjusted .47); size alone reported .44 (.42), density .40 (.38), and connectedness .31 (.28). The regression degrees of freedom imply complete-case analytic samples of about 202–204. The relation between the phase-level area under the density trajectory and performance changed sign: standardized β = −.46 in phase 1 and β = .90 in phase 2. This was a time-integrated summary, not a coefficient for density at one observation. The pretest was not included in the reported models; the design was observational and had no control group. The study establishes a criterion relation for an evolving individual co-enactment graph. It does not establish learning, transfer, durability, measurement invariance, or causality.

Ji, Wang, Wu, and Zhou's Human-Machine Collaboration-based Knowledge Tracing model (2026) supplies another close component precedent. Its spatiotemporal graph-convolution module learns a weighted adjacency matrix among knowledge components from correct-or-incorrect response sequences and updates temporal knowledge-state representations. The paper reports AUC .8593 for the full model and .8194 without its active-learning component. Its later heuristic analyses do not observe learners invoking a rule: they impose enhancement or attenuation on selected edges and node representations to simulate availability and representativeness. The method is non-Bayesian, the relation matrix is not reported as a longitudinal person-specific heuristic graph, and the criterion is response prediction rather than independent acquisition, transfer, or durability. It therefore occupies learned knowledge-element edges and heuristic-labeled graph manipulation, while leaving the focal observation model and construct-validity problem open. Ji et al. (2026)

The untested integration—and what would test it

The combination not located in this review is a study specification, not a demonstrated contribution.

Evidence would need to establish, in order:

  1. Attribution: independent human reference coding can distinguish learner enactment from content supplied or enacted by the AI.
  2. Opportunity: non-use is counted only when a task genuinely called for the heuristic and the trace made its use observable. Mere retrieval is not sufficient opportunity.
  3. Reliability: heuristic and edge evidence is reproducible across coders, models, tasks, and trace formats, with abstention where evidence is insufficient.
  4. Discriminant validity: edge change is not explained by exposure, each heuristic's marginal frequency, assistant verbosity, learner verbosity, session count, task difficulty, retriever behavior, or repeated presentation.
  5. Convergent and incremental validity: the graph relates to independent structural-knowledge and performance measures and adds information beyond node mastery, simple use counts, BKT/PFA predictions, and non-Bayesian co-occurrence networks.
  6. Learning-process interpretation: equivalent repeated tasks show changing selection or coordination rather than only growing certainty about a stable pattern.
  7. Acquisition, transfer, and durability: for the stronger outcome interpretation, learners later invoke the relation without supply, in a structurally related new situation, and after a meaningful delay.
  8. Fairness and generalization: coverage, error, and criterion relations remain stable enough to interpret across learner groups, domains, accessibility modes, and interaction styles.

The current evidence supports a proposed longitudinal record of reasoning-in-use. Its interpretation as learning-in-process requires temporal and discriminant validation; its interpretation as acquired, transferable, or durable capacity requires the independent criteria above.

References

  • Baker, R. S. J. d., Corbett, A. T., & Aleven, V. (2008). More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian Knowledge Tracing. Intelligent Tutoring Systems, 406–415. https://doi.org/10.1007/978-3-540-69132-7_44
  • Beck, J. E., & Chang, K.-m. (2007). Identifiability: A fundamental problem of student modeling. User Modeling 2007, 137–146. https://doi.org/10.1007/978-3-540-73078-1_17
  • Bernholt, S., Lossjew, J., & Gombert, S. (2026). Analyzing students' conceptual understanding over the course of a teaching unit: Tracking changes in knowledge structures over time. Unterrichtswissenschaft. https://doi.org/10.1007/s42010-026-00244-0
  • Chen, Y., González-Brenes, J. P., & Tian, J. (2016). Joint discovery of skill prerequisite graphs and student models. Proceedings of the 9th International Conference on Educational Data Mining, 46–53. Primary paper
  • Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253–278. https://doi.org/10.1007/BF01099821
  • Doroudi, S., & Brunskill, E. (2017). The misidentified identifiability problem of Bayesian Knowledge Tracing. Proceedings of the 10th International Conference on Educational Data Mining, 143–149. Primary paper
  • Han, S.-Y., Yoon, J., & Yoo, Y. J. (2017). Discovering skill prerequisite structure through Bayesian estimation and nested model comparison. Proceedings of the 10th International Conference on Educational Data Mining, 398–399. Primary paper
  • Ji, W., Wang, H., Wu, Q., & Zhou, G. (2026). Knowledge tracing model based on human-machine collaboration: An analysis of the impact of perceptual ambiguity, selective attention, and heuristic judgment on learning performance. Journal of Big Data, 13, Article 47. https://doi.org/10.1186/s40537-026-01385-w
  • Käser, T., Klingler, S., Schwing, A. G., & Gross, M. (2014). Beyond knowledge tracing: Modeling skill topologies with Bayesian networks. Intelligent Tutoring Systems, 188–198. https://doi.org/10.1007/978-3-319-07221-0_23
  • Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on Epistemic Network Analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10.18608/jla.2016.33.3
  • Yudelson, M. V., Koedinger, K. R., & Gordon, G. J. (2013). Individualized Bayesian Knowledge Tracing models. Artificial Intelligence in Education, 171–180. https://doi.org/10.1007/978-3-642-39112-5_18
  • Zemla, J. C., & Austerweil, J. L. (2018). Estimating semantic networks of groups and individuals from fluency data. Computational Brain & Behavior, 1(1), 36–58. https://doi.org/10.1007/s42113-018-0003-7