Research area

Writing assessment beyond AI detection

Focus

Generative AI has pushed many writing programs toward detection and surveillance. This strand argues that detection is unreliable, that students assessed by it learn to avoid being flagged rather than to learn, and that it answers the wrong question. Treating generative AI instead as a mediational tool whose value depends on how learners engage with it, the work proposes a portfolio-based framework that makes the writing process, including AI use, visible and assessable.

Beyond Detection: the framework

Beyond Detection critiques AI-detection-based assessment on empirical and conceptual grounds and reframes generative AI through Vygotskian sociocultural theory and Freirean critique. The proposed portfolio includes planning materials, multiple drafts, a structured AI-use process memo, and a final submission, assessed along four dimensions: product quality, depth of revision, quality of reflection, and transparency of tool use. The aim is to shift evaluation from surveillance toward dialogic, process-oriented accountability.

Beyond Detection: Rethinking writing assessment in the age of generative AI (2025 – present), independent research project.

Do AI-text detectors hold up against rewriting? An empirical study

A second strand tests whether AI-text detection verdicts are reliable enough to serve assessment, drawing on 15,985 pairs of human-written and AI-generated texts in two domains: 11,030 from pre-2021 arXiv articles and 4,955 from student essays in seven corpora. A recognizer and a rewriter trained from the same paired data show that, at thresholds flagging 1% of human writing, each of three detector families judged 81–86% of the rewritten AI texts as human, so for the detectors and corpora tested, detection verdicts are not reliable against such rewriting.

Xia, Y. (in progress). Beyond detection: Auditing AI-text detectors with a recognizer and a rewriter trained from the same paired data.

From AI Detection to Portfolio Assessment: a funded pilot

The framework is now being tested in a one-semester pilot in a required first-year composition course at Texas State University, funded by a Spark Grant with Yuwei Xia as Principal Investigator. The study replaces AI detection with the portfolio framework, collects student portfolios, AI-use process memos, and instructor feedback, and evaluates learner outcomes across the four assessment dimensions.

From AI Detection to Portfolio Assessment: A One-Semester Pilot Study in First-Year Writing (2026 – 2027), Spark Grant, Texas State University, Principal Investigator.

Structured mediation and learner agency in AI-supported writing

A companion line of work asks how structured mediation shapes learner agency when generative AI is part of the writing process, and how mediation is distributed across teacher-designed prompts, AI feedback, and the teacher.

Xia, Y., Xu, Q., & Zeng, Z. (under review). Beyond the tool: A sociocultural perspective on structured mediation and the development of learner agency in AI-assisted L2 writing.
Xia, Y., & Xu, Q. (2026). Structured mediation and learner agency in AI-supported L2 writing: A sociocultural perspective on generative AI feedback. Paper presented at the AAAL Conference, Chicago.
Xu, Q., Xia, Y., Lu, X., & Zeng, Z. (2026). Distributed mediation in AI-assisted dynamic assessment: Learner-AI interaction and revision across EFL writing continuation tasks. Language Learning & Technology, 30(1). Corresponding author.
Luo, Y., Xia, Y., & Lu, X. (2025). A systematic review of the role of artificial intelligence in second language writing education. Digital Studies in Language and Literature, 2(2), 302–325.

WriteMed, an AI mediator that keeps the learner as the author of each revision · Dynamic assessment research