Developing Alice: A Scaffolding Agent for AI-Mediated Computational Thinking

Abstract

<p>Recent advances in generative AI have increasingly highlighted the transformative potential of large language models (LLMs) within educational contexts. Nevertheless, the token-based generation characteristic of LLMs often results in responses that may lack depth, thereby potentially limiting their effectiveness as scaffolding tools. This paper introduces Alice, a scaffolding agent designed to provide unsolicited hints and adaptive support in computational thinking (CT) education. Alice's effectiveness was primarily evaluated through user feedback scores and benchmarked programming tasks, while further empirical research is underway to explore qualitative evidence of scaffolding effectiveness. Alice was initially optimized for both plugged and unplugged learning scenarios using a structured system prompt informed by a hierarchical framework for AI-mediated CT and the oDSP-HF approach. Subsequent fine-tuning with a LoRA-based method reduced perplexity from 9.5 to 6.6 and improved JavaScript-to-block-based code conversion accuracy from 45.8% to 69.5%. User ratings also increased from 64% to 85%. These findings tentatively indicate that careful system prompt design, combined with targeted fine-tuning, may enhance the adaptive support and learner engagement provided by LLM-based scaffolding agents in CT education.</p>