Academic Positions

  • Present 2015

    Postdoctoral Research Associate

    University of Pittsburgh, Learning Research & Development Center (LRDC) and University Center for Teaching and Learning

  • 2015 2013

    Assistant Director

    University of Toronto, Technologies for Aging Gracefully Lab (TAGlab)

  • 2014

    Visiting Researcher

    Kwansei Gakuin University, Graduate School of Language, Communication, and Culture

  • 2014 2013

    Visiting Researcher

    University of Birmingham (UK), School of Electronic, Electrical and Systems Engineering

Education & Training

  • Ph.D. 2016

    Ph.D. in Computer Science (Human-Computer Interaction - Educational Technology)

    University of Toronto

  • M.Sc.2010

    Master of Science in Computer Science (Artificial Intelligence - Educational Technology)

    University of Saskatchewan

  • B.Sc.2007

    Bachelor of Science in Computer Science Honors, Russian Honors

    University of Saskatchewan

Honors, Awards and Grants

  • 2014
    Robert E. Lansdale / Okino Computer Graphics Graduate Fellowship in DGP for the Department of Computer Science (Academic), University of Toronto
  • 2014
    Doctoral Completion Award (Academic), University of Toronto
  • 2014
    User Modeling, Adaptation and Personalization (UMAP) Best Poster Award (Research), User Modeling Inc.
  • 2013-2014
    Walter C. Sumner Memorial Fellowship (Academic), University of Toronto
  • 2012, 2014
    Ontario Graduate Scholarship (Academic), University of Toronto
  • 2013
    W. Garfield Weston Fellowship (Academic), University of Toronto
  • 2012
    ACM-W Scholarship (Travel), International Conference on Intelligent User Interfaces (IUI)
  • 2010
    Wolfond Fellowship (Academic), University of Toronto
  • 2010
    Best Graduate Student (Academic & Leadership), Dept. of Computer Science – University of Saskatchewan (medal)
  • 2008
    Google Canada Anita Borg Scholarship (Academic & Leadership), University of Saskatchewan
  • 2008
    Joseph Zlacky Scholarship (Academic), University of Saskatchewan
  • 2007
    University of Saskatchewan College of Arts & Science Study Abroad Award (Academic), Smolny Institute (Смольный институт) – St. Petersburg State University (Санкт-Петербургский государственный университет, СПбГУ)
  • 2007
    Mendelle and John Woodley Award (Academic), University of Saskatchewan
  • 2007
    CS Alumni Award (Leadership), Dept. of Computer Science – University of Saskatchewan
  • 2006
    John Labatt Scholarship (Academic & Leadership), University of Saskatchewan
  • 2006
    Shell Canada Scholarship (Academic), University of Saskatchewan

General Purpose Educational Technology Development and Evaluation Projects

  • iHelp Discussions and Pragmatics Project

    iHelp Discussions was a discussion forum that was used by students of all levels within the Department of Computer Science at the University of Saskatchewan. We are analyzing the language used within iHelp Discussions in an attempt to find patterns in language use and determine how these patterns change over time. More specifically, we are looking to see if students adopt the language of their community of practice. We are also investigating how power structures within the academic setting affect the formality and other linguistic features of student posts within this forum.

    More >>

    Project Lead: Carrie Demmans Epp

    Collaborators: Christopher Brooks, Greg Logan, Jennifer Seaton, Gord McCalla

  • Recollect

    Recollect is a video lecture recording and playback system. It was used to support many University of Saskatchewan undergraduate courses across a variety of disciplines. The courses that used Recollect were primarily lecture-based.

    More >>

    I performed the qualitative evaluation of open-ended survey responses from over 1000 students who had used Recollect.

    This work led to the development of models of system usage. Chris continued this work and identified the relationships between system usage patterns and learning outcomes. We are now expanding this work to look at the usage patterns of English-language learners.

    Project Leads: Christopher Brooks (University of Michigan) and Carrie Demmans Epp

    Collaborators: Jim Greer (ARIES Laboratory, U of Saskatchewan), Greg Logan

    Publications:

    Christopher Brooks, Carrie Demmans Epp, Greg Logan, & Jim Greer. (2011) The Who, What, When, Where, and Why of Lecture Capture. Learning Analytics and Knowledge 2011, Banff, Alberta, Canada 86-92

  • iHelp Courses Project

    iHelp Courses is a learning content management system that was developed by the Department of Computer Science and ARIES laboratory at the University of Saskatchewan. It was used to deliver materials for online and blended courses within the Department of Computer Science.

    More >>

    My involvement with this project included course content development and organization (content packaging and learning object preparation), as well as the analysis of learner behavior based on the logs of their interactions with the system.

    Project Leads: Monisha Shukla and Jim Greer (ARIES Laboratory, U of Sask.)

  • MyVoice

    MyVoice is a location-aware communication support tool that runs on mobile phones. We studied its use within special education contexts to see if it could better support student learning activities.

    More >>

    I was responsible for duties surrounding the automated logging of student activities within the application. I was also responsible for the processing and later analyses of this log information.

    Project Leads: Rachelle Campigotto and Rhonda McEwen

    Collaborators: Karyn Moffatt, Rachelle Campigotto, Rhonda McEwen, Ronald M. Baecker, MyVoice Inc.

    Publications:

    Carrie Demmans Epp, Rhonda McEwen, Rachelle Campigotto, & Karyn Moffatt. (2015). Information Practices and User Interfaces: Student Use of an iOS Application in Special Education. Education and Information Technologies. Online First.

    Rachelle Campigotto, Rhonda McEwen, & Carrie Demmans Epp. (2013) Especially Social: Exploring the Use of an iOS Application in Special Needs Classrooms. Computers & Education, 60(1), 74-86.

    Carrie Demmans Epp, Rachelle Campigotto, Alexander Levy, & Ron Baecker. (2011). MarcoPolo: Context-Sensitive Mobile Communication Support. In Proceedings of FICCDAT: RESNA/ICTA, 2011, Toronto, Ontario, Canada. (4 pages)

  • Selecting Oral Assessment Tasks for the University Undergraduate Classroom

    This project involved the development of a taxonomy and program that can be used to help undergraduate instructors select appropriate oral assessment tasks for their classrooms. The tool takes an instructor’s institutional context, teaching practices, and assessment goals into account and recommends the types of oral assessment tasks that are most appropriate to that instructor’s situation. The tool also makes suggestions that could improve the usefulness and fairness of different types of oral assessment tasks. Beyond this, it asks several questions that are intended to encourage reflection and help the instructor with his/her course planning.

    More >>

    I helped develop the taxonomy and programmed the application.

    Project Leads: Carrie Demmans Epp, Gina Park, and Christopher Plumb

    Publications:

    Carrie Demmans Epp, Gina Park, & Christopher Plumb. (2015). Developing an adaptive tool to select, plan, and scaffold oral assessment tasks for undergraduate courses. Educational Technology Review & Development (ETRD). 63(3), 475-498.

  • PeppeR

    PeppeR is an online learning environment that provides collaborative work spaces to support student activities and learning. The PeppeR system visually resembles a discussion forum but it has many features that specifically aim to support knowledge building.

    More >>

    I led the heuristic evaluation of the graphical user interface with the aim of improving the system's usability. I also helped doctoral students with their quantitative analysis of usage logs and questionnaire responses.

    Project Leads: Jim Hewitt and Clare Brett

    Collaborators: Jim Hewitt, Krystle Phirangee

    Publications:

    Krystle Phirangee, Carrie Demmans Epp, & Jim Hewitt. (2017). Community in Online Courses: The Roles Played By Course Length, Course Facilitation Methods, and Student Actions. American Educational Research Association (AERA) Annual Meeting 2017, San Antonio, TX, USA.

    Carrie Demmans Epp, Krystle Phirangee, & Jim Hewitt. (2017). The Interplay Between Students’ Usage of Pronouns and Community Levels in Online Courses. American Educational Research Association (AERA) Annual Meeting 2017, San Antonio, TX, USA.

    Krystle Phirangee, Carrie Demmans Epp, & Jim Hewitt. (2016). Exploring the relationships between facilitation methods, students’ sense of community and their online behaviours. Online Learning Journal, special issue on Learning Analytics, 20(2), 134-154.

    Krystle Phirangee, Carrie Demmans Epp, & Jim Hewitt. (2016). Exploring How Peer and Instructor Facilitation Relates to Students’ Activity Patterns in Online Learning Communities. American Educational Research Association (AERA) Annual Meeting 2016, Washington, D.C., USA.

  • Massive Open Online Courses (MOOCs)

    We are investigating various aspects of how learners interact with MOOC learning materials, learner background, and learner knowledge. This includes aspects of students' language use and their use of learning materials. It also includes aspects of instructional design.

    More >>

    Collaborators: Diane Litman, Chris D. Schunn, Alok Baikadi, Yanjin Long, Rae Mancilla, & Valerie Swigart

    Publications:

    Alok Baikadi, Christian D. Schunn, Yanjin Long, & Carrie Demmans Epp. (2016). Redefining “What” in Analyses of Who Does What in MOOCs. 9th International Conference on Educational Data Mining (EDM16), Raleigh, NC, USA. 569-570.

    Chandrasekaran, M.K., Demmans Epp, C., Kan, M-Y., & Litman, D.J. (2017). Using Discourse Signals for Robust Instructor Intervention Prediction. In AAAI 2017 (pp. 3415-3421). San Francisco, CA, USA.

  • PsychOut Mobile Learning

    PsychOut is a case-based mobile learning app that aims to expose nursing students to clinical situations before they begin their formal clinical training in psychiatric units. The app uses multimedia materials to enable nursing students to explore psychiatric cases through choose-your-own-adventure style activities.

    More >>

    I collaborate with the app developers to evaluate the effectiveness of this tool for supporting nurse training.

    Collaborators: Joe Horne (University Center for Teaching and Learning - University of Pittsburgh), Britney Kepler, Irene Cane, and Amy Bowser (University of Pittsburgh, School of Nursing)

Language Learning and Learner Modeling Projects

  • VocabNomad

    VocabNomad is a mobile assisted language learning and communication support application. It is designed to be location aware and adapt to the user’s changing vocabulary support needs. Adaptivity is provided through the use of learner models that track learners' language use and preferences. Standardized assessments of vocabulary knowledge were used to determine if the adaptive communication support that is provided by VocabNomad can help English language learners improve their vocabulary knowledge.

    More >>

    I have performed qualitative interviews with English language learners as part of a formative evaluation of VocabNomad’s base features. These interviews were used to determine additional application features that could better support learners’ needs.

    A classroom-based deployment of the system took place in a Japanese senior high school, where improvements in student vocabulary knowledge and English proficiency were observed.

    Project Leads: Carrie Demmans Epp (PhD Work)

    Collaborators: Stephen Tsourounis, Justin Djordjevic, Christopher Arnold, & Semaphore, Faculty of Information, University of Toronto

    Instruments: The MyLog experience sampling application

    Publications:

    Stephen Tsourounis, & Carrie Demmans Epp. (2016). Learning Dashboards and Gamification in MALL: Design Guidelines in Practice. In A. Palalas and M. Ally (Eds.), The International Handbook of Mobile-Assisted Language Learning, 370-398, China Central Radio & TV University Press Co., Ltd.

    Carrie Demmans Epp. (2014). Mobile Adaptive Communication Support for Vocabulary Acquisition. Journal of Learning Analytics. 1(3), 173-175.

    Carrie Demmans Epp, Stephen Tsourounis, Justin Djordjevic, & Ronald M. Baecker. (2013) Interactive Event: Enabling Vocabulary Acquisition while Providing Mobile Communication Support. Artificial Intelligence in Education (AIED) 2013, Memphis, TN, USA. 932-933.

    Carrie Demmans Epp. (2013) Mobile Adaptive Communication Support for Vocabulary Acquisition. Artificial Intelligence in Education (AIED) 2013, Memphis, TN, USA. 876-879.

    Carrie Demmans Epp, & Ronald M. Baecker. (2012) Employing Adaptive Mobile Phone Applications for Scaffolding the Communication and Vocabulary Acquisition of Language Learners. Accepted to LearnLab's Annual Learning Science Workshop Use of Technology Toward Enhancing Achievement and Equity in the 21st Century, Pittsburgh, PA, USA.

    Carrie Demmans Epp, & Ronald M. Baecker. (2011). VocabNomad: a Context-Sensitive Application for Mobile Assisted Language Learning. Young Researcher's Track poster Session at Artificial Intelligence in Education (AIED) 2011, Auckland, New Zealand.

  • Representing Uncertainty in Visualizations of Educational Data

    Various forms of uncertainty are present in assessments of student knowledge and abilities. Measurement error and a lack of precision are among these sources of uncertainty, but they are rarely communicated through the visual representations that are used to report on student learning (i.e., open learner models and learning dashboards). This is in spite of the fact that this information can help us to properly interpret reports of student knowledge and learning.

    More >>

    I analyzed the visual representation of uncertainty across disciplines as well as the representation of information in open learner models and learning dashboards. The analyses of these two bodies of literature were combined to define a design space where we can begin to explore the representations of uncertainty within the reporting of educational data.

    Project Lead: Carrie Demmans Epp

    Collaborators: Susan Bull

    Publications:

    Lamiya Al-Shanfari, Carrie Demmans Epp, & Susan Bull. (2016). Uncertainty visualisation in open learner models: Representing inconsistencies in the underlying data. In Proceedings of the Workshop on Learning Analytics for Learners at the 6th International Learning Analytics and Knowledge Conference (LAK).

    Carrie Demmans Epp, & Susan Bull. (2015). Uncertainty Representation in Visualizations of Learning Analytics for Learners: Current Approaches and Opportunities. IEEE Transactions on Learning Technologies (TLT). Online First.

    Carrie Demmans Epp, Susan Bull, & Matthew D. Johnson. (2014). Visualising Uncertainty for Open Learner Model Users. 22nd Conference on User Modelling, Adaptation and Personalization (UMAP 2014), Aalborg, Denmark. 9-12.

  • Assessing Instructor Attitudes Towards Open Learner Models

    A learner model is a system-level representation of the user's knowledge, beliefs, and abilities. An open learner model is a learner model where the user can see aspects of the underlying model. I am developing a scale that can be used to assess how instructors feel about their students use of e-learning systems that have open learner models. Results from the use of this instrument could be employed to target the development of e-learning systems to specific populations or to inform the refinement of these systems.

    More >>

    Project Lead: Carrie Demmans Epp

    Publications:

    Carrie Demmans Epp. (2012) Developing a Scale for Assessing Instructor Attitudes Towards Open Leaner Models. In E. Herder, K. Yacef, L. Chen, & S. Weibelzhal (Eds.), Workshop and Poster Proceedings of the 20th Conference on User Modeling, Adaptation, and Personalization (UMAP 2012), Montreal, Quebec, Canada. (3 pages)

  • ProTutor

    ProTutor was a Pronunciation Tutor for learners of Russian as a Foreign Language. It employed an open learner model to inform learners of changes in their abilities. This model also enabled the adaptive recommendation of learning materials to learners based on each learner’s pronunciation accuracy and the academic schedule of the course in which they were enrolled.

    More >>

    This system was deployed and evaluated with learners over the course of an academic year. Survey results along with usage statistics revealed that learners liked the system, thought it was ‘fun’, and that the historical information that had been included in the open learner model allowed them to see their progress and helped maintain their motivation.

    Publications:

    Carrie Demmans Epp, Gordon McCalla. (2011). ProTutor: Historic Open Learner Modeling for Pronunciation Tutoring. Artificial Intelligence in Education (AIED) 2011, Auckland, New Zealand. 441-443

Language Learning Projects (NOT Technology Based)

  • The Vocabulary Learning Project (VLP)

    The VLP is a reading tutoring program for teen-aged English language learners where a longitudinal assessment of the effect of the tutoring intervention on language acquisition is being performed. This includes the acquisition of phonemic knowledge, vocabulary knowledge, and reading comprehension.

    More >>

    Within this project, I have administered and scored standardized and other tests that are being used as measures of each participant’s language proficiency.

    Project Lead: Esther Geva

Filter:

Sort by year:

Learning Dashboards and Gamification in MALL: Design Guidelines in Practice

Stephen Tsourounis, & Carrie Demmans Epp
Chapter In A. Palalas and M. Ally (Eds.), The International Handbook of Mobile-Assisted Language Learning, ISBN:978-7-304-07464-7, 2016, Pages 370-398.

Summary

This book chapter discusses the role that gamification and a particular type of educational reporting (i.e., open learner models and learning dashboards) play in mobile language learning. The chapter provides guidelines that are meant to help help people design appropriate gamification and feedback features within a mobile language-learning context. These guideline are grounded in the results of a design-based study and the literature from gamification, persuasive technology, open learner models, goal setting, and psychology.

Migrants and Mobile Technology Use: Gaps in the Support Provided by Current Tools

Carrie Demmans Epp
Journal Paper Journal of Interactive Media in Education (JIME), special collection on migrants, education, and technologies, 2017(01): 2, 2017, Pages 1-13.

Our current understanding of how migrants use mobile tools to support their communication and language learning is inadequate. This study, therefore, explores the learner-initiated use of technologies to support their comprehension, production, and acquisition of English following migration to Canada. Information about migrant use of technologies and experiences was collected with interviews. The interview data was analyzed through the complementary lenses of noticing, from language learning, and appropriation, from human-computer interaction. Combining these lenses enabled the identification of unmet migrant communication, support, and learning needs. The manner in which migrants employed mobile and other tools to facilitate their learning and communication were identified through the application of these theories. This analysis indicates that migrants can use existing tools to access information. However, they need additional support if they are to take full advantage of existing mobile tools. Moreover, there is a need for tools that support larger gaps in their knowledge and skills. Migrant experiences indicate that they need additional social, meta-cognitive, and emotional support. These needs suggest opportunities for creating mobile tools that scaffold the development of new skills that include the learner’s ability to monitor and plan his or her learning and understand language produced by those who speak different varieties of English or who have non-majority accents.

Exploring the relationships between facilitation methods, students' sense of community and their online behaviours

Krystle Phirangee, Carrie Demmans Epp, & Jim Hewitt
Journal Paper Online Learning Journal, special issue on online learning analytics, 20(2), 2016, Pages 134-154.

The popularity of online learning has boomed over the last few years, pushing instructors to consider the best ways to design their courses to support student learning needs and participation. Prior research suggests the need for instructor facilitation to provide this guidance and support, whereas other studies have suggested peer facilitation would be better because students might feel more comfortable learning and challenging each other. Our research compared these two facilitation methods and discovered that students participated more in instructor-facilitated online courses where they wrote more notes, edited and reread notes more, and created more connections to other notes than students in peer-facilitated online courses. We identified student activity patterns and described differences in how those patterns manifest themselves based on the facilitation method that was used. Our findings also show that instructor-facilitated online courses had a stronger sense of community than peer-facilitated online courses.

Information Practices and User Interfaces: Student Use of an iOS Application in Special Education

Carrie Demmans Epp, Rhonda McEwen, Rachelle Campigotto, & Karyn Moffatt
Journal Paper Education and Information Technologies, 21(5), 2015, Pages 1-24.

Abstract

A framework connecting concepts from user interface design with those from information studies is applied in a study that integrated a location-aware mobile application into two special education classes at different schools; this application had two support modes (one general and one location specific). The five-month study revealed several information practices that emerged from student attempts to overcome barriers within the application and the curriculum. Students engaged in atypical and unintended practices when using the application. These practices appear to be consequences of the user interface and information processing challenges faced by students. Abandoning activities was a strategic choice and was an unanticipated information practice associated with the application’s integration into lessons. From an information processing perspective, it is likely that students reinterpreted information in the location mode as housing application content rather than being location specific and the information practice of taking photos emerged as an expressive use of the device when an instrumental task was absent. Based on these and other emergent practices, we recommend functionality that should be considered when developing or integrating these types of applications into special education settings and we seek to expand the traditional definition of information practice by including human-computer interaction principles.

Uncertainty Representation in Visualizations ofLearning Analytics for Learners: Current Approaches and Opportunities

Carrie Demmans Epp, & Susan Bull
Journal Paper IEEE Transactions on Learning Technologies (TLT), 8(3), 2015, Pages 242-260

Abstract

Adding uncertainty information to visualizations is becoming increasingly common across domains since its addition helps ensure that informed decisions are made. This work has shown the difficulty that is inherent to representing uncertainty. Moreover, the representation of uncertainty has yet to be thoroughly explored in educational domains even though visualizations are often used in educational reporting. We analyzed 50 uncertainty-augmented visualizations from various disciplines to map out how uncertainty has been represented. We then analyzed 106 visualizations from educational reporting systems where the learner can see the visualization; these visualizations provide learners with information about several factors including their knowledge, performance, and abilities. This analysis mapped the design space that has been employed to communicate a learner's abilities, knowledge, and interests. It also revealed several opportunities for the inclusion of uncertainty information within visualizations of educational data. We describe how uncertainty information can be added to visualizations of educational data and illustrate these opportunities by augmenting several of the types of visualizations that are found in existing learning analytics reports. The definition of this design space, based on a survey of the literature, will enable the systematic exploration of how different design decisions affect learner trust, understanding, and decision making.

Developing an adaptive tool to select, plan, and scaffold oral assessment tasks for undergraduate courses

Carrie Demmans Epp, Gina Park, & Christopher Plumb
Journal PaperEducational Technology Review & Development (ETRD), 63(3), 2015, Pages 475-498

Abstract

The increased linguistic and cultural diversity of undergraduate classrooms at English language institutions has imposed additional pedagogical and assessment challenges on instructors, many of whom lack the knowledge necessary to design classroom activities and assessments that are fair to all students regardless of students’ background and language abilities. The development of an adaptive instrument for instructors who do not specialize in English language learning represents an attempt to adjust instructional practices to meet this need. This paper reports on the development of an instrument that undergraduate instructors can use to plan their courses at universities where English is the language of instruction. The instrument’s intended use is illustrated through an example that involves the planning of an interdisciplinary undergraduate course. To build this adaptive tool, a taxonomy that describes the relevant components of assessments that involve oral communication was developed and externally reviewed. The questions used in the instrument were then developed and piloted with a group of university undergraduate instructors; after which, the instrument was further refined. Although piloting revealed an increase in instructor awareness of how language abilities relate to assessment, further research is needed to determine the extent to which this tool affects instructor’s classroom or assessment practices.

Mobile Adaptive Communication Support for Vocabulary Acquisition

Carrie Demmans Epp
Journal PaperJournal of Learning Analytics, 1(3), 2014, Pages 173-175

Abstract

This work explores the use of an adaptive mobile tool for language learning. A school-based deployment study showed that the tool supported learning. A second study is being conducted in informal learning environments. Current work focuses on building models that increase our understanding of the relationship between application usage and learning.

Especially Social: Exploring the Use of an iOS Application in Special Needs Classrooms

Rachelle Campigotto, Rhonda McEwen, & Carrie Demmans Epp
Journal PaperComputers & Education, 60(1), 2013, Pages 74-86

Abstract

A five-month exploratory study was conducted with iOS mobile devices in two Toronto area schools with students in grades 7 through 12. Both classrooms were identified as Special Education classes by the Ontario Ministry of Education, and each student was identified as having exceptionalities requiring additional support and differentiation within the curriculum to support their success. Participants used MyVoice, a mobile application that is customizable through a website, which allows users to input vocabulary words and link words with pictures. The application can also be used to speak on behalf of users. Based on findings that technology perceived to be ‘fun’ by students has the potential to improve motivation in learning contexts, we aimed to (i) assess the degree to which the use of iOS devices and the MyVoice application affects attention and motivation levels for students with special needs, and (ii) identify factors that had the greatest influence on the integration of the technologies with existing curricula. Results indicated a strong potential for successfully integrating mobile technology within special needs classrooms, with a high-degree of student support for using mobile devices to enhance classroom experiences. The results also highlighted the challenges of incorporating these technologies into curriculum in terms of practicality, teacher comfort, and the limitations of the device. We find that the differentiation enabled by the multi-modal features of the application and device has positive outcomes on student perception of success, and that increased self-confidence in areas of the curriculum fostered a more robust sense of community among special needs students. We conclude with study implications for information processing theory.

Evaluating the Effect of Uncertainty Visualisation in Open Learner Models on Students’ Metacognitive Skills

Lamiya Al-Shanfari*, Carrie Demmans Epp, & Chris Baber
Conference Papersto appear at AIED2017

Abstract

Self-assessment is widely used in open learner models (OLMs) as a metacognitive process to enhance students’ self-regulated learning. Yet little research has investigated the impact of the visualisation when the OLM shows the conflict (i.e., uncertainty) between the system’s beliefs about student knowledge and students’ confidence in the correctness of their answers. We deployed such an OLM and studied its use. The impact of student access to the OLM was determined by first comparing OLM users to non-OLM users. The impact of the uncertainty visualisation on student learning, confidence gains and actions was determined by comparing these measures across two treatment conditions and a control condition. Those who accessed the OLM performed significantly better on the post-test. Also, students in the treatment group that could see both sets of beliefs separately showed significantly greater confidence gains and used the system significantly more.

Student Preferences for Visualising Uncertainty in Open Learner Models

Lamiya Al-Shanfari*, Chris Baber, & Carrie Demmans Epp
Conference Papersto appear at AIED2017

Abstract

User preferences for indicating uncertainty using specific visual variables have been explored outside of educational reporting. Exploring students preferred method to indicate uncertainty in open learner models can provide hints about which approaches students will use, so further design approaches can be considered. Participants were 67 students exploring 6 visual variables applied to a learner model visualisation (skill meter). Student preferences were ordered along a scale, which showed the size, numerosity, orientation and added marks visual variables were near one another in the learner’s preference space. Results of statistical analyses revealed differences in student preferences for some variables with opacity being the most preferred and arrangement the least preferred. This result provides initial guidelines for open learner model and learning dashboard designers to represent uncertainty information using students’ preferred method of visualisation.

A synthesis study: Evaluating the applicability and generalisability of technology-supported vocabulary programs for adolescent ELLs

Jia Li*, Esther Geva, Carrie Demmans Epp, Catherine Snow, & Andrew Biemiller
Conference Papersto appear at the XVIIIth International CALL Conference: CALL in Context

Abstract

This article reports on a synthesis study that systematically assesses technology supported programs and mobile apps for enhancing adolescent English language learners’ (ELLs’) learning of vocabulary in the contexts of English as a foreign, second or additional language.

Using Discourse Signals for Robust Instructor Intervention Prediction

Muthu Kumar Chandrasekaran*, Carrie Demmans Epp, Min-Yen Kan, & Diane J. Litman
Conference Papers31st AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, CA, USA. 3415-3421.

Abstract

We tackle the prediction of instructor intervention in student posts from discussion forums in Massive Open Online Courses (MOOCs). Our key finding is that using automatically obtained discourse relations improves the prediction of when instructors intervene in student discussions, when compared with a state-of-the-art, feature-rich baseline. Our supervised classifier makes use of an automatic discourse parser which outputs Penn Discourse Treebank (PDTB) tags that represent in-post discourse features. We show PDTB relation-based features increase the robustness of the classifier and complement baseline features in recalling more diverse instructor intervention patterns. In comprehensive experiments over 14 MOOC offerings from several disciplines, the PDTB discourse features improve performance on average. The resultant models are less dependent on domain-specific vocabulary, allowing them to better generalize to new courses.

Redefining “What” in Analyses of Who Does What in MOOCs

Alok Baikadi*, Christian D. Schunn, Yanjin Long, & Carrie Demmans Epp
Conference Papers9th International Conference on Educational Data Mining (EDM16), Raleigh, NC, USA. 569-570.

Abstract

To advance our understanding of learning in massive open online courses (MOOCs), we need to understand how learners interact with course resources. Prior explorations of learner interactions with MOOC materials have often described these interactions through stereotypes, which does not account for the full spectrum of potential learner activities. A focus on stereotypes also limits our ability to explore the reasons behind learner behaviors. To overcome these shortcomings, we apply factor analysis to learner activities within four MOOCs to identify emergent behavior factors. The factors support characterizations of learner behaviors as driven heavily by types of learning activities and secondarily by time/topic; regression revealed demographic factors (especially country and gender) associated with these activity and topic preferences. Both factor and regression analyses revealed structural variability in learner activity patterns across MOOCs. The results call for a reconceptualization of how different learning activities within a MOOC are designed to work together.

English Language Learner Experiences of Formal and Informal Learning Environments

Carrie Demmans Epp
Conference PapersIn the 6th International Learning Analytics and Knowledge Conference (LAK16), Edinburgh, Scotland. 231-235.

Abstract

Many people who do not know English have moved to English-speaking countries to learn English. Once there, they learn English through formal and informal methods. While considerable work has studied the experiences of English language learners in different learning environments, we have yet to see analytics that detail the experiences of this population within formal and informal learning environments. This study used the experience sampling methodology to capture the information that is needed to detail the communication and affective experiences of advanced English language learners. The collected data reveals differences in how English language learners perceived their communication success based on their learning context, with higher levels of communicative success experienced in formal learning settings. No such differences were found for learners’, highly negative, affect. The data suggest a need for additional emotional support within formal and informal learning environments as well as a need for oral communication support within informal contexts.

Formative Assessment and Meaningful Learning Analytics – an Independent Open Learner Model Solution

Susan Bull, Matthew D. Johnson*, Carrie Demmans Epp, Drew Masci, Mohammad Alotaibi, & Sylvie Girard
Conference Papers14th IEEE International Conference on Advanced Learning Technologies (ICALT 2014), Athens, Greece. 327-329.

Abstract

We introduce an independent open learner model for formative assessment and learning analytics based on developments in technology use in learning, also maintaining more traditional numerical and qualitative feedback options.

Interactive Event: Enabling Vocabulary Acquisition while Providing Mobile Communication Support

Carrie Demmans Epp*, Stephen Tsourounis, Justin Djordjevic, & Ronald M. Baecker
Conference PapersArtificial Intelligence in Education (AIED) 2013, Memphis, TN, USA. 932-933.

Abstract

We have developed an adaptive communication support tool that also supports vocabulary acquisition. This tool is called VocabNomad; it is one of the few mobile assisted language learning tools that aims to support the call for activities that are fundamentally different than those provided by paper and pencil or computer assisted language learning [1]. VocabNomad meets this call by trying to support the communication of immigrants who are isolated from their surrounding environment because of their limited English language proficiency. In the US, these English language learners (ELL) make up more than 20 percent of the population [2, 3].

Mobile Adaptive Communication Support for Vocabulary Acquisition

Carrie Demmans Epp*
Conference PapersArtificial Intelligence in Education (AIED) 2013, Memphis, TN, USA. 876-879.

Abstract

Language learners are often isolated because of their inability to communicate. Adaptive mobile communication support tools could be used to scaffold both their interaction with others and their vocabulary acquisition. I propose the exploration of a new tool that is designed to meet this need.

Towards Providing Just-in-Time Vocabulary Support for Assistive and Augmentative Communication

Carrie Demmans Epp*, Justin Djordjevic, Shimu Wu, Karyn Moffatt, & Ronald M. Baecker
Conference PapersIntelligent User Interfaces (IUI), Lisbon, Portugal. 33-36.

Abstract

Many people cannot communicate effectively with those around them. The causes vary but several tools and strategies can support their communication. These tools, which collectively fall under the banner of Assistive and Augmentative Communication (AAC), are rarely adaptive. Of those that are, few provide context-based or just-in-time vocabulary support to users even though the proliferation of smartphones makes this possible. To meet this need, we developed four algorithms to retrieve relevant vocabulary from Internet-based corpora. We used discourse completion tasks to evaluate each algorithm’s ability to identify appropriate vocabulary across a set of specific contexts. The results indicate that our approach identifies appropriate context-specific words that complement general AAC vocabularies: when combined with a typical base vocabulary, the algorithms outperformed the support provided by the base vocabulary alone. They did this by adding small targeted vocabularies.

ProTutor: Historic Open Learner Modeling for Pronunciation Tutoring

Carrie Demmans Epp*, & Gordon McCalla
Conference PapersArtificial Intelligence in Education (AIED) 2011, Auckland, New Zealand. 441-443.

Abstract

Acquiring proper pronunciation is difficult for second language learners. We built a pronunciation tutor, called ProTutor, that uses open learner models (OLM). Its OLM incorporates historic information about learner performance to encourage reflection and maintain learner motivation. We performed a formative evaluation in which participants indicated that ProTutor was helpful and fun to use.

The Who, What, When, Where, and Why of Lecture Capture

Christopher Brooks*, Carrie Demmans Epp, Greg Logan*, & Jim Greer
Conference Papers Learning Analytics and Knowledge 2011, Banff, Alberta, Canada. Pages 86-92

Abstract

Video lecture capture is rapidly being deploying in higher-education institutions as a means of increasing student learning, outreach, and experience. Understanding how learners use these systems and relating this use back to pedagogical and institutional goals is a hard issue that has largely been unexplored. This work describes a novel web-based lecture presentation system which contains fine-grained user tracking features. These features, along with student surveys, have been used to help analyse the behaviour of hundreds of students over an academic term, quantifying both the learning approaches of students and their perceptions on learning with lecture capture.

Uncertainty visualisation in open learner models: representing inconsistencies in the underlying data

Lamiya Al-Shanfari, Carrie Demmans Epp, & Susan Bull
Conference Papers (adj. proc.)In Learning Analytics for Learners Workshop of the 6th International Learning Analytics and Knowledge Conference (LAK16), Edinburgh, Scotland, 23-30.

Abstract

This paper suggests different methods for uncertainty visualisation in open learner models (OLM). In order to apply uncertainty visualisation in OLMs, two factors need to be measured, namely the source of the uncertainty in the data and the level of uncertainty in the learner model. This paper proposes a method to detect the source of uncertainty within a learner model: outlier analysis is employed to identify inconsistencies in the data set from which the OLM is built. Two approaches for visu-alising this uncertainty within the OLM have been integrated into an existing system and additional potential representations are proposed.

Visualising Uncertainty for Open Learner Model Users

Carrie Demmans Epp, Susan Bull*, & Matthew D. Johnson
Conference Papers (adj. proc.)22nd Conference on User Modelling, Adaptation and Personalization (UMAP 2014), Aalborg, Denmark. 9-12. [Best Poster Award]

Abstract

There is now widening use of open learner models to support learning and promote metacognitive behaviours, but the learner model visualisations do not typically include information about uncertainty in the model. This paper considers findings from the field of information visualisation and applies these to open learner models. Examples are given for how visualisation of model uncertainty might be usefully achieved by open learner model designers.

Using Simulated Learners and Simulated Learning Environments within a Special Education Context

Carrie Demmans Epp* & Alexandra Makos
Conference Papers (adj. proc.)In Workshop on Simulated Learners at Artificial Intelligence in Education (AIED) (Vol. 4, pp. 1–10). Memphis, TN, USA: Springer.

Abstract

The needs of special education populations require specific support to scaffold learning. The design and use of intelligent tutoring systems (ITS) has the potential to meet these needs. Difficulty in the development of these systems lies in their validation due to the ethics associated in studying learners from this population as well as the difficulty associated with accessing members of this learner group. This paper explores the use of simulated learners as a potential avenue for validating ITS designed for a special education population. The needs of special education learners are discussed. Potential avenues for employing simulated learners and simulated learning environments to test ITS, instructional materials, and instructional methods are presented. Lastly, the expansion of an educational game designed to develop emotion recognition skills in children with autism spectrum disorder is used to illustrate how simulated learning environments can be used to support the learning of these students.

Developing a Scale for Assessing Instructor Attitudes Towards Open Leaner Models

Carrie Demmans Epp*
Conference Papers (adj. proc.)In E. Herder, K. Yacef, L. Chen, & S. Weibelzhal (Eds.), Workshop and Poster Proceedings of the 20th Conference on User Modeling, Adaptation, and Personalization (UMAP 2012), Montreal, Quebec, Canada. (3 pages)

Abstract

The attitudes of students with respect to open learner models have been extensively explored, whereas the attitudes of instructors have not been fully explored. I have, therefore, begun the development of a scale to assess this. I describe the initial item development and revision which was based on cognitive interviews. I then describe a pilot study that was performed to further refine the scale before attempting to validate it. Once the scale has been validated, it could be used to assess individual instructors’ views towards specific open learner models or open learner model use within specified contexts.

Posture monitoring and improvement for laptop use

Carrie Demmans Epp*, Sriram Subramanian, & Jon Titus
Conference Papers (adj. proc.)Proceedings of CHI Extended Abstracts 2007, San Jose, CA, USA. 2357-2362

Abstract

Both Repetitive Stress Injuries and laptop use have increased. The poor ergonomic design of laptops has the potential to create or exacerbate existing RSI. We propose a persuasive Attentive User Interface which provides feedback in order to improve user neck posture. This system measures the angle of the user.s neck and determines the quality of his/her neck posture. We then provide exercises to strengthen the neck and improve the user.s posture. We performed a study which showed an increase in neck comfort among our system.s users. The study demonstrated the potential of our system, which should be further tested.

The Interplay Between Students’ Usage of Pronouns and Community Levels in Online Courses

Carrie Demmans Epp, Krystle Phirangee, & Jim Hewitt
Workshops & OtherPresented at the 2017 American Educational Research Association (AERA) Annual Meeting.

Summary

Identifying which online behaviours and interactions help foster a sense of community is needed to better support students’ online learning experience. However, there is a need for more research exploring student use of verbally immediate behaviours since current research focuses more on instructor behaviours. We explore the verbally immediate behaviour of pronouns and their usage within online courses. Specifically, we look at how students used pronouns within the posts that they make to their online course and how their pronoun usage relates to community level. Findings suggest that students from Low CCS courses and High CCS courses used pronouns differently but their usage seems to be influenced by the facilitation method adopted by the instructor.

Community in Online Courses: The Roles Played By Course Length, Course Facilitation Methods, and Student Actions

Krystle Phirangee, Carrie Demmans Epp, & Jim Hewitt
Workshops & OtherPresented at the 2017 American Educational Research Association (AERA) Annual Meeting.

Summary

Fostering a strong sense of community among students within online courses is essential to supporting their learning experience. However, there is little consensus about the facilitation methods that best enable community development or how the length of the course influences the development of community. We explore students’ sense of community across four courses that employ either instructor or peer facilitation across a full or half term. We tie students’ sense of community to their actions within these courses both confirming and challenging prior findings. These findings suggest the presence of an interaction between term length and facilitation method when considering how students’ sense of community is manifested through their actions in online courses.

Exploring How Peer and Instructor Facilitation Relates to Students’ Activity Patterns in Online Learning Communities

Krystle Phirangee, Carrie Demmans Epp, & Jim Hewitt
Workshops & OtherPresented at 2016 American Educational Research Association (AERA) Annual Meeting, Washington, DC.

Summary

As online learning becomes more popular instructors must consider how to best design their courses to support student participation and learning needs. Some previous studies have emphasized the importance of instructor facilitation to help provide this support. Other scholars have suggested peer facilitation might be the better approach. Our research compared these two approaches and found that students had a higher level of participation in instructor facilitated online courses with more written notes, replies, rereads, revisions, and connections to other notes, than peer facilitated courses. Furthermore, instructor facilitated courses had a stronger sense of community compared to peer facilitated courses.

Participatory Design with Music Students: Empowering Children to Develop Instructional Technology

Heather Birch*, & Carrie Demmans Epp
Workshops & OtherPresented at 2015 American Educational Research Association (AERA) Annual Meeting, Chicago, Illinois.

Summary

Participants, aged 7 through 17, were recruited to be active collaborators in the creation of a technological solution to address various challenges they had experienced as music learners. Phase 1 of the study, which is currently ongoing, employs participatory design (PD) to develop a mobile application for motivating and supporting music learners, particularly in the area of technical skill development. Phase 2 will determine if the mobile application facilitates positive effects on students’ musical skill, and their attitudes toward practicing their musical instrument. We report on Phase 1 progress and highlight four challenges that arose while involving children in participatory design. These challenges, while causing some frustration and confusion among project stakeholders, can be overcome. A discussion of the potential benefits and advantages of engaging children in participatory design is included.

The Speaking Section of the Test of English for International Communication (TOEIC)

Carrie Demmans Epp, & Daniel Riccardi
Workshops & OtherTESL Ontario Contact, 39(3), 61–67.

Summary

This test section assesses one's ability to produce oral language. The article provides a brief history of the speaking test's development. The article also analyzes the strengths and limitations of the speaking section of the TOEIC. The test's strengths include its delivery method, which helps ensure consistency, and its limitations include the exclusion of dialog co-construction.

Employing Adaptive Mobile Phone Applications for Scaffolding the Communication and Vocabulary Acquisition of Language Learners

Carrie Demmans Epp*, & Ronald M. Baecker
Workshops & OtherIn LearnLab’s Learning Science Workshop on the Use of Technology Toward Enhancing Achievement and Equity in the 21st Century. Pittsburgh, PA, USA.

Abstract

Many recent immigrants are unable to obtain employment in the areas for which they were trained because they lack proficiency in the language of their new home. Considerable effort has gone into training programs to help members of this group, but these programs typically require learners to sit in a traditional classroom. This can further limit employment opportunities because the scheduling of these courses often conflicts with employment responsibilities, thus preventing newcomers from accessing the very programs that are intended to help them.

We propose the use of mobile devices for supporting the communication of second language (L2) learners when they are interacting with those around them. This has the potential to enable anytime-anywhere learning. It also places the learning in an authentic context.

I will describe preliminary results from a study where L2 learners used a smart-phone application that is location-aware and supports communication. I will also describe how the results of this evaluation are informing the design and implementation of a smart-phone application that builds a model of the learner’s language use and knowledge in order to scaffold that learner’s vocabulary acquisition. This application will enable situated learning opportunities by supporting the learner’s ability to interact with others in his or her surroundings.

MarcoPolo: Context-Sensitive Mobile Communication Support

Carrie Demmans Epp*, Rachelle Campigotto, Alexander Levy, & Ronald M. Baecker
Workshops & OtherIn FICCDAT: RESNA/ICTA (4 pgs). Toronto, Canada.

Summary

People with Autism Spectrum Disorder and aphasia often have impairments that limit their ability to communicate verbally. Consequently, their daily activities, social interactions, and quality of life are affected. We have developed a customizable context-sensitive application, MarcoPolo, that can help these populations overcome challenges by providing vocabulary support both orally and visually. Initial evaluations of MarcoPolo have helped refine the application and have illustrated that context-sensitive mobile applications can support communication. Several additional studies will be run to further evaluate the efficacy of context-sensitive communication support. Two of these will start in 2011 and early results will be presented at FICCDAT.

Supporting English Language Learners with an Adaptive Mobile Application

Carrie Demmans Epp
ThesisPhD (2016) - University of Toronto, Department of Computer Science

Abstract

English language learners (ELL) have dedicated considerable time and effort to the development of their language proficiency. This has included the use of a variety of mobile assisted language learning (MALL) tools that are either unproven or that have undergone limited evaluations of their effectiveness. The majority of these evaluations have been performed with beginner foreign-language learners at the post-secondary level. Moreover, dedicated MALL tools rarely support the learner’s ability to communicate in English. I propose and demonstrate the feasibility of an adaptive MALL approach that aims to scaffold ELL vocabulary and communication needs. This scaffolding recommends learning materials to ELLs by employing the ecological approach to dynamically reason over logs of learner interactions with a MALL tool.

The highly personalized approach to supporting learners that is operationalized through this tool was developed following user-centered design principles. The development of the learning content generation and recommendation mechanisms that are included as part of this approach to supporting English language learners was validated through two studies. An additional exploratory evaluation of this adaptive approach to supporting ELL communication and learning activities was performed before evaluating its influence on ELL vocabulary knowledge, communication, and affect through two studies.

These studies considered the effectiveness of the proposed MALL approach from multiple perspectives. The first took place in a Japanese high school and focused on the relationship between student vocabulary knowledge and system usage. The second involved advanced English language learners and took place in the greater Toronto area. This study aimed to determine the relationships among system usage, user communicative success, and user affect. The work presented in this thesis shows that the use of the proposed approach can support ELL communication, vocabulary development, and affect. The evaluation of this approach allowed the creation of models that predict learning outcomes based on learners’ MALL usage and knowledge. Combining the results of these studies with those of the formative evaluations, indicates that a mobile tool that employs the ecological approach to learner modeling can support the learning activities, vocabulary learning outcomes, affect, and communication of English language learners.

ProTutor: a Pronunciation Tutor that Uses Historic Open Learner Models

Carrie Demmans Epp
ThesisMSc (2010) - University of Saskatchewan, Department of Computer Science

Abstract

Second language learners face many challenges when learning a new language. To determine which challenges learners needed additional support in overcoming, we conducted a needs assessment of the Russian language program at the University of Saskatchewan and found that their students needed the most help with speaking in Russian. As a result, we designed an Intelligent Tutoring System (ITS) to help students learn how to pronounce Russian properly. We hoped to alleviate some of the challenges that learners face when learning to pronounce words in a second language by building an ITS that uses a Historic Open Learner Model (HOLM) to encourage learner reflection and to help maintain learner motivation. We designed, built, and performed a formative evaluation of a system, called ProTutor, using beginner learners of Russian as a second language at the University of Saskatchewan. This evaluation showed that learners have a positive perception of HOLMs and of the system as a whole. However, ProTutor needs further evaluation in order to determine its effectiveness as a learning aide.

Online Communities for Girls in the Sciences

Carrie Demmans Epp
ThesisBSc Honours (2006) - University of Saskatchewan, Department of Computer Science

Summary

I analyzed different websites and online communities that aimed to encourage or support girls' interest in science, technology, engineering, and math (STEM). I then conducted a survey to collect information about how girls and boys in high school perceived a representative sample of these online communities. This data was analyzed and a set of design guidelines for creating inclusive online STEM communities was created.

Noticing: ELL Use of MALL for Filling the Gap

Carrie Demmans Epp*
Conference TalksPresented at the 2016 Annual CALICO Conference, East Lansing, Michigan, USA.

Summary

Data from three studies that explored English language learner (ELL) use of Mobile assisted language learning (MALL) technologies was analyzed to better understand how ELLs employ MALL tools to support noticing. This data shows how ELLs have repurposed everyday mobile services and applications as well as dedicated MALL tools to help them fill gaps in their knowledge that they have noticed as a result of using English to interact with others.

VocabNomad: a Context-Sensitive Mobile Application - University English for Academic Purposes Case Study.

Bill Mboutsiadis*, Carrie Demmans Epp, & Dariyava Anayatova
Conference TalksPresented at the 2015 TESOL International Convention & Language Expo (TESOL), Toronto, Canada.

Summary

This presentation combined a system demonstration with guidelines for integrating the demonstrated application into English for academic pruposes (EAP) courses. These guidelines were based on a case study where we integrated two applications into an EAP program.

VocabNomad: Exploiting Gamification in a Mobile Vocabulary Learning Tool

Stephen Tsourounis*, Carrie Demmans Epp, & Ronald M. Baecker
Conference TalksPresented at 2014 Annual CALICO Conference. Athens, OH, USA.

Summary

VocabNomad is an adaptive communication support tool that also supports vocabulary acquisition. It aims to meet the needs of English language learners who are isolated from their surrounding environment because of their limited English proficiency. VocabNomad exploits information about the learner to provide personalized adaptive vocabulary support that can be used to transform everyday events into learning opportunities. We have integrated aspects of gamification to increase the motivation of learners, and in turn, increase the learner’s engagement with the system. Participants will get to interact with the system to see how it supports learners and personalizes its interactions with them.

Teacher and System Influences on Student Experiences and Actions

Carrie Demmans Epp
Talks Cognitive Psychology Brown Bag Series – University of Pittsburgh, United States. 2016.

Summary

For many, taking online courses has become an accepted and even desired part of their post-secondary experience. Most students have interacted with online platforms that either supplement their in-class experiences or that are meant to allow students to pursue their education whenever and wherever they can. Common wisdom and current teaching models encourage the use of student-led discussion forums. However, we do not fully understand how course-design choices or system interfaces influence student experiences. To better understand these factors, I conducted two studies. One explored how the facilitation methods employed in discussion forums supported students and influenced their behaviours. The other explored the interplay between the type of online course that students were taking and the specific learning technology that was used to deliver that course. Findings from both studies suggest that these issues are more complicated than suggested and require further attention if we are to make online learning experiences effective.

Using Mobile and Scalable Technologies to Support Language Learning and Communication

Carrie Demmans Epp
Talks Computer-Assisted Language Learning Colloquium – University of Illinois Urbana-Champaign, United States. 2016.

Summary

Computer assisted language learning holds the promise to support the language development of its users so that they can later use that language to interact with others. Recent advances in mobile and scalable solutions are not yet meeting this goal. I will discuss two projects that take steps in this direction. The first is a mobile application that aims to support the communication of English language learners who have recently migrated. The second develops learners’ ability to recognize and use clinical terminology through a massive open online course.

Supporting English Language Learners with an Adaptive Mobile Application

Carrie Demmans Epp
Talks Intelligent Systems Program Colloquium – University of Pittsburgh, United States. 2016.

Summary

English language learners (ELL) have dedicated considerable time and effort to the development of their language proficiency. This has included the use of a variety of mobile assisted language learning (MALL) tools that are either unproven or that have undergone limited evaluations of their effectiveness. I propose and demonstrate the feasibility of an adaptive MALL approach that aims to scaffold ELL vocabulary and communication needs. This scaffolding recommends learning materials to ELLs by employing the ecological approach to dynamically reason over logs of learner interactions with a MALL tool. The personalized approach to supporting learners that is provided by this tool was developed following user-centered design principles before the tool was evaluated through two studies. These studies investigated its influence on ELL vocabulary knowledge, communication, and affect. The first study focused on the relationship between student vocabulary knowledge and system usage. The second study aimed to determine the relationships among system usage, user communicative success, and user affect. These summative evaluations and the formative evaluations that were used to build the MALL tool show that the proposed approach can support ELL communication, vocabulary development, and affect. Going forward, similar approaches will be used within a MALL tool that aims to support the development of the reading comprehension of low-literacy adults.

Supporting English Language Learners with an Adaptive Mobile Application

Carrie Demmans Epp
Talks Reading and Language Group – University of Pittsburgh, United States. 2015.

Summary

English language learners (ELL) have dedicated considerable time and effort to the development of their language proficiency. This has included the use of a variety of mobile assisted language learning (MALL) tools that are either unproven or that have undergone limited evaluations of their effectiveness. The majority of these evaluations have been performed with beginner foreign-language learners at the post-secondary level. Moreover, dedicated MALL tools rarely support the learner’s ability to communicate in English. I propose and demonstrate the feasibility of an adaptive MALL approach that aims to scaffold ELL vocabulary and communication needs. This scaffolding recommends learning materials to ELLs by employing the ecological approach to dynamically reason over logs of learner interactions with a MALL tool.

The highly personalized approach to supporting learners that is operationalized through this tool was developed following user-centered design principles. The development of the learning content generation and recommendation mechanisms that are included as part of this approach to supporting English language learners were validated through two studies. An additional exploratory evaluation of this adaptive approach to supporting ELL communication and learning activities was performed before evaluating its influence on ELL vocabulary knowledge, communication, and affect through two studies.

These studies considered the effectiveness of the proposed MALL approach from multiple perspectives. The first took place in a Japanese high school and focused on the relationship between student vocabulary knowledge and system usage. The second involved advanced English language learners and took place in the greater Toronto area. This study aimed to determine the relationships among system usage, user communicative success, and user affect. The work presented in this thesis shows that the use of the proposed approach can support ELL communication, vocabulary development, and affect. The evaluation of this approach allowed the creation of models that predict learning outcomes based on learners’ MALL usage and knowledge. Combining the results of these studies with those of the formative evaluations, indicates that a mobile tool that employs the ecological approach to learner modeling can support the learning activities, vocabulary learning outcomes, affect, and communication of English language learners.

Learner Modelling and Learning Dashboards to Support Language and Domain Learning

Carrie Demmans Epp
Talks Mobile Interfaces and Pedagogical Systems (MIPS) Group – University of Pittsburgh, United States. 2015.

Summary

This informal talk will be organized around your interests as the audience. I will provide a high-level overview of my approach to research and the technology-enhanced learning projects that I have been involved with. This work includes mobile learning, intelligent-tutoring systems, learner modelling, and open learner models. You will then get to choose which aspects of my research are discussed.

Adaptive Technologies to Support Language Learning

Carrie Demmans Epp
Talks University of Ontario Institute of Technology, Canada. 2015.

Summary

Technology use is deeply rooted within language learning. From the early use of language labs and more recent use of multi-media, we have seen the wide use of technology by language learners. This talk will present two adaptive language-learning tools and discuss their deployment in various cultural and educational contexts. The first tool is a computer-based pronunciation tutor for Russian (ProTutor) and the second is a mobile-based English communication support and study tool (VocabNomad). Both systems employed representations of the user’s knowledge (learner models) to drive their adaptive content recommendation and personalized feedback. These adaptive features helped to motivate learners, enabled their self-regulation, and supported learning activities and outcomes. Their use by high school and university students also demonstrated the need for them to be accompanied by appropriate pedagogical practices when used as support tools in formal learning environments.

Supporting Language Learning and Learner Reflection through Technology-Enhanced Learning

Carrie Demmans Epp
Talks School of Information – University of Michigan. 2014.

Summary

Technology use is deeply rooted within language learning. From the early use of language labs and more recent use of multi-media, we have seen the wide use of technology by language learners. This talk will present two adaptive language-learning tools and discuss their deployment in various cultural and educational contexts. The first tool is a computer-based pronunciation tutor for the Russian language (ProTutor) and the second is a mobile-based English communication support and study tool (VocabNomad). The study of these environments has included the use of a particular type of educational reporting that is meant to support learner reflection and self-regulation by communicating information about the learner’s knowledge, skills, or activities. This class of tools is called open learner models or learning dashboards, and their use within these systems has opened an exploration of how to communicate uncertainty in the results of assessments in order to support informed decision making. I will conclude by discussing this newly opened design space.

Mobile Adaptive Communication Support for Vocabulary Acquisition

Carrie Demmans Epp
Talks Graduate School of Language, Communication, and Culture – Kwansei Gakuin University, Japan. 2014.

Summary

Current mobile assisted language learning (MALL) tools only support a subset of learner needs. This talk will discuss a research project that details which learner needs are not being met: situated second-language learning, just-in-time support, and communication. It will then describe the development of a new MALL tool that aims to support these needs, and a program for investigating the effectiveness of that tool will be discussed.

Technology Enhanced Learning to Support Language, Cognition, Recall, Knowledge Construction, and Reflection

Carrie Demmans Epp
Talks School of Computing – University of Dundee, UK. 2014.

Summary

This talk will provide an overview of several communication and knowledge construction tools as well as a particular type of information visualization, open learner models, that is common in technology enhanced learning environments. The talk will highlight the use and further development of an adaptive augmentative and alternative communication application with students in special education and English language learning contexts.

Technology Enhanced Learning and Supporting the Aging Process

Carrie Demmans Epp
Talks School of Computing – University of Leeds, UK. 2014.

Summary

This talk will provide an overview of several technologies that aim to support the social and communication needs of those who are aging. The re-purposing of cognitive and communication support tools for special education and language learning populations will be discussed in greater detail before discussing Carrie's explorations into teacher attitudes towards open learner models and the communication of uncertainty within open learner models.

From Language Learner to Researcher

Carrie Demmans Epp
Talks School of Continuing Studies – University of Toronto. 2013.

Summary

Technology has a long history of use in language learning. This talk will provide a history of the development of computer assisted language learning (CALL) and detail how I came to develop new CALL programs to fill gaps that I had noticed as a language learner and teacher.

CRAP, Heuristic Evaluation, & Paper Prototyping

Carrie Demmans Epp
Talks Pepper Research Group, Ontario Institute for Studies in Education (OISE) – University of Toronto. 2013.

Summary

This talk explained a variety of design principles and evaluation methods for graphical user interfaces. Among the topics covered were Nielsen’s heuristics, Heuristic Evaluation, paper prototyping, and the design principles of contrast, repetition, alignment, and proximity (CRAP). This tutorial was followed by paper prototyping activities, the later evaluation of a software system, and the system’s eventual redesign.

VocabNomad: Adaptive Mobile Assisted Language Learning

Carrie Demmans Epp
Talks Advanced Research in Intelligent Educational Systems (ARIES) Laboratory, Dept. of Computer Science – University of Saskatchewan. 2012.

Summary

Mobile Assisted Language Learning (MALL) is a developing sub-area of Computer Assisted Language Learning that uses mobile devices instead of a computer to deliver educational materials. However, most current MALL applications fail to fully exploit the features of mobile devices. A feature that MALL applications could better explore is the use of contextual information, such as location, to provide learners with more appropriate support exactly when it is needed (just-in-time) so that they can fully engage with the world around them.

I will report on the development of a MALL application, called VocabNomad, for Android platforms. It will provide learners with context-specific vocabulary in order to support their second language communication. VocabNomad will use a learner model to recommend personalized vocabulary that meets the learner's communication needs and preferences based on the learner's ever-changing context (location, activity, and preferences). Following this, I will describe the studies that are planned for evaluating the efficacy of the support provided by VocabNomad.

Student Supervision - BSc Theses

  • 2014 2014

    Zafer Sawaf

    PlayOn: A mobile piano practice support tool, Department of Computer Science - University of Toronto (co-supervised with Heather Birch)

  • 2014 2014

    Stephen Tsourounis

    VocabNomad: Exploiting Gamification in a Mobile Vocabulary Learning Tool, Department of Computer Science - University of Toronto

Current Teaching

  • Current

    I am not currently teaching.

Instructor

  • 2010 Summer

    Developing Object Oriented Systems

    (CMPT270 - Year 2, Undergraduate), Department of Computer Science - University of Saskatchewan (2010 Summer)

  • 2010 Summer

    Introduction to Computing

    (CMPT100 - Year 1, Undergraduate), Department of Computer Science - University of Saskatchewan

  • 2009 Summer

    Introduction to Computing

    (CMPT100 - Year 1, Undergraduate), Department of Computer Science - University of Saskatchewan

  • 2009 Winter

    Introduction to Computing

    (CMPT100 - Year 1, Undergraduate), Department of Computer Science - University of Saskatchewan

Tutorial Assistant or Marker

  • 2014 Fall

    Topics in Communication, Culture, Information & Technology

    (CCT395 - Year 3, Undergraduate), Institute of Communication, Culture, Information & Technology - University of Toronto Mississauga

  • 2010 Winter

    Introduction to Computer Organization and Design

    (CMPT215 - Year 2, Undergraduate), Department of Computer Science - University of Saskatchewan

  • 2009 Winter

    Language Technology

    (CMPT298 - Year 2, Undergraduate), Department of Computer Science - University of Saskatchewan

  • 2008 Summer

    Introduction to Computer Organization and Design

    (CMPT215 - Year 2, Undergraduate), Department of Computer Science - University of Saskatchewan

  • 2007 Summer

    Introduction to Computer Organization and Design

    (CMPT215 - Year 2, Undergraduate), Department of Computer Science - University of Saskatchewan

  • 2006 Fall

    Introduction to Computing

    (CMPT100 - Year 1, Undergraduate), Department of Computer Science - University of Saskatchewan

  • 2004 Fall

    Introduction to Computing

    (CMPT100 - Year 1, Undergraduate), Department of Computer Science - University of Saskatchewan

Course Developer

  • 2008 2008

    Language Technology

    (CMPT298 - Year 2, Undergraduate), Department of Computer Science - University of Saskatchewan

  • 2004 2004

    Introduction to Computing and Programming

    (CMPT102 - Year 1, Undergraduate), Department of Computer Science - University of Saskatchewan

Primary and Secondary School Level

  • 2009 2008

    Science Ambassador

    Mosquito Primary School

    Mosquito First Nation, Saskatchewan

    Developed and delivered hands on science, computing, and mathematics activities for students in grades 1-7

    Helped the kindergarten and pre-school classes in the computer lab

    Helped the Culture teacher find and use electronic langauge-learning resources for Plains Cree (Y dialect).

  • 2003 2002

    Official Language Monitor and Substitute Teacher

    Commission Scolaire des Chics-Chocs

    Gaspe, Quebec

    Developed & delivered oral English-langauge activities for students in Primary 3-6 and Secondary 1-5 (grades 3-11)

MobileHCI'17 - Paper Accepted

Our paper exploring different ways of augmenting content creation using finger-tracking on tablets will be out this fall!

posted Apr 28, 2017, 7:58 AM

AIED'17 - Submissions Accepted

Our submissions about how to show learners conflict between their beliefs and the system's assessment of their knowledge have been accepted :) Lamiya will be going to Wuhan this summer.

posted Mar 25, 2017, 4:03 PM

CALL 2017 - Submission Accepted

Jia Li will present our survey of mobile apps for vocabulary development.

posted Feb 24, 2017, 2:34 PM

IUI 2017 - Poster & Demos PC Member

Some really cool work will be demoed at IUI

posted Jan 04, 2017, 3:54 PM

ICALT 2017 - I'm joinging the APTeL track PC

I'm looking forward to reading papers about adaptive systems and learning :)

posted Dec 10, 2016, 10:29 AM

AAAI 2017 - Our MOOC paper was accepted!

My collabroators will be presenting this work about using discourse features to predict when instructors intervene in MOOC discussion forums.

posted Dec 1, 2016, 4:17 PM

GI - HCI 2017, Associate Chair

I'm serving as an AC on the GI program committee and am excited to read everyone's submissions

posted Nov 8, 2016, 8:19 AM

AERA 2017 - Both Submissions accepted

Now on to the hard work - preparing the presentations for our AERA submissions. I'm super lucky to be working with Krystle Phirangee and Jim Hewitt on interesting projects!

posted Nov 3, 2016, 11:03 AM

PITT - Cognitive Psychology Brown Bag Talk

I enjoyed talking with these students and faculty about my work. It was fun to go through techniques and approaches that were so different from what they typically use but that describe their experiences.

posted Oct 26, 2016, 2:15 PM

UIUC Talk - Computer Assisted Language Learning Series

Really enjoyed my talk and visit at UIUC. Thanks to my fabulous hosts: Luc Paquette, Aurore Mroz, and Xun Yan.

posted Sept 29, 2016, 8:47 PM

Learning Analytics in Practice at iKnow 2016

I will be serving on the program committee for LAP. I'm looking forward to reading about how people are presenting and using learning analytics.

posted Jul 23, 2016, 4:33 PM

EDM 2016 Submission Accepted

Our work on learner activity patterns within MOOCs will be appearing at EDM!

posted Apr 19, 2015, 1:33 PM

CHI LBW PC - Associate Chair

I've volunteered to serve on the Specific Application Areas subcommittee.

posted Jan 10, 2016, 4:00 PM

GI - HCI Associate Chair

I'll be serving on the program committee for the HCI track at Graphics Interface.

posted Dec 27, 2015, 8:45 AM

CALICO 2016 Submission Accepted

It looks like I'll be giving a talk about how English language learners use mobile technologies to support their noticing activities.

posted Dec 22, 2015, 11:30 AM

LAK '16 Submission Accepted

I get to present my work on English language learners' experiences of formal and informal learning environments in Edinburgh this upcoming April.

posted Dec 21, 2015, 1:58 PM

AERA Submission Accepted!

My work with Krystle Phirangee was accepted to AERA 2016!

posted Nov 5, 2015, 4:23 PM

Defended!

My final oral exam, which was on Monday, went well. I have a few revisions to make and then I'm all done.

posted Oct 21, 2015, 10:55 AM

Papers Accepted

I just had papers accepted to IEEE TLT and ETRD!

posted Mar 4, 2015, 12:06 PM

Vocabulary Improvement!

The vocabulary of students in the treatment group appears to have improved.

posted Jun 5, 2014, 11:29 PM

#lasi14

I was accepted to LASI and am excited about going again.

posted May 8, 2014, 8:32 PM

Deployment started

We handed the devices out to our students on Friday; we explained what could be done with the app and demonstrated several features. Overall, I would say that student response was positive. A few of them were somewhat concerned about how the introduction of the devices would affect their workload and how they were expected to use the app.

posted Apr 12, 2014, 10:28 PM

Settling into Life in Japan

My first week here has been great. My hosts at KGU and KGHS have been very kind and helpful. I'm really looking forward to working with my partner teachers to integrate my system into their class.

posted Apr 5, 2014, 4:49 AM

Visit to Dundee

I will be giving a talk in Dundee in a few weeks.

http://www.computing.dundee.ac.uk/events/index

posted Feb 2, 2014, 9:16 AM

Open Learner Modeling - University of Birmingham

I have just joined the open learner modeling group at the University of Birmingham and will be working with Professor Susan Bull for the next five months.

posted Oct 5, 2013, 8:47 AM

AIED 2013

Memphis was great and I got many good comments on my work when I was at #AIED2013 last week. https://sites.google.com/a/iis.memphis.edu/aied-2013-conference/

posted Jul 16, 2013, 8:40 AM

LASI 13

Had a great week at #LASI13 http://www.solaresearch.org/events/lasi/.

posted Jul 5, 2013, 7:45 PM

Awarded Sumner Fellowship

http://web.cs.toronto.edu/program/currentgradstudents/Awards/WCSumner.htm

posted May 21, 2013, 11:58 AM

Awarded a Weston Fellowship

I have recently been awarded one of the first Weston fellowships and will be going to work with Susan Bull at the University of Birmingham. I will also be conducting a study at a high school in Japan.

http://boundless.utoronto.ca/campaign-updates/inaugural-weston-fellows-announced/

posted May 10, 2013, 1:24 PM

  • image

    How to learn English faster and with less stress: Weston Fellow Carrie Demmans Epp has an app for that

    Carrie Demmans Epp. (2013)

    U of T News, University of Toronto

    http://www.news.utoronto.ca/how-learn-english-faster-and-less-stress

  • image

    Graduate Profiles: Carrie Demmans Epp

    Carrie Demmans Epp.(2012)

    Department of Computer Science, University of Toronto

    http://web.cs.toronto.edu/Page25336.aspx

  • image

    Undergraduates on Research Team Behind English Language Mobile App

    Carrie Demmans Epp, Naomi Cui, Amirezza Abdolmaleki, Justin Djordjevic, and Ronald M. Baecker. (2012)

    Faculty of Arts and Science Year in Review, University of Toronto

    http://issuu.com/year-in-review/docs/2012_year_in_review/24

  • image

    Hello from Saskatoon Canada! Ambassador: Julita Vassileva, Canada

    Julita Vassileva, Carrie Demmans Epp. (2009)

    ACM-W Women in Computing News Blog

    http://acmwnews.blogspot.com/2009/04/hello-from-saskatoon-canada-ambassador.html

My log is an experience sampling application that was designed to collect data about the communication and affective experiences of language learners. It uses signal-based sampling and collects data through a variety of established self-reporting instruments and other questions.

It can collect data for any combination of three time periods: the current moment, the entire day, or the previous week.

 
image
 
image
 
image


Context-related Questions

The context-related questions collect information that can help to detail users' experiences. It can also be used to interpret their responses to the questions about the variables that are of interest to the research project, mainly learner's communication and affect.

 
image
 
image
 
image


Communication Related Questions

The perceived competence scale (PCS) is used to collect information about user's sense of self-efficacy with respect to their ability to communicate. The momentary questions collect information about a recent communication attempt and the daily version collects information about the user's communication experiences over the course of his or her day.

Momentary
image
Daily
image
PCS
image
 
image


Affect Related Questions

The short form of the international version of the positive and negative affect schedule (I-PANAS-SF) and the self-assessment manikin (SAM) are measures that have been validated for their ability to collect information about a person's affective state. The I-PANAS-SF requires language knowledge and appears to have better discriminant validity. However, the version of the SAM that is used prevents limited language knowledge from impeding on the user's assessment of their affect.

Momentary
I-PANAS-SF
image
 
SAM
image
 
Additional Context
image

Daily
I-PANAS-SF
image
 
SAM
image


Language Learning Related Questions

The perceived competence scale (PCS) is used to collect information about users' sense of self-efficacy with respect to their ability to learn English.

PCS
image
 
image

Viewing Vocabulary on Web

The user can view a list of all of his or her active words.

image

The user can view a list of all of the vocabulary items that she or he has deleted.

image

Viewing Vocabulary on Android

image

The detailed view of a single word.

image

The list view of the user's words.

Adding/Editing Vocabulary on Web

Initial word list from which vocabulary entries can be selected for editing.

image

Screen after clicking the Edit button for "ambulance".

image

The User's changes.

image

The Screen after clicking the Save button.

image

The deleted words list after deleting "andouille sausage".

image

The word list after the Delete button has been clicked for "andouille sausage".

image

The word list after clicking on the Undo button for "agnolotti".

image

The deleted word list after restoring "agnolotti".

image

To add a new word the user clicks on the Add Word + button at the top of the screen. S/he is then taken to the same screen that is used to edit words.

image

The user then enters the word and its accompanying information.

image

After clicking on the Save button the user is shown his/her new word within the context of the larger vocabulary collection.

image

Adding/Editing Vocabulary on Android

image

Creating a new vocabulary entry

image

The word Editing screen.

image

A new word once the user has retrieved or written the definition and added an image. This screen shows the user editing the sentence that accompanies the word.

image

The image that is associated with a word can be changed using the device camera, items from a library that are automatically generated using Internet-based corpora, or information retrieval techniques.


image

The tags or categories that words are associated with can also be edited. The tags that are associated with a vocabulary entry are used during search and to refine the vocabulary based on the user's context.

image

The application searches through existing tags and suggests them when you are creating a new tag association for a vocabulary entry.

image

The same screen after adding another tag (i.e., fruit).

image

When the trashcan is selected the user is first given the option of removing the tag from the individual word.


image

The user is then asked if they would like to completely delete the tag (i.e., remove it from all of the words with which it is associated).

image

Individual tags can be edited.

image

We allow users to edit the tag for all of the vocabulary entries with which it is associated or the individual word on which they are working.

Searching for Vocabulary on Android

image

User's can search through vocabulary items based on the categories to which the item belongs (i.e., tags) or just search through the words. By default, the system searches both.

image

The system starts searching as soon as characters are entered.

image

The user has the option of adding tags as filters if they come up during the search.

image

Once a tag filter is added, all searched items will have some association with that filter.


image

The user can further refine his/her search and see both the tags and words that meet the search criteria.

image

The user has chosen to add another tag-based filter to his/her search.

image

The user has completed his/her search and can only see the vocabulary entries that match the search criteria.

Synonym Recommendation on Android

image

The system shows the user synonym sets for items that are believed to be known or nearly known.

image

The user can then collapse and expand these synonym sets as s/he sees fit.

Pronunciation Modeling on Android

image

One of the screens from which a user can obtain a pronunciation sample.

image

A pronunciation sample of the word is being provided using text to speech.

image

The user can also hear the accompanying sentence. The audio is generated using text to speech.

image

The user can record another person or him/herself saying the word.


image

A recorded sample pronunciation is being played. This can be used in 2 ways: to listen to a native speaker or to compare their own pronunciation to that of the text to speech or another English speaker.

image

Users can keep multiple recordings. This allows them to see how they are progressing and can allow them to record other English speakers who may exhibit regional pronunciations of vocabulary items.

image

Users can delete old recordings.

image

Users can also hear a model of the correct pronunciation of vocabulary items from the main word list. Clicking on the speaker will initiate the text-to-speech engine. Beyond pronunciation modeling, this allows users to support their communication with others.