Challenges in
Evaluating Educative Experiences of Flexible and Personal Learning Environments (E3FPLE)
21st September 2011 - Palermo, Italy
a one day workshop held in conjunction with ECTEL 2011
Workshop background
Technology-enhanced learning is a complex activity involving the use of various learning tools, services, resources, and processes, giving rise to a range of user as well as learning experiences.
Emerging software tools and services enable learners to emancipate from the bounds of traditional brick-and-mortar environments and to generate contents as well as consume other-created ones. This has led to a shift from a centralised institutional teaching approach to a more learner-centred decentralised learning approach (Wilson, 2008). Propelled by this shift engendered by lifelong learning, new generation learning environments (PLEs) are required to be more responsive and open, allowing breakthrough levels of personalization. It is considered a huge challenge to develop innovative technologies, concepts and approaches that can support lifelong learners to transfer smoothly through different stages of their personal and professional development, figuratively speaking, from cradle to grave (Wolpers, Law & Klamma, under review). Similar to PLEs, Virtual Learning Places (VLPs) aim to integrate personalization, social exchanges, cultural stratification and memory (Giovannella & Graf, 2010). For both types of environment, pedagogically open design accommodating the unpredictability of the use scenarios is deemed essential.
Furthermore, technology-enhanced learning processes are complex and trajectories of tools and services usage are not always predictable. Consequently, we can no longer entrust to fully deterministic design, especially when dealing with design for experience. Alternatively, we should embrace open design, which allows experiences to be shaped in a way that meet learners’ needs and values. Accordingly, the processes should have a high degree of flexibility and adaptability (Giovannella, 2010).
Evaluation of such flexible and personal learning environments (FPLEs) which will constantly change and be adopted by diverse user groups is extremely challenging. Such evaluation should be an ongoing process with empirical findings being used as well as provided by the developer and researcher communities; the successful interplay between software evaluation and development (Law et al., 2010) is hard to sustain. Another critical issue is the inherent difficulty in defining and operationalising qualities of user experience (UX) (Law et al., 2009) in general and those of learning processes in particular. Furthermore, technological, personal, and social factors are so highly intertwined that it seems an insurmountable task to ascribe weights for their respective influence on the learning effect. Hence, evaluation of FPLEs should not be limited to the assessment of products, but should address the quality of educative experience with close monitoring (Giovannella et al., 2010).
-> call for paper and research questions
Giovannella C. (2010). Beyond the Media Literacy. Complex Scenarios and New Literacies for the Future Education: the Centrality of Design. IJDLDC, vol. 1 N. 3, 2010 pp. 18-28
Giovannella C., Graf S. (2010). Challenging Technologies, Rethinking Pedagogy, Being Design-Inspired. The Grand Challenge of this Century. eLearn Magazine, ACM ed., 25 Feb.
http://www.elearnmag.org/subpage.cfm?section=articles&article=114-1
Giovannella C., Spadavecchia C., Camusi A. (2010). Educational complexity: centrality of design and monitoring of the experience. G. Leitner, M. Hitz, and A. Holzinger (Eds.): USAB 2010, LNCS 6389, (pp. 353-372) Springer, Heidelberg, and references therein
Law, E., Roto, v., Hassenzahl, M., Vermeeren, A., & Kote, J. (2009). Understanding, scoping and defining user experience: a survey approach. Proc. CHI 2009 (pp. 719-728), Boston, USA.
Law, E., Abraho, S., & Stage, J. (2010). Proc. of the Workshop on the Interplay between User Experience Evaluation and System Development (I-UxSED 2011), NordiCHI 2010, Iceland.
Wilson, S. (2008). Patterns of personal learning environments. Interactive Learning Environments,16(1), 17-34.
