Accepted contributions
• A Framework for Evaluating a Personal Learning Environment Using Digital Storytelling
Nikolaos Marianos, Nikos Manouselis, Madalina Ungur, John Stoitsis and Alexios Dimitropoulos
Abstract. “Learning about Politics” is a Personal Learning Environment developed by POLITICS project, which uses digital storytelling to help people learn about Politics and Active Participation. Digital Storytelling is a rather new inquiry-based educational technique which engages the learners in attractive interactive activities and stimulates their learning abilities. In digital storytelling, learners use Information and Communication Technologies, Learning Objects and multimedia, to write their own interactive stories and learn at the same time. The pathways used to guide the learners through the completion of the stories, the used Learning Objects and the developed stories themselves have to be evaluated to comply with quality standards. Only high quality content can actively engage the learners. This paper presents the evaluation methodology proposed in POLITICS and the results from an open validation trial.
• Evaluating First Experiences with an Educational Computer Game: A Multi-Method Approach
Marianna Obrist, Florian Foerster, Daniela Wurhofer, Manfred Tscheligi and Jörg Hofstätter
Abstract. This paper presents our evaluation approach for a specific case study, namely the evaluation of an early prototype of an educational game with children aged between 12 and 14 years. The main goal of this initial evaluation study was to explore children’s first impressions and experiences of the game on the one hand and to assess the students’ ideas and wishes for the further development of the game on the other hand. The main challenge for the evaluation activities was the selection of the appropriate methodological approach, taking into account children as a special user group. We opted for a combination of different methods that were reported beneficial for work with children in Human-Computer Interaction (HCI) and integrated these into different phases of the evaluation process. By presenting our multi-method approach in detail, other researchers can benefit from our experiences when evaluating games with children.
• Heuristic environments
Ines Giunta
Abstract. Assuming that the complexity of learning depends on the complexity of the environment within which it matures, then it suggests the need to structure learning environments flexible according to the corresponding levels of complexity. In light of this consideration it is possible to interpret the educational success like a dialectical spiral between the corresponding levels of learning and environments. Consequently, the experiences that are done within flexible learning environments are connoted, from time to time, for their adequacy with respect to the logic level of learning that intend to support. In particular, this research work focuses on the qualities they must possess in order to promote higher level learning (learning logic level 3).
• Monitoring Collaboration in Flexible and Personal Learning Environments
María Jesús Rodríguez Triana, Alejandra Martínez Monés and Juan Ignacio Asensio Pérez
Abstract. Both virtual and personal learning environments (VLEs and PLEs) are technologies widely used with educational purposes. Moreover, there also exists a pedagogical trend towards the development of high-level competences, such as the capacity to work collaboratively in a group. These two trends come together within thefield of computer supported collaborative learning (CSCL). The evaluation of CSCL situations requires to take into account, among other things, the process of collaboration. In response to this need, a number of collaboration analysis tools have been developed. However, the integration of these tools in the decentralized scenarios typically found in PLEs is a complex matter: the information retrieved and the access to it are heterogeneous; and the lack of data sharing standards between the learning software and the analysis tools reduces the probability of compatibility. The present work aims to delve into the problem of the integration of collaboration analysis tools and learning software. A solution is proposed to support monitoring in an architecture that already integrates third party external tools in PLEs and VLEs named GLUE!. The proposal is illustrated by means of an example based on a real CSCL experience that took place in a course at our University.
• PLE and PLN experience evaluation using 3D visualization indications
Francesco Di Cerbo, Gabriella Dodero and Laura Papaleo
Abstract. The paper describes our proposal for adopting 3D spatial information, based on suitable indications and metrics, for the evaluation of experiences conducted in a Personal Learning Environment (PLE) and its social extension, the Personal Learning Network (PLN). The use of 3D space is a mean for integrating different perspectives (e.g., structured and non-structured learning) but also to foster the cooperation between users. Moreover, information coming from the use of 3D space can represent a precious feedback for better understanding ongoing learning activities, but it will be extremely useful for enhancing and improving the development of existing PLE-PLN support tool.
• Understanding and designing Personal Learning Environments in Higher Education
Bernadette Charlier, Denis Gillet, France Henry and Daniel Peraya
Abstract. Recently, the Personal Learning Environment (PLE), based on the wide availability of Web.2.0 technologies, has been described as a new resource for the learning process. New technologies such as ubiquitous computing, wireless network and mobile communication allow learning to take place everywhere and every time. In addition, social software, that is predominant in PLEs, allows people to connect, collaborate, create and share. These new resources could support new learning styles, new relationships to knowledge and new opportunities to develop transfer and social learning. However, it has not yet been empirically demonstrated in what ways and to what extent PLEs contribute significantly and efficiently to the learning processes and outcomes in the context of Higher Education (HE). Our contribution aims at providing a complex, multi-level, consistent theoretical framework to study Personal Learning Environments.
• Challenges of Coordinating International Usability and User Experience Tests for Personal Learning Environments
Effie Law, Hans-Christian Schmitz, Marcel Berthold, Albert Dietrich
Abstract. Coordinating international usability and user experience tests (IU2XTs) is deemed extremely challenging, because each test site is situated in a specific organizational, technical and socio-cultural context. We have coordinated such IU2XTs in the project of ROLE (http://www.role-project.eu/) in which several testbeds are involved in deploying and evaluating Personal Learning Environments (PLEs). They include Higher Education Institutions and global enterprises in and beyond Europe, representing a range of values and assumptions. Whilst the diversity provides fertile grounds for validating our research ideas, it poses many challenges for conducting comparison studies. In the paper, we first describe the general research protocol of our IU2XTs and then present the issues we have experienced as well as the resolutions attempted. Next, we report some preliminary findings that serve as input for the future development of ROLE. Reflections on these processes and experiences can be distilled as best practices for other projects facing similar challenges.
• What and how to monitor complex educative experiences. Toward the definition of a general framework
Carlo Giovannella, Simone Carcone, Andrea Camusi
Abstract. A result of re-considering educative processes as 'experiences', i.e. multidimensional processes of high complexity and flexibility, is a decrease in the predictability of processes’ outcomes. As consequence, a natural transition from evaluation approaches (produced by a deterministic view) to monitoring practices is needed and expected. Such transition requires to invest a quite considerable effort in design and development of new and more powerful monitoring strategies, methods, tools and environments. In this paper, using appropriate examples, we present and discuss possible strategies that can be adopted to achieve the goal. We also present a parallel participatory strategy that can be adopted whenever tools to monitor specific dimensions of the educative experience are not yet available.
