October 4-6, 2006
Lima, Peru

Design and Evaluation of Computer-based
Learning Environments

Prof. Dr. Heinz Mandl
University of Munich, Germany

Description
When we look at the design and evaluation of computer-based learning environments, there are especially three different aspects which must be considered: (1) the aim of the computer-based learning environment, (2) the theoretical principles lying behind the design and the (3) evaluation criteria. The reason for designing a computer-based learning environment is the first important antecedent. This concerns e. g. knowledge, competences or skills learners are supposed to acquire by working with the learning environment. Depending on this first decision are design and evaluation. Design principles are based on theoretical approaches on learning and instruction. There are especially two theoretical approaches relevant: a cognitive and a constructivist theoretical concept. While in a learning environment based on cognitive assumptions all effort is devoted to instruction and the planning, organizing and guidance of the learning process, in a constructivist-oriented learning environment, the learner and its learning process is focused on. Evaluation criteria are relevant for measuring the effectiveness of the computer-based learning environment. According to Kirkpatrick (1996), there are especially four evaluation steps necessary: Reaction, learning, behaviour and results. Reaction is defined as the acceptance of the computer-based learning environment. Only if learners like it, will they learn with it effectively. Learning is the second step. In the third step it must be evaluated whether the knowledge, skills, or competences learned by an individual are transferred into one’s everyday behaviour. This change in behavior may have an impact on the organization as a result of the learning environment.

Aims, design principles and evaluation criteria of computer-based learning environments are a main topic in this session. Six different computer-based learning environments are presented and illustrated with data of its realization and implementation in higher and further education. Furthermore, a detailed discussion will follow to show advantages and disadvantages of the various learning environments.
The first contribution of Tergan gives an introduction to the topic of evaluating e-learning environments in general. He presents a comprehensive approach that includes every aspect relevant for evaluations in virtual learning environments.

The second contribution of Stark and Krause shows the hypertext-based learning environment WALe which was developed to facilitate competence change in educational argumentation in higher education with problem-oriented learning principles.

Also based on constructivist theories is the third learning environment of a virtual seminar which focuses on collaborative learning in higher education (Kopp, Schnurer and Mandl). Besides design principles and evaluation data, detailed process-product analyses are presented which shows the relevance of content-specific activities for learning in virtual learning environments.

This kind of virtual seminar is the object of investigation in the contribution of Diekamp, too. He evaluates the interpersonal knowledge of collaborating partners and its influence on satisfaction, learning outcome as well as acceptance. It will be shown, how interpersonal knowledge can develop in asynchronous learning environments. Further on, the effects of interpersonal knowledge on learning processes and outcomes are analysed.

Instructional design is the basis of Zumbach’s contribution which concerns an hyperaudio learning environment with cellular phones. The non-linear audio-based learning environment will be presented as well as results on learning processes and outcomes.

The last contribution of Niegemann exemplifies the design of learning environments by showing an E-Learning design assistant. This system aims to assist developers on designing multimedia learning environments with instructional design theory depending on six main design decisions.

Single contributions:

• Sigmar-Olaf Tergan: Evaluating e-Learning environments. A comprehensive approach.
• Robin Stark, & Ulrike-Marie Krause: Facilitating competence change in educational argumentation – conceptualization and evaluation of a problem-based learning environment.
• Birgitta Kopp, Katharina Schnurer, & Heinz Mandl: Collaborative learning in virtual seminars – design principles, evaluation data, and process-product-analyses.
• Oliver Diekamp: Evaluating interpersonal knowledge in virtual seminars – one important aspect for satisfaction, outcome, and acceptance?
• Jörg Zumbach: Instructional design for hyper-audio learning with cellular phones.
• Helmut Niegemann: Integrating and Conveying the Knowledge on How to Design Multimedia Learning Environments – The Development of an eLearning Design Assistant
  

Please send all paper submissions to: mandl@edupsy.uni-muenchen.de

Biography
Heinz Mandl is full professor of Education and Educational Psychology at the University of Munich since 1990. He was full Professor of Education at the University of Tuebingen from 1978 to 1990, head of the Department for Learning Research at the German Institute for Distance Education at the University of Tuebingen form 1978 to 1990 and Director of the German Institute for Distance Education at the University of Tuebingen from 1988 to 1990. He was elected speaker of the section "Educational Psychology" of the German Association for Psychology and of the interdisciplinary working group "Intelligent Learning Systems" (German Association of Computer Science) from 1987 to 1991, as well as president of the European Association for Research on Learning and Instruction from 1989 to 1991. Since 1999 he is a member of Societas Scientiarum Fennica (Finland). He received an award for the implementation of virtual learning environments to the University of Munich of the Bavarian State (Minister of Science, Research and Art) in 2001. His research topics concerns the analysis and support of knowledge acquisition and the application of knowledge in schools, universities and organizations/corporations (knowledge transfer), self-guided and cooperative learning, net-based knowledge communication in groups, personal and organizational knowledge management, and the design, implementation and evaluation of innovative learning environments with new media.

Key publications
Weinberger, A., & Mandl, H. (2003). Computer-mediated knowledge communication. Studies in Communication Science, 81-105.
Weinberger, A., Fischer, F., & Mandl, H. (2003). Gemeinsame Wissenskonstruktion in computervermittelter Kommunikation: Wirkungen von Kooperationsskripts auf den Erwerb anwendungsorientierten Wissens. Zeitschrift für Psychologie, 211 (2).
Fischer, F., Bouillion, L., Mandl, H,. & Gomez, L. (2005). Towards a conceptual and methodological anatomy of Pasteur’s Quadrant: Bridging theory and practice in learning environment research. International Journal of Educational Policy, Research & Practice.
Fischer, F., & Mandl, H. (2005). Knowledge convergence in computer-supported collaborative learning - the role of external representation tools. Journal of the Learning Sciences, 14(3), 405-441.
Kopp, B., & Mandl, H. (2006). Gemeinsame Wissenskonstruktion. In H.-W. Bierhoff & D. Frey (Eds.), Handbuch der Sozialpsychologie und Kommunikationspsychologie (pp. 504-509). Göttingen: Hogrefe.
Mandl, H., Ertl, B., & Kopp, B. (in press). Cooperative learning in computer supported learning environments. In L. Verschaffel, F. Dochy, M. Boekaerts, & S. Vosniadou (Eds.), Instructional psychology: Past, present and future trends. Sixteen essays in honor of Erik De Corte. Oxford: Elsevier.
Stark, R., Gruber, H., Renkl, A., & Mandl, H. (1998). Instructional effects in complex learning: do objective and subjective learning outcomes converge? Learning and Instruction, 8(2), 117-129.
Stark, R. & Mandl, H., (in press). Web-based learning in the field of empirical research methods. Computer & Education.

Back to CATE 2006 Home Page

© Copyright 2005 IASTED www.iasted.org