The Learner’s Role in Multimedia Systems

Alty (1991) criticizes the exclusive stress on the technological perspective of multimedia, the mixture of media, from the consumers’ point of view: »Whether users actually want to do this is never questioned« (32). Alty emphasizes a user-centred perspective, and agrees with Elsom-Cook’s (1991) statement: »The effort in developing the technology has not been matched by a similar concern with the pedagogy […] At present it is an article of good faith that multimedia is a good thing for education and training. There is no evidence that multimedia enhances learning, or makes it more cost effective«. I will go into the question whether this is an accurate assessment in the chapter on evaluation at the end of this book. What interests me here is the user perspective point of view demonstrated in these statements. To reduce the event space to the dimension of an interface would be to underestimate the importance of the learner and interaction. I am dealing with the role of the learner and of dialogue in this section. In the next section, I will throw some light on certain aspects of interaction.

Tetrahedron Model by Fischer and Mandl (1990)

One of the few approaches that include the learner in the modelling of multimedia architecture is the Tetrahedron Model by Fischer and Mandl (1990), which distinguishes between the following dimensions: They discuss the four triangular relations or interactions resulting from looking at the tetrahedron’s sides:
  1. the interaction of learner x goal x domain
  2. the interaction of learner x goal x learning activity
  3. the interaction of learner x domain x learning activity
  4. the interaction of learning activity x goal x domain.
I do not think much of graphical models as a rule, because they tend to disguise problems in the relationships between the individual categories rather than expose them. But in this case I can readily accept the model as a heuristic pointer for the necessary discussion of the role of the learner in hypermedia. It provides a systemic place for the learner’s role and stresses the intentions and ideas that the learner brings to learning with hypermedia. The learner’s role is related to both the domain and the intended learning objectives (interaction 1). The model points out the learner’s goals and ideas which influence his own learning ability (interaction 2). And it points out the interaction between learner and hypermedia object mediated by learning activities (interaction 3). Finally, it defines the »traditional« level of theories of instructions, the relation between domain, learning objective, and learning method (interaction 4).

Fischer and Mandl call hypermedia ‘virtual media’, because they consist of »subsources«, of partial objects, whose appearance and meaning only come into existence through an act of interpretation. This indication of the reception and interpretation work of the learner, who synthesizes the hypermedia as a meaningful object through his own cognitive constructs, seems very important to me. The hypermedia construct is an abstract entity as long as it exists independent of the user. I do not want to discuss here whether we should therefore, as Fischer and Mandl suggest, develop a »psychophysics« of hypermedia (because psychophysics distinguish between the objective stimulus and its subjective perception, a distinction that is problematic from an epistemological point of view if we follow the theory of Maturana and Varela (1987), according to which there is no objective reality independent of the cognizant subject), or whether it is sufficient to call this aspect user dialogue or learner interaction and to model it accordingly. The indication points toward the common epistemological characteristics of all cultural objects, at any rate, it applies equally to any unread book, any unwatched painting, and any unplayed record. These »sleeping« objects, too, are only »awakened« through the interpretation of the reader, viewer or listener, whom Fischer and Mandl call a ‘fuzzy entity’. In this sense, all cultural objects that are only opened up through interpretation would be virtual objects.

The meaningful level of hypermedia objects is thus an object level that is only constituted in an interpretation. To call it an ‘abstract entity’ for that reason, thus assigning an ‘objective structures’ status to uninterpreted objects, is at the very least misleading in my opinion. The objects do not have any objectivity outside the viewers interpretations, a fact already pointed out in neurobiology: »The meaning of signals does not at all depend on the nature of the signals, but on the conditions under which they are taken in by the receiver. It is the receiver that constitutes meaning.« [Roth (1997), 106ff] The hypermedia object’s dependency on the user points toward the social-hermeneutic dimensions of interaction with programs, and towards the necessity of viewing and modelling the event space as a pedagogical space as well (the term »pedagogical« is not meant in the sense underlying the concept of learning as instruction here).

Learner Modelling

Winograd and Flores (1987) arrive at a similar conclusion in their analysis of expert systems. Since it is impossible in their opinion to model the knowledge of experts in the knowledge component of expert systems, it would be sensible to complement the knowledge model in multimedia systems by a user model or learner model, and to model the multimedia dialogue with the methods of dialogue design and the aid of learning psychology theories as a user model in the form of intentions, plans and strategies. Unfortunately, however, the state of research is not very advanced in this field [Fischer/Mandl (1990), XXIV]. Card, Moran et al (1983), who sketch the learner’s interactive dialogue with the computer as a GOMS model (»Goals, Operators, Methods for achieving goals, and Selection rules for choosing among competing methods for goals«), have created a model in the triad of goals, methods, and operators that in my opinion is only suited for the technical dimension of user interactions, which only appear as discrete events for the computer [McIlvenny (1990)]. An extension of the model by linguistic speech acts, sociological dimensions of role-switching, and the power or dimension of communicative action seems impossible to me, and not sensible either.

Interaction Analysis

User modelling in human-machine-dialogues [Kobsa (1985); Bratman (1987)] does not look at the dialogue from a perspective of learning aspects, but rather as a functional interaction with a technological artefact. Approaches to learner modelling for IT systems do not analyse the learning process as an interaction of user and program, but merely as a cognitive model with regard to the respective domain.[Overviews of user modelling in IT systems can be found in Murray (1988), Nielsen (1990), Diaper and Addison (1991), Dillenbourg and Self (1992a)] The speech act theory does not (yet) deal with the linguistically reduced speech acts in communication with a technical interaction partner like a computer [Rehbein (1977)]. And conversation analysis approaches, even if the authors choose information science for their addressee like Wooffitt (1990), have an explicitly sociological focus and do not offer any base for dealing with the interpretative dialogue of a learner with a multimedia program [s. the contributions in the reader by Luff and Gilbert et al (1990)]. One of the few exceptions is Schank and Abelson (1977), who conceive the dialogue with the computer in the form of scripts, plans and goals. Dialogue systems may then be seen as a scenario within which certain language games are played out. The user is modelled through orientation knowledge and action knowledge. The protagonists’ actions are described as intentions, plans, and practical arguing [Bratman (1987)], i.e. as intentional acts with a »dialogue partner« restricted to functional interactions, with the term intentional act clearly only meant in the restricted meaning of instrumental intentions here.

Human-Computer Interaction as Instrumental Action

Schanck’s and Abelson’s model depends, however, on being able to count on a relatively limited domain and as complete as possible scripts for the interaction [s. Minsky’s (1992) criticism of ‘understanding’ in scripts]. This is just what is not possible in human communication, which can at any time clarify the assumptions underlying a communication, and progress into discourse, with the clarification possibly leading to further assumptions having to be clarified in turn. A computer program cannot handle this infinite regression or metacommunication. It must start out from a »fixed set of assumptions« [Suchman (1987), 61]. The user actions are restricted by the program design, the design »assumes, however, that it is the correspondence of the system’s plan to the user’s purposes that enables interaction« (100). This correspondence assumption takes away space for necessary interpretative acts, because »no action can fully provide for its own interpretation« (67). The analysis of human-computer interaction with plans and goals is an analysis of a goal-oriented event, or of a planning model for instrumental action. It is impossible to arrive at an agreement about statements and action orientations, or even to call into question the extent of the validity of propositions. Such a definition of human-computer interaction as instrumental action perfectly corresponds to the communication theory approach of Habermas (1981) and his distinction of instrumental, strategic, and communicative action (367ff). In instrumental domains, no action that is oriented towards understanding is possible (384ff).

A good part of learning – and also of learning with computers – takes place in dialogue, and this interactive learning is probably the most effective and lasting factor of learning, but there is no theory or model of the dialogue with the computer and learning program on the horizon as yet [Forrester (1991)], much less a concept of dialogic interactive learning.

Human-Program Interaction

There is another dimension that seems to me to have always been neglected in most reflections on computer-human interaction. In my opinion, most models start out too much from conversation or interaction with the computer as machine or tool. But especially if we are dealing with multimedia, we are less interested in human-computer interaction than in human-program interaction. If software plays a part in user modelling, it is usually the operating system, tool programs, or a word processor. A look at multimedia applications, however, which represent a »conversation partner« for the learner, is likely to make the computer as machine or its operating system retreat and become invisible behind the application. Such a view should be able to open up new perspectives, although this is still not a case of communicative action, because the program cannot supply the partner with whom it is possible to negotiate situation definitions, and the program user is the only protagonist who can furnish the necessary interpretations for communicative action.

Constructivism and the Computer

For Maturana and Varela (1987), actions with the computer are merely instructive interactions with structurally determined units (107) corresponding to a simple input-output model, while they demand an independent dynamics of internal relations even for the nervous system (184ff). The idea that the computer (even today) is a machine, and that communication with the computer follows other rules than communication between people, is a widespread one, and has led to a reduction of perspectives on interaction as machine communication, command language, or instruction.

But the overall configuration that we meet in learning with computers is a little more complex, really. More recent positions of the constructivists (I will go into constructivism in detail in chapters 3, 5 and 6) on human-machine-interaction show an intention to apply insights on human communication from psychology and ethnology to human-machine interaction as »shared understanding« [Suchman (1987)]: »The initial observation is that interaction between people and machines requires essentially the same interpretive work that characterizes interaction between people, but with fundamentally different resources available to the participants« (180). The computer is part of the learners’ empirical world, and at the same time, it is also a medium for a learning program in which the author has invested certain intentions, intentions directed towards the learner’s learning that are meant to more or less »communicate« with, or at least »mediate« themselves to him. The computer is thus no mere tool any longer, but a ‘culturally situated object’, as Winograd and Flores (1987) emphasize: »An understanding of what a computer really does is an understanding of the social and political situation in which it is designed, built, purchased, installed, and used« (84). Radical constructivist approaches start out from the computer’s social embedment and thus do no longer exclude the application of the paradigm of communicative acts to interaction with computers or conceiving this interaction as a discourse model (one must observe, however, that many information scientists no longer use the term »discourse« in the sense of communication theory, but in a sense exchangeable with technical dialogue).

Hermeneutics

With Winograd and Flores, and with regard to the occidental hermeneutic tradition of textual interpretation [or pointing to the current state of computer technology], one may answer the question put over and over again in discussions, whether computers are »intelligent«, negatively, and consider it irrelevant. The negative answer applies to the machine, the physical hardware, from which we must distinguish the program run by the machine, the software, [s. the clear indication of this difference in Brown (1985), 200]. The other question to be distinguished from the first one, i.e. whether hermeneutic understanding plays a role in learning with a learning program, e.g. a hypertext or an electronic book, can quite simply not be answered differently than it would be if the question referred to the reading of books. The computer has developed from a mere tool, which could be caused to execute certain events with the help of commands, into a device that presents cultural subject matter to the user in the form of texts, pictures, and videos. Dealing with multimedia information must be regarded in the hermeneutic tradition of reading as a matter of course.

Human-Software Dyad

The question, however, whether multimedia, which can sometimes »give life« to books through films and animations, adds new dimensions to this form of reading, is not so easy to answer. Multimedia certainly does not suddenly turn the asymmetrical relation reader-book into a symmetrical, reciprocal relation, a communicative discourse, even if the respective multimedia program could be called »interactive«. Nickerson (1987) discusses this question not on a level of communication theory, but pragmatically, using 16 criteria which for him characterize conversation, e.g. bidirectionality, alternating initiative, presence, a shared situational context and equal status of the partners, arriving at the conclusion: »The model that seems appropriate for this view of person-computer interaction is that of an individual making use of a sophisticated tool and not that of one person conversing with another« (691).

This view is still determined by the tool paradigm of the computer. After what I have just said about the change of paradigm from computer as tool to computer as vehicle of culture, the answer would turn out somewhat differently today: the user deals interpretatively with the subject matter and meanings that the multimedia author’s software has to offer. The dimensions that multimedia adds to the book will nevertheless be situated on quite different levels, i.e. those of perception psychology, motivation psychology, and cognition psychology, and not on the methodological level on which we decide on instrumental vs. communicative action, or symmetrical vs. asymmetrical communication, on judgements, and the validity of norms [cf. Penrose (1995), 397]. With regard to these functions one could speak of the computer as a communication tool, a »communication facilitator« [Brown (1985), 199]. That would make clear that neither computer nor multimedia program »understand«. They only understand unambiguous, discrete actions, they do not interpret the subject matter of this symbolic interaction. There is thus no relationship of a human to a »partner«, even if we do not regard the interaction from the point of view of the human-computer dyad, but the human-software dyad [Floyd (1990)].

Mental Models

A definition of multimedia thus consists of the dialogic, interactive component of the multimedia system, and of the interpretation and manipulation of multimedia objects by the learner. The learner triggers off events by manipulating multimedia objects. This presupposes a familiarity with the methods that multimedia objects trigger off in the representation level (ease of manipulation as an objective of multimedia designers). In doing this, the learner makes use of hypotheses about the methods triggered off by the objects in the deep structure. This is why the designers of learning programs would love to get at the learners’ mental models, their interpretations of multimedia objects, in order to be able to anticipate or ‘harmonize’ them in the design [Vacherand-Revel/Bessière (1992), 60]. But all attempts at developing user models have become stuck on relatively low levels of interaction up to now.

Interpretations

The individual interpretations, as well as the individual learning strategies of the learners, are brought into this hypothesis formation, and influence the strategies of interaction with the respective system and the preferment of certain systems, in alternative and free choice, as well as the perception of the multimedia product’s structure: »The user’s goal or intentions add partial structure to the hypermedium by overlaying expectations onto the hypermedium and its data, a structure which guides the user’s browsing« [Fischer/Mandl (1990), XXI].

Individual Styles of Learning

But this is another aspect of interaction that belongs to the underdeveloped aspects of research on the design of the dialogue component or the student models. Research on students’ cognitive styles of learning and learning strategies has found little attention, for example [Entwistle (1976), Entwistle/Entwistle et al (1993), Entwistle/Hounsell (1975), Hounsell (1983), Pask (1976), Marton/Saljö (1976a), Marton/Saljö (1976b), Marton (1983), Marton/ Hounsell et al (1984)]. Veenman, Elshout et al (1992) also point out the important interaction of learning ability and learning style or method, and their significance as predictors of learning success. This research has led to a description of individual styles and strategies of learning [Schulmeister (1983b)], which can designate factual differences in the learning behaviour of students (deep processing vs. surface processing; holist vs. serialist). According to Laurillard (1979), one must however take into account that the descriptive dichotomies of learning styles are not individual styles of learning, but context-dependent variables. She compared the learning behaviour of students over a sequence of several tasks, in which different demands where put on the learners. Their protocols were analysed using the method of Marton and Saljö, with a 77% correspondence of evaluators. Her study yielded the result that students use different styles of learning for different tasks, and she concludes from this that the characteristic styles are »context-dependent, rather than student-dependent« (407). Even the strategy students choose for dealing with a task seems to be dependent on their interpretation of the environment.

There have been only few studies up to now that researched concepts of learning styles in connection with the effects of multimedia programs and can make an empirical contribution to the discussion just reported. I would like to give a short review of some of these studies in the following (more will be reviewed in the chapter on evaluation, and the chapters dealing with the evaluation of intelligent tutorial systems and hypertext systems), without however entering into the problem here on what kind of learning style constructs the studies are based:

Shute, Woltz et al (1989) tested differences in learning sytles with an ITS (Intelligent Tutorial System, s. chapter 6). They found grave differences. What is interesting about their study is that attitudes changed over time without the tutorial component being able to accommodate this.

Cordell (1991) studied students using an inventory of learning styles (after Kolb), and analysed their interaction with a tutorial that was conceived as linear on the one hand, and on the other hand used branching. Unfortunately, the tutorial is not described in her essay, so that it is impossible for the reader to assess this study’s independent variable. Differences between learning styles proved to be insignificant, the methodical variable linear vs. branching however produced clear differences in posttests. This may be due to the fact that the pedagogic structure of the learning materials was not suited to address all styles of learning in the Kolb inventory (this cannot be checked because of the lack of information).

Yung-Bin (1992) studied the effect of styles of learning and instructional advice using a program on DNA and protein synthesis. The results indicated that performance in the posttest, the amount of time needed, and the frequency of selecting embedded information was influenced by the interaction between styles of learning and instruction strategies. A comparison of active and passive learners led to the result that active learners invested significantly more time in the tasks, looked at more information, and achieved better results in the final test.

Stanton and Baber (1992) analysed students with the Embedded Figures Test with regard to field dependence vs. field independence, and had them learn with different training programs, a top-down model, and a non-linear training module. The test was designed in such a way that the learning result was of no importance. It was not a question of which group learned more effectively, but whether the groups that were formed learned differently. This hypothesis was confirmed, but in the end Stanton and Baber themselves offer the thesis for discussion that the concept of styles of learning »may well be an artefact produced by the learners’ intelligent adaptation to fluctuating circumstances« (164). They are made suspicious by the fact that »every study of this type emerges with another set of learning styles«. This indication of the meagre and uncertain standing of research in this field of learning styles should indeed be taken seriously. Many of the scales of learning styles used for comparisons have neither received any theoretical foundation, nor have they been sufficiently empirically justified.

Allinson (1992a) analysed students using Entwistle’s study inventory with regard to various dimensions (deep processing vs. surface processing, serialist vs. holist strategy) and formed two groups: high reproducing and high meaning. She transferred the Hitch-Hiker’s Guide to HyperCard and conducted an experiment with two different versions. The first version was for the training phase, the second for the learning phase. The only difference resulting between the groups concerned the assessment of the degree of difficulty: reproducing learners thought there was a higher degree of difficulty. But the analysis of the automatically written protocols proved interesting. Significant differences with regard to the use of the index and guided tour mode were found. The analysis of the navigation pointed towards differences in navigation, even if the differences were not significant: »It appears that subjects in Group 1 do conform to the expected pattern, that is showing a preference for a linear and structured presentation of the materials. Subjects in Group 2 indicated a tendency to utilise the less well structured hypertext navigation« (69). An analysis after four phases of interaction with the material indicated functional differences in using the individual tools: thus utilization of the tour method decreased in the end, while utilization of the map increased.

Lee and Lehman (1993) compared active and passive learners in their interaction with two types of hypermedia: with and without instructional hints. While the learning style variables did not differ significantly, the instruction conditions resulted in clear differences: Both passive and neutral learners achieved better results with the instructional hints variant, while the differences between the two methods proved to be irrelevant for active learners.

Yoder (1994) executed a study comparing learners with a reflective-observant style of learning (n=16) and an active-experimental style of learning (n=42), and arrived at the conclusion that the former learned more effectively with normal video, while the latter did better with interactive video.

A definition of multimedia also includes the intentions and plans that the product designer has invested in the design of the environment, and which are perhaps realized in the user’s interpretation. But a definition of the overall multimedia environment also includes the teacher, if the system is employed in a situation that is moderated by the teacher. According to Heywood-Everett (1991), the dominant interaction styles of teacher and school environment also have an influence on the use of computers in teaching. Goodyear (1992) describes the role of the teacher in the multimedia system through the following activities as a form of »knowledge communication«: