Interactivity of Multimedia Systems

Interactions with the computer are actions mediated in a special way. Actions consist of punctuation marks possessing a syntax and a semantics. Seen from the point of view of computer technology, user actions are interrupts of other processes, e.g. the event cycle or a program procedure. It is important to distinguish the physical or technical aspect of interaction from the symbolic aspect of interaction [Dillon (1990), 186].

User interaction with the computer by way of hardware events usually occurs in the form of mouse clicks, i.e. discrete events. Giardina (1992) distinguishes the following kinds of interactivity, apparently according to the degree of independence of the learner’s actions:

Singer (1993) studies even selections with the mouse from the point of view of her analogy to functions of natural language, as anaphora, deictic, and ellipsis. But even if we could underlay mouse actions with a kind of grammar of interaction, the actions of an iconic interface user with an enactive character have an extremely fragmentary character from a communication theorist’s perspective: a click here, a click there, a brief entering of text, etc. The interaction of human and computer is often regarded as analogous to human communication. This becomes clear if one looks at Herrmann’s (1986) analyses of the meaning that interaction and communication assume under different aspects of human-machine communication or social interaction. Floyd (1990) complements this aspect by an analysis of the unsuitable metaphors disguising the fact that this is not a case of social interaction.

From a software technology point of view, interactions are regarded as a form of searching for information or of entering data. This characterization is trivial, it does not say anything, because the intentional and interpretative acts of the user underlying the bit actions are not included in such a technical description of interaction. These make up the actual interaction. Again, we encounter the distinction of foreground and background, or surface and deep structure, just as we did with multimedia architecture and multimedia objects.

The task of a multimedia designer is to achieve an agreement of the user’s interaction semantics with the interaction syntax of the program. The so-called »Human-Computer Interface« is finally nothing more than a spatial and temporal form of organisation for the exchange of such syntactic elements. To regard this type of direct manipulation [Shneiderman (1983)] already as a form of conversation [Brennan (1990)] or as »communicative dialogue« [Dillon (1990), 186], reduces the terms communication and conversation to their purely technical or instrumental dimension. How freely terms like communication and interaction are used in the literature is demonstrated by Kearsley and Frost (1985), who distinguish three levels of interactivity in videodisc systems: a videodisc without a program, a videodisc with an integrated program, and a videodisc with a computer program – this means the interaction of system, hardware and software, not between program and user.

Interactivity

Interactivity marks the essential distinction between a computer supported learning program and a film: »The first benefit is great interactivity« [Kay (1991), 106]. Interactivity is not only an objective characteristic of the multimedia system, this characteristic also has a decisive influence on the user’s experience: »the design of interactivity in multimedia systems, including the choice of user interface, fundamentally affects the experience of using them« [Feldman (1994), 8]. Even if Feldman recognizes that interactivity is »one of the most obviously unique features« of multimedia (9), he warns of overestimating the importance of this characteristic. Interactivity, according to Feldman, »has become a largely unquestioned gospel […] often has just the opposite effect […] becomes too much like hard work and makes users switch off, mentally and physically […] can be too demanding for some people’s taste«. The different assessment of the function and meaning of interactivity is mostly due to the fact that for Feldman interactivity is defined above all on a technical level. If the concept of interactivity only means events like »the mechanical link or the ability to search for images on the videodisc using a computer« [Giardina (1992), 52], however, the user may quickly become bored with interaction. For Giardina, the present discussion about interactivity is »too closely identified with the technological features of the ‘tool’« (62). Giardina distinguishes physical interactivity from cognitive interactivity. As soon as one makes this distinction, the program contents and objects assume a more important role for interaction than manual operation. For Giardina, the learner and his motivation move to the foreground, therefore, which calls for consequences in the dialogue quality of interaction in programs (56). Giardina does not come very much farther in the determination of interactivity than to demand an immediate control of visual, oral and written information, because he stops at a definition of »cognitive interaction« from epistemological psychology, and does not consult any communication theory approaches. This becomes more clear in Baumgartner and Payr (1994), when they call the interface a cross-border symbol system (113ff).

Mayes, Dolphin et al (1989) emphasize that science has not as yet come up with any evidence for the efficacy of the typical multimedia components. The lively impression that multimedia has on the user is not a characteristic of the system as such, they say, but due to interaction: »The impact will always depend on an interaction with user characteristics«. Clark (1983), and Clark and Craig (1992) likewise emphasize the aspect of high interactivity in their meta-analyses on computer supported learning: if there are indeed differences found in a comparison of different media, which is seldom the case, they can probably be put down to the chosen method rather than the medium, which, with computer-mediated instruction, would indicate the interaction factor. Gloor (1990), too, sees mostly those characteristics that are related to interaction as the advantages of computer learning programs: increased interaction with the student, individualization, flexible use, increased motivation, direct feedback, simple control of student performance, learner control (198ff). What distinguishes multimedia from all other media is apparently the high degree of interaction, as long as it is utilized by the program designers. Borsook (1991) argues that the best distinguishing feature of multimedia is its potential for interactivity. In his opinion, learning systems should imitate the wealth and flexibility of human interaction as far as possible, and make partners of computer and learner. As conditions for successful interaction, Borsook and Higgenbotham-Wheat (1991) name the interaction’s response behaviour immediacy, non-sequential access to information, adaptability, feedback, options, bidirectional communication, and grain size (the size of the smallest unit, s. Ch. 7) (12ff). Gentner (1992) researched continuing motivation of young persons in dealing with computer games through an analysis of programs in which the young people even take strenuous work upon themselves only to be able to solve an adventure. His thesis is that it is a mixture of learner control and external control that is responsible for the young persons’ motivation.

Dillenbourg and Mendelsohn (1992) designate the relation of intelligent learning environments and learners as interaction space. This view, which largely corresponds to the distinction of different spaces in multimedia that I have introduced above, does not so much emphasize the internal representation of knowledge on the part of learner or computer as the protagonists’ interactions. They divide the interaction space into representation space and event space, with pairs of representations and events constituting their own microworlds.

Reactive, Co-Active and Proactive Interaction

Rhodes and Azbell (1985) distinguish three forms of interactivity design in learning environments: reactive, co-active and proactive design. Reactive design comes from the behaviouristic stimulus-reaction paradigm, while proactive design assigns an actively constructing role to the learner. The distinction is taken up and modified by Thompson and Jorgensen (1989). They situate an interactive model between the poles of reactive and proactive design, a model that allows the learner to browse or select, or the form of behaviour that appears with an ideal tutor. These terms are interpreted by Lucas (1992) on the basis of learning theories. Schwier (1992) also takes up this terminology, rejects the hardware-based concept of »interaction levels« that dominated the literature on interactive video [e.g. Kearsley/Frost (1985)], and describes human-machine interaction as a taxonomy of learner-media interaction. This is based on the type of the learners’ cognitive activity. Interaction is described on three levels of different quality:

Reactive Interaction

A reactive interaction is a response to presented stimuli, e.g. an answer to a question that has been put.
Proactive Interaction
Proactive interaction stresses the constructing and generating activity of the learner. The learner’s actions go beyond selecting available information and reacting to existing structures, and generate individual constructions and elaborations beyond the rules set up by the designer.
Reciprocal Interaction
Reciprocal interaction takes place in designs of artificial intelligence or virtual reality, in which learner and system may reciprocally adapt to each other.
Reactive, proactive and reciprocal interactivity are described on five functional levels through the following transactions: confirmation, pacing, navigation, inquiry, and elaboration. One important implication of this description concerns learner control: on the reactive level, the designer maintains complete control over subject matter, its presentation, sequence, and the exercise levels. On the higher levels, rather more control passes into the hands of the user [cf. Schwier (1993a), Schwier (1993b)]. A certain similarity to this classification of the dimensions of interaction is present in the distinction of Midoro, Olimpo et al (1991), who likewise know three dimensions of the interaction space and build a classification of learning programs on that concept: adaptivity, reactivity, and navigability.

Classification of Learning Programs [Midoro/Olimpo et al (1991), 181]Interaction is determined not only by the technical dimension of design, but also by the dimensions of the subject matter and the instructional type of software, but it is also partly independent of these factors. Even in a dictionary or an electronic vocabulary book, interactivity can be realized differently than through mere selection of a term. This is demonstrated beautifully by the design for an individual user dictionary by Ferm, Kindborg et al (1987). The learning effect of passive page-turning, or the mechanical following of hypertext links is not assessed very highly: »A reasonable interpretation of the evaluations, however, might be that such a system promotes effective learning only in so far as the users are engaged in actively making their own connections at the conceptual level« [Mayes/Kibby et al (1990), 229]. Wishart and Canter (1988) argue for a classification of software according to the type of software and the degree of user involvement. The consequences of such an approach, which includes the contents of the application in the characterization of interaction, are indicated in Laurel (1989), when she classifies interaction with digital films as narrative, navigational or dramatic.

Giardina (1992) points out the wide spectrum of interaction forms that is determined by the educational ideas of the designers. He grants that one will always encounter designs in which »a particular learning framework is imposed on individuals, based on the erroneous notion that the designer-expert is in the best position to prescribe effective teaching« (54), but basically multimedia allows a design of bidirectional interactivity. Giardina stresses that this form of communication is characterized above all by the control and manipulation that an individual may exert over his learning environment and the objects contained in it [cf. La Follette (1993)].

Interactivity versus Instruction

Jaspers (1991) complains that despite the popularity of the interaction concept, scientific literature does not offer much on this subject (22), and he calls for a new definition of instruction and a concise description of interaction in reaction to Merrill and his concept of instructional transactions (21). This definition should take into account the development that has gained acceptance in Western countries and which is characterized by the fact that the learner is increasingly emancipated from control by the school, the teacher, or the instructional designer: »We must conclude that the point is not: interaction yes or no. The point is: more or less. All the named characteristics of interactivity are gradients« (22). He calls the expression »instructional delivery« a »contradictio in terminis«: »Thus, what we definitely need is a new definition of instruction. Or even more so, a concept that replaces the instruction concept. It is evident that this substitution will make reference to the concept of interactivity«. It is basically irrelevant whether one designates the goal as a redefinition of instruction or as a shift of emphasis from instruction to learning, as the constructivists do. The second version seems clearer to me, because it maintains the traditional meaning of instruction, but demonstrates the principal difference implied in the change of perspective to that of the learner.

Reciprocity and Symmetry of Communication

Jaspers describes the phenomenological level of interaction under the aspects of exchange of information and management of information. He distinguishes introducing a dialogue, setting goals, interruptions, agreeing on the topic, calling for feedback, breaking off the dialogue etc., and finds: »There is equality in all or in a certain number of aspects« (22). The reciprocity and symmetry of communication is that which distinguishes real dialogue from the artificial dialogues of the programs. I cannot agree with a program on the topic which has been predetermined by the author, I cannot cause the program to change its style of interaction and enter into a metacommunication. The reciprocity of communication is violated in human-program interaction. Therefore, Jaspers says rightly: »In fact, we would prefer to reserve the attribute of interactivity for systems in which each partner has the occasion to influence the common stream of events, including the operations of the other partner« (22ff). Stebler, Reusser et al (1994) seem to agree when they say: »The addition ‘interactive’ is given to those teaching-learning environments in which cooperation and discourse take centre position«. What we need, then, is a cooperative discourse, or, to put it in a more catchy way with Perelman (1992): »The focus is on learning as an action that is ‘done by,’ not ‘done to,’ the actor« (23). What is important is that the subject matter is dealt with in a manner corresponding to what Stebler, Reusser et al call »thorough understanding«, with the restriction that this discourse cannot be conducted with the program, but only in the learner’s head or in cooperation with other learners and teachers.

Absence of Sanctions in Interaction

In conclusion, I would like to point out another phenomenon that plays an important role in interaction with the computer and a learning program: It is usually overlooked that the interaction with a program is characterized by the fact that it is free from judgement and social consequences. Actions may even be undone without leaving any trace – quite contrary to interaction. Even if a learning program does give out judging statements, I may keep them to myself and can avoid them in repetition. In human-to-human interaction, nothing is retractable, a mistake or an impression once made cannot be deleted. The absence of sanctions in interaction with a computer or program is thus perhaps the most important aspect of the learning subject. One may well speculate that the computer is so attractive to young persons because it gives permanent feedback, but minus the judgement that is inherent in the personal feedback of a teacher. The research results of Rheinberg (1985), who questioned young hackers about the reasons for their hours of activity at the computer, speak in favour of this thesis. So does Twidale’s (1993) observation that students made a lot of intentional mistakes in learning with a tutorial system in order to get tutorial feedback. The interaction of young people with the computer apparently only functions without any fears because one may make mistakes without being punished: »One may suppose that the computer is one of the few competitive fields in which failure-oriented learners are not deterred« [Rheinberg (1985), 98].

This thesis does not at all contradict the observation that computer users usually wish for response times as short as possible, and immediate feedback. On the contrary, short and frequent feedback that causes the learner make a voluntary correction to his behaviour, but remains without consequences otherwise (e.g. no bad marks or moral judgements), is what makes the medium popular. Reinhardt (1995) quotes Schank, who sees as the problem at the core of computer learning that »people need to be able to experiment without fear of embarrassment and with experts looking over their shoulders« (70). The argument thus comes full circle from the topic of interaction to the topics of feedback and student control which I am going to address in one of the following chapters.

What conclusions can we draw from the reflections on the design of multimedia programs that have been discussed?