Evaluation of a One-year Trial of Auralog's TellMeMore Education (Version 7) Software Package in a Higher Education Context


Historical and pedagogical background

The TellMeMore series, which evolved from Aura-lang (1991) and Talk To Me, Auralog’s earliest forays into language learning software based on speech recognition, has, with its harnessing of two more challenging aspects of CALL, speech recognition and tracking functionality, taken CALL software design to new levels of interactivity, authenticity of communication and control of feedback. However, in doing so has it met the challenge of bringing CALL back into step with current language learning methodologies (Levy, 1997, pp. 28-9)? Has it gone beyond being ‘technology-drivenE(Morgenstern, 1986, p. 23)?

Prior to its version 7 - ESPAN class="italic">EducationE- the TellMeMore product had done well on the commercial and private-user markets, but this did not prove its ‘integrabilityE(McCarthy, 1999) into the pedagogies of higher education. TellMeMore’s pedigree was of the ‘self-contained tutorialEtype, which works fine with individual learners who can work their way through the package at their own pace and pick and choose activities at leisure, and where the program/computer acts as the tutor. Roblyer, Edwards & Havriluk had warned that, however soundly structured the instructional sequence might be, teachers might reject such a courseware design ‘because it does not cover the topic the way he or she presents itE(1997, p. 91).

Furthermore, the product had to struggle with the challenge of matching the built-in tendency towards a behaviouristic style of exercise, forced upon it by the limitations of the speech recognition software, with the need to be flexible enough to fit into a classroom environment into which teachers bring their own approaches. As McCarthy states: “software is not pedagogically inertE(1999, p. 3). Auralog’s answer to this seems to be the exhaustive Admin Tools and Tutor Tools features which enable teachers to customise their own learning paths and map their own way through the vast array of activities. Has this been enough to render the package mouldable to any outside pedagogy, or does there still remain inflexibility due to the hermetically-sealed nature of most of the exercises?

Fig.1 - screenshot of Tutor Tools variable settings.

When we started trialling the latest version with first year undergraduates of French, the main questions we had in mind related, indeed, to its adaptability to a pre-existing modular context. Could it serve as a tool for differentiatied whole class teaching or was it destined to serve as a self-access trainer? Would its emphasis on customisable learning paths and exhaustive tracking and feedback system be manageable and beneficial in a degree course? Would we be able to go beyond the drill-and-practice features, which the package is full of, to use it as a coach for any form of ‘constructivistE communicative or collaborative purposes, especially authentic, spontaneous communication? Would its cultural workshop be useful in an area studies module? Would it serve best for extension or for remedial tuition?

Over the course of an academic year (2004/5) we put these questions to the test in the context of two projects: TOLD and BLINGUA. The TOLD Project (Technology and Oral Language Development) looked at its adaptability to the setting of an oral class. This project was reported on at length in 2005 (Barr, Leakey & Ranchoux). The second, the BLINGUA Project, looked at the package in the context of a blended learning for CALL pedagogy delivering written French language and area studies. The findings from BLINGUA are currently with review. Key findings from both the above projects as they relate to the TellMeMore package itself are given below.

Features of TellMeMore Education (v.7).

  • 10,000 exercises spread over 37 different types of activity (as opposed to the 35 of the previous version). The glossary is the same size: 10,000 words. There are the same six different Workshops (Lesson, Oral, Written, Grammar, Vocabulary and Cultural Workshop). The Cultural Workshop came in new for Version 6.
  • The Guided and Free-to-Roam modes were also features of Version 6.
  • Version 7 makes much use of Flash technology.
  • Admin tools: for registering individual student password protected IDs and mapping them to languages, tutors, groups and classes. While this was available to some extent from Version 5, it is much enhanced in Version 7.
  • Tutor Tools: a highly detailed feature enabling the customising of learning paths and then mapping the students to these learning paths. Also this feature enables the tutor to track in great detail student usage of and performance in both the Free-to-Roam and the Guided Modes. Such tracking can then be exported into external packages such as spreadsheets.
  • Version 7 comes in seven languages: American English, British English, French, German, Spanish, Dutch, Italian.
  • There are three packs: Pack 1 = Complete Beginners, Beginners, Intermediate, Intermediate+. Pack 2 = Advanced, Advanced+, Business. Pack 3 = 3 Business Levels.
  • Version 7 has a new ‘Complete Beginner’ level comprising six lessons created for absolute beginners.
  • An Intermediate+ Level, also new.
  • Version 7 is also available both as a networkable CD-ROM and as an online e-learning package complete with tutor support. Our study only involved the networkable CD-ROM.  The online version can be hosted either on a local server or on Auralog’s server.
Fig.2 - Tutor feedback on correctness (excerpt from a learning path focussing on oral skills only)
Fig.3 - Student feedback on correctness (excerpt from a learning path focussing on oral skills only)

Evaluation of strengths and weaknesses

Lafford (2004) has already reviewed the Spanish version of what appears to be version 7.. Below is her resume of the strengths and weaknesses of the package as she saw them. These points apply to both versions 6 and 7:

ESPAN class="italic">Tell Me More Spanish is a technologically sophisticated multimedia program with high-end graphics and excellent speech recognition software that provides the learner multiple opportunities to practice speaking, listening, reading, and writing skills and to gain knowledge about some isolated cultural facts. It is suited to the needs of individual learners, who are given a great deal of control over various elements of the program so they can forge their own learning path. However, the program's focus on pronunciation, structurally-based curriculum, mechanical exercises, decontextualized interaction, and use of culture capsules (mostly isolated from vocabulary and grammar exercises and listening, speaking and writing activities) causes this program to be out of step with modern communicatively-based views of task-based foreign language pedagogy -- views which are grounded in cultural authenticity and the notion of language as social practice.E(p. 32)

Her study looked at the Beginner Level only and did not comment on networking issues, the Admin Tools nor the Tutor Tools.

The University of Ulster trial of the French version endorses most of Lafford’s points both positive and negative with the following caveats and additional points.

While the Learning Paths feature allows for the importing of externally produced material as discrete features of a learning path, TellMeMore still does not allow for adaptation or replacement of any of its own content. It is not an authoring package as such, and so one is limited to Auralog’s own content within the TellMeMore functionality. A major limitation of this is that the material, be it cultural issues or the images will become dated, and much of the content may be irrelevant to a particular course of study. Users of the online version, however, have free access to an authoring tool which will allow for the importing by the teacher of texts, multiple-choice questions and word lists. The process of importing such texts requires some training as well as the rebooting of the host server every time a text is added. .

As regards installation issues, the faculty technician who installed the program on our network found the application setup, which uses an HTTP Server, far simpler than for version 5 (previously installed). The only real criticism he had was the extensive use of Macromedia Flash, which he felt slowed the overall performance.

Pedagogically, the main problem area concerns the mismatch between the self-contained nature of the most of the activities and the way that teachers in a given situation like to teach. From staff feedback most echoed Roblyer et al’s concerns (1997, p. 91) and saw the package as ‘an all-or-nothingEchallenge where they felt that if they were going to use it in a whole class context then they would need to adapt completely their teaching style as well as the content of their lessons to accommodate the package. Most preferred its use as a self-access trainer in the ‘mediathequeE Some staff also felt the highest levels were not sufficiently taxing for the abler student at undergraduate level.

TOLD Project

In the TOLD classes the drill-and-practice qualities of the phonetic features leant themselves well to pronunciation and fluency coaching at the start of an oral class. These features were found to be excellent in the pre-communicative, or “rehearsalEstage of a sequence of instruction (Barr et al, 2005).

Where the program remains weak, from the point of view of communication, is at the performance stage, where the limits of technology do not allow for anything more real than the simulated interactive dialogues. Greater authenticity, as Lafford herself states, could be brought in by toggling out of the program at a given point to, say, a teacher-prepared activity involving research on the web followed by oral feedback in a face-to-face context.

Both students and native-speaker lectrices expressed the fact that, while they appreciated the coaching for fluency and pronunciation in the program as well as the chance for mistakes to be made without the embarrassment they might feel in a group, nevertheless, they missed the spontaneity and human element of an ordinary oral class.

The BLINGUA project

This project trialled a blended learning pedagogy for a written language and an area studies module. The cohort was split into two groups, one of which was given activities geared to their predominant learning style, the other was taught using similar activities yet without this differentiation. TellMeMore was used in the written language module for grammar rehearsal and testing, and in the area studies module to kick-start research on a given topic. Especially useful for the former were the dictation exercises and the sentence transformation activities. However, as with Lafford, we found the grammar workshop difficult to map to a sequenced programme of grammar tuition, and preferred to use a separate grammar drilling program for initial grammar input. For the area studies module the Culture Workshop had material on a wide range of topics. However, the passages were very short and could only serve as a brief introduction and comprehension exercise, not as source material of any depth at university level. We gave web links and other support material to take the students further. The product, if it is to fulfil the demands of an area studies support, would require a greater degree of flexibility to allow for teachers to bring in current texts and set up their own questions within a pre-existing template (similar, say, to the HotPotatoes format).

The Learning Paths feature of TellMeMore proved a useful means of differentiating activities for the various learning styles. Using Admin Tools and Tutor Tools to preset student IDs and map learning paths to different students did initially take a while to get used to but once understood proved to be a quick way to customise student learning.

Fig.4 - Screenshot showing how the tutor can customise topic, activity and settings within the program.

TellMeMore comes with a large number of preset learning paths mapped to language function, student level or skill required. Tutors can either use or adapt these, or else devise their own paths from the huge database of content.

Assessment

Below is an example of the exportability of the student tracking feedback. TellMeMore automatically scores student work, and displays this in tabular format which can then be exported as text-files, html pages etc. This is clearly a bonus timesaving feature, which all teachers will welcome. (see table below). The “DurationEcolumn shows time spent on a given activity.

Student: Stephanie GEE
Path: week9_post_sem2_2004/5
Date: Overview

Fig.5 - Tracking provides detailed and exportable data on student activity and performance

Cost

This will be the main prohibitive feature for any educational institution wishing to buy into software with speech recognition, student tracking, and the high-level programming that TellMeMore v 7 demonstrates. Institutions can buy up to three packs and a variety of licencing levels. Many institutions are preferring to adopt the online version and thereby also avoid the cost of network licences.  Payment for the online version is an annual fee per student.

Conclusion

TellMeMore has proved popular with the students. Because it is such a large package with a wide range of functionality it can be quite daunting to get used to it. Coping with the complexities of both the Admin Tools and the Tutor tools and customizing the settings is quite a challenge and there must be a virtue in simplifying these processes. For the student some of the screens can be a little busy and difficult to navigate, and the predominance of Flash programming tends to slow the navigation. Speech recognition still is not as sharp as one would like it to be.

Clearly a huge amount of developmental work and design planning has gone into making the program suitable for the 21st century educational market. One feels its greatest applicability will be in the elementary or secondary educational sectors, who may, paradoxically, be the least able to afford it. To raise the appeal for higher level and adult learners the images and the content of the videos, dialogues and some of the phonetic phrases need to be adjusted. The main developmental challenge seems to be now to develop a program whose content itself is not hermetically sealed but customizable to the needs of the different educational and cultural contexts of its users, and where tutors can import and manipulate their own content. TellMeMore Education will work well as a self-contained trainer, and in its current form may give value in a higher education context for guided remedial or extension work on a self-access basis, though it may prove less workable in whole class settings where its content and some of the activities do not match the existing content or teaching goals.

Author

Jonathan Leakey is a lecturer in French at the School of Languages and Literature at the University of Ulster (Northern Ireland) after teaching French and German at secondary level for fourteen years in Liverpool (England). His principal research interest is effectiveness research in Computer-Assisted Language Learning. He has published recently on two higher education CALL research projects: one on technology and oral language development, and the other on a blended learning pedagogy for CALL. He is currently writing his doctoral thesis on evaluating the effectiveness of CALL platforms, programmes and pedagogy in L2 acquisition.

References

  • Barr, D., Leakey, J. & Ranchoux, A. (2005). TOLD Like It Is! - An evaluation of an integrated oral development model. Language Learning & Technology, 9, (3), 55-78. Retrieved January 10, 2006, from http://llt.msu.edu/vol9num3/barr/default.html
  • Lafford, B.A. (2004). Review of Tell Me More Spanish. Language Learning & Technology, 8, (3), 21-34. Retrieved May 6, 2005, from http://llt.msu.edu/vol8num3/review1
  • Levy, M. (1997). Computer-Assisted Language Learning, Context and Conceptualization. Oxford: Clarendon Press.
  • McCarthy, B. (1999). Integration: the sine qua non of CALL. CALL-EJ online, 1, (2). Retrieved January10, 2006, from http://www.tell.is.ritsumei.ac.jp/callejonline/journal/1-2/mccarthy.html
  • Morgenstern, D. (1986). Simulation, Interactive Fiction and Language Learning: Aspects of the MIT Project. Bulletin de l’ACLA/Bulletin of the CAAL (Canadian Association of Applied Linguistics), 8, (2), 22-33. (Cited in Levy, M. (1997). Computer-Assisted Language Learning, Context and Conceptualization. Oxford: Clarendon Press, p. 89).
  • Roblyer M., Edwards J. & Havriluk M. (1997). Integrating Educational Technology into Teaching. New Jersey: Prentice-Hall.