Television Subtitles for Deaf and Hard-of-Hearing people
For the Deaf Studies Trust and supplied to BBC/ITC 1993
The study began with a very simple aim: to discover the views of deaf and hard-of-hearing people on the quality of teletext subtitles and of their preferences in relation to programmes. Because of the tremendous response of deaf and hard-of-hearing people and their organisations, the study has grown into a major examination of all aspects of subtitle viewing and of the reasons for different types of responses and reactions. Not surprisingly, the data collected has proved to be complex and to make full use of it has taken longer than originally intended. Nevertheless the information is of value to planners and broadcasters.
The Study - starting points
Aims and Schedule
The aim was to collect data from the deaf and hard-of-hearing viewing population concerning their experiences of using teletext subtitles. The data was drawn from a number of areas.
The extended survey proved very successful and produced over 2,500 responses.
A total of 284 were completed of which 275 were deemed suitable. There were interviews in Bristol London, Glasgow, Northern Ireland, Leeds, Oxford, and Kent.
Sub-studies on quality factors in subtitling
These were envisaged as clinics with an opportunity for groups of people to comment on subtitle form. On the whole however, it was found more effective to show video materials to individuals or to larger groups where a set of questions were provided on standard forms. Attempting to conduct group discussion with the hard-of-hearing groups simply created stress in the participants.
A postal corporate questionnaire was sent to the national deaf and hard-of-hearing organisations.
Children, Parents and Schools
Parents were contacted directly through the NDCS. A total of 98(68%) were returned.
A brief diary study was carried out in conjunction with the Deaf Broadcasting Council during the week of the 4th - 10th March. A total of 17 people returned the completed forms.
The various components can be grouped for our discussion under four headings: measured opinion, analysis of broadcast output, experimentation, and monitoring.
Under this heading we can draw together the Survey, the Interview study, the parents responses and the responses of the organisations. Of the two generally sampled components(survey, interview) we believe that the sample is typical of the deaf and hard-of-hearing viewing population. It cannot be fully representative in the way that Census based studies would be since we do not have the relative incidence figures for hearing loss and knowledge of geographical distribution which would make such sampling possible. However, the results are indicative of the population of teletext viewers who have a hearing loss.
The findings are very encouraging - teletext ownership and use is very much higher among this group than among reported figures for the general population(even allowing for the fact that teletext owners were more likely to have responded). In particular those who are members of deaf organisations seem more likely to possess sets and this reaches a very high level amongst those up to 40 years of age.
Older people(the majority of the deaf and hard-of-hearing population) are less likely to have sets. This may be a simple result of technophobia or of the increased reliability of older sets(which are not traded for teletext sets while they are still working). Once a set was in place people did not have great problems with the controls on the whole.
News is of particular concern to the vast majority of respondents. The Gulf War occurred mid way through the project and certainly increased the awareness of the news and meant that in all opinion measurement(except for children), news was seen as the highest priority. This may have made deaf people's views more analytical and it certainly made them more critical. The result is that both types of news subtitling(that used by the BBC - word-by-word scrolling and by ITV and C4 - traditional blocks) succeed in polarising the population. Large groups feel that the BBC version is too fast (though more see it now as having become established - so that it cannot be criticised any longer - failure to follow is then a personal problem). A significant minority think that the ITN version is too slow. Extreme viewpoints were frequently expressed in relation to viewer satisfaction with these services. As well as an expected split between deaf and hard-of-hearing people on the topic, there is a striking age effect where younger people are more likely to prefer the BBC version while older people are more likely to prefer the ITN version(see slide).
The survey and the interview study broadly agreed in terms of the programmes which people rated most highly. These were, on the whole, more serious programmes(see next slide). It was only when further analysis was carried out that we can begin to see the reasons why this "serious" pattern deviates from that expected from viewer responses directly to the television companies. (Following slides show this.)
The priorities for subtitling of hard-of-hearing people are broadly in line with their preferences for viewing, with exceptions in relation to deaf programmes and plays. In contrast the pattern for deaf people is rather different. News retains its pre-eminent position, but deaf programmes, films and comedy are very well liked and also rate very highly for subtitling. Parents differ in that they wish to have more subtitling for their children.
Although there is broad consensus on preference for news programmes, deaf sign users are more likely to prefer lighter programmes. Deaf people tend to give much higher priorities to subtitling all types of programmes.
Priorities also relate to age. Younger people want lighter programmes while the older people are more in favour of news and documentary programmes.
However, the most telling factor in all of these situations is the strategy used for television viewing. Many people retain much of their hearing ability. Relatively few are "totally" deaf. As a result, a significant proportion of the population are able to utilise sound from the television(over 50% of the hard-of-hearing and 13% of the deaf). For them subtitles perform a different role - as a support to the speech rather than as providing all the information.
Strategy affects programme choice with those able to use sound being more interested in serious programmes and less interested in lighter programmes.
Characteristics of Subtitles
In general questions about subtitle quality and in the open-ended questions, there were predictable responses concerning the extent of omission of information, speed, colour and breakdowns. Much of the comment while it is critical in both positive and negative senses, still seems to stem from a lack of understanding of how subtitles are constructed.
The editing of subtitles - ie where the subtitle does not appear to follow the speech verbatim - remains a subject of concern for many people(even though our examination of subtitle output did not detect major deletion of text, except in ITN's live news). However, it is perhaps one of the easier characteristics to identify and its expression seems to be related to particular strategies - where the viewer uses the subtitles to aid with lip-reading or where there is a belief that the precise words spoken are the key to the story, then the deviations from the spoken word will be very evident. Much depends on the viewer and the strategy which is adopted to watch subtitles.
Speed remains an issue and one which cannot easily be solved. If presenters and actors speak fast, then the subtitles will be faster if they are to keep up. Where the personal reading level is lower, the viewer may be unable to watch subtitles and see the programme images. They will tend to suggest that they have been reading the programme and not watching it.
There was very little direct investigation of the situation of deaf children but parental reports indicate that children under the age of 12 or 13 years will be unable to read the subtitles at the usual presentation rate(90-120 words per minute). (see slide on children).
The use of colour does not produce a set of simple preferences among the viewers. Those who do not have teletext sets tend to believe that colour is unnecessary(nearly all open subtitling will be in monochrome). Those with a set are much more likely to describe the benefits of speaker identification. It would appear that there is some effect of frequency of use.
This continues to be a problem in the eyes of the viewer. Although complete breakdown occurs less than one might expect, it occurs often enough for it to evoke strong feelings. Respondents suggested a breakdown rate of about 2-3 per week and the breakdowns centred on subtitles stopped or not supplied or where the wrong subtitles appeared for that programme. There was some evidence of reception problems and this may be an issue to be addressed by the broadcasters.
Breakdowns are more of a problem than quality factors in subtitle form, as they do significantly affect viewing enjoyment and they do undermine confidence in the programme. These stoppages may seem to be rather minor, only 10 minutes here and there or subtitles not appearing after the break in ITV. It is a significant problem area for the respondents in the survey and interview.
The general level of knowledge about breakdown is one factor to be considered in the issuing of a guide to teletext subtitles. The more informed viewers are the more easily they will be able to diagnose breakdown causes and transmission faults. This will be of help to the broadcaster and to the viewer.
Summarising Measured Opinion
If we can sum up this section of the study, it would be to say that the responses indicated a situation of "mostly satisfied, but want more". Most commonly, people emphasised the importance of access through subtitles and the fact that television viewing would not be possible at all without them. As a result there was considerable disappointment when there were no subtitles on a particular programme. There is considerable support for the increase in subtitling proposed in the Broadcasting Act.
Analysis of Broadcast Output
Most of the analysis done under this heading involved live programmes. In investigating this, one could focus on the programme makers, who make it difficult for the subtitlers to perform well, or on the technology which may not be adequate for the job in hand or on the subtitlers' performance. There is some scope for an examination of all of these. However, we chose to consider what may be an easier option - the examination of the end product as it would be seen by the viewer. In doing so, it was possible to try to develop measures of subtitle presentation which would allow broadcasters and others, to examine the quality of subtitles in future work.
All of the programmes examined had problems in them. Live Superbowl coverage produced text which was unreadable by most people and required a major English reconstruction for those who were prepared to try to watch. A significant number of viewers would prefer to do without than to try to cope in this way.
We found as expected, that people retain very little of what they read in the subtitles - News information, while seeming rather important does not seem to implant itself in memory.
Where a programme was subtitled live, there were considerable problems from the perspective of the viewer. Reading of subtitle text was made more difficult by the breaking of conventions of grammar and display. Such an examination of live news within the study, allowed the development of a number of metrics to describe the subtitles:
The experimental system tried by Central Television had problems on many of these variables and although it seems to allow more information to be transmitted in subtitle form, there remains work to be done on the system to obtain the best possible presentation for the viewer, using a "live" autocuing system to supply the subtitles. This in turn, implies the need to carry out more experimentation on the viewer responses to the use of this system.
When dedicated systems of live subtitling, such as the one used by the BBC, are available, considerable success in verbatim display can be reported. For much of the time the words displayed are very close in time to the speech. Yet to do this the screen display has to change each half second, and lines have to move up every 2 seconds. We have not yet measured direct effects of this on comprehension and significantly on picture watching, but opinions of viewers varied markedly from extremely positive to extremely resistant.
It will take a more detailed experimental study than this one to explore these variables further but they indicate that live subtitles on screen do not produce a display which is optimal for reading.
This part of the study was affected by the difficulties of producing stimulus material as a result of the Gulf War's pressure on preparation time. A number of the studies which had been planned were not carried out. Nevertheless, the findings of the studies completed are interesting and indicative of directions which might be taken in future.
Lag time is a variable which markedly affects programmes which are live, although an opposite effect(leading subtitles) occurs with prepared programmes. Lag occurs when the text appears after the words have been spoken on screen; leading occurs when the words are on screen before the speaker has used them. The latter seems to be less of a problem since it has the beneficial effect of allowing the lip-reader to "plan ahead"; by scanning the subtitle the lip-reader can then be prepared for the words as they appear on the lips and this will give a much stronger impression of being able to follow the programme directly(it will almost certainly also give the impression that the speaker is a clearer lip-speaker). Most concern has been expressed about lag as it is this which is a priority in live subtitling.
In the studies carried out we found some differential effects on programme content - programmes with dramatic content suffered more when there was a lag between speech and subtitle. Where there was mainly a single speaker, then lag time even at the limit of 6 seconds behind normal did not create a problem and did not affect overall comprehension. Lag may not be as important a factor in the transmission of programme meaning.
The same is true of colour and add-on lines. In the studies carried out, there were no systematic effects on comprehension of meaning which could be attributed to colour. This also applied to the use of add-on lines. It appears that these variables affect viewer enjoyment more than they affect the reception of information.
In the case of this type of study there remains some work to be done to develop adequate measures of viewer performance. While comprehension is important, all the evidence suggests that hearing viewers do not retain a great deal of the precise information presented. What appears to be retained is the storyline and what is activated is a series of scenarios(in psychological terms - scripts). In these the viewer explores the possibilities for the characters in the drama, matches these predictions to what occurs in the action and attempts to draw out a message, relieve tension or to learn. It is the extent to which this process occurs effectively and smoothly that allows one to enjoy the programme. In order to deal with this type of variable it would be necessary to carry out "stop the action" measurement, where the viewers with subtitles and without subtitles were asked to reconstruct the story, to draw out the implications and to fill in certain dynamic details. This remains to be done and is a more complex area of study.
On the basis of data collected in this study, we can see that comprehension is not affected by these variables of lag, colour and add-ons.
As a final aspect of the work, a diary study was carried out where a number of deaf and hard-of-hearing viewers were requested to log all the programmes which they watched during a single week. While tending to confirm a general level of satisfaction with current subtitling it did offer some additional points. It produced a clearer indication of the extent of breakdown in subtitles and it provided some information on the reception problems which people experienced.
It would be valuable to create such a regular system of monitoring as it allows a measurement of the longer term use of subtitles, as distinct from the snapshot offered by the experimental studies.
The primary purpose of this research was to provide data upon which guidance for subtitling could be built. Yet the main finding of the study which one returns to again and again is that there are several populations embedded in the classification "deaf" and they are typically untypical.
The age factor is one of the most significant. To satisfy the largest group, then one should ensure that the programmes which older people like, always carry subtitles. On the other hand, older people are proportionally less likely to have teletext subtitles and they experience much more difficulty with subtitle format. The live scrolling screen is a case in point. This may lead at times, to two forms of subtitle being transmitted - one for those who require full speed verbatim text and those who need subtitle blocks and are prepared to compromise on comprehensiveness.
Where there is preparation time prior to broadcast, the task for the subtitler is a little easier, with text blocks having become standard. The current refinements such as use of colour and add-on lines do not affect comprehension but may affect enjoyment especially if more information is supplied to viewers on the construction of subtitles.
The main dilemma has begun to be apparent in the results from the study. It is not the fact of the impossible task of subtitling(ie representing in written form at an appropriate speed the meaning which film-makers have created). It is rather the issue of how to deal with the diversity in the viewers. Subtitle users are not only different in personal terms but the way in which they use subtitles seems to vary. Comments or criticisms of subtitles do not arise solely in the person's background but such comments also represent the way in which the viewers are utilising the subtitles. Where used as an adjunct to heard speech the status of the subtitles is quite different - in the words of one viewer, "it gives me the impression that I am hearing what they say, even though I know I am not." For others the subtitle represents the only way information contained in the speech can be accessed. For the latter group, meaning is paramount, while for the former, it is the verbatim element which is vital. Unfortunately, even these strategies may be likely to shift according to programme type, with drama requiring a closer text-speech link.
Although we have identified some measures which could be used in relation to subtitle output, there remains a major question of who has the task of monitoring the output and carrying out the ongoing data collection which will allow standards to be maintained. It will be possible to generate guide-lines for subtitle quality from the results of the studies here, but it will then require some application. A monitoring body is needed - perhaps to be set up by deaf organisations.
The most fundamental question was asked right at the beginning and it is the one to which we have to return by the end - what is the aim of subtitling? Is it simply to add written words to what appears on screen or is it to create and to add to the enjoyment of the viewer? Just as producers and directors have advanced in their facility with the television medium, it seems clear that we have to make some progress with the subtitle dimension. This is already apparent in many programmes where in-vision captioning is being used to convey additional information - most simply in the news, but more creatively in music programmes and magazine programmes for young people. If such developments can begin in the hearing medium, it would seem obvious that we can begin to explore the use of subtitles for deaf people more effectively.
As long as the subtitlers have no influence on the programme production, their job will be a series of compromises and will be subject to criticism. The question of what is the aim of subtitling comes into focus. If the subtitle has to provide meaning to the viewer then it must be a consideration in the production stage of the programme. If it is to add a written representation of what is said, then it will also need to influence programme pace. If it is merely a bolt-on which is of lower priority in the whole process then deaf and hard-of-hearing people will always struggle to balance the tasks described.
One or two possibilities which have become apparent during the conduct of the research are:
Guide-lines for subtitle production:
already used by subtitlers, are a first step in quality control. The second step is a set of guide-lines which govern how the words look on the screen and this has been mentioned above. Where there are prepared programmes the problems seem to be less; in live programmes, these guide-lines could be invaluable.
We have not discovered a vast body of well-informed viewers who understand how subtitles are constructed and have developed strategies for using them. It would help if there were materials available to prospective teletext subtitle users which would make them more able to use subtitles and more able to extract and retain the information being presented. These materials could take the form of a broadcast programme, videotape or leaflet.
Such information material would be of value also to those who take on the role of monitoring subtitle output, and would lead to a more effective viewer panel, providing information to the broadcasters which would allow the service to be altered and changed more effectively.
Taken all together the study forms a large body of data which can be analysed further or used as a baseline for other studies. It remains only for us to begin to use those expressed views and the data which they have produced, in a constructive manner to the benefit of deaf and hard-of-hearing people.