The written, as we know, almost constantly appeared together with
other referential systems, especially images. In this respect, a
question regarding what we understand when we understand
language is whether images can be used as an aid to
understanding texts. Doubtless, pictures (at least some of them)
are, by their cognitive attributes, better bearers of
interpretation clues than are some words or writing devices.
Images, more so than texts, can stand in for the absent writer.
To the extent that they follow conventions of reality, pictures
can help the individual reconstitute, at least partially, the
frame of time and space, or one of the two. However, this
represents only one side of the issue. The other side reveals
that images are not always the best conveyors of information,
and that what we gain by using them comes at a cost in
understanding, clarity, or context dependence.
First of all, what is gained through the abstraction of the words
is almost entirely lost through the concreteness of the image.
The very dense medium of writing stands in sharp contrast to the
diluted medium of images. To download text on the network is
quite different from displaying images. If this were the only
reason, we would be alert to the differences between images and
texts. When the complexity of the image reaches high levels,
decoding the image becomes as tedious as decoding texts, and the
result less precise. All this explains why people try to use a
combination of images and words. It also helps in understanding
strategies for their combination. As a strategy of relating
text and image, redundancy helps in focusing interpretation. The
strategy of complementing helps in broadening the
interpretation. Other strategies, ranging from contrasting texts
and images to paraphrasing texts through images, or substituting
texts for images, or images for text, result in forceful ways of
influencing interpretation by introducing explanatory contexts.
A very large portion of today's culture-from the comic strip to
picture novels and advertisements, to soap operas on the
Internet-is embodied in works using such and similar strategies.
What interests us here is whether images can replace the
experience required to understand a text. If the answer is
affirmative, such images would be almost like the partner in
conversation. As products of human experience, images, just like
language, embody that particular experience. This automatically
makes the problem of understanding images more involved than
just seeing them. But we knew this from written language. Seeing
words or sentences or texts on paper (in script or in print) is
only preliminary to understanding. The naturalness of images
(especially those resembling the physical universe of our
existence) makes access to them sometimes easier than access to
written language. But this access is never automatic, and should
never be taken for granted. In addition, while the written word
does not invite to imitation, images play a more active role,
triggering reactions different from those triggered by words.
The code of language and visual codes are not reducible to each
other; neither is their pragmatic function the same.
Research reports are quasi-unanimous in emphasizing that the
usefulness of pictures in increasing text comprehension seems
not to depend on the mere presence of the image, but on the
specific characteristics of the reader. These make clear the
role played by what was defined as background knowledge, without
which texts, images, and other forms of expression stabilized as
languages make little sense, if any, to their readers, viewers,
or listeners. In order to arrive at such conclusions, researchers
went through real-time measurements of the so-called processing
of texts, in comparison to picture-text processing. The paradigm
employed uses eye movement recordings and comprehension measures
to study picture-text interactions. Pictures helped what the
researchers defined as poor readers. For skilled readers,
pictures were neutral when the information was important. The
presence of pictures interfered with reading when the
information in the text was less important. Researchers also
established that the type of text-expository or narrative-is not
a factor and that pictures can help in recall of text details.
This has been known for at least 300 years, if not longer.
Actors in Shakespeare's time were prompted to recall their lines
through visual cues embodied in the architecture of the theater.
After all was measured and analyzed, the only dependable
conclusion was that the effects of images on comprehension of
written language are not easy to explain. Again, this should not
come as a surprise as long as we use literacy-based quantifiers
to understand the limits of literacy. Whether images are
accidental or forced upon the reader, whether the text is
quasi-linear or very sophisticated (i.e., results from practical
experiences of high complexity), the relation does not seem to
follow any pattern. Such experiments, along with many others
based on a literacy premise, proved unsuitable for discovering
the sources and nature of reading difficulties.
Eye movement and comprehension measures used to study
picture-text interactions only confirmed that today there are
fewer commonalties, even among young students (not to mention
among adults already absorbed in life and work) than at the time
of the emergence of writing and reading. The diversification of
forms of human experience, seen against the background of a
relatively stable language adopted as a standard of culture,
hints at the need to look at this relation as one of the possible
explanations for the data, even for the questions that prompted
the experiments in the first place. These questions have bearing
on the general issue of literacy. Why reading, comprehension,
and recall of written language have become more uncertain in
recent years, despite efforts made by schools, parents,
employers, and governments to improve instruction, remains
unanswered. Regardless of how much we are willing to help the
understanding of a text through the use of images, the necessity
of the text, as an expression of a literate practical
experience, is not enhanced. Conclusions like these are not easy
to draw because we are still conditioned by literacy. Experiences
outside the frame of literacy come much more naturally together
because their necessity is beyond the conditioning of our
rational discourse. This is how I can explain why on the
Internet, the tenor of social and political dialogue is
infinitely more free of prejudice than the information provided
through books, newspapers, or TV. These observations should not
be misconstrued as yet another form of technological
determinism. The emphasis here, as elsewhere in the book, is on
new pragmatic circumstances themselves, not on the means
involved.
The research reported above, as any research we hear about in our
days, was carried out on a sample. A sample, as representative
as it can be, is after all a scaled- down model of society. The
issue critical to literacy being the scale of human practical
existence, scaled-down models are simply not suited for our
attempt to understand language changes when the complexity of
our pragmatic self-constitution increases. We need to consider
language, images, sounds, textures, odors, taste, motion, not to
mention sub-verbal levels, where survival strategies are encoded,
and beliefs and emotions are internalized, as they pertain to
the pragmatic context of our existence. Literacy is not adequate
for satisfactorily encoding the complexity and dynamics of
practical experiences corresponding to the new scale that
humankind has reached. The corresponding expectations of
efficiency are also beyond the potential of literacy-based
productivity. Ill-suited to address the mediated nature of human
experience at this scale, literacy has to be integrated with
other literacies. Its privileged status in our civilization can
no longer be maintained.
Korzybski was probably right in stating that language is a "map
for charting what is happening both inside and outside of our
skins." At the new stage that civilization has reached, it turns
out that none of the maps previously drawn is accurate. If we
really want details essential to the current and future
development of our species, we have to recognize the change in
metrics, i.e., in the scale of the charted entity, as well as in
dynamics. The world is changing because we change, and as a
result we introduce new dimensions in this world.
Even when we notice similarities to some past moment-let us take
orality as an example-they are only apparent and meaningless if
not put in proper context. Technology made talking to each other
at long distances (tele-communication) quite easy, because we
found ways to overcome the constraints resulting from the limited
speed of sound. The most people could do when living on two close
hills was to visit, or to yell, or to signal with fire or
lights. Now we can talk to somebody flying on an airplane, to
people driving or walking, or climbing Mount Everest. Cellular
telephony places us on the map of the world as precisely as the
global positioning system (GPS) deployed on satellites. The
telephone, in its generalized reality as a medium for orality,
defies co- presence and can be accessed virtually from anywhere.
Telephony as a practical experience in modern communication
revived orality under circumstances of highly integrated,
parallel, and distributed forms of human activity on a global
scale. On the digital networks that increasingly represent the
medium of self-constitution, we are goal and destination at the
same time. In one click we are wherever we want to be, and to a
great extent what we want to be or are able to do. With another
click, we are only the instantiation of someone else's interest,
acts, knowledge, or questioning. The use of images belongs to
the same broad framework. So does television, omnipresent and, at
times, seemingly omnipotent. We became connected to the world,
but disconnected from ourselves. As bandwidth available for
interacting through a variety of backchannels expands from
copper wire to new fiberglass data highways, a structure is put
in place that effectively resets our coordinates in the world of
global activity. Defying the laws of physics, we can be in more
than one place at the same time. And we can be more than one
person at the same time. Understanding language under such
circumstances becomes a totally new experience of
self-constitution.
Still, understanding language is understanding those who express
themselves through language, regardless of the medium or the
carrier. Literacy brought to culture the means for effectively
understanding language in a civilization whose scale was well
adapted to the linear nature of writing and reading, and to the
logic of truth embodied in language. However, literacy lacks
heuristic dimensions, is slow, and of limited interactivity. It
rationalizes even the irrational, taking into bureaucratic
custody all there is to our life. Common experience, in a
limited framework characteristic of the beginning of language
notation, is bound to facilitate interpretation and support
conflicting choices. Divergent experiences, many driven by the
search for the useful, the efficient, the mediating, experiences
having less in common among themselves, make language less
adapted to our self-constitution, and thus less easy to
understand. In such a context, literacy can be perceived only as
a phenomenon that makes all things it encomapsses uniform;
therefore literacy is resisted. Far from being only a matter of
skill, literacy is an issue of shared knowledge formed in work
and social life. Changes in the pragmatic framework brought
about the realization that literacy today might be better suited
to bridging various fragmented bodies of knowledge or
experiences, than to actually embodying them. Literacy might
still affect the manner in which we use specialized languages as
tools adapted to the various ways we see the world, the manner in
which we try to change it and report on what happens as a
result. But even under these charitable assumptions, it does not
follow that literacy will, or should, continue to remain the
panacea for all human expression, communication, and
signification.