Whenever people using language try to convince their partner in
dialogue, or even themselves, that they understood a
description, a concept, a proof, and answer by using the
colloquial "I see," they actually express the practical
experience of seeing through language. They are overcoming the
limitations of the abstract system of phonetic language and
returning to the concreteness of seeing the image. Way of
speaking equals way of doing-this sums up one of the many
premises of this book. We extract information about things and
actions from their images. When no image is possible-what does a
thought look like, or what is the image of right, of wrong, of
ideal?-language supports us in our theoretic experiences, or in
the attempt to make the abstract concrete. Language is rather
effective in helping us identify kinds of thoughts, in
implementing social rules that encode prescriptions for
distinguishing between right and wrong, for embodying the just
in the institution of justice, and ideals in values. But the
experience of language can also be an experience of images.
Once we reach the moment when we can embody the abstract in a
concrete theory, in action, in new objects, in institutions, and
in choices, and once we are able to form an image of these,
share the image, make it part of the visual world we live in, and
use it further for many practical or intellectual purposes, we
expand the literate experience in new experiences. So it seems
that we tend to visualize everything. I would go so far as to
say that we not only visualize everything, but also listen to
sounds of everything, experience their smell, touch, and taste,
and recreate the abstract in the concreteness of our
perceptions. The domination of language and the ideal of
literacy, which instills this domination as a rule, was and
still is seen as the domination of rationality, as though to be
literate equals being rational, volens nolens. In fact, the
rationality associated with language, and expressed with its
help, is only a small part of the potential human rationality.
The measure (ratio) we project in our objectification can as
well be a measure related to our perceptive system. It is quite
plausible to suspect that some of the negative effects of our
literate rationality could have been avoided had we been able to
simultaneously project our other dimensions in whatever we did.
The shift from a literacy-dominated civilization to the relative
domination of the visual takes place under the influence of new
tools, further mediations, and integration mechanisms required
by self-constitutive practical experiences at the new human
scale. The tools we need should allow us to continue exploring
horizons at which literacy ceases to be effective, or even
significant. The mediations required correspond to complexities
for which new languages are structurally more adequate. The
necessary integration is only partially achievable through
literate means since many people active in the humanities and
the sciences gave up the obsession of final explanations and
accepted the model of infinite processes.
Images, among other sign systems, are structurally better suited
for a pragmatic framework marked by continuous multiplication of
choices, high efficiency, and distributed human experience. But
in order to use images, the human being had to put in place a
conceptual context that could support extended visual praxis.
When the digital computer was invented, none of those who made
it a reality knew that it would contribute to more than the
mechanization of number crunching. The visionary dimension of
the digital computer is not in the technology, but in the concept
of a universal language, a characteristica universalis, or
lingua Adamica, as Leibniz conceived it.
This is not the place to rewrite the history of the computer or
the history of the languages that computers process. But the
subject of visualization-presented here from the perspective of
the shift from literacy to the visual-requires at least some
explanation of the relation between the visual and the human use
of computers. The binary number system, which Leibniz called
Arithmetica Binaria (according to a manuscript fragment dated
March 15, 1679), was not meant to be the definitive alphabet,
with only two letters, but the basis for a universal language, in
which the limitations of natural language are overcome. Leibniz
tried hard to make this language utilizable in all domains of
human activity, in encoding laws, scientific results, music. I
think that the most intriguing aspect, which has been ignored for
centuries, was his attempt to visualize events of abstract
nature with the help of the two symbols of his alphabet. In a
letter to Herzog Rudolph August von Braunschweig (January 2,
1697), Leibniz described his project for a medal depicting the
Creation (Imago Creationis). In this letter, he actually
introduced digital calculus. Around 1714, he wrote two letters to
Nicolas de Remond concerning Chinese philosophy. It is useful to
mention these here because of the binary number representation
of some of the most intriguing concepts of the Ih-King. Through
these letters, we are in the realm of the visual, and in front of
pages in which, probably for the first time, translations from
ideographic to the sequential, and finally to the digital, were
performed. It took almost 300 years before hackers, trying to
see if they could use the digital for music notation, discovered
that images can be described in a binary system.
This long historic parenthesis is justified by two thoughts.
First, it was not the technology that made us aware of images,
or even opened access to their digital processing, but
intellectual praxis, motivated by its own need for efficiency.
Second, visualization is not a matter of illustrating words,
concepts, or intuitions. It is the attempt to create tools for
generating images related to information and its use. A text on a
computer screen is, in fact, an image, a visualization of the
language generated not by a human hand in control of a quill, a
piece of lead or graphite, a pencil or a pen. The computer does
not know language. It translates our alphabet into its own
alphabet, and then, after processing, it translates it back into
ours. Displayed in those stored images which, if in lead, would
constitute the contents of the lower and upper cases of the
drawers in each typography shop, this literacy is subject to
automation.
When we write, we visualize, making our language visible on
paper. When we draw, we make our plans for new artifacts
visible. The mediation introduced by the computer use does not
affect the condition of language as long as the computer is only
the pen, keyboard, or typewriter. But once we encode language
rules (such as spelling, case agreement, and so on), once we
store our vocabulary and our grammar, and mimic human use of
language, what is written is only partially the result of the
literacy of the writer. The visualization of text is the
starting point towards automatic creation of other texts. It
also leads to establishing relations between language and
non-language sign systems. Today, we dispose of means for
electronically associating images and texts, for
cross-referencing images and texts, and for rapidly diagramming
texts. We can, and indeed do, print electronic journals, which
are refereed on the network. Nothing prevents such journals from
inserting images, animation and sounds, or for facilitating
on-line reactions to the hypotheses and scientific data
presented. That such publications need a shorter time to reach
their public goes without saying. The Internet thus became the
new medium of publication, and the computer its printing press-a
printing press of a totally new condition. Individuals
constituting their identity on the Internet have access to
resources which until recently were available only to those who
owned presses, or gained access to them by virtue of their
privileged position in society.
The visual component of computer processing, i.e., the graphics,
relies on the same language of zeros and ones through which the
entire computer processing takes place. As a result of this
common alphabet and grammar (Boolean logic and its new
extensions), we can consider language (image translations, or
number-image relations such as diagrams, charts, and the like),
and also more abstract relations. Creating the means to overcome
the limitations of literacy has dominated scientific work. The
new means for information processing allow us to replace the
routine of phenomenological observation with processing of
diverse languages designed especially to help us create new
theories of very complex and dynamic phenomena.
The shift to the visual follows the need to change the accent
from quantitative evaluations and language inferences based on
them, to qualitative evaluations, and images expressing such
evaluations at some significant moments of the process in which
we are involved. Let us mention some of these processes. In
medicine, or in the research for syntheses of new substances,
and in space research, words have proven to be not only
misleading, but also inefficient in many respects. New
visualization techniques, such as those based on molecular
resonance, freed the praxis of medicine from the limitations of
word descriptions. Patients explain what they feel; physicians
try to match such descriptions to typologies of disease based on
data resulting from the most recent data. When this process is
networked, the most qualified physician can be consulted. When
experimental data and theoretic models are joined, the result is
visualized and the information exchanged via high-speed broadband
digital networks.
Based on similar visualization techniques, we acquire better
access to sources of data regarding the past, as well as to
information vital for carrying through projects oriented towards
the future. Computed tomography, for instance, visualized the
internal structure of Egyptian mummies. Three-dimensional images
of the whole body were created without violating the casings and
wrappings that cover the remnants. The internal body structure
was visualized by using a simulation system similar to those
utilized in non-intrusive surgery.
The design and production of new materials, space research, and
nano- engineering have already benefited from replacing the
analytical perspective ingrained in literacy-based methods with
visual means for synthesis. It is possible to visualize
molecular structures and simulate interactions of molecules in
order to see how medicine affects the cells treated, the
dynamics of mixing, chemical and biochemical reactions. It is
also possible to simulate forces involved in the so-called
docking of molecules in virtual space. No literacy-based
description can substitute for flight simulators, or for
visualization of data from radio astronomy, for large areas of
genetics and physics.
Not the last among examples to be given is the still
controversial field of artificial intelligence, seduced with
emulating behaviors usually associated with human intelligence
in action. But it should not surprise anybody that while the
dynamics of the civilization of illiteracy requires freedom from
literacy, people will continue to preserve values and concepts
they are used to, or which are appropriate to specific knowledge
areas. Paradoxically, artificial intelligence is, in part, doing
exactly this.
When people grow up with images the same way prior generations
were subjected to literacy, the relation to images changes. The
technology for visualization, although sometimes still based on
language models, makes interactivity possible in ways language
could not. But it is not only the technology of visualization
applied within science and engineering that marks the new
development. Visualization, in its various forms and functions,
supports the almost instantaneous interaction between us and our
various machines, and among people sharing the same natural
environment, or separated in space and time. It constitutes an
alternative medium for thinking and creativity, as it did all
along the history of crafts, design, and engineering. It is also
a medium for understanding our environment, and the multitude of
changes caused by practical experience involving the life
support system. Through visualization, people can experience
dimensions of space beyond their direct perception, they can
consider the behavior of objects in such spaces, and can also
expand the realm of artistic creativity.
The print media, as an overlapping practical experience uniting
literacy and the power of sight, are more visual today than at
any previous time. We are no longer subjected-sometimes with
good reason, other times for dubious motives-to the
sequentiality of literacy-dominated modes of communication. An
entire shared visual language is projected upon us in the form
of comic strips, advertisements, weather maps, economic reports,
and other pictorial representations. Some of these
representations are still printed on paper. Others are displayed
through the more dynamic forms at public information kiosks, or
through interactive means of information dissemination, such as
computer-supported networks and non-linear search environments,
which Ted Nelson anticipated back in 1965. The World Wide Web
embodies many of his ideas, as well as ideas of a number of other
visionaries.
Parallel to these developments, we are becoming more and more
aware of the possibilities of using images in human activities
where they played a reduced role within literacy-civic action,
political debate, legal argumentation. Lawyers already integrate
visual testimony in their cases. Juries can see for themselves
the crime being committed, as well as the results of
sophisticated forensic tests. Human destinies are defended with
arguments that are no longer at the mercy of someone's memory or
another's talent for rhetoric or drama. The citizen is frequently
addressed by increasingly visual messages that explain how tax
dollars are spent and why he or she should vote for one or
another candidate. In becoming the Netizen, he or she will
participate in social interactions fundamentally new in nature.
On the Net, politicians claiming credit for some accomplishment
can be immediately challenged by the real image. Political
promises can be modeled and displayed while the campaign speech
is given. A decision to go to war can be subjected to an instant
referendum while the simulation of the war itself, or of
alternatives, is played on our monitors. But again, to idealize
these possibilities would be foolish. The potential for abusive
use of images is as great as that for their meaningful
application.
Many factors are at work slowing down the process of educating
visually literate individuals. We continue to rediscover the
wheel of reading and writing without advancing comprehensive
programs for visual education. Illustrative visual alternatives,
advanced more as an alibi for the maintenance of
literacy-dominated communication, are by the nature of their
function inappropriate in the context of higher efficiency
requirements. Utilized as alternatives, these materials can be,
and often are, irrelevant, ugly, insignificant, and expensive.
More often than not, they are used not to enhance
communication, but to direct it, to manipulate the addressee. It
will take more than the recognition of the role of the visual to
understand that visual literacy, or probably several such
literacies, comprising the variety of visual languages we need,
less confining, less permanent, and less patterned, are
necessary in order to improve practical experiences of
self-constitution through images. We are yet to address the
ethical aspects of such experiences, especially in view of the
fact that the visual entails constraints different from those
encoded in the letter of our laws and moral principles.
In discussing the transition to the visual, I hope to have made
clear that the process is not one of substituting one form of
literacy for another. The process has a totally different
dynamics. It implies transition from a dominating form of
literacy to a multitude of highly adaptive sign systems. These
all require new competencies that reflect this adaptability. It
also requires that we all understand integrative processes in
order to make the best of individual efforts in a framework of
extremely divided and specialized experiences of
self-constitution. If seeing is believing, then believing
everything we see in our day is a challenge for which we are, for
all practical purposes, ill prepared.