3.3 The visualisation system
Haber and McNabb recognised and discussed the critical role of the
intellectual phases of modelling and solution in simulations. However,
they excluded such mental visualisations of the nature of simulated
phenomena from their model of visualisation. They restricted the
latter to the visual post-processing of raw data. The resulting images
are deemed to play an important role in the Interpretation, evaluation
and revision of solutions. Again these cognitive activities did not
feature in their model. Consequently, Haber and MeNabb placed the
emphasisonthe variable methods for picture generation and deliberately
excluded the critical mental processes which initially conceive and
formulate and later verify these images. Their model of visualisation
does not consider the potential interactions which could occur within
an integrated psycho technical system for visualisation. We consider
these issues in more detail below.
Haber and McNabb identified three visualisation modes, namely
post-simulation analysis, runtime monitoring and interactive steering.
They pointed out that high end hardware performance has improved
dramatically and that their exploitation for interactive visualisation
is hindered by the lack of appropriate systems. We agree here. The
goal of their RIVERS project was to extend visualisation from a batch
procedure to a real time interactive process. They stated that real
time interactive steering closes the loop between the simulation and
visualisation systems. This gives the analyst immediate visual
feedback about the effects of making changes at run time to parameters
of the various transformations in the visualisation sequence. However,
even after elaboration for user configuration at runtime, their data
flow diagrams (p 88 89) indicate only a one way flow of data.
Consequently, the interaction effectively supports only iterative
picture generation. Their ambition was to upgrade the mode of
visualisation from a batch to an interactive one. Whilst this is
necessary and laudable, it offers only a dated and uninspiring vision
of the potential scope of scientific visualisation systems. With the
growing popularity of object-oriented WIMP systems and the escalating
interest in multimedia hypertext, even the general public are aware of
the growing potential for interactive exploration of data and/or
information.
Researchers in disciplines such as cartography, are excited not just
by the convenience with which they may generate and fine tune
alternative visualisations of complex phenomena and processes, but
more so by the expanding scope for treating such displays as two way
virtual devices for probing and investigating the phenomena by direct
exploration of the abstract data model and of raw data as indicated in
Figure 1. Just as the left and right hemispheres of the brain hold and
communicate separate encodings of the same pheriomena, there is a need
for interaction between the dual complementary representations of the
same phenomena in the database (digital map) and in image form (visual
maps).
The ability to transform geometrically, animate, slice, expose layers
and change the colour mappings of three dimensional and volume
visualisations is no doubt of great value to many scientific
applications. Haber and McNabb's data flow diagrams accommodated these
forms of investigation. However, visual thinking requires much greater
flexibility. Already image processing systems offer the scope for not
only generating multiple views of the same data but also for effecting
changes in related displays by direct manipulation of the underlying
model through key displays, such as histograms. A Tektronix 4406 AI
Workstation running Smalltalk was supplied to us in 1986 with a
prototype expert system for trouble shooting by cross referencing of
displays of a printed circuit board and its network schematics, a
table of component parts, a rule set and help information to the
operator. Visvalingam (1985) described how manual cross referencing of
a set of displays and tables of data led to insights about socio
spatial data. Visvalingam and Whyatt (1990 and 1991) described the
value of such visual cross referencing in the evaluation of line
simplification algorithms.
These and similar investigations, conducted since the mid 1970s, have
reliedonlaborious manual cross referencing of elementsondifferent
displays with the help of basic computer graphics. Often when
embarkingonsuch visual explorations, it is not possible to predict in
advance the number or type of displays required nor the manner in
which they will be cross referenced. There is a two way, fine grained
connection between displays and the applications which drive them. An
application module must not only be capable of generating the initial
display according to set parameters and modifying it when parameters
are changed, it must also be capable of selectively responding to user
manipulations of the display. Each display process too must be capable
of responding at any time to either direct manipulation by the user or
to transformations of user input routed through the driving
application. Moreover, the operation of two way virtual graphic
devices requires the definition of an appropriate minimal set of
mappings for effecting transformations in the reverse direction to
that indicated by Haber and McNabb. Some of these ideas were explored
by Visvalingam (1987), who was concerned with the inadequacy of the
Graphical Kernel System for this purpose.
3.4 The focusonscientific insight
Computer graphics is already serving a variety of uses, from graphic
design and art to communication of information. Systems for scientific
visualisation must subsume some of these functions, especially the
concepts and techniques for communication of information. However, as
stressed by McCormick et al, the new quest of ViSC is the exploitation
of developments in technology for invoking scientific insight. We too
believe that ViSC could become part of the apparatus of science but we
do not believe that it does so adequately at present. This is partly
because computer graphics has been concerned mainly with the
achievement of realism. Pretty pictures and entrancing movies may
tickle the senses and generate visual insight but can they on their
own generate intellectual insight? No doubt, developments in imaging
and computer graphics technology, spurred initially by the lucrative
markets in advertising and media communications, are already finding
uses in engineering and medicine. But, can this technology be used to
extend the frontiers of science? Science seeks to explain phenomena
not just understand data. We also need to examine the nature of
scientific insight and its perceived role within the broader spectrum
of science lest we distort the role of ViSC within science. We will
now outline our thoughtsonthese matters in the hope that it spurs
wider discussion.
The direction of advance in computer graphics
Although two dimensional computer graphics continues to have many
uses, the thrust of computer graphics has been towards the achievement
of simulated realism in three dimensional, volumetric and animated
graphics. These developments have been spurred by many sponsoring
applications, from flight simulation and media advertising to medical
imaging and environmental impact analysis. Within science such realism
articulates the requirements of empiricism. However, as Einstein
pointed out, empiricism is only the beginning and end of science it is
not the major source of scientific insight. He believed that "pure
thought is competent to comprehend the real as the ancients dreamed"
(see Madden, 1960, p 83). He expressed the view that every attempt at
logical deduction of the basic concepts and postulates of mechanics
from elementary experiences is doomed to failure. Scientific insights
are often grounded on successive layers of logically derived concepts.
The capacity to derive such abstractions varies from individual to
individual but it is believed to be essential for pattern recognition,
a task which humans can still perform better than computers.
Haber and McNabb referred to the very important issue of accuracy
versus efficiency in rendering and noted that accuracy should not
imply photorealism. Simple models, such as Lambert's cosine law of
diffuse reflection, were regarded as potentially more useful than more
elaborate ones, such as radiosity and ray tracing. This merely limits
the degree of realism for reasons of computational efficiency. We need
to promote efficient human processing through abstraction. The latter
runs counter (in reverse direction) to the traditional preoccupation
of computer graphics with realism. Visual thinking involves
abstraction and graphic notation. The process of adding calculated
realism to abstract descriptions of objects and scenes is thus
contrary to the requirements of visual thinking. Scientific
applications of Cartography employ a variety of transformational
processes to derive abstractions about reality and then to describe
these through generalised graphic symbolism. The map is a graphic
précis of reality.
Whilst there have been considerable advances in the achievement of
realism there have not been corresponding advances in automated
generalisation nor map design. Automated map production is still
limited to the realm of map reproduction rather than creative
visualisation. Despite more than 20 years of research we still do not
have algorithms which can emulate the cartographer's skill in map
generalisation and name placement. Kasturi et al (1989) pointed to
some contemporary research on aspects of this problem. Even seemingly
trivial tasks, like line simplification, are not performed adequately
by even the most celebrated line simplification algorithm in
cartography, which is also used in pattern recognition (Visvalingam
and Whyatt, 1990 & 1991). Consequently, the ideal goal of a scale free
database is not yet feasible (DoE, 1987). Existing systems for
visualisation cannot as yet produce the stylised images which a
skilled cartographer can create. This is because we do not fully
understand the process of visualisation, which is partly unconscious
and intuitive.
Creative thinking
Wallas (1926) suggested that creative
thinking involves four stages, namely preparation, incubation,
illumination and verification. ViSC can facilitate preparation and
verification but incubation and illumination are mental stages. As
Gregory and Zangwell (1987) put it, in order to innovate, scientists
must break the grip imposed on our imagination by the powers of
logical story telling; we must be willing to subvert the conventional
wisdom on which our everyday competence depends. We do this without
difficulty in dreams. The unconscious is known to have led to many a
"Eureka" in science. Often the vision is symbolic. For example, the
dream of a snake biting its tail was seen by Kekule as the clue to the
structure of the benzene ring.
Scientific visualisation need not be spontaneous; it could be directed
but it implies the power and the process of abstracting and
representing the essence of a complex problem in symbolic or
schematic, and not just photorealistic, form. In discussions about the
psychology of his creative work, Einstein (See Madden, 1960, p 90)
stated that the "physical entities which seem to serve as elements in
thought are certain signs and more or less clear images which can be
'voluntarily' reproduced and combined .... taken from the
psychological viewpoint, this combinatory play seems to be the
essential feature in productive thought." It is only when this
associated play is sufficiently established and can be produced at
will does the laborious process of "connection with the logical
construction in words or other signs which can be communicated to
others" begin.
We are not all Einsteins and we need to externalise our visualisations
in order to develop and refine our thoughts and spur our imagination.
Productive thinking is characterised by divergent thinking (Gregory
and Zangwell, 1987). Divergent, like lateral, thinking does not rely
necessarilyonlogical deduction. Current systems for visualisation do
not encourage a "combinatory play" with multiple images; they do,
however, offer opportunities for brain storming which could reveal
anomalies or unexpected trends and patterns during the preparation
stage. But the current euphoria over the prospect for brain storming
with ViSC must be tempered by at least a basic understanding of the
nature of the scientific apparatus.
The apparatus of science
The scientific apparatus, like all
good tools, is constructed and used in a manner which fits its
purpose. Some instruments, like the barometer, were specifically
developed to fit in with and further the process of scientific
explanation. When the existence of a prototype telescope was brought
to the attention of Galileo, he saw the potential that it offered and
sought to understand and modify the principles underpinning its
construction so that it could become an instrument of science.
Scientists do not normally use any tool at hand without it first
becoming a part of the framework of science. The euphoric
literatureonViSC gives the Impression that sheer eye balling of
coloured displays and entrancing movies could somehow generate insight
although McCormick et al did point out that "the most exciting
potential of widespread availability of visualisation tools is not the
entrancing movies produced, but the insight gained and mistakes
understood by spotting visual anomalies while computing". Even this
requires some a priori expectations.
The skill in seeing
Scientific instruments, particularly
when in prototype form, do not reveal the hitherto unseen in a direct
and unproblematic way. It is theory that helps the prepared mind
extract significant objects and relationships from artefacts, noise
and other contextual information. Many readers will have come across
images which may be seen in different ways, for example as a young
woman or an old hag. The presence of illusions, subjective contours,
ambiguities and artefacts in images are discussed in the texts by Rock
(1990) and Gordon (1989). It is well known that visual images have the
power not only to add clarity and spur insight but equally to distort
reality and mislead the uncritical. Reichmann (1961) wrote on the use
and misuse of statistics. Others, such as Monmonier (1977), Bertin
(1983) and Tufte (1983), have shown that it is equally easy to become
confused, even if not deceived, by graphics.
Furthermore, as Comte is reported to have pointed out in 1829, our
eyes would not even notice relevant facts unless they were guided by
prior expectations. Unless we are actively looking for an insight the
meaning of an image may be completely lost. The snake in the dream
would have meant nothing to most chemists but Kekule recognised its
significance and appreciated that it was the shape and not any logical
or mystical significance of the serpent that was significant as it was
in many cultures and religions, including Christianity. The mind of
Kekule was looking for the shape.
Seeing is a subjective process; this is why the sciences have sought
more objective and replicable quantitative procedures.
ViSC within the process of science
ViSC is being offered as a tool for
understanding data and invoking insight. We do not deny the potential
offered by ViSC for this purpose. However, this under sells the
potential role and value of ViSC to science. Insight plays an
undeniable role within science. It does not have to be massive nor
revolutionise to advance science. A fertile mind can derive insight
from a variety of sources past experience, intuition, dreams,
accidents and false starts. While ViSC can serve the insightful mind,
it must be appreciated that mere technology cannot of itself make
competent but conventional and conforming minds more creative. There
is, however, the distinct danger that ViSC could become a
sophisticated game. The UK Science and Engineering Research Council
(1989, p 8) cautioned postgraduates and their supervisors that a
common reason for students not completing their thesis within the
allotted time is because they get 'hooked on' computing. Whether ViSC
contributes towards the making of genius, to mental constipation or to
diversionary activities depends upon the mind and inclinations of the
individual scientist.
The source of insight is not the central concern within modern
science; it appears to be of greater interest to the history and
philosophy of science since even modern psychology seems more inclined
towards investigating matters which may be subjected to the scientific
method. An insight only assumes a place within science if it is
productive, verifiable and proven to be probably true and useful. The
insight therefore must lead to the formulation of useful and testable
hypotheses. As science advances it becomes more theoretical. Whereas
facts may be verified by an appeal to primary sources of knowledge and
immediate experience, hypotheses and theories are often verified by
laborious and indirect means which involve semantic mapping of
abstract concepts and symbols onto everyday concepts and experiences.
Unlike an artist who is free to express his personal insight and
visualisation in a free and unfettered manner, the scientist must
explore the many ramifications and implications of his unique insight
and develop a convincing argument in support of his propositions. ViSC
can expedite this stage of verification. Once the evidence is
sufficiently established to win support, the process of verification
is continued as a part of the humdrum of normal science by a community
of scientists. The bulk of scientific activity is concerned with the
verification, refinement and application of insights. This reliesonthe
existence of a pool of tacit knowledge about the domain of inquiry and
appropriate methods of investigation.
The language of mathematics has been the primary tool for symbolic
abstraction and manipulation of concepts. The emerging field of
Scientific Visualisation is basedonthe proposition that the symbolic
language of graphics is an equally powerful tool for visual thinking.
The French cartographer Bertin (1983) had already advocated this in
1967 noting that the eye brain system can 'See' multiple and complex
patterns in a single well designed visual display of data. If ViSC can
transcend photorealism and address the need for a "combinatory play"
with symbolic graphics through direct manipulation, it can become a
powerful tool not just for observation and simulation but also for
ideation and expression; i.e. it will become valued not just as
another means for invoking insight but also, and perhaps more
importantly, as a convenient and flexible apparatus for developing and
verifying it.
3.5 Visualisation Idioms
Haber and McNabb (p 87) pointed out
that "scientists and engineers are no more willing to get involved
with graphics languages, windowing systems, communications mechanisms,
and the like than they are to write low level device drivers".
Scientists are equally likely to be put off by unnecessary jargon such
as the term, 'visualisation idiom', coined by Haber and McNabb. This
jargon does not really convey anything new. Moreover, inadvertantly,
it places the emphasis on the communicative, rather than an the
exploratory, role of graphics as explained below.
Haber and McNabb cited Websters Dictionary definition of an idiom as
"an accepted phrase, construction or expression .... having a meaning
different from the literal". The term idiom also has a further
connotation; it is often a discrete expression characteristic of a
particular form of spoken or written language which is not logically
or grammatically explicable (Chambers ZOth Century Dictionary). This
implies that an idiom has a specific meaning, endowed by usage, which
need not necessarily be derived from the logical meaning or
grammatical structure of the words, as in the idiom "It is raining
cats and dogs".
Haber and McNabb stated that just as a listener has difficulty
understanding a verbal idiom in a foreign language, a viewer is unable
to interpret a graphic display without understanding the steps in the
'visualisation idiom' used to generate it. They defined a
'visualisation idiom' as any specific sequence of the transformations
(see Section 2) that produce an abstract display of a scientific data
set. They used different examples of 'visualisation idioms' to
demonstrate the scope for user configurability and to point out that
minor variations in the mapping transformations (note this reversal to
the phrase "mapping transformations", which has been more widely used
in the past in place of visualisation) can lead to dramatic changes in
the final images. They suggested that proper technical documentation
and visual aids are essential for proper understanding of both the
process of visualisation and the display. Equally, they believed that
visualisation idioms should not require excessive effort or
explanation for qualitative understanding; AVOs should therefore be
basedonintuitive analogies which take into account human factors.
Scientists have an obligation to be precise about the meaning of
jargon they use. We found it difficult to understand and thus adopt
the term, 'visualisation idiom', because we cannot grasp its intended
value. We suggest below that this is due to differences in our
assumptions about the formal language of graphics and its role in
scientific enquiry.
It is difficult to generalise about the complex medium of visual
imagery since it takes different forms to serve a variety of quite
different purposes. A full exploration of the nature of graphics is
irrelevant to our objection to the phrase, 'visualisation idiom'. The
thread of our argument starts with the popular assumption that
graphics is a natural language for communication. The ease of use of
Graphical User Interfaces (GUIs) by direct manipulation of widgets and
icons has persuaded many to accept the vendor's claim that graphics is
natural. They have thus come to expect that all graphics should be
natural. Haber and McNabb, for example, believed that AVOs could be
made intuitive by use of appropriate analogies.
There is no doubt that graphic representations of experienced reality
may be grasped intuitively. Such reproductions need not be real nor
photorealistic. Neither do we need to know how they were produced to
share the dreams (visualisations) of the creators of Luxor Jnr and
Snow White nor to experience virtual reality in a simulator. However,
as noted earlier, scientific visualisation is not always about the
tangible and it is not always possible to map abstract concepts onto
intuitively grasped forms. Directed thinking is expedited by symbolic
languages, such as mathematics, chemical nomenclatures and notations
(Cooke Fox et al, 1989), and the natural languages. We do not expect a
scientist to perceive intuitively some insight in some mathematical
expression without some appreciation of mathematical principles and
notations. A scientist cannot write papers without a grasp of the
vocabulary and grammar, let alone the nuances, of the language of
communication. Yet, it appears that we somehow assume that graphicacy
(a term coined by Balchin, 1972) is universal.
Haber and McNabb recognised that not all visualisations are
intuitively grasped and implied that this is due to a lack of
consideration of human factors and the presence of idioms. They also
implied that these problems may be resolved by user centred design,
technical documentation and on screen keys and legend. We are not
questioning the value or requirement for such measures. Instead, we
are concerned a) by the assumption that a need for explanatory legend
implies the presence of idioms and b) by the lack of stressonthe need
for education and training in mental visualisation.
In theory, using ViSC systems, even 5 year olds should be able to turn
out displays by plugging together modules from libraries in the same
way that many are already able to produce pretty maps of dubious value
given a base mapona 'paint' system. Whilst catchy phrases like 'hi' or
'the mind boggles' can easily become idioms of a language by popular
usage, arbitrary mappings seldom become a part of the idiolect of a
science. The abstractions and the graphic representations must be
logically derived to facilitate the intended mental visualisations
even though disciplines, such as cartography, do encourage some
limited measure of artistic licence. This is to provide some latitude
for minor adjustment to facilitate a stretch of the imagination which
is necessary to achieve the desired mental visualisation. Cartographic
maps are of different kinds (Robinson et al, 1984). Intuitively
perceived maps are largely used for purposes of communication,
especially when directed at those not trained in the use of graphics.
Analytical maps, which support scientific research, can be much more
complicated. It is expected that the scientist has been educated and
trained in the processes of construction and use of these logically
derived graphic formalisms as a part of his apprenticeship.
