Diagrammatics:
Thinking with Diagrams '97 Position Statement

Zenon Kulpa

[This is a slightly modified and updated version
of my Position Statement for the Thinking with Diagrams '97 workshop]

My interests in diagrammatic representation and reasoning have their origin in the idea of computer-aided visual communication I had called in early eighties by the name of "iconics" [1]. After moving with my work into other (though related) areas, I have returned to the just emerging field of diagrammatic reasoning in early nineties. To put my understanding of the subject in order, first I wrote the survey paper [2]. Subsequently, I am working, till now, mostly on applications of diagrams in Qualitative Physics [3], proposing recently a diagrammatic representation of interval space and interval relations [4, 5].

My understanding of main issues of this field can be summarized as follows:

Diagrams are a kind of analogical (or direct) knowledge representation mechanism that is characterized by a parallel (though not necessarily isomorphic) correspondence between the structure of the representation and the structure of the represented [6]. E.g., relative positions and distances of certain marks on a map are in direct correspondence to relative positions and distances of the cities they represent, whereas in a propositional representation, its parts or relationships between them need not correspond explicitly to any parts and relations within the thing denoted. The analogical representation can be said to model or depict the thing represented, whereas the propositional representation rather describes it. A similar distinction can be made regarding the method of retrieving information from the representation. The needed information can usually be simply observed (or measured) in the diagram, whereas it must be inferred from the descriptions of the facts and axioms comprising the propositional representation.

Knowledge representation and reasoning schemes originating in logic (specifically, predicate calculus), called propositional, were for long predominant in AI, as opposed to analogical schemes, including diagrammatic representations. This was only partially due to limitations of early computers and software in handling visual (or graphical) data. Possibly even more important cause was that most of the researchers in computer science and AI were (and still are) of mathematical background. And mathematics has been generally ruled by an implicit dogma stating that propositional reasoning using logic is the ultimate tool of precise and formal thinking. Many mathematicians, especially logicians (but even researchers in geometry!), tend to use diagrams rarely, sometimes as heuristics to prompt certain trains of inference, but mostly only as informal aids to understanding for uninitiated. On occasion, some of them explicitly stated that the diagram has no proper place in the proof as such. This state of affairs seems to change now - as recent research has shown, it is quite possible to formalize diagrammatic reasoning so as to make it no less precise and formal than logical reasoning [7, 8].

The main advantages of diagrammatic representations I would like to summarize as follows [2, 9]:

Diagrams are (at least) two-dimensional:

Richer possibilities of specifying and structuring in two dimensions (topological, metric, morphological: by connectivity, proximity, shape, etc.), compared to those available in a one-dimensional (linear) strings of symbols in propositional formulations, lead to the reduction of:

Diagrams represent analogically:

They permit explicit representation and direct retrieval of information (especially structural and spatial relations) that can be represented only implicitly in other types of representations and then has to be computed (or inferred), sometimes at great cost, to make it explicit for use. Also, onstruction of an analogical representation (e.g. a diagram) for the given set of facts usually causes the emergence of certain new entities, properties of the problem elements, and relations between them that follow from the given facts. These so-called "implicit facts" or "emergent properties" are a kind of ready-made inferences that can be directly read from the diagram at little or no cost.

Visual processing is easy:

Humans possess a well developed apparatus for making easy perceptual inferences on a diagram. However, this does not yet apply in full to computers which are still somewhat better at brute-force number crunching, rather than at visual data analysis.

The main criteria characterizing usefullness of a diagrammatic language can be summarized as follows [2, 10, 11, 12]:

Expressiveness:

The visual language must be able to express all the facts and only the facts we want to visualize. The first requirement is obvious; the second takes into account the "implicit facts" feature - the visual representation should not, as a by-product of representing the wanted facts, represent implicitly unwanted (false, incorrect) facts.

Effectiveness:

It determines how easy it is to state the facts in the language and how easy it is to perceive the facts from the representation. Effectiveness is relative with respect to the user of the representation - other rules apply when it is to be used by humans, other when by computers. Human visual perception criteria are quite well known and codified in extensive human factors and graphic design literature. E.g., it is well known that color and shading are far less effective in representing accurately numerical quantities than position on a scale or length of a line. Much less can be said about the preferences of computers, although some results, both negative and positive, are emerging.

Presentation goal:

It takes into account what kind of operations the user wants to perform with the representation, or what kind of tasks the representation will be used to aid. Examples of different goals might be, e.g., accurate value lookup (then representing the values as marks on a numerical scale is advisable) versus general magnitude comparison (then representing the values by size of graphical entities might be better).

Relevant Papers

[1]
Z. Kulpa. Iconics: computer-aided visual communication.
In: Levialdi S. (Ed.): Digital Image Analysis. (Proc. 2nd Conference on Image Analysis and Processing. Fasano, Italy, 1982). Pitman, London (1983), 280-282.
[2]
Z. Kulpa. Diagrammatic representation and reasoning.
Machine GRAPHICS & VISION 3(1/2), (1994), 77-103.
[PostScript file available (380KB, gzipped)]
[3]
M. Kleiber, Z. Kulpa. Computer-assisted hybrid reasoning in simulation and analysis of physical systems.
CAMES, 2(3), (1995), 165-186.
[PostScript file available (170KB, gzipped)]
[4]
Z. Kulpa. Diagrammatic representation of interval space in proving theorems about interval relations. Reliable Computing, 3(3): 209-217, 1997.
Also presented at INTERVAL'96 Conference (Würzburg, Germany, September 30 - October 2, 1996).
[PostScript file of the Extended Abstract (2 pp., 2 figs.) available (170KB, gzipped)]
[5]
Z. Kulpa. Diagrammatic representation for a space of intervals.
Machine GRAPHICS & VISION 6(1): (1997), 5-24.
(Special Issue on Diagrammatic Representation and Reasoning)
[PostScript file available (620KB, gzipped)]
[6]
A. Sloman. Afterthoughts on analogical representations.
Proc. 1st Workshop on Theoretical Issues in Natural Language Processing (TINLAP-1), Cambridge, MA (1975), 164-171. (Reprinted in: Brachman R.J., Levesque H.J. (Eds.): Readings in Knowledge Representation. Morgan Kaufmann, San Mateo, CA (1985), 432-439).
[7]
D. Wang, J.R. Lee. Visual reasoning: its formal semantics and applications.
J. Visual Languages and Computing, 4(4), (1993), 327-356.
[8]
G. Allwein, J. Barwise, Eds. Logical Reasoning with Diagrams.
Oxford Univ. Press, Oxford (1996).
[9]
J.H. Larkin, H.A. Simon. Why a diagram is (sometimes) worth ten thousand words.
Cognitive Science, 11, (1987), 65-99.
[10]
J. Mackinlay, M.R. Genesereth. Expressiveness and language choice.
Data & Knowledge Engineering, 1, (1985), 17-29.
[11]
S.F. Roth, J. Mattis. Data characterization for intelligent graphics presentation.
In: Human Factors in Computing Systems - VII (Proc. of the Conf. on Computer-Human Interaction (CHI'90)). ACM Press, (1990), 193-200.
[12]
J. Bertin. Semiology of Graphics: Diagrams, Networks, Maps.
University of Wisconsin Press (1983).

Back to my [DIAG] diagrammatics page
Original version: Jan 17, 1997.
Revised and updated: Jun 8, 1999.