Semantic Networks
This document concerns the management of the output of insight generators, the software agents utilized in the insight generation systems. The solution to managing these reports involves the automatic creation of a repository for all materials generated by various insight generators; this repository allows the user to navigate through this continually growing space of marketing reports, gaining new insights about the relationships between items of interest and adding new insights in the process. The goal of the system is to make all marketing information and insights generated by the man/machine interaction available to the user, so that there is a convergence towards a "conservation of information". To use a geometric metaphor, the goal is to make the user equidistant from all information at all times, as illustrated below.
The output of insight generators like I Want is paper; every time the system is run, a paper report is produced. If the system were run for every retail account in every market, it would produce hundreds of reports. If similar insight generators focused on other insights (distribution, variety, coupons, shelf space, prices, etc.), the collection of agents would produce thousands of reports. If each agent were run for each brand item (each size, package type, flavor, etc.), there could be many thousands of reports. Finally, if all of the agents were run each month for all the brands, then millions of reports could accumulate.
Further, these agents could be run for a firm's competitors. The system could be run backwards; that is to say, it could be run from the perspective of the brand's competitors in a particular category. From this, the brand group could learn where their brand is vulnerable to attack from competitors seeking to take merchandising support away from it. Also, the sales force could be informed where not to ask for more support, such as in the instance where their brand is receiving far more feature support than its volume share warrants. Such use of this system can be called exposure analysis and it was further explored in the Market Opportunity Inspector (MOI) prototype.
One approach to dealing with all of these insights is summarization, which is explored in the Marketing Opportunity Inspector (MOI) document. The MOI system assumes that an "I Want..." system has been run for a brand in all accounts in all markets. Each page of output corresponds to an exposure, an opportunity or neither for the brand. MOI takes these and summarizes the number and magnitude of the opportunities and exposures. The output of the system is again a sheet of paper. MOI helps to generate insights about a brand's situation, by locating its strengths and its weaknesses. The output of one insight generation system (I Want) became the input of another insight generating system (MOI). In both systems the output is a high-quality paper report. Insight generating systems such as MOI are employed to summarize the lower level information and pinpoint insights from this mass of textual output. But, the summaries themselves also add to the ever-growing output and must also be made available to users.
A second approach to the problem of insight management is to manage the output itself as a sort of library of information known about the brand. This library would be part of a system that manages information and insights that are in the form of compound documents, i.e. pictures and text, that goes beyond a strict hierarchical arrangement. The user needs to be able to move in multiple directions from any vantage point and create hierarchies as needed. The information needs to be arranged in a structure that follows the intrinsic relationships between the context of the reports and their contents. The reports must be managed according to some "mental-model" of the world of marketing.
One technology for capturing and reasoning with such mental models is a semantic network ... the topic of this document.
Semantic networks are knowledge representation schemes involving nodes and links (arcs or arrows) between nodes. The nodes represent objects or concepts and the links represent relations between nodes. The links are directed and labeled; thus, a semantic network is a directed graph. In print, the nodes are usually represented by circles or boxes and the links are drawn as arrows between the circles as in Figure 1. This represents the simplest form of a semantic network, a collection of undifferentiated objects and arrows. The structure of the network defines its meaning. The meanings are merely which node has a pointer to which other node. The network defines a set of binary relations on a set of nodes.
Pick up almost any technical book and look in the preface or introduction. Invariably there is a chapter dependency diagram. It is a node-link structure, a semantic network in which the nodes represent chapters and the links represent the relationship of which chapters should be read before which other chapters.
To move semantic nets from this abstract realm to something more concrete, let us consider an example from the structure of marketing. To begin simply, let us introduce two nodes and a link.
The node on the left labeled "Quad Cities" is linked to the node on the right, labeled "Market", and the arrow is labeled "is-a". Quad Cities is an example of a market. The diagram, in other words represents the fact that there is a binary relation between a market, Quad Cities, and the concept of a market. Another node with the label "Los Angeles" and a "is-a" link from this node to the "Market" node could be added, again representing that "Los Angeles" is a type of "Market".
If a retailer node is added to Figure 2, the structure of the network becomes apparent as shown in Figure 4. Markets generally contain retailing entities. To add an example of a retailer, add a node labeled "Chain56" and two links - one from the retailer "Chain56" to "Quad Cities" labeled "is-a-retailer-in" and one from the node "Chain56" to the node "Retailer" labeled "is-a". This illustrates that Chain 56 is a retailer in the Quad Cities market.
It is now important to note a point or two of possible semantic confusion. Notice that the nodes in this small network are not all of the same "type". The node labeled "Market" represents the generic or meta or class concept of a market; it represents the abstract concept of a market. It can be thought of as possessing properties common to all markets. The node "Quad Cities" represents an individual instance of the node "Market". The node "Quad Cities" represents a particular market. The same is true of the relation between the node labeled "Retailer" and the node labeled "Chain56". The node "Retailer" again represents the concept of a retailer that is common across all particular retailers. One instance of such a retailer is the node labeled "Chain56". In order to distinguish between these two types of nodes, the class nodes become boxes and the instance nodes become ellipses, as in Figure 5.
Another class node, labeled "Item", that represents the abstraction of items in a category, can now be added. Along with that, an instance of an item, labeled "87481", is added. Notice that there is a strong relationship between the type or class nodes and the column headings or entities of a relational database table. We will exploit this similarity later in this paper. Thus, another "is-a" link and a new link, "item-carried-in", must be added to the node "87481" and the node "Chain56" respectively. These new additions are shown in Figure 6. The information now being represented is that Chain56 is a retailer in the Quad Cities market and that Chain56 carries the item 87481.
As the nodes proliferate, the meanings of these links need to be considered. It should become apparent that not all links are alike. Some links express only relationships between nodes, and are therefore "assertions" of the nature of the relationship between two different nodes. For example, the link "item-carried-in" in Figure 6, which illustrates the relationship that retailer Chain56 carries the item 87481. The "is-a" links in Figure 6 are "structural" links in that they convey "type" information about the node. This information is about the node itself and not about the relationship it has to a different "type" of node. For instance, the node labeled "87481" is an instantiation of the class node labeled "Item".
In Figure 7, more nodes and links are introduced to the original network. There is now a "Brand" class node with an instance node "Ivory". The link "is-brand" conveys the information that the item 87481 is the Ivory brand. There are now also class nodes labeled "Manufacturer", "Category", and "Category Attributes". The Category Attributes class is linked to three other class nodes labeled "Size", "Color" and "Segment". These represent particular attributes of a particular category; in this instance, the liquid light duty detergent category, of which Ivory is a member. The "is-a" links between the class node Category Attributes and the class nodes Size, Color, and Segment represent a relationship of class to subclass and, hence, "structural" links. Here again are links that do not denote a relationship between different types of instance nodes, but give information about a class node itself. The class node Color is a type of Category Attribute.
Our network in Figure 7 now has a representation for information about the item node 87481. For instance, it is a form of Ivory which is manufactured by Procter & Gamble; it is the 22 ounce size, white in color and competes in the Mildness market segment of the liquid light-duty detergent category. This is one item in one chain in one market. The database used in the Marketing Information Center prototype has 100 items in five chains in one market. Each item can be one of seventeen brands made by twelve manufacturers which comes in seven sizes and eight colors and can compete in one of five segments. This database is a pared-down version of a scanner database for the Quad Cities market which has many more retailers and a good many more items. The network in Figure 7 becomes very complex with a 100-fold increase in the amount of information.
None of the networks have shown any structural links among class nodes, except for Figure 7 which shows only a subclass relationship between Category Attributes nodes and various class node attributes. Figure 8 shows possible structural relationships between class nodes.
In this figure the instance nodes have been left out in order to show more clearly possible relationships between classes. Remember class nodes represent larger, more general concepts and just as general concepts can have more refined sub-concepts, the particular types of category attributes, such as Size, are represented as a sub-class of the broader concept of Category Attributes as shown previously. Notice that the class Category Attributes is a kind of abstract class that probably would never have an instance node tied directly to it. It can only have relationships to other class nodes. So, general concepts are represented such as the concept that there are Manufacturers who create things termed Brands which are suppied to things called Retailers. Retailers are in an abstraction called a Market and carry instances of Items. All of this is obvious from the diagram. What is not so obvious is that the nodes themselves can contain more than meets the eye. The Retailer node is a short-hand notation for a bundle of concepts that make up a real-live retailer, such as the fact that retailers have a headquarters and stores and control shelf space, price and display. The links, such as the one labeled "supplies" between the Manufacturer node and the Retailer node, are really more like a co-axial link than a simple arc. This particular link represents a bundle of various relationships between Manufacturer and the abstraction, Retailer. This detail is shown in Figure 9.
Another import characteristic of the node-link representation is the implicit "inverse" of all relationships represented by the directional arrows. If there is an arrow going from one node to another, this also implies the reverse - that there is an arrow from the second node to the first. In Figure 10, there are the nodes labeled "P & G" and "Ivory" with the link labeled "makes". The direction of the relationship is that "P & G makes Ivory". Further, some linguistic terminology for our binary relationships could be used: "P & G" is the subject and "Ivory" is the object, and "makes" is the verb or action or link between them. This will be discussed in greater detail later.
This "P & G makes Ivory" relation implies the inverse relationship that "Ivory is-made-by P & G", as shown in Figure 11.
The representational or expressive power of semantic networks has been discussed thus far. As with any kind of knowledge representation scheme, a way of inferring knowledge that is not directly represented by the scheme is needed. The ability to work with incomplete knowledge sets a knowledge representation apart from a database. To give an example of what can be gleaned from the semantic network in Figure 7 that is not directly represented, consider Figure 12. It is an extraction of Figure 7 containing only three nodes and two links.
The information explicitly represented is that the item numbered 87481 is the Ivory brand and that Procter & Gamble makes Ivory. The inverse relationship of the item 87481 to the brand Ivory, i.e. that Ivory is-item-number 87481 is shown in Figure 13.
By tracing the path from the node P & G to the node Ivory via the arrow labeled "makes" and then from the node Ivory to the node 87481 via the arrow labeled "is-item-number", we can infer that Procter & Gamble manufactures the item 87481 by inferring a link labeled "makes-item" between the node P & G and the node 87481, as shown in Figure 14. This may seem obvious, but remember this small amount of new information need not be explicitly represented in the original network.
Described mathematically, composing arrows occurs by placing them end-to-tail. This composition creates a new arrow. In Figure 14, a triad of nodes is formed by arrows said to "commute". It is not possible to compose every pair of arrows, only those whose destinations and sources correspond. The destination of the first must be the source of the second. By composing arrows, new relationships between nodes can be found and described. This process is sometimes called "chasing arrows" and the terminology introduced stems from a branch of mathematics called Category Theory.
Figure 15 shows the results of more "arrow chasing". Additional relationships are derived, such as Ivory is a Brand carried by Chain 56, Procter & Gamble makes a product that competes in the Mildness segment,and Ivory is a Brand competing in the Mildness segment. Notice that layers of relationships between nodes can be built.
This discussion has introduced the concept of a semantic network consisting of nodes and links. The nodes represent concepts and the links represent relationships between these concepts. A distinction was made between instance nodes and class nodes: the former represents general notions of the latter of which there may be many types. The concept of links which extend from the instance node level to the class node level was given along with an introduction of the notion of abstract classes. The reversibility of the arrows and the method of inferring new relationships between nodes from existing ones was also given. Several figures illustrated these concepts using an example semantic network built from a scanner database header file of the liquid light-duty detergent category.
Limitations of Semantic Networks
This chapter should not end without some discussion of the limitations of semantic networks, and a comparison of their traditional use versus their use in the management of marketing insights.
Semantic networks as a representation of knowledge have been in use in artificial intelligence (AI) research in a number of different areas. Some of the first uses of the nodes-and-links formulation were in the work of Quillian and Winston, where the networks acted as models of associative memory. Quillian's work centers on how natural language is understood and how the meanings of words can be captured in a machine. Winston's work concentrates on machine learning and specifically on structural descriptions of an environment. Winston's work describes pedestals and arches formed from more elementary pieces such as wedges and blocks; these make up the famous "blocks world" that has been utilized by many research efforts in semantic networks.
The other major area in which the use of semantic networks is prevalent, is in models based on linguistics. These stem in part from the work of Chomsky. This latter work is concerned with the explicit representation of grammatical structures of language. It is opposed to other systems that tried to model, in some machine-implementable fashion, the way human memory works. Another approach combining aspects of both the previously mentioned areas of study was taken by Schank in his conceptual dependency approach. He attempted to break down the surface features of language into a network of deeper, more primitive concepts of meaning that were universal and language independent.
Such creations and uses of semantic networks have led to any number of epistemological problems. Numerous researchers have attempted to address these problems. Barr and Feigenbaum state that:
In semantic network representations, there is no formal semantics, no agreed-upon notion of what a given representational structure means, as there is in logic, for instance.
Of course, the success of logic in this respect is debatable, but semantic networks do tend to rely upon the procedures that manipulate them.
For example, the system is limited by the user's understanding of the meanings of the links in a semantic network. As pointed out previously in the example of the network of marketing insights, links between nodes are not all alike in function or form. Hence we need to differentiate between links that assert some relationship and links that are structural in nature). A paper by Brachman on the subtleties of the "is-a" link revealed even more distinctions in the uses of this link. According to Brachman "is-a" links can be divided into two groups, depending upon the nodes involved. One use of the "is-a" link resembles our distinction between an instance node and a class node. The link represents the relationship of instance nodes to abstract, generic qualities shared by many instance nodes. Brachman's other use of a link is between two instance nodes. These two major divisions can then be further broken down into finer uses of the link.
If problems and sublime uses characterize links, then nodes are not much better. The seeming simplicity of a node that represents a single concept or object in the world is actually fraught with complications. The question "what is a node?" is on par with "what is a market?" As in the discussion of links above, we need to distinguish between nodes that represent some set of objects and nodes that represent classes of qualities shared by these objects. There can be properties for instances of a class that all members of a class share as well as properties of the class itself. A property of a market is that it is made up of retailers. For example, a member of the class Market, such as Quad Cities, contains specific retailers, such as Chain 56. Chain 56 is an instance of the class Retailer. The problems of epistomology and the semantics of semantic network representations are discussed further in Brachman.
It should be noted that the sample network discussed at length in this book shares the advantages and disadvantages of any semantic network. We take "naive semantic networks" aka "naive set theory". We do not worry overly about the epistemologico-semantic problems associated with the use of the representation. Some of the difficulties are in fact side-stepped since the final goal for this network is not quite the same as the goals of other researchers. One fundamental difference between the use of the network representation in this book as opposed to elsewhere, is that we are not trying to represent natural language or associative memory independent of users. In fact we rely heavily on the user's intuitive understanding of the world. Semantic networks are used in this project as structures of knowledge that encourage the user to interact with them.
Having marketing knowledge visible to the marketing professional is one of the major advantages of AI technologies and of tools such as spreadsheets. This topic is pursued in more depth in Chapter 18, while the next chapter describes a computer-based tool for implementing semantic networks.