Scoring Manual Paper.11272002.wpd

has to do with concepts. The senses only deceive us; hence we should, in acquiring knowledge, ignore sense impressions and develop reason.

Aristotle (384-322), in codifying logical reasoning, set down rules of inference and recognized the importance of axioms for logic, postulates for the subject at hand, definitions of terms and the importance of giving logical arguments starting with the postulates. The model of hierarchical complexity follows in that tradition. Combining of Aristotle's precise formulation of logic with Thales' method, the main elements of modern science were then in place

The Model of Hierarchical Complexity on which scoring and problem construction is based, is a mathematical theory of the ideal. It is a perfect form as Plato would have described. It is like a circle. Once one draws it, it is no longer perfect. The lines have width, it is not perfect. If can be be perfectly round.

Events

Scientific accounts of behavior are built out of both analytical and empirical accounts of events. One problem that continually arises is what perturbations to consider as existing, or in other words, what constitutes an event. There only seems to be one necessary restriction on saying that something exists. The restriction is rather weak compared to those required by operationalism but strong with respect to intuitionism and phenomenonology. With the quantitative behavioral developmental theory that follows, we have to consider events as the basis. This notion is less restrictive than behaviorists' notions of stimuli and responses and so allows the theory to consider events that may not be clearly stimuli or responses. On the other hand, we do not want to make the mistake of Piagetians that thoughts, "schema," and verbalizations that belong to mental structures are the only causes of actions.

How do we know that something is an event? Events are potentially detectable perturbations. Perturbations are classed as events when they achieve some potential to be observed, witnessed, and in some way distinguished from the remaining noise by two independent paths of detection. The term event is used here to include all such perturbations, both public and private. The notion of paths of detection is not deniable or reducible lest we get into an infinite regress. These paths do not require direct observation. Note also that more experiencers or more experiences do not count as more independent paths.

Potential events may be inferred as long as there are two distinct paths leading to that inference, such as the case with electrons. Electrons may be detected through a multitude of paths by which inferences as to the existence of an "electron event" can be made. One can measure the magnetic moment of a single electron moving along a path in a magnetic field, the electric charge in an electric field, or the ionizing potential in a liquid hydrogen bubble chamber. There are numerous other ways of detecting the electron.

The reason two paths are required for events is because one path alone could mean that the perturbation could serve as its own causal explanation of itself. Some perturbations are deemed as having the status of being only singly detectable by one path. For example, if someone reports that the president is talking to them, there is one path, their report. They do not have a radio, telephone or any other such device and the president is nowhere close by. One other path is necessary to confirm that the president is actually talking to them and they are not reporting a hallucination. Behaviors and causes detected from a personal experience alone have this character. Robert Stickgold (personal communication, 1999) has shown that people think that of what they think, see, and dream as "real" while thinking, seeing and dreaming. The status of events and perturbations is even more complex when activity is not potentially observable, as is with gyrations of the soul or will. These perturbations may be studied in theological and theosophical terms ( Lowenthal, 1989). The best we can do within science is to discuss the report of these perturbations as data to be explained or refer to these perturbations in metaphorical terms.

Behavioral constructs (such as stimuli, behaviors, or consequences) are events. In the case of a verbal report, an observer may hear it. A microphone and meter will show it. There is a difference between the appearance of a perceived event and the actual event. Perceptual activity can transform events. Illusions refer to those instances where people report the appearance of stimuli in ways that distort the physical properties of the objects or events. Let us say one was looking at a color patch and the person said, "I see the color brown." But the color brown has no unique

Three Ways of Knowing about Development

Three Ways of Knowing about Development

With the definitions of perturbations and events, it is possible to show what are the minimum conditions necessary for having a quantitative behavioral developmental theory. One needs to recognize the different ways in which we might know and understand development. The argument is very simple. There are three ways of knowing as shown in Table 1. Knowledge is treated in a much more complex manner in philosophy. Here, the number of paths needed for detecting a perturbation is associated with the field and methodology that claims knowledge.

Table 1

Ways of Knowing

Ways of Knowing	Example of Fields Utilizing These Ways of Knowing	Number of Paths of Detections of Perturbations
1. Analytic: Proved material always true no matter what "data" or "experience" shows	Mathematics, Logic, Parts of Philosophy	No paths of detections of perturbations
2. Phenomenological: Experienced material a property of organisms and sometimes organisms interacting with environments.	Religion, Law, Art, Literature, Dance and Music	One independent path of detection. This means that if one observes an action and hypothesizes a cause, such as free will, then the putative cause may represent one path of detection. Detecting the behavior, however, does not prove that the hypothetical "causal" event is an actual event. If only one path is available, that is, if only one effect can be detected–that is the experience (and its report), there is no way to determine the cause of that experience. The experience is sometimes erroneously said to "cause itself."
3. Empirical: Resultant material from investigations moves scientific towards the truth.	Science, History	Two independent paths. An event can be said to be real in a scientific sense if and only if it is detectable by two independent paths. An independent second path for detecting the hypothesized causal event must be found.

There can be combinations such as 1 and 3, which define most of science. Problems arise with combinations of 2 with 1 (Folk Psychology of Aristotle), 2 with 3 (current mixes of experimental and phenomenonological accounts of free will such as Libet's, 1985). These can lead to various dangerous policies and practices. That does not mean that 2 is not prized for itself. It is.

There were three further developments necessary

The first, we are familiar with. Copernicus (1530) showed that the sun is the center of the solar system. He used mathematics to represent the orbits of the planets. In some sense, this was the first mathematical model. Second, modern thinking about the brain and behavior began with the French philosopher René Descartes (1596-1650). According to Descartes (1637), all action is a response to an event. He thereby introduced the notion of the stimulus and the response. Descartes suggested that 'animal spirits' flowing through the nerves of animals or humans served a similar function in automatic behavioral responses in man and animals or reflexes. The term 'reflex' is derived from the notion that the flow of animal spirits produced by a stimulus is somehow reflected by the brain into an outgoing flow which eventually produces some behavior. G. T. Fechner (1860) G. T. Fechner (1860 Fechner (1860) laid the basis for the application of the experimental method to psychology. His establishment of psychophysics through his publication of Elements of Psychophysics in 1860. He showed introduced the psychophysical scale and showed how to relate psychological variable to stimulus ones. This is exactly what the model of Hierarchical Complexity does. It relates stage of performance to the order of hierarchical complexity of tasks. Lastly, in the early 1960, Krantz, Luce, Suppes, and Tversky ( Krantz, Luce, Suppes, and Tversky, 1971 Krantz, Luce, Suppes, and Tversky, 1971); Suppes, Krantz, Luce & Tversky, 1989 Suppes, Krantz, Luce & Tversky, 1989); Luce, Krantz, Suppes, & Tversky, 1990 Luce, Krantz, Suppes, & Tversky, 1990) introduced the representational theory of measurement. It is the basis for the model of hierarchical complexity.

Tasks

One major basis for this developmental theory is task analysis. The study of ideal tasks, including their instantiation in the real world, has been the basis of the branch of stimulus control called Psychophysics. Tasks are defined as sequences of contingencies, each presenting stimuli and requiring a behavior or a sequence of behaviors that must occur in some non-arbitrary fashion. Properties of tasks (usually the stimuli) are varied and responses to them measured and analyzed. In the present use of task analysis, the complexity of behaviors necessary to complete a task can be specified using the complexity definitions described next. One examines behavior with respect to the analytically known complexity of the task.

History of Stage Scoring

Ever since the introduction of the idea that development proceeds in discrete stages, scientists have argued over the framework for defining and analyzing such stages. Many models were presented to conceptualize development, including the mentalistic theory of Jean Piaget (1954 Piaget (1954), a pioneer in the field of developmental psychology. Though Piaget’s theory did not define all stages precisely, it clearly established that there is one invariant pathway along which stage development proceeds irrespective of content or culture ( Piaget, 1973 Piaget, 1976). Other developmental models followed Piaget’s, and each usually focused on development within a particular domain of information. As more content-oriented models were introduced, the “theme of uniqueness [of each model] was increasingly dropping out” ( Kohlberg, 1990 Kohlberg, 1990). Because the varying informational frameworks of different domains have often concealed the common underlying process of stage development, standardization of research methods has been difficult to achieve. Nevertheless, researchers soon recognized the need for a broadly applicable model of developmental assessment that is necessary in order not only to better conceptualize the patterns and themes of development, but also to conduct comparable cross-cultural studies.

Model of Hierarchical Complexity

The Model of Hierarchical Complexity (MHC) developed by Commons ( Commons, Trudeau, Stein, Richards, & Krause, 1998) offers a standard method of examining the universal pattern of development. This model is based on a theory of General Stage development ( Commons & Richards, 1984a, 1984b Commons & Richards, 1984a, 1984b). The MHC states that all stages are hard distinct stages varying only in the degrees of hierarchical complexity. To counter the possible objection of arbitrariness in such an inclusive and uniform definition of stages, the MHC stage orders are grounded in the hierarchical complexity criteria of mathematical models ( Coombs, Dawes, & Tversky, 1970 Coombs, Dawes, & Tversky, 1970), and information science ( Commons & Richards, 1984a, 1984b Commons & Richards, 1984a, 1984b; Lindsay & Norman, 1977 Lindsay & Norman, 1977; Commons & Rodriguez, 1990, 1993 Commons & Rodriguez, 1990, 1993). The Model of Hierarchical Complexity is not based on the assessment of domain specific information, but instead on the analysis of the complexity of the participant’s attempted solution to a task of a specific complexity. That is, the participant’s successful performance on a task of a given order of complexity represents the stage of development achieved by that participant. The stage score is derived from the evaluation of performances on tasks.

The MHC has a broad range of applicability. The mathematical foundation of the model makes it an excellent research tool to be used by anyone examining performance that is organized into stages. It is designed simply to assess development based on the level of complexity which the individual utilizes to organize information. The MHC offers a singular mathematical method of measuring stages in any domain because the tasks presented can contain any kind of information. The model thus allows for a standard quantitative analysis of developmental complexity in any cultural setting. Other advantages of this model include its avoidance of mentalistic or contextual explanations, as well as its use of purely quantitative principles which are universally applicable in any context. Cross-cultural developmentalists and animal developmentalists; evolutionary psychologists, organizational psychologists, and developmental political psychologists; learning theorists, perception researchers, and history of science historians; as well as educators, therapists, and anthropologists can use the MHC to quantitatively assess developmental stages.

The MHC and Skill Theory have ordered problem-solving tasks of various kinds, including:

Social perspective-taking ( Commons & Rodriguez, 1990; 1993) Commons & Rodriguez, 1990; 1993)

Informed consent ( Commons & Rodriguez, 1990 Commons & Rodriguez, 1990, 1993 Commons & Rodriguez, 1990, 1993).

Attachment and Loss ( Commons & Rodriguez, 1990, 1993 Commons, 1991; Miller & Lee, 1998) Miller & Lee, 1998)

Workplace organization ( Bowman, 1996a; 1996b Bowman, 1996a; 1996b)

Workplace culture ( Commons, Krause, Fayer, & Meaney, 1993)

Political development ( Sonnert & Commons, 1994 Sonnert & Commons, 1994)

Leadership before and after crises ( Oliver, 2004 Oliver, 2004)

Honesty and Kindness ( Lamborn, Fischer & Pipp, 1994) Lamborn, Fischer & Pipp, 1994)

Relationships ( Cheryl Armon, 1984a, 1984b Cheryl Armon, 1984a, 1984b)

Good Work ( Cheryl Armon, 1993 Cheryl Armon, 1993)

Good Education ( Dawson, 1998 Dawson, 1998)

Good interpersonal ( Armon, 1990 Armon, 1990)

Views of the “good life” (Armon, 1984c; Danaher, 1993; Dawson, 2000; Lam, 1994 Lam, 1994 Lam, 1994)

Evaluative reasoning ( Dawson, 1998 Dawson, 1998)

Epistemology ( Kitchener & King, 1990 Kitchener & King, 1990; Kitchener & Fischer, 1990)

Moral Judgment ( Armon & Dawson Armon & Dawson, 1997 Armon & Dawson, 1997 Armon & Dawson, 1997; Dawson, 2000 Dawson, 2000)

Language stages ( Commons, et. al., 2004 Commons, et. al., 2004)

Writing ( DeVos & Commons, unpublished manuscript)

Algebra (Commons, in preparation)

Music (Beethoven) ( Funk, 1990 Funk, 1990)

Physics tasks ( Inhelder & Piaget, 1958 Inhelder & Piaget, 1958)

Four Story problem ( Commons, Richards & Kuhn, 1982 Sonnert & Commons, 1994 Commons, Richards & Kuhn, 1982; Sonnert & Commons, 1994 Kallio & Helkama, 1991)

Balance beam and pendulum ( Commons, Goodheart, & Bresette, 1995) Commons, Goodheart, & Bresette, 1995)

Spirituality ( Miller & Cook-Greuter, 2000 Miller & Cook-Greuter, 2000)

Atheism (Nicholas Commons-Miller, in preparation)

Animal stages ( Commons and Miller, in press Commons and Miller, in press)

Contingencies of reinforcement (Commons, in preparation)

Hominid Empathy ( Commons & Wolfsont, 2002 Commons & Wolfsont, 2002)

Hominid Tools Making ( Commons & Miller 2004 Commons & Miller 2004)

Counselor stages ( Lovell, 2004 Lovell, 2004)

Loevinger’s Sentence Completion task ( Cook-Greuter, 1990 Cook-Greuter, 1990)

Informed consent ( Commons, Rodriguez, Cyr, Gutheil et. al., in preparation)

Report patient’s prior crimes ( Commons, Lee, Gutheil, et. al., 1995) Commons, Lee, Gutheil, et. al., 1995)

Orienteering ( Commons, in preparation Commons, in preparation)

Most of the earlier scoring schemes have not presented ways of assessing to what extent the quality of a participant’s performance on a task should influence the stage score independent of the content of the participant’s discussion. To remedy this problem, the MHC presents a framework that quantifies the order of hierarchical complexity of a task based on mathematical principles of how the information is organized, not what information is presented. The hierarchical complexity of a task to be solved is determined by the mathematical analysis of task demands. The order of performance on the task, or the stage, is also derived by analyzing the mathematical complexity of successful performance, not merely by observing what the participant does or says. The scores of the MHC indicate the stage achieved by the participant as indicated by his ability to successfully meet task demands of varying degrees of complexity. Results are not subjectively weighted based on the considerations of culture or the environment.

Hierarchical complexity refers to the mathematical complexity of the task presented to the participant, but not directly to the complexity of the participant’s performance that will successfully complete the given task. Every task contains a multitude of subtasks ( Overton, 1990 Overton, 1990). When the subtasks are carried out by the participant in a required order, the task in question is successfully completed. Therefore, the model asserts that all tasks fit in some sequence of tasks, making it possible to precisely determine the hierarchical order of task complexity. Tasks vary in complexity in two ways: either as horizontal (involving classical information); or as vertical (involving hierarchical information).

Horizontal (Classical Information) Complexity

Classical information describes the number of “yes-no” questions it takes to do a task. For example, if one asked a person across the room whether a penny came up heads when they flipped it, their saying “heads” would transmit 1 bit of “horizontal” information. If there were 2 pennies, one would have to ask at least two questions, one about each penny. Hence, each additional 1-bit question would add another bit. Let us say they had a four-faced top with the faces numbered 1, 2, 3, or 4. Instead of spinning it, they tossed it against a backboard as one does with dice in a game. Again, there would be 2 bits. One could ask them whether the face had an even number. If it did, one would then ask if it were a 2. Horizontal complexity, then, is the sum of bits required by just such tasks as this.

Vertical (Hierarchical) Complexity

Specifically, hierarchical complexity refers to the number of recursions that the co-ordinating actions must perform on a set of primary elements. Actions at a higher order of hierarchical complexity: a) are defined in terms of actions at the next lower order of hierarchical complexity; b) organize and transform the lower-order actions; c) produce organizations of lower-order actions that are new and not arbitrary, and cannot be accomplished by those lower-order actions alone. Once these conditions have been met, we say the higher-order action co-ordinates the actions of the next lower order. Stage of performance is defined as the highest-order hierarchical complexity of the task solved. Using Adey, in press Rasch (1980) analysis, Commons, Goodheart, and Dawson (1995; 1997) found that hierarchical complexity of a given task predicts stage of a performance, the correlation being r = .92 (hierarchical complexity of the task that is completed).

The nonarbitrary organization of several lower order actions constitutes one action of a higher order of complexity. For example, completing the entire operation 3 x (4 + 1) constitutes a task requiring the distributive act. That act non-arbitrarily orders adding and multiplying to coordinate them. The distributive act is therefore one order more hierarchically complex than the acts of adding and multiplying alone and it indicates the singular proper sequence of the simpler actions. Although someone who simply adds can arrive at the same answer, people who can do both display a greater freedom of mental functioning. Therefore, the order of complexity of the task is determined through analyzing the demands of each task by breaking it down into its constituent parts. the hierarchical complexity of any complex task is thus mathematically determined The participant is scored at the stage this complexity when he successfully completes the task using the integrated approach of coordinated combination of lower order actions.

The hierarchical complexity of a task refers to the number of concatenation operations it contains. An order-three task has three concatenations operations. A task of order three operates on a task of order two and a task of order two operates on a task of order one (a simple task).

Tasks are also quantal in nature. They are either completed correctly or not completed at all. There is no intermediate state. For this reason, the General Stage Model characterizes all stages as hard and distinct. The orders of hierarchical complexity are stepped like the rings around the nucleus. Each task difficulty has an order of hierarchical complexity required to complete it correctly. Once again, since tasks of a given order of hierarchical complexity require actions of a given order of hierarchical complexity, the stage of the participant’s performance is equivalent to the order of complexity of the successfully completed task. The quantal feature of tasks is thus particularly instrumental in stage assessment because the scores obtained for stages are likewise discrete.

Hierarchical complexity of actions refers to the number of recursions that the coordinating actions must perform on a set of primary elements. Like tasks, actions at a higher order of hierarchical complexity:

1) are defined in terms of the actions at the next lower order of hierarchical complexity;

2) organize and transform the lower order actions;

3) produce organizations of lower order actions that are new and not arbitrary, and cannot be accomplished by those lower order actions alone.

The hierarchical complexity of tasks and the actions they require to be successfully completed provide the mathematical foundation for deriving scores for stages of reasoning. The MHC, however, does not dismiss the influences of the environment on one’s reasoning stage development, it simply does not quantify contextual variables during the scoring process as do other scoring manuals which are designed to measure stages in a particular domain of information and may give more weight to the overall score if particular issues are addressed by participants, regardless of the manner in which the references are made.

Stages

The notion of stages is fundamental in the description of human, organismic, and machine evolution. Previously it has been defined in some ad hoc ways; here we describe it formally in terms of the model of hierarchical complexity. Given a collection of actions A and a participant S performing A, the stage of performance of S on A is the highest order of the actions in A completed successfully, i.e., it is

stage(S, A) = max {h(A) | A ∈ A and A completed successfully by S}.

Thus, the notion of stage is discontinuous, having the same gaps as the orders of hierarchical complexity. This is in agreement with previous definitions (Commons et al, 1998; Commons & Miller, 2001, Commons & Pekker, submitted).

Relationship between Piaget and Commons notions

There are some common elements between Piaget and Commons notions of stage and many more that are different. In both one finds:

1. Higher order actions defined in terms of lower order actions

This forces the hierarchical nature of the relations and makes the higher order tasks include the lower ones

2. Higher order of complexity actions organize those lower order actions

This makes them more powerful

What Commons et al have added includes:

3. Higher order of complexity actions organize those lower order actions in an non-arbitrary way

This makes it possible for the organization to meet real world requirements, including the empirical and analytic

4. Task and performance are separated

5. All tasks have an order of hierarchical complexity

6. There is only one sequence of orders of hierarchical complexity.

Hence, there is structure of the whole for ideal task actions

7. All orders of hierarchical complexity are equally spaced

8. There are gaps between the orders of hierarchical complexity

9. Stage is most hierarchically complex task solved.

10. There are gaps in Rasch Scaled Stage of Performance

11. Rasch Scaled Stage of Performance are also equally spaced

12. Performance stage is different task area to task area

There is no structure of the whole – horizontal decaláge for performance.

It is not inconsistency in thinking within a developmental stage.

Decaláge is the normal modal state of affairs

Stages of Development

The MHC specifies 14 order of hierarchical complexity and their corresponding stages, showing that each of Piaget’s substages, in fact, are hard stages. Commons also adds three postformal stages. The sequence is as follows: (0) computory, (1) sensory & motor, (2) circular sensory-motor, (3) sensory-motor, (4) nominal, (5) sentential, (6) preoperational, (7) primary, (8) concrete, (9) abstract, (10) formal, (11) systematic, (12) metasystematic, (13) paradigmatic, and (14) cross-paradigmatic. The first four stages (0-3) correspond to Piaget’s sensorimotor stage at which infants and very young children perform. The sentential stage was added at Fischer’s suggestion. Adolescents and adults can perform at any of the subsequent stages. MHC stages 4-6 correspond to Piaget’s pre-operational stage; 6-8 correspond to his concrete operational stage; and 9-11 correspond to his formal operational stage. The three highest stages in the MHC are not represented in Piaget’s model. Because MHC stages are conceptualized in terms of the hierarchical complexity of tasks rather than in terms of mental representations (as are Piaget’s stages), the highest stage represents successful performances on the most hierarchically complex tasks rather than intellectual maturity.

Stages 0-5 normally develop during infancy and early childhood in people.

At the calculatory stage (0), machines can do simple arithmetic on 0s and 1s.

At the sensory and motor stage (1), infants may see or touch shapes, make generalized discriminations, as well as babbling vocalizations.

At the circular sensory and motor stage (2), reaching and grasping actions occurs. These actions generate simple gestures.

At the sensory-motor stage (3), the actions become associated with vocalizations. For instance, an infant may hold up an object and make sounds while doing so.

At the nominal stage (4), first single words are formed. These words such as “cup” or “water” relate concepts to others.

At the sentential stage (5), toddlers form short sentences and phrases. The use pronouns, and say numbers and letters in order as well. Sentences might be “want water,” “cup of water,” etc.

At the preoperational stage (6), these sentences are organized into paragraph long utterances.

At the primary stage (7), these paragraph long utterances are organized into stories which may be matched to reality.

At the concrete stage (8), two primary stage operations may be co-ordinated. For example, children think that a deal is fair after looking at from the perspective of simple outcomes for each person who is entering the deal. Negotiations make sense but there are not social norms for setting prices or values.

At the abstract stage (9), variables, stereotypes, personalities, traits, etc are introduced. Quantification words like “everyone in my group,” What would other’s think?” appear. The dimensionalized qualities may be used to express preferences.

At the formal operational stage (10), discussions are logical and empirical support is logical brought. Words like “if ...then,” “in every case, it turned out the same,” “the reasons were” occur.

Few individuals perform at stages above formal operations. More complex behaviors characterize multiple system models ( Adey, in press Kallio, 1995; Adey, in press Kallio & Helkama, 1991). Some adults are said to develop alternative to, and perspectives on, formal operations. They use formal operations within a “higher” system of operations and transcend the limitations of formal operations. In any case, these are all ways in which these theories argue and present converging evidence that adults are using forms of reasoning that are move complex than formal operations.

At the systematic stage (11), the new concepts are referred to as 3rd order abstractions. These coordinate elements of abstract systems. Words like bureaucratic, capitalist, functional, and structural are common. The systematic stage concept, structure, for example, can be employed to ask whether the structure of camp helps instill the qualities we want in future citizens. The logical structure of this stage coordinates multiple aspects of two or more abstractions, as in: “relationships are built on trust and though we can’t always keep them, making promises is one way we build trust, so it’s generally better to make promises than not to make them.” Here, the importance of trust to relationships, building trust, and the possibility that promises can be broken, are all taken into account while formulating the conclusion that promises are desirable.

At the metasystematic stage (12), the new concepts are referred to as 1st order principles. These coordinate formal systems. Words like autonomy, parallelism, heteronomy, and proportionality are common. The metasystematic stage concept of parallelism, for example, can be employed to compare the structures of the military and of camp as institutions. The logical structure of this stage identifies one aspect of a principle or an axiom that coordinates several systems, as in: “contracts and promises are articulations of a unique human quality, mutual trust, which coordinates human relations.” Here, contracts and promises are seen as the instantiation of a broader principle coordinating human interactions.

At the paradigmatic stage (13), people create new fields out of multiple metasystems. The objects of paradigmatic acts are metasystems. When there are metasystems that are incomplete and adding to them would create inconsistences, quite often a new paradigm is developed. Usually, the paradigm develops out of a recognition of a poorly understood phenomenon. The actions in paradigmatic thought form new paradigms from supersystems (metasystems).

Paradigmatic actions often affect fields of knowledge that appear unrelated to the original field of the thinkers. Individuals reasoning at the paradigmatic order have to see the relationship between very large and often disparate bodies of knowledge, and co-ordinate the metasystematic supersystems. Paradigmatic action requires a tremendous degree of decentration. One has to transcend tradition and recognize one's actions as distinct and possible troubling to those in one's environment. But at the same time one has to understand that the laws of nature operate both on oneself and one’s environment—a unity. This suggests that learning in one realm can be generalized to others.

At the cross-paradigmatic, paradigms and coordinated. This is the fourth postformal stage. Cross-paradigmatic actions integrate paradigms into a new field or profoundly transform an old one. A field contains more than one paradigm and cannot be reduced to a single paradigm. One might ask whether all interdisciplinary studies are therefore cross-paradigmatic? Is psycho biology cross-paradigmatic? The answer to both questions is ‘no’. Such interdisciplinary studies might create new paradigms, such as psychophysics, but not new fields.

This order has not been examined in much detail because there are very few people who can solve tasks of this complexity. It may also take a certain amount of time and perspective to realize that behavior or findings were cross-paradigmatic. All that can be done at this time is to identify and analyze historical examples.

Several tables are provided to help the reader better understand the concepts of stages as defined by the MHC. Tables 1, 4a, and 4b present the definitions of stages with examples of task demands of respective complexities. Table 1 particularly explains how behavior may form classes and how stimuli may be place into classes both functionally and analytically. This table gives examples of behaviors as they may be observed, and each stage behaviors is also broken down into substeps, showing the organizing functions of varying complexities. Table 1 is especially useful for scoring behaviors up to stage 12.

Table 2 provides examples of the kinds of vocal remarks made by various family members performing at specified stages. This table is a particularly useful tool for evaluating stages among related individuals which may be helpful in studies examining development and controlling for hereditary factors. In addition, in many developing nations families tend to play particularly important roles in lives of individuals, and increases in complexity might be particularly evident if the content of the assessment deals with topic of family. Family affairs constitute a practical source of universally relevant content that could be used to evaluate stages in any cross-cultural setting. Familiarity with table 2, therefore, is particularly useful for researchers.

Table 3 exemplifies various verbal relationships formed by vocalizations characterizing different stages. Table 3 is especially helpful since it shows some key grammatical structures associated with each stage, as well as the key phrases used by people performing at given stages. Though an in-depth mathematical analysis is necessary for an accurate assessment of stage, familiarity with Table 3 will assist the scorer in initial classification of behavior.

Tables 4a and 4b describe the orders of hierarchy and sequence of stages, respectively. That is, the table elaborates the nonarbitrary coordination process underlying MHC stages. This table explains how concepts are constructed and vocalized at each stage with increasing complexity. These table also clearly show how each subsequent discriminations is vocalized at the subsequent verbal relationship is formed. Understanding the examples provided in Tables 4a and 4b enables the researcher to classify stage based on complexity of vocalizations and the verbal relationships they form.

The new concepts formed at each stage can be viewed as “summaries” of the constructions of previous stages. Although the MHC proposes no mental model to explain this process ( Halford, 1999 Halford, 1999), suggests that this summarizing or “chunking” makes advanced forms of thought possible by reducing the number of elements that must be simultaneously coordinated to produce an argument or a conceptualization at a higher order of hierarchical complexity. Interestingly, at the preoperational, abstract, and metasystematic stages of the MHC the new concepts not only coordinate or modify constructions from the previous stage, but they are also qualitatively distinct conceptual forms: representations, abstractions, and principles, respectively. The appearance of each of these conceptual forms ushers in three repeating logical forms: definitional, linear, and multi variate. Other researchers have confirmed these distinct conceptual forms and repeating logical structures ( Fischer, 1980 Fischer, 1980; Fischer & Kenny 1986 Fischer & Kenny 1986; Fischer & Lazerson, 1984 Fischer & Lazerson, 1984).

External Influences

Psychological, sociological, and anthropological data address why the participant’s performance develops in a given manner. However, why development takes place is linked to how participants can demonstrate the stage of development. The successful completion of a task requires an ideal action of a given order of hierarchical complexity which had developed as a result of influences by psychological and sociological variables. For example, Table 5 shows how stimulus control with or without support can change the relative difficulty of a task. The level of support during task completion, therefore, changes the scored order of performance. Other models have often used the participant’s reference to an informational set as an index of stage development without considering such variable as the level of support. We believe that this approach is oversimplified. Accurate, consistent results could only be obtained when the system of evaluation is based on a universally applicable groundwork, such as the mathematical foundation of the MHC. According to the MHC, the participant’s approach to a given task is quantified to produce a score for the stage of reasoning in any domain. Inferences regarding the factors influencing the performance can be made independent of obtaining the stage scores.

The Model of Hierarchical Complexity posits that individual’s perceptions of the world (and the stimuli in it) are influenced by frameworks. These frameworks embody the individual’s conditioning history, including cultural, educational, religious, political, and social backgrounds, among other factors. These combined frameworks are referred to as one’s perspective. Perspectives differ in terms of hierarchical complexity. As the hierarchical complexity of an individuals’s response to task demands increases (i.e., as complexity of performance goes up), the individual is increasingly likely to have taken many such perspectives into account ( Commons & Rodriguez, 1990 Commons & Rodriguez, 1990).

There are task demands that certain professions require of individuals. Although the job demands of a secretary may not exceed formal stage of complexity, those of managers or judges often require development beyond the systematic stage. Tables 9 and 10 provide examples of types of social organizations and professional settings which require development to various stages of complexity.

Conceptualizing Stages

Each of Piaget’s stages is defined by a set of formal properties that constitute a structure d’ensemble, or a structure of the whole. This has sometimes been taken to mean that the entire knowledge system forms a single unified global structure ( Fischer, 1980 Fischer, 1980). In some interpretations of stage transitions based on the notion of structure d’ensemble, development is characterized by abrupt global reorganizations of the knowledge system which is modeled as a single staircase. However, studies of performances on various tasks do not provide evidence for this type of a global structuring of knowledge. Instead, assessment models such as the MHC posit that several analogous structures of knowledge exist, however, they do not appear to develop in parallel. This is especially true of analogous structures in different knowledge domains ( Fischer, 1980 Fischer, 1980; Fischer & Kenny 1986 Fischer & Kenny 1986; Fischer & Lazerson, 1984 Fischer & Lazerson, 1984). In fact, there is no evidence whatsoever supporting a single, global, stepwise pattern of development. Instead, it has been argued that the cognitive system can best be conceptualized as a set of interrelated dynamic knowledge systems ( Fischer, 1980; 1986; 1984 Fischer, 1980; Fischer & Kenny, 1986; Fischer & Lazerson,1984; van Geert, 1991 van Geert, 1991), each developing in a hierarchical manner. As discussed earlier, the MHC does not propose a direct link between mental processes and performance. However, inferences can be made about mental processes on the basis of patterns of performance, and these inferences can inform research into mental functioning when cultural and other factors influencing performance are evaluated along with the actual actions.-

Task Theory

To further elucidate the concepts involved in quantifying task demands as the basis for generating stage scores we present an overview of how tasks are constructed. Task demands increase along a continuum of complexity: from preoperational, concrete, and abstract, to formal, systematic, and metasystematic.

Series of tasks in different domains

Each task can only be correctly addressed at a given point in development. If the successful completion of the task requires a higher stage then one at which the person is performing, the scored stage will be lower than if the task demands actions at the reasoning stage the participant has already achieved. Fischer has shown that presenting a task that is above the participant’s stage of performance depressed the performance index below the actual stage for reasons related to emotional development ( Fischer, 1980 Fischer, 1980; Fischer & Kenny, 1986 Fischer & Kenny, 1986; Fischer & Lazerson, 1984 Fischer & Lazerson, 1984), to be additionally discussed in section called Stage Transition. Therefore, using only a stage task that’s too demanding may result in underscoring performance. Presenting a task demanding the response that the participant can actually display is a more accurate method of assessment. At the outset of the study, this stage is hard to predict. The most efficient way to assess stage, therefore, is to administer several tasks of varying complexity for the participant to attempt, including tasks of low orders of complexity. The completed task of the highest order of hierarchical complexity of all the tasks presented would most accurately represent the actual stage of the participants’ reasoning. In other words, the Model of Hierarchical Complexity not only does not focus on any particular domain of knowledge for reasoning stage assessment, but it also recommends that several tasks from multiple domains are presented in order to obtain the most accurate results. The stage scores may differ in each domain depending upon the mathematical complexity of performance.

Dimensions of tasks

Tasks are comprised of three basic dimensions: action, description or reflection upon that action ( King & Kitchener, 1989 King & et al., 1989; Tappan, 1990 Tappan, 1990), and the number of element that a person can work with at a given time which are required to perform that action and to report on it. The theory underlying the development of tasks is that different tasks require different levels or values of each of the three dimensions. The values of each dimension are important in assessing the stage at which a person is able to successfully execute a task. Often, these three dimensions are ignored and only one measurement, stage of action, is specified. This oversimplified process does not yield comparable measures of stage across tasks because the scoring is based on different values in one or more of the other three dimensions. In other words, the action demands of executing a certain task in one domain may differ from the action demands of executing a task in another domain. The same would be true for the demands of reflection required in performing a task in one domain versus another; and for the amount of memory required to execute a task in one domain versus another. The MHC is primarily concerned with the first dimension of task, the action dimension, because it interprets the stage of reasoning to correspond other stage of performance. However, the stages may differ in different domains because task demands also differ.

In making comparable stage interpretations across tasks, each of the values in the dimensions of action, reflection and memory should be specified. In other words, when discussing stage one must be specific about the reference to the dimensions of action, reflection, and memory.

I. Dimension 1: Action

The dimension of action consists of a number of requirements for a series of concatenated actions to form a stage hierarchy of actions. The chain of steps may not be rearranged. If doing the action was at the sensory motor stage¹, reporting on the action would be at the nominal stage, reporting on why one chooses that particular action would be at preoperational stage, and justifying those justifications would be at the primary stage. That is, more complex tasks and actions coordinate lower order tasks and actions in a nonarbitrary fashion, yielding the process to quantitative analysis.

For example, children might be told to put their toys into the toy box. Putting toys into the toy box is an action that a sensory motor child might perform. At the nominal stage they might say "Put toys," or "Put toys away." Preoperational children might say, "We are putting the toys away so we can get some cookies." Primary operational children might justify putting the toys away by saying, "We must put the toys away now before we do the next thing because that is the rules."

The order of hierarchical complexity of tasks composed of subtasks is easily determined. When the tasks are from the same domain, if one task operates on the other, the order of complexity increases. The same is true across domains. When tasks from different domains are added to one another to form a new task, the number of required concatenations of actions also add. This assumes that stage requirements form an interval scale. The order of hierarchical complexity required by a task is written as o. Hence, for stage:

o = the order n requirements - order n - 1 requirements where the order n requirements is the order of hierarchal complexity required by the task.

Hence for stage:

stage n requirements - stage n - 1 requirements =

stage n - k - stage n - k - 1

The predicted stage required by a task is written as o.

o = order n - k - order n - k - 1

This assumption also holds for describing action and reflecting upon the description.

Dimension 2: Reflection

The dimension of reflection on action consists of the following steps:

1. Doing the action

2. Reporting on doing the action (shadowing)

3. Reporting on why one chooses that particular action

4. Reporting on why that justification is good

5. Reporting on why that system of justifications is good

Each step requires the previous step. The question is whether the fact that each step requires a previous step represents a change of stage.

Dimension 3: Memory

Remembering an action in order to reflect upon it requires non-structural actions that increase the task difficulty. Little children can describe what they are doing before they can describe what they have done earlier ( Piaget, 1976 Piaget, 1976 Piaget, 1976; Karmiloff-Smith, 1986 Karmiloff-Smith, 1986 Karmiloff-Smith, 1986) although their exact report of what they have done may differ from what they actually did. Karmiloff-Smith clearly explains that there are mechanisms of thought in operation before the child comes to be able to report on those actions. Recalling previous actions may or may not require an extra stage depending on how the recall is triggered. For example, if the recall is in the sensory-motor stage as is the remembering of a comfortable sleeping position in order to attain that level of comfort again, one stage is not required to solve the problem, one simply moves around until that position is again attained. Yet, the explanation of what that position is requires additional stages. If the recall depends upon having a sense of time (i.e., recalling something ordered by time) it might require the attainment of at least one stage of development.

A given developmental stage represents a measure of successful performance on tasks of the same order of hierarchical complexity. The General Stage Model ( Commons & Richards, 1984a, 1984b Commons & Richards, 1984a, 1984b; Commons & Rodriguez, 1990; Commons & Rodriguez, 1990; Commons & Rodriguez, 1993 Commons & Rodriguez, 1993) defines the stage in terms of task performance. When people successfully perform tasks of a given order of hierarchical complexity, they are performing at the stage of the equivalent order. However, the dimensions of reflection and memory also influence the performance or action and are shaped by the developmental environment of the individual. The MHC incorporates ideas about how task performance develops and how transition progresses from one stage to the next.

Dimension 4 Familiarity

Task can vary in their important both between different cultures and within cultures. Individuals may have more interest or training in certain tasks. Familiarly affects the difficulty of tasks. With practice, support and reinforcement, the effects of Familiarly maybe wiped out.

Dimension 5 Placement of Key Information within tasks

Information place at the beginning or end of tasks are more easily remember and sometimes seen.

Dimension 6 Degree of symbolization provided

Surprisingly, mathematical problems are the easiest in educated populations because they come in a compact symbolized form. That form requires a minimum of coding by the participants

Dimension 7 Level of Support

Stage Transition

Measuring transition is extremely important. Many interventions do not produce a change of a complete stage. Some population only vary between transition-to-the-next stage and the next stage such as professors at research universities who study ethics.

There are two forms of stage transition. One is transition steps. These steps represent how two lower stage behaviors alternate increasingly rapidly. At step -1 or 4, the same stage behavior there is no alternation. The rate of alternation is 0. Then at step 0, which is getting off the dime when the present stage is seen to fail, to using an alternative same stage action, to alternating such an action with the previously more used action to finally smashing elements of both previous stage actions together – an infinitely fast alternation rate.

The second form of transition if the proportion of current and next stage action as Theo Dawson shows using Rasch analysis. If one looks at where the person scores fall, they fall between the stage marked by item those two stages or scores for those two stages.

In order to understand how the dimension of performance increases in hierarchical complexity we must examine the factors implicated in driving stage transition. That is, we must examine the various contingencies that promote the development of performance at higher reasoning stages which is only possible when the dimensions of reflection and memory coordinately increase in complexity along with the dimension of action. There are a large number of such contingencies. They include but are not limited to providing reinforcement or support for next stage behavior, showing contradiction for present stage behavior, exposing people to models of next stage behavior and the reinforcement that such behaviors attain. Here we also consider emotional and various environmental factors that shape the individual’s transition from one stage to the next.

Every participant’s behavior could be categorized to a transition step between stages. Varying factors such as the impact of emotions, personality, and environment, etc influence how long someone may stay at each step-. Most people only traverse up to 12 stages by the age of 24. Evidence shows people may transition every two years at most, sometimes even less. The only time when fast transitions occur is perhaps during infancy. Again, the participant’s performance on a task can only be scored at a given stage of complexity when the task of a corresponding order of complexity is successfully completed. Table 7, for example, focuses on four types of personalities associated with various transition substeps. Adults are simply not meant to “get stuck” at these substeps, and the examples provided are often associated with psychological or personality disorders. This knowledge of transitional mechanisms underlying development is a great asset for therapists and psychiatrists to possess because it could be so useful in diagnosing patients. Because reinforcement moves people along the substeps toward the successful achievement of stage, using various modifiers of reinforcement would help clinicians treat patients. The crucial insights of the MHC, therefore, are clearly applicable not only in research, but in medical practice as well.

When one successfully completes a task of a given order of hierarchical complexity, one is performing at that stage of complexity. Therefore, static coping is what occurs when one is not required to perform above one’s characteristic stage of performance. Often one must meet or solve other problems successfully, or assume additional perspectives and skills in order to change stages. In those cases, dynamic coping occurs during stage transition and it involves several steps. During steps 0-2, deconstruction of previous stage behavior occurs (e.g. Swan & Benack, 2002 Swan & Benack, 2002), during steps 3-4, new stage behavior is constructed.

At the beginning of each transition the perceived rate of reinforcement drops. The more one confronts failure, the more one might expect avoidance. In fact, Commons, Grotzer, and Davidson (in preparation) found that feedback alone in higher stage tasks led to a decrease in stage of performance, rather than an increase. Perhaps its defensive behavior, with is fear accompanying transition through the steps, decreases the stage of performance. Another explanation could be that one does not see a stage of performance higher than one’s own in others and this impedes learning through support. Please refer to Table 5 which elaborates the role of support in stage change. In any case, it is important to note that emotions are usually associated with transition of stages.

Transition steps

At step 0, the demands for performance beyond the final step of the last stage are perceived. Without changing performance from step 4 of the previous stage, there is a perceived reduction of reinforcement for task performance. This characterizes step 0. A person feels stupid and upset, sometimes even angry, while failing to fulfil a task. One may also feel elated about task mastery of the previous stages tasks.

At step 1, the person feels dejection in addition to the previous feeling of sadness (or anger). In both of these first transitional steps, one may want to “give it all up” and forget about it all. These are defense mechanists, ways of switching the point and rejecting frustration.

At step 2, relativism becomes the key concept. One sees the possibility of solving a problem but does not necessarily know the right means of doing it. Someone can be seen as competent for a special task, but not to any task. Relativism has to do with contexts, and because contextualization is a sort of concretizing, it is an attempt to cope with each better way. But concretizing is not the same as coordinating. One just knows there is a way of comparing situations and means, but not how to do so. Keep in mind that actions of the full higher order of hierarchal complexity not only put together actions of the lower order, but organize them in a non-arbitrary fashion. Random contextualization, therefore, is characteristic of a transitional step from one stage of performance to another.

Table 6a explains the steps involved in stage deconstruction, also providing the logical scheme underlying this process. The substeps organizing each deconstruction step are provided in dialectical form, even though the organization is based on mathematical laws. The construction of each step out of substeps is written out to facilitate the readers’s understanding of the very mathematical notions involved in organizing complexity.

At this point during transition, between the deconstruction and construction steps, one may feel conflicted, anxious and not sure of anything, because the individual does not perceive any control over the situation. People may ask themselves whether they are independent or dependent, but they most likely can not find an answer. Who is the one that really holds the reins? One might enjoy the excitement of the uncertainty, such as s tourist feels upon visiting a strange land and experiencing other cultures for the first time. One might defend the relativism as a necessary reality and feel that it justifies one’s behavior.

At step 3, the first step in constructing new stage behavior, people begin to show more creativity in handling problems. There are several conditioning substeps comprising this step:

a) The first substep is described as "getting chaotic". One simply tries anything to get going. What is often done is just smashing (or lumping) of all the existing systems of acting together without any formal integration. Smashing has an aggressive and desperate tone characterizing attempts to "survive"– i.e. building a life raft out of anything. On the first substep, people feel somewhat manic as part of a normal process.

b) The second substep is the "learning what to do” substep. Templates are formed that are inclusive. The instance of the relationship at the target stage will be detected and used. This second substep brings with it a beginning in producing correct results. One is not able to eliminate those acts that do not bring good solutions, but the right direction is at hand. The most common feelings experienced at this point are excitement and a sense of frustration because of making errors.

c) The third conditioning substep is "learning when and where to do" each subset of action. People know what to do but not when to do it. They may feel uncomfortable and confused, but not helpless. One knows what to do, but not when. On the other hand, people who do not know what to do, may have a feeling of deep incompetence and helplessness. When people feel both confused and helpless, they have no sense of power nor the ability to act progressively. One learns to eliminate over generalization errors. Everything has to be compulsively cleaned up. One may be obsessive, fussy, and "sticking." Templates constructed here exclude rather than include. There is reconstruction. One is just not meant to get stuck here.

During the final step 4 which completes the construction of new stage behavior, inclusion and exclusion templates are finally coordinated. One feels glorious for combining right elements successfully. A post-reinforcement pause may follow. At this step the closure makes one feel personally satisfied. As Rosenberg (1979 Rosenberg points out, how this momentary stability is perceived will effect how one feels socially ( Rosenberg, 1979 Rosenberg, 1979). Quite often the demands for further development occur. This affects how long such positive feeling persist.

When entering a score into an analysis, we use the following:

Stage	Stage #
Concrete	8
Abstract	9
Formal	10
Systematic	11
Metasystematic	12

To the previous stage, we add the following for the transition steps.

Step	Points
0	.2
1	.4
2	.6
3	.8
4	1.0

For example, a performance transitional to metasystematic at step 2 would be 11- 2 or 11.6 points. Performing fully at the Metasystematic stage is therefore 11-4 or 12= 11 + 1 points

Table 6b explains the deconstruction steps, also elaborating the logical scheme involved. Like in Table 6a, the substeps are clearly written out to clarify the mathematical principles of organization.

Reinforcement moves everyone along the substeps according the melioration law ( Herrnstein, 1982 Herrnstein, 1982, Herrnstein & Vaughan, 1980 Herrnstein & Vaughan, 1980), which dictates that behavior progresses at a rate proportional to an increase in reinforcement. Increases in hits increase the likelihood of making hits, which reinforces the generalization. Decreases in over generalization also increase reinforcement. This law also explains how and why the lack of satisfaction reinforcement that occurs when tasks are completed which are below the actual stage of the individual-- underestimate the stage since the individual may actually perform at a lower stage due to the lack of such reinforcement. Indeed, as explained in Table 7, emotional states and personality types affect stage transition, and this factor must be considered when formulating a complete conception of the processes involved in stage development.

Knowing how stage transition occurs is important in the proper application of the quantitative methods of the MHC. Since stage is assessed from performance, the best performance must be elicited properly. The failure of the researchers to administer the tasks so as to provide an adequate environment for the expression of ability may result in underscoring stage. Therefore, researchers must understand the psychological and sociological variables not only of how performance on tasks develops, but also how it can be demonstrated during assessment procedures.

How to Measure Transition

Transition can be measured using four different methods:

1. Scoring interviews directly for statements that reflect transition

2. Finding the rate and acceleration of alternations of old stage and newer stage actions.

3. Finding the proportion of new stage versus old stage behavior.

4. Determining the hierarchical complexity of stimulus items (or tasks) and using a Rasch analysis to show that they form a continuous scale. The Rasch analysis scales performance and items on the same log linear line. Transitional performance is shown by the mixtures of performances at different stages. The mixtures range from 0% at the higher stage to 100%. We call 95% at a stage consolidated performance and 0% up to 95% transitional. The advantages of the Rasch analysis are that: (a) it reduces measurement variance to a minimum; (b) This yields direct comparability which is useful in assessing both the possible natural number and nature of the items and the corresponding performances ( Mislevy & Wilson, 1996 Mislevy & Wilson, 1996; Wilson, 1989 Wilson, 1989).

Acquisition of New Stage Behavior

To overcome the huge gap between the lower stage behavior and the higher-stage behavior, Piaget suggested two processes: assimilation of new behaviors and performances to the present stage; accommodation to the higher stage performance. In both cases, we argue that the laws of learning apply. Different forms of instruction produce both assimilation and accommodation. The general finding is the more solid the performance at the lower stage behaviors, the more easily the new stage behavior may be acquired.

We describe five ways of advancing stage change as discussed with respect to adult development. First is the didactic method of teaching about higher stage behaviors. Second is the Piagetian notion of immersion and the use of contradictions. There are a wide range of programs and variation on this theme (See Adey, in press Adey, in preparation; Brendel, Kolbert, & Foster, this issue Brendel, Kolbert, & Foster, in press; Lovell, this issue Lovell, 1999; McAuliffe, this issue McAuliffe, in press). Third is the use of reinforcement for correct answers and outcomes. Fourth is the use of support. Last is the use of direct instruction and charted performance as feedback.

Didactic teaching has many variants. The most common is show and tell. At the high school level and above, this is referred to as lectures. Lecturers seem to have been derived in form from sermons. The information is imparted by speaking to the multitude. Seeing films or videos, DVD’s or other electronic form of one-way media including seeing films, or listening to tapes, are all variants. Sometimes there is a lecture followed by a discussion section, which may include more detailed lectures with some possibility for questions.

A second and related form to lectures is reading material. Not surprisingly, it is more effective. It allows for self-pacing, reviewing, and highlighting. Reading is a much more active process. The order from least active to most active is as follows: show and tell, listening, and reading.

Whereas the Piagetian notion of immersion works well for children and adults who care about contradictions in academic settings, less motivated children do not change stage very readily under these conditions ( Commons & Miller, 1998 Commons & Miller, 1998). In one experiment, performing the correct task lets the children earn points. The children’s points are then pooled for different teams and then the teams are put into competition between each other. These competitions for points led 75% of fifth and six grade students to acquire formal operations on a number of Piagetian tasks.

Fischer (personal communication Fischer (personal communication Fischer (personal communication) reports that various forms of support–providing examples or prompts for what is the correct response--leads to the acceleration of the acquisition of new stage behavior. This is probably due to the fact that such types of support reduce the required task demands by one order of hierarchical complexity (see Table 5). This makes it possible to perform the higher stage task. Repeated performances at the higher stage are reinforced and therefore acquired.

Finally, fields such as Precision Teaching offer actual training of new actions. Two basic notions in Precision Teaching are elements (components) and compounds (combinations) of those element behaviors. Precision Teachers first train individuals on the elements or components, and only later on combining them. In Precision Teaching one makes decisions about the effectiveness of current instructional interactions based on charted performances. The chart shows the rate of completing tasks and compares the rates to how experts would perform. Fluency training on the element behaviors consists of getting those behaviors to occur at an extremely rapid rate. When the rate of behavior reaches a maximum, that is, it most closely matches the rate of an expert –the behavior is considered fluent. If it is learned to the extent that very little effort or special attention is required, that is, the performance is automatic. Fluency training on the elements seems to increase the speed at which compounds are acquired from elements. The implications of this work are that Precision Teaching in behavior analysis provides an empirical account of development.

The Upper Limits of Stage Transition

The discussion of stage transition may give the impression that under ideal conditions no factors in the stage transition theory necessitate an upper limit on stage. The current formulation of stages includes 14 orders of complexity, suggesting that the number of times a series of elements can be turned into a higher order combination is 14. This may, in fact, be the upper limit, at least for human beings. There have been an increasing number of empirical reports claiming that a limit exists on the number of times a series of elements can be turned into a combination. These reports can be found in training studies, which show that at a given age, there are limits to how much training is effective in bringing about change. We also know from training graduate students that no matter how much training one provides for this group, some students will never move beyond the systematic stage in their problem solving because of getting stuck somewhere in the transition, not because of an inadequate testing environment.

It is also suggested that whatever the upper limit may be for a particular individual, that ceiling is almost totally heritable. For example, there does not seem to be any variation among identical twins who have been provided with similar training. Providing additional training to both twins merely causes acceleration of transition in the slower twin, but only up to the limit achieved by the other twin, not beyond.

This theory of stage transition makes six predictions regarding the stages, all of which ( Dawson, Commons, & Wilson, in review; Dawson, Commons, & Wilson, 1999, June Dawson et al., 1999, June) have confirmed:

The Hierarchical Complexity Scoring System (HCSS) entails several steps for assessing performance on a task:

1. Transcribe the interview or the material and put it onto a disk in a file. The file should be continuous. There should be a participant number, age and sex. The interview should be recorded verbatim. The analysis follows the interview. It is done systematically.

2. Divide the interview into individual statements about an issue in a domain. In an interview, each statement is numbered. These are usually propositions. Number the propositions a_1, a₂, a₃, ... a_k etc.

3. The individual statements are coordinated to build bigger statements. For example, two abstract stage statements, a_1, a₂, may be coordinated to form one formal operational statement, f₁. Two or more formal operational statements, f₁, f₂. may be coordinated into a system, s₁. Two or more systems, s₁, s₂, may be coordinated into a metasystem, m₁. Hence, the beginning statements may appear low in stage. The overall statement that is being scored usually ends with the highest stage coordination. The overall concluding statement is used to determine overall complexity for a number of reasons:

a. The mixture of lower order items distort the score of a statement or action.

b. All higher order statements require such lower order substatement.

c. Hence, scores the most integrative statement or action because it is the only consistent way to score

Sometimes the last coordination comes when the participant is asked why something is not true, caring, fair, beautiful, important, etc. The overall statement is the series of sub-statements the ends with the highest stage coordination.

3. Statements are classified as scorable or unscorable. A scorable statement consisted of the assertion of a solution to what the participant perceived as a problem (often an interview question) and the justification for that assertion. Statements are considered unscorable if no justifications were given regardless of whether solutions were asserted.

4. Scorable statements contain either positive or negative assertions. An assertion was a positive if it affirmed some position or relationship. A assertion was a negative if it denied or rejected a position or relationship.

5. To determine the category in choice theory (hit, miss, correct rejection, or false alarm) into which a answer falls the correct and incorrect information is combined with the positive and negative assertions.

6. To determine whether the statement's conclusion is correct or incorrect for the stage of reasoning the participant used to make the conclusion,

a. the hierarchical complexity of the implicit task a participant is trying to perform was systematically abstracted;

b. the most complex task attempted in a statement was identified;

c. the participant's argumentation was compared with the criteria for the stage.

7. Whether an assertion resulted from successful or unsuccessful reasoning was distinguished in two ways, depending on whether the assertion was positive or negative. A positive assertion which affirmed a conclusion that is correct for the stage of reasoning it used was called a hit. If the conclusion was incorrect, by the criteria of the stage, the positive assertion was labeled a false alarm. A negative assertion which correctly rejected a conclusion that is false for the stage of reasoning used was a correct rejection. And if the conclusion was incorrectly rejected, by the criteria of the stage, it was called a miss.

8. The last step is to calculate an index of sensitivity and a Rasch estimate of both item (score given to a statement) difficulty and participant proclivity. These are related to the given order of hierarchical complexity of the item. The nominal traditional categories of domain, issue, and norm are also recorded.

Problems with other forms of scoring

I. Scoring manuals have been domain specific for the most part. Colby and Kohlberg's (1987) Colby and Kohlberg's (1987a, 1987b) scoring system is reliable only for those dilemmas contained in the scoring manual. The Washington Sentence Completion Test (WSCT) scoring system is reliable only for those sentence stems contained in the scoring manual. This restriction follows from the fact that the logic of interstage relationships has not been made explicit.

a. bootstrapping method of developing the scoring system

b. manuals generated by applying a particular stage theory to pilot data

c. manual consists of standard dilemmas with representative answers for each stage

d. criterion judgments for each issue in the argument are reviewed, and matched to participants' responses

e. the matching process requires scorer to be familiar with the manual and to make fine discriminations between arguments

f. matches are ultimately based on the particular conceptual content of elements employed by the participant, rather than upon the relations among these elements. Standard Issue Scoring is thus limited by being content bound.

Definition of Common Terms across Stage Models

Concatenation: In a concatenation, a coordinating action is performed on two or more elements. The products of this action then become the elements and the action is performed again on the new elements. The products of the second performance of the action are taken as elements and the action is performed again. In theory, such a concatenation may be extended indefinitely.

Domain: Domain describes a set of tasks that share certain qualities in common. Such tasks are similar in both their actions and the objects acted upon (content).

Downward assimilation: Participants sometimes give answers or solutions that are derived from a higher orders of reasoning than the one the participant uses to justify the answer. This suggests that people may be attracted to the arguments of a given stage even if they themselves are unable to generate them. The adoption of such arguments is downward assimilation. For example, there is a social domain

Interview instrument: An interview instrument is a construction such as the Heinz Dilemma. It presents a framework around which the interview can be directed.

Order of Hierarchical Complexity of Performance: A Participant's stage of performance is the order of hierarchical complexity of a successfully performed task.

Variable: A variable is defined as an element with more than one possible solution (value). Variables can be continuous (like size) or discrete (like age in years). Discrete variables can be binary (dichotomous), like clean/dirty or new/old, or they can be multivalent, like number of dependents.

Instruments of assessment

In assessing development there is always the question as to whether the tasks should be presented to participant as a series of problems or in an interview format. In either case, the task may be used to examine various issues, such as moral reasoning, social-perspective taking, attachment, causality detection, etc. The participant may deal with each issue at a different stage, depending on the order of the performance on the task connected to each issue. When the three task dimensions earlier described are uniformly taken into account, both types of the assessment instruments, the interview and problem set, yield equivalent stage results because stage is a single measure of the hierarchical complexity of the task that the participant is solving. As long as the task demands presented in an interview or in a problem set are the same, performance stage should not vary. Though the format of the task can possibly add demands not related to stage of performance, the stage scores remain unchanged because the task related to stage is what must be correctly completed. Like all answers during an interview solve some implied task, all solutions to a problem series solve an issue at a stage of a particular hierarchical complexity ( Commons, Kantrowitz and Buhlman, 1984 Commons, Kantrowitz, et al., 1984).²

Once a task has been constructed and administered, the scoring scheme specifies the relevant data that the researcher evaluates to produce a stage score. Since most scoring schemes use standard assessment tools, the implied tasks that the participant carries out at a given stage are clear to the researcher. Once the implied tasks for each stage are uncovered from analyzing participants’ responses, a direct problem with more categorical answers is constructed.

Interview answers are relatively less constrained than stimulus driven problems. Problems and dilemmas always have specific contexts and ways of assessing performances. Because interview answers are much less constrained, one would expect that the plausible responses are essentially infinite. The validly of the answers may be great because the probability of selecting a non-self-representative response is virtually nil. On the other hand, dilemma presentation risks missing the responses that a participant would choose if less constrained by the instrument of assessment. Hence, we suggest that both dilemma and response sets are used as measurement tools during the evaluations. In some cultures, on the spot social discussions may also prove useful and necessary. Keep in mind, that presenting a variety of tasks of varying orders of complexity comprise the most efficient method of accurately assessing stage.

The two forms of measurement are mirrors of one another. With fixed problems, one can study the processes of transition and acquisition of new stage behaviors, as well as the specific ways in which problems are solved. Small variations may be introduced into the fixed problems administered to the participants, by varying single aspects of interest each time an assessment tool is used.

Turning interview questions into problems

In addition, though this involves a considerable effort, researchers can turn the interview responses into problems. The main difficulty lies in delineating the implied tasks and in showing how the actions that are needed to carry out a desired sequence of tasks order the more complex stage sequence.

Pros and cons of interviews

There is a number of reasons for choosing whether or not to use interviews. Open ended interviews, for instance, create variability, delaying the operationalization of the variables that the researcher thinks are important. During such interviews the participant may or may not choose to discuss a particular topic which may be important for raising the stage score according the system based on informational references. While solving specific problems, on the other hand, the participant is more likely to obtain clearer instructions from the researcher regarding which particular topics it’s important and relevant to address. The MHC does not make this contextual distinction and any combination of open-ended questions and more specific problems may comprise an acceptable format of testing stage.

Pros and cons of problems

Problems commit the researcher to an operationalization of the issues. They often do not measure the actual proficiency of the participant, because they contain demands that are not central to the concerns of the researcher and therefore may also underestimate the stage score. As many testing manuals point out ( Anatasi, 198 Anastasi, 1982), more than one form of the problem has to be generated in order to help the researcher make appropriate evaluations. So-called warm-up effects reflect the transfer of competence from one domain to the one being examined. Unless the transfer process is also examined, initial results of single items can be very misleading ( Commons & Ducheny 1979 Commons & Ducheny 1979).

Examples of scoring

Attachment A. Male, Age 8, Stage 7 Primary-Step 3 Smash, Substep 2: Over generalization: Transition to Concrete:

Interviewer: What happened to the toy that your cousin lost?

Child: Yeah. He threw it up someplace. It must've landed in a gutter or in the streets.

Analyses: He talked about his own point of view in an earlier response. Now he has reversed and is talking about his cousin (who threw his toy “some place”). He is at least considering what his cousin did and how that affected not being able to find the toy. However there is no specific co-ordination between what the cousin did and the fact that the toy cannot be found. The substep of transition that he is showing is over generalization. He would blame his cousin for anything, so he does not have “correct rejection” strategies - just a large number of hits.

Attachment N. Male, Age 9, Stage 7 Primary-Step 3 Smash: Transitional to Concrete

Interviewer: Why weren’t you very mad when your friend moved?

Child: Because I did have a say in it, sort of. I asked them to stay, but he said Oh we’ve been planning to move for about a year.

Analyses: This child spoke about these experiences almost entirely in primary order terms. But he made three statements approaching the concrete order (of which this is one). This statement involves more than just himself. He recognizes that others have points of view, but he does not really refer to their point of view. As a result this was coded as being at the transition substep smash.

Attachment K. Female, Age 8, Stage 8 Concrete-Step 2 Relativism: Transition to abstract

[When asked whether she was afraid due to the loss of her hat:]

Child: I just wasn't afraid. Because, I don't get afraid when I lose something. But if it's something very, very special to me, really, really important and I always loved it, then I would be a little more scared and worried that I lost it. I'll never see it again.

Analysis: She seems to have two ideas: some things that are not very important do not make her get very afraid; other things that are very, very important would make her get more afraid. She seems to be beginning to deal with different values of “importance” and of “fear” and relating them to each other, but she is not doing the relating explicitly. She also seems to be thinking hypothetically; she does not have a specific thing in mind but says “If it’s something…” As far as what step of transition she is showing here, it is relativism: she has both points of view, but does not co-ordinate them, instead she alternates between them.

Attachment C. Female, Age 9, Stage 8 Concrete-Step 3 Smash: Transition to abstract

[When her cat died:]

Child: It made me feel like I had to do something because I wasn't taking it that hard and like, the other two were.

Analysis: This is a story, with specified roles: self and other family members. She is stating what sounds like a social norm, but it is not a general social norm. It is specific: because these two people are upset, she should be upset. This is transitional to abstract, because generalized social norms are abstract. The transition substep is smash because the social norm is not free, it is stuck on these two people and this situation.

Therapy Stage 10 Formal, Step 4 (0):

Participant: I play slowly enough to anticipate each upcoming section of the music.

Analysis: Formal, stage 10. An implied “if… then” relationship logically connects two abstract variables. The first variable is the speed of playing, and the second is the anticipation of upcoming sections of music.

Therapy Stage 10 Formal-Step 1 Negation: Transition to Systematic

Participant: “He sees intimacy in a different way than me.”

Analysis: This is a functional relation: “If he sees intimacy as ‘x’ then I see it as ‘y’ and vice-versa.” This is a comparison between two abstract propositions.

Therapy Stage 10 Formal-Step 3 Smash: Transition to Systematic

Participant: “Need to explore and respect each other’s wants and desires and function as a team [to build intimacy].”

Analysis: The adult has a “needs to do” list of the conditions required for building intimacy. This is multiple causation; the conditions are combined in an additive fashion at the formal stage. The social relationship, as a system that builds intimacy, is not explained as a co-ordinated system of viewpoints that balances individual with common needs or desires. “Explore and respect each other’s wants and desires” indicates a notion of maintaining independence and “function as a team” dependence, but the adult doesn’t account for how to co-ordinate them.

Attachment M. Male, Age 41, Stage 11 Systematic-Step 0 Failure: Transition to Metasystematic

Participant: I lost my car, my marriage, my job, my health and a whole lot of other things at that same period of time so I can’t say, you know, it was point 0. 0 centimeters of sadness associated with losing my motorcycle.

Analysis: Systematic because there was this whole system of losses impacting on him that he cannot point to one event or one variable as the cause of his sadness. It is seen as transitional step 0 because it is just loss with nothing else.

Therapy Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic

Participant: I need to understand that John is a man of few words when it comes to love.

Analysis: The adult is negating blame. She is taking responsibility for constructing her view of John as an element of her overall understanding of building intimacy instead of blaming him (entirely) for blocking the process. (Negating blame is a rejection of a formal, linear view of causality. ) The systematic level of complexity as explained here involves a context (i.e., “when it comes to love” is a distinct context) in which the self (i.e., “I need to understand that …”) takes a view of the other’s view of love (i.e., “John is a man of few words when it comes to love”).

Attachment D. Female, Age 41 Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic

Interviewer: How did you come to change your mind [about your whole way of looking at life, as a result of living through the war in your country]? Was it just the fear of death? …

Participant: Well, it was the fear of death. [and somewhat further down in the same statement:]

Things like this, you can’t have pink ideals when the situation around here is like that. And you have to live day by day. You just cannot plan anything not even for a week. Lack of water, lack of sometimes bread…

Analyses: The reason that she changed her mind is partly because of the fear of death. Her whole way of looking at life changed as a result of death becoming so immediate. This was coded as being at the Systematic stage. She was talking about having one view of life (her first system) before the war, and having a second and totally different view of life (her second system) after the war. Further down, she is saying you can’t choose a system, you can’t have ideals - and the war is what made her that way - she couldn’t choose the path - the war made her see things on a day-to-day basis. So, this is a comparative statement about two systems: the way things would have been, and the way they ended up, but there is no explicit comparing. She articulates each one, but alternates between describing one or the other; or rather she mainly describes the new system and leaves the interviewer to understand that the old system had none of this. This was scored as relativism.

Attachment E. Male, Age 23, Stage 11 Systematic-Step 2, Relativism: Transition to Metasystematic

[When asked to decide what was his greatest loss:]

Participant: ... but that hasn’t [happened] to me yet, though, ummm, and I’m not sure I feel comfortable saying that the biggest loss I’ve ever had in my entire life, I think for some it might be easy to do. They could tell right off, but I really have different experiences, so I could say that something was a big loss, but I’m not sure it was the most…so is that going to be…?

Analyses: He talks about one system: his set of experiences that have given him a particular set of losses. He also refers to potential other systems containing the experiences of others that may produce other losses or more losses. Because he does not co-ordinate these two together, but talks on the one hand about his experiences, and on the other hand about the experiences of others, he is at the relativism substep.

Therapy Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic

Participant: I understand that it may not be possible to be both at the same time [to be a friend and pastor to an individual], and that what I am looking for from an individual at each particular time will be different as I am friend and pastor.

Analysis: The adult succeeds in bringing together the two roles of friend and pastor within the same individual. He alternates them in a systematic fashion so they do not conflict with one another. The adult does this by placing himself with the parishioner into two different temporal contexts. Here, there are two perspectives (from the same person) but they are not fully integrated. The transition process is not yet complete for this stage.

Attachment J. Female, Age 25, Stage 11 Systematic-Step 3 Smash: Transition to Metasystematic

[When asked to describe her emotions after breaking up with her boyfriend:]

Participant: And yeah, I was angry too. I was angry at him because… because I knew there were some things about him that were wrong, and created these adverse reactions in me, and I didn’t really know what they were, but I was really mad at him for just being himself.

Analyses: She was angry for at least two reasons: he did things or had characteristics that were wrong, but there was almost something about her that had adverse reactions to the things he did. So she is describing a kind of multi-variable system that determines her emotion, in this case, anger. Also, just the phrase “being himself” is a systematic notion; it consists of multiple behaviors occurring at multiple times and occasions. But this is not fully metasystematic because she does not know what is driving her nuts. She does not fully specify either her self system enough, or the “other” system enough to have a clear sense of what is wrong. She is at step 3, smash, in the transition to metasystematic, and most likely at substep 1.

Therapy Stage 11 Systematic-Step 3 Relativism: Transition to Metasystematic

Participant: I relax. I breathe. I visualize the pages ahead and the intent of playing and the feeling for the music to be expressed, all from a calm place. I keep practicing all the little snags to smooth out. I get plenty of rest, do Brain Gym and chi activities. I visualize success and calmness. I pay attention to the music and not the worry. I breathe some more, and repeat. I remember this and create it from out of my larger goals and purposes.

Analysis: The adult explains that he integrates success and calmness through visualizing both together. Also, coming from a “calm place” promotes visualization, and paying attention to the music and not the worry. Success and calmness are two “systems” that he is co-ordinating. At this point, he appears to be overgeneralizing how the two are combined. As systems, he explains how success depends on calmness, but not clearly how calmness depends on success.

Therapy Stage 12 Metasystematic-Step 4 (0)

Participant: “To find things easily is to pursue a thought until it intersects perceptions or images of the misplaced item, to sense its presence without clearly identifying it and then it appears as if to fulfill some sense of it being there already.”

Analysis: The adult integrates “locating” and “identifying” into a subjective, intuitive system of thinking about an object. This is a strategy of “scanning and zeroing in” on the misplaced item. In this process, the adult first thinks about an item, which elicits perceptions or images of it. Then, the adult implies that he looks around for the item. The scanning is done by first using the “ground” to sense an object intuitively, and then by disembodying the object more fully so it becomes the “figure. “ At this point, the appearance of the object is gradually matched against the images of the thought about the object until the client becomes fully conscious of the object’s presence. At the end of his explanation, the client is just starting to construct the related system of objective appearances and locations. The variables of thinking about, locating, and identifying a misplaced item are fully co-ordinated into a strategy for finding misplaced items. When looking about, he first senses the item's presence without fully identifying it. At the end of the process, the item is fully located and identified, confirming the adult's “sense” of the item's location.

Good Education Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic Stage

Participant: Teacher says look, we're going to tell you things that you can write down. And if you forget you can look them up in the textbook. I promise I won't tell you anything that's not in a book you can look up. And you write them down and memorize them. And then we're going to have an exam. And you tell us back and we'll check off whether you told us right and whether you told us everything. And if you did, then we'll give you an A.

Now this is a very safe process for all concerned. There's very little risk for the faculty. I mean anybody can give an adequate lecture of that type. So it won't reveal you as a bad teacher unless you just don't prepare. It's also very safe for the student, right? Because if you do your homework, if you don’t screw around and play tennis and waste your time, you can pass almost any course that is taught that way. And the conspiracy is that neither party, neither the professor nor the student do anything to reveal that not much learning has gone on. What do I mean by not much learning has gone on? What I mean is, when you confront people with problems for which the knowledge you have transmitted is supposed to be useful later on, they can't solve them.

Analysis: Participant includes components from an educational system based on predictable outcomes and rote memorization, as well as components from a system with risk. Participant does not co-ordinate these components and gives as examples subsets of different ways of education and assessing students at the Kennedy School of Government.

Good Education Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic Stage

Participant: “I want to say one more thing about this. One of the ways, that it is very difficult to evaluate a process like this, is to ask the victims or participants of it at the time. It's characteristic of every experiment that we have made that…(in my view)…in every experiment that we have made that really involves learning, that the students and this includes mid-career adult students, hate it. Or say they hate it. They say, oh, don't do that. That's a terrible idea. They plead with us to teach statistics by the lecture method. They…we asked them for example to grade each other in class performance. We don’t do it any more. They grade each other and that's half their course grade. So they are responsible for each other and responsible to the classroom and we are no longer the policeman of classroom behavior. And they have six dozen different elaborately reasoned explanations of why that's inappropriate and unethical and why we shouldn't do that. And it's our job to grade them and so on….

Analyses: Participant includes components from an educational system based on traditional lecture-type teaching and teachers doing all the evaluation, as well as components from a system based on the case method and students’ evaluation of one another. Participant produces hits at stage 12 by describing components of an ideal system of student empowerment, but overgeneralizes by relying too much on examples and not co-ordinating components into a cohesive system –

You know I think the best example is of Ulysses tying himself to the mast. You know, where he sails past the sirens? Do you know the story? He sails past the sirens and he knows that they're going to sing and lure the ship to its death so he says, “OK, sailors, stuff your ears with wax so everybody can't hear anything. OK now tie me to the mast and unstuff my ears. So they sail past the sirens and he hears the sirens but he can't do anything about it and they don't hear it so they keep rowing. SO he's the only man in the world who's heard them and survived. And he knew that when he heard them that if he didn't tie himself to the mast, right, if he didn't restrict his behavior. Somehow we have to trust the students that have made an agreement like this. At the same time they can not want to do this week's homework and hope that we will not listen to them. At that moment. Tricky problem.”

Moral Reasoning Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic

Participant: Alright, a business is trying to provide some product or some service to the society. Trying to make a profit. The university is trying to educate people. So obviously, the activity is going to be different… I suppose you could say that there's a certain kind of interaction that's crucial for a university's place students and faculty which doesn't have a comparable place in other kinds of institutions.

Analysis: Participant does not fully describe or co-ordinate the systems of business and education

Good Education Stage 11 Systematic-Step 1 Negation: Transition to Metasystematic Stage

Participant: “Well, there are some people who like to think of everything as a business. It's a metaphor people use. And maybe there's some point to that, but it also can be misleading. There's a point to it in the sense that the university should worry about using its resources efficiently and should make sure that it's accomplishing its mission. But it's misleading in the sense that you can't simply take the standards that apply in a business enterprise and transfer them wholesale to a university, judge it by the same standards. You've got to realize that there's a different kind of enterprise going on.”

Analysis: Participant does not fully describe or co-ordinate the systems of business and education.

Good Education Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic Stage

Participant: Well, because, I guess these goals are sort of Aristotelian [in] that the truth is always somewhere in between. And if we used all the resources of higher education merely to prepare for a career for example then the career wouldn't be worth having because life wouldn't be worth living. I mean if what you learned was how to do your job, then there wouldn't be much point in doing the job because you couldn't enjoy anything else. Conversely, if all you did was get very good at reading books and consuming experience, you wouldn't be any good to anybody else so why we should we care that you are having a good time?

Analysis: Participant alternates between the “preparing for a career” and “getting very good at reading books and consuming experience,” without co-ordinating them.

Good Education Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic Stage

Participant: So if higher education is in the service of these three objectives, and if turns out to only be good for one of them, then, I'd say it won't even advance that one. That these aren't additive. You can't say, well, this is the Harvard School of Getting an MBA and Making a Million Dollars. The accusation that's leveled at the Business school is that all it does is teaches people to make a lot of money and not to be good people in some other sense. And then it turns out to be not worth it that they are making a lot of money. Because they are deficient in these other dimensions. So that I can't separate the quality of interdependence and talk about any one of these by themselves in isolation.

Analysis: Participant alternates between students to make money and becoming a good person without co-ordinating them.

Good Education Stage 11 Systematic-Step 2 Relativism: Transition to Metasystematic Stage

Participant: .., any large institution needs governance, which is to say that some people have to be in a position to make decisions. So the university's no different from other institutions in that respect. What may be characteristic about universities is that much of the authority is decentralized, much so than any other social institutions. And that's because it's very important to allow faculty members to have a good deal of autonomy.

Analysis: Participant does not co-ordinate the two systematic goals of governance and autonomy

Good Education Stage 11 Systematic-Step 2: Transition to Metasystematic Stage

Participant: Whereas, if you want to call what the university does is producing a product, the product is the very interaction between students and faculty. I: Ok. What should that interaction be and why? I: What quality should it have? There are some very general features one might say as openness, willingness to explore new ideas, a willingness to question, to debate, to provide support for claims that are made...no one has monopoly or a lockhold on the truth. And if for every individual to a considerable extent has to make a judgement for him herself as to what's true. What makes sense. What connects to what.

Analysis: Participant produces a hit at stage 12, but basis for exclusion what is or isn’t an element in the ideal university system is not sharp

Good Education Stage 11 Systematic, Step 2: Transition to Metasystematic Stage

Participant: I think that everyone has an inclination, maybe even a desire to be certain about how the world is. Among other inclinations. And what you want to do is lead a person to perceive that they are adults, so you explore the beliefs that people have and you show where there are questions and why one has to keep a certain openness in regard to those things because they are questions.

Analysis: Participant produces a hit at stage 12, but basis for exclusion what is or isn’t an element in the ideal system of training is not sharp.

Good Education Stage 11 Systematic, Step 3 Smash: Transition to Metasystematic Stage

Participant: I don't mean by that there should be some separate course in a professional school because that seems to me a way of simply keeping ethos concerns marginal … in law there is a guiding ideal that the course system should be doing justice… when a lawyer helps two parties write a contract, the lawyer should have some understanding of what a fair contract is and what equal bargaining is. I: Why? W: Those are moral notions. Because this is a way of treating people with respect.”

Analysis: Participant correctly rejects reducing moral concerns to a marginal level, but under generalizes the need for respect to one professional area, the law.

Table 3b (LoCicero, Personal Communication, 2005)

Model of Hierarchical Complexity	Kegan Stages	Kegan Descriptions	WWK Descriptions	Womens’ Ways of Knowing Stages
Stage 9 Abstract	Stage 3: Interpersonal	Able to coordinate own needs and interests with others; unable to prioritize.	Distrusts conventional authorities; trusts experience, intuition	Position 3: Subjective
Stage 10 Formal	3-4 Transition	Shaky sense of self-as-authority, self-sufficiency.	Beginning to appreciate objectivity; procedures for sharing and evaluating knowledge.	3-4 Transition
Stage 11 Systematic	Stage 4: Institutional	Self-sufficiency, authority. Identification secure. Knows where one stands.	Concerned with methods and procedures for obtaining and sharing knowledge. Interested in objectivity. Learns by doubting game (separate) or believing game (connected).	Position 4: Procedural (Separate and Connected)
	4-5 Transition	Sees limits of self-sufficiency. Somewhat aware of isolation. Moving towards intimacy and directness.	Beginning to see self and others as creators of knowledge.	4-5 Transition
Stage 12 Meta-systematic	Stage 5: Interindivid-ual	Relates to others directly. Not role-bound. Intimacy. Recognizes reality as co-constructed and self as continuously created by multiple relationships.	Knowledge as situated. Integration of subjective and procedural. Sees self and others as creating and co-creating knowledge.	Stage 5: Constructed

Table 3c

Table 11.2

Comparative Table of Concorded Theories of Early to Postformal Stages

Researchers	Abstract	Formal	Systematic	Meta-systematic	Paradigmatic	Cross-Paradigmatic	Transcen-dental
Commons & Richards (1984)	9 (= 4a)	10 ( = 4b)	11 ( = 5a)	12 ( = 5b)	13 ( = 6a)	14 ( = 6b)
Sonnert & Commons (1994)	group	bureaucratic	institutional	universal	dialogical
Inhelder & Piaget (1958)	formal III-A	formal III-B	postformal	polyvalent logic; systems of systems
Fischer, Hand, & Russell (1984)	7	8	9	10
Sternberg (1984)		first-order relational reasoning		second-order relational reasoning
Kohlberg (1981)	3 mutuality	3/4	4 social system	5 prior rights/ social contract 6 universal ethical principles
Benack (1994)	4	5	6	7
Pascual-Leone (1983)	late concrete	formal and late concrete	pre-dialectical	dialectical			transcendental
Armon (1984)	3 affective mutuality	3/4	4 individuality	5 autonomy		6 universal categories
Powell (1984)	early formal	formal	stage 4a/ interactive empathy	category operations [?]
Labouvie-Vief (1984)		intra-systematic	inter-systematic	autonomous
Arlin (1984)	3a low formal (problem- solving)	3b high formal	4a postformal (problem-finding)	4b relativism of thought 4c over-generalization, 4d displace-ment of concepts	4e late postformal (dialectical)
Sinnott (1984)		formal	relativistic/ relativize systems, metalevel rules	unified theory: interpretation of contradict-tory levels
Basseches (1984)	phase 1b: formal early foundations	phase 2 intermediate dialectical schemes	phase 3: 2 out of 3 clusters of advanced dialectical schemes	4. advanced dialectical thinking
Koplowitz (1984)		formal	systems	general systems		unitary concepts
King & Kitchener (2002)	4	5	6	7
Torbert (1994)	diplomat	technician	achiever	ironist
Kegan (1994)	3:interpersonal	3/4	4: institutional	5 interindividual
Loevinger (1998)	conformist-conscientious	conscientious	individualistic	autonomous integrated ¹
Cook-Greuter (1990)	3/4	4	4/5	5	5/6		6
Gray (1999, June, personal comm.)	early formal	formal	systematic	metasystematic
Bond (1999, June, personal comm.)	early formal	formal	systematic	metasystematic
Dawson (1998)	9	10	11	12	13	14
Kallio (1991, 1995)	formal 1	formal 2	formal 3 generalized formal	postformal
Demetriou (1990; 1995)
Broughton (1977; 1984)	3: person vs. inner self	4. dualist or positivist; cynical, mechanistic	5. inner observer differentiated from ego	6. mind & body experiences of an integrated self
Belenky, Clinchy, Goldberger, Tarule women’s ways of knowing (1997)	position 3 subjective	transition	position 4 procedural separate and connected)	position 5 constructed

Lots of the things studied in animal behavior are same order of hierarchical complexity. Because there are so many other ways of making things more difficult, the order of hierarchical complexity is a very good source for measuring all these other dimensions of how to make things difficult. One can use order as a metric against which to measure the other ways. One varies the amount of the other difficulty producing variable, until it produces a decrement in performance equivalent to lower the order of complexity by one. Then one knows that that amount is the equivalent of one order of hierarchical complexity for that organism in that situation.

Let us say one gets, what is true about hierarchical complexity is that the sequence, the numbers and the properties of the orders of complexity are the same irrespective of task, content, method, modality, organism or anything. They have nothing to do with the empirical world. They are analytic truths, but that does not mean the performances are all identical at all. In fact, the performances should be highly different in each animal depending on each of the characteristics of the tasks that do not vary in hierarchical complexity, but hierarchical complexity is so powerful that when one look across organisms and tasks, it should be, the prediction is that it should explain the most variance. Let us say one did not know what the task was and one gave it to an organism, one did not know anything about the organism or the task or the niche or anything, one randomly selected them from a common set of tasks, the best predictor would be hierarchical complexity that they can generally do.

The stage of performances and G

G, The General Intelligence Factor, is found from applying factor analysis to a set of performances. Non-primate animals most often respond successfully to lower orders of hierarchical complexity tasks. They are clearly are more specialized at solving niche problems than organism that solve higher order tasks. The higher the stage of performance of an organism, the more likely it is to be able to solve a variety of tasks and one thing that we find is that social animals generally do better even on arithmetic tasks than nonsocial animals. So there is a general G and if I had to guess I would say it has to do with social communication in a flexible manner. That the more complex the social communication that that would be the best measure of G and attempts to look at G in animals in visual spatial tasks will have a similar history as it does in humans. It does not do very well and obviously a few things such as assortativeness in insects so that they only mate with their own kind and so forth would be highly specialized, but most organisms have to solve that problem of who to mate with and there quite often is in pretty low level organisms there is competition for mates. A lot of it is ritualized, but we would get a much better understanding of what organisms are about if we had these profiles and we could see what kind of skills hang out with what kind of skills and which ones do not. And we could test these hypotheses, but to me that is what comparative is and there is at the moment, other than measuring sensory sensitivities and some very basic kind of sensory and perceptual processes. That is not really about cognition, that is about sensory and perception and this is really the only measure that allows us to compare cognition.

Tasks, Domains and Subdomains

This is about the pattern and profile of multiple tasks with possible different hierarchical complexity. Now, in a homo-centered way, the tasks have already been classified as a visual task or as a spatial task. It would be good to further break down the domains into subdomains. Empirically, what is needed is test for any transfer across the tasks. If one gets better at one task, then is there an improved performance or improved acquisition on another task. If there is, then the two tasks are related and they would be functionally in the same subdomains. What the domains and subdomains are can vary in different organisms, different niches, different modalities. Let me give some extreme examples. Humans do not get to order one, the discrimination and habituation of ultra violet light or inferred light. When it comes to many smells, humans do not get to order one. When humans have no sensitivity, we do not get even to order one. That part of the profile in a human would show absolute idiocy.

We are not ranking organisms per se for each task. But when we want to compare organisms, one thing that we do is say we use what is called the maximum rule. **[needs to be edited from here to the next bracket] We take the maximum complexity of any task that this animal can do in any domain with any modality. The organism is characterized by the max of this. That is fair to the organism because they are not compared in different domains that they do not function well in. For example, using mice with something that involves smell and probably spatial would probably a niche that they normally function very well and flexibly.

One should characterize a number of things about the tasks. One, one has to choose the simplest version of the task at a given order of complexity. For example, if the issue is whether pigeons recognize that pictures represent objects, one have to make it very simple. One way to make it idiotically simple, is to put the same frame around the object and the picture so if one show it to a human being they have a hard time telling them apart. One is sure to get it correctly. But that is a degenerate situation. What one is really asking is how different can the representation or the picture be from the object ? It is a much more complicated issue. One would not know that unless one had some sort of model of how one test these things and this is such a model.

Level of Support and the Hierarchical Complexity of a Task That Can Be Addressed

Seven levels of support or lack of support that have been identified ( Commons & Miller, 1998 Commons & Miller, 1998).

Level 1 of support is imitation, this is widely used by Fischer ( Fischer & Lazerson, 1984 Fischer & Lazerson, 1984). This consists of showing the animal how to do the task and then after showing them and they do it, with reinforcement.

Commons, Miller and Kuhn (1982 Commons, Miller and Kuhn (1982) presented such material over and over. There was an improvement in performance. Inhelder and Piaget (1958) Inhelder and Piaget (1958) were probably wrong not to present the problems over and over for at least a few trials with reinforcement for correct solutions. There is a difference in doing it that way. People work harder, they learn to what to attend. Commons, Grotzer and Davidson (in preparation) found that about 75% of fifth and sixth graders performed at formal stage after three or four trials, whereas for Inhelder and Piaget, only 50% of tenth graders did so. It is interesting even just adding reinforcement and practice, but it has to be reinforcement. It is not enough just to use feedback because we found that was a disaster with human beings. One should follow good animal behavior rules that the animals have to be reinforced appropriately and the situations have to be designed appropriately. One can make tasks hard in a lot of stupid ways

Level 0 of support is illustrated with formal stage behavior by Inhelder and Piaget (1958) Inhelder and Piaget (1958), who just presented problems and did not use any reinforcement nor with any repetition.

Level -1 is problem finding. So finding out what one needs to do is really even harder than formal operations.

Starting a Fire Example and Levels of Support

Let us take the task of starting a fire, which hominids did.

Level 1: Imitating somebody starting a fire. If somebody says I want one to build a fire and someone else says this is how I start a fire, it would be one level of support.

Level 0: If one wants someone to discover how to start a fire, but tells them it is possible even though that person never saw it done, that person could fool around for a long time. But it would not be the simplest instance of a problem because there is a number of steps and a number of tools that have to be constructed. For instance, one has to collect tinder if one are using the striking stone method. So one would first have to collect tinder and then one would have to have the right stones and then one would have to hit them together in exactly in the right way to produce a spark and then one would have to blow on the tinder, have to be able to organize the fire, the tinder on the bottom, the small sticks next and then the bigger sticks and one would have to learn.

A simple formal operational level 0 task would be that if one found fire one could feed it with wood. This involves trying out various materials. One would find out that there are some things burn and others do not. Burning versus not burning is one variable, plant material versus non-plant material is another. that stones do not work, sand does not work, wet word barely works, dry wood works well, small pieces of wood burn easily, big pieces last long, etc. Making a fire is much harder than maintaining the fire and it is very interesting that humans have evolved to love, to watch fire. Kids love to build fires and the fact that it is reinforcing is interesting in and of itself.

We can use nonverbal sequences that really are very complex, especially things like starting and maintaining a fire, but by the time fire evolved there already was some language so we do not know that was the way humans did it, but one could imagine some other animal doing it, but it already started at a stage higher than a stage the chimps get to. We could show that.

The usual range of support varies from no support, minus one -- having them discover what the problem

The use and maintenance of fire can be a nonverbal activity. It could possibly be accomplished by chimpanzees but it is not. But making fire is formal order task finding it the first time is a much more difficult task.

First, no one understands that fires cook meat and tubers. Finding the problem is -1 level of support. The task of finding that fire cooked meat is a lot easier to eat than raw meat is formal order. It requires finding that meat in the fire is more tender and meat out of the fire is not. Finding meat that is tender is reinforcing. More of the meat in a carcass may be eaten. It is easier to get off of the animal. The first hominid would have to discover the utility of fire, keeping themselves warm, cooking meat. Then one might figure out that cooking certain tubers makes them edible whereas before they are not. That way hominids with fire have a much wider supply of food. It also means one they do not need such a large stomach because cooked tubers and meat do not require as huge a stomach as uncooked tubers and meat do to digest. The higher the caloric value of the food, it can support a bigger brain, it can require less effort and social animals tend to have higher values of caloric food than non-social animals. But there are solitary animals like bears that fish and eat incredibly high caloric food. They actually skin and only eat the skin of the fish, not even the meat when they are very plentiful.

Developing an explanation of why cooking food entailed even less support.

For Every Task, There Are a Number of Sequences.

Meredith Root-Bernstein (Personal communication, January 18, 2005) in Kacelnik laboratory presents an interesting example. Consider the end task of making a spider web. The important lesson here is that one must construct the task sequence, not just the end task. For a human and for a spider, the end looks somewhat similar. But the task belongs to two entirely different sequences. The human probably cannot make a spider web with a tool dispensing the web material until they perform at the concrete stage. For the human, it is a complex task. They have to understand the structure of the web – that it has axle arms connected by straight connectors between the arms. They have to start the tool at the arm and stop it at the next. This is somewhat akin to model car and airplane making. But for the spider, this is just a circular sensory motor order 2 task. All they have to do is squirt the suff out of the correct spinaret at the back (an sensory or motor order 1 task) and coordinate it with making sure the tension is constant, using their legs to gauge that tension (a second order 1 sensory or motor task). The web silk is made inside the spider and comes out of a spinaret at the back of the spider. This is an organ that makes the protein come out in thin fibers. It just has to coordinate the squirting of the sild with the tension the gauge between their legs. They start their web by sending out a thread that floats in the air until it sticks to a branch or leaf. Then more thread that sticks to other branches and also goes from the center to the sides. They keep spinning around and around in circles. Soon after, The flat round web is finished.

Preliminary analysis and scoring the crow example

Weir, Chappell & Kacelnik (2002) observed New Caledonian crows performing what will be shown to be a Preoperational order 6 action. The way sequences are built, is from the top down. So the preoperational actions has to organize two sequences. Each of the sequences has to organized in a different manner the same or different elements from the nominal.

At the Preoperational order, the crows have to coordinate two sentential order 5 sequences of action that finally result in the crow using a bent wire to obtain food from a crooked clear plastic tube.

The first Sentential order 5 straight-wire tool-use sequence consists of action steps: 1, 2, 3, 4, N

The second Sentential order 5 bent-wire tool-making sequence consists of action steps: 1, 2, 3, 4, K

Many birds pick up twigs. They also weave their nests with twigs. These require circular sensory-motor actions if they do not need a concept of a nest to do it. Instead of a concept of a nest, there is something that guides the process.

Pre actions – Where does this fit in? What sequence does this belong to? Is it necessary or just sufficient?

I. The first Sentential order tool use sequence consists of two Nominal order 4 representations. The “template” or representations are the nominal order names.

A. The first Nominal action is that Crows have to represent that sticking wire in the tube to get the food.

1. The first concept is that stinking stick like things into cavities to get at things. [Where two instances of concept are there here? ]

Normally pecking at food obtains it (Circular Sensory Motor). Pecking at the food in the tube does not obtain the food (Circular Sensory Motor). Sticking their beak down the tube fails also (Circular Sensory Motor). Failing at obtaining the food in a tube, leads to seeing that an extension of their own beak is needed (Nominal). They might have to visualize the wire as an extension of their beak. (Nominal order 4). This picturing of an extension is a representation at the nominal stage.

B. The second Nominal action is that the crow must see that the wire might poke the food out of the tube. That is, the crow needs to form a representation of the wire as a pusher of food out of the tube. (Nominal) They do not have practice doing that.

But seeing the food in the tube would not lead them poke with out without a representation at the nominal order 4 of some sort.

Distinguishing putting a stick into a tube to push out food from other tool use actions (putting a different stick into a hole to push out something from there) is the same as naming the two different tool use actions.]

Crows stick sticks and other things into a tube (circular sensory motor, order 2)

Crows pick up straight wires or sticks (circular sensory motor, order 2)

Crows see food in a clear plastic tube. (Sensory or motor)

Crows peck at objects with their beaks (Sensory or motor)

II. The second tool making sentential order sequence also consists of two nominal order 4 representations.

A. The first nominal order actions is seeing that using a straight wire failed because of the bend in the tube – straight wires do not go around bends. They form a picture of the bend in the tube and a picture of the straight wire.

B. The second nominal order actions is that the crow then forms a representation or template for the bend in the tube. From the representation of the bend in the tube, the crow then bends a wire.

Sentential order 5 They are

substep 3, Sticking it into the tube,

substep 4. Pushing wire around a curve, and

substep 5. Reaching the food and getting it

3. Sticking it into the tube;

4. Pushing wire around a curve;

5. Reaching the food and getting it.

So it is not a concept that has been formed by trial and error. (Sensory Motor)

The first sentential order 5 tool use sequence of actions then consists of substeps: 1, 2, 3, 4.

Maybe trying to get the food with the beak (subsubstep 1)

Failing leads to, seeing that an extension is needed (nominal).

Picking up a straight wire or stick (circular sensory motor, order 2)

Sticking it into a tube (circular sensory motor, order 2)

Push the food out of the tube (circular sensory motor, order 2) producing a concept at sensory motor order 3.

How to Construct a Task at the Next order of Complexity.

Let us say one have a ordered complexity task and I have advised people tests, where the testing one above and they are not pushing the organism, if the organism can do one above, they have to do two above until the organism fails to do the task. To make sure that one is in the right sequence and things are not weird, one do one below. So one do one above and one below. And they better fail the top one.

One can really write up a profile and of course, much of this work has already been done and what we need to do is to mine it. One way one make tasks more complex, is one ask organisms to reflect on the task. Let us say there subitizing how many pellets they get. One can ask them to tell 5 from 6 and that has been done, one can go 7 from 8 and one can try different spacing and one can have a whole thing. That would be a first ordered test. Are these tests more about whether things, what rule they are using, Brannon ( Brannon & Terrace, 1998 Brannon & Terrace, 1998) suggests the following. Let us say one are doing arithmetic and there are two rules either multiply or add and if one wanted to do a very simple concrete task one would ask them, one would ask them very simply a left press would mean I am doing addition, a right press would mean I am doing multiplication. And since they are reflecting on the operation that is a concrete task.

Variability of stage of performance and stage

One would expect that the variability in organisms that do not get to very high levels of order of complexity is very low. The variability should be across tasks of different sorts and of tasks with different modalities and different sensory systems

Today is 3-27-2004. Driving from Melbourne Florida to Orlando Airport.

Multisystems Tasks

Michael Lamport Commons and Francis Asbury Richards

For Educational Purposes

Consent Form

I, the undersigned, understand that the basic intent of these tasks is to understand reasoning and how it develops. This is entirely voluntary and I may stop at any time. I understand that the information I have given regarding my personal history, as well as the responses I give to the tasks, will be kept strictly confidential. I understand that my identity and personal history will not be divulged in any discussion, student report, lecture, address, or publication derived from this project.

| |

Signature Month Day Year

Instructions

Please answer all parts of this questionnaire. Read each section and answer the questions in the order given. Do not go to the following section before you have finished the previous section. Remember this is not a test of your ability as an individual. Rather, we wish to know how adults, in general, reason about the issues presented here. The order of answering is essential to this study about adult reasoning.

What is most important is not only what you consider to be good, but the reasons for your choices. Therefore, it is essential that you state your reasons why you make the choices that you make.

Participant Number: _______

Date of Birth: month _______ day _____

Now that you have read the stories and are familiar with them, answer the questions below. Make your comparisons on the basis of properties of the orderings found in each story, in the best and most complete way that you can.

Write out all the comparisons that you can in a systematic way. Use symbols to represent the order of things, and explain what the symbols stand for. Also include an explanation in English. You may want to use a chain; for example, Oregon → Texas, in addition to an ordering, Oregon > Texas. Then explain what are the most important similarities and differences in the stories, and explain how you arrived at deciding the relative importance of these similarities and differences. You may refer to more than one framework.

Make sure to give the strongest, most thorough, broad, inclusive, and complete explanations possible for the similarities and differences. It is necessary that you show all of your work in forming the orders and making the comparisons, as well as your commentary in English.

11. 6 & 7 1 2 3 4 5 6

12. 6 & 8 1 2 3 4 5 6

13. 6 & 9 1 2 3 4 5 6

14. 6 & 10 1 2 3 4 5 6

15. 7 & 8 1 2 3 4 5 6

16. 7 & 9 1 2 3 4 5 6

17. 7 & 10 1 2 3 4 5 6

18. 8 & 9 1 2 3 4 5 6

19. 8 & 10 1 2 3 4 5 6

20. 9 & 10 1 2 3 4 5 6

21. How do you know that the order you have indicated is correct (Fischer)?

22. What did you think of what Richard Reagan proposed to do with the people in the various states in story one? What is important in that first story to you?

23. Why is it important?

24. What do you think of the social, and political consequences of Richard Reagan's proposal?

25. How fair what Richard Reagan wanted to do with the states?

Not at all 1 2 3 4 5 6 Totally

26. Why?

27. Why not?

28. How might you deal with such a proposal and why would you deal in that manner (Gutheil, Appelbaum, Pauline, & Commons, in preparation)? Sorry

29. How much did you care about what happened to the people in these states and especially the ones you know (Gilligan, 1982)?

Not at all 1 2 3 4 5 6 Totally

30. Why?

31. Why not?

32. To what extant do you consider this an issue (Gilligan; Kegan)?

Not at all 1 2 3 4 5 6 Totally

33. "Should Vanktesh have been able to choose between x and y?" ___No ___Yes

34. Why should Vanktesh been able to choose or why not?

35. In the fourth story about weighing jewelry, could it ever be that even with a good scale one might see (get) similar results (King & Kitchener)? ___No ___Yes

36. Why or Why not?

37. What is it about weighing, taking states, chips and food preference that are all similar or different?

38. What is their core meaning?

39. How do you make sense of it (Kegan)?

40. If things are not decidable is it because they are relative (Sinnott, 1984; Kramer, 1990).

Table 3 General Description of Sequence

	Vocalizations and words	Verbal relationships	Examples
0	calculatory	None	None	None
1	sensory & motor actions	Babbling (Universal)	None	None
2	circular sensory-motor actions	Phonemes		Gestures, “aa”
3	sensory-motor	Morphemes	Phoneme action relationships	Holding cup out and saying “wa”
4	nominal	Single words: ejaculatives & exclamations, verbs, nouns, number names, letter names	Relating concepts to others through a word	Says "water"
5	sentential	Pronouns	Incomplete sentences; then complete sentences, crude dichotomies, ordered numbers & letters	States a rule "Don't touch"
6	preoperational		Multiple sentence stories with logically order sequenced events. Counting materials laid out in a line	Says "The water is filling the bathtub"
7	primary		Stories with logically order sequenced events connected solidly to reality with some specified times, places, acts, actors. Either perspective of self or other.	The water is filling the bathtub so I can take a bath
8	concrete	times, places, acts, actors	Stories about things, incidents, events, actors, actions, places in the context of the interaction between self and other.	If I turn on the faucet, the water will run out and fill the bathtub. My parents like it when I fill the tub myself. They are more willing to buy me clothes when I am always clean.
9	abstract	Variable time, place, act, actor, state, type; Quantifies (all, none, some) play an important role in the preadolescent's formulation of categorical assertions (e.g., "All teachers do that!").	Variable time, places, acts, actors, state, type Variables such as Stereotypes, Logical Quantification Words: all, none, some, never, sometimes, always, no one, somebody, everyone Propositions with variables	The water can run out quickly or slowly and the temperature may get colder after awhile. All teachers do that!
10	formal	Words: linear, logical, one dimensional	Variables connected by "if... then" and other relations. Empirical, or logical evidence and argument. Relations among Variables Single variables on input side Empirical, or logical evidence and argument.	Turning of a faucet changes the size of the opening of the passage way and thereby controlling the amount of flow.
11	systematic	Systems words appear in adult's utterances: legal system, society, our company, the economy, the country.	Systems of relations. Relations among relationship among variables	For slow flows, amount of flow is controlled by 1) Water pressure, 2) size of faucet opening 3) smallest of pipe size to source.
12	meta-systematic	Metalogical, meta-analytic words appear. Properties of systems can be considered: homomorphic, isomorphic, incomplete, inconsistent system, consistent system, incomplete system, complete system, incommensurable system, orders of complexity of the system, commensurable systems, etc.	Supersystems of systems	The systems that describe non-turbulent water flow and electric flow (amperes) are isomorphic. Flow equals pressure (volts) x resistence (ohms)
13	paradig-matic	Newly created paradigmatic words are used whether metasystems may be integrated.		There are many metasystems that belong to non-relativistic and non statistical mechanics
14	cross-paradig-matic	Newly created word about integration of paradigms.		By crossing mathematics and physical experimentation, classical mechanics arises.

Table 4a General Description of Hierarchy

Order or Stage	Discriminations	Vocalizations
0	calculatory	Exact–no generalization	none
1	sensory & motor actions	Rote, Generalized	Babbling (Universal)
2	circular sensory-motor actions	Open-Ended Classes	Phonemes
3	sensory-motor	Concepts	Morphemes
4	nominal	Relations among concepts	Single words: ejaculatives & exclamations, verbs, nouns, number names, letter names
5	sentential	Imitates and acquires sequences. Follows short sequential acts	Pronouns: my, mine, I; yours, you; we, ours; they, them
6	pre-operational	Simple deduction but contradiction is not excluded. Follows lists of sequential acts	Connectives: as, when, then, why, before
7	primary	Simple logical deduction and empirical rules involving time sequence. Simple arithmetic	Times, places, acts, actors
8	concrete	Full arithmetic	Interactions, social events, what happened among others
9	abstract	Discriminates variables such as Stereotypes; Logical Quantification; (all, none, some)	Variable time, place, act, actor, state, type; Quantifies (all, none, some); Categorical assertions (e.g., "All teachers do that!").
10	formal	Argue using empirical or logical evidence. Logic is linear, 1 dimensional.	Words: linear, logical, one dimensional, if... then, thus, therefore, because.
11	systematic	Constructs multi variate systems and matrices, coordinating more than one variable. Events and ideas situated in a larger context.	Systems: legal system, society, our company, the economy, the country.
12	metasystematic	Integrate systems to construct multisystems. Compare systems and perspectives in a systematic way (across multiple domains). Reflects on systems.	Metalogical, meta-analytic. Properties of systems named: homomorphic, isomorphic, incomplete, inconsistent or consistent, incomplete or complete, commensurable, incommensurable
13	paradigmatic	Discriminate how to fit metasystems together to form new paradigms
14	crossparadigmatic	Discriminate how to form new fields by crossing paradigms.

Table 4b General Description of Sequence

Kohlberg		Discriminations	Vocalizations and words	Verbal relationships	Examples	Receptive speech examples
0	-1	calculatory	Exact–no generalization	none	none	none	none
1	0	sensory & motor actions	Rote, Generalized	Babbling (Universal)	none	none	none
2	0	circular sensory-motor actions	Open-Ended Classes	Phonemes		gestures, “aa”	Follows one act directions
3	0/1	sensory-motor	Concepts	Morphemes	phoneme action relationships	Holding cup out and saying “wa”	Follows simple commands
4	1	nominal	Relations among concepts	Single words: ejaculatives & exclamations, verbs, nouns, number names, letter names		Says "water"	Follows simple chain of commands such as:
5		sentential	Imitates and acquires sequences. Follows short sequential acts.	Pronouns	incomplete sentences; then complete sentences, crude dichotomies, ordered numbers & letters	States a rule "Don't touch"	Follows instructions: "Turn off VCR when tape is over"
6	½	preoperational			Multiple sentence stories with logically order sequenced events. Counting materials laid out in a line	Says "The water is filling the bathtub"
7	2	primary	Simple logical deduction and empirical rules involving time sequence. Simple arithmetic	times, places, acts, actors	Stories with logically order sequenced events connected solidly to reality.	The water is filling the bathtub so I can take a bath	Can follow directions without being supervised for an extended period of time.
8	2/3	concrete	Full arithmetic	interactions, social events, what happened with among others.	Stories with some specified times, places, acts, actors, and the relation among the above. Things, Incidents, Events, Actors, Actions, Places	If I turn on the faucet, the water will run out and fill the bathtub.
9	3	abstract	Discriminates variables such as Stereotypes; Logical Quantification; (all, none, some)	Variable time, place, act, actor, state, type; Quantifies (all, none, some) play an important role in the preadolescent's formulation of categorical assertions (e.g., "All teachers do that!").	Variable time, place, act, actor, state, type Variables such as Stereotypes, Logical Quantification Words: all, none, some, Propositions with variables	The water can run out quickly or slowly and the temperature may get colder after awhile. All teachers do that!	All teachers do that!
10	3/4	formal	Argumentation contains empirical or logical evidence. Logical reasoning is linear, logical, one dimensional.	Words: linear, logical, one dimensional	Variables connected by "if... then" and other relations. Empirical, or logical evidence and argument. Relations among Variables Single variables on input side Empirical, or logical evidence and argument.
11	4	systematic	The adult constructs multi variate systems and matrices, coordinating more than one variable as input. Events and ideas can be situated in a larger context.	Systems words appear in adult's utterances: legal system, society, our company, the economy, the country.	Systems of relations. Relations among relationship among variables
12	5	meta-systematic	Some adults can integrate systems to construct multisystems. Comparison of systems and perspectives occurs in a systematic way (i.e., across multiple domains). The metasystematic adult reflects on systems.	Metalogical, meta-analytic words appear. Properties of systems can be considered: homomorphic, isomorphic, incomplete, inconsistent system, consistent system, incomplete system, complete system, incommensurable system, orders of complexity of the system, commensurable systems, etc.	Supersystems of systems
13	6	paradig-matic	There are a few adults who can discriminate how to fit metasystems together to form new paradigms
14	7	cross-paradig-matic	Historically there a few people who have discriminated how to form new fields by crossing paradigms.

Levels of Support

Expanding upon Vygotsky's (1966a, 1966b Vygotsky's (1966a, 1966b) notion of scaffolding, we introduce seven values of stimulus control (Dimension 3). Think of these as levels of independence of control by stimuli autonomy in responding to stimuli. Each level changes the relative difficulty of a task. Level of support or demand given during problem solving derived from Arlin (1975, 1984 Arlin (1975, 1984 Arlin (1975, 1984), Fischer (1984 Fischer et al. (1984), Gewirtz (1969b Gewirtz (1969), and Vygotsky (1966a; 1966b Vygotsky (1962; 1966). The name of support is stated and how it changes the measured order of performance relative to unaided problem solving. Then the action with respect to the participant is stated. The explanations follow.

Table 5 Level of support

Support Name	Change in measured complexity	Form of support	Action	Description
. Manipulation	-3	Being moved though each step.	Literally being moved through each step of how to solve a problem.	Part of the stimulus is the push that guides the movement.
Transfer of stimulus control	-2	Being told each step (direct instruction).	Do a task based on a set of verbal instructions or other direct stimuli telling one what to do.	Train a discrimination with one set of stimuli on one task. Use the same set of stimuli to control performance in another task. Slowly remove first set of stimuli. This is like an errorless learning procedure (Moore & Goldiamond, 1964; Terrace, 1963).
Pervasive imitation	-1	Being shown.	Includes delayed imitation or observational learning ( Gewirtz, 1969 Gewirtz, 1969). The imitated action may be written, depicted or otherwise reproduced.	Fischer and Lazerson (1984) call this form of control the optimal level.
Direct	0	No help or support is given.	Problem-solving or hacking (without support).	Fischer and Lazerson (1984 Fischer and Lazerson (1984) call this the functional level. Most of Piaget’s work was at this level.
Problem finding	1	In addition, to not getting help, one must discover a task to answer a known question.	Persons are given an issue and they are asked to give a example of a problem that reflects that issue.	Arlin (1975, 1977, 1984) Arlin (1975, 1977, 1984) Arlin (1975, 1977, 1984 Arlin (1975, 1977, 1984) introduced postformal complexity (systematic order) by requiring the construction of a formal-operational problem without aid or definition.
Question finding	2	In addition, to not getting help and having to discover, one must discover the question	With a known phenomenon, people find a problem and an instance in which to solve that problem.	One has to discriminate the phenomenon clearly enough to create and solve a problem based on that discrimination.
Phenomenon finding	3	No direct stimulus control is possible without a description of phenomenon.	Discovering a new phenomenon.	No reinforcement history with phenomenon.

Table 6a Deconstruction in the Transition Steps

Step	Sub-step	Relation	Name	Dialectical Form
0 (4)		a = a' with b'	Temporary equilibrium point (thesis)	Previous stage synthesis does not solve all tasks. (Deconstruction Begins) Extinction Process
1		b	Negation or complementation (antithesis)	Negation or complementation, Inversion, or alternate thesis. Participant forms a second synthesis of previous stage actions). (antithesis)
2		a or b	Relativism (alternation of thesis and antithesis)	Relativism. Alternates among thesis and antithesis. The schemes coexist, but there no coordination of them). (alternation of thesis and antithesis)

Table 6b Construction in the Transition Steps

3		a and b	Smash (attempts at synthesis)	The following substeps transitions in synthesis.
	1		Hits and Excess False Alarms and Misses	Elements from a and b are included in a non-systematic, non-coordinated manner. Incorporates various subsets of all the possible elements.
	2		Hit and Excess False Alarms.	Incorporates subsets producing hits at stage n. Basis for exclusion not sharp. Over generalization
	3		Correct Rejections and Excess Misses	Incorporates subsets that produce correct rejections at stage n. Produces misses. Basis for inclusion not sharp. Under generalization
4(0)	4	a with b	New temporary equilibrium (synthesis and new thesis)	New temporary equilibrium (synthesis and new thesis)

Personality and Transition in Performance Table 7

In personality, steps in adults tend to be stable but substeps are not.

Step 0	Fault finders	At low orders of performance in the social domain, this may result in antisocial behavior. These people perceive tremendous unfairness. People get stuck on this here because much of their current order behavior is maintained by “punishment” that reinforces the failure behavior. They have experienced a huge drop in perceived reinforcement rate because they see the failures of the behavior of the present order to obtain what others do. They quite often have unshakable negative depressive scripts.
Step 1	Nay sayers	May consist of persons who enter therapy, rebels, radicals, discontents. They have given up their old ways. If A is wrong, then the opposite of A is right. There is the second largest drop in reinforcement rate because they drop their previous successful behavior from A and substitute behavior from B for it.
Step 2	Relativists	Clinically at its worst, they may be Narcissists (Situationalists) or Borderlines (That nothing works proves no one loves them). In this culture, it is quite often the largest group. They fill academia. They stop progress by insisting that there is more than one way to look at things but cannot decide. There is some gain in reinforcement but the conflict between whether to chose A or B produces anxiety, angst, mood swings and uncertainty about roles, values etc.
Step 3	Movers	They are moving from smash to consolidate. They create great trouble for themselves and others by throwing ideas and actions together in a creative haphazard way, taking a great deal of risk.
Step 4	Unshakables	Everything is OK if it is not OK. They avoid conundrums, apparent contradiction, comparisons to people they look up to. Everything is good enough.

Layers of Contingency Setting

Michael Lamport Commons

Table 8 Talk Radio Example

Michael L. Commons, Ph.D., Patricia Roy

Whose Contingencies	Stated reasons for behavior and for conditioning	Stimulus	Behavior	Outcome
Listener	Received some punishment	Listens to other saying they feel alienated	Calls in	Gets listened to
Talk-Show Host	Money and Axe to Grind	Gets call	Engages Audience, Listen, Comments, Soliloquizes	Audience comment, Money
Producer	Money and Power	Auditions, previous experience, recommendations	Set contingencies for Talk Show Host. Informs Host of Ratings, Problems, Feedback, Studies’ Results	Talk show host behaves better, Station renews show, Producers increased price of package O.K.’D, % to producer
Radio Station Manager, CEO, Board of Directors Lumped here but could be separated	Money, Name recognition, spill over to other shows	Proposal and data from producer	Gives Contract to Producer and Host, Pays producer Advertises Show, Sells time to Advertisers	Ups ratings, Sells more advertising to adjacent shows, gets more money.
Advertising Agency	Contract for Advertising product. More business. More power, money by selling product.	Demographics of Audience, Previous experience with outlet.	Buys Air time, Put on advertisements	Contract renewed if product sales and/or name recognition improves
Company	Increase Sales and Name recognition	Proposal, previous performance, reputation of Agency	Contract with Advertising Agency	Increase in sales, market share, profitability, name recognition
Public	Prefers Good Products and services and lower risk for trying such, prefers to be entertained	Hears advertisement while listening to show	Buys product or Services	“Enjoys” the effects of products or services

Table 9

Classification of organizations, and positons as minimally requiring the following stages.

Stage	Type of Organization Activity	Positions Person	Social Armon Kegan Kohlberg Selman	Social Structure
Primary 7	Individual who works for others and is closely supervised	Cleaning person, Errand runner	2	Individual
Concrete 8	Individual vendors	Individual physical laborer	2/3	Clique
Abstract 9	Family or small number of individual's company	Filing Clerks, Typists, Elementary School Teachers	3	Small Group
Formal 10	Single niche company	Secretaries, Technicians, High School Teachers, Whistle Blowers	3/4	Bureaucracy
Systematic 11	Regular small to large companies and organizations	Managers, Professionals, College Professors	4	Institution
Metasystematic 12	Learning and developing companies and organizations (See Linda Morris)	Innovators & Leaders, Research Scientists and Professors, Major Artists, Appellate Court Judges	5	Universal
Paradigmatic	Organizations that reinvent themselves by adopting new paradigms		6

Table 10: Levels of Reflection of Actions Governed by Implicit and Explicit Rules

Basis for Reasoning	Implicit Rules	Explicit Rules	Tappan's Corresponding analysis as Modified by Commons:
Action & Decision Including Use of Implicit Rules			Use of Someone Else's Rules
Present (Expressed Decision) (Melioration Theory) (No memory)			Action Controlled by someone else past rules & Action is Shadowed (Reflective)
Past Action (Expressed Decisions) (Melioration Theory) Memory			Past Action Description of Actions Guides Action (Reflective)
Prospective Action (Expressed Decision) (Melioration Theory) (Memory applied)			Prospective Action Pragmatic Description of Action Guides Action (Reflective)
Prospective Action With Explanation (Would)			Prospective Action Pragmatic Explanation of With Explanation of Value & Structure in Cognitions (Meta-reflective)
Prescriptive Action (Ought or Should)			Normative: Combination of Value & Structure in Cognitions (3d Order Reflective)
Reflective Actions (How to Prescribe)			Justification of Normative system (4th Order Reflective) Epistemic Cognitions

My current idea is that there is a simpler table. But what kind of reflection and metacognition interacts with stage.

So the problem of refection just generating all the higher stages is maybe not a problem. In some sense they do but once they are automatized they do not require such computational space.

References

Binder, C. (May, 2000). Component/composite analysis and programming from the “real world.” Presented at the Association for Behavior Analysis, Washington D.C.

Bowman, A. K. (1966a). The relationship between organizational work practices and employee performance: Through the lens of adult development. Unpublished doctoral dissertation. The Fielding Institute, Santa Barbara, CA.

Brogden, H. E. (1977). The Rasch model, the law of comparative judgement and additive conjoint measurement. Psychometrika, 42, 631-634.

Campbell, R. L. (1991). Does class inclusion have mathematical prerequisites? Cognitive Development, 6(2), 169-194.

Campbell, R. L., & Bickhard, M. H. (1986). Knowing levels and developmental stages. Human Development, 16 Entire Issue.

Commons, M. L., Goodheart, E. A., & Bresette, L. M. with Bauer, N. F., Farrell, E. W., McCarthy, K. G., Danaher, D. L., Richards, F. A., Ellis, J. B., O'Brien, A. M., Rodriguez, J. A., and Schraeder, D. (1995). Formal, systematic, and metasystematic operations with a balance-beam task series: A reply to Kallio’s claim of no distinct systematic stage. Adult Development, 2 (3), 193-199.

Commons, M. L., Krause, S. R., Fayer, G. A., & Meaney, M. (1993). Atmosphere and stage development in the workplace. In J. Demick & P. M. Miller (Eds.). Development in the workplace (pp. 199-220). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Commons, M. L., Lee, P., Gutheil, T. G., Goldman, M., Rubin, E. & Appelbaum, P. S. (1995). Moral stage of reasoning and the misperceived "duty" to report past crimes (misprision). International Journal of Law and Psychiatry, 18(4), 415-424.

Commons, M. L., & Miller, P. M. (1998). A quantitative behavior-analytic theory of development. Mexican Journal of Experimental Analysis of Behavior. 24(2) 153-180.

Commons, M. L., & Miller, P. A. (2001). A quantitative behavioral model of developmental stage based upon hierarchical complexity theory. Behavior Analyst Today, 2(3), 222-240.

Commons, M. L., & Pekker, A. (submitted). Hierarchical Complexity: A Formal Theory. Journal of Mathematical Psychology.

Commons, M. L., & Richards, F. A. (1984). A general model of stage theory. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development (pp. 120-140). New York: Praeger.

Commons, M. L., & Richards, F. A. (2002). Organizing components into combinations: How stage transition works. Journal of Adult Development, 9 (2): 159-177.

Commons, M. L., Richards, F. A., Trudeau, E. Dawson , T. L., & Goodheart, E. A. (1997, March). The Psychophysics of Stage: The General Stage Model analytically measures hierarchical task complexity and the Rasch Model statistically models corresponding stage of performance. Presented at the International Objective Measurement Workshop, Judd Hall, University of Chicago.

Commons, M. L., Rodriguez, J. A. (1990). “Equal access" without "establishing" religion: The necessity for assessing social perspective-taking skills and institutional atmosphere. Developmental Review, 10, 323-340.

Commons, M. L., Rodriguez, J. A. (1993). The development of hierarchically complex equivalence classes. Psychological Record, 43, 667-697.

Commons, M. L., Trudeau, E. J., Stein, S. A., Richards, F. A., & Krause, S. R. (1998). The existence of developmental stages as shown by the hierarchical complexity of tasks. Developmental Review, 8(3), 237-278.

Commons, M. L., & White, M. S. (2003). A complete theory of tests for a theory of mind must consider hierarchical complexity and stage: A commentary on Anderson and Lebiere target article, The Newell Test for a theory of mind. Behavioral and Brain Sciences, 26, 587–64, 20-21.

Cook-Greuter, S. R. (1990). Maps for living: Ego-development theory from symbiosis to conscious universal embeddedness. In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.). Adult Development: Vol. 2, Comparisons and applications of adolescent and adult developmental models (pp. 79-104). New York: Praeger.

Danaher, D. (1994) Sex role differences in ego and moral development: Mitigation with maturity. Unpublished doctoral dissertation, Harvard University, Cambridge, MA.

Dawson, T. L. (2000). Moral reasoning and evaluative reasoning about the good life. Journal of Applied Measurement, 1, 372-397.

Dawson, T. L. (2002). A comparison of three developmental stage scoring systems. Journal of Applied Measurement. 3(2) 2002, 146-189.

Dawson, T. L., Goodheart, E. A., Draney, K., Wilson, M., Commons, M. L. (1997, March). Concrete, abstract, formal, and systematic operations as observed in a “Piagetian” balance-beam task Series. Paper presented at the annual meeting of the AERA. Chicago.

Dawson, T. L., Commons, M., & Wilson, M. (in review). The shape of development.

Draney, K. L. (1996). The polytomous Saltus model: A mixture model approach to the diagnosis of developmental differences. Unpublished doctoral dissertation, University of California at Berkeley, Berkeley, CA.

Fischer, G. (1968). Psychologische Test Theorie. Bern, Switzerland: Huber.

Green, D. M. & Swets, J. A. (1966). Signal detection theory and psychophysics. Huntington, NY: Krieger.

Goel. A., & Chandrasekaran, B. (1992). Case-Based Design: A Task Analysis In Artificial Intelligence Approaches to Engineering Design, in Tong and D. Sriram (editors). Volume II. Innovative Design, pp. 165-184. San Diego, CA: Academic Press..

Goodheart, E. A., Dawson, T. L., Draney, K., & Commons, M. L. (1997, March). A Saltus analysis of developmental data from the laundry problem task series. Presented at the International Objective Measurement Workshop, Judd Hall, University of Chicago.

Guttman, L. (1950) The basis for Scalogram analysis. In Stouffer et al. Measurement & Prediction, The American Soldier, Vol IV. New York: Wiley.

Holland, J.G. & Skinner, B.F. (1961). The analysis of behavior. New York: McGraw-Hill.

Inhelder, B., & Piaget, J. (1958). The growth of logical thinking from childhood to adolescence. New York: Basic Books.

Kauffman S. A. (1993). The origins of order: self-organization and selection in evolution. New York: Oxford University Press.

Keats, J. A. (1967). Test theory. Annual Review of Psychology, 18, 217-238.

Keats, J. A. (1971). An Introduction to Quantitative Psychology. Sydney, Australia: John Wiley.

Kubina, R. M., & Morrison, R. (2000). Fluency in education. Behavior and Social Issues, 10, 83-99.

Linacre, J. M. (2004). Winsteps: Multiple-Choice, Rating Scale and Partial Credit Rasch Analysis. Chicago, IL: MESA Press.

Mislevy, R. J., & Wilson, M. (1996). Marginal maximum likelihood estimation for a psychometric model of discontinuous development. Psychometrica, 61, 41-47.

Pelton, T. W., & Bunderson, C. V. (2003). The Recovery of the Density Scale using a Stochastic Quasi-realization of Additive Conjoint Measurement. Journal of Applied Measurement. 4 (3) 269-281.

Perline, R., Wright, B.D. & Wainer, H. (1979). The Rasch model as additive conjoint measurement. Applied Psychological Measurement, 3, 237-256.

Rasch, G. (1960). Probabilistic model for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research. Expanded edition, 1980. Chicago: University of Chicago Press.

Rasch, G. (1966). An item analysis which takes individual differences into account. British Journal of Mathematical and Statistical Psychology, 19, 49-57

Rost, J. (2001). The Growing Family of Rasch Models, In A. Boomsma, M. A. J. van Duijn, & T. A. B. Snijders (Eds.). Essays on item response theory. (Pp. 25-42). New York: Springer-Verlag.

Siegler, R. S. (1986). Children's thinking. Englewood-Cliffs, NJ: Prentice-Hall.

Tomasello, M., & Farrar, M. J. (1986). Joint attention and early language. Child Development, 57, 1454-1463.

Wilson, M. (1984). A psychometric model of hierarchical development. Unpublished doctoral dissertation, University of Chicago.

Wilson, M. (1989). Saltus: A psychometric model of discontinuity in cognitive development. Psychological Bulletin, 105, 276-289.

Wolfram, S. (2002). A new kind of science: Notes for Chapter 10: Processes of perception and analysis: Section: defining complexity. (Pp. 1067- 1069). Champaign, IL: Wolfram Media.

Wright, B. D., & Stone, M. H. (1979). Best test design. Chicago, IL: MESA Press.

Adey, P. (In preparation). Promoting the development of formal operations. Journal of Adult Development

Alexander, C. N., Druker, S. M., & Langer, E. J. (1990). Major issues in the exploration of adult growth. In C. N. Alexander & E. J. Langer (Eds.), Higher stages of human development : perspectives on adult growth (pp. 3-34). New York: Oxford University Press.

Anastasi, A. (1982). Psychological testing. 5^th Edition. New York, NY: Macmillan Publishers.

Andrich, D. (1988). Rasch models for measurement. Thousand Oaks, CA: Sage Publications, Inc.

Antell, S., & Keating, D. (1983). Perception of numerical invariance in neonates. Child Development, 54, 695-701.

Arlin, P. K. (1975). Cognitive development in adulthood: A fifth stage? Developmental Psychology, 11, 602-606.

Arlin, P. K. (1977). Piagetian operations in problem finding. Developmental Psychology, 13, 247-298.

Arlin, P. K. (1984). Adolescent and adult thought: A structural interpretation. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development. New York: Praeger.

*Armon, C. (1984). Ideals of the good life and moral judgment: Ethical reasoning across the life span. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development (pp. 357-380). New York: Praeger.

*Armon, C. & Dawson, T. L. (1997). Developmental trajectories in moral reasoning across the life-span. Journal of Moral Education, 26, 433-453.

*Baer, D. M. & Rosales, J. (1994, May). Development cusps: A relevant concept for behavior analysis. Paper presented at the Association for Behavior Analysis, Atlanta, GA.

*Baldwin, J. M. (1906). Thoughts and things or genetic logic (3 Volumes). New York: Macmillan.

*Baldwin, J. M. (1906). Social and ethical interpretations in mental development. New York: Macmillan.

*Baltes, P. B. (1987). Theoretical propositions of life-span developmental psychology: On the dynamics between growth and decline. Developmental Psychology, 23, 611-626.

*Basseches, M. A. (1984). Dialectical thinking as a metasystematic form of cognitive organization. In M. L. Commons, F. A. Richards, and C. Armon (Eds.), Beyond formal operations: Vol. 1.Late adolescent and adult cognitive development. (pp. 216-238). New York: Praeger.

*Benack, S., & Basseches, M. A. (1989). Dialectical thinking and relativistic epistemology: their relation in adult development. In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.), Adult Development: Vol. 1, Comparisons and applications of adolescent and adult developmental models [(pp. ). New York: Praeger.

*Bidell, T. R., & Fischer, K. W. (1992). Beyond the stage debate: Action, structure, and variability in Piagetian theory and research. In, Berg, C. A. & Sternberg, R. J. (Eds.), Intellectual development. New York: Cambridge University Press.

*Binder, C. (2000). Component/composite analysis and programming from the “real world..” Presented at the association for Behavior Analysis, Washington D.C.

*Binet, A., & Simon, T. (1905/1916). The development of intelligence in children. Baltimore: Williams & Wilkins.

Bond, T. G, & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Bowman, A. K. (1966b). Examples of task and relationship 4b, 5a, 5b statements for task performance, atmosphere, and preferred atmosphere. In M. L. Commons, E. A. Goodheart, T. L. Dawson, P. M. Miller, & D. L. Danaher, (Eds.) The general stage scoring system (GSSS). Presented at the Society for Research in Adult Development, Amherst, MA.

*Brainerd, C. D. (1978). Learning research and Piagetian theory. In L. S. Siegel & C. J. Brainerd (Eds.), Alternatives to Piaget, critical essays the theory (pp. ). New York: Academic Press.

Brannon, E. M., & Terrace, H. S. (1998). Ordering of the numerosities 1 to 9 by monkeys. Science, 282, 746-749.

*Brendel, J. M., Kolbert, J. B., & Foster, V. A. (In press). Promoting student cognitive development. Journal of Adult Development

*Broughton, J. M. (1984). Not beyond formal operations but beyond Piaget. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development (pp. 395-411). New York: Praeger.

Brown, E. H. (2004). The "Two Cultures", Mathematics and Postmodernism. September 10, 2004 http://people.brandeis.edu/~buchsbau/FACSEMPAPERS/2Cult.pdf.

*Campbell, R. L., & Bickhard, M. H. (1986). Knowing levels and developmental stages. Contribution to Human Development, 16.

*Campbell, R. L., & Bickhard, M. H. (1987). A deconstruction of Fodor's anticonstructivism. Human Development, 30(1), 4, 8-59.

*Campbell, R. L., & Richie, D. M. (1983). Problems in the theory of developmental sequences: Prerequisites and precursors. Human Development, 26, 156-172.

*Case, R. (1974). Structures and strictures: Some functional limitations in the course of cognitive growth. Cognitive Psychology, 6, 544-573.

*Case, R. (1978). Intellectual development from birth to adulthood: A Neo-Piagetian interpretation. In R. Siegler (Ed.), Children's thinking: What develops? Proceedings of the Thirteenth Annual Carnegie Symposium on Cognition.

*Case, R. (1982). The search for horizontal structures in children's development. The Genetic Epistemologist, 11(3), 1-12.

*Case, R. (1985). Intellectual development: Birth to adulthood. Orlando Florida: Academic Press.

Colby, A., & Kohlberg, L. (1987a). The measurement of moral judgement: Vol. 1. Theoretical foundations and research validation. New York: Cambridge.

Colby, A., & Kohlberg, L. (Eds.) (1987b). The measurement of moral judgment: Vol. 2. Standard form scoring manuals. New York: Cambridge University Press.

*Commons, M. L., & Calnek, A. D. (1984). On the empirical undecidability between the hypotheses that stage change is or is not a discrete, discontinuous, stepwise process. The Genetic Epistemologist, 14(2), 11-16.

*Commons, M.. L., Danaher, D., & Meaney, M. (June, 2000). Transition and stage in performance of Harvard University faculty and staff. Data presented at Society for Research in Adult Development Symposium, Pace University, New York City, New York.

Commons, M. L., & Ducheny, J. R. (1979). The relationship between perceived density of reinforcement in a schedule sample and its reinforcement value. Paper presented at the Second Harvard Symposium on Quantitative Analyses of Behavior: Matching and Maximizing Accounts, Harvard University, Cambridge.

Commons M. L. Goodheart E. A. Danaher D. L. 1995. Scoring individual and organizational material for moral stage using the General Stage Scoring System (GSCSSS) and Rasch analysis. How we score almost any text or discourse in any domain for stage in ½ stages with 95% reliability and transition step. AME, New York.

*Commons, M.L., Goodheart E. A., Danaher, D. L., Miller, P.M., Broderick, M. A., Rodriguez, J.A., Weinstein, G. (1994). Scoring text and discourse and choice for stage and transition using The General Stage Scoring System.

Commons, M. L., Goodheart, E. A., & Dawson T. L. 1997. The general hierarchical complexity scoring system (GHCSS): How to score anything. Scoring individual and organization material for hierarchical complexity of performance using the General Hierarchical Complexity Scoring System and Rasch Analysis: How to score almost any text or discourse in any domain for hierarchical complexity of performance (stage) with 95% reliability and transition steps.

*Commons, M. L., & Grotzer, T. A. (1990). The relationship between Piagetian and Kohlbergian stage: An examination of the "necessary but not sufficient relationship." In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott (Eds.), Adult development: Vol. 2. Models and methods in the study of adolescent and adult thought (pp. 205-231). New York: Praeger.

Commons, M. L., Grotzer, T. A., & Davidson, M. N. (In preparation). The necessity of reinforcing problem solutions for transition to formal operations: An examination of Piaget's equilibration theory of stage change.

*Commons, M. L., & Hallinan, P. W., with Fong, W., & McCarthy, K. (1990). Intelligent pattern recognition: Hierarchical organization of concepts. In M. L. Commons, R. J. Herrnstein, S. M. Kosslyn, & D. B. Mumford (Eds.), Quantitative analyses of behavior: Vol. 9. Computational and clinical approaches to pattern recognition and concept formation (pp. 127-154). Hillsdale, NJ: Lawrence Erlbaum Associates.

*Commons, M. L., Jenisch, J., & Straughn, J. B. (submitted). Operant conditioning and the sense of free will: A response to the libertarian position.

Commons, M. L., Kantrowitz, S., Buhlman, R. A., Ellis, J. B., & Grotzer, T. A. (1984). Immediately detecting a causal-relation stimulus embedded in similarly appearing possible causal relations. In M. L. Commons, M. C. Davison, J. A. Nevin, (Eds), Models of Behavior: Vol. 11, Signal Detection. Hillsdale, NJ: Lawrence Erlbaum Associates.

*Commons, M. L., Krause, S. R., Fayer, G. A., & Meaney, M. (1993). Atmosphere and stage development in the workplace. In J. Demick & P. M. Miller. Development in the workplace (pp. 199-220). Hillsdale, NJ, Lawrence Erlbaum: Associates.

*Commons, M. L., Lee, P., Gutheil, T. G., Goldman, M., Rubin, E. & Appelbaum, P. S. (1995). Moral stage of reasoning and the misperceived "duty" to report past crimes (misprision). International Journal of Law and Psychiatry., 18(4), 415-424.

Commons, M. L., & Miller, P. M. (1998). A quantitative behavior-analytic theory of development. Mexican Journal of Experimental Analysis of Behavior. 24(2) 153-180.

*Commons, M. L., Miller, P. M., & Kuhn, D. (1982). The relation between formal operational reasoning and academic course selection and performance among college freshmen and sophomores. Journal of Applied Developmental Psychology, 3, 1-10.

*Commons, M. L., & Richards, F. A. (1995). Behavior analytic approach to dialectics of stage performance and stage change. Behavioral Development, 5(2), 7-9.

Commons, M. L., & Richards, F. A. (1984a). A general model of stage theory. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development (pp. 120-140). New York: Praeger.

Commons, M. L., & Richards, F. A. (1984b). Applying the general stage model. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development (pp. 141-157). New York: Praeger.

*Commons, M. L., Richards, F. A., & Armon, C. (Eds.). (1984). Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development. New York: Praeger.

*Commons, M. L., Richards, F. A., & Armon, C. (1986). More than a "valuable scorecard." Reply to Keating, D. B. (1986). Beyond Piaget: The evolving debate. Contemporary Psychology, 30(6), 449-450.] Contemporary Psychology, 31(6), 470-471.

*Commons, M. L., Richards, F. A., & Kuhn, D. (1982). Systematic and metasystematic reasoning: A case for a level of reasoning beyond Piaget's formal operations. Child Development, 53, 1058-1069.

Commons, M. L., & Rodriguez, J. A. (1990). "Equal access" without "establishing" religion: The necessity for assessing social perspective-taking skills and institutional atmosphere. Developmental Review, 10, 323-340.

Commons, M. L., Rodriguez, J. A. (1993). The development of hierarchically complex equivalence classes. Psychological Record, 43, 667-697.

*Commons, M. L., Stein, S. A., & Richards, F. A. (April, 1987). A general model of stage theory: Stageof a task. Paper presented at the Society for Research on Child Development, Baltimore, MD.

*Commons, M. L., Strachan, D. D., Rodriguez, J. A., & Barnett, M. A. (1987, January). The effect of moral judgment stage on the utility function form in young adults: The decision to sacrifice one's own grades to keep tutoring commitments. Paper presented at the symposium "From moral action to judgement and back: The relationships between action and stage" in Cambridge, MA.

Commons, M. L., Trudeau, E. J., Stein, S. A., Richards, F. A., & Krause, S. R. (1998). The existence of developmental stages as shown by the hierarchical complexity of tasks. Developmental Review, 8(3). 237-278.

*Commons, M. L., Woodford, M., & Ducheny, J. R. (1982). The relationship between perceived density of reinforcement in a schedule sample and its reinforcing value. In M. L. Commons, R. J. Herrnstein, & H. Rachlin (Eds.), Quantitative analyses of behavior: Vol. 2. Matching and maximizing accounts (pp. 25-78). Cambridge, MA: Ballinger.

*Cook-Greuter, S. R. (1990). Maps for living: Ego-development theory from symbiosis to conscious universal embeddedness. In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.), Adult Development: Vol. 2. Comparisons and applications of adolescent and adult developmental models (pp. 79-104). New York: Praeger.

Coombs, C. H., Dawes, R. M., & Tversky, A. (1970). Mathematical psychology: An elementary introduction. Englewood Cliffs, New Jersey: Prentice-Hall.

Copernicus, N. (1530/1543/1992), On the Revolutions,[De revolutionibus orbium coelestium] edited and translated by E. Rosen, MD, Baltimore: The Johns Hopkins University Press,

*Damen, P. (1977). Piagetian psychology: Cross-cultural contributions. New York: Gardner Press.

*Dasen, P. R. (1980) Psychological differentiation and operational development: a cross-cultural link. The Quarterly Newsletter of the Laboratory of Comparative Human Cognition, 2, 81-86.

Davis, H., & Perusse, R. (1988). Numerical competence in animals: definitional issues, current evidence, and a new research agenda. Behavioral and Brain Sciences, 11, 561-615.

*Dawson, T. L. (2000). Moral reasoning and evaluative reasoning about the good life. Journal of Applied Measurement, 1 (372-397).

*Dawson, T. L. (in press). New tools, new insights: Kohlberg moral reasoning stages revisited. International Journal of Behavioral Development.

Dawson, T. L., Commons, M. L., & Wilson, M. (in review). The shape of development.

Dawson, T. L., Commons, M. L., Wilson, M., & Xie, Y. (1999, June). The General Model of Hierarchical Complexity Scoring System: Refinements and additions. Paper presented at the annual symposium of the Jean Piaget Society, Mexico City, Mexico.

*Demetriou, A. (1990). Structural and developmental relations between formal and postformal capacities: Towards a comprehensive theory of adolescent and adult cognitive development. In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott (Eds.), Adult development: Vol. 2., Models and methods in the study of adolescent and adult thought (pp. 147-173). New York: Praeger.

*Demetriou, A., & Efklides, A. (1985). Structure and sequence of formal and postformal thought: General patterns and individual differences. Child Development, 56, 1062-1091.

*Denton, K. L., & Krebs, D. L. (1990). From the scene of the crime: The effect of alcohol and social context on moral judgement. Journal of Personality and Social Psychology, 59(2), 242-249.

Descartes, R. (1637) Discourse on Method. Cambridge University Press, edited Haldane and Ross; first five parts.

*Dromie, E. (1984). The one word period as a stage in language development: Quantitative and qualitative accounts. In I. Levin (Ed.), Stage and structure (pp. ). Norwood, NJ: Ablex.

*Draney, K. L. (1996). The polytomous Saltus model: A mixture model approach to the diagnosis of developmental differences. Unpublished doctoral dissertation, University of California at Berkeley, Berkeley, CA.

*Ellis, J. B., Commons, M. L., Rodriguez, J. A., Grotzer, T. A., & Stein, S. A. (June, 1987). Immediately detecting a causal relation stimulus embedded in similarly appearing possible causal relations. Paper presented at the Tenth Symposium on Quantitative Analyses of Behavior Held at Harvard: Signal Detection, Cambridge, MA.

*Ennis, R. H. (1978). Conceptualization of childrens' logical competence: Piaget's propositional logic and an alternative proposal. In L. S. Siegel, C. J. Brainard, Alternatives to Piaget: Critical essays on the theory. (pp. ). New York: Academic Press.

Fechner, G. T. (1860/1966). Elements of Psychophysics, vol. I (D. H. Howes and E. G. Boring, eds.), trans. H. E. Adler (New York: Rinehart and Winston, ).

*Erickson, E. H. (1959). Identity and the life cycle. Psychological Issues Monograph, I.

*Erickson, E. H. (1978). Adulthood. New York: Norton.

*Erickson, E. H. (1982). The life cycle completed: A review. New York: Norton.

*Feldman, D. H. (1980). Beyond universals in cognitive development. Norwood, NJ: Ablex.

Fischer, K. W. (1980). A theory of cognitive development: The control and construction of hierarchies of skills. Psychological Review, 87, 477-531.

Fischer, K. W., & Bidell, T. R. (1998). Dynamic development of psychological structures in action and thought. In R. M. Lerner (Ed.), & W. Damon (Series Ed.), Handbook of child psychology: Vol. 1. Theoretical models of human development (5th ed., pp. 467-561). New York: Wiley.

Fischer, K. W., & Bullock, D. (1981). Patterns of data: Sequence, synchrony, and constraint in cognitive development. In K. W. Fischer (Ed.), New Directions for Child Development: Vol. 12. Cognitive Development (pp. 69-78). San Francisco: Jossey-Bass.

Fischer, K. W., Hand, H. H., & Russell, S. (1984). The development of abstractions in adolescence and adulthood. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development (pp. 43-73). New York: Praeger.

Fischer, K. W., & Kenny, S. L.(1986). Environmental conditions for discontinuities in the development of abstractions. In R. A. Mines & K. S. Kitchener (Eds.), Adult cognitive development: Methods and models.(pp. 57-75). New York: Praeger.

Fischer, K. W., Knight, C. C., & Van Parys, M. (1993). Analyzing diversity in developmental pathways: Methods and concepts. In W. Edelstein & R. Case (Eds.), Contributions to Human Development: Vol. 23. Constructivist approaches to development (pp. 33-56). Basel, Switzerland: S. Karger.

Fischer, K. W., & Lazerson, A. (1984). Human development: From conception through adolescence. New York: W. H. Freeman.

*Flavell, J. H. (1981). Monitoring social-cognitive enterprises: something else that may develop in the area of social cognition. In J. H. Flavell & L. Ross (Eds.), Social cognitive development: Frontiers and possible futures (pp. 272-287). New York: Cambridge University Press.

*Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906-911.

*Flavell, J. H. (1971).Comments on Beilin's "The development of physical concepts." In T. Mischel (Ed.). Cognitive development and epistemology. (pp. 121-128). New York: Academic Press.

*Flavell, J. H. (1963). The developmental psychology of Jean Piaget. Princeton: Van Norstrand.

*Gardner, R., Sainato, D. M., Cooper, J. O., Heron, T E., Heward, W. L., Eshleman, J. Grossi, T. A. (1994). Behavior Analysis in Education Focus on Measurably Superior Instruction. California Pacific Grove.

*Gesell, A. L., & Amatruda, C. S. (1964). Developmental diagnosis (2nd ed.). New York: Hoeber.

Gewirtz, J. L. (1969). Mechanisms of social learning: Some roles of stimulation and behavior in early human development. In D. A. Goslin (Ed.), Handbook of socialization theory and research (pp. 57-212). Chicago: Rand McNally. Goodman in preparation.

*Gewirtz, J. L. (1991). Identification, attachment, and their developmental sequencing in a conditioning frame. In J. L. Gewirtz & W. M. Kurtines (Eds.), Intersections with attachment (pp. 247-255). Hillsdale, New Jersey: Erlbaum.

*Gibbs, J. (1977). Kohlberg's stages of moral judgment: A constructive critique. Harvard Educational Review, 47, 43-61.

*Gibbs, J. (1979). Kohlberg's moral stage theory: A Piagetian revision. Human Development, 22, 89-112.

*Gödel, K. (1977). Some metamathematical results on completeness and consistency; On formal undecidable propositions of Principia Mathematica and related systems I; On completeness and consistency. In J. Heijehoort (Ed.), From Frege to Gödel: A Source Book in Mathematical Logic 1879-1931 (pp. 592-617). Cambridge: Harvard University Press, 1977 (Originally published 1930, 1931, 1931, respectively).

Goffman, E. (1963). Behavior in public places; notes on the social organization of gatherings. New York, NY: Free Press of Glencoe.

*Grotzer, T. A. McCarthy, K. G., Frome, K. W., & Commons, M. L. (June, 1987). The process of stage change. Paper presented at Beyond Formal Operations 3: Positive development during adolescence and adulthood. The Development of Adolescent and Adult Thought and Perception, Harvard University, Cambridge.

Halford G. (1999). Developmental of mental representation: theories and applications. Sigel I. (Ed.). Mahwah, NJ, US: Lawrence Erlbaum Associates, Inc. Publishers.

Herrnstein, R. J. (1982). Melioration as behavioral dynamism. In M. L. Commons, R. J. Herrnstein, & H. Rachlin (Eds.), Quantitative analyses of behavior: Vol. 2. Matching and maximizing accounts (pp. 433-458). Cambridge, MA: Ballinger.

Herrnstein, R. J., & Vaughan, W., Jr. (1980). Melioration and behavioral allocation. In J. E. R. Staddon (Ed.), Limits to action: The allocation of individual behavior (pp. 143-176). New York: Academic Press.

*Holland J. G., & Skinner, B. F.(1961). The analysis of behavior: a program for self-instruction. New York, McGraw-Hill, 1961.

*Inhelder, B., & Piaget, J. (1958). The growth of logical thinking from childhood to adolescence: An essay on the development of formal operational structures. (A. Parsons, & S. Seagrim, Trans.). New York: Basic Books (originally published 1955).

*Jackson, M. A., & Malott, R. W. (1994). Helping high-risk Black college students. In R. Gardner, D. M. Sainato, J. O. Cooper, T. E. Heron, W. L. Heward, J. Eshleman, & T. A. Grossi (Eds), Behavior Analysis in Education Focus on Measurably Superior Instruction (pp. 349-363). California Pacific Grove.

Kallio, E. (1995). Systematic reasoning: formal or postformal cognition? Journal of Adult Development, 2, 3, 187-192.

Kallio, E., & Helkaman, K (1991). Formal operations and postformal reasoning: A replication. Scandinavian Journal of Psychology, 32, 1, 18-21.

*Kantrowitz, S., Buhlman, R. A., & Commons, M. L. (April, 1985). Application of signal-detection theory to the measurement of stage of development found in performance on formal-operational isolation-of-variables problems. Paper presented at the annual meeting of the Eastern Psychological Association, Boston, MA.

Karmiloff-Smith, A. (1986). From meta-processes to conscious access: Evidence from children's metalinguistic and repair data. Cognition, 23 95-147.

*Keating, D. B. (1985). Beyond Piaget: The evolving debate. [Review of Beyond formal operations: Late adolescent and adult cognitive development]. Contemporary Psychology, 30(6), 449-450.

*Kegan, R. (1982). The evolving self. Cambridge: Harvard University Press.

King, P. M., Kitchener, K. S., Wood, P. K., & Davison, M. L. (1989). Relationships across developmental domains: A longitudinal study of intellectual, moral, and ego development. Adult Development: Vol. 1. Comparisons and applications of adolescent and adult developmental models (pp. 57-72). New York: Praeger.

*Kitchener, K. S. (1983). Cognition, Metacognition, and epistemic cognition: A three-level model of cognitive processing. Human Development, 26, 222-232.

*Kitchener, K. S., & King, P. M. (1990). Reflective judgement: Ten years of research. In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott (Eds.), Beyond formal operations: Vol. 2. Models and methods in the study of adolescent and adult thought (pp. 63-78). New York: Praeger.

*Klahr, D., & Wallace J. G. (1976). Cognitive development: An information-processing view. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

*Kohlberg, L. (1971). From is to ought: How to commit the naturalistic fallacy and get away with it in the study of moral development. In T. Mischel (Ed.), Cognitive development and epistemology. New York: Academic Press.

*Kohlberg, L. (1981). The philosophy of moral development: Vol. 1. Moral stages and the idea of justice. San Francisco: Harper and Row.

Kohlberg, L. (1984). Essays on moral development: Vol. 2. The psychology of moral development: Moral stages, their nature and validity. San Francisco: Harper & Row.

Kohlberg, L. (1990). Which postformal levels are stages? In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott (Eds.). Adult Development. Vol. 2. Models and methods in the study of adolescent and adult thought (pp. 263-268). New York: Praeger.

*Kohlberg, L., & Armon, C. (1984). Three types of stage models used in the study of adult development. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development (pp. 357-381). New York: Praeger.

*Kohlberg, L., & Ryncarz, R. A. (1990). Beyond justice reasoning: Moral development and considerations of a seventh stage. In C. N. Alexander, & E. J. Langer (Eds.), Higher stages of human development: Perspectives on adult growth (pp. 191-207). New York: Oxford University Press.

Krantz, D. H., Luce, R. D., Suppes, P., & Tversky, A. (1971). Foundations of measurement: Vol. 1. Additive and polynomial representations. New York: Academic.

*Krebs, D. L., Denton, K. L., Vermeulen, S. C., Carpendale, J. I. & Bush, A. (1991). Structural flexibility of moral judgment. (Personalty Processes and Individual Differences). Journal of Personality and Social Psychology, 61(6), 1012-1024.

Kuhn, D., Pennington, N., & Leadbeater, B. (1983).. Adult thinking in developmental perspective: The sample case of juror reasoning. In P. Baltes & 0. Brim (Eds.), Life-span development and behavior, Vol. 5 (pp. ) New York: Academic Press.

*Lahey, L., Souvaine, E., Kegan, R., Goodman, R., & Felix, S. (1982). A Guide to the Subject-Object interview: Its administration and interpretation. Unpublished manuscript available from Robert Kegan, Harvard Graduate School of Education.

Lam, M. S. (1995). Women and men scientists' notions of the good life: A developmental approach. Unpublished doctoral dissertation, University of Massachusetts, Amherst, MA.

*Levinson, D. J. (1986). A conception of adult development. American Psychologist, 41, (pp. 3-13). Lindsay, P. H., & Norman, D. A. (1977). Human information processing: An introduction to psychology, (2nd Edition), New York: Academic Press.

Libet, B. (1985). Unconscious cerebral initiative and the role of conscious will in voluntary action. Behavioral and Brain Sciences, 8:529-566.

Lindsay, P. H., & Norman, D. A. (1977). Human information processing: An introduction to psychology, (2nd Edition), New York: Academic Press.

*Loevinger, J. (1970). A theory of ego development. San Francisco: Jossey-Bass.

*Loevinger, J., & Blasi, A. (1976). Ego development: Conceptions and theories. San Francisco: Jossey-Bass.

*Loevinger, J., & Wessler, R. (1970). Measuring ego development. San Francisco: Jossey- Bass.

Lovell, C. W. (1999). Development and disequilibration: Predicting counselor trainee gain and loss scores on the Supervisee Levels Questionnaire. Journal of Adult Development.

Luce, R. D. (1959). On the possible psychological laws. The Psychological Review, 66(2) 81-95.

Luce, R. D. (1963). Detection and recognition. In R. Luce & R. Bush, (Eds.), Handbook of Mathematical Psychology (pp. 103-190). New York: Wiley.

Luce, R. D., Krantz, D. H., Suppes, P., & Tversky, A. (1990). Foundations of measurement: Vol. 3. Representation, axiomatization, and invariance. San Diego, CA: Academic.

Luce, R. D. & Tukey, J. W., (1964). Simultaneous conjoint measurement. Journal of Mathematical Psychology, 1, 1-27.

*Luchins, A. S., & Luchins, E. H. (1965). Logical foundations of mathematics for behavioral scientists. New York: Holt, Rinehart and Winston.

McAuliffe, G. J. (In press). Student changes, program influences, and adult development in one program of counselor training: An exploratory inductive inquiry. Journal of Adult Development.

*Miller, P. M., Lee, S. T., & Commons, M. L. (June, 2000). Scoring for stage and transition of losses in children and adult’s lives. Data presented at Society for research in Adult Development Symposium, Pace University, New York City, New York.

*Miller, P. M., &. Lee, S. T. (June, 2000). Stages and transitions in child and adult narratives about losses of attachment objects. Paper presented at the Jean Piaget Society. Montreal, Québec, Canada.

Mislevy, R. J., & Wilson, M. (1996). Marginal maximum likelihood estimation for a psychometric model of discontinuous development. Psychometrica, 61, 41-47.

*Moerk, E. L. (1992). A first language taught and learned. Baltimore, MD: Paul H. Brookes Publishing

Moore, R., & Goldiamond, I. (1964). Errorless establishment of visual discrimination using fading procedures. Journal of the Experimental Analysis of Behavior, 7, 269-272.

*Munsinger, H. (1970). Light detection and pattern recognition: Some comments on the growth of visual sensation perception. In L. R. Goulet & P. B. Baltes (Eds.), Life-span developmental psychology (pp. 228-247). New York: Academic.

*Newell, A., Shaw, J. C., & Simon, H. A. (1964). Elements of a theory of human problem- solving. In R. J. C. Harper, C. C. Anderson, C. M. Christensen, & S. M. Hunda (Eds.), The cognitive processes: Readings (pp. 339-356). Englewood Cliffs, NJ: Prentice Hall.

*Newell A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.

Overton, W. F. (1990). Reasoning, necessity, and logic: Developmental perspectives. Hillsdale, NJ: Lawrence Erlbaum Associates.

*Pascual-Leone, J. (1970). A mathematical model for the decision rule in Piaget's developmental stages. Acta Psychologica, 32, 301-345.

*Pascual-Leone, J. (1984). Attention, dialectic, and mental effort: Towards an organismic theory of life stages. In M. L. Commons, F. A. Richards, & C. Armon (Eds.), Beyond formal operations: Vol. 1. Late adolescent and adult cognitive development (pp. 182-215). New York: Praeger.

Pepperberg, I. M. (1987). Evidence for conceptual quantitative abilities in the african grey parrot: labeling of cardinal sets. Ethology, 75, 37-61.

*Perry, W. G. (1968, 1970). Forms of intellectual and ethical development in the college years: A scheme. New York: Holt, Rinehart and Winston.

*Perry, W. G. (1981). Cognitive and ethical growth: The making of meaning. In A. Chickering (Ed.),

The modern American college (pp. 76-116). San Francisco: Jossey-Bass.

*Piaget, J. (1953). Logic and Psychology. (W. Mays and F. Whitehead, Trans.). New York: Basic Books.

Piaget, J. (1954). The construction of reality in the child. (M. Cook, Trans.). New York: Ballantine Books.

Piaget, J. (1976). The grasp of consciousness: Action and concept in the young child. Cambridge: Harvard University Press.

*Piaget, J., & Inhelder, B., with Sinclair-de Zwart, H. (1973). Memory and intelligence. A. Pomerans (trans.). New York: Basic Books.

*Piaget, J., Inhelder, B. & Szeminska, A. (1960) The child's conception of geometry. New York: Basic Books.

Rasch, G. (1960/1980). Probabilistic model for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research. Expanded edition, 1980. Chicago: University of Chicago Press.

*Rest, J. R., Turiel, E., & Kohlberg, L. (1969). Level of moral development as a determinant of preference and comprehension of moral judgement made by others. Journal of Personality, 37, 225-252.

*Richard, D. C., Unger, C. M., & Commons, M. L. (1988, April 22). Strategies as knowledge: Subjects' methods of determining causality during the shift from concrete to formal operations. Paper presented at the annual meeting of the Eastern Psychological Association at Buffalo. Friday, April 22, 1988, 8:20 am (Available from the Dare Institute, 234 Huron Avenue, Cambridge, MA 02138).

*Richards, F. A., & Commons, M. L. (1990b). Applying signal detection theory to measure subject sensitivity to metasystematic, systematic and lower developmental stages. In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, and J. D. Sinnott (Eds.), Adult Development: Vol 2. Models and methods in the study of adolescent and adult thought (pp. 175-188). New York: Praeger.

*Riegel, K. F.(1973).Dialectic operations: The final phase of cognitive development. Human Development, 16: 346-370.

Rodriguez, J. A. (1989). Exploring the notion of higher stages of social perspective taking. Unpublished Qualifying Paper, Harvard Graduate School of Education.

Rodriguez, J. A. (1992). The adult stages of social perspective-taking: assessment with the doctor-patient problem. Unpublished doctoral dissertation, Harvard Graduate School of Education, Cambridge, MA.

*Rosales, J., & Baer, D. M. (in press). A behavior-analytic view of development. In E. Ribes & S. W. Bijou (Eds.), Recent approaches to behavioral development. Guadalajara, Jal. México: Editorial, Universidad de Guadalajara.

*Rosales-Ruiz, J. (May, 2000). Divide and teach. Presented at the Association for Behavior Analysis, Washington D.C.

*Rosales-Ruiz, J., & Baer, D. M..(1997). Behavioral cusps: A developmental and pragmatic concept for behavior analysis. Journal of Applied Behavior Analysis. 30(3), 533-544.

Rosenberg, M. (1979). Conceiving the self. New York: Basic Books.

*Scandura, J. M. (Ed.) (1977). Problem solving: A structural/process approach with instructional implications. New York: Academic Press.

*Schrader, D. E. (1988). Moral metacognition and its relation to moral judgment: A description and analysis. Unpublished doctoral dissertation, Harvard University.

*Schrader, D. E. (1987, June 30-July 4). Overview of moral development. Presented at the annual Moral Judgment Interview Scoring Workshop, Harvard graduate School of Education, Cambridge, MA (Available from the author, Cornell University).

*Selman, R. L. (1980). The growth of interpersonal understanding. New York: Academic Press.

*Siegler, R. S. (1981). Developmental sequences within and between concepts. Monographs of the Society for Research in Child Development, 46, (Whole No. 189).

*Siegler, R. S. (1986). Children's thinking, What develops? Englewood Cliffs, NJ: Prentice Hall.

Skinner, B. F. (1931). The concept of the reflex in the description of behavior. Journal of General Psychology, 5, 427-458.

*Skinner, B. F., & Vaughan, M. E. (1983). Enjoy old age: A program of self-management. New York: W. W. Norton.

*Spada, H., & McGraw, B. (1985). The assessment of learning effects with linear logistic test models. In S. E. Embretson (Ed.), Test design. Developments in psychology and psychometrics. Orlando, FL Academic Press.

*Sonnert, G., & Commons, M. L. (1994). Society and the highest stages of moral development. Politics and the Individual, 4(1), 31-55.

Sonnert, G., & Commons, M.L. (1994). Society and the highest stages of moral development. The Individual and Society, 4(1), 31-55.

*Stein, S. A., & Commons, M. L. (May, 1987). Verbal description and explanation appear lower in stage than nonverbal performance on a formal operational task. Paper presented at the meeting of the Jean Piaget Society, Philadelphia, PA.

Strauss, M., & Curtis, L. (1984). Development of numerical concepts in infancy. In C. Sophian (Ed.), The origins of cognitive skills. Hillsdale, NJ: Erlbaum.

Suppes, P., Krantz, D. H., Luce, R. D., & Tversky, A. (1989). Foundations of measurement: Vol. 2. Geometrical, threshold, and probabilistic representations. San Diego, CA: Academic

Swan, T. & Benack, S. (2002). Narcissism in the Epistemological Pit. Journal of Adult Development, 9(3).

*Tappan, M. B. (1990). The development of socio-moral cognition during young adulthood: A three-dimensional model. In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.), Adult Development: Vol. 2. Comparisons and applications of adolescent and adult developmental models. New York: Praeger.

*Terman, L. M., & Merrill, M. A. (1937). Measuring intelligence. Boston: Houghton Mifflin.

*Terrace, H. S. (1961). Discrimination learning with and without "errors". Unpublished doctoral dissertation, Harvard University, Cambridge, MA.

Terrace, H. S. (1963). Discrimination learning with and without "errors." Journal of the Experimental Analysis of Behavior, 6, 1-27.

*Tappan, M. B. (1990a). The development of justice reasoning during young adulthood: A three-dimensional model. In M. L. Commons, C. Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer, & J. D. Sinnott (Eds.), Adult development: Vol. 2. Models and methods in the study of adolescent and adult thought (pp. 235-248). New York: Praeger.

*Tappan, M. B. (1990b). The development of socio-moral cognition during young adulthood: A three-dimensional model. In M. L. Commons, J. D. Sinnott, F. A. Richards, & C. Armon (Eds.), Adult Development: Vol. 2. Comparisons and applications of adolescent and adult developmental models. New York: Praeger.

*Terman, L. M., & Merrill, M. A. (1937). Measuring intelligence. Boston: Houghton Mifflin.

*Terrace, H. S. (1961). Discrimination learning with and without "errors". Unpublished doctoral dissertation, Harvard University, Cambridge, MA.

Terrace, H. S. (1963). Discrimination learning with and without "errors." Journal of the Experimental Analysis of Behavior, 6, 1-27.

Van Geert, P. (1991). Theoretical problems in developmental psychology. In P. Van Geert, & L. P. Mos (Eds.), Annals of Theoretical Psychology, 7 (pp. 1-54). New York: Plenum Press.

Van Loosbroek, E. & Smitsman, A. (1990). Visual perception of numerosity in infancy. Developmental Psychology, 26, 916-922.

Vygotsky, L. S. (1962) [or (1986) rpt.]. Thought and language. Cambridge, MA: MIT Press.

Vygotsky, L. S. (1966). Development of the higher mental function. In Psychological Research in the U.S.S.R. (pp. 44-45). Moscow: Progress Publishers.

*Walker, L. J. (1986). Experiential and cognitive sources of moral development in adulthood. Human Development, 29(2), 113-124.

*Weinreb, N., & Brainerd, C. H. (1975). A developmental study of Piaget's Groupment Model of the emergence of speed and time concepts. Child Development, 46, 176-185.

*Weir, A. A., Chappell, J., Kacelnik, A. (2002). Shaping of hooks in New Caledonian crows. Science. 9;297(5583): 981.

*Wexler, K. A. (1982). Language acquisition: The state of the art. Cambridge: At the University Press.

Wilson, M. (1989). Saltus: A psychometric model of discontinuity in cognitive development. Psychological Bulletin, 105, 276-289.

*Wolfsont, C. (June, 2000). The brain gym and scoring adult developmental transition and stage. Data presented at Society for research in Adult Development Symposium, Pace University, New York City, New York.

Young, F. W. (1972). A model for polynomial conjoint analysis algorithms. In R. B. Shepard, A. K. Romney, & S. B. Nerlove (Eds.), Multidimensional scaling. Theory and applications in the behavioral sciences. (69-104). New York: Seminar Press

Zheng Yan, Z, Fischer, K. W. (2002). Always Under Construction. Dynamic Variations in Adult Cognitive Microdevelopment, Human Development 45(3):141-160

Adey, in press 7, 9, 12, 18

Anatasi, 198 22

Antell & Keating, 1983 46

Aristotle (384€322) 2

Arlin (1975, 1977, 1984 68

Arlin (1975, 1977, 1984) 68

Arlin (1975, 1984 68

Arlin (1984) 39

Armon (1984) 39

Armon & Dawson 6

Armon & Dawson, 1997 6

Armon, 1990 6

Basseches (19 39

Benack (1994) 39

Bond (1999, June, personal comm.) 40

Bowman, 1996a; 1996b 6

Brannon & Terrace, 1998 51

Brendel, Kolbert, & Foster, this issue 18

Broughton (1977; 1984) 40

Cheryl Armon, 1984a, 1984b 6, 37

Cheryl Armon, 1993 6, 37

Colby and Kohlberg's (1987) 20

Commons & Ducheny 1979 22

Commons & Miller 2004 6, 37

Commons & Miller, 1998 18, 48

Commons & Pekker, submitted 45

Commons & Richards (1984) 39

Commons & Richards, 1984a, 1984b 5, 14

Commons & Rodriguez, 1990 6, 11

Commons & Rodriguez, 1990, 1993 5, 6

Commons & Rodriguez, 1990; 14

Commons & Rodriguez, 1990; 1993) 6

Commons & Rodriguez, 1993 14

Commons & Wolfsont, 2002 6, 37

Commons and Miller, in press 6, 37

Commons, et. al., 2004 6

Commons, Goodheart, & Bresette, 1995) 6

Commons, Goodheart, & Bresette, 1995) 37

Commons, Goodheart, and Dawson ( 1995; 1997 7

Commons, Grotzer and Davidson (in preparation) 48

Commons, Grotzer, and Davidson (in preparation 15

Commons, in preparation 6, 37

Commons, in preparation) 37

Commons, Kantrowitz and Buhlman, 1984 21

Commons, Kantrowitz, Buhlman, Ellis, Grotzer, in preparation 71

Commons, Krause, Fayer, & Meaney, 1993 6

Commons, Lee, Gutheil, et. al., 1995) 6, 37

Commons, Miller and Kuhn (1982 48

Commons, Pekker, & Adams, in press). 42

Commons, Richards & Kuhn, 1982 6, 42

Commons, Richards & Kuhn, 1982); 44

Commons, Rodriguez, Cyr, Gutheil 37

Commons, Rodriguez, Cyr, Gutheil et. al., in preparation 6

Commons, Trudeau et al 1998 45

Commons, Trudeau, Stein, Richards, & Krause, 1998) 5

Cook-Greuter, 1990 6

Cook€Greuter (1990) 40

Cook€Greuter, 1990) 37

Coombs, Dawes, & Tversky, 1970 5

Copernicus (1530) 4

Davis & Perusse, 1988 45

Dawson (1998) 40

Dawson & Commons, in preparation 19

Dawson, 1998 6, 37

Dawson, 2000 6

Dawson, Commons, & Wilson, 1999, June 19

Dawson, Commons, & Wilson, in review; Dawson, Commons, & Wilso 19

Demetriou (1990; 1995) 40

Descartes (1637 4

DeVos & Commons, unpublished manuscript 6

Dickenson and McGonigle (1996) 45

Edger Brown (2004) 1

Fechner (1860 4

Fischer (1984 68

Fischer (personal communication 18

Fischer & Kenny 1986 11

Fischer & Kenny, 1986 12

Fischer & Lazerson, 1984 11, 12

Fischer & Lazerson, 1984 11, 48

Fischer and Lazerson (1984 68

Fischer, 1980 11, 12

Fischer, 1980; 1986; 1984 12

Fischer, Hand, & Russell (1984) 39

Funk, 1990 6

G. T. Fechner (1860) 4

Gewirtz (1969b 68

Gewirtz, 1969 68

Goffman, 1967 71

Gray (1999, June, personal comm.) 40

Halford, 1999 11

Herrnstein & Vaughan, 1980 17

Herrnstein, 1982 17

Inhelder & Piaget (1958) 39

Inhelder & Piaget, 1958 6

Inhelder and Piaget (1958) 48

Kallio (1991, 1995) 40

Karmiloff-Smith, 1986 13

Kegan (1994) 40

King & Kitchener (2002) 40

King & Kitchener, 1989 12

Kitchener & Fischer, 1990) 6

Kitchener & King, 1990 6

Kohlberg (1981) 39

Kohlberg (1985) 71

Kohlberg, 1984 71

Kohlberg, 1990 5

Koplowitz (1984) 40

Krantz, Luce, Suppes, and Tversky, 1971 4

Kuhn, Pennington, and Leadbeater (1983 44

Labouvie€Vief (1984) 39

Lam, 1994 6

Lamborn, Fischer & Pipp, 1994 37

Lamborn, Fischer & Pipp, 1994) 6

Libet's, 1985 4

Lindsay & Norman, 1977 5

Loevinger (1998) 40

Lovell, 2004 6, 37

Lovell, this issue 18

Lowenthal, 1989 2

Luce, Krantz, Suppes, & Tversky, 1990 4

McAuliffe, this issue 18

Meredith Root-Bernstein (Personal communication, January 18, 2 49

Miller & Cook-Greuter, 2000 6

Miller & Cook€Greuter, 2000 37

Miller & Lee, 1998) 6

Mislevy & Wilson, 1996 18

Nicholas Commons€Miller, in preparation 37

Oliver, 2004 6, 37

Overton, 1990 6

Pascual€Leone (1983) 39

Piaget (1954 5

Piaget, 1973 5

Piaget, 1976 13

Powell (1984) 39

Rasch (1980 42

Rodriguez, 1989 44

Rosenberg (1979 16

Rosenberg, 1979 16

Sinnott (1984) 39

Sonnert & Commons (1994) 39

Sonnert & Commons, 1994 6, 37

Sternberg (1984) 39

Strauss & Curtis, 1981 46

Suppes, Krantz, Luce & Tversky, 1989 4

Swan & Benack, 2002 15

Tappan, 1990 12

Thales (640 € 546 1

Torbert (1994) 40

van Geert, 1991 12

van Loosbroek & Smitsman, 1990) 46

Vygotsky (1966a; 1966b 68

Vygotsky's (1966a, 1966b 68

Wilson, 1989 18, 19

¹Loevinger table contents as modified by Commons (present volume).

Order of Hierarchical Complexity	Name	Example
0	Calculatory	Simple Machine Arithmetic on 0's and 1's
1	Sensory & Motor	Either seeing circles, squares, etc. or instead, touching them. O #
2	Circular Sensory- Motor	Reaching and grasping a circle or square. O #
3	Sensory-Motor	A class of filled in squares may be formed # # # # #
4	Nominal	That class may be named, “Squares”
5	Sentential	The numbers, 1, 2, 3, 4, 5 may be said in order
6	Preoperational	The objects in row 5 may be counted. The last count called 5, five, cinco, etc * * * * * # # # # # O O O O O # / "} Q
7	Primary	There are behaviors that act on such classes that we call simple arithmetic operations 1 + 3 = 4 5 + 15 = 20 5(4) = 20 5(3) = 15 5(1) = 5
8	Concrete	There are behaviors that order the simple arithmetic behaviors when multiplying a sum by a number. Such distributive behaviors require the simple arithmetic behavior as a prerequisite, not just a precursor 5(1 + 3) = 5(1) + 5(3) = 5 + 15 = 20
9	Abstract	All the forms of five in the five rows in the example are equivalent in value, x = 5. Forming class based on abstract feature
10	Formal	The general left hand distributive relation is x * (y + z) = (x * y) + (x * z)
11	Systematic	The right hand distribution law is not true for numbers but is true for proportions and sets. x + (y * z) = (x * y) + (x * z) x (y z) = (x y) (x z)
12	Metasystematic	The system of propositional logic and elementary set theory are isomorphic x & (y or z) = (x & y) or (x & z) Logic x (y z) = (x y) (x z) Sets T(False) φ Empty set T(True) Ω Universal set

Order or Stage	What they do	How they do it	End result
0	calculatory	Exact–no generalization	Human made program manipulate 0, 1	None
1	sensory & motor	Discriminate in a rote fashion, stimuli generalization, move	Move limbs, lips, eyes, head View objects and movement	Discriminative and conditioned stimuli
2	circular sensory-motor	Form open-ended classes	Reach, touch, grab, shake objects, babble	Open ended classes, phonemes
3	sensory-motor	Form concepts	Respond to stimuli in a class successfully	Morphemes, concepts
4	nominal	Find relations among concepts Use names	Use names and other words as successful commands	Single words: ejaculatives & exclamations, verbs, nouns, number names, letter names
5	sentential	Imitate and acquire sequences Follows short sequential acts	Generalize match-dependent task actions. Chain words	Pronouns: my, mine, I; yours, you; we, ours; they, them
6	preoperational	Make simple deductions Follows lists of sequential acts Tell stories	Count random events and objects Combine numbers and simple propositions	Connectives: as, when, then, why, before; products of simple operations
7	primary	Simple logical deduction and empirical rules involving time sequence Simple arithmetic	Adds, subtracts, multiplies, divides, counts, proves, does series of tasks on own	Times, places, counts acts, actors, arithmetic outcome from calculation
8	concrete	Carry out full arithmetic, form cliques, plan deals	Does long division, follows complex social rules, takes and coordinates perspective of other and self	Interrelations, social events, what happened among others, reasonable deals,
9	abstract	Discriminate variables such as Stereotypes; logical quantification; (none, some, all)	Form variables out of finite classes Make and quantify propositions	Variable time, place, act, actor, state, type; quantifiers (all, none, some); categorical assertions (e.g. “We all die ")
10	formal	Argue using empirical or logical evidence Logic is linear, 1 dimensional	Solve problems with one unknown using algebra, logic and empiricism	Relationships are formed out of variables; words: linear, logical, one dimensional, if then, thus, therefore, because; correct scientific solutions
11	systematic	Construct multivariate systems and matrices	Coordinates more than one variable as input Consider relationships in contexts	Events and concepts situated in a multivariate context; systems are formed out of relations; systems: legal, societal, corporate, economic, national
12	metasystematic	Construct multi-systems and metasystems out of disparate systems	Create supersystems out of systems Compare systems and perspectives Name properties of systems: e.g. homomorphic, isomorphic, complete, consistent, commensurable	Supersystems and metasystems are formed out of systems of relationships
13	paradigmatic	Fit metasystems together to form new paradigms	Synthesize metasystems of	Paradigms are formed out of multiple metasystems