Chapter 19

Measuring Treatment

Merle Canfield

During the past 50 years there has been a steady increase in the number of schools or styles of psychotherapy. In 1984 Karasu, T., et. al. reported 418 systems. At the same time there has been a moderate increase in the elements of psychotherapy. In recent years there has been a trend to merge these systems with common factors. It is the purpose of this section to show how research designs empirically support the process of sorting out the characteristics of these system. At the same time the designs should support the further elaboration and search for the elements of psychotherapy. Frank (1971) proposed that there were common factors in addition to specific factors in psychotherapy that might be related to outcome (see also Parloff, 1986).

In an attempt to determine whether the treatment has been implemented three approaches have been used: (1) developing therapy manuals, (2) labeling or coding psychotherapy as it unfolds, and (3) rating the process by the use of scales. In 1979 Russell and Stiles reviewed the coding systems and attempted to devise a taxonomy and resulting coding system that would include all elements of the existing coding systems. They generated a logical or rational taxonomy. Although their task was different they did attempt do develop a taxonomy of the psychotherapeutic elements. Many of the taxonomies that have been developed have been developed for specific style of school of therapy. We are proposing methods to perform empirical taxonomies, or a combination of judgments and empirical

These methods can be used in to ways: (1) develop a taxonomy of the styles of therapy, or (2) develop a taxonomy of the elements of therapy. Probably both of these would be useful. If both were developed they would complement each other so that identifying a particular style or school of therapy would be a matter of selecting a set of elements of therapy. The techniques are similar for the two areas. Finally, modes other than psychotherapy are presented.

For example, of the 400 different schools what is the overlap and how would one determine the overlap between the schools? It would be useful to identify the overlap or common factors.

What are the interactions that would place a therapeutic interaction within a specific school and separate the interaction from other schools (unique factors)? What therapeutic interactions overlap with other schools (common factors). There are two major tasks to be accomplished if one is to make such discriminations. The first task is to be able to identify and measure (either by counting or by assessing some degree) of the client/therapist interactions. If that can be accomplished the second task is to indicate the taxonomy of performances that each of the styles need. That is, a profile of the style in terms of the performances is needed. The most fruitful method of identifying these performances has been to code the utterances of the psychotherapy process.

There are four statistical methods that might be used for this process: (1) cluster analysis, (2) discriminant function analysis, (3) multidimensional scaling, and (4) factor analysis. Four basic therapy processes discussed are: (1) psychotherapy, (2) group therapy, (3) ancillary therapies, and (3) milieu therapy. The literature indicates that the descriptive or taxonomy process has be accomplished most the psychotherapy, next with group therapy, next the ancillary therapies and finally milieu is the least identified.

Trochim (19--) and _______________ used a combination of cluster analysis and multidimensional scaling to develop maps of attitudes of toward organizations. It is proposed here that the same method could be used to build a taxonomy of the elements of psychotherapy.

In this example participants were asked to identify processes or characteristics of psychotherapy that they thought were curative. The following is that list (along with an abbreviated name):

Develop insight INSIGHT

desensitize DESENS

reflect REFLECT

introspection INTROSP

develop trust DEVTRUST

reframe REFRAME

acceptance ACCEPT

interpret INTERP

being consistent CONSIT

being nurturing BEINGNUR

address anxiety ADDRESA

correct faulty cognition CORRECT

try new behaviors TRYBEHAV

challenge CHALLENG

set limits SETLIMIT

help cope HLPCOPE

define expectations DEFEXP

demythetize DEMYTH

counter transference CONTRAN

be a good mom BGDMOM

identify conflicts IDCONFL

These statements were put on slips of paper and the participants were asked to place them into stacks. They were instructed that there must be fewer stacks than slips of paper and there must be more than one stack. Once these stacks were created the information was transferred to a coding sheet in the following manner (the coding sheet is on the following page). Assume that ACCEPT, DEVTRUST, BEINGNUR, and BGDMOM were placed in the same stack. Marks would be place on the coding sheet at the intersection of all of these pairs. Note that there is a mark where DEVTRUST intersects with DEVTRUST, ACCEPT, BEINGNUR, AND BGDMOM. Again there is a mark where ACCEPT intersects with DEVTRUST, ACCEPT, BEINGNUR, and BGDMOM. The same procedure is performed for BEINGNUR and BGDMOM. The coding sheet has the marks filled in for this one stack (DEVTRUST, ACCEPT, BEINGNUR and BGDMOM). The same sheet would be used to complete the remaining stacks.

Twenty-four participants completed the task of sorting the items and completing the tally sheets. The cells of a summary sheet were then completed by counting the number of participants who had a check (or one (1)) in each in the corresponding cell. That data is presented in Frame CURET.DBF the labels across the top are not part of the file. The tallies are the number students who raised their hand when the cells were identified. The tallies are actually an estimate of the number of hands raised when they were more than about 5. The cells now give an indication of the similarity of the items or labels for the cell. For example, the cell in Figure __ identified by REFRAME and REFLECT is 12 indicating that 12 of the respondents put those two items in the same stack. That indicates a moderate to high similarity of the items. The cell labeled CHALLENG and DEVTRUST has a 0 indicating that none of the respondents put those two items in the same stack and therefore judge them to be dissimilar. Consequently, a high score indicates similarity and a low score indicates dissimilarity. The upper right triangle and lower left of the triangle are identical. The estimates were in fact not identical (because of errors in estimation) but the computer program requires and the lower left was used to duplicate the upper right.

Figure 1. A coding sheet for recording .....

NAME

INSIGHT

DESENS

REFLECT

INTROSP

DEVTRUST

REFRAM

ACCEPT

INTERP

CONSIT

BEINGNUR

ADDRESA

CORRECT

TRYBEHAV

CHALLENG

SETL

IMI

T

HLPCOPE

DEFEXP

DEMYTH

CONTRAN

BGDMOM

IDCONFL

INSIGHT

DESENS

REFLECT

INTROSP

DEVTRUST

1

REFRAM

ACCEPT

1

INTERP

CONSIT

BEINGNUR

1

ADDRESA

CORRECT

TRYBEHAV

CHALLENG

SETLIMIT

HLPCOPE

DEFEXP

DEMYTH

CONTRAN

BGDMOM

1

IDCONFL

Table 2. Representation of data base file CURET.DBF (in dBase IV format).

NAME

INSIGHT

DESENS

REFLECT

INTROSP

DEVTRUST

REFRAM

ACCEPT

INTERP

CONSIT

BEINGNUR

ADDRESA

CORRECT

TRYBEHAV

CHALLENG

SETL

IMI

T

HLPCOPE

DEFEXP

DEMYTH

CONTRAN

BGDMOM

IDCONFL

INSIGHT

24

6

10

2

6

1

8

0

1

5

1

4

2

3

7

2

6

5

1

7

DESENS

6

24

4

0

5

0

2

1

0

12

5

6

4

6

3

6

4

1

5

REFLECT

6

4

24

6

3

12

5

8

4

3

2

4

2

3

8

3

2

6

INTROSP

10

4

6

24

3

5

3

12

1

2

4

2

4

3

2

0

7

5

2

4

DEVTRUST

2

0

3

24

2

18

1

12

14

1

2

1

0

4

3

1

12

2

REFRAM

6

5

12

5

2

24

3

4

5

2

5

7

2

5

3

9

2

4

ACCEPT

1

0

5

3

18

3

24

1

12

15

0

3

1

4

1

3

1

0

13

0

INTERP

8

2

8

12

1

4

1

24

1

0

2

5

2

4

3

2

7

1

3

CONSIT

0

1

4

1

12

5

12

1

24

9

1

4

1

0

7

2

6

0

9

0

BEINGNUR

1

0

4

2

14

2

15

0

9

24

0

3

1

3

1

3

1

0

15

0

ADDRESA

5

12

4

1

5

0

2

1

0

24

6

7

5

3

6

5

0

7

CORRECT

1

5

3

2

5

3

5

4

3

6

24

4

6

4

7

4

6

4

2

TRYBEHAV

4

6

2

4

1

5

1

2

1

7

4

24

9

6

7

3

4

2

1

4

CHALLENG

2

6

4

3

0

7

1

4

0

1

5

6

9

24

3

8

4

8

7

0

4

SETLIMIT

3

4

2

4

2

4

3

7

3

5

4

6

3

24

2

8

5

1

3

4

HLPCOPE

7

6

3

2

3

5

1

2

1

5

7

8

2

24

3

4

2

7

DEFEXP

2

3

0

3

2

6

3

4

3

4

8

3

24

8

1

3

6

DEMYTH

6

8

7

3

9

1

7

0

1

6

4

8

5

4

8

24

4

1

9

CONTRAN

5

4

3

5

1

2

0

7

0

5

6

2

7

1

2

1

4

24

0

6

BGDMOM

1

2

12

2

13

1

9

15

0

4

1

0

3

2

3

1

0

24

1

IDCONFL

7

5

6

4

2

4

0

3

0

7

2

4

7

6

9

6

1

24

The first method used to develop a taxonomy is cluster analysis. It should be remembered that this process is a descriptive process and not hypothesis testing. The purpose is to describe the relative position of one element to another. The result of cluster analysis is a distance indicator of one element to another. Frame CURCLS1.SPS is a jobstream for SPSS, Frame CURET.DBF contains the data in the dBase IV file that the jobstream will use.

File Name = curcls1.sps

get file = '\proeval\curet.sav'/keep=

NAME INSIGHT DESENS REFLECT INTROSP DEVTRUST REFRAM

ACCEPT INTERP CONSIT BEINGNUR ADDRESA CORRECT TRYBEHAV

CHALLENG SETLIMIT HLPCOPE DEFEXP DEMYTH CONTRAN BGDMOM

IDCONFL .

cluster insight to idconfl

/id=name

/print=distance

/print=schedule cluster(9)

/plot=dendrogram hicicle.

┌───────────────────────────────────────────────────────────────────────────┐

│ CURCLS1.LIS │

├───────────────────────────────────────────────────────────────────────────┤

│Dendrogram using Average Linkage (Between Groups) │

│ │

│ Rescaled Distance Cluster Combine │

│ │

│ C A S E 0 5 10 15 20 25 │

│ Label Seq +‑‑‑‑‑‑‑‑‑+‑‑‑‑‑‑‑‑‑+‑‑‑‑‑‑‑‑‑+‑‑‑‑‑‑‑‑‑+‑‑‑‑‑‑‑‑‑+ │

│ │

│ DEVTRUST 5 ‑+‑‑‑+ │

│ ACCEPT 7 ‑+ +‑‑‑‑‑+ │

│ BEINGNUR 10 ‑‑‑+‑+ +‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑+ │

│ BGDMOM 20 ‑‑‑+ | | │

│ CONSIT 9 ‑‑‑‑‑‑‑‑‑‑‑+ | │

│ SETLIMIT 15 ‑‑‑‑‑‑‑‑‑‑‑‑‑+‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑+ | │

│ DEFEXP 17 ‑‑‑‑‑‑‑‑‑‑‑‑‑+ | | │

│ DESENS 2 ‑‑‑‑‑+‑‑‑‑‑‑‑‑‑‑‑‑‑+ | | │

│ ADDRESA 11 ‑‑‑‑‑+ +‑+ | | │

│ TRYBEHAV 13 ‑‑‑‑‑‑‑‑‑‑‑‑‑+‑+ | | +‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑+ │

│ CHALLENG 14 ‑‑‑‑‑‑‑‑‑‑‑‑‑+ +‑‑‑+ +‑‑‑+ | │

│ HLPCOPE 16 ‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑+ | | | │

│ CORRECT 12 ‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑+ | | │

│ REFLECT 3 ‑‑‑‑‑‑‑+‑‑‑‑‑‑‑‑‑‑‑+ +‑‑‑+ │

│ REFRAME 6 ‑‑‑‑‑‑‑+ +‑‑‑+ | │

│ DEMYTH 18 ‑‑‑‑‑‑‑‑‑‑‑‑‑+‑‑‑‑‑+ | | │

│ IDCONFL 21 ‑‑‑‑‑‑‑‑‑‑‑‑‑+ +‑+ │

│ INTROSP 4 ‑‑‑‑‑‑‑+‑‑‑‑‑+ | │

│ INTERP 8 ‑‑‑‑‑‑‑+ +‑‑‑‑‑‑‑+ | │

│ INSIGHT 1 ‑‑‑‑‑‑‑‑‑‑‑‑‑+ +‑+ │

│ CONTRAN 19 ‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑‑+ │

└───────────────────────────────────────────────────────────────────────────┘

The frame CURCLS1.LIS contains part of the output from the CURCLS1.SPS computer run. The horizontal axis of the dendrogram represents distance between the variables listed on the vertical axis. Moving to the right indicates greater distance. A plus (+) indicates that two variables have joined to form a cluster. In the diagram DEVTRUST and ACCEPT were the first to join (when moving from left to right) are the most similar. The next pair to join are BEINGNUR and BGDMOM indicating they are next pair in close proximity. The next pair to join are DEFEXP and DESENS. The next joining is not a pair of variables but the joining of two clusters; the cluster formed by DEVTRUST and ACCEPT is joined with BEINGNUR and BGDMOM. The final joining (the further to the right) represents the joining of two clusters that are the most distant. One of the clusters is made up of DEVTRUST, ACCEPT, BEINGUUR, and BGDMOM and the cluster to join it is made up of all of the other variables. The method proposed for determining the number of clusters is to find the greatest horizontal distance where no variables or clusters join and draw a vertical line. All clusters that have formed up to that line are considered to be clusters. In the Figure that would be line A. That is, there are no joinings between about 15 and 25; there is no other distance that great when no variables or clusters join. Using that criteria there are two clusters in this solution since there are two clusters to the left of line A. This solution is not very satisfying theoretically. Many of the elements in cluster two seem different it does not help our taxonomy to combine them all in a single cluster. Like factor analysis there is a second method for determining the number of clusters and that is interpretability. Further, we are not testing hypotheses but building a taxonomy. The next greatest distance when no joinings occur is at line B. That vertical line intersects 9 horizontal lines indicating that 9 clusters have be formed up to that point. The 9 clusters are presented along with the cluster names.

1. Intrapshychic

INSIGHT

INTROSP

INTERP

2. Anxiety

DESENS

ADDRESA

3. Give Feedback

REFLECT

REFRAME

4. Warmth

DEVTRUST

ACCEPT

CONSIT

BEINGNUR

BGDMOM

5. Correct

CORRECT

6. Directive

TRYBEHAV

CHALLENG

HLPCOPE

7. Set Limits

SETLIMIT

DEFEXP

8. ??

DEMYTH

IDCONFL

9. Countertransference

CONTRAN

This solution appears to give a better taxonomy than does the first solution. Cluster 1 INSIGHT, INTROSP, and INTERP would appear to similar type of therapist interventions; REFLECT and REFRAME are similar and so forth. There are two clusters that contain single items and they do not seem to belong to any of the clusters that exists.

Although there is some indication in the dendrogram of the distance between clusters it does not give a graphic picture. For example, in the 9 cluster solution the distance between cluster REFLECT and REFRAME and the cluster SETLIMIT and DEFEXP is not readily apparent. Is that distance about the same or much greater than the distance between DESENS and ADDRESA and the cluster SETLIMIT and DEFEXP?

The method of multidimensional scaling offers a more graphic picture of the distance between variables. The following jobstream uses the same set of data as that used in the cluster analysis. The task requests a three dimension solution.

File Name = curcls3.sps

get file = '\proeval\curet.sav'/keep=

NAME INSIGHT DESENS REFLECT INTROSP DEVTRUST REFRAM

ACCEPT INTERP CONSIT BEINGNUR ADDRESA CORRECT TRYBEHAV

CHALLENG SETLIMIT HLPCOPE DEFEXP DEMYTH CONTRAN BGDMOM

IDCONFL .

als var = insight to idconfl

/level=ordinal(similar)

/criteria=dimensions(3)

/plot=all.

The weights for each item on the three dimensions are presented in Frame CURALS3.LST.

┌─────────────────────────────────────────────────────────────────────────┐

│ CURALS3.LST │

├─────────────────────────────────────────────────────────────────────────┤

│ Dimension 1 Dimension 2 Dimension 3 │

├─────────────────────────────────────────────────────────────────────────┤

│BEINGNUR ‑2.2475 INTROSP ‑1.5347 CORRECT ‑1.2737 │

│ACCEPT ‑2.1976 INTERP ‑1.4272 CONTRAN ‑1.1266 │

│BGDMOM ‑2.1645 REFLECT ‑1.1540 CHALLENG ‑0.9879 │

│CONSIT ‑2.1478 CONTRAN ‑1.0630 REFRAME ‑0.8759 │

│DEVTRUST ‑1.8399 INSIGHT ‑0.9176 HLPCOPE ‑0.6125 │

│DEFEXP ‑0.4503 ACCEPT ‑0.4117 TRYBEHAV ‑0.4709 │

│SETLIMIT ‑0.4328 DEVTRUST ‑0.3511 BGDMOM ‑0.2777 │

│CORRECT ‑0.1885 BEINGNUR ‑0.3222 INTROSP ‑0.2690 │

│REFLECT ‑0.0692 DEMYTH ‑0.2027 DESENS ‑0.1460 │

│REFRAME 0.0569 REFRAME ‑0.1758 BEINGNUR ‑0.0709 │

│INTROSP 0.3780 BGDMOM ‑0.0736 ADDRESA ‑0.0234 │

│HLPCOPE 0.5617 IDCONFL 0.0142 INTERP ‑0.0207 │

│TRYBEHAV 1.0587 CONSIT 0.4554 ACCEPT 0.0023 │

│DEMYTH 1.0670 CORRECT 0.5163 CONSIT 0.0217 │

│INTERP 1.0758 CHALLENG 0.6593 REFLECT 0.3745 │

│IDCONFL 1.1449 ADDRESA 0.7462 DEVTRUST 0.6676 │

│INSIGHT 1.1548 DESENS 0.8300 DEMYTH 0.7658 │

│CHALLENG 1.1846 SETLIMIT 0.9680 INSIGHT 0.8362 │

│CONTRAN 1.2479 DEFEXP 1.0765 SETLIMIT 1.1014 │

│ADDRESA 1.3813 HLPCOPE 1.1151 IDCONFL 1.1337 │

│DESENS 1.4265 TRYBEHAV 1.252? DEFEXP 1.2520 │

└─────────────────────────────────────────────────────────────────────────┘

It should be noted that this is not direct output from the SPSS run CURALS3.SPS, each dimension has been arranged from the most negative weight to the most positive weight. Dimension 1 has at one pole BEINGNUR, ACCEPT, BGDMOM, CONSIT, and DEVTRUST while the other pole is DESENS, ADDRESA, CONTRAN, CHALLENG, and INSIGHT. This dimension seems to be warmth (possibly emotional) to relearning (possibly cognitive). Dimension 2 has at one pole INTROSP, INTERP, REFLECT, CONTRAN, and INSIGHT; at the other pole is TRYBEHAV, HLPCOPE, DEFEXP, and SETLIMIT. The continuum seems to go from intrapsychic understanding to a directive or didactic approach. The third dimension has CORRECT, CONTRAN, CHALLENG, and REFRAME at one pole and DEFEXP, IDCONFL, and SETLIMIT at the other pole. Dimensions 1 and 2 have been plotted in the next figure while dimensions 1 and 3 have been plotted in the subsequent figure.

The figure gives a graphic picture of the distance between cluster 1 (from the previous calculation; DEVTRUST, ACCEPT, BEINGNUR, and BGDMOM) and cluster 2 (SETLIMIT and DEFEXP). It also shows the distance between cluster 1 and cluster 3 (DESENS and ADDRESA; the variable CHALLENG is added to this cluster). Further, the distance between cluster 2 and cluster 3 is presented in this graphic. It is important to remember that the task as presented here is not to test theory but develop taxonomies (in a sense to develop theory). The task is to help the researcher visualize (understand) the complexities of the relationships among the variables.

Multidimensional scaling provides information beyond cluster analysis as presented here. The two dimensions represented in the circumplex provides to additional bits of information: (1) distance between the clusters (and individual variables) and (2) where along each of dimensions each variable and cluster lies. Although the dendrogram in cluster analysis does provide information of the distance between cluster 1 (INSIGHT, INTROSP, and INTERP) and cluster 3 (REFLECT and REFRAME) it is a much clearer in the circumplex model of multidimensional scaling. Further, one can readily note the relation to other clusters.

Multidimensional scaling is not limited to two dimensions, like factor analysis there can be as many dimensions as their are variables and in the same manner that there can be as many factors as there are variables. Unlike factor analysis the methods of determining the number of dimensions is not as advanced as is the method for determining the number of factors. As multidimensional scaling is presented here that is not a problem

One could think of these 21 elements being used to describe a school or style of psychotherapy. In a simplified form psychoanalysis might be thought of as made up of interpretation, transference and countertransference, and working through.

This set of statistics can be used on a range of taxonomic or descriptive problems. The creation of the input matrix determines the issue studied. The method presented here combined the data from a panel as described by Trochim. This process assists the clinician in sorting out their judgments. However, a single clinician could fill in the above chart by making judgments of the similarity of the pairs (zero might represent similar--or no difference while 8 might represent a great difference). In the cell identified by ACCEPT (acceptance) and BEINGNUR (being nurturing) the judgment might be 1 (quite similar). The cell identified by DEVTRUST and CHALLENG might be judged 6 (quite dissimilar). The same set of statistics could then be computed on the matrix of this single clinician. This would result in a map of the clinician. Such maps could be used be used in comparing theories. Students could be compared to a panel of experts. These methods could be used to empirically support the judgements of clinicians.

Personality Theory Rating Scale

Name: _________________________________________ Date: ________________

Use the scale below to rate the personality theory of ____________________________.

╔═══════════════════════════════════════════════════════════════════════════╗

║ None A Little Somewhat Quite a Bit A Lot ║

╟───────────────────────────────────────────────────────────────────────────╢

║ 0 1 2 3 4 5 6 7 8 ║

╚═══════════════════════════════════════════════════════════════════════════╝

╔══════════════════════╗

║LEAVE THE QUESTION ║

║BLANK IF YOU DON'T ║

║KNOW OR IF IT DOESN'T ║

║APPLY. ║

╚══════════════════════╝

ACCORDING TO THIS THEORY:

_____ ...motivation is based on drive reduction.

_____ ...the person is an intentional (goal-oriented) being.

_____ ...people are hedonistic.

_____ ...cognition accounts for the actions of people.

_____ ...values account for the actions of people.

_____ ...people are actively involved in the development of their personality.

_____ ...people's early experiences influence their personality.

_____ ...the person imposes perception on the world.

_____ ...the environment or learning accounts for the person's actions.

_____ ...people are basically good.

_____ ...heredity effects the person's actions.

_____ This theory stresses the individual's conscious view of the world.

_____ This theory stresses the individual's unconscious view of the world.

_____ This theory stresses the individual's social consciousness.

_____ This theory accounts for the individual's perception of reality.

_____ This theory has influenced psychology (clinical, research, literature).

_____ This theory focus on "the here and now", the past, or the future.

(0 = past, 4 = here and now, 8 = future)

_____ This theory is empirically based.

_____ This theory is parsimonious.

_____ This theory assumes that the individual has free choice.

_____ This theory employs a method of therapeutic intervention.

_____ This theory emphasizes psychopathology.

_____ I agree with this theory.

The names for the respective items are as follows:

TDATE

THER

THID

CLUS

DRIVE

GOAL

HEDON

COG

VALUE

ACTIVE

EARLY

IMPOSE

LEARN

GOOD

HERED

CONSCI

UNCONS

SOCIAL

PERCEP

INFLU

TIME

DATA

PARSI

FREE

THERA

PATH