Essay Writing Service http://StudyHub.vip/Assessing-Therapeutic-Problem-Solving-S

1. Scandinavian Journal of Psychology, 1993, 34, 27-38 zyxwv
Assessing therapeutic problem solving skills: Empirical
analysis of a measuring operation
JOHN A. RMNNING' and KNUT A. HAGTVET2 zyxwvu
'Department of General Psychology, University of Bergen, Bergen, Norway
Department of Pediatrics, Regional Hospital, Tromsfl,Norway
Rwning, J. A. zyxwvutsrq
& Hagtvet, K. A. (1993): Assessing therapeutic problem solving skills:
Empirical analysis of a measuring opcration. Scandinavian Journal of Psychology, 34,21-38.
Employment of caregivers zyxwvuts
as trainers is an imperative in the training and treatment of
mentally retarded persons. Besides being taught specific behavior modification skills, these
paraprofessional trainers have to obtain a set of general skills in order to adjust their specific
skills to new clients, situations, and behaviors across time. Empirical research on the
characteristics of therapeutic problem-solving skills are scarce. This may be due to a
reluctance of behavior researchers against including cognitive factors in the analysis of
behavior influence. However, the development of basic knowledge on the existence of
therapeutic problem-solving skills should be one of the primary research goals. This article
dekribes the development and application of four scales assumed to be representative of an
important set of general skills-therapeutic problem-solving skills. By means of these four
scales 21 caregivers were evaluated before and after extensive training. A multifaceted
measuring operation is employed with the main objective of empirically testing the dimen-
sionality of problem-solving skills. Rater consistency and individual differences in change
from pre- to post-training level are also uncovered. It is concluded that the four scales of
therapeutic skills, largely reflect the same underlying concept of a rather general therapeutic
problem-solving skill. Discussion of the findings also pointed to future research.
Key words: Assessment, specific skills, general skills, generalization and maintenance, thera-
peutic problem-solving skills, multifacet analysis.
John A. Running, Regional Centre zyxwvuts
for Child and Yourh Psychiatry, ksgdrdsveien 86, zyxw
9OCQ Troms0
Today it is accumulated enough empirical evidence to conclude, that a necessary condition
for developing appropriate and reducing inappropriate behaviors of mentally retarded
persons is to use their caregivers as trainers and therapists (e.g. Kazdin, 1973; Lmaas et zyx
al., 1973; R~nning
& Thorsen, 1984;Thompson & Grabowski, 1972; Whitman et al., 1983).
The caregivers have to be taught specific skills in how to train and treat the mentally
retarded persons. They have also to acquire a set of general skills in order to adjust their
specific skills to new clients, situations and behaviors, and to maintain their skills across
time.
For both specific and general skills it is necessary to assess the caregivers level of
skill-acquisition, in order to evaluate the effect of training, and in order to know when they
can continue on their own without extensive training and supervision by professionals.
Concerning specific skills there are developed instruments for deciding level of skill-acquisi-
tion which are empirically tested (e.g. Watson, 1972). This is not the case with regard to
the general skills. Although at present their empirical characteristics seem unclear, the
necessity of developing a set of general therapeutic skills in the caregivers are repeatedly
underscored by clinical practitionars (e.g. Heppner, 1978; Whitman et at., 1983).
One set of skills which zyxwvuts
is usually thought of as general skills, and often applied in direct
therapeutic encounters, e.g. alcoholics, (Intagliata, 1978); antisocial child behavior (Kazdin zy
efal., 1987), is called problem-solving skills. Researchers within an explicit problem-solving

2. 28 zyxwvutsrqp
J. A . zyxwvutsrqpo
RBnning and zyxwvut
K. zyxwvu
A. Hagtuet zyxwvu
Scand J Psycho1 3
4zy
( zy
1993) zy
framework have traditionally postulated the existence of several “stages” in the problem-
solving process (e.g. D’Zurilla & Goidfried, 1971; Goldfried & Goldfried, 1976). Five stages
are common to most models of problem-solving: general orientation, problem definition,
generation of alternatives, decision making, and evaluation.
The process of behavior therapy can, in general, be traced along five similar steps, each
step consisting of several specific skills:
1. Formulation of the problem (this means that the staff is able to describe the problem in
behavioral terms).
2. Analysis (this means that the staff can pinpoint probable releasing and/or maintaining
conditions of the problem).
3. Formulation of treatments (this means that the staff can formulate some treatment
strategies on different levels to cope with the problem).
4. Choice of treatments (this means that the staff can combine the analysis of releasing
and/or maintaining conditions of the problem with the knowledge of treatments, and
formulate a realistic and manageable treatment streategy for the specific case).
5. Evaluation (this means that the staff is aware of the importance of evaluation, and has
knowledge of methods to formally evaluate the effects of the treatment strategy chosen).
Although some evidence has suggested that problem-solving is a function of different
activities (Johnson et al., 1968; Spivack & Shure, 1974), no research, as far as we know, has
empirically investigated the existence of these stages. It is also unclear whether there are
components underlying the applied problem-solving process, and whether the process is most
accurately described in terms of distinct stages, or components that cut across stages. The
ideology of behavior modification underscores the necessity of operating with conceptual
unities which are descriptive and behavioral. General skills, on the other hand, are difficult
to define in behavioral descriptive terms.
As a point of departure we consider these problem-solving skills to be underlying operative
components in every aspect of the therapeutic process. Experience has made us conscious of
four components or general types of skills typical for those staff members who become able
behavior modifiers. Lack of these skills or requirements, on the other hand, seem to
characterize those staff members who don’t perform well in the behavior modification
programs. We will conceptualize these skills in the following way:
1) Initiative-the person is able to independently carry out the five-step process as described
above.
2) Systematics-the person is able to focus at one aspect of the information at a time, and
then put this information together in a broader perspective.
3) Specificity-the person is able to present the information in concrete terms, describe
strategies, and refer to theories and research with direct relevance for the presented
case.
4) Analytical reasoning (creativity)-the person is able to analyse the problem in light of
the information presented, to transcend from daily work situation, analyse frame condi-
tions and detect practical limitations and constraints. and with regard to this, describe
treatment strategies which can be realized.
These skills or requirements are important within all the five before mentioned problem-solv-
ing stages or stages of behavior therapy. We have chosen to call the set of these characteris-
tics “therapeutic problem-solving skills”.
The present study reflects upon the existence and nature of therapeutic problem-solving
skills, and presents a way of measuring these skills. A meaningful first step to investigate the

3. Scand zyxwvutsrqponm
J Psycho1 34 zyxwvutsrqp
( 1993) zyxwvutsrqp
Assessing therapeutic problem solving skills 29 zy
present concept of therapeutic problem-solving skill is to penetrate its dimensional nature.
Therefore the primary purpose of this study is to examine if the four therapeutic problem-
solving skills referred to above may be conceptualized as different indicators of a general
therapeutic problem-solving skill.
METHOD
Subjects
Twenty-one staff members assigned to three wards in an institution for mentally retarded served as
subjects. None of the staff in these three units had ever received training based on behavioral principles.
Their initial interaction with the residents, when it took place, was characterized by activation, stopping
and custodial care.
The resident population, with whom they worked, all belonged to the severely mentally retarded. They
demonstrated few self-help skills such as eating, dressing, undressing and personal hygiene. On the whole
there existed a lack of social skills, such as mutual interaction, eye contact, sharing etc. The majority
displayed high rates of inappropriate behaviors, especially repetitious (self-stimulating), aggressive and
self-destructive behaviors. zyxwvuts
Training procedure
Each group of staff was given 300 hours of training within a period of three months. The training
consisted of goal delineation, demonstration of actual treatmentltraining work in the real situations,
feedback on staff training skills, training in registration and graphical displaying. Theoretical and
problem-solving training were continually integrated in the practical training, but also given separately
in three daily seminars. For a more complete description of this training procedure, the reader is referred
to Rmning & Thorsen (1984, 1986a, 1986b).
Construction of a cue-point scale
A scale for each of the four characteristics of therapeutic problem solving skills was then developed. zyx
A
differentiation and specification of each scale was undertaken. This was accomplished by establishing
three anchors (Nunnally, 1978) on each scale by which the staff's verbal behavior could be compared.
These anchors represent different levels of each scale (cf. Figs. I, 2, 3, and 4).
The anchors were established in the following way: Each scale was represented by a horizontal line of
18 zyxwvutsrqpo
cm.Four experienced clinical psychologists independently provided three descriptions representing
three different levels on each scale. Each description was then discussed until consensus was reached.
Then the four clinicians independently placed the three descriptions on the appropriate scale along the
horizontal line of 18 cm.If the discrepancy between each description on the specific scale was less than
1 cm,a final placement was reached by using the median of the single placements. A discrepancy greater
than 1 cm initiated a discussion which was ended by an agreement on the final placement of the anchor.
By this procedure a cue-point scale was constructed for each of the four components. The anchors on
each scale were operationally defined by specific descriptions. These descriptions were placed on a scale
where the end-points are defined as the absolute minimum and maximum of this category of behavior.
The anchors and the end-points of the scale, are the cue-points (or reference points) for scoring the
interview (cf. Figs. 1-4). The descriptions of the different cue-points are intended to make the raters
more attentive to the same aspects of the verbal behavior to which they are listening. The component
analytical reasoning (cf. Fig. 4) is somewhat more comprehensive than the other three. As a helpful
device to score the performance along this scale, the descriptions of cue points are divided into four
points-A, B, C and D.
The interviews
In order to evaluate the staffs therapeutic problem-solving skills, a semi-structured interview on a
hypothetical case was administered before and after training with an interval of six months. The pre-test
assessment took place a few days before the training period was initiated, while the post-test assessment
took place three months after closing the training period. The interview covered the five phases of the
above mentioned behavior therapy process considered to be important in designing and conducting a
treatment/training program. The interviewswere performed by two of the clinicians who had constructed

4. 30 zyxwvutsrqp
J. A. zyxwvutsrqpo
RBnnh~
and zyxwvut
K.A . Hagtvet zyxwvut
Scand J Psycho1zy
34 zyx
( 1993)
Cm m i a t e furtherfromherown answerswhen being asked.
b not &pendent u p questions Brings up new quatiom andzyxwvuts
points otview by herowp
iFig. zyxwvutsrqp
1. Initiative:
the cue-point scale. The interviews were taped with a casette recorder. The two other clinicians who had
participated in the construction of the scale. scored all the interviews independently of each other. The
tapes were only identified by numbers, and the tapes from pre- and post-intervention were presented to
the raters in random order. When all the interviews were scored along the four scales the scores were
transformed to an interval scale with 9 intervals, each of 2 cm.
Data-analytic models
The above measuring procedure encompassed three facets: a) pre- and post-training measures, b) four
scales of therapeutic problem-solving skills, and c) two independent raters of all the measures. These
features were combined in an entirely crossed three facet structure (cf. Cronbach er al., 1972). Thus,
altogether 16 variablcs were incorporated in the present measuring Operation. This implies that the
present measuring design will provide information about, (a) the dimensionality of the present therapeu-
tic problem-solving skills, (b) the stability of the dimensionality pattern across training occasions, (c)
rater consistency and (d) stability in individual differences from pre- to post-training occasions. The use
of different but equivalent raters is essential for estimating the reliability of the different linear
combinations entering into the measuring operation.
Associated with this measuring design the data were analyzed by means of a correlationally oriented
analysis of variance of which different features have been described and discussed by Bock (1960),
Crocker & Algina (1986), Cronbach er al. (1972), Eikeland (1972, 1973). Eisenhart ( 1947) and Winer
(1962). The present application of analysis of variance is in particular described and discussed by
Hagtvet (1989) and illustrated by Hagtvet & Halpern (1992). In the present context it is important to
. emphasize that due to the small number of individuals included in the present study, a mathematical
descriptive application of ANOVA will be used. Nevertheless, the question of relative importance of
identifiable linear combinations and their reliability/generalibility will be addressed.

5. -
- zyx
Failsto put diffmnt p
r
t
s
zyxwvu
of the information togetherin a cogent manner, zyxwvut
and pnwnls the information in a blurry a d haph.zardly
way.zyxwvutsrqp
Usesonly pul of h e relevantinfomahi.
- Uses a p a t deal of the relevantinfoornution. but dependson help from the interviewer to delimit h e different p ~ a
- Uses all zyxwvutsrqpo
of the revelant information, discusses each part at it's time in a structured way, and is able to yt
the different parts of information in a broader perspective.
7
Since our concern is to describe individual differences in therapeutic problem-solving skills, the
observed mean squares in the present analysis of variance for person and the interactions in which
person occurs are of primary interest. Only these sources of variation with their associated latent
(expected) variance structures are reported in Table 1. The observed mean squares for the person-ori-
ented sources of variation are at the outset decomposed into their respective latent components of
variance within a complete random model.
The relative importance of the eight mean squares, each representing one or more linear combinations,
is indicated by the descriptive percentaged trace components in Table 1.
RESULTS
Table 1 clearly reports that the general component is by far the most important component,
accounting for about 68 per cent of the trace of the associated 16 by 16 covariance matrix
of the variables in the design (cf. Hagtvet, 1989).
Furthermore, independently of the general component the person by occasion (PO)
component accounts for about 11 per cent of the total variance which represents a far weaker
contribution but nevertheless is a recognizable finding. This component simply reflects the
relative impact of individual differences in change of the general problem-solving skill level
across training occasions. On average, the group improved over the interval of six months.
Further inspection of data revealed the frequent finding of a negative correlation zyxw
(r = -0.51)
between pre-training level and the extent of improvement across the six month period (cf.
Rogosa et al., 1982). It should be noted that the present measuring design does not allow any
causal interpretation related to the individual differences in change from pre- to post-mea-
sures. It allows only a descriptive measure of change.

6. 32 zyxwvutsrqp
J. A. RBnning and zyxwvut
K.A. Hagtuet Scand J Psycho1zy
3
4zy
( zy
1993)
-
Makesspecificreferencest
o the presentinformationandzyxwvu
uses behavioralterms i
n an adequate manner.
Describesstrategies preciselyand concretely.
Refersto theoriesarid research with d k ! relevanceforthe prsent case.
t
L
Fig. 3. Specificity.
The small contribution from the person by rater (PR) component indicates that the two
raters display close agreement throughout the measuring operation indicating acceptable
inter-rater reliability. Estimated by the Spearman-Brown formula (Ferguson & Takane,
1989) the inter-rater reliability at the pre-scores vaned from 0.89 to 0.93 for the different
scales. The corresponding figures for post-scores were 0.81 and 0.93, Likewise, the small
contribution from the person by skill dimension (PD) component indicates a relative weak
support for the impact of differentiality aspects among the four scales. No further informa-
tion of interest seem to be associated with the remaining components.
Since the trace measures of relative importance referred to above are influenced by
measurement errors, like in ordinary principal components, it is important to estimate the
reliability/generalizabilityof the components of some importance, before they are considered
worthwhile of substantive interpretation. Furthermore, the question of measurement errors
or reliability in the present context requires decision about considering the levels or
conditions of the three facets fixed or random (Cronbach zyxw
et al., 1972). It may be a moot
question whether the four included aspects of therapeutic problem-solving skills are exhaus-
tive of potential aspects belonging to this type of skill. A realistic approach suggests that the
included four scales listed above represent important and essential aspects of these types of
therapeutic problem-solving skills. This reasoning suggests that the Dimension facet would
operate as a random facet for the present generalizability estimates. For the purpose of

7. Scand zyxwvutsrqponm
J Psycho1 34zyxwvutsrqpo
( 1993) zyxwvutsrqpo
Assessing therapeutic problem solving skills zy
33
- zyx
A. Poalydevelopedstillsin analyzingtheproblemin light of the pmcnt infomYtioe I
s na able tot-nd b
u daily working
situation zyxwvutsrqpo
8. Poorlydevelopedstillsin analyzingthe fnmr cmditioru, pndiul limiutiom a d conslrainmin an imtitutiond settins and With
regard10 t
h
i
s drsribe V c a M d Sntegies which u n be rulizcd.
C R d U r i ~ ~ l
a d thoughtfulappracherto t
k problemam a i m aknL .
D. Indiutiomand WM&odiutiom f a the zyxwvutsrqp
use ol I tlcrunmt m
k
g
y am a k a
-
A. Average developedskills in analyzing the problem in light of the present information. Is able ta tnnwcnd her daily working
situation to a moderate degree.
B.Avenge developedskillsin analyzingthe fnmc mnditiopr. pracliul limiutiom a d ccaslraintsin an inaitulimdsetting, a d With
regardl o t h i d&be UcaUIIcntslrakgia which a n be malimd.
C
.Prrsents fewpdaional and thoughtfulsppmchu to the problem
D
.Resentsfew indiutiom a d coaurindiutiom f a the we da vealment hruegy.
-
A. Highly developedskillsin alulydngIkc problemin light ofthe pmenl informuionl
shighly uprblc 10transcendh a daily working
situation
B. Highly developed stills i
n mrlyling the fnmc d i t i m a p
n
d
u
l limitationsand coaaninm in an imtitdmd rcmng. a d zyxw
With
reprd 10 Ihi.ducribe (rulmcnt sntegiu which u n be realized
C
.PRIcnls a lot of profcopimal and thoughtfula p p r 0 K k tothe problem
D.Rescnts many indiutiom a d mwraindiutiolu f a the uu d I tnrtment stntcgy
I
L
Fig. 4. Analytical reasoning (creativity).
comparison, however, generalizabilityestimates will also be considered under the assumption
of a fixed universe of dimensions. Given these two assumptions we have two models for
estimating generalizability coefficients for the most important linear combinations derived
from the present measuring operation. For both models occasions in the present study are
considered fixed since they were not chosen to represent anything beyond themselves. Raters
Table 1. Threefacet analysk of therapeutic problem-solving skill
Trace components
Source Df. Mean sq. Nc Comp "
h Latent variance structure-random model
P
PO
PR
POR
PD
PDO
PDR
PDOR
20
20
20
20
60
60
60
60
~ ~~
48.690
7.831
3.210
I .556
1.515
1.018
0.445
0.374
1 48.690
1 7.831
1 3.210
I 1.556
3 4.545
3 3.054
3 1.335
3 1.122
68.25
10.98
4.50
2.18
6.37
4.28
1.87
I .57
Total 16 71.343 100.00
Nc = the number of linear combinations going into each mean square. P = person; 0 = occasion (pre-post);
D =dimension; R = rater. For each component a letter 0,r. d and p is assigned, respectively, which is used to
designate a source of variation used as a subscript. and to designate the number of levels of the source when used as
a weight. Concerning the rationale for writing the observed and latent (expected) mean squares, the reader is referred
to Winer ( 1962)or Millman & Glass ( 1970).Trace components are the diagonal elements of the corresponding 16 x 16
covariance matrix of the variables included in the measuring design which sum up to Trace -71.343.

8. 34 zyxwvutsrqp
J. A. h i n g and zyxwvut
K.A . Hagtvet zyxwvu
Scand J Psycho1 34zy
( zy
1993)
Table 2. Estimation zyxwvutsrqp
o
f generalizability coeficients under zyxwvu
two mixed model assumptions
Variance components
Mixed-I Mixed-I1
Model
Source Mean squares p PO Pd P PO Pd
P 48.690
PO 7.831
PR 3.210
POR 1.556
PD 1.515
P W 1.018
PDR 0.445
PDOR 0.374
Generalizability coefficient:
45.480
2.836
6.275
1.182
1.070
0.071
0.374 0.374 0.374
0.93 0.80 0.71
44.410
5.631
2.765
1.182
1.070 1.070
0.644
0.071 0.071
0.374 0.374 0.374
0.91 0.72 0.71
~ ~ ~
Mixed-I: D =fixed, 0=fixed, R =random; Mixed-11: D =random, 0 =fixed, R =random. For other
symbols, the reader is referred to Table 1.
are considered random due to the very idea of raters representing equally competent raters
from a larger pool of potential raters. Thus the one model, called Mixed-I, considers
Dimension and Occasion as fixed and Raters as random, while the Mixed-I1 model assumes
Dimensions to be random, otherwise being identical to Mixed-I model. Given these two sets
of assumptions, generalizability estimates will be provided for both models.
Generalizability estimates are derived from a decomposed structure of variance components
as reported in Table 1. However, depending upon the assumptions associated with each model
the latent structure of the random model (cf. Table 1) is changed accordingly (cf. Millman
8c Glass, 1970; Winer, 1962). How the generalizability coefficients for both the general (P),
the change (PO) and the person by skill dimensions (PD) components are estimated under
the two models, is presented in Table 2.
The two different sets of assumptions do not seem to affect the estimates of the
generalizabilitycoefficientfor the general component being 0.93 under Model4 and 0.91 under
Model-I1assumptions. The generalizability coefficientfor the change component is somewhat
lowered from 0.80 to 0.72 under the Mixed-I1 model assumptions. However a generalizability
coefficientof 0.72is still considered acceptable. This coefficientcan be thought of as equivalent
to the reliability of the sum of eight difference scores between pre- and post-measures
throughout the measuring operation (Eikeland, 1973). The generalizability coefficient is 0.71
for the person by skill dimension (PD) component under both sets of assumptions.
The same analytic procedure was repeated with standardized scores.'Only trivial discrepan-
cies to those reported above were obtained.
The conclusion of the present combined measurement and data-analytic approach suggests
that the most pronounced finding is reflected by the impact of the general component both
with respect to its relative importance and generalizability. The present measuring procedure
'The ordinary calculations of mean squares in analysis of variance were carried out by means of the
subroutinesAVDAT, AVCAL and MEANQ, suggested by Hartley ( 1962), adopted from IBM ( 1966)
and implemented at the UNIVAC 1110 system by Tom Backer Johnsen, Psychometrics Unit, University
of Bergen, Norway, to whom we are greatly indebted.

9. Sfand zyxwvutsrqponm
J Psycho1 zyxwvutsrq
34 zyxwvutsrqpo
( zyxwvutsrq
1993) zyxwvutsrqpon
also demonstrates acceptable rater consistency, and reliable change in the general problem-
solving score across the training period. The PD component is difficult to interprete in the
present analysis. however. It consists of three linear combinations that altogether explain
about zyxwvutsrq
6 per cent of the trace, which in turn indicates that each of them explain on the
average about only 2 per cent. The reliability of an average linear combination going into
this component is reported to be 0.71 in Table 2 within both sets of assumptions. Even
though this variance component or its linear combinations explain a rather small or trivial
amount of variance in the present study, its generalizability coefficient suggest that a
potential existence of differentiality aspects of the therapeutic problem-solving skills should
not be entirely rejected.
Assessing therapeutic problem solving skills 35
DISCUSSION
The concept of general skills in the training of para-professionals is often given different
content in the treatment research literature (i.e. Burgio et zyxwvu
al., 1983; Gladstone & Spencer,
1977; Page et al., 1982). However, the function of general skills is to create a flexible and an
adaptive practice on behalf of the therapist or trainer. In this way, therapeutic problem-
solving skills may be considered as a set of general mediating skills-a tool in the adaptation
of specific skills in different clinical contexts. We assume that the skills delineated in the
present study cover a variety of clinical change processes. Important professional character-
istics associated with the clinician are initiative in constructing, carrying out and evaluating
a treatment, and being systematic and specific in her conduct. Lastly, it is of great
importance that she is capable of rationally examining the problems encountered. We have
labelled these skills therapeutic problem-solving skills, and presented a method for how they
can be evaluated.
The present analysis clearly suggests that the four included skills serve as indicators of a
rather general therapeutic problem-solving skill concept. Referring to the notion of therapeu-
tic process in terms of stages, the present approach and results suggest the existence of an
underlying dimension across stages within the therapeutic process. Such a notion may be a
parsimonious and adequate way in portraying the complexity of therapeutic problem-solving.
Describing therapeutic problem-solving only in terms of content-stages may be a simplifi-
cation which may also mask important stable individual differences in the therapeutic
problem-solving process.
Even though the present findings clearly suggest the existence of a general therapeutic
problem-solving skill concept, there are factors that may limit generalizations of the present
findings, which in turn may initiate further research in this field.
First, the present findings are restricted to the types of skill included in the dimension facet
of this study. Other skills frequently emphasized, particularly in the clinical psychodynamic
tradition, like therapeutic patience and clinical empathy are not explicitly included in the
present conception of therapeutic problem-solving skills. Future research is therefore needed
to examine if the impact of a general component would still be dominant within a broader
selection of therapeutic skills.
Secondly, the results are based on a hypothetical case. According to Janis & Mann (1977)
there is evidence indicating that people respond differently to hypothetical situations than
they would do to real life situations. Future validation of the present findings should
therefore include the context of real life therapeutic problem-solving.
Third, there has been an absence of instruments and methods attempting to measure
constructs associated with therapeutic problem-solving. Thus, the present instrument may
serve as a helpful tool for researchers who want to assess the abilities of paraprofessionals in

10. 36
therapeutic problem-solving. An advantage of the instrument is the ease with which it is
administered and scored in contrast to the cumbersome scoring procedures used for assessing
presence versus non-presence of specific skills at different stages. However, until further
research is conducted on the instrument, it would best be restricted to research functions for
reasons listed above.
Fourth, the strong contribution of the general component also indicates that the individual
differences in this component stay rather stable from the pre- to the post-test assessment. zyx
Also the weak support of the differentiality aspect was not modified over the same period.
One may zyxwvuts
expect that the variance of the general component at post-occasion should have
been reduced compared to the corresponding variance at pre-occasion due to training effects.
The same reduction in variance may be a consequenceof the frequent observation, also made
in this study, that those low on pre-training level improve more over the training period than
those high on the pre-training occasion (cf. the correlation of -0.51 between pre-level and
improvement). Inspection of data indicated that the variance zyxwv
of the post-scores (254.94) was
less than the variance of the pre-scores (197.21). However, the observed difference between
the two correlated variances did not satisfy conventional level of significance zyx
(t,9=0.82; zyx
p >0.05). In a sum, there is evidence to state that stability indicated in both the amount of
individual differences of general therapeutic skill, and in the dimensionality pattern over
time, is a prominent aspect of the present findings. One may therefore suspect if the relative
small impact of differentiality has been restricted due to a relative short training period for
differentiality aspects to occur. This suggestion may in turn inspire a research program to
examine if the concept of therapeutic problem-solving skill would be more differentiated as
a function of a longer lasting experience and training.
The present study also illustrates one approach of evaluating measured therapeutic
problem-solving skills with respect to their dimensionality. Inter-rater consistency in this
methodology is represented by parameters of a multivariate model. zyxw
In this way its existence
in the model will directly affect the parameters of dimensionality. This procedure is in
contrast to the common routine of estimating inter-rater consistency by means of a single
correlation where its effect on other parameters is not explicitly estimated. In the present
methodology different types of inter-rater consistency can be estimated and in turn applied
in estimating generalizability of different components of substantive interest.
Instead of inspecting all correlations between the 16 variables in our measurement design.
the present procedure are able to support a parsimonious interpretation in terms of few
components to account for most of the correlational information. Even though the basic
ideas of this assessment have been known for a long time (cf. Bock, 1960). it's potential
application in assessing substantively based complex measuring procedures in small data sets
has not been frequently used.
J. A. Renning and zyxwvuts
K. A . Hagtvet %and J Psycho1 34 (1993)
REFERENCES
Bock, R. D. (1960). Components of variance analysis as a structural and discriminal analysis for
psychological tests. Brirish Journal o
f Statistical Psychology, i3*I51-163.
Burgio, L. D., Whitman, T. L. & Reid, 0. H. (1983). A participative management approach for
improving directtare staff performance in an institutional setting. Journal o
f Applied Behuuior
Analysis, 16, 37-53.
Crocker, L. & Algina, J. ( 1986). Introduction to classical and modem test theory. Fort Worth: Holt.
Rcinhart & Winston.
Cronbach. J. L., Gleser, G. C., Nanda, H. A. & Rajardtnam. N. ( 1972). The dependability of behavioral
meusurements. New York:Wiley.
DZurilla, T. J. & Goldfried. M. R. ( 1971). Problem-solving and behavior modification. Journul 0
1
Abnormul Psychology. 78, 107- 126.

11. Scand zyxwvutsrqp
I zyxwvutsrqponm
Psycho1 34 (1993) zyxwvutsrqp
Assessing therapeutic problem solving zy
skills zy
31 zy
Eikeland, H.-M. zyxwvutsrq
(1972). The structure of generalizability theory for hierarchically stratified tests.
Eikeland, H.-M. (1973). Generalizability estimates for difference scores: An aspect of the construct
Eisenhart, C. (1947). The assumptions underlying the analysis of variance. Biometries, 3, 1-21.
Gladstone, B. W. & Spencer, C. J. (1977). The effects of modeling on the contingent praise of mental
retardation counselors. Journal o
f Applied Behavior Analysis, 10, 15-84.
Goldfried, M. R. & Goldfried, A. P. (1975). Cognitive change methods. In F. H. Kanfer & A. P,
Goldstein (Eds.), Helping people change. New York: Pergamon Press.
Hagtvet, K. A. ( 1989).A rationally based simultaneous description of group- and individual differences:
A combination of aspects of experimental and correlational application of analysis of variance.
Report from Department of Psychometrics, University of Bergen; Bergen, Norway.
Hagtvet, K. A. & Halpern, E. (1992). A dimensional analysis of a pictorial experimental paradigm
measuring children’s developmental anxieties. In K. A. Hagtvet (Ed.), Advances in test anxiety
research. (Vol. 7). Lisse, The Netherlands: Swets & Zeitlinger.
Hartley, H. D. (1962). Analysis of variance. In A. Ralston & H. Wilf (Eds.), Mathematical method for
digital computers. New York: Wiley.
Heppner, P. P. (1978). A review of the problem-solving literature and its relationship to the counseling
process. Journal o
f Counseling Psychology, 28, 366-375.
IBM ( 1966). Scientific subroutines Package. Programmers Manual. New York: IBM, Technical Publica-
tions Department.
Intagliata, J. C. ( 1978). Increasing the interpersonal problem-solving skills of an alcoholic population.
Journal of Counsulting and Clinical Psychology, 46, 489-498.
Ivancic, M. T., Reid, 0. H., Iwata, B. A., Faw. G. 0 . & Page, T. J. (1981). Evaluating a supervision
program for developing and maintaining therapeutic staff-resident interactions during institutional
care routines. Journal o
f Applied Behavior Analysis, 14, 95- 107.
Janis, I. L. & Mann, L. (1977). Decision making: A psychological analysis o
f conflict, choice, and
commitment. New York Free Press.
Johnson, D. M., Parrot, G. R. & Stratton, R. P. (1968). Production and judgement of solutions to five
problems. Journal o
f Educational Psychology Monograph, 59 (6, E
’
t
. 2).
Kazdin, A. E. (1973). Issues in behavior modification with mentally retarded persons. American Journal
o
f Mental Deficiency, 78, 134-140.
Kazdin, A. E., Esveldt-Dawson, K., French, N. H. & Unis, A. S. (1987). Problem-solving skills training
and relationship therapy in the treatment of antisocial child behavior. Journal o
f Consulting and
Clinical Psychology, 55, 76-85.
Kiuel, R. L., Whitman, T. L. & Reid, 0.H. ( 1983). An institutional staff training and self-management
program for developing multiple self-care skills in severely/profoundly retarded individuals. Journal
o
f Applied Behavior Analysis, 16, 395-415.
Lmaas, 0. I., Koegel, R., Simmons, J. Q. & Long, J. S. (1973). Some generalization and follow-up
measures on autistic children in behavior therapy. Journal o
f Applied Behavior Analysk, 6, 131-
166.
Millman, J. & Glass, G. V. ( 1967).Rules of thumb for writing the ANOVA table. Journal o
f Educational
Measurement, 4, 41-51.
Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill.
Page, T. J., Iwata, B. A. & Reid, D. H. ’(1982). Pyramidal training. A large-scale application with
institutional staK. Journal o
f Applied Behavior Analysis, 15, 335-351.
Rogosa, D., Brandt, D. & Zimowski, M. (1982). A growth curve approach to the measurement of
change. Psychological Bulletin, 92, 726- 748.
Running, J. A. & Thorsen, S. A. (1984). Utviklingsfremmende tiltak for psykisk utviklingshemmete som
bor i institusjon. Personalrettede tiltak (Efforts in the education of staff in institutions for mentally
retarded). Tihskrift for Norsk Psykologforening, 21, 241-252.
Running, J. A. & Thorsen, S
. A. (1986~).The effectiveness of a staff training-and management
program on inappropriate behaviors of institutionalized severely and profoundly retarded. Unpub-
lished manuscript. Department of Clinical Psychology, University of Bergen.
Rsnning, J. A. & Thorsen, S. A. (19866). On the investigation and evaluation of problem solving skills
in developing appropriate staff behavior in institutions for the mentally retarded. Unpublished
manuscript. Department of Clinical Psychology, University of Bergen.
Mimeographed. University of O
s
l
o
.
validity of test. Mimeographed. University of Oslo.
Spivack, G. & Shure, M. B. ( 1978). Social adjustment o
f young children. San Francisco: Jossey-Bass.

12. 38 zyxwvutsrqp
J. A. RBnning andzyxwvut
K.A. Hagtvet Scandzyx
J Psycho1 34 zy
(1993)
Thompson, T. & Grabowski, 3. zyxwvuts
(1972). Behavior modgearion of -the mentally retarded. New York:
Oxford University Press.
Watson, L. S. Jr. (1972). How to use behavior modification with mentally retarded and autistic children:
Programsfor administrators, teachers,parents and nurses. Illinois: Behavior Modification Technology,
Inc.
Whitman, T. L., Schibak, J. W. & Reid, D. H. (1983). Behauior mod$cation with the severely and
profoundry retarded. New York: Academic Press.
Winer, B. J. (1962). Statistical principles i
n experimenfa!design. New York: McGraw-Hill.
Received 6 February 1989