2. Test Specifications
A test’s specifications provide the official statement
about what the test tests and how it tests it
What is the purpose of the test?
What sort of learner will be taking the test?
How many sections should the test have?
What text types should be chosen – written/spoken?
What language skills should be tested?
What language elements should be tested?
What sort of tasks are required
2TEST CONSTRUCTION PROCEDURES
3. Test Specifications
How many items are required for each section?
What test methods are to be used?
What rubrics are to be used as instructions?
Which criteria will be used for assessment by markers?
3TEST CONSTRUCTION PROCEDURES
4. Test Specifications
Test specifications should include all or some of the following:
The test’s purpose
Description of the test taker
Test level
Construct (theoretical framework for test)
Description of suitable language course or textbook
Number of sections/papers
Time of each section/paper
Target language situation
Text-types
Text length
Language skills to be tested
4TEST CONSTRUCTION PROCEDURES
5. Test Specifications
Test specifications should include all or some of the following:
Language elements to be tested
Test tasks
Test methods
Rubrics
Criteria for marking
Descriptions of typical performance of each level
Descriptions of candidates at each level can do in the
real world
Sample papers and Samples of students’ performance
on tasks
5TEST CONSTRUCTION PROCEDURES
7. Test Specifications
Cognitive
Cognitive domains are objectives which emphasize
remembering or reproducing something which has
presumably been learned, as well as objectives which
involve the solving of some intellectual tasks for which
the individual has to determine the essential problem
and then reorder given material or combine it with
ideas, methods, or procedures previously learned.
7TEST CONSTRUCTION PROCEDURES
8. Test Specifications
Cognitive Domain
Six Major Areas
Evaluation
Synthesis
Analysis
Application
Knowledge
Comprehension
New Forms (Verbs)
Creating
Evaluating
Analyzing
Applying
Understanding
Remembering
8TEST CONSTRUCTION PROCEDURES
9. Test Specification
Cognitive Domain
Knowledge refers to the recall of specific information;
Comprehension refers to an understanding of what was
read;
Application refers to the converting of abstract content to
concrete situations;
Analysis refers to the comparison and contrast of the content
to personal experiences;
Synthesis refers to the organization of thoughts, ideas, and
information from the content; and
Evaluation refers to the judgment and evaluation of
characters, actions, outcome, etc., for personal reflection and
understanding
9TEST CONSTRUCTION PROCEDURES
10. Test Specification
Cognitive Domain
Remembering indicates recalling information
(recognising, listing, describing, retrieving, naming, finding).
Understanding signifies explaining ideas or concepts
(interpreting, summarising, paraphrasing, classifying, and
explaining).
Applying conveys using information in another familiar situation
(implementing, carrying out, using, executing).
Analysing denotes breaking information into parts to explore
understandings and relationships
(comparing, organising, deconstructing, interrogating, and
finding).
Evaluating implies justifying a decision or course of action
(checking, hypothesising, critiquing, experimenting, and judging).
Creating refers to generating new ideas, products, or ways of
viewing things
(designing, constructing, planning, producing, inventing).
10TEST CONSTRUCTION PROCEDURES
11. Test Specification
Affective Domain
Affective domains are objectives, which emphasize a
feeling tone, an emotion, or a degree of acceptance
or rejection. Affective objectives vary from simple
attention to selected phenomena to complex but
internally consistent qualities of character and
conscience. A large number of such objectives in the
literature expressed as
interests, attitudes, appreciations, values, and
emotional sets or biases.
11TEST CONSTRUCTION PROCEDURES
12. Test Specification
Affective Domain
Affective domains:
Spiritual Attitude: pious, virtuous
Social Attitude:
1. Honest to oneself and others
2. Disciplined punctual and orderly
3. Responsible take risk and apologize
4.Tolerance respect others
5. Cooperative work together
6.Polite respect and behave well
7. Self-confident brave and courageous
12TEST CONSTRUCTION PROCEDURES
13. Test Specification
Psychomotor Domain
Psychomotor domains are objectives, which
emphasize some muscular or motor skill, some
manipulation of materials and objectives, or some
acts that requires a neuromuscular coordination.
13TEST CONSTRUCTION PROCEDURES
14. Test Specification
Example
Subject: _________________________________________
Semester/Class: __________________________________
School: __________________________________________
Test description: _________________________________
No
Basic
Competence
Objective/
Indicators
Domain Techniques Items
1. Listening … C1, C2 MC 1 – 10
2. Speaking C3 … …
3. Reading …
4. Writing
14TEST CONSTRUCTION PROCEDURES
15. Test Specification
Example: Reading
Subject: _________________________________________
Semester/Class: __________________________________
School: __________________________________________
Test description: _________________________________
Item Cognitive Techniques Objectives Item Types Remarks
1. C2 MC Finding main
idea
Topic
sentence
2. C4 MC Comparing
Paragraphs
Difference …
3. C1 MC Finding
explicit info
Facts or
Realities
…
4. … … … … …
15TEST CONSTRUCTION PROCEDURES
16. Test Construction and Moderation
Item Writing and Revision
In writing test items, one should ideally combine both necessary
formal professional
The item writing must be based on the test specifications
The method used for testing a language ability may itself affect
the students’ score, which is called the method effect
It is likely that particular test methods will lend themselves to
testing some abilities, and not be so good at testing others
In terms of test editing or moderation, each item and the test as
a whole are considered for the degree of match with the test
specifications, likely level of difficulty, possible unforeseen
problems, ambiguities in the wording of items and of
instructions, problems of layout, match between stems and
choices, and overall balance of the subtest or paper.
16TEST CONSTRUCTION PROCEDURES
17. Try-out
However well designed a test may be, and however carefully
it has been edited, it is not likely to know how it will work
until it has been tried out on students.
We do not only need to know how difficult the test items
are, but we also need to know whether they work.
The number of students on whom a test should be trialed
depends on the importance and type of test, and also the
availability of suitable students. The only guiding rule is
the more the better, since the more students there are, the
less effect change will have on the results.
17TEST CONSTRUCTION PROCEDURES
18. Test Analysis
This analysis will show us the extent to which each item works.
For objective test items, traditionally there are two measures of
calculation – the facility value and the discrimination index.
For subjectively marked tests although item analysis is
inappropriate, such as summaries, essays, and oral
interviews, these tests still need to be tried out to see whether
the items elicit the intended sample of language; whether the
marking system, which should have been drafted during the
item writing stage, is usable; and whether the examiners are able
to mark consistently.
Once the papers or interviews have been administered, there
should be trial marking sessions to see whether the test item
prompts have produced the intended kinds of responses, and
whether the marking guidelines and criteria are working
satisfactorily.
18TEST CONSTRUCTION PROCEDURES
19. Validation
The most important question of all in language testing
is validity.
The validation process involves the terms internal and
external validity, with the distinction being that
internal validity relates to studies of the perceived
content of the test and its perceived effect, and
external validity relates to studies comparing students’
test scores with measures of their ability gleaned from
outside the test.
The commonest types of external validity are
concurrent validity and predictive validity.
Expert Validation
19TEST CONSTRUCTION PROCEDURES
20. Public or User Trial
The tests that have been constructed, tried out, and
analyzed should also be evaluated by public, especially
the future users of the tests. The tests are presented to
the future users and they analyze the tests, give
comments or suggestions to the improvement of the
tests, and approve the tests.
20TEST CONSTRUCTION PROCEDURES
21. Exercises
What are the procedures in constructing a test?
What should be included in the test specifications?
Explain the three domains of taxonomy.
What are the new terms in cognitive domains?
21TEST CONSTRUCTION PROCEDURES