The most common objective test questions are:

• multiple-choice
• true-false
• matching items
• cloze

The most common is the multiple choice question (MCQ) test where students must select the correct answer from a number of possible answers.

The incorrect answers in MCQs are termed distractors.

Distractors should cotnain:

• misconceptions
• partly correct answers
• common errors of fact or reasoning (these distract students who are not well prepared for the test from giving the correct answer)

MCQs are usually used to test the test-taker’s ability to:

• recall information
• interpret data/diagrams
• analyse/evaluate material

Main strengths of MCQs:

• test a wide range of issues in a short time
• assessment is not affected by a student’s ability to write
• can be reliably marked as all answers are predetermined
• can be quickly marked by computer
• computer marking gives easy access to an item analysis of questions to pinpoint problem areas for students
• a large bank of questions can be built up to reduce future preparation time
• can be used for quick revision at the start or end of a class and marked by the students

Main weaknesses of MCQs:

• do not test the student’s ability to develop and organize ideas and present these in a coherent argument
• takes a long time to write plausible distractors (especially in cases where higher order cognitive skills are being tested)
• restrictions are placed on the test-taker’s answers as they must select from given alternatives
• guessing may result (but plausible distractors will result in intelligent guessing)
• questions are often re-used which means special attention to security
• questions need to be pre-tested and items reviewed to ensure the validity of the items

Writing MCQs is a relatively difficult task. However, the effort expended in item construction is rewarded by the ease and reliability of marking

MCQs must have:
a clear and unambiguous stem
a correct answer
several (usually 3 or 4) distractors which appear plausible to students who do not know the correct answer
coherence to the content matter to be examined

E.g.

Tips for constructing MCQs:

• use simply worded stems
• present only one issue in the stem
• avoid use of negative premises (may especially disadvantage ESL students)
• ensure that the answer to one question cannot be obtained from another
• Keep the distractors brief and as homogeneous as possible
• ensure the distractors are plausible (i.e. common errors made by students)
• use at least 3 distractors (reduces chance of guessing the correct answer)
• avoid distractors that provide clues (e.g. phrases from text books)
• group similar types of MCQs together
• avoid using a pattern for the position of the correct response

The purpose of a Table of Specifications is to identify the achievement domains being measured and to ensure that a fair and representative sample of questions appear on the test.

As it is impossible, in a test, to assess every topic from every aspect, a Table of Specifications allows us to ensure that our test focuses on the most important areas and weights different areas based on their importance / time spent teaching. A Table of Specifications also gives us the proof we need to make sure our test has content validity.

Tables of Specifications are designed based on:

course objectives

topics covered in class

amount of time spent on those topics

textbook chapter topics

emphasis and space provided in the text

A Table of Specification could be designed in 3 simple steps:

1. identify the domain that is to be assessed

2. break the domain into levels (e.g. knowledge, comprehension, application …)

3. construct the table

The more detailed a table of specifications is, the easier it is to construct the test.

When important decisions are made based on test scores, it is critical to avoid bias which may unfairly influence test-takers’ scores

Fairness and bias are not the same thing.

Fairness has to do with how a test is used.

A biased test may be used fairly.

E.g.

For a test biased such that males score 5 points higher on average than do females if we simply add 5 points to the observed scores of the females and use that score for making decisions, the biased test will prove to be fair in use.

An item may be biased if it contains content or language that is differentially familiar to different subgroups and/or if the item structure or format is differentially difficult to different subgroups.

E.g.

If a test designed to measure a specific trait, then each time the test is administered to a subject, the results should be approximately the same.

Unfortunately, it is impossible to calculate reliability exactly but there are several different ways to estimate reliability. The different types of reliability that could be estimated are:

Test-Retest Reliability

Inter-rater Reliability

Parallel-Forms Reliability

Internal Consistency Reliability

To gauge test-retest reliability, the test is administered twice at two different points in time. This kind of reliability is used to assess the consistency of a test over a period of time. Test-retest reliability assumes that there will be no change in the quality or construct that is being measured.

Inter-rater reliability is assessed by having two or more independent raters score the test, then comparing the scores to determine the consistency of the raters’ estimates.

Parallel-forms reliability is estimated by comparing different tests that were created using the same content. The two tests should then be administered to the same subjects at the same time.

Internal consistency reliability is used to judge the consistency of results across items on the same test. i.e.  test items that measure the same construct are compared in order to determine the tests internal consistency.

Testing is a matter of making judgments about test-takers competence in view of their performance on certain tasks.

These judgments are inferences as tests do not collect concrete evidence about test-takers’ ability, in the natural state, but only abstract inferences

Evidence of test performance is used to draw conclusions about candidates’ ability to handle the demands of the criterion situation.

For high-stakes tests procedures need to be taken to investigate the procedure by which the conclusions were drawn.

Test validation is this process of investigating the quality of the test-based conclusions

The different types of validity are:

Content validity

     Face validity

     Content (sampling) validity

Criterion-related validity

-[        Concurrent Validity

<!        Predictive Validity

Construct validity

Face validity is the extent to which a test meets the expectations of those involved in its use –  stake-holders

This type of validation is designed to decrease opposition by ensuring that nobody is too unhappy with it.

An example of an instrument that measures face validity is Rosenberg’s self esteem scale.

When a test has content (sampling) validity, the items on the test represent the entire range of possible items the test should cover.

To ensure this, individual test questions may be drawn from a large pool of items that cover a broad range of topics.

Content validity establishes that the measure covers the full range of the concept’s meaning, i.e. covers all dimensions of a concept

When a test has content validity, the test reflects the syllabus on which it is based

A test is said to have criterion-related validity when the test is demonstrated to be effective in predicting criterion or indicators of a construct.

There are two different types of criterion-related validity:

<!   concurrent Validity

<!   predictive validity

Concurrent validity occurs when the criterion measures are obtained at the same time as the test scores.

This indicates the extent to which the test scores accurately estimate an individual’s current state with regards to the criterion.

Predictive validity occurs when the criterion measures are obtained at a time after the test.

A test has construct validity if it demonstrates an association between the test scores and the prediction of a theoretical trait.

Construct under-representation and construct irrelevant variance are two major threats to validity too.

A test is said to demonstrate construct under-representation if tasks included in the test fail to measure important dimension of the construct. If this happens, results of the test are unlikely to reveal test-taker’s ability within the domain the test claims to measure.

A test is said to demonstrate construct irrelevant variance if tasks measure variables which are irrelevant to the domain the test claims to measure. This type of invalidity can take two forms:

<!     construct irrelevant easiness

<!     construct irrelevant difficulty

Communicative competence is a linguistic term which refers to a learner’s second language ability. It refers to a learner’s ability to:

• apply and use grammatical rules
• form correct utterances
• use these utterances appropriately

The term was coined by Dell Hymes (in 1966) who was inspired by Noam Chomsky’s distinction on linguistic competence and performance.

According to Chomsky (1965) a speaker’s language ability comprised two components:

• linguistic competence
• linguistic performance

Hymes proposed that knowing a language entailed knowing more than its grammar and rules

According to Hymes there culturally specific rules that created a relationship between:

• the language used
• features of the communicative context

e.g.

What is appropriate language for communication with a sibling may not be appropriate for communication with an employer or lecturer.

Before the theory of communicative competence, language was often described from a psychological perspective, but this theory marked a profound shift in how language was perceived as it presented language as an internal phenomenon.

With the appearance of the communicative competence theory the focus shifted to a more sociological one, where the focus was on external, social functions of language.

The relevance of Hymes’ theory to language testing was almost immediately recognized when it appeared.

However, it was a decade later that its actual impact was felt on practice with the development of communicative language testing

Communicative language tests (CLT) are distinguished by two main features:

• CLTs are performance tests and therefore require assessment to be carried out when the learner or candidate in engaged in an extended (receptive/productive) act of communication
• CLTs pay attention to the social roles candidates would assume and hence considers the roles that candidates would assume in the real world on passing the test and offers a means of specifying the demands of such roles in detail

Models of communicative ability

Thought it was a challenge to shift perspectives not to mention focuses of language tests, there was a continuous theoretical engagement with the idea of communicative competence and its implications for the performance requirements of communicative language testing since the advent of the theory of communicative competence.

A number of writers have tried to specify the components of communicative competence in second languages and their role in performance.

The purpose of this is to provide a comprehensive framework for:

• test development
• testing research
• interpretation of test performance

The first such models specified the components of knowledge of language without dealing in detail with their role in performance.

In 1980, Michael Canale and Merrill Swain published a paper that specified four components of communicative competence:

• Grammatical competence - knowledge of systematic features of grammar, lexis and phonology
• Sociolinguistic competence - knowledge of rules of language use in terms of what is appropriate in different contexts
• Strategic competence - ability to compensate for incomplete or imperfect linguistic resources in a second language by using (other) successful communication strategies
• Discourse competence - ability to deal with extended use of language in context (cohesion and coherence)

They differ mainly in terms of design (method) and purpose.

In terms of method, a broad distinction can be made between pen-and-paper language tests and performance tests

Paper-and-pen tests are typically used for the assessment of

• separate components of language (grammar, vocabulary …)

• receptive understanding (listening & reading comprehension)

Test items in such tests (especially if they are professionally made, standardized tests) are often in fixed response format (e.g. MCQ)

In performance tests language skills are assessed in an act of communication.

e.g. tests of speaking and writing where:

• extended samples of speech/writing is elicited
• judged by trained markers
• common rating procedure used

Main distinction in terms of test purpose:

Achievement tests

Proficiency tests

Achievement Tests are associated with the process of instruction and should support the teaching to which they relate by measuring what students (would) have learned as a result of teaching.

E.g.

• end of course tests
• portfolio assessments
• observation procedures for recording & assessing classroom work/participation

Achievement tests may be self-enclosed i.e. may not bear any direct relationship to language use in the real world

E.g. focus on knowledge of particular areas of grammar/vocabulary

However, if the curriculum is designed to reflect language use in the real world, achievement tests will automatically reflect normal language use and can be designed in innovative ways to reflect progressive aspects of the curriculum.

That is why achievement tests are associated with the most interesting development in language assessment - alternative assessment.

Alternative assessment stresses the need for assessment to be integrated with goals of curriculum and promotes a constructive relationship with the teaching/learning process.

Proficiency Tests look to the future situation of language use without necessarily any reference to the process of teaching.

In these tests, performance is measured in relation to a targeted level known as the criterion.

Main purpose of performance tests is to make inferences, however they are not valued in themselves but as indicators of how the test-taker will perform similar (or related) tasks in the real world setting of interest.

A test can be defined as any one of the following:

procedure for critical evaluation; a means of determining the presence, quality, or truth of something;
series of questions, problems, or physical responses designed to determine knowledge, intelligence, or ability.
basis for evaluation or judgment:

So we could conclude that testing is a form of assessment used for evaluation purposes

Role of Testing in Education

Tests can be used in education to achieve any of the following purposes:

accountability:

is the teacher/school actually doing what it claims to be doing?

information on results of teaching:

feedback

backwash:

positive effects of backwash — change and improvement

negative effects of backwash — teaching for the test

Costs of inaccurate testing

Financial and social

Gate-keeper role of testing:

effect on peoples’ lives

What is a Language Test?

What is true of testing in general is also true of language testing.

Images of language tests usually involve examination rooms, test papers, desperate scribbling and racing against the clock to finish on time or waiting nervously in an uncomfortable chair to be called into an interview …

But there is more to language testing than this.