Azu’s Notes

A Table of Specifications is a two-way chart which describes the topics to be covered in a test and the number of items or points which will be associated with each topic. Sometimes the types of items are described as well.

The purpose of a Table of Specifications is to identify the achievement domains being measured and to ensure that a fair and representative sample of questions appear on the test.

As it is impossible, in a test, to assess every topic from every aspect, a Table of Specifications allows us to ensure that our test focuses on the most important areas and weights different areas based on their importance / time spent teaching. A Table of Specifications also gives us the proof we need to make sure our test has content validity.

Tables of Specifications are designed based on:

course objectives

topics covered in class

amount of time spent on those topics

textbook chapter topics

emphasis and space provided in the text

 

A Table of Specification could be designed in 3 simple steps:

1. identify the domain that is to be assessed

2. break the domain into levels (e.g. knowledge, comprehension, application …)

3. construct the table

The more detailed a table of specifications is, the easier it is to construct the test.

A phrase is a group of words acting as a single part of speech and not containing both a subject and a verb.

It is a part of a sentence, and does not express a complete thought.

Noun Phrases

Noun phrases must have a Noun (or Pronoun) and may or may not have other modifiers

e.g.

Adjective phrases

An Adjective phrase must consist an adjective (A) and may or may not have an adverb phrase (AdvP)

e.g.

Adverb Phrases

Adverb phrases must consist an Adverb (Adv) and may or may not have a degree adverb (deg)

Adverb phrases may be dominated by either a VP, an AP or the sentence.

Adverb phrases that modifies a whole sentence can be moved to different positions in the sentence without altering the meaning or making it ungrammatical.

Sentence adverbs often express an attitude or evaluation (e.g. clearly, frankly, actually …)

e.g.

Preposition Phrases

Preposition Phrases must consist of a preposition (P) and may or may not include an NP

e.g.

Verb phrases

Verb phrases must consist a verb and may or may not have other phrases within it

e.g.

A piece of language is said to be coherent (therefore discourse) if it has a discernible, unified meaning.

A piece of discourse is said to be cohesive if its components (ie. sentences/phrases/words) are bound together through linguistic and non-linguistic features to form a unified whole.

The linguistic features used to link one word/phrase/sentence to another are called formal links.

Some common formal links are:

Referring Expressions

Repetitions

Lexical chains

Substitutions

Ellipsis

Connectives

The most obvious example of formal link is third person pronouns

In a piece of language, cohesion is achieved by using these referring expressions that direct the hearer/reader to look elsewhere for their interpretation.

Reference expressions can be:

Endophoric

Anaphoric

Cataphoric

Exophoric

Endophoric references are linguistic references to something within the same text.

There are two types of endophoric references:

anaphoric references

cataphoric references

Anaphoric references refer back to another unit that was mentioned before.

e.g.

Aiminaibee asked Thakuru to buy her a diamond ring.

Cataphoric references refer ahead to another unit that is mentioned later.

e.g.

Waving at him happily, Thakuru saw Aiminaibee come out.

Exophoric references refer to entities outside the text, in the context of the utterance or speaker.

e.g.

That is where Aiminaibee first saw the Foolhudhiguhandi.

(said while pointing to the place)

Repetition of a key term or phrase in the text helps to focus your ideas and to keep your reader/listener on track.

E.g.

The problem with modern art is that it is not easily understood by most people. Modern art is deliberately abstract, and that means it often leaves the viewer wondering what she is looking at.

Lexical chains are also a form of repetition but without repeating the exact same phrase/word.

i.e. use different words that are lexically related (e.g. hypernyms)

e.g.

Myths are an important part of a country’s heritage. Such traditional narratives are, in short, a set of beliefs that are a very real force in the lives of the people who tell them.

Cohesion is often achieved by substituting special words for ones that have already been used.

The most common substitutes used in English are

one

do

so

Each of these are used to substitute for a different type of clause

‘one’ is used to substitute for nouns / noun phrases:

I left the school and went to the one in Thuraakunu.

I left Hithadhoo secondary school and went to the Thuraakunu one.

I left the Hithadhoo secondary school with many students and went to the one with few students.

I left the Hithadhoo secondary school with few students and went to the Thuraakunu one with few students.

Verbs are substituted with ‘do’.

Since do is a verb (and an irregular one at that) is also has the forms does, did, done and doing.

I have not finished yet, when I do you can start.

I like coffee and so does he.

The word ’so‘ is often used to substitute for a whole clause:

Thakuru: “We’ll be watching you close, smart guy.”

Haadi: “I hope so. You might learn something.”

 

Thakuru: “I think we have got rid of him for good.”

Aiminaibee: “You really think so?”

In certain contexts it is possible to leave out a word/phrase rather than repeat it.

This device is called ellipsis.

A child learns to speak almost ‘by chance’. He imitates his parents without knowing why < >.

Students continue to wear faded jeans to class even after being told not to < >.

Connectives are words/phrases used to indicate a specific connection between different parts of a text.

Various kinds of words and phrases can function as connectives:

conjunctions

It posed several problems for me, but it was all worthwhile.

adverbs

It posed several problems for me; nevertheless, it was all worthwhile.

prepositions

In spite of the severe problems it posed for me, it was all worthwhile.

There are 4 basic types of connectives:

Addition connectives (AC)

adds on to the idea presented before (also, and)

Opposition connectives (OC)

contrasts with the idea presented before (but, nevertheless )

Cause connectives (CC)

shows a causal connection with the ideas presented before (therefore, since)

Time connectives (TC)

shows a sequence or simultaneous actions (first, finally)

Test bias is the presence of some characteristics of an item in the test, that results in differential performance by individuals of the same ability but different sub-group

When important decisions are made based on test scores, it is critical to avoid bias which may unfairly influence test-takers’ scores

Fairness and bias are not the same thing.

Fairness has to do with how a test is used.

A biased test may be used fairly.

E.g.

For a test biased such that males score 5 points higher on average than do females if we simply add 5 points to the observed scores of the females and use that score for making decisions, the biased test will prove to be fair in use.

An item may be biased if it contains content or language that is differentially familiar to different subgroups and/or if the item structure or format is differentially difficult to different subgroups.

Reliability refers to the consistency of a measure. A test is considered reliable if we get the same result repeatedly.

E.g.

If a test designed to measure a specific trait, then each time the test is administered to a subject, the results should be approximately the same.

Unfortunately, it is impossible to calculate reliability exactly but there are several different ways to estimate reliability. The different types of reliability that could be estimated are:

Test-Retest Reliability

Inter-rater Reliability

Parallel-Forms Reliability

Internal Consistency Reliability

To gauge test-retest reliability, the test is administered twice at two different points in time. This kind of reliability is used to assess the consistency of a test over a period of time. Test-retest reliability assumes that there will be no change in the quality or construct that is being measured.

Inter-rater reliability is assessed by having two or more independent raters score the test, then comparing the scores to determine the consistency of the raters’ estimates.

Parallel-forms reliability is estimated by comparing different tests that were created using the same content. The two tests should then be administered to the same subjects at the same time.

Internal consistency reliability is used to judge the consistency of results across items on the same test. i.e.  test items that measure the same construct are compared in order to determine the tests internal consistency.