Relaibility

Reliability refers to the consistency of a measure. A test is considered reliable if we get the same result repeatedly.

E.g.

If a test designed to measure a specific trait, then each time the test is administered to a subject, the results should be approximately the same.

Unfortunately, it is impossible to calculate reliability exactly but there are several different ways to estimate reliability. The different types of reliability that could be estimated are:

Test-Retest Reliability

Inter-rater Reliability

Parallel-Forms Reliability

Internal Consistency Reliability

To gauge test-retest reliability, the test is administered twice at two different points in time. This kind of reliability is used to assess the consistency of a test over a period of time. Test-retest reliability assumes that there will be no change in the quality or construct that is being measured.

Inter-rater reliability is assessed by having two or more independent raters score the test, then comparing the scores to determine the consistency of the raters’ estimates.

Parallel-forms reliability is estimated by comparing different tests that were created using the same content. The two tests should then be administered to the same subjects at the same time.

Internal consistency reliability is used to judge the consistency of results across items on the same test. i.e.  test items that measure the same construct are compared in order to determine the tests internal consistency.

Syntax

All languages have rules. These rules together form the grammarof that language. These rules are what enables a person to produce an infinite number of phrases/sentences from a finite number of possibilities, that is understood by both the speaker and the listener. It would be rather difficult to learn any language if each sentence had to be learnt separately!

Syntax is the areas of linguistics that attempts to describe what is grammatical in a particular language in term of rules. It is the study of sentence structure. Syntactic rules detail an underlying structure and a transformational process.

The underlying structure of English for example would have a subject-verb-object (SVO) sentence order

e.g.

Usman hit the ball.

The transformational process would allow an alteration of the word order which could give you something like:

The ball was hit by Usman.

All languages have ways of referring to entities –  people, places, things, ideas, events. These expression used to refer to entities are known as referring expressions. All languages can also make predictions about what is signaled by the referring expressions (i.e. they have ways of making statements, asking questions, issuing directions …) These are generally called the predicate of the sentence.

In syntactic terms a referring expression is called a Noun Phrases – NP; and the predicate is called a Verb Phrases – VP. All languages have NPs and VPs

In syntax all other phrases (that occur within the NPs & VPs) are also termed with reference to the head component:

e.g.

Adjective Phrase (AP)

Adverb Phrase (AdvP)

Preposition Phrase (PP)

There are three main kinds of sentences:

Simple sentences

Conjoined sentences (also known compound sentences)

Complex sentences

Simple sentences contain only one idea

E.g.

Adnan fell.

Simple sentences comprise only one idea, clause and verb group. We say verb group (instead of verb) because a verb group itself can consist of one word (fell, assembled, cooked, won) or more than one word (will buy, had put, should have believed).

 

Conjoined sentences have two (or more) clauses joined together using coordinating conjunctions (and, but, or …)

E.g.

Adnan fell and twisted his ankle.

 

Complex sentences have embedded clauses. i.e. one clause is incorporated into another one

E.g. The clause

Adnan fell

can be incorporated into another clause to produce the sentence –

Zahir said Adnan fell

Unlike in conjoined sentences, complex sentences contain clauses of unequal status. i.e. one clause is subordinated into another and functions as a grammatical part of it. The subordinate clause is called an embedded clause and the clause in which it is embedded is called a matrix clause.

Every subordinate clause is embedded in a matrix clause and serves a grammatical function in it.

Lexical Semantics

Lexical semantics is the subfield of linguistics that studies how and what words of a language denote and thus involves the meaning of individual words.

Lexical semantics focuses on theories of:

classification and decomposition of word meaning

differences and similarities in lexical semantic structure between different languages

the relationship of word meaning to sentence meaning and syntax

The study of lexical semantics includes the study of:

lexical fields

lexical relations

Lexical fields are sets of words that share semantic affinity

E.g.

Lexical field of colour includes words such as

black, orange, red, green …

There are two types of lexical fields:

Syntagmatic

words share a particular syntactic relationship to one another

Paradigmatic

words organized into a head term and other subordinate terms in a hierarchical organization pattern

A lexical relation is a pattern of association that exists between lexical units in a particular language.

Lexical relations we will be looking at are:

synonymy

meronomy

hyponymy & hypernymy

homonymy

polysemy

antonymy

Synonyms are words that have identical or similar meanings that can be used interchangeably in most cases; although no synonyms have exactly the same meaning (in all contexts or social levels of language)

E.g.

smooth, silky

give, deliver, provide

house, home, hut, dwelling, abode

Synonyms can be nouns, adverbs or adjectives as long as they belong to the same part of speech

Meronomies describe part-whole relations. A meronym denotes a constituent part of, or a member of something

E.g.

finger is a meronym of hand

Hyponym is a word whose semantic range is included within that of another word

E.g.

car, van, train, dhoni, ship, aircraft are all hyponyms of vehicles.

Hypernym is a word whose meaning encompasses the meaning of (an)other word(s)

E.g.

Vehicle is the hypernym for each of the following:

car, van, train, dhoni, ship, aircraft

Homonym is a word whose senses are not obviously related (other than through coincidence or accident)

E.g.

I, eye and aye

Homonyms can be divided into further sub-categories:

homographs

same spelling (bark of tree; bark of dog)

homophones

same pronunciation (there, their, they’re)

Heteronyms

same spelling, different meaning & pronunciation (desert (abandon) and desert (arid region))

capitonymy

different meanings when capitalized (polish, Polish)

heterologues

from different languages; have same spelling but different meaning (‘Hell’ in English & ‘Hell’ in German which means bright)

A polyseme is a single word with two distinct but very closely related senses.The distinction between polysemy and homonymy is often very subtle and subjective. Some sources state that homonym meanings must be unrelated (rather than just different), or that the words must have a different origin. Whereas polysemes have related meanings often with the same origin

E.g.

fork (in road / instrument to eat)

Antonyms are word pairs that are opposite in meaning and can be divided into four main types:

gradable antonyms

extremes of a scale (hot >< cold)

complementary antonyms

mutually exclusive (married >< single)

converse/relational antonyms

a sort of binary opposition (parent >< child)

Validity

Validity is the extent to which a test measures what it claims to measure.

Testing is a matter of making judgments about test-takers competence in view of their performance on certain tasks.

These judgments are inferences as tests do not collect concrete evidence about test-takers’ ability, in the natural state, but only abstract inferences

Evidence of test performance is used to draw conclusions about candidates’ ability to handle the demands of the criterion situation.

For high-stakes tests procedures need to be taken to investigate the procedure by which the conclusions were drawn.

Test validation is this process of investigating the quality of the test-based conclusions

The different types of validity are:

Content validity

     Face validity

     Content (sampling) validity

Criterion-related validity

-[        Concurrent Validity

<!        Predictive Validity

Construct validity

Face validity is the extent to which a test meets the expectations of those involved in its use –  stake-holders

This type of validation is designed to decrease opposition by ensuring that nobody is too unhappy with it.

An example of an instrument that measures face validity is Rosenberg’s self esteem scale.

When a test has content (sampling) validity, the items on the test represent the entire range of possible items the test should cover.

To ensure this, individual test questions may be drawn from a large pool of items that cover a broad range of topics.

Content validity establishes that the measure covers the full range of the concept’s meaning, i.e. covers all dimensions of a concept

When a test has content validity, the test reflects the syllabus on which it is based

A test is said to have criterion-related validity when the test is demonstrated to be effective in predicting criterion or indicators of a construct.

There are two different types of criterion-related validity:

<!   concurrent Validity

<!   predictive validity

 

Concurrent validity occurs when the criterion measures are obtained at the same time as the test scores.

This indicates the extent to which the test scores accurately estimate an individual’s current state with regards to the criterion.

Predictive validity occurs when the criterion measures are obtained at a time after the test.

A test has construct validity if it demonstrates an association between the test scores and the prediction of a theoretical trait.

Construct under-representation and construct irrelevant variance are two major threats to validity too.

A test is said to demonstrate construct under-representation if tasks included in the test fail to measure important dimension of the construct. If this happens, results of the test are unlikely to reveal test-taker’s ability within the domain the test claims to measure.

A test is said to demonstrate construct irrelevant variance if tasks measure variables which are irrelevant to the domain the test claims to measure. This type of invalidity can take two forms:

<!     construct irrelevant easiness

<!     construct irrelevant difficulty

Morphology (continued …)

Word structure

In morphology, word structure is described in terms of roots and affixes

Simple words consist one morpheme – the root

fun, go, danger

Complex words consist more that one morpheme – the root + affix(es)

funny, goes, endanger

Languages have three principal ways of extending their vocabulary:

invention of entirely new words

borrowing from other languages

formation of new words from already existing words and word parts

Invention of new words

This is very rare

It is much easier for languages to either incorporate new meanings to existing words or borrow from another language than to make new words from scratch

Borrowing from other languages

Most language users have borrowed words from other languages and incorporated them into their own.

Deriving new words

Some ways in which new words are built from existing ones, in English are:

compounding

Shortening

Acronyms

blends

back formation

functional shift or conversion

semantic shift (metaphorical Extension)

Compounding

Compounding is a very common form of creating new words in English. It is the combination of two words to form one.

e.g.

waterbed

Shortening

Shortenings of various sorts are a popular means of multiplying the words of a language.

e.g.

feds  –  federal agents

Acronyms

These are words formed by joining the initial letters of an expression and pronouncing them as a word.

e.g.

FIFA

Blends

Blends are words created by combining parts of existing words.

e.g.

motel  – motor + hotel

Backformation

Words that are formed (again) from derivations of existing words to include a different meaning.

e.g.

computer

originally formed by adding –er to (existing) verb compute (calculate using a mathematical function) then, the computer was invented machine that computed (in the mathematical sense), however now compute has been back-formed carrying the meaning ‘to use a computer’

 

Functional Shift

In some languages (e.g. English) words belonging to one lexical category get converted to another lexical category without any overt markings on the words itself.

e.g.

local (noun; adjective)

Semantic Shift

This occurs when existing words take on new meaning by shrinking or extending their domain or usage.

When a word undergoes a functional shift in meaning, they do not replace the old one, but instead extend their range of application.

e.g.

Computer users today use a mouse and bookmark an Internet address.

Semantic Shift

Functional shifts create metaphors, then the metaphorical use of the words often leads to new meanings that come to seem perfectly natural and hence all but lose their metaphorical content.

Types of Meaning

There are three main types of meaning:

Referential meaning

Social Meaning

Affective Meaning

Referential Meaning

This is the object, notion, or state of being described by a word, phrase or sentence

e.g.

Safaru Kaidha

the meaning of this is the person who goes by that particular name.

Hadigilla’s trousers

This refers to the particular piece of clothing that belongs to that particular person

i.e the meaning of the sentence is the particular piece of clothing that belongs to Hadigilla.

The piece of clothing described by the phrase Hadigilla’s trousers is the referent of the referring expression Hadigilla’s trousers.

Santhi Mariyabu is sleeping on the holhuashi.

The meaning of this sentence is that the person by the name of Santhi Mariyambu is lying down (or sitting, perhaps) on the bed-like structure made of bamboo trunks.

Therefore, the referent of the sentence is Santhi Mariyabu’s state of being on the particular structure mentioned.

 

Social Meaning

This is the information about the identity of the speaker that is conveyed by an utterance.

e.g.

Then I says to him he can’t do nothin’ right.

The use of the verb ‘says’ with the first-person singular pronoun reveals something about the social class of the speaker

Is it a doctor in here?

The form ‘it’ where most other varieties of English would use ‘there’ indicates a speaker of an ethnically marked variety (African American Vernacular English)

Y’all gonna visit over the holiday?

The pronoun ‘y’all’ and the verb ‘gonna’ indicates a particular regional dialect of American English (Southern)

Great chow!

The choice of words here indicates that the comment was made in an informal context.

Social class, ethnicity, regional origin and context are all types of social meaning.

In addition to referential meaning, every utterance also conveys social meaning – not only in the sentence as a whole but in word choice and pronunciation.

Affective Meaning

This is the speaker’s feeling / attitude towards the content or the ongoing context.

e.g.

Rehendhi, who always brags about her cooking skills, lectured me throughout the dinner on how to improve my cooking skills.

What does the speaker think of Rehendhi?

The utterance gives the impression that the speaker considers Rehendhi a pompous bore who thinks too much of herself.

What about this one?

Rehendhi, who is a skilled cook herself, gave me some wonderful tips on how to improve my cooking skills.

What does this speaker think of Rehendhi?

This gives the impression that the speaker believes Rehendhi to be a skilled and interesting person.

Although both sentences may be describing the same event (i.e they have similar referential meaning), on another level, the information they convey is very different

i.e. ‘stance’ of the two sentences are very different.

Different use of stress and intonation also provides a striking contrast in the feelings and attitudes communicated through an utterance.

e.g.

Fulhu is very kind.

How many different feelings / attitudes about Fulhu can you convey with this simple utterance?

Denotation & Connotation

The reference meaning of a word or sentence is frequently called its denotation, in contrast to the connotation, which includes both social and affective meaning.

Sense, Denotation and Reference

Linguistic expressions have both detonation and sense.

Denotation has to do with relations between linguistic expressions and the world.

It is the set of entities to which a word or expression refers (also called its referents and extension).

However, denotation is not enough to decide the meaning of a word.

Sense is the linguistic version of meaning, and is the ‘presentation of the denotation’ – which lets one work out what the denotation is.

Sense is something possessed by a name or expression, whether or not it has a reference / denotation.

e.g.

The following expressions are intelligible, and therefore have sense, even though there is no individual object (its reference) to which the expressions correspond:

The 32nd day of February.

The Mercurian ambassador to the planet Venus.

Present King of the Maldives.

Denotation is different from reference in that:

Reference is also about the relations between expressions and the world

Reference of an expression is specific to that particular occasion of utterance

Sense, Denotation and Reference

e.g.

That man is really dumb.

The denotation of man:

{all males .. e.g. Ahmed, Thakuru, Ibrahim, Fulhu …}

The reference of man:

the specific man meant by that particular sentence, at that particular time, by that particular speaker (e.g. just Alifulhu)

Words and sentences are two units of language that carry meaning.

Content words (mainly nouns, verbs, prepositions, adjectives and adverbs):

  •  refer to concrete objects and abstract concepts

  •  are marked as being characteristic of particular social, ethnic and regional dialects and of particular contexts

  • convey information about the feelings and attitudes of the language user

Function words (e.g. conjunctions, determiners and auxiliaries):

  • signal grammatical relationships

The meaning of a sentence is dependent on the meaning of the individual words it contains.

Grammatical Aspect

In linguistics, grammatical aspect is a property of a verb that defines the nature of temporal flow in the described event or state.

In most modern Indo-European languages, including English, the concept of aspect has become conflated with the concept of tense.

It is somewhat difficult to explain the idea of aspect in English because it uses the same patterns to encode in tense both the time and the aspect of a verb together.

Time signals whether an action or event happens in the past, present, or future.

Aspect signals the duration that the event covers (and perhaps its commencement, continuation, completion, or repetition, etc.).

Time and aspect do not necessarily have to be represented together; but any clear distinction has long been lost in English, where the verb tense-form now encodes both aspect and time together.

Aspect is often indicated by verbal affixes or auxiliary verbs.

In English present and past are expressed using direct modifications of the verb, which is then modified further by one or more non-simple aspects

i.e. either

progressive/continuous,

prefect/completed  or

both

Each tense is named according to its combination of aspects and time.

Morphology – the structure of words

Technically, a word is a unit of language that carries meaning and consists of one or more morphemes which are linked more or less tightly together, and has a phonetic value.

Typically a word will consist of a root or stem and zero or more affixes.

Words can be combined to create phrases, clauses, and sentences.

A word consisting of two or more stems joined together is called a compound.

It is quite hard to define what exactly a ‘word’ is, because

what is classified as words in different language are different

determining word boundaries in speech is very complex (e.g. short words are often run together and long words are often broken up)

If a word is a unit of language that consists of one or more morphemes, then we need to know what a morpheme is.

A morpheme is the smallest linguistic unit that has semantic meaning

E.g

rude

un-true

smooth-ly

dis-organize-d

A word can consists of:

one morpheme (simple)

cat

travel

appear

more than one morpheme (complex)

cat-s

travel-ed

dis-appeare-d

There are 6 main types of morphemes:

free

bound

lexical

grammatical

inflectional

derivational

Free morphemes can constitute a word on their own:

Thakuru

will

a

Bound morphemes must appear with one or more morphemes to form a word:

Thakuru’s

helped

enable

Words often consist of a free morpheme with one or more bound morphemes attached to it:

en-danger-ed

In this sort of word, the free morpheme is called the root or stem, and the bound morphemes are affixes

An affix attached to the front of a word is called a prefix

An affix attached to the back of a word is called a suffix

 

lexical morphemes have lexical (semantic) meanings:

help

impressive

race

Grammatical morphemes provide grammatical information:

help-ed

under

endanger

Lexical morphemes tend to be free morphemes:

Hiyala

jump

afternoon

Grammatical morphemes may be either free or bound:

Hiyala’s

jump-ed

afternoon-s

 

Inflectional & Derivational Morphemes 

Bound grammatical morphemes seem to come in (at least) two types:

Inflectional

derivational

The precise difference between inflectional and derivational morphemes is hard to define

But the most obvious difference is:

derivational morphemes build new words by changing the meaning and/or syntactic category of the word

inflectional morphemes permit a word to agree with other words in its context by providing grammatical information

Communicative Language Testing

A new theory of language and language use that exerted significant influence on language teaching and therefore language testing from the early 1970s was the theory of communicative competence

Communicative competence is a linguistic term which refers to a learner’s second language ability. It refers to a learner’s ability to:

  • apply and use grammatical rules
  • form correct utterances
  • use these utterances appropriately

The term was coined by Dell Hymes (in 1966) who was inspired by Noam Chomsky‘s distinction on linguistic competence and performance.

According to Chomsky (1965) a speaker’s language ability comprised two components:

  • linguistic competence
  • linguistic performance

Hymes proposed that knowing a language entailed knowing more than its grammar and rules

According to Hymes there culturally specific rules that created a relationship between:

  • the language used
  • features of the communicative context

e.g.

What is appropriate language for communication with a sibling may not be appropriate for communication with an employer or lecturer.

Before the theory of communicative competence, language was often described from a psychological perspective, but this theory marked a profound shift in how language was perceived as it presented language as an internal phenomenon.

With the appearance of the communicative competence theory the focus shifted to a more sociological one, where the focus was on external, social functions of language.

The relevance of Hymes’ theory to language testing was almost immediately recognized when it appeared.

However, it was a decade later that its actual impact was felt on practice with the development of communicative language testing

Communicative language tests (CLT) are distinguished by two main features:

  • CLTs are performance tests and therefore require assessment to be carried out when the learner or candidate in engaged in an extended (receptive/productive) act of communication
  • CLTs pay attention to the social roles candidates would assume and hence considers the roles that candidates would assume in the real world on passing the test and offers a means of specifying the demands of such roles in detail

 

Models of communicative ability

Thought it was a challenge to shift perspectives not to mention focuses of language tests, there was a continuous theoretical engagement with the idea of communicative competence and its implications for the performance requirements of communicative language testing since the advent of the theory of communicative competence.

A number of writers have tried to specify the components of communicative competence in second languages and their role in performance.

The purpose of this is to provide a comprehensive framework for:

  • test development
  • testing research
  • interpretation of test performance

The first such models specified the components of knowledge of language without dealing in detail with their role in performance.

In 1980, Michael Canale and Merrill Swain published a paper that specified four components of communicative competence:

  • Grammatical competence – knowledge of systematic features of grammar, lexis and phonology
  • Sociolinguistic competence – knowledge of rules of language use in terms of what is appropriate in different contexts
  • Strategic competence – ability to compensate for incomplete or imperfect linguistic resources in a second language by using (other) successful communication strategies
  • Discourse competence ability to deal with extended use of language in context (cohesion and coherence)

Types of Language Tests

All language tests are not of the same kind.

They differ mainly in terms of design (method) and purpose.

In terms of method, a broad distinction can be made between pen-and-paper language tests and performance tests

Paper-and-pen tests are typically used for the assessment of

  • separate components of language (grammar, vocabulary …)

  • receptive understanding (listening & reading comprehension)

Test items in such tests (especially if they are professionally made, standardized tests) are often in fixed response format (e.g. MCQ)

In performance tests language skills are assessed in an act of communication.

e.g. tests of speaking and writing where:

  • extended samples of speech/writing is elicited
  • judged by trained markers
  • common rating procedure used

Main distinction in terms of test purpose:

Achievement tests

Proficiency tests

Achievement Tests are associated with the process of instruction and should support the teaching to which they relate by measuring what students (would) have learned as a result of teaching.

E.g.

  • end of course tests
  • portfolio assessments
  • observation procedures for recording & assessing classroom work/participation

Achievement tests may be self-enclosed i.e. may not bear any direct relationship to language use in the real world

E.g. focus on knowledge of particular areas of grammar/vocabulary

However, if the curriculum is designed to reflect language use in the real world, achievement tests will automatically reflect normal language use and can be designed in innovative ways to reflect progressive aspects of the curriculum.

That is why achievement tests are associated with the most interesting development in language assessment – alternative assessment.

Alternative assessment stresses the need for assessment to be integrated with goals of curriculum and promotes a constructive relationship with the teaching/learning process.

Proficiency Tests look to the future situation of language use without necessarily any reference to the process of teaching.

In these tests, performance is measured in relation to a targeted level known as the criterion.

Main purpose of performance tests is to make inferences, however they are not valued in themselves but as indicators of how the test-taker will perform similar (or related) tasks in the real world setting of interest.