Test Retest Reliability
From ICE Primer: A Tobacco Control Research Methodology Primer
Measurements obtained from subjects in tobacco control studies, whether these be anthropometric such as height or weight, from questionnaires such as smoking behaviours, attitudes, etc., or biochemical, should meet certain standards of reliability and validity.
A reliable measuring system is one where we obtain the same measure from the same individual on different occasions when the measure would not be expected to have changed. Validity refers to whether the measured value is equal to the actual value of the variable being measured.
Test-retest reliability is one method for assessing the reliabity of a measuring system. To illustrate, consider a questionnaire that is collecting information on smoking attitudes and behaviour. If we administer the same questionnaire to the same subjects on different occasions, we should expect to see the same responses to those items where there has been no change in the true value of the response over the time period. By measuring the degree of association between the responses, we can get a measure of the test-retest reliability of the items on the questionnaire.
Test-retest reliability studies are important in the design of a questionnaire. The main concern is to choose the time interval between successive administrations of the questionnaire that is long enough that a subject cannot merely remember the previous response, but not so long that responses to the items will have changed. One or two weeks are common time intervals for test-retest studies of smoking related studies.
Correlation coefficients are often used to measure the degree of association between the responses for the two times. Correlations greater than 0.7 would be expected for decent measures. Note, however, that correlation coefficients measure association and not agreement, so confirming that the average of the difference between the responses for the two occasions is zero is important, too.
