Ep 168: Reliability – the Foundation of Any Good Test

MichaelCritical Thinking, Personality, Research and Stats5 Comments

Can you tell your personality from what flavor ice cream you like? There are lots and lots of so called Personality tests on the web. How do you know when you’ve come across a good one? For example, are the inner workings of your personality revealed in the playlists you have on your smartphone or mp3 device?

In this episode we take a look at the first thing you should demand from any test – reliability. If you’re interested in psychology you are going to learn A LOT about the different kinds of reliability.


Some Types of Test Reliability

    • Test-Retest reliability: people should get about the same score every time they take your test. You can’t score highly on an introversion scale one week and then low the next week (well, you could, but then we’d know that the test is no good).
    • Split-Half reliability: if your test had 20 questions (and was measuring just one concept, like shyness) and we correlated the odd questions with the even ones we should get a high positive correlation.
  • Alternate Form reliability: in this scenario, psychologists create two questionnaires, both of which measure the same concept (again, such as shyness). We give you one form one week and the other form a week (or month) later. You should score about the same on both forms.

Resources on Personality and Reliability

    • Miserandino, M. (2006). “I Scream, You Scream: Teaching Validity and Reliability Via the Ice Cream Personality Test.” Teaching Psychology. 33(4), 265-68.
    • The Distorted Tunes test
    • The What is Reliability page on psychology.about.com lists a few more types of test reliability.

Related Posts Plugin for WordPress, Blogger...

5 Comments on “Ep 168: Reliability – the Foundation of Any Good Test”

  1. Yes, you’re right Stu: a lot of things change during the adolescent years, so it’s possible for a test score to change during a year and thus it could be seen as unreliable – at least as compared to the score a year earlier. Good point. I was thinking of aspects of adults that typically don’t change from year to year.

    Glad you’re enjoying the podcast!

  2. have a question to do with reliability. you mentioned how if you came back in a year your results could be the same. true this should be in most people like adults. i was wondering if this is the same with teenagers and adolescents as i believe personalities could change due to life changing events or concepts in the idea of life. maybe this links to eysenck’s dimension of neuroticism. or maybe im over complicating the issue. any way it was a good episode, keep ’em coming

  3. AJ: Glad you liked this episode. I decided to discuss reliability and validity in separate episodes. Heard that joke about the “two types of people”. It’s a goodie.

  4. Great to see you getting into the subject of personality tests, a favourite topic of mine.
    I am looking forward to the follow-up episodes.
    I personally think there are only two types of people in the world, those who split people into two group and those who don’t.

Leave a Reply

Your email address will not be published. Required fields are marked *