In episode 168 I discussed reliability but the real test of a test’s “mettle” – or how confident you can be in the results – comes when you subject it to lots of validity tests. You’ll learn a lot about validity if you take a psychology class. High validity is what separates the many fun-to-take but essentially meaningless tests you’ll find on the web, and a truly solid test of your personality.

Types of Test Validity

• • Face Validity: does the test measure what it appears to measure? The Distorted Tunes test has high face validity, i.e., it’s pretty obvious what the test is measuring. The Rorshach, on the other hand, has low face validity: it’s not all clear to the test-taker what the test is measuring. Sometimes you want low face validity. If you think people won’t tell the truth, or that they really don’t know what their personality is like, or you’re afraid of the Social Desirability effect (people will give you responses that make them look good) then you might want to use a projective test like the Rorshach or the Thematic Apperception test (TAT).
  • Concurrent validity: do the results of the test agree with other aspects of the person’s current life? If, for example, the results of an achievement test indicate that you have a high achieving personality, then you should probably have high grades in school or you should make a good deal of money in your job. The test should correlate in a predictable way with data this is currently available from you.
  • Predictive validity: do the results of the test predict some future state of your life? If the test says you’re a high achiever, then 10 years from now you should probably be making good money in your job. If the test says you have high musical ability, then 10 years from now you might likely be employed in some way in the music industry.
  • Convergent validity: your test results should agree with other tests that measure similar concepts (or constructs as we often call them in psychology). Example: high achievers are probably outgoing people, so if you score highly on an achievement personality test then you should probably also score highly on a test of extroversion.
  • Divergent validity: your test results should disagree (or not correlate at all) with tests that have nothing to do with the concept the test measures. An achievement-orientation test, for example, probably shouldn’t correlate at all with a test of whether or not you are a “trusting” person. The two ideas really don’t have much to do with one another. Achievement-orientation might, however, correlate negatively (diverge) with a test of your tendency to avoid conflict. High achievers probably don’t avoid conflict – in face, they may enjoy it.

Resources on Personality and Validity

• • Miserandino, M. (2006). “I Scream, You Scream: Teaching Validity and Reliability Via the Ice Cream Personality Test.” Teaching Psychology. 33(4), 265-68.
  • The Distorted Tunes test

• • The What is Reliability page on psychology.about.com lists a few more types of test reliability.