Monday, January 11, 2010

Methdology: Rasch Analyses - what they might add

The past two blogs I have written have been about factor analysis - a data reduction technique I have used frequently. For example, I have often used exploratory factor analyses to determine which items to keep as part of survey constructs. However, I was recently shown some Rasch analyses which showed me such characteristics as item fit and construct reliability, as well as how well items discriminate among persons (e.g. are they easy to agree too or hard to agree to) and how persons viewed response options.

For example, while I was able to use factor analyses to determine that a specific item loaded onto my construct at a weight of .765, I could have found the same thing ("good infit") using Rasch analyses. However, the use of Rasch analyses also showed that this was one of the easier items to agree to (meaning it had low discrimination among persons, not what I wanted). Rasch analyses also told me something about my response options: the Rasch analyses showed that persons did not follow a trajectory from Highly disagree to Somewhat disagree, to Neither agree or disagree, to Somewhat agree, to Highly agree. Rather, a person would go through this sequence but without using the middle response option (Neither agree or disagree). This suggests that instead of a 5-point scale with a middle "neutral" response option, respondents in actuality responded as if the response options represented a 4-point scale, treating the middle option as "Not applicable".

That's a lot of information from one analysis! I plan to try conducting a Rasch analyses next time I would have used factor analyses. One popular program used to conduct such analyses is Winsteps (www.Winsteps.com). It has a primer / how-to book, but I would also suggest reading Bond and Fox's book, "Applying the Rasch Model". It shows some examples using Winsteps as well as the text one could use to reproduce their findings.

*With special thanks to Andrew Swanlund at Learning Point Associates for making me a believer!

No comments: