Not common: If your data is a census data (everyone is in the dataset, like US Census), you do not need to use HLM. You do not even need statistical testing because every estimate you get is a true estimate. (Note: my friend wrote me and said he disagrees. He said even if he had all people's data he wants to do stat testing to compare male and female when the difference is very small. Again I disagree.)
Common: If your data is a sample and data are hierarchically structured and thus errors are dependent (violation of the independence assumption), you may consider HLM to alleviate the clustering problem. Examples of hierarchically structured data are:
- Students nested within schools (2-levels)
- Repeated measures nested within subjects who are nested within schools (3-levels)
It is sometimes said that the motivation for the use of HLM must be whether the group units are a random sample of the population. The argument claims that if, for example, schools in the sample are a convenient sample, one cannot use HLM. This is not exactly correct. I state the following using RCT (randomized control trial) or QED (Quasi-experimental design) impact studies as a context.
If the group units are randomly sampled, one can generalize the result of impact analysis to the whole population. If Intervention program A was found effective (or not effective) and the sample was a random sample of US population, this finding is generalizable. If impact estimation relied on a convenient sample, one cannot generalize it to the whole population. If RCT, the random sample vs. convinient sample difference should not affect the internal validity of the impact estimate.
There is a tricky case. HLM is inappropriate if the group units are, for a lack of better word, distinct groups with apparent identity and as a researcher you are genuinely interested in the pure group estimates. This is a case when the exact group estimate, derived as fixed effects not as random effects, are of interest.
For example, if group units are US states, HLM is most likely inappropriate. State specific estimates should be interpreted as such and should not be treated as random effects. Imagine the outcome of interest is income level and the state you live in had the average income of 50,000. Just because your state had a small number of survey respondents (and thus reliability of the estimate is lower and HLM will pull your average closer to the grand average), you do not want to see your state’s average to be changed to look more like the national average. Another example would be a study of 20 regional hospitals. You should be interested in the fixed estimates of hospital outcomes.
When HLM treats schools as the mixed effects, we are treating school units somewhat instrumentally (a bit rude thing to do :>) in order to obtain the best value for the intercept (=grade average of the group specific effects estimated as random effects). So if you are a school, you may feel HLM is treating you without respect. HLM will not respect your data if the sample size is small and outcome variance in your school is large. But HLM is respecting you in a different way. Your data is unreliable, so let me just adjust it to be more normal, so you won't embarrass yourself.