Structural mean models (SMMs) were originally formulated to estimate causal effects among those selecting treatment in randomised controlled trials affected by non-ignorable non-compliance. It has already been established that SMM estimators identify these causal effects in randomised placebo-controlled trials where no-one assigned to the control group can receive the treatment. However, SMMs are starting to be used for randomised controlled trials without placebo-controls, and for instrumental variable analysis of observational studies; for example, Mendelian randomisation studies, and studies where physicians select patients' treatments. In such scenarios, identification depends on the assumption of no effect modification, namely, the causal effect is equal for the subgroups defined by the instrument. We consider the nature of this assumption by showing how it depends crucially on the underlying causal model generating the data, which in applications is almost always unknown. If its no effect modification assumption does not hold then an SMM estimator does not estimate its associated causal effect. However, if treatment selection is monotonic we highlight that additive and multiplicative SMMs do identify local (or complier) causal effects, but that the double-logistic SMM estimator does not without further assumptions. We clarify the proper interpretation of inferences from SMM estimators using a data example and simulation study.