Estimating causal moderation effects with randomized treatments and non‐randomized moderators

Researchers are often interested in analysing conditional treatment effects. One variant of this is ‘causal moderation’, which implies that intervention upon a third (moderator) variable would alter the treatment effect. This study considers the conditions under which causal moderation can be identified and presents a generalized framework for estimating causal moderation effects given randomized treatments and non‐randomized moderators. As part of the estimation process, it allows researchers to implement their preferred method of covariate adjustment, including parametric and non‐parametric methods, or alternative identification strategies of their choosing. In addition, it provides a set‐up whereby sensitivity analysis designed for the average treatment effect context can be extended to the moderation context. To illustrate the methods, the study presents two applications: one dealing with the effect of using the term ‘welfare’ to describe public assistance in the United States, and one dealing with the effect of asylum seekers’ religion on European attitudes towards asylum seekers.

BANSAK researchers can better explain the phenomena they are studying and determine the likelihood that a treatment will be effective if introduced to other populations in the future. Most commonly, researchers will investigate whether the effect of a treatment T on an outcome Y is conditional upon the value of a third variable S.
However, if researchers find evidence of treatment effect conditionality, they then face an additional hurdle in interpreting the nature of that conditionality. On the one hand, they may simply observe a difference in the treatment effect across different values of S because the treatment is more or less impactful for different segments of the population. This phenomenon is often referred to as treatment effect heterogeneity. In this case, it can be established that the treatment effect is different across subgroups with different values of S, but what cannot be established is whether that difference is attributable to S itself.
On the other hand, however, researchers may be interested in further claiming that S is precisely the reason for the difference in treatment effects-that is, changes in S actually cause changes in the effect of T on Y. This phenomenon, which is the focus of the present study, will be referred to as the 'causal moderation' of treatment effects, which is the causal effect of a third variable on the treatment's effect on the outcome. In contrast to treatment effect heterogeneity, which deals with extant variation in the treatment effect across the population, causal moderation implies that counterfactual intervention upon a third (moderator) variable would alter the treatment effect. This study examines whether, when and how experimental researchers can actually investigate causal moderation rather than limit their focus on treatment effect heterogeneity.
Evidence of causal moderation has important theoretical and policy implications that do not follow from the existence of treatment effect heterogeneity in general, and as such, researchers are often interested in making claims about causal moderation. For instance, the existence and magnitude of causal moderation determines the extent to which a treatment effect would be stable, amplified or attenuated if a third variable were also manipulated, thereby providing unique insights on the potential generalizability of the treatment effect.
Researchers focused on causal moderation, however, must confront additional estimation and identification hurdles when the moderator variable has not been randomized. This is a common situation in experimental social science, where financial, logistical, physical or other constraints allow the researcher to randomize certain variables (the treatments) but not randomize other variables that are believed to interact with the treatment. In this situation, treatment effect conditionality is most commonly analysed on the basis of conventional subgroup analysis and/or underspecified interaction regression models. As this study will show, these standard approaches generally do not provide unbiased or consistent estimates of causal moderation effects given non-randomization of the moderator. Yet applied researchers using these standard approaches will often make claims suggestive of causal moderation without fully grappling with the identification hurdles. For instance, a study on partisan identity and political emotion evaluates the extent to which political messages threatening a party's electoral loss cause anger for copartisans (Huddy et al., 2015). In assessing whether the 'partisan threat' experimental treatment leads to greater anger among respondents with stronger partisan identity, the authors describe this as the 'effect of partisan identity…on emotional response to partisan threat and reassurance', suggesting a causal moderation phenomenon.
This study considers the conditions under which causal moderation can be identified and presents a generalized framework for estimating causal moderation effects in the specific design context of randomized treatments and non-randomized moderators. As part of the estimation process, it allows researchers to implement their preferred method of covariate adjustment, including parametric and non-parametric methods, or alternative identification strategies of their choosing. To illustrate the methods, two applications are also presented: one dealing with the effect of using the term 'welfare' to describe public assistance in the United States and whether that effect is moderated by party | 67 BANSAK identification, and one dealing with the effect of asylum seekers' religion on European attitudes towards asylum seekers and whether it is moderated by nationalism.

| Set-up and notation
Suppose a simple random sample of size N is drawn from a larger population of interest. Units in the sample are indexed by i = 1, …, N, and are assigned to either a treatment condition or a control condition. In addition to measuring an outcome for each unit, also assume measurement of a third variable which will be called a moderator. Employing the potential outcomes framework of Neyman (1923) and Rubin (1974) to define the causal effects of interest, this study posits for each unit i the existence of Y i (t, s) ∈ at all values of the treatment t ∈ and moderator s ∈ . Y i (t, s) denotes the potential outcome of unit i if that unit had treatment status t and moderator value s.
For each unit in the sample, we observe where Y i ∈ denotes the realized outcome, T i ∈ denotes the assigned treatment status, S i ∈ denotes the observed moderator value, and X i ∈ denotes a vector of other observed covariates. Note that S i and X i will be assumed to be pre-treatment. Under the definition of the potential outcomes, the realized outcome Thus, only one potential outcome is observed for each unit. However, implicit in the postulation of potential outcomes Y i (t, s) for all t ∈ and s ∈ is that both the treatment and moderator are assumed to be mutable at the individual level. The variables contained in X i may or may not be manipulable.
Note that this study proceeds from the perspective of sampling from a larger population. Hence, the causal estimands that will be presented pertain to the target population, with expectations taken over the population distribution. However, similar to standard frameworks for identifying an average treatment effect (ATE), the general identification results are not exclusive to this particular sampling context and would proceed similarly in both the super-population approach and the sample-as-population approach, where the sample observed is treated as the population of interest (see Aronow & Samii, 2016;Imai et al., 2008;Imbens & Rubin, 2015). Indeed, the sample-as-population perspective can be viewed as a special case that nests within the larger-population perspective, which follows from considering a sample of size N (without replacement) from a population of the same size.
For simplicity, the rest of the study will consider binary versions of the treatment and moderator such that ≡ {0, 1} and ≡ {0, 1}. As will be presented, the causal estimands of interest will employ comparisons of potential outcomes Y i (t, s) where t and s take on two separate values at a time, and the values 0 and 1 will be used for convenience. In practice, however, the treatment and moderator could take on any number of values, and 0 and 1 used to denote any pairs of values of interest.

| Treatment effect heterogeneity versus causal moderation
To illustrate what is at stake in investigating causal moderation, in contrast to other types of treatment effect conditionality, consider the following hypothetical experiment. Imagine a study where, in order to investigate a potential pathway for increasing local public goods provision (Y) across towns (the units of analysis) in a developing country, researchers run a field experiment that draws a sample of towns and then randomly assigns certain towns to a program (T) that provides specialized training to its local public officials. The researchers find that on average the treatment works-the training program improves public goods provision-but also that it appears to have differential effects for 68 | BANSAK different types of towns. Specifically, the researchers suspect that the treatment is more effective in towns that have more officials per capita, that is, towns with higher governmental capacity (S). Deciding how to extend their analysis and interpret their results then depends upon the researchers' goals.
As a first goal, the researchers may simply want to find treatment effect heterogeneity. That is, they may want to know the types of places where the training program is most effective, so that a future training policy can be rolled out in a targeted and cost-efficient manner. They do not necessarily need to know why it is more effective in certain places; they just need to be able to predict what those places are likely to be. In this particular example, investigation of treatment effect heterogeneity across towns with low and high capacity levels could proceed using simple subgroup analysis. From this perspective, researchers would be interested in the following estimand, which captures treatment effect heterogeneity: where the expectations are taken with respect to the distribution of potential outcomes over the population from which the sample was drawn.
Given that the expectation for each potential outcome is taken conditional on the observed moderator value being the value of the moderator in the potential outcome, this estimand effectively treats the moderator as a purely observational variable with no causal component of its own being measured. In other words, it captures the difference between the ATE of T for units whose moderator value is observed to be 1 and the ATE for units whose moderator value is observed to be 0. Accordingly, this estimand could be re-expressed using reduced form potential outcomes Y i (t), t ∈ {0, 1}, as follows: . (Note also that for treatment effect heterogeneity, the moderator would not need to be mutable at the individual level.) Researchers may also want to identify other variables across which the treatment effect is heterogeneous but may not know which observed variables to focus attention on. Recent years have seen an increase in interesting research into new methods to discover, estimate and perform statistical inference on treatment effect heterogeneity beyond simple subgroup analysis, much of which integrates machine-learning techniques (Athey & Imbens, 2016;Beygelzimer & Langford, 2009;Ding et al., 2016;Dudík et al., 2011;Foster et al., 2011;Green & Kern, 2012;Imai & Ratkovic, 2013;Künzel et al., 2017;Powers et al., 2017;Ratkovic & Tingley, 2017;Su et al., 2009;Tian et al., 2014;Wager & Athey, 2018;Weisberg & Pontes, 2015;Zeileis et al., 2008).
In some cases, however, discovering the existence and extent of treatment effect heterogeneity may not be the ultimate goal of a researcher's inquiry into causal effect conditionality. Instead, researchers may want to estimate causal moderation effects. At stake here is the ability to claim that a moderator variable truly causes changes in the treatment effect, or that the treatment effect would remain stable if the moderator itself were to be manipulated. The corresponding causal moderation estimand of interest will be referred to as the average treatment moderation effect (ATME) and denoted by δ.

Definition 1
The ATME is defined as where the expectation is taken over the population distribution.

BANSAK
Whereas treatment effect heterogeneity deals with variation in the treatment effect across different spaces of the observed distribution in the population, causal moderation implies that counterfactual intervention upon the moderator variable would alter the treatment effect.
In the example above, the researchers may have found that the training program is more effective in high-capacity towns, but they did not randomly assign capacity and thus this result could be because capacity is related to any number of other things that are truly responsible for increasing the effectiveness of the treatment. If their interest is in causal moderation, then the researchers need to know if it truly is higher capacity that causes the training program to be more effective. The answer to this question has theoretical and practical policy importance. If there is reason to believe that policy or other shocks in the future may lead to local capacity growth (or loss), then the researchers should be interested to know how this will likely impact the success of training programs that are introduced in the future (e.g. whether the treatment effect will generalize to new scenarios). Alternatively, they may be interested to know if it would be cost-efficient to add funding for additional public official employment to future training program roll-outs (e.g. whether the treatment regime can be optimized via additional interventions). To achieve this, finding treatment effect heterogeneity is not sufficient; what is needed is the estimation of causal moderation.

| Causal moderation and interaction in other contexts
The ATME as defined in this study is an estimand that has been discussed in previous scholarship on variable interactions. VanderWeele (2009VanderWeele ( , 2015 has referred to this phenomenon as 'causal interaction' or more simply 'interaction' (while referring to treatment effect heterogeneity as 'effect modification') and Egami and Imai (2019) use the term 'average interaction effect'. However, these previous studies consider the estimand within different design contexts than that of the present study. VanderWeele (2009VanderWeele ( , 2015 considers identification and estimation of the estimand in the general case, without explicitly considering whether the variables have been randomized. In contrast, the present study considers the specific case faced by experimental researchers of one randomized variable and one non-randomized variable, providing a more targeted set of identification assumptions that researchers can think through intuitively, as well as a flexible and straightforward estimation strategy for this context. Conversely, Egami and Imai (2019) discuss the estimand in the special case of two randomized treatments. Consequently, their context does not confront the same identification and estimation challenges present in the design context of interest in this study.
Finally, it is important to explicitly note that this study formulates the ATME (as well as treatment effect heterogeneity defined earlier) in terms of a difference model. In certain scholarly contexts, such as the study of risk factors in epidemiology, a key distinction is made between exposure variable interactions measured on a difference scale versus a ratio or other scale (for discussion of this issue and its relationship to the concept of 'biologic' vs. statistical interactions, see Andersen & Skrondal, 2015;Rothman et al., 1980). This can be an important distinction to make because an interaction may manifest on one scale but not another. In addition, as has been observed with respect to the difference-in-differences method of estimating treatment effects (e.g. Meyer, 1995), nonlinear transformations of the outcome variable may also affect the results. Throughout its entirety, this study adopts a difference perspective on variable interaction and moderation, as is clear from the 'difference-in-differences' form of both the ATME and treatment effect heterogeneity estimands defined above.

| IDENTIFICATION AND THE PARALLEL ESTIMATION FRAMEWORK
This section lays out the assumptions under which the ATME can be identified, presuming a design context involving a randomized treatment and non-randomized moderator, and introduces a generalized framework for estimating the ATME.

| Identification
To achieve identification of the ATME, this subsection presents a strategy in which selection on observables is applied to the moderator. (However, note that other identification strategies could also be applied under alternative assumptions, a matter that will be revisited later.) The first identification assumption is the stable unit treatment value assumption (SUTVA).
S denote the full treatment and moderator vectors across subjects in the sample i = 1,2, …, N.
In accordance with the design context of interest, the second identification assumption is that the treatment has been randomized. While both the identification results and estimation approaches that will be presented could accommodate any randomization scheme where all units share the same treatment assignment probability-including Bernoulli randomization, complete randomization and stratified randomization with constant probabilities across strata-complete randomization simplifies variance derivation and estimation.

Assumption 2 (Complete Randomization of the Treatment)P(
for all a and a ′ such that T a = T a � where ι is the N-dimensional column vector with all elements equal to one.
The third assumption is that the moderator S and a set of observed confounder variables X are pre-treatment, and hence the treatment can have no individual-level effects on S or X. Combined with Assumption 2, this implies that the moderator, confounder variables, and potential outcomes are jointly independent of the treatment.

Assumption 3 (Pre-Treatment Moderator and Confounders)The moderator S and a set of observed
confounder variables X are pre-treatment such that, given randomization of the treatment, The fourth identification assumption is conditional independence between the potential outcomes and the moderator.

Assumption 4 (Moderator Conditional Independence)
Note that given a randomized treatment and pre-treatment status of S and X, the identification burden implied by Assumption 4 is no more stringent than the burden involved in the standard causal BANSAK identification of an ATE via selection on observables. In addition, identification via selection on observables also requires a common support assumption.

Assumption 5 (Moderator Common Support)
The assumptions delineated above allow for identification of the ATME. See Appendix A in the Supplementary Materials (SM) for a proof.

Proposition 1 Given Assumptions 1-5,
where the outer expectation is taken with respect to the population distribution of X.

| Estimation
The expression in Proposition 1 provides a preliminary basis for estimation of δ. However, for a number of reasons that are described below, it is useful to disaggregate the expression into two separate components. Specifically, Assumption 3 implies that F X|T (x|t) = F X (x) for all x and t, where F X denotes the joint distribution function of X and F X|T denotes the joint distribution function conditional on T.
for any t, s. Hence, Equation (1) can be re-expressed as follows: While this may seem a simple mathematical re-expression, seeing the estimand explicitly in this form is valuable and informative for several reasons. First, it points to an initially non-obvious opportunity for estimation in a manner that is highly accessible to applied researchers. Specifically, it highlights that researchers can simply subset their data by treatment level and then proceed to apply covariate-adjustment estimation strategies separately to the treated units' data and control units' data. That is, can be estimated separately for T i = 0 and T i = 1 by a covariate-adjustment estimator of the researcher's choice. This is equivalent to holding T constant and then estimating the causal effect of S on Y as one normally would in an observational study. A variety of straightforward and widely used methods for performing this estimation already exist, such as regression, matching and propensity score weighting. As a result, applied researchers can easily adapt the tools they are already familiar with and prefer using. The simple difference between the two subsetted estimates then constitutes an estimate of the ATME. While estimation of the ATME in this manner is not strictly necessary, it provides a highly accessible and flexible approach, which will be referred to as the 'parallel estimation framework'.
Second, the disaggregation and ensuing parallel estimation approach improves not only accessibility but also tractability in the case of certain non-parametric covariate-adjustment methods. In particular, it facilitates the use of matching, one of the most popular methods used in the estimation of causal effects of non-randomized variables. Again, note that estimation of the ATME via the parallel estimation framework is not strictly necessary; one could instead specify a general model for E[Y i |T i , S i , X i ] as the basis for estimating the ATME. Of course, the key caveat for this (and also for specifying subsetted models in the parallel estimation approach) is that the model must be correctly specified. To avoid such model dependence, researchers interested in estimating causal effects involving non-randomized variables often turn to matching (see Ho et al., 2007;Stuart, 2010). Employing matching would also be fruitful in this regard for estimating the ATME. Looking at the expression as presented in Proposition 1, one would be inclined to match quadruples of observations, in contrast to the normal requirement to match pairs. This would be problematic for a couple of reasons. Seeking to minimize the distance between four units requires minimizing some chosen metric that aggregates six pairwise distances, and hence achieving a certain quality of match (e.g. average pairwise distance) with finite data becomes more difficult with quadruples than with pairs. In addition, as a more practical matter, while there already exist reliable implementations of pair matching that are accessible to applied researchers across a range of popular statistical software, implementations of quadruple matching are not similarly pervasive. Thus, by allowing for the matching of pairs, the parallel estimation framework effectively enables the use of matching in the ATME estimation.
Third, the disaggregation also provides simplifying insights on how to characterize the statistical properties of and conduct inference on estimates of the ATME. As already described above, the parallel estimation framework allows researchers to employ their preferred method of covariate adjustment to estimate the quantity in expression (3) separately for t = 0, 1. Under the assumptions presented earlier along with any functional form assumptions made by the researcher to accompany their estimation method (e.g. linearity in the case of employing linear regression), the estimates of expression (3) for t = 0, 1 will exhibit the unbiasedness and/or consistency of the estimation method. In turn, given the difference form, the estimate of the ATME will inherit this unbiasedness and/or consistency. Furthermore, the two subsets (T i = 0 and T i = 1) can be viewed as approximately independent. As a result, the variance of the ATME estimate can be approximated and estimated directly as the sum of the variances of the two within-subset estimates. In addition, the parallel estimation framework also facilitates a sensitivity analysis of the ATME estimate, described in detail in Appendix D.
Finally, the fourth benefit of the disaggregation is that it helps to formally reveal how certain common approaches used by researchers in the analysis of treatment effect conditionality are not well suited to estimating the ATME. As described, estimation of expression (3) is performed separately for t = 0, 1. In contrast, however, subsetting the units by moderator level and taking the difference between estimates of the treatment effects within those subsets would lead to biased and inconsistent estimates of the ATME, as F X|S (x|s = 0) ≠ F X|S (x|s = 1) under the assumptions. This is an important, even if simple, insight given that a researcher's first instinct may be to subset along the moderator variable. Similarly, as will be described later, common regression specifications that include interactions also generally fail to recover unbiased and consistent estimates of the ATME.

| Alternative identification strategies
The parallel estimation framework also provides the opportunity to employ alternative within-subset identification strategies in place of selection on observables. Because of the randomization of the | 73 BANSAK treatment, the data can be split into treatment-level subsets, and a causal effect of the moderator on the outcome within subset can be identified and estimated according to other identification strategies, provided that the relevant variables and additional identification features are pre-treatment.
Specifically, given Assumptions 1 and 2, identification of the ATME could be achieved via an alternative strategy if (a) the additional requisite assumptions are met such that the strategy achieves identification of the causal effect of S on Y in both subsets T i = 0 and T i = 1, and (b) the auxiliary variables and features used in the alternative identification strategy are pre-treatment. If a regression discontinuity (or encouragement/instrumental variables) design were used as the within-subset identification strategy, the forcing variable (or encouragement/instrument and principal strata) would need to be pre-treatment or not otherwise impacted by the treatment. Using such identification strategies would also require viewing the ATME as an appropriately local estimand.
As an example, consider the encouragement/instrumental variables approach. Continue to make Assumptions 1 and 2, and now let Z i ∈ {0, 1} denote for each unit i the encouragement/assignment to the moderator S i ∈ {0, 1}. However, we allow for noncompliance with the encouragement/assignment such that for some units Z i ≠ S i . In this approach, Z i replaces X i as the auxiliary variable used in identification, and Assumption 3 is amended such that the moderator and instrument are pre-treatment. Then within each subset T i = 0, 1, a causal effect of S on Y can be identified using the local average treatment effect (LATE) framework applied to Y, S, and Z. In other words, Z can be regarded as the instrument and S as the treatment variable in the standard LATE framework, and the standard LATE assumptions then applied (Imbens & Angrist, 1994). Then within each subset T i = 0, 1, it is possible to estimate (e.g. via the Wald IV estimator) the LATE, which is the average causal effect of S on Y for compliers, defined as units for whom it would always be true that Z i = S i regardless of the value of Z i assigned. The difference between these two effects for each subset T i = 0, 1 then constitutes an estimate of the ATME for compliers.

ESTIMATION FRAMEWORK
To provide concreteness, given a randomized treatment and non-randomized moderator and under Assumptions 1-5, estimation of the ATME using the parallel estimation framework proceeds as follows: 1. Subset the data by treatment level. 2. Separately for each treatment subset T i = 0, 1, estimate the effect of S on Y.
• Use preferred method(s) of covariate adjustment (e.g. regression, matching, propensity score weighting, etc.) to perform the estimation controlling for X. • Alternative identification strategies (e.g. regression discontinuity, instrumental variables, selection on unobservables) could also be employed under alternative assumptions pertaining to the relationship between S and Y, provided that any auxiliary variables used for identification are pre-treatment. 3. Calculate the difference between the two estimates from Step 2, which provides an estimate of the ATME, ̂ .
• Provided that unbiased and/or consistent estimators of the effect of S on Y are employed in Step 2, then ̂ is also unbiased and/or consistent. • Var(̂ ) can be approximated and estimated directly as the sum of the variances of the two estimates recovered in Step 2.

BANSAK
The following subsections will describe a number of specific implementations of the parallel estimation framework using covariate adjustment and illustrate the problems with using standard approaches to estimate the ATME.

| Parallel within-treatment regressions
The simplest method of estimation would be to use a set of parallel linear least squares regressions. In each regression, the outcome Y is regressed on the moderator S, along with the full set of covariates X conditional upon which it is assumed that S is independent of the potential outcomes. This yields estimates of the average causal effect of S on Y within each treatment level, and their difference constitutes the estimate of the ATME, ̂ PR .

| Full interaction regression
The parallel regression equations also nest within a single regression equation. To see this, consider that the parallel regression equations can be combined as follows: As a result, ̂ FR =̂ PR ; the estimates are equivalent using both the parallel regression framework and the full interaction regression as specified above.
Note that T i X ′ i corresponds to the interactions between the treatment and all of the control variables. This highlights the problem-a problem that is common among researchers seeking to estimate moderation effects-with estimating an ATME in a single regression framework by merely controlling for the covariates by themselves and only including the interaction between the treatment and moderator. As is made clear by Equation (4), such a specification would not recover an unbiased or consistent estimate of the ATME. As a simple case, consider a data-generating process determined by the following model: where ε is distributed with mean zero and, for simplicity, X denotes a single covariate. This model corresponds to a situation where X not only affects the outcome itself but also, similar to the moderator of interest S, interacts with the treatment's effect on the outcome. Hence, it is not sufficient to simply include X as a regression control variable by itself in order to estimate the ATME, as omission of the TX term would lead to omitted variable bias that affects ̂ FR given a non-zero value of ξ and a non-zero covariance between S and X.
An interesting special feature to note about this linear context, already evident from the discussion above, is that the full interaction regression simultaneously recovers an estimate of the causal BANSAK moderation effect of S (i.e. the ATME) as well as estimates of the causal moderation effects of any number of other variables contained in X = (X 1 , …, X k , …) � in Equation (4), provided that those variables also satisfy the necessary conditions. In particular, each must also be manipulable at the individual level to have a well-defined causal moderation effect, and each must be independent of the potential outcomes conditional on the other included variables. If either of these conditions is not met for any given variable X k , then while inclusion of k TX k in the Equation (4) specification is necessary for estimation of the ATME, k itself does not correspond to an identified causal moderation effect. Note, however, that the possibility of estimating multiple causal moderation effects simultaneously does not generally carry over to other methods of estimation researchers would likely be interested in employing, as presented below.

| Parallel matching and alternative methods
Linear regression approaches to estimating causal effects, whether for the ATME as described above or for individual ATEs with observational data, rely upon specific model assumptions. This includes, of course, linearity. In addition, less obvious is the implicit assumption of constant effects given that linear regression in general performs a conditional-variance-weighting scheme that diverges from the type of weighting necessary to achieve proper average effect estimates (see Angrist & Pischke, 2009, chapter 3). These assumptions may be undesirable given certain data and when the estimation results are likely to be sensitive to functional form and model specifications in general. Thus, researchers may wish to avoid the model dependence of linear regression (or any other parametric method for that matter) in the parallel estimation process. Hence, an attractive non-parametric alternative for estimating the ATME would be to use parallel within-treatment-subset matching methods. The parallel matching process is analogous to the parallel regression approach: 1. Split sample into treated and control subsets. 2. Within each subset, match 'moderated' units (S i = 1) and 'unmoderated' units (S i = 0) on the confounding variables (X i ). Within each matched sub-sample, estimate the effect of S on Y. 3. Compute the difference between these two estimated effects, which provides an estimate of the ATME.
As with the implementation of matching in the ATE context, standard diagnostics can be performed for both the treated and control sub-samples to assess common support and the quality of the matching process in producing covariate balance, and various post-matching techniques can be applied to estimate the within-subset effects of S on Y (Ho et al., 2007;Stuart, 2010). More generally, any method of causal effect estimation under selection on observables can be applied to step 2 above. See Appendix B for additional details.

| Comparing the parallel estimation framework to conventional approaches
The two standard approaches most commonly used in the literature to estimate moderation effects are subgroup analysis and interaction regressions. As the results presented in this study demonstrate, these conventional methods generally do not constitute unbiased or consistent estimators of the 76 | BANSAK ATME given a non-randomized moderator. The first approach, subgroup analysis, involves subsetting the data by different levels of the moderator variable, estimating the treatment effect within subsets, and calculating the difference between the two. This may be referred to alternatively as treatment-bycovariate interactions and comparison of conditional average treatment effects (CATEs) (cf. Gerber & Green, 2012). Except in the special case where both the treatment and moderator are randomly assigned, such estimates can clearly not be interpreted as causal moderation effects.
The second common approach is using a controlled interaction regression, in which the outcome is regressed on the treatment, the moderator, the treatment-moderator interaction, and control variables. As already shown above, this approach also falls short for estimating causal moderation in the case of a randomized treatment and non-randomized moderator. While this result has already been observed in specific fields of scholarship, including social psychology (Hull et al., 1992;Yzerbyt et al., 2004) and gene-environment interaction (Keller, 2014), it appears not to be widely appreciated. Instead, empirical researchers continue to investigate causal moderation phenomena using interaction regressions that control for possible confounders but not their interactions with the treatment.
The controlled interaction regression specification can, of course, be extended for estimation of causal moderation by including additional interaction terms. That this appears to be done rarely suggests that, in spite of the large corpus of methodological literature on interaction terms in regression (e.g. Aiken & West, 1991;Brambor et al., 2006;Hainmueller et al., 2019;Kam & Franzese, 2009), there remains a lack of clarity on how to choose which interaction terms to include in one's regression model and which to omit. One problem is that interaction/moderation effects are often discussed without explicitly stating what type of estimand (descriptive versus causal) is of interest. For more discussion on this, see VanderWeele (2015, pp. 268-270) and Kraemer et al. (2008). A second problem is that interaction/moderation effects are often discussed without reference to whether the treatment and moderator of interest are exogenous or randomized. The result is a lack of explicitness in terms of what types of confounding are possible and, accordingly, what types of adjustments must be made in response.
In sum, while there already exists extensive methodological literature on interaction and moderation effects, the disparate assumptions and analytical goals tacitly underlying different recommendations makes it difficult for researchers to know exactly what to do given their own specific data and goals. Furthermore, because most of the literature is anchored within the regression context, the recommendations often lack general formal results to justify certain specifications; researchers must instead fall back on the usual, slightly uncomfortable assumption that the posited parametric model is correct. This is not reassuring for researchers who want a robust model that is both motivated by clearcut theoretical guidance yet also acknowledges that the estimation models we specify are rarely correct or complete. By formalizing the definition of a causal moderation effect in the form of the ATME and discussing it within the specific design context of a randomized treatment and non-randomized moderator, this study clarifies for experimentalists precisely what is meant by causal moderation, what assumptions its identification requires, and how its estimation can be undertaken.

| APPLICATIONS
This section presents two applications to provide concrete illustrations of the parallel estimation framework. Both applications come from previous studies involving experimentally manipulated treatments, and under consideration is the extent to which a separate, non-randomized variable moderates the treatment effect. Thus, the applications fall within the present scope of dealing with a randomized treatment and non-randomized moderator. Furthermore, both applications deal with moderators that are mutable at the individual level.

BANSAK
As already described, in addition to randomization of the treatment, identification of the ATME requires a number of other assumptions. Of particular note is the observation of a set of control variables (X) conditional upon which the moderator is independent of the potential outcomes. Each application specifies a plausible set to meet this condition. Yet just as in the case of estimating ATEs for non-randomized treatments, this conditional independence assumption is non-refutable, and subject matter experts may identify other variables missing from the conditioning set. Note, however, that the applications are meant to be illustrative, and they provide the structure for further research should the correct conditioning set be measured in the future. In addition, identification of the ATME also requires the SUTVA assumption. In both applications below, the experimental designs make SUTVA defensible within the study contexts. However, this of course does not guarantee that SUTVA would be met in less highly controlled contexts, or that more real-world versions of the treatment would have similar effects. Finally, in both applications, the samples were not drawn through pure simple random sampling. However, we may adopt the sample-as-population approach and interpret the estimates as pertaining to sample estimands (Imai et al., 2008).

| Framing experiment on support for public assistance
Many scholars of American voter behaviour and political psychology study how issue frames can be used by politicians to influence public attitudes, either positively or negatively, towards specific policies. One of the simplest yet most powerful and prominent examples is the framing of policies on public assistance to the poor as 'welfare'. While the term welfare is simply another label for public assistance, scholars have long noted that substantial segments of the U.S. voting population hold a peculiar aversion towards this label (Kluegel & Smith, 1986;Rasinski, 1989;Shaw & Shapiro, 2002;Smith, 1987). Furthermore, years of concrete empirical evidence of this phenomenon have been compiled, after the General Social Survey (GSS) beginning in the 1980s introduced a question-wording experiment that measures the effect of using the label 'welfare' rather than 'assistance to the poor' on support for assistance to the poor. As the results across years of GSS waves have shown, referring to welfare causes enormous increases-on the order of 30-50 percentage points-in the probability of a respondent asserting that 'too much' is being spent on such policies in the United States.
The experimental evidence in the GSS has led to an abundance of subsequent scholarship focused on explaining the underlying sources of this 'welfare-label effect' and variation in its magnitude across different subsets of voters. In one example, Green and Kern (2012) develop a method of applying Bayesian Additive Regression Trees (BART) to model and discover treatment effect heterogeneity and apply the method to the GSS welfare framing experiment data. In their study, they find that substantial amounts of heterogeneity in the welfare-label effect manifest as a function of a number of covariates, including party identification, with the welfare-label effect being more pronounced among Republicans than Democrats.
However, Green and Kern's finding that the welfare-label effect magnitude depends upon party identification does not, nor do Green and Kern claim it to, demonstrate that party identification itself is causing variation in the welfare effect. There are, of course, reasons to believe that party identification itself may moderate the welfare effect to some extent. A voter's party identification relates directly to how responsive that voter is to party cues, and party cues involve delivery of partisan ideological messages, position preferences and issue frames. Given the economically conservative preference for limited redistribution of the Republican Party, it is reasonable to expect that Republican Party cues to voters include calls for limiting social assistance, and that such calls might frame social assistance using the stigmatized label of welfare. For these reasons, we might expect that identification with the 78 | BANSAK Republican Party would, in fact, make an individual more sensitive to the welfare frame and hence exhibit a stronger welfare-label effect than had s/he not adopted that party identification. Furthermore, even if most voters do not change their party identification often or at all, party identification is a characteristic that is clearly mutable at the individual level.
In contrast, however, there are many non-political factors known to influence voters' party identification choices that may also make voters more sensitive to the welfare frame for various reasons not directly related to party identification itself. For instance, some researchers have explored the possibility of interactions between the welfare-label effect and such variables as education and racial perceptions (Federico, 2004;Henry et al., 2004). Such variables may also correlate with party identification such that any conditionality in the welfare-label effect across voters with different party identifications may be at least partially the result of these other variables.
To investigate the causal moderation of the welfare effect attributable to party identification, the parallel estimation framework is applied to the GSS data, and estimates of the ATME using parallel regression and parallel matching are compared to moderation effect estimates using conventional methods. As described above, the randomized treatment is whether the respondent was asked about 'welfare' (coded as 1) or 'assistance to the poor' (coded as 0), and the outcome is whether the respondent stated that 'too much' is being spent on such policies (coded as 1) or that 'too little'/'about the right amount' is being spent on such policies (coded as 0). The moderator is a dichotomous version of party identification, with those respondents identifying as Republicans coded as 1 and all others coded as 0. Covariates used as control variables include age, sex, race, highest education level, real income, number of children, region, city/town type (based on size and urbanicity), year surveyed and racial attitudes. While scholars of American voting behaviour have long considered racial attitudes to be possible determinants of party identification (Carmines & Stimson, 1989), some studies have questioned the precise relationship between racial attitudes and party identification (e.g. Abramowitz, 1994). Although the two are likely co-determined, for the purposes of the present investigation, it is illuminating to isolate the party identification moderation effect that is not related to racial attitudes. For this reason, racial attitudes are included as a control variable, based on the coding used by Green and Kern (2012). Figure 1 presents the results, with point estimates and 95%confidence intervals estimated using the conventional approaches and parallel estimation framework respectively. As illustrated in the upper half of the plot, the results from the conventional estimators suggest substantial moderation. The first estimator is the difference in subset effects, where the two subsets are Republican respondents and non-Republican respondents, and each subset effect is the simple difference between the probability of believing that too much is being spent on poverty assistance in the treatment condition (i.e. given the 'welfare' frame) and the probability of believing that too much is being spent on poverty assistance in the control condition (i.e. given the 'assistance to the poor' frame). The difference in subset effects highlights important heterogeneity of the welfare effect: while there is a welfare effect among all partisan groups, the magnitude of the estimated welfare effect is almost 10 percentage points higher among Republicans than among other respondents. Specifically, the estimated effect among non-Republicans is 0.329, while it grows over 25%to 0.418 for Republicans.
The second estimator whose results are presented in Figure 1 is the controlled interaction regression estimator, which is the estimate of the interaction effect between the treatment and moderator in a linear regression of the outcome on the treatment, moderator, treatment-moderator interaction, and control variables. While a number of the control variables are correlated with party identification (as should be expected), and while we should also expect a number of these control variables to bear upon voters' susceptibility to the welfare effect, the figure shows that the inclusion of controls in the interaction regression has virtually no effect on the moderation effect estimate, compared to the simple difference in subset effects.

BANSAK
As explained in this study, it would be incorrect to interpret the upper two estimates in Figure 1 as causal moderation effects because they are based on estimators that do not perform covariate adjustment in a way that allows for estimation of an identified causal moderation estimand. In contrast, ATME estimates using the parallel regression method and the parallel matching method are reported in the lower half of Figure 1. The matching procedure-implemented with the Matching package in R (Sekhon, 2011)-used Mahalanobis distance and, within each treatment-level subset, matched each Republican (non-Republican) unit with the closest non-Republican (Republican) unit with replacement. As can be seen, these results paint a much more conservative picture of the extent to which party identification per se moderates the welfare effect. Specifically, the estimates yielded by these methods are attenuated towards zero and cut roughly in half.
These results indicate that a substantial portion of the moderation effect attributed to party identification by the conventional estimators is actually the result of party identification being correlated with other variables that themselves moderate the welfare effect. Indeed, the results from the parallel regression method include statistically significant interactions between the treatment and several other variables-including education level, real income, city/town type and racial attitudes-that are also correlated with party identification.

| Asylum seeker conjoint experiment
European governments have recently struggled with the largest refugee crisis since World War II, with Europe receiving approximately 1.3 million new asylum claims in 2015. Exacerbating the political difficulties of accommodating asylum seekers while minimizing domestic public backlash is that, while most of these asylum seekers have originated from Muslim-majority countries, research has shown that European voters would prefer to grant asylum to non-Muslim applicants (Bansak et al., 2016). Furthermore, some right-wing parties have sought to mobilize citizens around asylum policy issues, appealing to nationalistic impulses and promoting anti-asylum policies in order to increase their vote shares (Dustmann et al., 2018;Dinas et al., 2019;Hangartner et al., 2019). This episode is important in and of itself, and it also represents an example of a longstanding theme explored by political scientists whereby nationalism is considered a malleable dimension of public opinion that may fluctuate over time, is endogenous to current events, and constitutes a potential target for manipulation via government intervention and elite political communication (e.g. Baker & Oneal, 2001;Mueller, 1970;Weiss, 2013). In this case, we may ask: to what extent might nationalism increase Islamophobia? More specifically, does increased nationalism strengthen voters' preferences for non-Muslim asylum seekers? To investigate, the parallel estimation framework is applied to data from a previous conjoint experiment embedded in an online survey fielded in 15 European countries with approximately 18,000 respondents (Bansak et al., 2016). In the conjoint experiment, respondents were presented with pairs of hypothetical asylum-seeker profiles comprised of various attributes with randomly assigned levels. The analysis here focuses on one attribute in particular, which serves as the randomly assigned treatment T in the parallel estimation process: whether the asylum seeker was Muslim (1) or Christian (0). For each pair of asylum seekers, the respondent was asked to specify which s/he preferred, providing a binary preference outcome variable Y. Each respondent's level of nationalism was also measured via a set of questions in the survey combined into an index. This measure is dichotomized into a binary measure of high-versus-low nationalism, with the median index value chosen as the cut point, for use as the moderator variable S. Thus, being investigated is whether nationalism moderates the effect of an asylum seeker's religion on voters' acceptance of the asylum seeker. Implicit in this investigation is the assumption that nationalism is indeed mutable at the individual level, in line with previous political science scholarship that treats nationalism as endogenous to social and political factors. Finally, a range of demographic and other characteristics are used as the set of control variables X. This includes gender, age, income, education and left-right political ideology. More details on the study design and variables can be found in Appendix C. Figures 2 and 3 present the results, pooled and across countries, with moderation effect point estimates and 95%confidence intervals estimated using the conventional approaches and parallel estimation framework respectively. The results from the conventional estimators, shown in Figure 2, highlight strong heterogeneity of voters' preferences regarding the religion of asylum seekers: the lower probability of accepting a Muslim asylum seeker relative to a Christian asylum seeker is more pronounced among high-nationalism respondents than low-nationalism respondents in all countries, a statistically significant finding at α = 0.05 in 10 of the 15 countries included in the survey. Substantively, the magnitude of that heterogeneity varies by country. For instance, in Austria, the difference in subset effects is about −0.1, meaning that the negative gap in the probability of accepting a Muslim asylum seeker relative to a Christian asylum seeker grows by 10 additional percentage points when moving from the low-nationalism segment of voters to the high-nationalism segment.
In contrast, however, the causal moderation results using the parallel estimation framework, shown in Figure 3, paint a more conservative picture of the extent to which increased nationalism can strengthen Islamophobic biases towards asylum seekers. Specifically, with only a few exceptions the causal moderation effect estimates yielded by these methods are attenuated towards zero compared to the conventional estimates. Of the 10 countries where the conventional estimators yielded statistically significant moderation effects, only one is found to have statistically significant causal moderation effects according to both the parallel regression and parallel matching estimators, and only four others are found to have statistically significant causal moderation effects by either estimator. Note that the matching procedure used Mahalanobis distance and matched all of the moderated (high nationalism) units with the closest unmoderated (low nationalism) units, with replacement. Hence the matching estimates should be interpreted as the ATMEs for the highly nationalistic.

| Sensitivity analysis
Identification of the ATME via selection on observables relies on the assumption that all variables that confound the relationship between the moderator and the outcome are conditioned upon. Of course, as with the estimation of ATEs using observational data, it is impossible to know and difficult to argue