Remove outliers from ffq_total_sum_meat and other FFQ variables.

Currently, we examine the relationship between concerns and interest in clean meat by comparing baseline concerns to treatment wave interest among the 308 individuals in the control group. We should replicate this analysis using all ~3k individuals but only using baseline measures of concerns and interest in clean meat.

Description

Re-format for submission to Journal of Consumer Research

Tasks

• 2 columns --> 1 column.
• change to double-spaced 12pt Times New Roman.
• put page number in upper-right hand corner.
• use 1 inch margins.
• remove all underlines.
• change "here" links to references.
• add full question to each figure caption.
• make sure each figure is self-sufficient
• change two decimal points to one decimal point.

Description

A natural question is what kinds of individuals were most vs. least willing to pay. Using the attitudinal outcomes, there were no obvious differences along various covariates, but perhaps the discrete choice analyses will yield different results.

Tasks

• (3hr) estimate discrete choice results broken down by core demographic groups (age, gender, education).
• (3hr) estimate discrete choice results for key attitudinal variables of interest (e.g. ideology, baseline meat consumption).
• (3hr) write up the new results.

Description

User story: As a reader, I want to be shown that the treatment groups are balanced on the average number of days between treatment and followup.

Because not all respondents participated in each wave at exactly the same time (such that the followup was 9 weeks after treatment for some respondents, 11 weeks after for other respondents, et cetera), we should confirm that the treatment groups are balanced on the time between treatment exposure and followup survey. Otherwise, this could bias the results.

Tasks

• create a variable in R representing the number of days between when a respondent took the treatment survey and the endline survey.
• regress this days between surveys variable on the treatment conditions in order to confirm that the groups are balanced.

Description

User story: As a reader, I want to know whether the pro-clean meat appeals were more effective among any particular demographic groups.

People in marketing departments will care most about the least theoretically interesting predictors of interest in clean meat: demographics. So we should examine whether appeal effects are larger in any particular subgroup, even though this won't make any kind of intellectual contribution.

Description

User story: As a PI, I want to be sure that my false discovery rate is restricted to 0.05 so that I do not make Type I errors.

We conduct a lot of hypothesis tests (in paper version 0b2154e), but do not correct our p-values in any way. We should use the Benjamini-Hochber method or another method for correcting these p-values.

Description

User story: As someone marketing clean meat products, I want to know how these findings should affect my practices so that I can maximize public acceptance of clean meat.

We should add a paragraph or two aimed at clean meat advocates describing how our findings can be applied to improve the effectiveness of clean meat advocates.

The key question is: what can we confidently recommend based on our findings, beyond the obvious that an embrace unnatural appeal may be effective, and advocates need to be wary of negative social information?

Self-explanatory.

Description

User story: As a psychologist, I want to know about what cognitive processes might lead each treatment appeal to be effective or ineffective.

In the paper (0b2154e), we make some remarks about how the embrace unnatural appeal may be particularly effective because it leverages individuals' "mental models" and "cultural intuitions". This appeal may also induce cognitive dissonance that is easiest to resolve by becoming more accepting of clean meat, whereas the debunk unnatural appeal induces cognitive dissonance that is easiest to resolve by coming up with some other argument why clean meat is bad.

However, these remarks are fairly shallow and ad hoc, and we do not do a very good job in the most recent version of the paper (0b2154e) as setting up (or citing) what existing theories would lead us to expect one pro-clean meat appeal to be more effective than another. Also, our discussion of results does not relate the findings to more general phenomena of interest to behavioral economists and psychologists, such as motivated reasoning, asymmetric updating from new information, and cognitive dissonance.

So we should put more effort into setting up clear expectations about why each pro-clean meat appeal might be effective or ineffective, drawing on recent work in social psychology, behavioral economics, and political science on belief updating, congitive dissonance, mental models, et cetera. This will increase the reach of our readership beyond those who are just interested in clean meat and closely related topics (e.g. GMOs, vaccines).

Description

Clearly describe main results.

Description

User story: As a reader, I want more qualitative insight into attitudes towards clean meat so I better understand how important different concerns are.

We should allocate more time to reading the open-ended responses and seeing how they inform our interpretation of the results in the paper. For one, this will necessitate spending more time reading through the open-ended responses to get an intuition for the various ways in which individuals responded to cultured meat and the treatment appeals.

Here are a few thoughts on more structured analyses of the open-ended responses we might want to do:

• construct a measure of a respondent's main concern about clean meat, as extracted from open-ended responses rather than the closed multichoice response. This would be a good sensitivity check on the results reported in the most recent version of the paper (0b2154e). We could try doing this by: (a) using a topic model or PCA; (b) hand-labeling all ~1500 open-ended responses; or (c) hand-labeling ~200 hundred responses and then training a classifier to predict the rest.
• Examine whether the "embrace unnatural" treatment led to more positive sentiment towards the naturalness of clean meat, even though it increased concerns about its naturalness. Could examine this by comparing positive/negative sentiment across treatment groups, or even by examining whether the negative relationship between interest in clean meat and concerns about naturalness became weaker in this group. See #18.

Tasks

...?

Description

User story: As a reader, I want to be convinced of the far-reaching benefits of clean meat so that I see the importance in your research.

To sell this paper to people interested in environmental degradation and health, but who have not paid much attention to clean meat, we should put more effort into the opening 1-2 paragraphs at describing the anticipated benefits of clean meat (with citations).

For some context/stats on the costs of factory farming, see: https://openletteranimalfarming.com/welcome/.

Description

Why is existing work inadequate to answer the questions we pose in this study?

Acceptance criteria

• clear articulation of how this study fits into existing literature, and why existing work does not adequately answer the questions at hand.

Tasks

• (3hr) describe existing research on the importance of food naturalness to consumers, including research on attitudes towards GMOs.
• (3hr) in brief, mention existing research on the psychology of meat-eating that would lead us to expect public skepticism about clean meat.
• (2hr) in brief, describe recent observational research on clean meat.
• (3hr) describe why existing research does not answer the questions we seek to answer in this study.

Description

Need to improve the readability of figures.

Tasks

• ...

Description

User story: As a reader, I want to know whether "conflicted" omnivores were more/less affected by the treatments.

We might expect people who are "conflicted omnivores" to be more susceptible to the treatments, since they are looking for a reason to becoming more accepting of clean meat products, which they can use to continue to avoid going vegetarian in the short term while promising themselves they'll eat clean meat in the future. On the other hand, we might expect conflicted omnivores to be more motivated to reject treatment appeals that are more threatening to their current patterns of meat consumption.

So we should examine whether effects were largest for "conflicted" omnivores, who we would define as individuals who (a) eat a reasonably large amount of meat; (b) are sympathetic about the suffering and intelligence of animals; (c) are willing to accept that eating conventional meat products are at least moderately harmful; and (d) have an interest in reducing their meat consumption.

Tasks

• construct a measure of whether a respondent is a "conflicted omnivore" (e.g. by combining baseline measures of concern about factory farming, self-reported meat consumption, ...)
• examine the effects of the treatment appeals and negative social information conditional on whether respondents were conflicted omnivores at baseline.

Description

User story: As a reader, I want to know the exact survey timeline.

This is not very important, but it would be good to provide a more accurate description of the survey timeline of our study, especially since not all respondents took each survey wave at exactly the same time.

From looking at the survey responses, it looks like the following are the correct (or at least close enough) start/end dates of each survey wave:

Baseline: 2017-03-07 -- 2017-03-18
Treatment: 2017-03-07 -- 2017-04-18
Followup: 2017-06-19 -- 2017-06-26

Tasks

• add a couple of sentences describing these start/end dates (or add a small table in the appendix).

Description

User story: As a reader, I want to know whether relationship between interest and naturalness is driven by becoming concerned or becoming unconcerned.

Readers will want to know whether the observed relationship between change in concern and change in interest in clean meat (as reported in this version of the paper: 0b2154e) is driven by people becoming concerned or people becoming unconcerned.

Tasks

• examine relationship between change in interest in clean meat and change in concerns about naturalness, conditional on whether respondents were concerned or unconcerned at baseline.

Description

User story: As a reader, I want to see the results replicated with outliers removed so that I can have more confidence in the reported results.

Could any of our results be driven by outliers? Probably not, since none of our outcomes allow users to report extreme values. Instead, most of our outcomes are dichotomous or on 5-point and 7-point scales. Nevertheless, it may provide readers with some comfort to know that removing outliers on these likert scales does not alter the results.

Tasks

• write an R function to remove the top/bottom x% of outliers from a variable.
• replicate the main results after removing the top/bottom 2.5% of outliers, 5% of outliers, 10% of outliers.

Description

...

Description

Clearly describe how we estimate willingness-to-pay and treatment effects.

Tasks

• (1hr) (actual: 1hr) describe WTP estimation
• (1hr) (actual: 1hr) describe difference in means estimation.

Description

Re-run discrete choice analyses using clogt, and then plot average marginal effects.

Tasks

• (1hr) (actual: 2hr) re-estimate results using clogt.
• (2hr) (actual: 2hr) compute marginal effects (and SE) and output to a nice publication plot.

Description

User story: As a reader, I want to be shown treatment effects on willingness-to-pay broken down by subgroups.

In the version of the paper pushed in 0b2154e, we examine subgroup effects on interest in clean meat. We should replicate these results using willingness-to-pay as the outcome variable.

Description

Need to describe discrete choice experiment results clearly to allow for interpretation.

Tasks

• WTP:
  • (1hr) describe interpretation of point estimates in the discrete choice WTP figure.
  • (1hr) describe how WTP treatment effects compare to price changes.
  • (0.5hr) mention how clean meatballs are more appealing than vegetarian meatballs.
  • (1hr) describe how WTP treatment effects compare to price changes.
• Concerns and attitudes:
  • (1hr) describe interpretation of point estimates in the attitudes and concerns effects figure.
  • (1hr) describe effects of each treatment appeal.
  • (1hr) describe effects of anti-adoption social information.

Description

Include a balance table in the appendix.

Tasks

• (1hr) produce balance regressions.
• (1hr) summarize balance in a facetted plot.

Description

User story: As a reader, I want to know whether the treatment appeals affected ancillary outcomes.

Readers may want to know how the treatment appeals and negative social information affected ancillary outcomes (e.g. perceptions of vegetarians, perceived ease of reducing meat consumption, attitudes towards factory farming), as these results may offer insight into how/whether related attitudes shift collectively.

Tasks

• decide on a shortlist of plausible theories/reasons why we might expect certain outcomes to be affected by the treatments (e.g. is there reason to expect that the embracing unnatural appeal may have led individuals to think it is easier to reduce their meat consumption?)
• replicate main results using the relevant set of variables needed to test these theories.

Description

Replace hard-coded numbers in the paper with imported data from R, so that the paper can be rendered with updated data without the need for any hard-coding.

Tasks

• (1hr) write general function in R that takes in a list and saves data as a json.
• (1hr) find all numbers reported in the paper and add them to the list in R and then export to json.

Description

User story: As a reader, I want to see what kinds of individuals are most susceptible to the naturalistic heuristic.

What kinds of individuals are susceptible to the naturalistic heuristic? This heuristic can be operationalized in one of two ways. First, people who employ the naturalistic heuristic should be more likely to be concerned about the naturalness of clean meat. Second, among people who employ this heuristic, we should expect there to be a strong negative relationship between interest in clean meat and concerns about its naturalness.

The second approach seems to better capture the naturalistic heuristic. While there is already old code in analysis/main_analysis.r that examines the predictors of who is concerned about the naturalness of clean meat (resulting in few interesting insights), I have not examined susceptibility to the naturalistic heuristic using this second approach.

Tasks

• regress change in interest in (or WTP for) clean meat on change in whether concerned about its unnaturalness, interacted with categorical variables such as education, age, et cetera. This will shed light on whether the naturalistic heuristic is more prevalent among certain groups of individuals.

On average, respondents in our study are much less willing to pay for clean meat than conventional meat products. But are there any subsets of consumers that would be willing to pay more for clean meat than conventional meat products? We can examine this question through a set of subgroup analyses where we estimate willingness-ot-pay conditional on respondents' baseline responses.

Description

User story: As a reader, I want to know whether the embrace unnatural appeal severed the negative relationship between interest and naturalness concerns.

The embrace unnatural appeal led to sustained increases in consumer interest in clean meat, but it may have also increased consumers' tendency to be concerned about the naturalness of the products. So what is going on here? Did the appeal sever the negative relationship between perceived naturalness and interest? Or is something else going on. This is something we should investigate.

Tasks

• for respondents in each treatment appeal, run a regression of change in interest in clean meat on change in concerns about unnaturalness. Then examine whether this relationship is significantly weaker in the embrace unnatural group than in other groups.
• ...?