soubhikbarari / svmkr Goto Github PK

The SurveyMonkey research team has a set of standard variable coding practices. It would be great to institutionalize those into our R package.

For example, a function that looks for the usual SurveyMonkey Age question in the columns of a dataframe and then codes it into the Pew generations. Something like this (but accounts for present year):

svy_df$Age <- cut(as.numeric(svy_df$`What is your age?`), 
                  breaks=c(-Inf, 26, 42, 59, Inf),
                  labels=c("18-26 (Gen-Z)","27-42 (Millennial)","43-58 (Gen X)","59+ (Boomer)"))
attr(svy_df$Income, "label") <- "Age (4) / Pew Generation" ## needed to show up in banners

A provisional list of such variables to code would be:

• Which gender best describes you? (gender_non_binary)
• age3 (age3)
• Age 6 (age6)
• Race 5 (racethn5)
• Pew Generations 2022 (generations)
• Education 2 (educ2)
• Income groups (income3)
• Party3 (party3)

See 2022 procedures doc for more info

Updated TODOs Jun 16 2023

Functionality:

• have an option to specify keep.margins and remove.margins to keep/remove margins in the passed-in target
• auto.map option if it makes sense, i.e. allow user to turn off auto-mapping dataframe columns to population variables
• have an option to save a custom target distribution, or list of dataframes that the user is created, e.g. save_target

Specific distributions to save out the target files in inst/extdata/targets -- drawing from all the files we've created already from various projects -- and then also to turn into functions (in order of priority):

• us_smallbiz_acs_23 (or something similar) for the small business weights that Zoe created
• us_aapi_acs_19 (or something similar) for the weights we use for AAPI Data
• tx_acs_19, ma_acs_19, etc. etc. for each state gen pop
• us_acs_teens_19 or something similar for US teens (these weights should be on the dashboard backend folders somewhere)

See svmkR_demo_0602.R.

See svmkR_demo_0602.R as a reference.

It would be nicer to have a more consistent interface for the survey downloading that mirrors the qdoc survey creation interface:

• download_qdoc: does what fetch_survey_obj() does but turns into a qdoc object
• download_responses: takes either ID or qdoc (with ID parameter) and returns dataframe as parse_survey() works
• backwards compatibility with fetch_survey_obj and parse_survey

It would be nice if read_qdoc provided some more precise and easily readable warnings / error messages when a question is malformed.

e.g. [[Template:12122:4]]

We need to have some QDOC tag that the user can use to specify an explicit "Other" and "None of the above" options. Relevant code in make_survey.R.

There's a way to specify this in the Developer API, just need to:

• have qdoc.question be able to parse a particular tag (maybe a :Other subtype) for Other and similarly for NOTA
• incorporate this into the vignette
• have it hook up to the developer API

• delete a question
• replace a question
• modify an attribute of a question
• add a question

There's already an option to import questions into a survey -- have an option that dumps a QDOC into a format compatible with this.

Can only import multiple choice and text box questions -- maybe should make this clear in the text outputted.

Need to specify:

• template ID
• where qdoc questions go relative to template questions

Request from Zewei.

For example, we could add stars or bolden numbers within a particular block if they are significant in a pairwise test.

Will need to think through the exact design, generalizability, etc.

Not this again ...

Duplicated questions issue arising for survey number 407853711.

If a particular column in a multi-answer matrix question is never selected, it drops out of the dataset returned by parse_survey() altogether.

Can either fix this in process_matrix() or in the fetch_survey_obj() stage (probably less work for current release to do the former).

upload_qdoc can take a theme_id parameter. It would be great if we could test out uploading the same survey using different theme IDs and seeing that this feature works.

It would also be great to know what the theme number is for our usual research insights surveys (check out one of our recent research team surveys and then try looking at the dev docs to see if theres a list of theme numbers or see if there's a theme number on the SurveyMonkey platform itself when creating a survey...).

If there is, it would be nice to have theme_id in upload_qdoc automatically set to that!

• test out whether random assignment parameters in API can get us this
• otherwise, will need some functionality to split questionnaire into separate surveys
• create new tags for [[Split]]s
  • have option to pull splits out into either (a) separate consecutive blocks of the same survey (b) separate surveys themselves

• Descriptive text question -- i.e. a presentation family question
• Slider type question

For example, for the SVMK marriage season survey, there is (or was) a comment linking to the ideas document that, when downloading the doc as a .txt file gets appended to the very end of the survey doc like so:

In read_qdoc (parse_survey.R), should check for any comments at the very end of the .txt file in this footnote-y format if the user is reading a Google Doc.

Server returns a 504 error either immediately or after a little while.

• Try removing format_date() for start_created_at in parse_survey()
• Try paging like we're paging in the API patch script I created on the SVMK server bypassing the API and just manually checking for response dates.

Take tests in create_survey_tests.R and format/copy over into /tests.