Remember to document all arguments in load_dataset

Prepare for release:

• git pull
• Check current CRAN check results
• Polish NEWS
• devtools::build_readme()
• urlchecker::url_check()
• devtools::check(remote = TRUE, manual = TRUE)
• devtools::check_win_devel()
• rhub::check_for_cran()
• revdepcheck::revdep_check(num_workers = 4)
• Update cran-comments.md
• git push

Submit to CRAN:

• usethis::use_version('patch')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• git push
• usethis::use_github_release()
• usethis::use_dev_version()
• git push

some authors have requested that citation information be displayed in the download prompt along with in the documentation.

It looks like the structural change in the source file that led to issue #50 has been rolled back, so lexicon_nrc() fails due to a missing file.

I think the path used in process_nrc() should once again be:
"NRC-Emotion-Lexicon/NRC-Emotion-Lexicon-v0.92/NRC-Emotion-Lexicon-Wordlevel-v0.92.txt"

Prepare for release:

• Check current CRAN check results
• devtools::check(remote = TRUE, manual = TRUE)
• devtools::check_win_devel()
• rhub::check_for_cran()
• revdepcheck::revdep_check(num_workers = 4)
• Update cran-comments.md
• Polish NEWS
• Review pkgdown reference index for, e.g., missing topics
• Draft blog post

Submit to CRAN:

• usethis::use_version('minor')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• usethis::use_github_release()
• usethis::use_dev_version()
• Finish blog post
• Tweet
• Add link to blog post in pkgdown news menu

Prepare for release:

• git pull
• Check current CRAN check results
• Polish NEWS
• urlchecker::url_check()
• devtools::build_readme()
• devtools::check(remote = TRUE, manual = TRUE)
• devtools::check_win_devel()
• revdepcheck::revdep_check(num_workers = 4)
• Update cran-comments.md
• git push

Submit to CRAN:

• usethis::use_version('patch')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• usethis::use_github_release()
• usethis::use_dev_version(push = TRUE)

The text file unzipped for the NRC Valence, Arousal, and Dominance lexicon does not include column names, but the read_tsv() assumes it does. As a result, the function returns a tibble with columns named "aaaaaaah", "0.479", "0.606", and "0.291".

A PR has been prepared to address this issue.

Pre-history

• usethis::use_readme_rmd()
• usethis::use_roxygen_md()
• usethis::use_github_links()
• usethis::use_pkgdown_github_pages()
• usethis::use_tidy_github_labels()
• usethis::use_tidy_style()
• usethis::use_tidy_description()
• urlchecker::url_check()

2020

• usethis::use_package_doc()
  Consider letting usethis manage your @importFrom directives here.
  usethis::use_import_from() is handy for this.
• usethis::use_testthat(3) and upgrade to 3e, testthat 3e vignette
• Align the names of R/ files and test/ files for workflow happiness.
  usethis::rename_files() can be helpful.

2021

• usethis::use_tidy_dependencies()
• usethis::use_tidy_github_actions() and update artisanal actions to use setup-r-dependencies
• Remove check environments section from cran-comments.md
• Bump required R version in DESCRIPTION to 3.5
• Use lifecycle instead of artisanal deprecation messages, as described in Communicate lifecycle changes in your functions

2022

• usethis::use_tidy_coc()
• Handle and close any still-open master --> main issues
• Update README badges, instructions in r-lib/usethis#1594
• Update errors to rlang 1.0.0. Helpful guides:
  https://rlang.r-lib.org/reference/topic-error-call.html
  https://rlang.r-lib.org/reference/topic-error-chaining.html
  https://rlang.r-lib.org/reference/topic-condition-formatting.html
• Update pkgdown site using instructions at https://tidytemplate.tidyverse.org
• Ensure pkgdown development is mode: auto in pkgdown config
• Re-publish released site; see How to update a released site

Currently load_dataset() calls printer(), which in turn calls menu(), which throws an error if R is run non-interactively.

See

• https://github.com/EmilHvitfeldt/textdata/blob/master/R/load_dataset.R#L28
• https://github.com/EmilHvitfeldt/textdata/blob/master/R/printer.R#L16

An example of wanting to use this function non-interactively is including the datasets in a Docker image.

I think that in a non-interactive session you can always assume that the user wants to download the dataset, so a possible reworking of printer() might be something like the following.

printer <- function(name) {
  info_name <- print_info[[name]]
  if(interactive()) {
    cat("Do you want to download:\n",
      "Name:", info_name[["name"]], "\n",
      "URL:", info_name[["url"]], "\n",
      "License:", info_name[["license"]], "\n",
      "Size:", info_name[["size"]], "\n",
      "Download mechanism:", info_name[["download_mech"]], "\n"
    )
    menu(choices = c("Yes", "No"))
  } else {
    cat("Downloading:\n",
      "Name:", info_name[["name"]], "\n",
      "URL:", info_name[["url"]], "\n",
      "License:", info_name[["license"]], "\n",
      "Size:", info_name[["size"]], "\n",
      "Download mechanism:", info_name[["download_mech"]], "\n"
    )
    1
  }
}

That is, the message is changed from "Do you want to download" to "Downloading" and menu() is replaced by always returning 1.

Hi! Thanks for your hard work on this repo :) I've been using it for an analysis lately and I run into a problem.

My goal was to deploy a Shiny App that uses the NRC dataset. The app would work locally but it wouldn't when I deployed it. This is because every time I tried to run get_sentiments("nrc"), a menu is used asking whether I wanted to download the dataset. This is shown in the demo:
https://github.com/EmilHvitfeldt/textdata/blob/master/man/figures/textdata_demo.gif

I would suggest to add a parameter so the answer to that menu is automatically "Yes", and it downloads whatever it needs without asking.

I managed to finish my app by using the dataset locally so no rush for me, but I think I would be a nice addition. Tell me what you think :)

https://github.com/yumeng5/Spherical-Text-Embedding

https://github.com/r-lib/archive

Doublecheck that vignette specify that new data should be mentioned on the readme.

When install textdata package via remotes::install_github last night I got an error message below when trying to load the library. Please could you kindly advice? Thank you so much.

Error: package or namespace load failed for ‘textdata’:
package ‘fs’ does not have a namespace

Prepare for release:

• devtools::build_readme()
• Check current CRAN check results
• devtools::check(remote = TRUE, manual = TRUE)
• devtools::check_win_devel()
• rhub::check_for_cran()
• revdepcheck::revdep_check(num_workers = 4)
• Update cran-comments.md
• Polish NEWS
• Review pkgdown reference index for, e.g., missing topics

Submit to CRAN:

• usethis::use_version('patch')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• usethis::use_github_release()
• usethis::use_dev_version()

Hello and thanks for this great package!

Unfortunately my corporate firewall prevents me from downloading the data from within R (I can download the file manually with the browser).

What should I do to make it work?

Thanks!

I'd like to add the GloVe pre-trained word vectors, for use in tidymodels/textrecipes#20

The datasets are available here: https://nlp.stanford.edu/projects/glove/

There are 4 downloads, that break down like this:

• glove.6B.zip = 4 datasets
• glove.42B.300d.zip = 1 dataset
• glove.840B.300d.zip = 1 dataset
• glove.twitter.27B.zip = 4 datasets

The first one is all I'm directly in need of right now, but it feels worthwhile to work out a standard for all of them while I'm at it.

I don't want to make the functions too complicated to understand, but it feels like maybe it should be one set of textdata functions (download_glove, process_glove, dataset_glove), with arguments about the specifics (something like dataset_glove({normal stuff plus}, token_set, dimensions)).

Let me know what you think and I can knock this out (I'm doing it anyway for personal/work use, so formalizing it won't be a lot of extra work).

• nma_words
• parts_of_speech
• sentiments
  • nrc
  • bing
  • loughran
  • AFINN
• stop_words
  • onix
  • SMART
  • snowball

As i know that NRC Emolex is available in 40+ languages including Indonesian language. is indonesian laguage of NRC also available on this package?

I get a 404 error when I try to download the AFINN sentiment lexicon. It looks like https is being forced somewhere in the function, but I can't figure out how to fix it myself. I've tried editing the textdata::catalogue data.frame but that did not work. The only way I was able to fix it was to manually download the zip file.

Prepare for release:

• devtools::check()
• devtools::check_win_devel()
• rhub::check_for_cran()
• revdepcheck::revdep_check(num_workers = 4)
• Polish NEWS
• Polish pkgdown reference index
• Draft blog post

Submit to CRAN:

• usethis::use_version('minor')
• Update cran-comments.md
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• usethis::use_github_release()
• usethis::use_dev_version()
• Finish blog post
• Tweet
• Add link to blog post in pkgdown news menu

For some reason, the afinn dataset seems to have improperly named columns on my local mac installation.

The columns are "word" and "value", instead of "word" and "sentiment" like the documentation would suggest (and a previous version of the tidytext package reflects a third specification, "word" and "score").

For reference:

> afinn=lexicon_afinn()
> names(afinn)
[1] "word"    "value"
> sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.5

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] textdata_0.3.0

loaded via a namespace (and not attached):
 [1] readr_1.3.1     compiler_3.5.2  R6_2.4.0        hms_0.4.2       tools_3.5.2     pillar_1.3.1    fs_1.2.7        rstudioapi_0.10
 [9] rappdirs_0.3.1  tibble_2.1.1    yaml_2.2.0      crayon_1.3.4    Rcpp_1.0.1      pkgconfig_2.0.2 rlang_0.3.4    

Prepare for release:

• Check that description is informative
• Check licensing of included files
• usethis::use_cran_comments()
• devtools::check()
• devtools::check_win_devel()
• rhub::check_for_cran()
• Polish pkgdown reference index
• Draft blog post

Submit to CRAN:

• usethis::use_version('minor')
• Update cran-comments.md
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• usethis::use_github_release()
• usethis::use_dev_version()
• usethis::use_news()
• Update install instructions in README
• Finish blog post
• Tweet
• Add link to blog post in pkgdown news menu

Prepare for release:

• git pull
• Check current CRAN check results
• Polish NEWS
• devtools::build_readme()
• urlchecker::url_check()
• devtools::check(remote = TRUE, manual = TRUE)
• devtools::check_win_devel()
• rhub::check_for_cran()
• revdepcheck::revdep_check(num_workers = 4)
• Update cran-comments.md
• git push

Submit to CRAN:

• usethis::use_version('patch')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• git push
• usethis::use_github_release()
• usethis::use_dev_version()
• git push

Possible also a delete_all() function.

lexicon_nrc() fails to run due to missing file:

It appears that there has been a change to the structure of the file downloaded from: http://saifmohammad.com/WebDocs/Lexicons/NRC-Emotion-Lexicon.zip
(according to http://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm this file was updated in August 2022)

Seems like the path that is currently specified in process_nrc():
"NRC-Emotion-Lexicon/NRC-Emotion-Lexicon-v0.92/NRC-Emotion-Lexicon-Wordlevel-v0.92.txt
should actually be:
"NRC-Emotion-Lexicon/NRC-Emotion-Lexicon-Wordlevel-v0.92.txt

ref: juliasilge/tidytext#169

Prepare for release:

• Check current CRAN check results
• devtools::check()
• devtools::check_win_devel()
• rhub::check_for_cran()
• revdepcheck::revdep_check(num_workers = 4)
• Update cran-comments.md
• Polish NEWS
• Review pkgdown reference index for, e.g., missing topics
• Draft blog post

Submit to CRAN:

• usethis::use_version('minor')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• usethis::use_github_release()
• usethis::use_dev_version()
• Finish blog post
• Tweet
• Add link to blog post in pkgdown news menu

http://mpqa.cs.pitt.edu/lexicons/

ref EmilHvitfeldt/R-text-data#8

Prepare for release:

• git pull
• Check current CRAN check results
• Polish NEWS
• devtools::build_readme()
• urlchecker::url_check()
• devtools::check(remote = TRUE, manual = TRUE)
• devtools::check_win_devel()
• rhub::check_for_cran()
• revdepcheck::revdep_check(num_workers = 4)
• Update cran-comments.md
• git push

Submit to CRAN:

• usethis::use_version('patch')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• git push
• usethis::use_github_release()
• usethis::use_dev_version()
• git push

Implement an argument that delete intermediate files.

Default to FALSE to adhere to current implementation

NRC Emotion Intensity Lexicon (aka Affect Intensity Lexicon) http://saifmohammad.com/WebPages/AffectIntensity.htm

NRC Valence, Arousal, Dominance Lexicon
http://saifmohammad.com/WebPages/nrc-vad.html

For lexicon_nrc_eil:

http://saifmohammad.com/WebDocs/NRC-AffectIntensity-Lexicon.txt​
@inproceedings{LREC18-AIL,
author = {Mohammad, Saif M.},
title = {Word Affect Intensities},
booktitle = {Proceedings of the 11th Edition of the Language Resources and Evaluation Conference (LREC-2018)},
year = {2018},
address={Miyazaki, Japan}
}

For lexicon_nrc_vad:

http://saifmohammad.com/WebDocs/VAD/NRC-VAD-Lexicon-Aug2018Release.zip
@inproceedings{vad-acl2018,
title={Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words},
author={Mohammad, Saif M.},
booktitle={Proceedings of The Annual Conference of the Association for Computational Linguistics (ACL)},
year={2018},
address={Melbourne, Australia}
}

Make sure code will work on Windows OS too.

The original data file doesn't seem to contain headers.

textdata::lexicon_nrc_vad()
# A tibble: 19,970 × 4
   aaaaaaah    `0.479` `0.606` `0.291`
   <chr>         <dbl>   <dbl>   <dbl>
 1 aaaah         0.52    0.636   0.282
 2 aardvark      0.427   0.49    0.437
 3 aback         0.385   0.407   0.288
 4 abacus        0.51    0.276   0.485