From CRAN maintainers:

Please see the problems shown on
https://cran.r-project.org/web/checks/check_results_pkglite.html.

In particular, please see the "Found the following HTML validation problems" NOTEs in the "HTML version of manual" check for the r-devel debian checks results.

R 4.2.0 switched to use HTML5 for documentation pages. Now validation using HTML Tidy finds problems in the HTML generated from your Rd files of the form

• <big> element removed from HTML5
• <center> element removed from HTML5
• <img> attribute "align" not allowed for HTML5
• <img> attribute "hspace" not allowed for HTML5
• <img> attribute "width" has invalid value "120px"
• <img> attribute "width" has invalid value "480px"
• <img> attribute "width" has invalid value "50px"
• <img> attribute "width" has invalid value "72px"

For the first four, please see https://html.spec.whatwg.org/#obsolete-but-conforming-features for info on these: in principle, all can be fixed by using style attributes, e.g.

style='text-align: right;'

instead of align='right' etc., which will work for both the new and old ways of converting Rd to HTML.

For the second four, simply drop the px units: the HTML5 standard asks for a non-negative integer implied to be in CSS pixels.

Note that the problems are found in Rd files auto-generated with roxygen2: to fix it might suffice to re-generate these using the current CRAN version of roxygen2.

Use stop(call. = FALSE) is recommended by https://style.tidyverse.org/error-messages.html

We should update the customized template/mechanism to accommodate pkgdown 2.0.0 which uses Bootstrap 5.

Add a function to combine file collection objects from multiple collate() calls.

pkglite follows a simple rule when unpacking packages:

• It will create directory(s) under the output directory named after the parsed package name(s) from each DESCRIPTION file of the package(s), and put the content of each package under them.
• This design implies that the simplest way to avoid unnecessary confusions is to keep package names and folder names an identical mapping.
• Users can adjust the directory names and hierarchy in userland easily, but that would belong to individual business logic.

See https://win-vector.com/2018/02/19/r-tip-use-seq_len-to-avoid-the-backwards-sequence-bug/

Document the approaches to merge, prune, and sanitize file collections.

Prepare for release

• Check current CRAN check results
• Check licensing of included files
• Review pkgdown reference index for, e.g., missing topics
• Bump version
• Update cran-comments.md (optional)
• Update NEWS.md
• Review pkgdown website
• urlchecker::url_check()
• Check with local machine
• Check with GitHub Actions
• Check with win-builder

Submit to CRAN

• Draft GitHub release
• Submit to CRAN via web form
• Approve emails

Wait for CRAN

• Accepted 🎉
• Post on r-packages mailing list
• Tweet

Prepare for release

• Check current CRAN check results
• Check licensing of included files
• Review pkgdown reference index for, e.g., missing topics
• Bump version
• Update cran-comments.md (optional)
• Update NEWS.md
• Review pkgdown website
• urlchecker::url_check()
• Check with local machine
• Check with GitHub Actions
• Check with win-builder

Submit to CRAN

• Draft GitHub release
• Submit to CRAN via web form
• Approve emails

Wait for CRAN

• Accepted 🎉

Would you think this line of code can be used to list files? Just want to avoid folders

setdiff(list.files(), list.dirs(recursive = FALSE, full.names = FALSE))

https://stackoverflow.com/questions/22069095/r-get-list-of-files-but-not-of-directories

Add file specification template file_tests() to allow the collation of files under tests/.

Might be related to r-lib/pkgdown #1843

Test sketch in https://github.com/Merck/pkglite/blob/master/tests/testthat/test-independent_test_collection.R

user may need to save .stan file in the inst/ folder. Shall we add this extension in the folder structure?

It seems pkgdown website allow float TOC. But our template did not enable it.

https://pkgdown.r-lib.org/articles/pkgdown.html

Add function to remove files from file collection objects. Instead of asking users to modify .$df directly.

It would be helpful to include checks before sending code into a seq(...) function.
In my case I had an empty file in a folder and got an error while running the pack function:

Error in seq.default(from = 1L, to = nchars, by = nmax) :
wrong sign in 'by' argument

I traced it all the way to

seg_char <- function(x, nmax) {
    nchars <- length(x)
    pos <- seq(from = 1L, to = nchars, by = nmax)
    short <- nchars <= nmax
    nlines <- if (short) 1L else length(pos)
    pos_start <- if (short) 1L else pos[seq_len(nlines)]
    pos_end <- if (short) nchars else c(pos[2L:nlines] - 1L, nchars)
    lapply(seq_len(nlines), function(i) x[pos_start[i]:pos_end[i]])
  }

Since the binary file had 0 bytes and no content the seq(from = 1L, to = 0, by = 64) gave the error. I propose to built in a safety net to check if files have content (eg remove empty files in a sanitize function, eg if nchars>1 then do ... else give informative error to the user to investigate)

Please use tests/testthat/test-independent-test_utils_cli.R to add the tests for the 10 new functions in R/utils-cli.R.

Should we add a file_all function?

It would be helpful if we need to move all files from a sub-folder. (e.g. inst/ folder)

The logo is huge in pkgdown https://merck.github.io/pkglite/index.html

https://github.com/Merck/pkglite/settings/branches

It feels like it would generate more benefits than harms if we make read_pkglite() an exported function, because people might need to parse pkglite.txt directly sometimes. This would avoid the use of :::.

What do you think? @elong0527

When evaluating file specifications to create file collections, we should follow this:

• If a file has a known extensions, mark it as text or binary based on the dictionary (implemented)
• Include files that do not have a file extension, and files with extensions not covered by the dictionary
  • Guess if the file is (canonically) text, otherwise mark them as binary
    • I'd prefer zlib's algorithm: https://github.com/madler/zlib/blob/master/doc/txtvsbin.txt
    • If a file does not have any content, then mark it as binary
• Document this flow in the specification section

The page navigator is now at the bottom of the page. E.g.

https://merck.github.io/pkglite/articles/pkglite.html

https://cran.r-project.org/web/views/ReproducibleResearch.html

We should test whether the output path is writable, and give friendlier error messages if not before writing to it.

When packing only one file, I get an error that I traced back to the following piece of code:

idx_content_end <- c(idx_pkg[2L:nblocks] - 2L, nlines)

This line of code identifies the end of each block. If you have three files packed in the .txt file it will look for:

• the start of the second block of content = end of first block
• the start of the third block of content = end of second block
• the end of the .txt file = end of third block

When you only pack one file, this logic does not work because nblocks is 1.

This can be solved by eg

if (nblocks == 1) {
idx_content_end <- c(nlines)
} else if (nblocks>1) {
idx_content_end <- c(idx_pkg[2L:nblocks] - 2L, nlines)
}

It would be good to include a unit test for this.

I don't really understand why I would use this. Can you include some info in the README or vignettes discussing why it is useful? Also in the README which vignette should be read first.

Prepare for release

• Check current CRAN check results
• Check licensing of included files
• Review pkgdown reference index for, e.g., missing topics
• Bump version
• Update cran-comments.md (optional)
• Update NEWS.md
• Review pkgdown website
• urlchecker::url_check()
• Check with local machine
• Check with GitHub Actions
• Check with win-builder

Submit to CRAN

• Draft GitHub release
• Submit to CRAN via web form
• Approve emails

Wait for CRAN

• Accepted 🎉

When checking on R-devel since a week ago, we see:

checking LazyData ... NOTE
  'LazyData' is specified without a 'data' directory

This is from the R CMD check updates: wch/r-source@3280ff0.

I would like to add SAS binary data extensions to ext_binary() in the dictionary, as some packages might contain such datasets.

The current binary file extensions I consider include

• .sas7bdat
• .sas7bcat
• .xpt
• xpt5
• xpt8

Any other recommendations? @elong0527

Note that some units tests might need an update for this change.

It would be great to support defining the packing scope in a configuration file.

One of the potential benefits is automating the packing action in a CI/CD workflow, without modifying the pipeline code.

For the file format, my personal preference is a YAML file like in pkgdown, but see lintr for the syntax (not using the DCF format).

I was trying to remove some folders before packing but ran into issues. Then I realized you make the assumption that the files are already residing in a subfolder.

sanitize.file_collection <- function(x) {
  pkg_name <- fc$pkg_name
  df <- fc$df
  df <- df[!grepl(pkglite:::cat_patterns(pattern_file_sanitize()), df$"path_abs"), ]
  new_file_collection(pkg_name, df)
}

pattern_file_sanitize()
[1] "/\\.DS_Store$"     "/Thumbs\\.db$"     "/\\.git$"          "/\\.svn$"          "/\\.hg$"           "/\\.Rproj\\.user$" "/\\.Rhistory$"    
[8] "/\\.RData$"        "/\\.Ruserdata$"   


grepl("/\\.git", ".git/blah") # trying to remove the .git folder but this does not work
FALSE
grepl("/\\.git", "/.git/blah") # it failed above because . git was in the absolute folder
TRUE

It would be more intuitive (at least to me) to not make assumptions and change the pattern to below so the function finds all files:

[1] "\\.DS_Store$"     "Thumbs\\.db$"     "\\.git$"          "\\.svn$"          "\\.hg$"           "\\.Rproj\\.user$" "\\.Rhistory$"    
[8] "\\.RData$"        "\\.Ruserdata$"   

r-lib/cli is now using C to format certain things and thus becomes a dependency that requires compilation.

To minimize the number of dependencies, we should consider replacing it with wrappers created around r-lib/crayon. This would also avoid one other dependency of cli: tidyverse/glue.

.brew files are text files used as brew templates and should be added.

Test sketch in https://github.com/Merck/pkglite/blob/master/tests/testthat/test-independent_test_collection.R