Bug: need to generalize assumptions in base R data split about spectratrait HOT 10 CLOSED

Oh, sorry, missed that you linked another data set

from spectratrait.

Pretty sure this is the area causing problems

create_data_split <- function(approach=NULL, split_seed=123456789, prop=0.8,
                              group_variables=NULL) {
  set.seed(split_seed)
  if(!is.null(approach)) {
    if (approach=="base") {
      plsr_data$CalVal <- NA
      split_var <- group_variables
      plsr_data$ID <- apply(plsr_data[, split_var], MARGIN = 1, FUN = function(x) paste(x, collapse = " "))

from spectratrait.

can you email me the data set you are using? I agree, this needs to work on any data set.

from spectratrait.

I provided the EcoSIS link to the dataset? Do you actually need me to email it to you?

from spectratrait.

@serbinsh Ok, I see the problem. It's the grep looking for NA's. Which will cause issues. I've solved this once upon a time and don't remember the fix right now. The real question is, how do we want to handle say species that has NA's in it? Drop them? Leave them?

from spectratrait.

@serbinsh What do we want to do with NA's in the grouping variables?

from spectratrait.

We dont want to drop NA's. I think we may just have to shunt them as another "factor" that is pool all NA's together. What do you think? The risk is dropping NAs could mean we drop a lot of good data for training

from spectratrait.

Good point. I'll keep them in. We should check Julien's method and see what it does.

from spectratrait.

I'll get you the fix after lunch. I think I need to update all the things before submitting the fix.

from spectratrait.

Fixed by @neo0351

from spectratrait.