Why do all functions have Age = AgeStart = AgeEnd = AgeInt <- NULL # hack CRAN note? Can this be removed?

Combining the *Name and *ID columns with get_recorddata seems to be an idea that Patrick likes. However, dplyr::bind_rows and rbind seem to have some trouble merging haven labels. I've reported it here: https://github.com/tidyverse/dplyr/issues/5492

When this is fixed, make sure you restore the haven labels inside get_recorddata. In the mean time, the code will be commented out, as well as the vector with the *Name and *ID pairs in Utils_Get.R.

Some heaping methods are now called check_heaping_*, when the output_msg function refers to the old name.

The API will eventually do this:

The recordset that come out from the structureddatarecordsadditional queries, I see that they include the extra SortOrder fields I asked, but the names return include only “Sort” as suffix. Can you label them with SortOrder as suffix. I also noticed that you are returning SubGroupTypeSortOrder, can you also return SubGroupSortOrder.

Since get_recorddata does column subsetting, you'll need to update this subset of columns to include these new columns and rename the Sort columns to SortOrder.

After a long discussion Dennis now allows get_recorddata to accept these two sets of arguments:

• indicatorIds or indicatorTypeIds
• dataProcessIds or dataProcessTypeIds

We need to add some tests checking that, for example, if I set indicatorTypeIds=8, the result should contain many different indicatorIds such as 58, 60, 311. Conversely, if I specify indicatorId=58, the resulting indicatorIds should only be 58. Although this comes directly from the API, a test on this could flag the UNPD whether something is off.

Similarly, we need to apply this for dataProcessTypeIds and dataProcessIds, dataProcessTypeIds being the umbrella that contains several values of dataProcessIds.

Just so I don't forget, here's an example that Patrick mentioned that will allow me to implement this for indicatorTypeIds:

For a country like AFG (locID=4) indeed the recordsets are limited so you are going to see similar recordsets based on these two sets of queries. Try with India (locIds=356). But if isComplete=0 is also used in these queries, iot helps also to narrow the list of indicator.

I sent him these two problems:

'# With indicatorTypeIds = 8, this result should contain results for all indicatorIds 58, 60, 311, etc.. when in fact we only get 58
http://24.239.36.16:9654/un3/api/structureddatarecordsadditional?dataProcessIds=2&startYear=1920&endYear=2020&indicatorTypeIds=8&isComplete=0&locIds=4&locAreaTypeIds=2&subGroupIds=2

'# With indicatorIds = 58 we get the same result as above
http://24.239.36.16:9654/un3/api/structureddatarecordsadditional?dataProcessIds=2&startYear=1920&endYear=2020&indicatorIds=58&isComplete=0&locIds=4&locAreaTypeIds=2&subGroupIds=2

and he responded with:

A more meaningful test of these differences can be seen if you use IndicatorTypeID=29, you’ll get a collection of life table indicators. If isComplete=0 is used you’ll get only the indicators only for abridged life tables. But if we use directly IndicatorID=245 we should get only “m(x,n) – abridged”

There's no current example for this on dataProcessTypeIds and we should get one from them

Can this be fixed or calculated internally? This is to make sure everything is harmonized across functions.

The API is currently unstable so we'll comment out this part for now to be able to do the refactoring reliably by running tests and passing.