CRS: Composite Reference Standard in R
The functions are written to perform sensitivity and specificity calculations using CRS for any binary outcome (and produce valid confidence intervals).
The main function perform_crs(...)
takes in a dataset containing one row per sample and at least three columns that represent the index text, imperfect truth and the resolver. The index test and the imperfect truth are expected to have no missing values, however as per the method, the resolver column may contain missing (NA
) values as not every sample is expected to have a resolver result.
Details are available in the paper: Clinical evaluation of the APAS Independence: Automated imaging and interpretation of urine cultures using artificial intelligence with composite reference standard discrepant resolution.
Additional background information on CRS can be found in Hawkins et al. (2001) and Alonzo and Pepe (1999).
To install and load in R, run:
library(devtools) # see http://cran.r-project.org/web/packages/devtools/README.html
devtools::install_github('tystan/crs')
library(crs)
### see help file to run example
?perform_crs
A PDF manual of the function descriptions is included in the repository here: /inst/doc/crs-help.pdf.
You can reproduce the analysis in Brenton et al. (2019). First load one of the dataframes that come with the package: brenton2019
.
data(brenton2019) # load data from Brenton et al. (2019)
?brenton2019 # will bring up a description of the data
brenton2019 # have a look at data
## A tibble: 881 x 3
## A S P
## <int> <int> <int>
## 1 1 1 NA
## 2 1 1 NA
## 3 2 2 NA
## 4 2 2 NA
## 5 2 1 2
## 6 1 1 1
## 7 1 1 1
## 8 1 1 NA
## 9 1 1 NA
##10 1 1 1
## ... with 871 more rows
To run CRS, pass the data (or any dataset in the required form) to the perform_crs()
function.
# run CRS analysis on brenton2019 data
perform_crs(
brenton2019, # data
index = "A", # index test
imperfect = "S", # imperfect truth
resolver = "P" # resolver test (with NAs present)
)
The resulting output:
param | p | var_p | se_p | trans | p_lo | p_up |
---|---|---|---|---|---|---|
sens | 0.919 | 0.00031 | 0.0176 | probit | 0.879 | 0.948 |
spec | 0.877 | 0.00051 | 0.0226 | probit | 0.827 | 0.916 |
You can also recreate the results of Hawkins et al. (2001) by running the below.
data(hawkins2001) # load data from Hawkins et al. (2001)
hawkins2001 # have a look at data
# run CRS analysis on hawkins2001 data
perform_crs(
hawkins2001, # data
index = "index", # index test column in the data
imperfect = "ref", # imperfect truth column in the data
resolver = "resolve" # resolver test column in the data (with NAs present)
)
param | p | var_p | se_p | trans | p_lo | p_up |
---|---|---|---|---|---|---|
sens | 0.975 | 0.000032 | 0.00567 | probit | 0.962 | 0.984 |
spec | 0.550 | 0.000922 | 0.03036 | probit | 0.490 | 0.609 |
CRS can be compared to the index test against the imperfect truth by using the get_sens_spec()
function.
For the brenton2019
data:
# the sens+spec estimates using the imperfect truth only
get_sens_spec(
table(brenton2019[["S"]], brenton2019[["A"]])
)
param | cases | correct | est | lo | up |
---|---|---|---|---|---|
sens | 449 | 423 | 0.942 | 0.917 | 0.960 |
spec | 432 | 342 | 0.792 | 0.751 | 0.827 |
Similarly, for the hawkins2001
data:
# the sens+spec estimates using the imperfect truth only
get_sens_spec(
table(hawkins2001[["ref"]], hawkins2001[["index"]])
)
param | cases | correct | est | lo | up |
---|---|---|---|---|---|
sens | 2000 | 1800 | 0.900 | 0.886 | 0.912 |
spec | 1000 | 400 | 0.400 | 0.370 | 0.431 |
Please run the following to get citation information when referencing this code.
citation("crs")