oxwearables / asleep Goto Github PK
View Code? Open in Web Editor NEWasleep: a sleep classifier for wearable sensor data using machine learning
License: Other
asleep: a sleep classifier for wearable sensor data using machine learning
License: Other
Probably a corner case.
This bug can be reproduced e.g. with the file "ukb/accelerometer/group3/5563512_90001_0_0.cwa.gz"
See attached file for the log.
pandas_error.txt
Please see: OxWearables/stepcount#33
Very interesting package, which I would like to use on data we collected with the Axivity devices. Do you have tips for parallelisation of the get_sleep command to speed up the analysis process of multiple files? I tried to use the command in a shell script using xargs for parallelisation but ran into an error thrown by the Java Runtime (see attached file).
Currently, the summary statistics only generate the sleep parameters for overnight sleep duration but not the for the total sleep time.
To do this, we will need to add the computation for all the sleep blocks for each day in the day
class object.
Having parsed the data into the right format, we will need to
Is there any chance you could add time zone correction argument(s) similar to GGIR/Accelerometer? Some of our watches were improperly configured, so their time zones need adjusted.
Thanks!
This bug can be reproduced using e.g. the file "ukb/accelerometer/group10/4877373_90001_0_0.cwa.gz"
See attached file for the log.
index_error_bug.txt
Hey @angerhang , the last bug fix worked well. I have now got through the processing on all our files and have come across a couple of additional issues.
For a number of files we are getting the type error in the title - I have attached a txt output of this. This happens in the JR001.gt3x file that you had previously.
I am also seeing a less frequent issue where I run on a given file and it appears to start, produces the text Saving files to dir: outputs/'filename', but then stops processing and waits for the next file input. It appears that it does not successfully read the file. I didn't know if you may have come across this/had any immediate thoughts? If not I would plan to analyse all the data we can, if we can fix the type error issue and then if all is looking good with those work through everything we need to, to get some of our study files to you to see if you can replicate.
Run the asleep package on some of the UKB files to see if the results line up
Update the model weights with the latest one.
Actipy now renders all the non-wear as NA but this is not ideal because it is likely to yield false positives during sleep periods.
Having obtained all the required sleep predictions, we will want to generate the common sleep parameters for the downstream analysis
actipy
. We would want to store the non-wear information in certain format that would allow to do QAs based on the non-wear threshold.When dealing with NAs, it is probably to specify the the min_duration is 22 hours because it could be that at the start and the end of particular recording, we have a very short day hence also a short NA duration. This might change a bit of the statistics at the beginning and the end.
Before we publish this package we should do the following:
get_sleep
and visu_sleep
Since sleepnet requires a proxy for time-in-bed as an input, we will need to use the random forest model and its output:
We will take this RF to train a sleep vs non-sleep detector:
https://github.com/OxWearables/stepcount/blob/e53ec0594a2bad40214a7efdda8f7b773869ceb4/stepcount/models.py#L229
The asleep
package should behave the same way as the step count page at least in terms of input and output interface
I believe it is simply a matter of better error handling.
This bug can be reproduced e.g. with the file "ukb/accelerometer/group1/4380443_90001_0_0.cwa.gz"
See attached file for the log.
index_error_log.txt
Add versioneer to support for automatic publishing for version number increment
I've encountered several critical issues while attempting to use this project, which have significantly hindered my ability to effectively leverage its capabilities. These issues pertain to the lack of dependency pinning, compatibility problems with newer versions of Python, and the overall usefulness of the results produced. Below is a detailed account of each problem and its impact on my experience.
It took a while to finde the working dependencies. env.yml.txt
New versions just break it enirely.
The project is missing sample data to verify that it works as expected.
Testing muliple datasets the results were always 0 or a few minutes of sleep. As I cannot upload our samples the same happens with sample data from another issue:
#33 (comment)
Output:
Saving files to dir: outputs/sample
Raw data file already exists. Skipping raw data parsing.
Unnamed: 0 time x y z non_wear
0 0 2023-08-15 15:00:00.017000000 -0.461000 0.894996 -0.116996 False
1 1 2023-08-15 15:00:00.050333333 -0.443447 0.781798 -0.121022 False
2 2 2023-08-15 15:00:00.083666666 -0.527073 0.767216 -0.148742 False
3 3 2023-08-15 15:00:00.117000000 -0.605458 0.903918 -0.139534 False
4 4 2023-08-15 15:00:00.150333333 -0.689671 1.284508 -0.111031 False
Data2model file already exists. Skip data transformation.
data2model shape: (8640, 3, 900)
times shape: (8640,)
Non_wear flag shape: (8640,)
prediction set sample count: 8640
Using local /home/tobi/asleep/src/asleep/torch_hub_cache/OxWearables_ssl-wearables_v1.0.0
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [01:38<00:00, 19.62s/it]
(8640,)
(array([0., 1., 2.]), array([5807, 451, 2382]))
time label is_wear
0 2023-08-15 15:00:00.017 0 True
1 2023-08-15 15:00:30.017 0 True
2 2023-08-15 15:01:00.017 0 True
3 2023-08-15 15:01:30.017 0 True
4 2023-08-15 15:02:00.017 0 True
No sleep windows >30 mins detected. Exiting...
Non-wear time has been written to outputs/sample/non_wear.npy
Current sleep classification has been written to outputs/sample/ssl_sleep.npy
Any thoughts @angerhang
Right now, get_sleep
function is a bit messy.
We probably should clean it up soon/
This is very interested work, and we are interested in using the algorithm with some of our actigraphy data but have been running into an error. We are using .gt3x files and when running the code, the initial steps seem to work and a handful of outputs are produced: a raw csv, an info JSON file and 2 NPY files (data2model and times), but we then get the error: RuntimeError: torch.cat(): expected a non-empty list of Tensors.
Are you able to advise what the issue may be?
I believe this is also a matter of better error handling.
This bug can be reproduced e.g. with the file "ukb/accelerometer/group5/3801805_90001_0_0.cwa.gz"
See attached file for the log.
index_error_log.txt
The monitoring StartTime and EndTime is automatically extracted for CWA files but not csv files, leading to a processing error.
Therefore, for csv files, we need to be able to extract this information from the provided 'time' column.
In contrast to other OxWearables packages, asleep produces an "info.json" file for actipy parameters and a "summary.json" file for the sleep-related parameters. I believe these two should be one file.
Also, the --remove_intermediate_files argument deletes "info.json", which I believe is wrong. It is not an intermediate file but an important part of the final output.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.