Comments (9)
scollis
commented on May 29, 2024
what does a ncdump look like? Sounds like the format is horrible!
from act.
scollis
commented on May 29, 2024
we might need to make a custom ingest from netCDF4 in this case.
from act.
rcjackson
commented on May 29, 2024
You can specify decode_coords=False to xarray.open_dataset and that should at least disable the dimension labelling for nasty datasets like this.
…________________________________
From: Adam Theisen <[email protected]>
Sent: Wednesday, May 22, 2019 9:33:25 AM
To: ANL-DIGR/ACT
Cc: Subscribed
Subject: [ANL-DIGR/ACT] Xarray not able to read old MMCR data (#67)
Some older data might not be compatible with xarray. sgpmmcrmomC1.b1.20041107 data yielded this error:
xarray.core.variable.MissingDimensionsError: 'heights' has more than 1-dimension and the same name as one of its dimensions ('mode', 'heights'). xarray disallows such variables because they conflict with the coordinates used to label dimensions.
Is there a way around this?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<#67>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFIQA5FHMG6NXEAFKAGDYNLPWVKTLANCNFSM4HOU6TAQ>.
from act.
AdamTheisen
commented on May 29, 2024
netcdf sgpmmcrmomC1.b1.20041107.000014 {
dimensions:
time = UNLIMITED ; // (44959 currently)
mode = 10 ;
namelength = 64 ;
heights = 167 ;
hourly = 24 ;
variables:
int base_time ;
base_time:string = "7-Nov-2004,0:00:14 GMT" ;
base_time:long_name = "Base time in Epoch" ;
base_time:units = "seconds since 1970-1-1 0:00:00 0:00" ;
double time_offset(time) ;
time_offset:long_name = "Time offset from base_time" ;
time_offset:units = "seconds since 2004-11-07 00:00:14 0:00" ;
double time(time) ;
time:long_name = "Time offset from midnight" ;
time:units = "seconds since 2004-11-07 00:00:00 0:00" ;
float heights(mode, heights) ;
heights:long_name = "Range Heights (center of radar sample volume)" ;
heights:units = "m MSL" ;
heights:missing_value = -9999.f ;
int DataQualityStatus(time) ;
DataQualityStatus:long_name = "Data Quality Status" ;
DataQualityStatus:units = "code" ;
DataQualityStatus:missing_value = -9999 ;
float MeanDopplerVelocity(time, heights) ;
MeanDopplerVelocity:long_name = "Mean Doppler Velocity" ;
MeanDopplerVelocity:units = "m/s" ;
MeanDopplerVelocity:resolution = 0.001f ;
MeanDopplerVelocity:missing_value = -9999.f ;
float SignalToNoiseRatio(time, heights) ;
SignalToNoiseRatio:long_name = "Signal to Noise Ratio" ;
SignalToNoiseRatio:units = "dB" ;
SignalToNoiseRatio:resolution = 0.001f ;
SignalToNoiseRatio:missing_value = -9999.f ;
float Power(time, heights) ;
Power:long_name = "Power (uncalibrated)" ;
Power:units = "dB" ;
Power:resolution = 0.001f ;
Power:missing_value = -9999.f ;
float SpectralWidth(time, heights) ;
SpectralWidth:long_name = "Spectral Width" ;
SpectralWidth:units = "m/s" ;
SpectralWidth:resolution = 0.001f ;
SpectralWidth:missing_value = -9999.f ;
float NoiseLevel(time, heights) ;
NoiseLevel:long_name = "Mean Noise Level" ;
NoiseLevel:units = "dB" ;
NoiseLevel:resolution = 0.001f ;
NoiseLevel:missing_value = -99999.f ;
float Reflectivity(time, heights) ;
Reflectivity:long_name = "Reflectivity" ;
Reflectivity:units = "dBZ" ;
Reflectivity:resolution = 0.001f ;
Reflectivity:missing_value = -9999.f ;
Reflectivity:comment1 = "Nearfield correction has been applied" ;
Reflectivity:comment2 = "Reference: Sekelsky, S. (2002), JAOT, v19, no. 4" ;
float RangeCorrectedPower(time, heights) ;
RangeCorrectedPower:long_name = "Range Corrected Calibrated Power" ;
RangeCorrectedPower:units = "dBm" ;
RangeCorrectedPower:resolution = 0.001f ;
RangeCorrectedPower:missing_value = -9999.f ;
float CircularDepolarizationRatio(time, heights) ;
CircularDepolarizationRatio:long_name = "Circular Depolarization Ratio" ;
CircularDepolarizationRatio:units = "dB" ;
CircularDepolarizationRatio:resolution = 0.001f ;
CircularDepolarizationRatio:missing_value = -9999.f ;
from act.
AdamTheisen
commented on May 29, 2024
decode_coords=False did not work for this case
from act.
rcjackson
commented on May 29, 2024
Then we would need a procedure that looks for variables that have the same name as a dimension in a netCDF file. Then it would suggest a change to the user and save a new file for xarray compatibility.
Bobby
…________________________________
From: Adam Theisen <[email protected]>
Sent: Wednesday, May 22, 2019 9:45:31 AM
To: ANL-DIGR/ACT
Cc: Jackson, Robert; Comment
Subject: Re: [ANL-DIGR/ACT] Xarray not able to read old MMCR data (#67)
netcdf sgpmmcrmomC1.b1.20041107.000014 {
dimensions:
time = UNLIMITED ; // (44959 currently)
mode = 10 ;
namelength = 64 ;
heights = 167 ;
hourly = 24 ;
variables:
int base_time ;
base_time:string = "7-Nov-2004,0:00:14 GMT" ;
base_time:long_name = "Base time in Epoch" ;
base_time:units = "seconds since 1970-1-1 0:00:00 0:00" ;
double time_offset(time) ;
time_offset:long_name = "Time offset from base_time" ;
time_offset:units = "seconds since 2004-11-07 00:00:14 0:00" ;
double time(time) ;
time:long_name = "Time offset from midnight" ;
time:units = "seconds since 2004-11-07 00:00:00 0:00" ;
float heights(mode, heights) ;
heights:long_name = "Range Heights (center of radar sample volume)" ;
heights:units = "m MSL" ;
heights:missing_value = -9999.f ;
int DataQualityStatus(time) ;
DataQualityStatus:long_name = "Data Quality Status" ;
DataQualityStatus:units = "code" ;
DataQualityStatus:missing_value = -9999 ;
float MeanDopplerVelocity(time, heights) ;
MeanDopplerVelocity:long_name = "Mean Doppler Velocity" ;
MeanDopplerVelocity:units = "m/s" ;
MeanDopplerVelocity:resolution = 0.001f ;
MeanDopplerVelocity:missing_value = -9999.f ;
float SignalToNoiseRatio(time, heights) ;
SignalToNoiseRatio:long_name = "Signal to Noise Ratio" ;
SignalToNoiseRatio:units = "dB" ;
SignalToNoiseRatio:resolution = 0.001f ;
SignalToNoiseRatio:missing_value = -9999.f ;
float Power(time, heights) ;
Power:long_name = "Power (uncalibrated)" ;
Power:units = "dB" ;
Power:resolution = 0.001f ;
Power:missing_value = -9999.f ;
float SpectralWidth(time, heights) ;
SpectralWidth:long_name = "Spectral Width" ;
SpectralWidth:units = "m/s" ;
SpectralWidth:resolution = 0.001f ;
SpectralWidth:missing_value = -9999.f ;
float NoiseLevel(time, heights) ;
NoiseLevel:long_name = "Mean Noise Level" ;
NoiseLevel:units = "dB" ;
NoiseLevel:resolution = 0.001f ;
NoiseLevel:missing_value = -99999.f ;
float Reflectivity(time, heights) ;
Reflectivity:long_name = "Reflectivity" ;
Reflectivity:units = "dBZ" ;
Reflectivity:resolution = 0.001f ;
Reflectivity:missing_value = -9999.f ;
Reflectivity:comment1 = "Nearfield correction has been applied" ;
Reflectivity:comment2 = "Reference: Sekelsky, S. (2002), JAOT, v19, no. 4" ;
float RangeCorrectedPower(time, heights) ;
RangeCorrectedPower:long_name = "Range Corrected Calibrated Power" ;
RangeCorrectedPower:units = "dBm" ;
RangeCorrectedPower:resolution = 0.001f ;
RangeCorrectedPower:missing_value = -9999.f ;
float CircularDepolarizationRatio(time, heights) ;
CircularDepolarizationRatio:long_name = "Circular Depolarization Ratio" ;
CircularDepolarizationRatio:units = "dB" ;
CircularDepolarizationRatio:resolution = 0.001f ;
CircularDepolarizationRatio:missing_value = -9999.f ;
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<#67>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFIQA5GMZQJFE2RADGDTQZDPWVMAXANCNFSM4HOU6TAQ>.
from act.
scollis
commented on May 29, 2024
Oh crud.. it puts all the modes in one file!
Bobby is right.. we effectively need a mode splitter
From: Bobby Jackson <[email protected]>
Reply-To: ANL-DIGR/ACT <[email protected]>
Date: Wednesday, May 22, 2019 at 9:51 AM
To: ANL-DIGR/ACT <[email protected]>
Cc: Scott <[email protected]>, Comment <[email protected]>
Subject: Re: [ANL-DIGR/ACT] Xarray not able to read old MMCR data (#67)
Then we would need a procedure that looks for variables that have the same name as a dimension in a netCDF file. Then it would suggest a change to the user and save a new file for xarray compatibility.
Bobby
…________________________________
From: Adam Theisen <[email protected]>
Sent: Wednesday, May 22, 2019 9:45:31 AM
To: ANL-DIGR/ACT
Cc: Jackson, Robert; Comment
Subject: Re: [ANL-DIGR/ACT] Xarray not able to read old MMCR data (#67)
netcdf sgpmmcrmomC1.b1.20041107.000014 {
dimensions:
time = UNLIMITED ; // (44959 currently)
mode = 10 ;
namelength = 64 ;
heights = 167 ;
hourly = 24 ;
variables:
int base_time ;
base_time:string = "7-Nov-2004,0:00:14 GMT" ;
base_time:long_name = "Base time in Epoch" ;
base_time:units = "seconds since 1970-1-1 0:00:00 0:00" ;
double time_offset(time) ;
time_offset:long_name = "Time offset from base_time" ;
time_offset:units = "seconds since 2004-11-07 00:00:14 0:00" ;
double time(time) ;
time:long_name = "Time offset from midnight" ;
time:units = "seconds since 2004-11-07 00:00:00 0:00" ;
float heights(mode, heights) ;
heights:long_name = "Range Heights (center of radar sample volume)" ;
heights:units = "m MSL" ;
heights:missing_value = -9999.f ;
int DataQualityStatus(time) ;
DataQualityStatus:long_name = "Data Quality Status" ;
DataQualityStatus:units = "code" ;
DataQualityStatus:missing_value = -9999 ;
float MeanDopplerVelocity(time, heights) ;
MeanDopplerVelocity:long_name = "Mean Doppler Velocity" ;
MeanDopplerVelocity:units = "m/s" ;
MeanDopplerVelocity:resolution = 0.001f ;
MeanDopplerVelocity:missing_value = -9999.f ;
float SignalToNoiseRatio(time, heights) ;
SignalToNoiseRatio:long_name = "Signal to Noise Ratio" ;
SignalToNoiseRatio:units = "dB" ;
SignalToNoiseRatio:resolution = 0.001f ;
SignalToNoiseRatio:missing_value = -9999.f ;
float Power(time, heights) ;
Power:long_name = "Power (uncalibrated)" ;
Power:units = "dB" ;
Power:resolution = 0.001f ;
Power:missing_value = -9999.f ;
float SpectralWidth(time, heights) ;
SpectralWidth:long_name = "Spectral Width" ;
SpectralWidth:units = "m/s" ;
SpectralWidth:resolution = 0.001f ;
SpectralWidth:missing_value = -9999.f ;
float NoiseLevel(time, heights) ;
NoiseLevel:long_name = "Mean Noise Level" ;
NoiseLevel:units = "dB" ;
NoiseLevel:resolution = 0.001f ;
NoiseLevel:missing_value = -99999.f ;
float Reflectivity(time, heights) ;
Reflectivity:long_name = "Reflectivity" ;
Reflectivity:units = "dBZ" ;
Reflectivity:resolution = 0.001f ;
Reflectivity:missing_value = -9999.f ;
Reflectivity:comment1 = "Nearfield correction has been applied" ;
Reflectivity:comment2 = "Reference: Sekelsky, S. (2002), JAOT, v19, no. 4" ;
float RangeCorrectedPower(time, heights) ;
RangeCorrectedPower:long_name = "Range Corrected Calibrated Power" ;
RangeCorrectedPower:units = "dBm" ;
RangeCorrectedPower:resolution = 0.001f ;
RangeCorrectedPower:missing_value = -9999.f ;
float CircularDepolarizationRatio(time, heights) ;
CircularDepolarizationRatio:long_name = "Circular Depolarization Ratio" ;
CircularDepolarizationRatio:units = "dB" ;
CircularDepolarizationRatio:resolution = 0.001f ;
CircularDepolarizationRatio:missing_value = -9999.f ;
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<#67>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFIQA5GMZQJFE2RADGDTQZDPWVMAXANCNFSM4HOU6TAQ>.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
from act.
AdamTheisen
commented on May 29, 2024
Yep, this was the old fashion way of handling modes. Interleaved in the data. Mode varied with time and you had to pull out the modes. Fun times
from act.
AdamTheisen
commented on May 29, 2024
I think I have a method down here that uses the NetCDF4 library. The only question then is if we want to expand the function to breakout the modes.
from act.
Related Issues (20)
- Add ARM Data Surveyor Command Line Tool
- Attempting to set identical low and high xlims HOT 1
- CAPE Calc Problem HOT 6
- Valid Min in act.utils.decode_present_weather
- Documentation error: ACT Cheat Sheet HOT 1
- Setting y-range for groupby causes error HOT 5
- Missing kwargs in plot_stacked_bar_graph HOT 3
- Example Incorrect
- act.utils.decode_present_weather classification tables HOT 2
- Feedstock failing due to pandas datetime
- Histogram Display Examples HOT 2
- New Example Template
- Standardize discovery module HOT 7
- DistributionDisplay Consistent Names
- Adding plotting options to Skew-T HOT 4
- 2023 ARM-ASR Joint PI Meeting - I/O Suggestions HOT 3
- QC-aware transformations HOT 8
- UserWarning for Time HOT 5
- Replace current CVD colormaps with CMWeather HOT 1
- Epoch addition to ACT Discovery HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from act.