Code Monkey home page Code Monkey logo

wreport's Introduction

Build Status Build Status Build Status Build Status Build Status

WREPORT

Introduction

wreport is a C++ library for working with weather reports.

wreport is a powerful decoder and encoder for the BUFR and CREX formats.

It also provides a useful abstraction to handle values found in weather reports, with awareness of significant digits, measurement units, variable descriptions, unit conversion and attributes on variables.

Features provided:

  • Read and write BUFR version 2, 3, and 4
  • Read and write CREX
  • Unit conversion
  • Handling of physical variables

C++ and Python API documentation.

Installing wreport

wreport is already packaged in both .rpm and .deb formats, and that provides easy installation for most Linux distributions.

For CentOS and Fedora, rpm files are hosted in a copr repo: https://copr.fedorainfracloud.org/coprs/simc/stable/

For Debian, wreport is available in the testing distribution: https://packages.debian.org/testing/libwreport3

Using docker images with wreport preinstalled is also possible:

docker run -it arpaesimc/fedora:36 /bin/bash
docker run -it arpaesimc/centos:8 /bin/bash

If you want to build and install wreport yourself, you'll need to install Meson and run the following commands:

meson setup builddir && cd builddir
meson compile
meson test
meson install

For more details about Meson see the official documentation https://mesonbuild.com/.

If you want to build the package yourself:

  • rpm: the packaging files are in fedora directory of the master branch.
  • deb: the packaging files are in the debian branches (e.g. debian/sid, ubuntu/jammy, etc.)

AFL instrumentation

To run wreport using American Fuzzy Lop:

CXX=/usr/bin/afl-g++ meson --default-library=static builddir && cd builddir
AFL_HARDEN=1 meson compile
afl-cmin  -i testdata/bufr/ -o afl-bufr -- src/afl-test @@
afl-fuzz -t 100 -i afl-bufr -o afl-bufr-out src/afl-test @@

Contact and copyright information

The author of wreport is Enrico Zini [email protected]

wreport is Copyright (C) 2005-2024 ARPAE-SIMC [email protected]

wreport is licensed under the terms of the GNU General Public License version 2.

wreport's People

Contributors

brancomat avatar dariomas avatar dcesari avatar edigiacomo avatar spanezz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wreport's Issues

New tables required

Abbiamo incontrato alcuni messaggi SYNOP (WIGOS?) che richiedono le tabelle versione 33 (centro 98 ECMWF) per la decodifica, ne allego uno, BABNO_POLJE.zip, a titolo di esempio.

Inserendo le tabelle
B0000000000000033000.txt
B0000000000098033002.txt
B0000000000098033001.txt
C0000000000000033000.txt
C0000000000098033002.txt
C0000000000098033001.txt
D0000000000000033000.txt
D0000000000098033002.txt
D0000000000098033001.txt

da uno degli ultimi pacchetti di bufrdc_tables in https://confluence.ecmwf.int/display/BUFR/BUFRDC+Home in una cartella locale puntata da $WREPORT_TABLES la decodifica (e successiva trasformazione in netcdf) funziona. Come possiamo procedere per completare le tabelle nei sorgenti minimizzando l'entropia? Ha senso inserire anche tutte le versioni intermedie?

Quality information value out of range

This issue vaguely reminds #48: the attached aircraft report qinfo_overflow.bufr.gz fails with

Value 127 is outside the range [0,2] for 033002 (Quality information) at offset 0.

the only clue is given by wrep -t which ends with

...
   011105 EDR algorithm version[NUMERIC]
           -
    204007 7 bits of associated field
    011076 Peak turbulence intensity (eddy dissipation rate)[M2/3 S-1]
qinfo_overflow.bufr:0:Value 127 is outside the range [0,2] for 033002 (Quality information)

does it mean that 33002 has been resized to 7 bits and the missing value (127) is not recognized?

Bug in applying C table modifier

While trying to decode the attached bufr (TEMP-like vertical profile) which makes use of the C table descriptor 207YYY (Increase scale, reference value and data width) wreport crashes because of a bug.

Value -3.06757e+06 is outside the range [-1e+06,-2.85908e+06] for B10009 (Geopotential height)

I allow myself to commit a patch to master since it is an evident and localised bug and it is clear that we had never tested wreport on such a modifier. I just create this issue to keep track, maybe the message can be used as a test case.

I tracked the problem thanks to TRACE instructions in interpreter.cc, BTW is make CPPFLAGS=-DTRACE_INTERPRETER the correct way to enable trace?

TEMN_2020022200_001.bufr.zip

ARM support in Travis build

In arm32v7 branch I added the ARM builds. I am using the images built in the multiarch organization, that supports Fedora (armhfp and aarch64) and CentOS 7 (armhfp, aarch64 and i386).

Open issues:

failed test

After #26 commits builds fail.

See https://travis-ci.org/ARPA-SIMC/wreport/builds/497104739

e.g.:

bulletin_dds_validator: BUFR 0:2:255 234:0 2019-02-07 00:00:00 1 1 subsets
 Tables: /root/rpmbuild/BUILD/wreport-3.19-1/wreport/../tables/B0000000000000028000.txt /root/rpmbuild/BUILD/wreport-3.19-1/wreport/../tables/D0000000000000028000.txt
 Data descriptors:
  203014
  007030
  007031
  203255
  301150
  307080
 BUFR details: ed4 t0:28:0 - osl14
 Variables:
dump interrupted: define_c03_refval_override is not implemented in this interpreter
✘.
tests: .
lua: .
 * Test failures
bulletin_dds_validator.bufr: define_c03_refval_override is not implemented in this interpreter
  bulletin/dds-validator-test.cc:67:validate(*msg1) [bufr/wigos.bufr]
 * Result summary
1/141 tests failed

Installation from source

The INSTALL file says:

 `cd' to the directory containing the package's source code and type
     `./configure' to configure the package for your system.

But there is no configure.ac. How is one meant to build from source?

Wrong decodification of a descriptor

La decodifica del messaggio allegato B005.zip, AMV, Atmospheric Motion Vectors, derivato da dati satellitari, il cui template è probabilmente cambiato di recente, fallisce perché il primo descrittore sembra male interpretato:

# wrep -D B005_2021030300
310077 (group)
  000103 (missing in B table /usr/share/wreport/B0000000000000031000.txt)
  001034 Identification of originating/generating sub-centre[COMMON CODE TABLE C-12]
  025061 Software identification and version number[CCITTIA5]
...

mentre:

# bufr_dump -d B005_2021030300
***** FILE: B005_2021030300 
001033  centre  IDENTIFICATION OF ORIGINATING/GENERATING CENTRE [Common CODE TABLE C-1]
001034  subCentre       IDENTIFICATION OF ORIGINATING/GENERATING SUB-CENTRE [Common CODE TABLE C-12]
025061  softwareVersionNumber   SOFTWARE IDENTIFICATION AND VERSION NUMBER [CCITT IA5]
...

cioè 001033 al posto di 000103 che è in effetti fuori dai limiti dei valori presenti nella relativa sezione della tabella. Tutti gli altri descrittori sembrano decodificati correttamente. Se trucco la tabella aggiungendo la riga 000103 uguale a 001033 la decodifica arriva in fondo. C'è un bug in decodifica o il messaggio ha qualche stranezza inedita?

Build python bindings documentation on CentOs

Building the Python API documentation on Centos7 fails with this error:

Running Sphinx v1.2.3
1734loading pickled environment... failed: [Errno 2] No such file or directory: '/root/rpmbuild/BUILD/wreport-3.23-1/python/apidocs/.doctrees/environment.pickle

For now, I'm disabling building of the documentation on Cetnos7. It can be reenabled if we find the cause of that error, or this bug can be closed if we drop support for Centos7 at some point.

error in decoding bufr (missing table?)

The attached file SPC.zip contains 4 GTS bufr messages which cannot be parsed by wreport:

$ dbamsg scan --verbose IUSD01ERSA190000.BIN
could not detect the encoding of IUSD01ERSA190000.BIN

$ dbamsg scan -t bufr --verbose IUSD01ERSA190000.BIN 
Cannot parse BUFR message #0: BUFR table for center 80:0 table 0:29:0 not found at offset 20.
#0 BUFR message: 109724 bytes.

I labeled this as a question since I'm not sure if the bufr is malformed (their fault) or we're missing a table (our fault)

Inconsistent overflow check for string values

It seems that the scale is applied after checking the range:

python3 <<EOF
import wreport
table = wreport.Vartable.get_bufr(master_table_version_number=24)
print("B13011 scale:", table["B13011"].scale)
print('"-1" =', wreport.Var(table["B13011"], "-1"))
print('"-9" =', wreport.Var(table["B13011"], "-9"))
EOF

B13011 scale: 1
"-1" = -0.1
Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
OverflowError: Value -9 is outside the range [-1,16381] for 013011 (Total precipitation/total water equivalent)

print('"-9" =', wreport.Var(table["B13011"], "-9")) should print -0.9 instead of throwing an error.

Issue with compiling latest wreport on centos 6.5

Hi

I followed the instructions in the README and run: autoreconf -if after untaring the source. I get the following error:

[root@fep wreport-3.3-1]# autoreconf -if
/usr/bin/m4:configure.ac:172: recursion limit of 1024 exceeded, use -L<N> to change it
autom4te: /usr/bin/m4 failed with exit status: 1
aclocal: autom4te failed with exit status: 1
autoreconf: aclocal failed with exit status: 1

What should I do to be able to compile it?

Thanks

[root@fep wreport-3.3-1]# rpm -qf /usr/bin/autoreconf
autoconf-2.63-5.1.el6.noarch

CI test of packages that use wreport after a wreport update

with libwreport 3.21 and bufr2netcdf 1.5 (Fedora 28 and Centos):

bufr2netcdf -o obs obs_1.bufr
...
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
...
(gdb) where
#0  0x0000000000000000 in ?? ()
#1  0x00007ffff459e244 in wreport::bulletin::Interpreter::run (this=0x7fffffffd4a0) at bulletin/interpreter.cc:38
#2  0x00007ffff459e130 in wreport::bulletin::Interpreter::run (this=0x7fffffffd4a0) at bulletin/interpreter.cc:70
#3  0x0000000000413c11 in b2nc::Plan::build (this=0x64eb58, bulletin=...) at plan.cc:516
#4  0x000000000040bcef in b2nc::Arrays::add(std::unique_ptr<wreport::Bulletin, std::default_delete<wreport::Bulletin> >&&) (this=0x64eb58, 
    bulletin=<unknown type in /usr/lib/debug/usr/bin/bufr2netcdf.debug, CU 0x21f1a, DIE 0x2dd7a>) at arrays.cc:367
#5  0x0000000000406ef3 in b2nc::NCFiller::add(std::unique_ptr<wreport::BufrBulletin, std::default_delete<wreport::BufrBulletin> >&&, std::string const&) (this=0x64eb58, bulletin=<unknown type in /usr/lib/debug/usr/bin/bufr2netcdf.debug, CU 0xbd08, DIE 0x1cdf3>, 
    raw="BUFR\000\000\222\004\000\000\026\000\000\377\000\377\000\000\004\000\b\r\000\a\343\a\001\022\000\000\000\000\070\000\000\001\200\313\005\b\004\002@\r\003\fg\r\002B\000\037\001\vK\vL\v%\v'\vM\024*\024+\024,\024-\024)\002\005\002>\002F\002A\a\004!\032\000\000\000\070\000BVOEOGBA\377\261;\317\t\334\324G\343pd\000\001\371\213V\006/\377ۏ?\377\377\377\377\376\001\377\377\377\377\377\377\377\377\360\000\000\000\017\377\377\000\067\067\067\067") at convert.cc:219
#6  0x00000000004075e9 in b2nc::OutfileImpl::add_bufr(std::unique_ptr<wreport::BufrBulletin, std::default_delete<wreport::BufrBulletin> >&&, std::string const&) (this=0x64eb50, bulletin=<unknown type in /usr/lib/debug/usr/bin/bufr2netcdf.debug, CU 0xbd08, DIE 0x1cdf3>, 
    raw="BUFR\000\000\222\004\000\000\026\000\000\377\000\377\000\000\004\000\b\r\000\a\343\a\001\022\000\000\000\000\070\000\000\001\200\313\005\b\004\002@\r\003\fg\r\002B\000\037\001\vK\vL\v%\v'\vM\024*\024+\024,\024-\024)\002\005\002>\002F\002A\a\004!\032\000\000\000\070\000BVOEOGBA\377\261;\317\t\334\324G\343pd\000\001\371\213V\006/\377ۏ?\377\377\377\377\376\001\377\377\377\377\377\377\377\377\360\000\000\000\017\377\377\000\067\067\067\067") at convert.cc:309
#7  0x0000000000405f9b in b2nc::Dispatcher::add_bufr(std::unique_ptr<wreport::BufrBulletin, std::default_delete<wreport::BufrBulletin> >&&, std::string const&) (this=0x7fffffffd940, bulletin=<unknown type in /usr/lib/debug/usr/bin/bufr2netcdf.debug, CU 0xbd08, DIE 0x1cdf3>, 
    raw="BUFR\000\000\222\004\000\000\026\000\000\377\000\377\000\000\004\000\b\r\000\a\343\a\001\022\000\000\000\000\070\000\000\001\200\313\005\b\004\002@\r\003\fg\r\002B\000\037\001\vK\vL\v%\v'\vM\024*\024+\024,\024-\024)\002\005\002>\002F\002A\a\004!\032\000\000\000\070\000BVOEOGBA\377\261;\317\t\334\324G\343pd\000\001\371\213V\006/\377ۏ?\377\377\377\377\376\001\377\377\377\377\377\377\377\377\360\000\000\000\01---Type <return> to continue, or q <return> to quit---
7\377\377\000\067\067\067\067") at convert.cc:155
#8  0x00000000004055ff in b2nc::read_bufr (in=0x6472d0, out=..., fname=0x647058 "obs_1.bufr") at convert.cc:67
#9  0x0000000000405517 in b2nc::read_bufr (fname="obs_1.bufr", out=...) at convert.cc:47
#10 0x00000000004052e8 in main (argc=4, argv=0x7fffffffdab8) at bufr2netcdf.cc:112

with libwreport 3.18 and same bufr2netcdf (on Fedora24) it works.

I attach the bufr, but I think it does not depend on the message.

obs_1.zip

Wrong decoding of B33007 descriptor

In the attached bufr.zip file (satellite derived products, with subsets) wreport incorrectly decodes the data confidence information B33007:

wrep B005_2018031500_exmpl.bufr
...
  [0][14] 002023 SATELLITE DERIVED WIND COMPUTATION METHOD(CODE TABLE): 5
  [0][15] 007004 PRESSURE(PA): 43700
           033007 PER CENT CONFIDENCE(%): 0
           033035 MANUAL/AUTOMATIC QUALITY CONTROL(CODE TABLE): (undef)
           033036 NOMINAL CONFIDENCE THRESHOLD(%): (undef)
  [0][16] 011001 WIND DIRECTION(DEGREE TRUE): 265
           033007 PER CENT CONFIDENCE(%): 0
           033035 MANUAL/AUTOMATIC QUALITY CONTROL(CODE TABLE): (undef)
           033036 NOMINAL CONFIDENCE THRESHOLD(%): (undef)
  [0][17] 011002 WIND SPEED(M/S): 11.0
           033007 PER CENT CONFIDENCE(%): 0
           033035 MANUAL/AUTOMATIC QUALITY CONTROL(CODE TABLE): (undef)
           033036 NOMINAL CONFIDENCE THRESHOLD(%): (undef)
  [0][18] 002153 SATELLITE CHANNEL CENTRE FREQUENCY(Hz): 47966800000000
...

The same file dumped with bufr_dump from eccodes or with other emos tools gives values !=0 for B33007, while wreport gives sistematically zero for every occurrence of the descriptor in this and other files.

Replace /usr/bin/python shebang

From https://fedoraproject.org/wiki/Packaging:Python#Multiple_Python_Runtimes:

Packages in Fedora MUST NOT use /usr/bin/python. Instead packages for Python 3 MUST use /usr/bin/python3 (even if upstream supports both Python 2 and 3). As a result of that /usr/bin/python (as well as /usr/bin/env python and similar) MUST NOT be used in shebang lines or as a dependency of a package. As of Fedora 30, all uses of unversioned python executables in shebang lines will fail the build

Pacchettizzare con git-buildpackage

Pacchettizzare per debian usando git-buildpackage per allinearsi allo standard e in caso di bisogno poter avere dipendenze diverse per bullseye, sid, o jammy

Value out of range with MODES

Again an issue with this kind of aircraft report MODE_12.zip:

wrep MODE_12.bufr
MODE_12.bufr:0:Value 15 is outside the range [0,14] for 008009 (DETAILED PHASE OF FLIGHT)

Actually B08009 has 4 bit width and zero offset and scale, I guess from this comment in varinfo.cc

                int bit_max = exp2(bit_len) + bit_ref;
                // We subtract 2 because 2^bit_len-1 is the
                // BUFR missing value.
                // We cannot subtract 2 from the delayed replication
                // factors because RADAR BUFR messages have 255
                // subsets, and the delayed replication field is 8
                // bits, so 255 is the missing value, and if we
                // disallow it here we cannot import radars anymore.
                if (WR_VAR_X(code) != 31)
                    bit_max -= 2;

that this is a desired behavior and 15 should be a missing value. OTOH bufr_dump -p allegedly reports

detailedPhaseOfFlight={
      3, 3, 15, 3, 3, 3, 3, 3, 3, 3, 
      6, 3, 3, 3 }

so, first we should understand whether that is meant to be a missing value or not (but that's not @spanezz job) and, second, assuming it is really intended as a missing value, is there a way (environmental variable?) to avoid raising the error and code it as missing in (bufr2)netcdf?

No NEWS.md file in wreport

For future releases we should consider adding a NEWS.md file with the changes (as in arkimet and dballe)

build errors on Fedora 38

full log:
https://simc.arpae.it/moncic-ci/wreport/202305171212/master/fedora38/build.log

relevant bit:

[6/86] g++ -Iwreport/libwreport.so.3.0.3.p -Iwreport -I../wreport -I. -I.. -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Wpedantic -std=c++11 -Wextra -Wno-unused-parameter -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -fPIC -MD -MQ wreport/libwreport.so.3.0.3.p/utils_string.cc.o -MF wreport/libwreport.so.3.0.3.p/utils_string.cc.o.d -o wreport/libwreport.so.3.0.3.p/utils_string.cc.o -c ../wreport/utils/string.cc
FAILED: wreport/libwreport.so.3.0.3.p/utils_string.cc.o
g++ -Iwreport/libwreport.so.3.0.3.p -Iwreport -I../wreport -I. -I.. -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Wpedantic -std=c++11 -Wextra -Wno-unused-parameter -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -fPIC -MD -MQ wreport/libwreport.so.3.0.3.p/utils_string.cc.o -MF wreport/libwreport.so.3.0.3.p/utils_string.cc.o.d -o wreport/libwreport.so.3.0.3.p/utils_string.cc.o -c ../wreport/utils/string.cc
../wreport/utils/string.cc: In function ‘std::string wreport::str::encode_base64(const void*, size_t)’:
../wreport/utils/string.cc:449:11: error: ‘uint8_t’ does not name a type
  449 |     const uint8_t* str = (const uint8_t*)data;
      |           ^~~~~~~
../wreport/utils/string.cc:3:1: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
    2 | #include <vector>
  +++ |+#include <cstdint>
    3 |
../wreport/utils/string.cc:456:23: error: expected primary-expression before ‘[’ token
  456 |             enc = (str[i] << 16) | (str[i + 1] << 8) | str[i + 2];
      |                       ^
../wreport/utils/string.cc:456:40: error: expected primary-expression before ‘[’ token
  456 |             enc = (str[i] << 16) | (str[i + 1] << 8) | str[i + 2];
      |                                        ^
../wreport/utils/string.cc:456:59: error: expected primary-expression before ‘[’ token
  456 |             enc = (str[i] << 16) | (str[i + 1] << 8) | str[i + 2];
      |                                                           ^
../wreport/utils/string.cc:459:23: error: expected primary-expression before ‘[’ token
  459 |             enc = (str[i] << 16);
      |                       ^
../wreport/utils/string.cc:461:27: error: expected primary-expression before ‘[’ token
  461 |                 enc |= str[i + 1] << 8;
      |                           ^
../wreport/utils/string.cc:463:27: error: expected primary-expression before ‘[’ token
  463 |                 enc |= str[i + 2];
      |                           ^

do not compile on fedora 20 i686

make[2]: Entering directory `/root/rpmbuild/BUILD/wreport-3.4-1/src'
g++ -DHAVE_CONFIG_H -I. -I.. -DTABLE_DIR="/usr/share/wreport" -I.. -D_FILE_OFFSET_BITS=64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-
gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -std=gnu++11 -c -o options.o options.cc
g++ -DHAVE_CONFIG_H -I. -I.. -DTABLE_DIR="/usr/share/wreport" -I.. -D_FILE_OFFSET_BITS=64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -std=gnu++11 -c -o wrep.o wrep.cc
In file included from wrep.cc:20:0:
input.cc: In function 'void read_bufr_raw(const Options&, const char_, RawHandler&)':
input.cc:49:63: error: no matching function for call to 'wreport::BufrBulletin::read(FILE_&, std::string&, const char_&, long int_)'
while (BufrBulletin::read(in, raw_data, fname, &offset))
^
input.cc:49:63: note: candidate is:
In file included from input.cc:22:0,
from wrep.cc:20:
../wreport/bulletin.h:293:17: note: static bool wreport::BufrBulletin::read(FILE_, std::string&, const char_, off_t_)
static bool read(FILE_ in, std::string& buf, const char* fname=0, off_t* offset=0);
^
../wreport/bulletin.h:293:17: note: no known conversion for argument 4 from 'long int_' to 'off_t_ {aka long long int_}'
In file included from wrep.cc:20:0:
input.cc: In function 'void read_crex_raw(const Options&, const char_, RawHandler&)':
input.cc:92:63: error: no matching function for call to 'wreport::CrexBulletin::read(FILE_&, std::string&, const char_&, long int_)'
while (CrexBulletin::read(in, raw_data, fname, &offset))
^
input.cc:92:63: note: candidate is:
In file included from input.cc:22:0,
from wrep.cc:20:
../wreport/bulletin.h:440:17: note: static bool wreport::CrexBulletin::read(FILE_, std::string&, const char_, off_t_)
static bool read(FILE* in, std::string& buf, const char* fname=0, off_t* offset=0);
^
../wreport/bulletin.h:440:17: note: no known conversion for argument 4 from 'long int_' to 'off_t_ {aka long long int_}'
In file included from wrep.cc:21:0:
output.cc: In member function 'virtual void PrintTables::handle(wreport::Bulletin&)':
output.cc:82:89: warning: format '%zd' expects argument of type 'signed size_t', but argument 4 has type 'off_t {aka long long int}' [-Wformat=]
m->master_table_version_number, m->master_table_version_number_local);
^
output.cc:93:94: warning: format '%zd' expects argument of type 'signed size_t', but argument 4 has type 'off_t {aka long long int}' [-Wformat=]
m->master_table_number, m->edition_number, m->master_table_version_number);
^
output.cc:98:46: warning: format '%zd' expects argument of type 'signed size_t', but argument 4 has type 'off_t {aka long long int}' [-Wformat=]
b.fname.c_str(), b.offset);
^
output.cc: In member function 'virtual void PrintFeatures::handle(wreport::Bulletin&)':
output.cc:113:58: warning: format '%zd' expects argument of type 'signed size_t', but argument 4 has type 'off_t {aka long long int}' [-Wformat=]
fprintf(out, "%s:%zd:", b.fname.c_str(), b.offset);
^
make[2]: *_* [wrep.o] Error 1

Impossible to decode bufr

The attached file wigos.zip containing bufr messages cannot be decoded with wrep, for each message I get "end of buffer while looking for 4 bits of bit-packed data". I cannot swear it is coded correctly, but I know that somebody uses these data with other libraries.
Could you give it a check?
wrep -t or wrep -D show something reasonable before crashing.

Test failures

Non sono sicuro che fedora 20 debba essere supportata, ma a 32 bit compare questo errore nei test, cosa che potrebbe segnalare comunque un problema.

Running /root/rpmbuild/BUILD/wreport-3.12-1/wreport/test-wreport
options: .
error: ...........
conv: ...
tableinfo: ..
varinfo: ......
vartable: ....
var: ................
opcode: .
dtable: ..
tables: .
internals_fs: ...
tabledir: ......
subset: .
bulletin: .
bufr_input: .
bufr_decoder: ...................................................✘...
bufr_encoder: ...
crex_decoder: ........
buffers_bufr: .
buffers_crex: .
bulletin_associated_fields: .
bulletin_bitmaps: .
bulletin_interpreter: .
bulletin_internals: .
bulletin_dds_validator: ..
tests: .
lua: .

  • Test failures

bufr_decoder.bufr/afl-src01flip1-pos10.bufr: 'killing signal catched' does not contain 'looking for data descriptor list'
bufr/decoder-test.cc:36:actual(e.what()).contains(errmsg)

  • Result summary

1/135 tests failed
134 tests succeeded
Failed with result 1
make[2]: *** [check-local] Error 1
make[2]: Leaving directory /root/rpmbuild/BUILD/wreport-3.12-1/wreport' make[1]: *** [check-am] Error 2 make[1]: Leaving directory /root/rpmbuild/BUILD/wreport-3.12-1/wreport'
make: *** [check-recursive] Error 1
errore: Stato d'uscita errato da /var/tmp/rpm-tmp.3K1Jba (%check)

deploy fails for centos:8 due to undefined encrypted_6d2e60986cdb_iv

Hi,

I am running the build and test as part of the Ubuntu distribution for ppc64le.
This helps us simplify testing later when distributions are re-building and re-releasing.

During the build and test, I found that build for centos:8 fails during deploy stage with error as below

$ openssl aes-256-cbc -K $encrypted_6d2e60986cdb_key -iv $encrypted_6d2e60986cdb_iv -in .copr.enc -out .copr -d
The command "openssl aes-256-cbc -K $encrypted_6d2e60986cdb_key -iv $encrypted_6d2e60986cdb_iv -in .copr.enc -out .copr -d" failed and exited with 1 during .
Your build has been stopped.
iv undefined

I think it is looking for something to be defined in encrypted_6d2e60986cdb_iv.

Could you pl. take a look into the same.

Wrong decoding of MODE S message

The attached bufr message mode-s.zip, presumably correct, a "MODE S" (new kind of aircraft report), shows decoding errors with wrep:

Value 59776.5 is outside the range [0,409.4] for B11002 (WIND SPEED)

Other messages of the same kind show errors on different descriptors, e.g.

Value -30706 is outside the range [0,62] for 002170 (AIRCRAFT HUMIDITY SENSORS)
When decoding compressed BUFR data, the difference bit length must be 0 (and not 4 like in this case) when the base value is missing
Value 212308 is outside the range [0,2] for 002064 (AIRCRAFT ROLL ANGLE QUALITY)
etc.

I can attach other examples, but I think the source of error is the same. Are they using some non managed modifiers?

N.B. Decoding requires the new bufr tables just committed.

Fix Travis build on Fedora >= 29

Fedora >= 29 docker images come now with coreutils-single package that conflicts with coreutils (installed by @buildsys-build).

gcc 4.8.3 not supported

When compiling with gcc 4.8.3:

make[2]: Entering directory `/autofs/nethomes/edigiacomo/src/wreport/wreport'
...
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -DTABLE_DIR=\"/usr/local/share/wreport\" -I.. -g -O2 -std=gnu++11 -Wall -MT utils/tests.lo -MD -MP -MF utils/.deps/tests.Tpo -c utils/tests.cc  -fPIC -DPIC -o utils/.libs/tests.o
In file included from utils/tests.cc:9:0:
utils/tests.h: In member function 'void wreport::tests::TestCase::add_method(const string&, FUNC, Args&& ...)':
utils/tests.h:701:56: error: expected ',' before '...' token
         methods.emplace_back(name, [test_function, args...]() { test_function(args...); });
                                                        ^
utils/tests.h:701:56: error: expected identifier before '...' token
utils/tests.h:701:59: error: parameter packs not expanded with '...':
         methods.emplace_back(name, [test_function, args...]() { test_function(args...); });
                                                           ^
utils/tests.h:701:59: note:         'args'
utils/tests.h: In lambda function:
utils/tests.h:701:83: error: expansion pattern 'args' contains no argument packs
         methods.emplace_back(name, [test_function, args...]() { test_function(args...); });
...

Maybe is related to gcc bug #41933?

documentation (managing btables)

I just added a variable (in 80f8b07) and I suffered a bit for lack of basic documentation, like where to find a description of the file fields (talking with @edigiacomo the order of fields seems: "B code - description - unit - scale - offset - byte lenght"), the basic hierarchy of files, things like that.

It might be a case of RTFM but I'm unable to find the FM

Wrong year in bufr section 1

Il problema è comparso in bufr2netcdf ma risale a wreport. In allegato un file con 2 messaggi bufr, wind profiler, presumibilmente corretti. Se li vado a decodificare, la sezione 1 in un caso ha l'anno corretto, nell'altro caso un anno assurdo 7120:

wrep B002_2020012300.100.bufr
BUFR 2:255:96 98:0 2020-01-22 22:15:00 0 1 subsets
...
wrep B002_2020012300.100.bufr
BUFR 2:255:96 98:0 7120-01-22 23:30:00 0 1 subsets
...

L'anno codificato successivamente nella sezione dati con B04001 è giusto. Analizzando con bufr_dump -O vedo che nella sezione 1 c'è solo l'anno del secolo ma non il secolo, possibile? come viene calcolato il secolo in questo caso? Da cosa dipende? Le due sezioni 1 differiscono solo per il byte di padding e poco altro.

wprof.zip

Drop autotools

Once #39 is done, we can remove autotools support to simplify packaging, and leave only one (easier) build infrastructure to maintain

Missing table?

The attached file unpars.zip contains 2 of many bufr messages recently retrieved form ECMWF MARS archive (SYNOP(?) BUFR), which cannot be parsed by wreport, with the error

variable 001101 not found in table /usr/share/wreport/B000000000981301.txt

Do we need to update tables, are they inconsistently coded, or...?

Error on compressed strings

The attachment GPS_bufr.zip contains two almost equal GPS bufr messages, with a different coding of station name. The decoding of GPSR_fail.bufr fails with the error message compressed strings with 20 bit deltas have non-zero reference value. According to the source code, this seems to be due to an unimplemented feature.

Is the interpretation correct? Could that feature be implemented? Or any workaround?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.