Hi, first cool project!

I think there is a bug in computing the padding for some messages:

I have a messgae format in DBC like this:

BO_ 1 MSG0: 8 E1
 SG_ A : 6|15@0+ (1,0) [0|32768] ""  E1
 SG_ B : 7|1@0+ (1,0) [0|1] ""  E1
 SG_ C : 38|15@0+ (1,0) [0|32768] ""  E1
 SG_ D : 39|1@0+ (1,0) [0|1] ""  E1

It results in a fmt string with more than 64 bit due to too large padding like this:
p24u2u15p30u1u15

I don't know the exact syntax of the DBC file, but the definition of start in not intuitive.
The Vector DBC editor reports as startbits:
D->39
C->40
B->7
A->8

I think the logic in this function of start + length may not be right. May be even the start bit coputed by your code before is wrong.

def _create_message_encode_decode_format(signals):
    fmt = ''
    end = 64
    for signal in signals:
        padding = end - (signal.start + [signal.length)

Let me know it this it the right way to report the issue, if you plan to fix it or would like to have more information.

Otherwise, I see the following error:
default_send_type = database.attribute_definition_defaults['GenMsgSendType']
TypeError: list indices must be integers, not str

Hi,

I tried to decode my .dbc file using this tool and when I loaded the .dbc file, I got this following error

cantools.db.UnsupportedDatabaseFormatError: DBC: "Invalid DBC syntax at line 2724, column 33: 'BA_DEF_DEF_ "SamplePointMax" 81>!<.5;': Expected ";"."

After spending a little bit of time to find the error, it turned out, that this error is related to the following signal which has FLOAT data type
BA_DEF_ "SamplePointMax" FLOAT 0 100;

I really appreciate your help.

Thanks in advance.

Hi there,
I am trying to save a self generated can data base as kcd but the method "as_kcd_string()" is not returning a string as the method "as_dbc_string()" correctly does.

Instead is returning a '<cantool.db.database.Database object at 0x0.....>'

Code:

I was trying it once with a dbc file

db = cantools.db.File()
filename = r"...\tests_cantools\files\example.dbc"
db.add_dbc_file(filename)

and a second time with a kcd file

db = cantools.db.File()
filename = r"...\tests_cantools\files\example.kcd"
db.add_kcd_file(filename)

signals = [cantools.db.Signal(name='POPOTO_SIG',  start=0, length=4, nodes=['foo'], byte_order='big_endian', is_signed=False, scale=1.0, offset=10, minimum=10.0, maximum=100.0, unit='m/s', choices=None, comment=None)]

message = cantools.db.Message(frame_id=37, name='PEPETO_message',  length=8,  nodes=['bar'],  signals=signals, comment='')

db.add_message(message)

texto = db.as_kcd_string()  # this one returns an object
texto1 = db.as_dbc_string()  # this one works correctly

In both cases "as_kcd_string()" returns an oject

When trying to use:
candump can0 | cantools decode --single-line ../j1939.dbc

I get the following error:
'utf-8' codec can't decode byte 0xb0 in position 4832: invalid start byte

I've tried decoding an encoded message in both Cantools 28.6.0 and 22.0.1, and I'm seeing the same behavior where one signal's value is lost somehow.

>>> msg.decode(msg.encode({ 'signal1': -250.0, 'signal2': -250.0, 'signal3': 1, 'signal4': 2, 'signal5': 3, 'signal6': 4, 'signal7': 0 }))
{'signal1': -250.0, 'signal2': -250.0, 'signal3': 1, 'signal4': 0, 'signal5': 3, 'signal6': 4, 'signal7': 0}

I noticed that https://github.com/eerimoq/cantools/blob/master/tests/test_database.py contains quite a few instances of checking whether the decoded message matches the original message passed through something equivalent to msg.decode(msg.encode(msg_payload)), so I'm unsure where the issue lies. All of my signals in this CAN message are big-endian, and I also noticed this weird inconsistency when printing the start and length values for my signals in this message:

>>> sorted([(sig.start, sig.length) for sig in msg.signals], key=lambda sig: sig[0])
[(6, 15), (22, 15), (35, 4), (39, 4), (41, 2), (55, 8), (59, 3)]

Some of the signals seem to overlap for some reason. Am I misunderstanding how the signals are laid out in this particular message?

I took a look at the message dump, which looks fine however: https://pastebin.com/0UufZbuj

Please let me know if more information is necessary.

Hi,
I suggest to support a parameter to disable the decoding of signal choices (e.g. enumerated signal values such as 'ON', 'OFF') and to decode the integer value instead.
This can be useful for post processing.

Can't use cantools with versions after 22.0.0 it raise exception when tries open any dbc file if it has a message VECTOR__INDEPENDENT_SIG_MSG. Due to DLC = 0. Following exception with package 28.8.0:

  File "build/bdist.linux-x86_64/egg/cantools/database/__init__.py", line 112, in load_file
  File "build/bdist.linux-x86_64/egg/cantools/database/__init__.py", line 149, in load
  File "build/bdist.linux-x86_64/egg/cantools/database/__init__.py", line 204, in load_string
  File "build/bdist.linux-x86_64/egg/cantools/database/__init__.py", line 194, in load_can_database
  File "build/bdist.linux-x86_64/egg/cantools/database/can/database.py", line 130, in add_dbc_string
  File "build/bdist.linux-x86_64/egg/cantools/database/can/formats/dbc.py", line 1101, in load_string
  File "build/bdist.linux-x86_64/egg/cantools/database/can/formats/dbc.py", line 961, in _load_messages
  File "build/bdist.linux-x86_64/egg/cantools/database/can/message.py", line 59, in __init__
  File "build/bdist.linux-x86_64/egg/cantools/database/can/message.py", line 838, in refresh
  File "build/bdist.linux-x86_64/egg/cantools/database/can/message.py", line 818, in _check_signal_tree
  File "build/bdist.linux-x86_64/egg/cantools/database/can/message.py", line 782, in _check_signal
cantools.database.errors.Error: The signal NumberOfMembersInWorkingSet does not fit in message VECTOR__INDEPENDENT_SIG_MSG.

So I have to limit package version into 22 where such files processed without errors.

I have many dbc files with same message, but different signals. For example the message from my j1939.dbc:

BO_ 3221225472 VECTOR__INDEPENDENT_SIG_MSG: 0 Vector__XXX
 SG_ NumberOfMembersInWorkingSet : 0|8@1+ (1,0) [0|0] "" Vector__XXX
 SG_ SourceAddress : 0|8@1+ (1,0) [0|0] "" Vector__XXX
 SG_ NncntnuslyMonitoredSystemsStatus : 0|16@1+ (1,0) [0|0] "" Vector__XXX
 SG_ NncntnslyMonitoredSystemsSupport : 0|16@1+ (1,0) [0|0] "" Vector__XXX
 SG_ CntnslyMntrdSystmsSupport_Status : 0|8@1+ (1,0) [0|0] "" Vector__XXX
 SG_ AETCTorqueValue : 0|16@1+ (1,0) [0|64255] "Nm" Vector__XXX
 SG_ TrailerWeight : 0|16@1+ (2,0) [0|128510] "kg" Vector__XXX
 SG_ ReferenceTirePressSetting : 0|8@1+ (8,0) [0|2000] "kPa" Vector__XXX
 SG_ ReferenceTirePress : 0|8@1+ (8,0) [0|2000] "kPa" Vector__XXX
 SG_ TirePressThresholdDetection : 0|3@1+ (1,0) [0|7] "" Vector__XXX
 SG_ TireAirLeakageRate : 0|16@1+ (0.1,0) [0|6425.5] "Pa/s" Vector__XXX
 SG_ CTIWheelEndElectricalFault : 0|2@1+ (1,0) [0|3] "" Vector__XXX
 SG_ CTITireStatus : 0|2@1+ (1,0) [0|3] "" Vector__XXX
 SG_ CTIWheelSensorStatus : 0|2@1+ (1,0) [0|3] "" Vector__XXX
 SG_ TireTemp : 0|16@1+ (0.03125,-273) [-273|1734.96875] "deg C" Vector__XXX
 SG_ TirePress : 0|8@1+ (4,0) [0|1000] "kPa" Vector__XXX
 SG_ EngTurningGearEngaged : 0|2@1+ (1,0) [0|3] "" Vector__XXX
 SG_ AETCSpeedValue : 0|16@1+ (0.125,0) [0|8031.875] "rpm" Vector__XXX

If open such file with editor CANdb++ I can see those signals but not this message. It seems like it's ignoring such message.

Not sure that using strict=False with load_file() it's a correct way to avoid such exception. Because, it will turn off checks for any others possible errors, too.

The dump subcommand currently does not display the choice values. It would be nice to support this.

I think it makes sense to add this to the dump subcommand, as opposed to adding a new subcommand.

Maybe we can do something like:

================================= Messages =================================

  ------------------------------------------------------------------------

  Name:       DRIVER_HEARTBEAT
  Id:         0x64
  Length:     1 bytes
  Cycle time: 1000 ms
  Senders:    DRIVER
  Layout:

                          Bit

             7   6   5   4   3   2   1   0
           +---+---+---+---+---+---+---+---+
     B   0 |<-----------------------------x|
     y     +---+---+---+---+---+---+---+---+
     t       +-- DRIVER_HEARTBEAT_cmd
     e

  Signal tree:

    -- {root}
       +-- DRIVER_HEARTBEAT_cmd

  Signal choices:

    DRIVER_HEARTBEAT_cmd
        0 DRIVER_HEARTBEAT_cmd_NOOP
        1 DRIVER_HEARTBEAT_cmd_SYNC
        2 DRIVER_HEARTBEAT_cmd_REBOOT

  ------------------------------------------------------------------------

Hello guys,

I'm trying to open and parse CDD but I get this error
File "C:\Users\gaia.littarru\AppData\Local\Programs\Python\Python37\lib\site-packages\cantools\database_init_.py", line 228, in load_string
raise UnsupportedDatabaseFormatError(e_dbc, e_kcd, e_sym, e_cdd)
cantools.database.UnsupportedDatabaseFormatError: CDD: "invalid literal for int() with base 10: '2,0,0'"

It seems there is a problem with data type.
Have you already noticed this issue?

Thanks for the help,

Gaia

Hi,
I like cantools a lot, but I got a problem when I want to use it with my .dbc file.
In this dbc files I got signal names longer than 32 characters. Vector allowes signal names longer than 32 characters. But these longer names are only accesible through a special attribute.
For example:
When i want to look for this signal name:
"FAILURE_ZELL_UEBERSPANNUNG_WARNUNG_IDX_01"
I cant find it with cantools.
The signal name is stored as:
"FAILURE_ZELL_UEBERSPANNUNG__0024"
This is because of the 32 character limitation.

I think when you interpret the .dbc file with cantools it only looks at the attribute SG_

But when you got longer signal names the full name ist stored in BA_ "SystemSignalLongSymbol"

Is it possible to fix this?

The generated C code decode-function isn't accomplishing the offset and scaling calculation. I allways have to do this by myself.

hello,

I am using a Peak USB2CAN device to read in information from a force sensor. I am using Ubuntu 16.04, is it possible to get the can messages directly into a python script (instead of using candump in command line and add decode there) and decode it and have engineering units.
I have a *.dbc file already and am able to decode in the command line tool already

Thank you in advance

Hi.

I just noticed that cantools will default to UTF-8 if the user does not pass the encoding argument, regardless of the database type (e.g. 'dbc', detected or passed by the user). I feel that this inconvenient since one could safely assume the encoding to be CP1252 for dbc files. Other encodings for the dbc file format are the exception.

I would suggest database file type specific defaults. If that is not an option, adding the encoding argument to the dbc reading related examples in the docs would present a less intrusive approach to take that off the users plate.

What do you think?

Could you please return a DTC list for cdd. Thanks!

I am instantiating a Database with an attribute definition like so:

Database(messages, nodes, buses, version,
                           attribute_definitions=[
                                                      ('BA_DEF_',
                                                       'BO_',
                                                       'GenMsgCycleTime',
                                                       'INT',
                                                       [ '0',
                                                         '1000000'
                                                       ],
                                                      )
                                                 ]
)

And one of the messages has a non-null cycle_time attribute set, which does show in the DBC. However, the attribute definition itself does not. I can't seem to figure out why.

I did notice that the _dump_attribute_definitions() function in formats/dbc.py seems to expect attribute definitions in a different format than they show when going FROM a dbc TO a Database object; see below:

>>> db = cantools.db.load_file("mydbc.dbc")
>>> print db.__dict__
...
 '_attribute_definitions': [(['BA_DEF_', 'BO_', 'GenMsgCycleTime', 'INT', (['0', '1000000'], {})], {})],
...

As you can see there is a lot more nesting going on than what I passed to the constructor -- the simpler format seems to be what _dump_attribute_definitions expects.

This is great tool and great work. Thank you.

I have a code which was working with 23.0.0 but is failing after that to add_message to database. I do not see any change in add_message function in database.py. What could be causing the problem? Basically what I am doing is iterating though messages and adding specific messages to new database. I am getting a message with its name but failing to add it to new database. It was working fine before and generating new database.

Hello Erik,

what do you think about adding arxml format to the supported database formats?

https://automotive.wiki/index.php/AUTOSAR_XML_Schema

Message.nodes returns a string not a list as the documentation suggests

With package version 28.8.0 when trying loading dbc file having the overlapped signals message:

db = cantools.db.load_file(db_path, strict=False)

It produces exception:

  File "build/bdist.linux-x86_64/egg/cantools/database/__init__.py", line 112, in load_file
  File "build/bdist.linux-x86_64/egg/cantools/database/__init__.py", line 149, in load
  File "build/bdist.linux-x86_64/egg/cantools/database/__init__.py", line 204, in load_string
  File "build/bdist.linux-x86_64/egg/cantools/database/__init__.py", line 194, in load_can_database
  File "build/bdist.linux-x86_64/egg/cantools/database/can/database.py", line 130, in add_dbc_string
  File "build/bdist.linux-x86_64/egg/cantools/database/can/formats/dbc.py", line 1102, in load_string
  File "build/bdist.linux-x86_64/egg/cantools/database/can/formats/dbc.py", line 962, in _load_messages
  File "build/bdist.linux-x86_64/egg/cantools/database/can/message.py", line 59, in __init__
  File "build/bdist.linux-x86_64/egg/cantools/database/can/message.py", line 830, in refresh
  File "build/bdist.linux-x86_64/egg/cantools/database/can/message.py", line 112, in _create_codec
  File "build/bdist.linux-x86_64/egg/cantools/database/utils.py", line 200, in create_encode_decode_formats
  File "build/bdist.linux-x86_64/egg/cantools/database/utils.py", line 194, in create_little
  File "python2.7/site-packages/bitstruct-4.0.0-py2.7.egg/bitstruct.py", line 462, in pack
    return CompiledFormat(fmt).pack(*args)
  File "python2.7/site-packages/bitstruct-4.0.0-py2.7.egg/bitstruct.py", line 300, in pack
    bits = self._pack_value(info, args[i], bits)
  File "python2.7/site-packages/bitstruct-4.0.0-py2.7.egg/bitstruct.py", line 257, in _pack_value
    value_bits = info.pack(value)
  File "python2.7/site-packages/bitstruct-4.0.0-py2.7.egg/bitstruct.py", line 71, in pack
    arg))
bitstruct.Error: "u64" requires 0 <= integer <= 18446744073709551615 (got 4703919738795935662272)

After investigation I found the message leading to failing:

BO_ 2566687742 AFT1PSI2: 8 Vector__XXX
 SG_ HtrRes : 40|16@1+ (0.1,0) [0|6425.5] "ohm" Vector__XXX
 SG_ MaxRes : 32|16@1+ (250,0) [0|62500] "kohm" Vector__XXX
 SG_ Temp : 16|16@1+ (0.03125,-273) [-273|1734.96875] "�C" Vector__XXX
 SG_ RegenFailedCount : 8|8@1+ (1,0) [0|250] "" Vector__XXX
 SG_ PwrSupply : 4|2@1+ (1,0) [0|3] "" Vector__XXX
 SG_ DetectionStatus : 0|4@1+ (1,0) [0|15] "" Vector__XXX

By the way, version 22.0.0 open such dbc file without any error.

In my projects, I typically, at a bare minimum, enable -Wall -Werror -Wextra -Wpedantic. I suspect I'm not the only person who does so, and GCC/Clang has a bunch of additional useful flags as well.

Perhaps it would be good to work towards enabling these additional flags in the tests, so that it is easy for other people to include the generated C code in their projects (ie. make the code compile under the most stringent options, so that integrating will always have either equal or less-stringent options)? Currently the test files are only compiled with -Wall -Werror -Wpedantic.

One way to do this is starting with enabling all these flags and ignoring warnings (via -Wno-* flags) that the generated C code doesn't comply against so that CI is passing. Then ignored warnings can be gradually dropped as the code is made compliant, so that it doesn't have to happen all at once, since I suspect that this might be a pretty significant effort and it may make more sense to do this incrementally.

Thoughts?

Edit: Just for kicks, I enabled it on a temporary branch and ran Travis on it, and it seems like there's only really 2 classes of failures that need fixing if only -Wextra is added. So it might not be as involved as I initially thought... However, I can merge these Makefile changes if this plan of attack seems reasonable?

I created a unit test to demonstrate that KCD labels (enums) aren't always being matched against raw value, as it is specified in the KCD schema.
https://github.com/maxchill/cantools/tree/bugfix/enum_scaling

I included a commented out potential fix, however I'm not sure what the expected DBC enum matching behavior is (scaled vs raw). I tried searching around for details but couldn't find any hard answers. Short of testing out Vector tools behavior, or comparing with other code bases I'm not sure if my proposed change fixes, or breaks DBC decoding.

Here is an excerpt from KCD schema

<xs:element name="Label">
    <xs:annotation>
      <xs:documentation>Descriptive name for a single value e.g. to describe an enumeration, mark special,invalid or error values.</xs:documentation>
    </xs:annotation>
    <xs:complexType>
      <xs:complexContent>
        <xs:extension base="BasicLabelType">
          <xs:attribute name="value" type="xs:nonNegativeInteger">
            <xs:annotation>
              <xs:documentation>Signal raw value that is described here.</xs:documentation>
            </xs:annotation>
          </xs:attribute>
        </xs:extension>
      </xs:complexContent>
    </xs:complexType>
  </xs:element>

  <xs:element name="LabelGroup">
    <xs:annotation>
      <xs:documentation>Descriptive name for a sequence of adjacent values.</xs:documentation>
    </xs:annotation>
    <xs:complexType>
      <xs:complexContent>
        <xs:extension base="BasicLabelType">
          <xs:attribute name="from" use="required" type="xs:nonNegativeInteger">
            <xs:annotation>
              <xs:documentation>Signal raw value the label group is starting with.</xs:documentation>
            </xs:annotation>
          </xs:attribute>
          <xs:attribute name="to" use="required" type="xs:nonNegativeInteger">
            <xs:annotation>
              <xs:documentation>Signal raw value the label group is ending with.</xs:documentation>
            </xs:annotation>
          </xs:attribute>
        </xs:extension>
      </xs:complexContent>
    </xs:complexType>
  </xs:element>

It's a small nit, but I was trying to compile the generated files with clang's -Wdocumentation and noticed that the Doxygen comments for the message structs isn't actually valid Doxygen.

In file included from tests/main.c:6:
tests/files/c_source/motohawk.h:50:5: error: '@param' command used in a comment
      that is not attached to a function declaration [-Werror,-Wdocumentation]
 * @param enable Value as on the CAN bus.
    ^~~~~
tests/files/c_source/motohawk.h:54:5: error: '@param' command used in a comment
      that is not attached to a function declaration [-Werror,-Wdocumentation]
 * @param average_radius Value as on the CAN bus.
    ^~~~~
tests/files/c_source/motohawk.h:58:5: error: '@param' command used in a comment
      that is not attached to a function declaration [-Werror,-Wdocumentation]
 * @param temperature Value as on the CAN bus.
    ^~~~~

I think this is because @param isn't actually meant to be put on struct attributes.

This StackOverflow question gives a workaround, if you want to have the comment outside the struct, the way to do this is something like

/** @struct motohawk_example_message_t
 *  @brief Signals in message ExampleMessage.
 *  @var motohawk_example_message_t::enable
 *  Value as on the CAN bus.
 *            Range: -
 *            Scale: 1
 *            Offset: 0
 *  @var motohawk_example_message_t::average_radius
 *  Value as on the CAN bus.
 *            Range: 0..50 (0..5 m)
 *            Scale: 0.1
 *            Offset: 0
 *  @var motohawk_example_message_t::temperature
 *  Value as on the CAN bus.
 *            Range: -2048..2047 (229.52..270.47 degK)
 *            Scale: 0.01
 *            Offset: 250
 */

Alternatively, this can be placed inside the struct above each member, if you want to fix this.

The newest version of a cantools (4.0.0) crashes while parsing dbc on string:
BA_ "BusType" "CAN";

Error log:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/cantools/db.py", line 547, in add_dbc
    tokens = self._grammar.parseString(dbc.read())
  File "/usr/local/lib/python3.5/dist-packages/pyparsing.py", line 1632, in parseString
    raise exc
  File "/usr/local/lib/python3.5/dist-packages/pyparsing.py", line 1622, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/usr/local/lib/python3.5/dist-packages/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/lib/python3.5/dist-packages/pyparsing.py", line 3395, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/usr/local/lib/python3.5/dist-packages/pyparsing.py", line 1383, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/lib/python3.5/dist-packages/pyparsing.py", line 3183, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pyparsing.ParseException: Expected end of text (at char 10209), (line:218, col:1)

File fragment:

BA_DEF_DEF_  "BusType" "CAN";
BA_ "BusType" "CAN";
BA_ "GenMsgCycleTime" BO_ 13 1;

Old, 1.0.0, works as expected.

In DBC files when a signal has one receiver which Vector__XXX, that means this signal is received by all nodes.

Is it possible that signal.receivers return the complete list of node names present on the same bus instead of Vector__XXX?

ax

I am using the version 16.2.0. When I try to parse a dbc file I get the following error message:

AttributeError: no such attribute name

I can reproduce the error with the following line of code:

import cantools
cantools.db.formats.dbc.load_string("")

The complete backtrace is:

> Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:09:15)
> Type "copyright", "credits" or "license" for more information.
> 
> IPython 5.3.0 -- An enhanced Interactive Python.
> ?         -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help      -> Python's own help system.
> object?   -> Details about 'object', use 'object??' for extra details.
> 
> In [1]: import cantools
> In [2]: cantools.db.formats.dbc.load_string("")
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> <ipython-input-2-c260aaeb5b25> in <module>()
> ----> 1 cantools.db.formats.dbc.load_string("")
> 
> /lib/python2.7/site-packages/cantools/db/formats/dbc.pyc in load_string(string)
>     867     """
>     868
> --> 869     grammar = _create_grammar()
>     870
>     871     try:
> 
> /lib/python2.7/site-packages/cantools/db/formats/dbc.pyc in _create_grammar()
>     273                                                              - Suppress('-')
>     274                                                              - Suppress(positive_integer)))
> --> 275                                        - scolon)
>     276     signal_multiplexer_values.setName(SIGNAL_MULTIPLEXER_VALUES)
>     277
> 
> /lib/python2.7/site-packages/pyparsing.pyc in delimitedList(expr, delim, combine)
>    3129        as a list of tokens, with the delimiters suppressed.
>    3130     """
> -> 3131     dlName = _ustr(expr)+" ["+_ustr(delim)+" "+_ustr(expr)+"]..."
>    3132     if combine:
>    3133         return Combine( expr + ZeroOrMore( delim + expr ) ).setName(dlName)
> 
> /lib/python2.7/site-packages/pyparsing.pyc in _ustr(obj)
>     120             # If this works, then _ustr(obj) has the same behaviour as str(obj), so
>     121             # it won't break any existing code.
> --> 122             return str(obj)
>     123
>     124         except UnicodeEncodeError:
> 
> /lib/python2.7/site-packages/pyparsing.pyc in __str__(self)
>    2388
>    2389         if self.strRepr is None:
> -> 2390             self.strRepr = "{" + " ".join( [ _ustr(e) for e in self.exprs ] ) + "}"
>    2391
>    2392         return self.strRepr
> 
> /lib/python2.7/site-packages/pyparsing.pyc in _ustr(obj)
>     120             # If this works, then _ustr(obj) has the same behaviour as str(obj), so
>     121             # it won't break any existing code.
> --> 122             return str(obj)
>     123
>     124         except UnicodeEncodeError:
> 
> /lib/python2.7/site-packages/pyparsing.pyc in __str__(self)
>    2388
>    2389         if self.strRepr is None:
> -> 2390             self.strRepr = "{" + " ".join( [ _ustr(e) for e in self.exprs ] ) + "}"
>    2391
>    2392         return self.strRepr
> 
> /lib/python2.7/site-packages/pyparsing.pyc in _ustr(obj)
>     120             # If this works, then _ustr(obj) has the same behaviour as str(obj), so
>     121             # it won't break any existing code.
> --> 122             return str(obj)
>     123
>     124         except UnicodeEncodeError:
> 
> /lib/python2.7/site-packages/pyparsing.pyc in __str__(self)
>    1379
>    1380     def __str__( self ):
> -> 1381         return self.name
>    1382
>    1383     def __repr__( self ):
> 
> /lib/python2.7/site-packages/pyparsing.pyc in __getattr__(self, aname)
>    1422             return ret;
>    1423         else:
> -> 1424             raise AttributeError("no such attribute " + aname)
>    1425
>    1426     def __eq__(self,other):
> 
> AttributeError: no such attribute name
> 

linting_output.txt
Hi.

Great to see cantools progressing with promising features like the c source generator. I cannot contribute myself right now but since I have access to a linter with MISRA 2012 rules set up, I thought I might at least provide its feedback to you.

I generated from vehicle.dbc in 29.5.0 and linted using PC Lint.

The output is as redundant as you would expect, but still, there are a few pieces of interest in here.

If you are not interested right now, you can just close the issue again.

Best regards.

While parsing a few new .dbc-files, I got stuck with cantools when reaching lines with the commands:

• "SIG_VALTYPE_"
• "BO_TX_BU_"
• "BA_DEF_REL_"
  • "BU_SG_REL_"
• "BA_DEF_DEF_REL_"
• "BA_REL_"
  • "BU_SG_REL_"

just wanted to let you know

SIG_VALTYPE sets the type of the value of a signal in a message - example:
SIG_VALTYPE_ 1649 dCANxxTime : 2;
Sets the datatype in message (0x671), for signal (dCANxxTime) to "2" (IEEE Double)

Tries to read multiple non conflicting DBC files into the same database object.
Guess this should be possible?

`dbfiles = ["file1.dbc", "file2.dbc"]

db = cantools.db.load_file(dbfiles[0])

for file in dbfiles[1:]:
db.add_dbc_file(file)`

Hi,
I have a dbc file which defined some signals as 32bit IEEE float. I think in DBC file this is done by an extra line, but I'm unsure.

SIG_VALTYPE <can-id> <signame> : 1;

With cantools 9.1.1 I got parsing errors.
With cantools 14.4.0 the dbc is accepted but the decoded signals looks like to be interpreted as INT32 and not as float.

Thank you for any hints.

After updating up to 22 version, it was lose ability to load DBC files with extended frames.

import cantools
db = cantools.db.load_file(some_extended_db)

If frame ID has high bit is on, I get assert error on such frame:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/cantools/db/__init__.py", line 67, in load_file
  File "build/bdist.linux-x86_64/egg/cantools/db/__init__.py", line 89, in load
  File "build/bdist.linux-x86_64/egg/cantools/db/__init__.py", line 123, in load_string
  File "build/bdist.linux-x86_64/egg/cantools/db/database.py", line 123, in add_dbc_string
  File "build/bdist.linux-x86_64/egg/cantools/db/formats/dbc.py", line 1139, in load_string
  File "build/bdist.linux-x86_64/egg/cantools/db/formats/dbc.py", line 702, in _load_attributes
KeyError: 3221225472

But, if I remove only high bit from frame id, the same dbc successfully loaded.

Sample of code with the issue:

db = cantools.db.Database()
db.add_dbc_file('some.dbc')
ext_msg = db.get_message_by_name('SomeExtendedMessage')

if ext_msg.is_extended_frame:
    db_out = cantools.db.Database()
    db_out.add_message(ext_msg)
    with open('result.dbc', 'w') as dbc_result:
        dbc_result.write(db_out.as_dbc_string())

Check frame_id in the result output and see that 0x80000000 bit is absent.
And if open result.dbc with your library it also show the is_extended_frame flag is False:

db = cantools.db.Database()
db.add_dbc_file('result.dbc')
for m in db.messages:
	print m

Hello,

congratulations for cantools and bitstruct packages!

I'm using them in my package asammdf. A few days ago I made my package available to anaconda users on the conda-forge channel. Do you think you can also provide your packages on conda-forge? It's not so hard since they are pure-python.

hey do you know how to use regular expression to simplify your project with one fifth or even one tenth of your code, and ten times of readability?
https://github.com/AlvinYee/GodLoveMe/blob/AlvinYee-patch-2/dbcParser/dbcParser

Value tables are currently omitted when a db is dumped to dbc format. Include them. Further a proper access interface to these value tables (e.g. in the dbc specifics) might make sense.

Hi,

I'm working on CAN projects and I'm currently using the cantools lib to parse my DBC files and it's work great! (I'm using it as a complement to python-can lib ) .

I'm wondering if you plan to integrate parsers for CDD (CANdelaStudio diagnostic description) files and maybe a more open format like ODX (Open diagnostic data exchange) also?

I could give you some help about the CDD file format.

ax

Heya,

Great project! I've been using it to support dbc loading for an ncurses based can monitoring tool I've written. However we have a situation where we have multiple dbc files for different components within the network as a whole (yes there are inevitably conflicts...!). This means that when monitoring the network I have to load multiple dbc files. Has this use case been given any consideration as yet?

There are two possible approaches I see are :

1. Merging dbcs ; I was originally instantiating one database object and was just calling add_dbc_file() multiple times. This used to appear to work in the past (on the surface at least!) but has now been hobbled by recent changes to database.py where refresh clears _frame_id_to_message every time add_dbc_string() is called. (This can be bodged by changing add_xxx_string() to effectively merge in new databases from load_string rather than re-creating). I quite like this approach as it highlights clashes between multiple dbcs and keeps it convenient. However there are lots of potential pitfalls and loading all of our databases takes nearly 40 secs on startup of the tool :-O

2. Load databases independently ; so as a second workaround I wrote a small wrapper class that creates a new database every time I load in a dbc file and then provides an abstraction of interfaces such as get_message_by_frame_id() that iterates through all loaded databases. The downside is that if a frame id (or other object) exists in multiple databases, there's no clash detection ... However parsing is much faster : the same set of dbcs load in 15 secs instead of 40...

Do you have any thoughts?

Hello,

I have a dbc file which cannot be parsed by cantools. I looked at pyparsing Exception and it looks like the problem comes from one of the comments.

The error can be seen in the following dbc string:

VERSION "TEST"
CM_ "";

This crashes and pyparsing outputs Expected end of text (at char 15), (line:2, col:1)

I also tried with :

VERSION "TEST"
BS_: 

And it worked fine.

Best regards,

Hi Erik,

Cheers! Excellent job you have done, especially the generating c source code part that I found particular useful to me. When reading the generated code, a colleague of mine reminded me that using bit field in the message structs might be a good idea, and I very much like to know your opinion. I'll use the ExampleMessage from motohawk.dbc as an example.

Right now the struct definition is as below. The packing and unpacking functions are done with bit operations to set and extract data.

struct motohawk_example_message_t {
    uint8_t enable;
    uint8_t average_radius;
    int16_t temperature;
};

If adding bit fields, the packing and unpacking can be simplified or eliminated, by force converting the incoming data to fit the struct, and return the pointer directly for outgoing data.

struct motohawk_example_message_t {
    uint8_t enable: 1;
    uint8_t average_radius: 6;
    int16_t temperature: 12;
} __attribute__((packed));

It introduces some new issues as well. So if applicable, at least some conversions are needed.

• The bit-wise endianess in a CAN signal can be both little-endian or big-endian, but in computer system it is always big-endian.
• In my experiment, when instantiating a message from the struct, the signals seem to be stored in a reverse order compared to their sequence.
• The total size of a CAN message could be larger than the message struct. The above struct takes only 3 bytes, while it should take 8 bytes from .dbc file definition.

I am still studying the generated code, so let me know any of my misunderstandings. Do you think the bit field struct has valid advantage here? Or it may complicate things in an unnecessary way?

Yang

Hey,

I'm trying to decode the following can frame: can0 599 [8] 00 00 73 D9 FF 1D 06 C3
I created a dbc file which looks like this:

BO_ 1433 DSP_DRIVE_STATUS_0: 8 DSP
 SG_ DSP_DRIVE_STATUS_0_odometer : 0|32@1+ (1,0) [0|0] "km" DBG
 SG_ DSP_DRIVE_STATUS_0_range : 40|8@1+ (1,0) [0|0] "km" DBG
 SG_ DSP_DRIVE_STATUS_0_speed : 48|16@1+ (0.01,0) [0|0] "km/h" DBG

When decoding i get

DSP_DRIVE_STATUS_0(
    DSP_DRIVE_STATUS_0_odometer: 3648192512 km,
    DSP_DRIVE_STATUS_0_range: 29 km,
    DSP_DRIVE_STATUS_0_speed: 499.26 km/h
)

which is obviousely wrong. Looking at the frame directly 06 C3 is big endian for 17.31km/h (from 0x06C3 * 0.01) and 0x73D9 is 29657km which fits my car's total km.

When I change 48|16@1+ to 48|16@0+ (for big endian) i get unpack requires at least 71 bits to unpack (got 64).

What am I doing wrong? How do I correctly specify a big endian signal in a dbc file?

Hello,

Thanks for putting this nice library out!

I found what I think is a defect: when I encode a message, it is always returned on full length of 8 bytes, even though a smaller length is correctly set in the description file (Kayak in my case).

Steps to reproduce:
1- Have a message of total length 1 byte defined in a .kcd file. Example (just the message itself):

<Message id="0x275" length="1" name="Command">
  <Notes>A command</Notes>
  <Signal name="Start" offset="0" length="8" endianess="little" />
  <Producer>
    <NodeRef id="0"/>
  </Producer>
</Message>

2- Encode it:

import cantools
db = cantools.db.load_file('Test_file.kcd')
messages = {msg.frame_id: msg for msg in db.messages}
print(messages[0x275])
# Printing: message('Command', 0x275, False, 8, 'A command')
data = messages[0x275].encode({'Start': 1})
print(len(data))
# Printing: 8

Not only does it returns 8 bytes long data. The Message object itself believe it is of length 8.

I am willing to find the source of the problem and submit a PR. But before, I wanted to know if this was an intentional feature or if this is a bug?

Cheers,
Axel

I have .dbc with extended multiplexing message done according AN-ION-1-0521_Extended_Multiplexing_in_DBC_Databases.pdf Vector documentation.
And following code:

import cantools
db = cantools.database.load_file('ext_multiplexed.dbc')
msg = db.get_message_by_name('ExtMX_Message')
msg.signal_tree

Produce result like here only one multiplexer:
[{'S0_m': {0: ['S1_m'], 1: ['S6'], 2: ['S1_m'], 4: ['S2', 'S3'], 5: ['S4'], 6: ['S5']}}, 'S7', 'S7']
On my point of view it's a wrong behavior. If the Library does not support such kind of CAN messages I expect get exception or error response. But more valuable will be support for such messages, too.

Description from document: Extended_Multiplexing_in_DBC.pdf
Example based on the document: ext_multiplexed.dbc.gz

Note: Here is not only nested multiplexers, but and a list of possible multiplexer ID values mapped on same signals.

It's not only theoretical idea, but in real J1939.dbc I see at least one such message.

Hi, just wanted to ask a question.

The last I checked, cantools did not properly support database-, message-, and signal-level attributes passed to constructors. Has this since been updated? I am hoping to build up a CAN database using cantools classes and assign certain attributes to the objects within the database, and finally render this to a DBC file. Is this possible at this time?

Thanks

First of all, great tool and nice coding!

A question regarding attributes of enum-type:
Examining DBC-files generated by Vectors tool "CANdb++" (version 3.5.4), I see that the attribute default is defined with the enum-name as a string, but each object is defined with the enumerated value as integer (0 => "cyclic" in example below), not a string.

BA_DEF_ BO_  "GenMsgSendType" ENUM  "cyclic","NotUsed","IfActive","NotUsed","NotUsed","NotUsed","NotUsed","NotUsed","noMsgSendType";

BA_DEF_DEF_  "GenMsgSendType" "noMsgSendType";

BA_ "GenMsgSendType" BO_ 2348875518 0;

It seems as if cantools always applies the enum-name-string in all cases?
Which DBC-version do you support?

Thanks!

Hi @eerimoq,
I really like your project cantools.
I'm working a lot on CAN related tools, as well. In the past, I started to contribute to the Scapy project to support various automotive protocols, like UDS, ISOTP, OBD-II, GMLAN and DoIP.
You can have a look to my work here:
https://scapy.readthedocs.io/en/latest/advanced_usage.html#automotive-usage
and here
https://github.com/secdev/scapy/tree/master/scapy/contrib/automotive

We also want to integrate the DBC file format and the j1939 protocol, in the future.
Since you did all of this things in the past, I would like to ask you, if you want to
join us and help us to make Scapy ready for the automotive world.

Since Scapy is all about packets and packet manipulation, I think, it's the perfect project
to provide an open source tool for automotive networks.

Have a great day,
Nils

I believe I've identified a few big endian related decoding errors or improvements.

1. Skip bigendian signal start calculation / decode logic if signal <= 8 bits as "Endianness refers to the sequential order in which bytes are arranged into larger numerical values "
2. When the signal at the end of a message is bigendian a sanity check fails
   "unpack requires at least 71 bits to unpack (got 64)". I included a unit test triggering this, but have not decided on the correct fix yet, so wanted to open the topic for discussion to get the opinion of the maintainer.

I've started a branch here, and can submit a PR after I get your opinion on fixing item 2 I listed.
https://github.com/maxchill/cantools/tree/bugfix/bigendian_decoding_errors

Using Python 3.4.4 and cantools 16.2.0 I get the following error when trying to load my DBC:

cantools.db.UnsupportedDatabaseFormatError: DBC: "Invalid DBC syntax at line 5569, column 80: 'BA_REL_ "GenSigComment" BU_SG_REL_ ICCM_CE SG_ 569 SomeString >!<"";': Expected positive integer."

Is it possible to add support for this?

I am having issues parsing SG_MUL_VAL_ parameter in .dbc files. The following exception is raised:

>>> db = cantools.db.load_file('TestDB.DBC')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\cantools\db\__init__.py", line 41, in load
_file
    return load(fin)
  File "C:\Python27\lib\site-packages\cantools\db\__init__.py", line 58, in load

    return load_string(fp.read())
  File "C:\Python27\lib\site-packages\cantools\db\__init__.py", line 88, in load
_string
    raise UnsupportedDatabaseFormatError(e_dbc, e_kcd)
cantools.db.UnsupportedDatabaseFormatError: DBC: "Invalid DBC syntax at line 77,
 column 1: '>!<SG_MUL_VAL_ 2148676694 BIT_L Multiplexor 8-8, 16-16, 24-24;': Exp
ected end of text.", KCD: "syntax error: line 1, column 0"

The SG_MUL_VAL_ defines the multiplexer for the signal. For example:
SG_MUL_VAL_ 2148676694 BIT_L Multiplexor 8-8, 16-16, 24-24;

• 2148676694: Message ID
• BIT_L: Signal to multiplex
• Multiplexor: Multiplexer field
• 8-8, 16-16, 24-24: Values to multiplex

Here is the dbc: TestDB.zip
dbc is generated using CANdb++

When there is a default string attribute which is numeric, it is converting it to int. As result there is type conflict.

Let's say there is a .dbc file with attribute with type of string and its default value is "100". When you load the .dbc file and then do db.as_dbc_string(), it would not have default value for the attribute.

https://github.com/eerimoq/cantools/blob/d1127b07efea9ec74844afc9d36f0f7095160d86/cantools/db/formats/dbc.py#L522

Hi,

I was playing around with Kvaser Database Editor and realized that DBC files can contain double types. This is done via:

SIG_VALTYPE_ <can-id> <signame> : 2;

It seems like cantools is is currently treating this as an (u)int64_t instead of a double, which leads to incorrect C struct member types being generated.

Here's a sample DBC I've created

VERSION "HIPBNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY/4/%%%/4/'%**4YYY///"


NS_ : 
	NS_DESC_
	CM_
	BA_DEF_
	BA_
	VAL_
	CAT_DEF_
	CAT_
	FILTER
	BA_DEF_DEF_
	EV_DATA_
	ENVVAR_DATA_
	SGTYPE_
	SGTYPE_VAL_
	BA_DEF_SGTYPE_
	BA_SGTYPE_
	SIG_TYPE_REF_
	VAL_TABLE_
	SIG_GROUP_
	SIG_VALTYPE_
	SIGTYPE_VALTYPE_

BS_:

BU_: TestNode


BO_ 1024 NewMessage0: 8 TestNode
 SG_ NewSignal0 : 0|64@1- (1,0) [0|0] "" Vector__XXX


CM_ BU_ TestNode "";
SIG_VALTYPE_ 1024 NewSignal0 : 2;

Is this something that would be supported?

Thanks!

Edit: On further inspection, I guess it's technically supported in the DBC parser, but since the check for is_float is what determines whether to perform special behaviour for non-integer types for code generation, we generate int types. Would it be sufficient to modify is_float to check whether the type is in [1, 2] instead?