Code Monkey home page Code Monkey logo

odxtools's Introduction

PyPi - Version PyPI - License CI Status

odxtools

odxtools is a set of utilities for working with diagnostic descriptions of automotive electronic control units using the data model and the associated technologies of the ODX standard.

ODX stands for "Open Diagnostic data eXchange" and is primarily an XML based file format to describe the diagnostic capabilities of the electronic control units (ECUs) of complex distributed technical systems (usually cars and trucks). ODX is an open standard maintained by ASAM e.V. and is also standardized internationally by ISO-22901.

Usually, ODX is used to complement the UDS automotive diagnostics standard -- which itself can be considered to be an extension of OBD-II -- to provide a machine-processable description of the vendor-specific diagnostics functionality of a vehicle's ECUs. That said, the functionality which is described by ODX files neither needs to be a super- nor a subset of OBD-II/UDS, e.g., ODX can be used to describe diagnostic functionality that uses fundamentally different wire formats and conventions than the ones mandated by OBD-II/UDS. (In practice, the ODX-described functionality usually adheres to these standards, though.)

The functionality provided by odxtools encompasses parsing and internalizing ODX diagnostic database files as well as de- and encoding the data of diagnostic requests and their responses send to/received from ECUs in an pythonic manner.

Table of Contents

Use Cases

Here are some of the intended use cases of odxtools:

  • Prototype development: Interacting with the diagnostic services of electronic control units directly from python (requires taping into the car's relevant CAN or ethernet bus)
  • End-of-production calibration/quality control: Initial set up and running a self diagnosis of newly produced cars to ensure that everything works as specified
  • After-sales: Implementing servicing functionality for workshops, i.e., defining test schedules based on the legally mandated functionality of ISO 15031-6 (OBD II) as well as manufacturer-specific routines
  • Prototype development (II): Analyzing and debugging diagnostic sessions done using third-party software
  • Prototype development (III): Implementing bridges to higher-level protocols such as HTTP
  • Development for mass production: Accelerating the implementation of diagnostic servicesfor low-cost ECUs by using odxtools-based code generators for the diagnostic glue code on system-level languages like C++ or rust

Please be aware that some of the use cases listed above are currently rather aspirational.

Installation

The easiest way of installing odxtools on your system is via pip:

python3 -m pip install odxtools

If you want to develop odxtools itself, you need to install it from source using git. The first step is to clone the repository:

cd $BASE_DIR
git clone https://github.com/mercedes-benz/odxtools

After this, make sure that all python dependencies are installed:

cd $BASE_DIR/odxtools
python3 -m pip install -e .

Next, you can optionally build a package and install it on the system:

cd $BASE_DIR/odxtools
python3 -m pip install --upgrade build
python3 -m build
sudo python3 -m pip install dist/odxtools-*.whl

Finally, update the PYTHONPATH environment variable and the newly cloned module is ready to be used:

export PYTHONPATH="$BASE_DIR/odxtools:$PYTHONPATH"

Now, you can check whether the installation worked:

python3 -m odxtools list -a "$YOUR_PDX_FILE"

Usage Examples

Python snippets

  • Load an ODX database from file somersault.pdx:

    import odxtools
    
    db = odxtools.load_pdx_file("somersault.pdx")
  • List the names of all available services of the somersault_lazy ECU:

    # [...]
    
    ecu = db.ecus.somersault_lazy
    print(f"Available services for {ecu.short_name}: {ecu.services}")
  • Determine the CAN IDs which the somersault_lazy ECU uses to send and receive diagnostic messages:

    # [...]
    
    print(f"ECU {ecu.short_name} listens for requests on CAN ID 0x{ecu.get_receive_id():x}")
    print(f"ECU {ecu.short_name} transmits responses on CAN ID 0x{ecu.get_send_id():x}")
  • Encode a session_start request to the somersault_lazy ECU:

    # [...]
    
    raw_request_data = ecu.services.session_start()
    
    print(f"Message for session start request of ECU {ecu.short_name}: {raw_request_data}")
    # -> bytearray(b'\x10\x00')
  • Encode the positive response to the start_session request:

    # [...]
    
    raw_request_data = ecu.services.session_start()
    raw_response_data = ecu.services.session_start.positive_responses[0].encode(coded_request=raw_request_data)
    
    print(f"Positive response to session_start() of ECU {ecu.short_name}: {raw_response_data}")
    # -> bytearray(b'P')
  • Decode a request:

    # [...]
    
    raw_data = b"\x10\x00"
    decoded_message = ecu.decode(raw_data)
    print(f"decoded message: {decoded_message}")
    # -> decoded message: [start_session()]
  • Decode a response to a request:

    # [...]
    
    raw_request_data = b"\x10\x00"
    raw_response_data = b'P'
    decoded_response = ecu.decode_response(raw_response_data, raw_request_data)
    print(f"decoded response: {decoded_response}")
    # -> decoded response: [session()]

Using the non-strict mode

By default, odxtools raises exceptions if it suspects that it cannot fulfill a requested operation correctly. For example, if the dataset it is instructed to load is detected to be not conformant with the ODX specification, or if completing the operation requires missing features of odxtools. To be able to deal with such cases, odxtools provides a "non-strict" mode where such issues are ignored, but where the results are undefined. The following snippet shows how to instruct odxtools to load a non-conforming file in non-strict mode, and after this is done, enables the safety checks again:

import odxtools

[...]

odxtools.exceptions.strict_mode = False
botched_db = odxtools.load_file("my_non-conforming_database.pdx")
odxtools.exceptions.strict_mode = True

[...]

Interactive Usage

Python REPL

python's interactive read-reval-print-loop (REPL) supports tab-completion on most plattforms, i.e., in this case, all data can be conveniently interactivly discovered and this makes odxtools a very convenient tool to explore the capabilities of a given ECU.

A notable exception is the Microsoft Windows platform: Most python distribtions for Windows do not enable tab-completion by default in their REPL. For more convenience in such a scenario, we recommend using ptpython. ptpython can be installed like any other python package, i.e., via python3 -m pip install ptpython. Then, the REPL ought to be started using

c:\odxtest>python3 "C:\Python39\Lib\site-packages\ptpython\entry_points\run_ptpython.py"

Alternatively, pyreadline can be used after installing it via python3 -m pip install pyreadline. With this, basic tab-completion for python under Windows in Interactive Mode should work.

Command line usage

Based the python module, odxtools also provides a set of command line utilities for quick interactive explorations. Amongst others, these utilities allow the inspection ODX/PDX files, snooping on diagnostic sessions, etc. If odxtools is installed on a system-wide basis, these commands can be invoked using odxtools SUBCOMMAND [PARAMS], if the repository has been manually cloned via git and odxtools has not been installed on a system-wide basis, the way to invoke these utilities is via python3 -m odxtools SUBCOMMAND [PARAMS].

Generic parameters

Available generic parameters and a list of subcommands can be obtained using odxtools --help:

$ odxtools --help
usage: odxtools [-h] [--version] {list,browse,snoop,find,decode,compare} ...

Utilities to interact with automotive diagnostic descriptions based on the ODX standard.

Examples:
  For printing all services use:
   odxtools list ./path/to/database.pdx --services
  For browsing the data base and encoding messages use:
   odxtools browse ./path/to/database.pdx

positional arguments:
  {list,browse,snoop,find,decode,compare}
                        Select a sub command
    list                Print a summary of automotive diagnostic files.
    browse              Interactively browse the content of automotive diagnostic files.
    snoop               Live decoding of a diagnostic session.
    find                Find & display services by their name
    decode              Find & print service by hex-data. Can also decode the hex-data to its named parameters.
    compare             Compares two ecu versions or databases with each other. Checks whether diagnostic services and its parameters have changed.

optional arguments:
  -h, --help            show this help message and exit
  --version             Print the odxtools version

All subcommands accept the --help parameter:

$ odxtools list --help
usage: odxtools list [-h] [-v VARIANT [VARIANT ...]] [-s [SERVICE [SERVICE ...]]] [-p] [-d] [-a] PDX_FILE
[...]

It follows is an inexhaustive list of the subcommands that are currently available:

The list subcommand

The list subcommand is used to parse a .pdx database file and print the relevant parts of its content to the terminal.

$ odxtools list -h
usage: odxtools list [-h] [-v VARIANT [VARIANT ...]] [-g] [-s [SERVICE [SERVICE ...]]] [-p] [-d] [-a] [-po] PDX_FILE

List the content of automotive diagnostic files (*.pdx)

Examples:
  For displaying only the names of the diagnostic layers use:
    odxtools list ./path/to/database.pdx
  For displaying all content use:
    odxtools list ./path/to/database.pdx --all
  For more information use:
    odxtools list -h

positional arguments:
  PDX_FILE              path to the .pdx file

optional arguments:
  -h, --help            show this help message and exit
  -v VARIANT [VARIANT ...], --variants VARIANT [VARIANT ...]
                        Specifies which variants should be included.
  -g, --global-negative-responses
                        Print a list of the global negative responses for the selected ECUs.
  -s [SERVICE [SERVICE ...]], --services [SERVICE [SERVICE ...]]
                        Print a list of diagnostic services specified in the pdx.
                        If no service names are specified, all services are printed.
  -p, --params          Print a list of all parameters relevant for the selected items.
  -d, --dops            Print a list of all data object properties relevant for the selected items
  -a, --all             Print a list of all diagnostic services and DOPs specified in the pdx
  -po, --plumbing-output
                        Print full objects instead of selected and formatted attributes

The options --variants and --services can be used to specify which services should be printed.
If the --params option is specified, the message layout and information about the service parameters (reuest as well as responses) are printed for all specified variants/services. If the --global-negative-responses option is specified, all global negative responses are printed for all specified variants. If the --dops option is specified, a list of all data object properties (their names) is printed for all specified variants/services. With the parameter --all all data of the file that is recognized by odxtools is printed. The default output does not display all information of the specified objects but a selection. To see all object information choose the parameter --plumbing-output.

Example:

$ odxtools list $BASE_DIR/odxtools/examples/somersault.pdx --variants somersault_lazy --services do_forward_flips --params
Overview of diagnostic layers:
    | Name            | Variant Type | Num. of Services | Num. of DOPs | Num. of comparams
----+-----------------+--------------+------------------+--------------+-------------------
  0 | somersault_lazy | ECU-VARIANT  |                5 |           10 |                10

Diagnostic layer: 'somersault_lazy'
 Variant Type: ECU-VARIANT
 Description: Sloppy variant of the somersault ECU (lazy < assiduous)

The services of 'somersault_lazy' are:
 do_forward_flips <ID: OdxLinkId('somersault.service.do_forward_flips')>
  Service description: Do a forward flip.

  Request Properties:
   Request Name: do_forward_flips
   Byte-Array: ---      Hex-String: 0x---
   Service Parameters: [sid, forward_soberness_check, num_flips]

    | Name                    | Byte Pos. | Bit Length | Param. Type | Data Type | Value | Value Desc. | Linked DOP
----+-------------------------+-----------+------------+-------------+-----------+-------+-------------+-----------------
  0 | sid                     |         0 |          8 | CODED-CONST | A_UINT32  | 0xBA  | coded value |
  1 | forward_soberness_check |         1 |          8 | VALUE       | A_UINT32  |       |             | soberness_check
  2 | num_flips               |         2 |          8 | VALUE       | A_UINT32  |       |             | num_flips

  Message format of a request:
           7     6     5     4     3     2     1     0
        +-----+-----+-----+-----+-----+-----+-----+-----+
      0 | sid(8 bits)                                   |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      1 | forward_soberness_check(8 bits)               |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      2 | num_flips(8 bits)                             |
        +-----+-----+-----+-----+-----+-----+-----+-----+
   
  Positive Response Properties:
   Number of Positive Responses: 1
   Positive Responses: [grudging_forward]
   Service Parameters: [sid, num_flips_done]

    | Name           | Byte Pos. | Bit Length | Parameter Type         | Data Type | Value | Value Desc. | Linked DOP
----+----------------+-----------+------------+------------------------+-----------+-------+-------------+-------------
  0 | sid            |         0 |          8 | CODED-CONST            | A_UINT32  | 0xFA  | coded value |
  1 | num_flips_done |         1 |          8 | MATCHING-REQUEST-PARAM |           |       |             |

  Message format of a positive response:
           7     6     5     4     3     2     1     0
        +-----+-----+-----+-----+-----+-----+-----+-----+
      0 | sid(8 bits)                                   |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      1 | num_flips_done(8 bits)                        |
        +-----+-----+-----+-----+-----+-----+-----+-----+
   
  Negative Response Properties:
   Number of Negative Responses: 1
   Negative Responses: [flips_not_done]
   Service Parameters: [sid, rq_sid, reason, flips_successfully_done]

    | Name                    |Byte Pos. | Bit Length | Parameter Type         | Data Type | Value     | Value Desc. | Linked DOP
----+-------------------------+----------+------------+------------------------+-----------+-----------+-------------+-------------
  0 | sid                     |        0 |          8 | CODED-CONST            | A_UINT32  | 0x7F      | coded value |
  1 | rq_sid                  |        1 |          8 | MATCHING-REQUEST-PARAM |           |           |             |
  2 | reason                  |        2 |          8 | NRC-CONST              | A_UINT32  | [0, 1, 2] | coded value |
  3 | flips_successfully_done |        3 |          8 | VALUE                  | A_UINT32  |           |             | num_flips

  Number of negative responses: 1
  Message format of a negative response:
           7     6     5     4     3     2     1     0
        +-----+-----+-----+-----+-----+-----+-----+-----+
      0 | sid(8 bits)                                   |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      1 | rq_sid(8 bits)                                |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      2 | reason(8 bits)                                |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      3 | flips_successfully_done(8 bits)               |
        +-----+-----+-----+-----+-----+-----+-----+-----+
   

The browse subcommand

The browse subcommand uses InquirerPy to interactively navigate through the database of a .pdx file. For example, using the browse subcommand you can select the ECU and service without spamming the terminal:

$ odxtools browse $BASE_DIR/odxtools/examples/somersault.pdx
? Select a Variant.  somersault_lazy
ECU-VARIANT 'somersault_lazy' (Receive ID: 0x7b, Send ID: 0x1c8)
? The variant somersault_lazy offers the following services. Select one!  do_forward_flips
? This service offers the following messages.  Request: do_forward_flips
             7     6     5     4     3     2     1     0
          +-----+-----+-----+-----+-----+-----+-----+-----+
        0 | sid(8 bits)                                   |
          +-----+-----+-----+-----+-----+-----+-----+-----+
        1 | forward_soberness_check(8 bits)               |
          +-----+-----+-----+-----+-----+-----+-----+-----+
        2 | num_flips(8 bits)                             |
          +-----+-----+-----+-----+-----+-----+-----+-----+
     Parameter(short_name='sid', type='CODED-CONST', semantic=None, byte_position=0, bit_length=8, coded_value='0xba')
     Parameter(short_name='forward_soberness_check', type='VALUE', semantic=None, byte_position=1, bit_length=8, dop_ref='somersault.DOP.soberness_check')
      DataObjectProperty('soberness_check', category='LINEAR', internal_type='A_UINT32', physical_type='A_UINT32')
     Parameter(short_name='num_flips', type='VALUE', semantic=None, byte_position=2, bit_length=8, dop_ref='somersault.DOP.num_flips')
      DataObjectProperty('num_flips', category='LINEAR', internal_type='A_UINT32', physical_type='A_UINT32')
[...]

The snoop subcommand

The snoop subcommand can be used to decode a trace of a or a currently running diagnostic session.

$ odxtools snoop -h
usage: odxtools snoop [-h] [--active] [--channel CHANNEL] [--rx RX] [--tx TX] [--variant VARIANT]
                      [--protocol PROTOCOL]
                      PDX_FILE

Live decoding of a diagnostic session.

positional arguments:
  PDX_FILE              path to the .pdx file

options:
  -h, --help            show this help message and exit
  --active, -a          Active mode, sends flow control messages to receive ISO-TP telegrams successfully
  --channel CHANNEL, -c CHANNEL
                        CAN interface name to be used (required in active mode)
  --rx RX, -r RX        CAN ID in which the ECU listens for diagnostic messages
  --tx TX, -t TX        CAN ID in which the ECU sends replys to diagnostic messages  (required in active mode)
  --variant VARIANT, -v VARIANT
                        Name of the ECU variant which the decode process ought to be based on
  --protocol PROTOCOL, -p PROTOCOL
                        Name of the protocol used for decoding

Example:

# create a socketcan `vcan0` interface
sudo ip link add dev vcan0 type vcan
sudo ip link set vcan0 up

# start the snooping on vcan0
odxtools snoop -c vcan0 --variant "somersault_lazy" $BASE_DIR/odxtools/examples/somersault.pdx

# on a different terminal, run the diagnostic session
$BASE_DIR/odxtools/examples/somersaultlazy.py -c vcan0

The snoop command will then output the following:

$ odxtools snoop -c vcan0 --variant "somersault_lazy" $BASE_DIR/odxtools/examples/somersault.pdx
Decoding messages on channel vcan0
Tester: do_forward_flips(forward_soberness_check=18, num_flips=1)
 -> 7fba7f (bytearray(b'\x7f\xba\x7f'), 3 bytes)
Tester: start_session()
 -> session()
Tester: do_forward_flips(forward_soberness_check=18, num_flips=1)
 -> grudging_forward(num_flips_done=bytearray(b'\x01'))
Tester: do_forward_flips(forward_soberness_check=35, num_flips=1)
 -> flips_not_done(rq_sid=bytearray(b'\xba'), reason=0, flips_successfully_done=0)
Tester: do_forward_flips(forward_soberness_check=18, num_flips=3)
 -> grudging_forward(num_flips_done=bytearray(b'\x03'))
Tester: do_forward_flips(forward_soberness_check=18, num_flips=50)
 -> flips_not_done(rq_sid=bytearray(b'\xba'), reason=1, flips_successfully_done=6)

The find subcommand

The find subcommand can be used to find a service and its associated information by a partial name via cli.

$ odxtools find -h
usage: odxtools find [-h] [-v VARIANT] -s [SERVICES ...] [-nd] [-ro] [-po] PDX_FILE

Find & print services by name

Examples:
  For displaying the services associated with the partial name 'Reset' without details:
    odxtools find ./path/to/database.pdx -s "Reset" --no-details
  For more information use:
    odxtools find -h

positional arguments:
  PDX_FILE              Location of the .pdx file

options:
  -h, --help            show this help message and exit
  -v VARIANT, --variants VARIANT
                        Specifies which ecu variants should be included.
  -s [SERVICES ...], --service-names [SERVICES ...]
                        Print a list of diagnostic services partially matching given service names
  -nd, --no-details     Don't show all service details
  -ro, --relaxed-output
                        Relax output formatting rules (allow unknown bitlengths for ascii representation)
  -po, --plumbing-output
                        Print full objects instead of selected and formatted attributes

Example: Find diagnostic services with the name session_start

$ odxtools find $BASE_DIR/odxtools/examples/somersault.pdx -s session_start

=====================================
somersault_lazy, somersault_assiduous
=====================================


 session_start <ID: OdxLinkId('somersault.service.session_start')>

  Request Properties:
   Request Name: start_session
   Byte-Array: bytearray(b'\x10\x00')   Hex-String: 0x1000
   Service Parameters: [sid, id]

    | Name | Byte Pos. | Bit Length | Param. Type | Data Type | Value | Value Desc. | Linked DOP
----+------+-----------+------------+-------------+-----------+-------+-------------+------------
  0 | sid  |         0 |          8 | CODED-CONST | A_UINT32  | 0x10  | coded value |
  1 | id   |         1 |          8 | CODED-CONST | A_UINT32  | 0x00  | coded value |

  Message format of a request:
           7     6     5     4     3     2     1     0  
        +-----+-----+-----+-----+-----+-----+-----+-----+
      0 | sid (8 bits)                                  |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      1 | id (8 bits)                                   |
        +-----+-----+-----+-----+-----+-----+-----+-----+

  Positive Response Properties:
   Number of Positive Responses: 1
   Positive Responses: [session]
   Service Parameters: [sid, can_do_backward_flips]

    | Name                  | Byte Pos. | Bit Length | Param. Type | Data Type        | Value | Value Desc. | Linked DOP
----+-----------------------+-----------+------------+-------------+------------------+-------+-------------+------------
  0 | sid                   |         0 |          8 | CODED-CONST | A_UINT32         | 0x50  | coded value |
  1 | can_do_backward_flips |         1 |          8 | VALUE       | A_UNICODE2STRING |       |             | boolean

  Message format of a positive response:
           7     6     5     4     3     2     1     0  
        +-----+-----+-----+-----+-----+-----+-----+-----+
      0 | sid (8 bits)                                  |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      1 | can_do_backward_flips (8 bits)                |
        +-----+-----+-----+-----+-----+-----+-----+-----+

  Negative Response Properties:
   Number of Negative Responses: 1
   Negative Responses: [general_negative_response]
   Service Parameters: [sid, rq_sid, response_code]

    | Name          | Byte Pos.| Bit Length | Parameter Type         | Data Type | Value | Value Desc. | Linked DOP
----+---------------+----------+------------+------------------------+-----------+-------+-------------+------------
  0 | sid           |        0 |          8 | CODED-CONST            | A_UINT32  | 0x7F  | coded value |
  1 | rq_sid        |        1 |          8 | MATCHING-REQUEST-PARAM |           |       |             |
  2 | response_code |        2 |          8 | VALUE                  | A_UINT32  |       |             | error_code

  Message format of a negative response:
           7     6     5     4     3     2     1     0  
        +-----+-----+-----+-----+-----+-----+-----+-----+
      0 | sid (8 bits)                                  |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      1 | rq_sid (8 bits)                               |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      2 | response_code (8 bits)                        |
        +-----+-----+-----+-----+-----+-----+-----+-----+

The decode subcommand

The decode subcommand can be used to decode hex-data to a service, and its associated parameters.

$ odxtools decode -h
usage: odxtools decode [-h] [-v VARIANT] -d DATA [-D] PDX_FILE

Decode request by hex-data

Examples:
  For displaying the service associated with the request 10 01 & decoding it:
    odxtools decode ./path/to/database.pdx -D -d '10 01'
  For displaying the service associated with the request 10 01, without decoding it:
    odxtools decode ./path/to/database.pdx -d '10 01'
  For more information use:
    odxtools decode -h

positional arguments:
  PDX_FILE              Location of the .pdx file

options:
  -h, --help            show this help message and exit
  -v VARIANT, --variants VARIANT
                        Specifies which ecu variants should be included.
  -d DATA, --data DATA  Specify data of hex request
  -D, --decode          Decode the given hex data

Example: Decode diagnostic services with the request 10 00

$ odxtools decode $BASE_DIR/odxtools/examples/somersault.pdx -d '10 00'
Binary data: 10 00
Decoded by service 'session_start' (decoding ECUs: somersault_lazy, somersault_assiduous)

Example: Decode diagnostic services with the request 10 00, and parameters

$ odxtools decode $BASE_DIR/odxtools/examples/somersault.pdx -d '10 00' -D
Binary data: 10 00
Decoded by service 'session_start' (decoding ECUs: somersault_lazy, somersault_assiduous)
Decoded data:
  sid=16 (0x10)
  id=0 (0x0)

The compare subcommand

The compare subcommand can be used to compare databases (pdx-files) and diagnostic layers with each other. All diagnostic services as well as its parameters of specified databases and variants are compared with each other and changes are displayed.

database comparison:

  • new diagnostic layers
  • deleted diagnostic layers
  • diagnostic layer comparison

diagnostic layer comparison:

  • new services
  • deleted services
  • renamed services
  • service parameter comparison

service parameter comparison:

find changes in following properties:

  • Name
  • Byte Position
  • Bit Length
  • Semantic
  • Parameter Type
  • Coded Value
  • Data Type
  • Data Object Property (Name, Data Type, Bit Length, Default Value)
$ odxtools compare -h
usage: odxtools compare [-h] [-v VARIANT [VARIANT ...]] [-db DATABASE [DATABASE ...]] [-nd] [-od] PDX_FILE

Compares two ecu versions or databases with each other. Checks whether diagnostic services and its parameters have changed.

Examples:
  Comparison of two ecu versions:
    odxtools compare ./path/to/database.pdx -v variant1 variant2
  Comparison of two database versions:
    odxtools compare ./path/to/database.pdx -db ./path/to/old-database.pdx
  For more information use:
    odxtools compare -h

positional arguments:
  PDX_FILE              Location of the .pdx file

options:
  -h, --help            show this help message and exit
  -v VARIANT [VARIANT ...], --variants VARIANT [VARIANT ...]
                        Compare specified ecu variants to each other.
  -db DATABASE [DATABASE ...], --database DATABASE [DATABASE ...]
                        Compare specified database file(s) to database file of first input argument.
  -nd, --no-details     Don't show all service parameter details
  -od, --object-details
                        Print all object details instead of just the name

Example: Compare the ecu variants somersault_lazy and somersault_assiduous

$ odxtools compare $BASE_DIR/odxtools/examples/somersault.pdx -v somersault_lazy somersault_assiduous

Overview of diagnostic layers:
    | Name                 | Variant Type | Num. of Services | Num. of DOPs | Num. of comparams
----+----------------------+--------------+------------------+--------------+-------------------
  0 | somersault_lazy      | ECU-VARIANT  |                5 |           10 |                10
  1 | somersault_assiduous | ECU-VARIANT  |                8 |           10 |                10


Changes in ecu variant somersault_lazy
 (compared to somersault_assiduous)

 Changed diagnostic services for ecu variant: somersault_lazy

 Deleted services
    | Name                 | Semantic   | Hex-Request
----+----------------------+------------+---------------
  0 | set_operation_params | FUNCTION   |
  1 | do_backward_flips    | FUNCTION   |
  2 | headstand            | UNKNOWN    |

Example: Compare two databases

$ odxtools compare $BASE_DIR/odxtools/examples/somersault_modified.pdx -db $BASE_DIR/odxtools/examples/somersault.pdx -nd

Changes in file somersault_modified.pdx
 (compared to somersault.pdx)

Overview of diagnostic layers (for somersault_modified.pdx)
    | Name                 | Variant Type | Num. of Services | Num. of DOPs | Num. of comparams
----+----------------------+--------------+------------------+--------------+-------------------
  0 | somersault           | BASE-VARIANT |                6 |           10 |                10
  1 | somersault_lazy      | ECU-VARIANT  |                5 |           10 |                10
  2 | somersault_assiduous | ECU-VARIANT  |                8 |           10 |                10
  3 | somersault_young     | ECU-VARIANT  |                6 |           10 |                10


Overview of diagnostic layers (for somersault.pdx)
    | Name                 | Variant Type | Num. of Services | Num. of DOPs | Num. of comparams
----+----------------------+--------------+------------------+--------------+-------------------
  0 | somersault           | BASE-VARIANT |                7 |           10 |                10
  1 | somersault_lazy      | ECU-VARIANT  |                5 |           10 |                10
  2 | somersault_assiduous | ECU-VARIANT  |                8 |           10 |                10

 Changed ecu variants:
         New ecu variants:
                         somersault_young
         Deleted ecu variants:

 Changed diagnostic services for ecu variant: somersault_lazy

 Renamed services
    | Name          | Semantic   | Hex-Request   | Old service name
----+---------------+------------+---------------+--------------------
  0 | session_start | SESSION    | 0x1000        | start_session

 Services with parameter changes
    | Name           | Semantic      | Hex-Request | Changed parameters
----+----------------+---------------+-------------+-----------------------------------------------------------------------------------------------
  0 | session_start  | SESSION       | 0x1000      | positive response parameter 'can_do_backward_flips',
  1 | session_stop   | SESSION       | 0x1001      | request parameter 'id', positive response parameter 'can_do_backward_flips',
  2 | tester_present | TESTERPRESENT | 0x3E00      | request parameter 'id', positive response parameter 'status',
  3 | report_status  | CURRENTDATA   | 0x2200      | positive response parameter 'dizzyness_level', positive response parameter 'happiness_level',

 Detailed changes of diagnostic service session_start

Properties of 2. positive response parameter can_do_backward_flips have changed
    | Property      | Old Value   | New Value
----+---------------+-------------+------------------
  0 | Linked DOP    | uint8       | boolean
  1 | DOP data type | A_UINT32    | A_UNICODE2STRING

 Detailed changes of diagnostic service session_stop

Properties of 2. request parameter id have changed
    | Property   |   Old Value |   New Value
----+------------+-------------+-------------
  0 | Bit Length |          16 |           8

Properties of 2. positive response parameter can_do_backward_flips have changed
    | Property      | Old Value   | New Value
----+---------------+-------------+------------------
  0 | Linked DOP    | uint8       | boolean
  1 | DOP data type | A_UINT32    | A_UNICODE2STRING

 Detailed changes of diagnostic service tester_present

Properties of 2. request parameter id have changed
    | Property   | Old Value   | New Value
----+------------+-------------+-------------
  0 | Value      | 0x01        | 0x00

Properties of 2. positive response parameter status have changed
    | Property   | Old Value   | New Value
----+------------+-------------+-------------
  0 | Bit Length | 16          | 8
  1 | Value      | 0x0043      | 0x00

 Detailed changes of diagnostic service report_status

Properties of 2. positive response parameter dizzyness_level have changed
    | Property       | Old Value       | New Value
----+----------------+-----------------+-----------------
  0 | Parameter name | happiness_level | dizzyness_level
  1 | Linked DOP     | happiness_level | dizzyness_level

Properties of 3. positive response parameter happiness_level have changed
    | Property       | Old Value       | New Value
----+----------------+-----------------+-----------------
  0 | Parameter name | dizzyness_level | happiness_level
  1 | Linked DOP     | dizzyness_level | happiness_level

 Changed diagnostic services for ecu variant: somersault_assiduous

 New services
    | Name     | Semantic   | Hex-Request
----+----------+------------+---------------
  0 | flicflac | FUNCTION   |

 Deleted services
    | Name                 | Semantic   | Hex-Request
----+----------------------+------------+---------------
  0 | set_operation_params | FUNCTION   |

 Renamed services
    | Name          | Semantic   | Hex-Request   | Old service name
----+---------------+------------+---------------+--------------------
  0 | session_start | SESSION    | 0x1000        | start_session

 Services with parameter changes
    | Name              | Semantic      | Hex-Request | Changed parameters
----+-------------------+---------------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------
  0 | session_start     | SESSION       | 0x1000      | positive response parameter 'can_do_backward_flips',
  1 | session_stop      | SESSION       | 0x1001      | request parameter 'id', positive response parameter 'can_do_backward_flips',
  2 | tester_present    | TESTERPRESENT | 0x3E00      | request parameter 'id', positive response parameter 'status',
  3 | do_backward_flips | FUNCTION      |             | request parameter 'backward_soberness_check', positive response parameter 'num_flips_done', positive response parameter 'grumpiness_level',
  4 | report_status     | CURRENTDATA   | 0x2200      | positive response parameter 'dizzyness_level', positive response parameter 'happiness_level',
  5 | headstand         | UNKNOWN       |             | request parameter list,

 Detailed changes of diagnostic service session_start

Properties of 2. positive response parameter can_do_backward_flips have changed
    | Property      | Old Value   | New Value
----+---------------+-------------+------------------
  0 | Linked DOP    | uint8       | boolean
  1 | DOP data type | A_UINT32    | A_UNICODE2STRING

 Detailed changes of diagnostic service session_stop

Properties of 2. request parameter id have changed
    | Property   |   Old Value |   New Value
----+------------+-------------+-------------
  0 | Bit Length |          16 |           8

Properties of 2. positive response parameter can_do_backward_flips have changed
    | Property      | Old Value   | New Value
----+---------------+-------------+------------------
  0 | Linked DOP    | uint8       | boolean
  1 | DOP data type | A_UINT32    | A_UNICODE2STRING

 Detailed changes of diagnostic service tester_present

Properties of 2. request parameter id have changed
    | Property   | Old Value   | New Value
----+------------+-------------+-------------
  0 | Value      | 0x01        | 0x00

Properties of 2. positive response parameter status have changed
    | Property   | Old Value   | New Value
----+------------+-------------+-------------
  0 | Bit Length | 16          | 8
  1 | Value      | 0x0043      | 0x00

 Detailed changes of diagnostic service do_backward_flips

Properties of 2. request parameter backward_soberness_check have changed
    | Property       | Old Value      | New Value
----+----------------+----------------+--------------------------
  0 | Parameter name | backward_check | backward_soberness_check

Properties of 2. positive response parameter num_flips_done have changed
    | Property      |   Old Value | New Value
----+---------------+-------------+-------------
  0 | Byte position |           1 |

Properties of 3. positive response parameter grumpiness_level have changed
    | Property       | Old Value   | New Value
----+----------------+-------------+-------------
  0 | Byte position  | 2           |
  1 | Parameter type | CODED-CONST | VALUE

 Detailed changes of diagnostic service report_status

Properties of 2. positive response parameter dizzyness_level have changed
    | Property       | Old Value       | New Value
----+----------------+-----------------+-----------------
  0 | Parameter name | happiness_level | dizzyness_level
  1 | Linked DOP     | happiness_level | dizzyness_level

Properties of 3. positive response parameter happiness_level have changed
    | Property       | Old Value       | New Value
----+----------------+-----------------+-----------------
  0 | Parameter name | dizzyness_level | happiness_level
  1 | Linked DOP     | dizzyness_level | happiness_level

 Detailed changes of diagnostic service headstand

List of request parameters for service headstand is not identical.
    | List     | Values
----+----------+---------------------
  0 | Old list | [sid, id, duration]
  1 | New list | [sid, duration]

Testing

The included unit tests can be run via

python -m unittest tests/test_*.py

The static type checker can be run via

python3 -m mypy --ignore-missing-imports odxtools

Contributing

We welcome any contributions. If you want to contribute to this project, please read the contributing guide.

Code of Conduct

Please read our Code of Conduct as it is our base for interaction.

Provider Information

Please visit https://mbition.io/en/home/index.html for information on the provider.

Notice: Before you use the program in productive use, please take all necessary precautions, e.g. testing and verifying the program with regard to your specific use. The program was tested solely for our own use cases, which might differ from yours.

Acknowledgements

This work includes research of the project SofDCar (19S21002), which is funded by the German Federal Ministry for Economic Affairs and Climate Action.

License

This project is licensed under the MIT LICENSE.

odxtools's People

Contributors

andlaus avatar dmholtz avatar floroks avatar josglch avatar kakoeh avatar kayoub5 avatar klba avatar louwersj avatar qwander avatar smileprem avatar willzhang05 avatar yuebit avatar zzpbuaa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

odxtools's Issues

Can we make a flag to make it support parsing PDX file even validation fail?

We discussed in a previous question about parsing PDX files when validation fails. The validation failure is caused by the keyword "assert", for example,

def parse_int(value: str) -> int: try: return int(value) except ValueError: v = float(value) assert v.is_integer() return int(v)

In this scenario, if the input string is "0.05", the entire process fails. However, we would like to continue parsing the rest of the validated PDX file without letting one type of issue block the entire process. Editing the PDX file is a costly process, and we would prefer not to modify it.

Is it possible to introduce a flag that allows us to control the behavior of the library, enabling it to parse the entire PDX file even if validation fails in a testing environment? For instance, we can toggle the flag to false during testing, allowing the parsing of the PDX file. However, in the production environment, we can set the flag to true to ensure parsing fails if validation fails. The production environment should prioritize safety considerations.

ODXTools : Admin data access in Individual Variant in the ECU

can not access the admin data for the individual variant inside the one ECU, It seems it is not available now, as able to access the admin data in the diag layer container for the doc revision for ECU, but not individual variants admin data like doc revisions in one of the variants of the ECU - like for boot variant, base variant etc.

Support for odx-c and odx-cs file formats

Hi @andlaus - We have a requirement to parse the communication parameters from PDX files (odx-c and odx-cs) to be able to configure the diagnostic clients. I would like to know, if this file format support is in the road map. Please let us know.

If not planned already, we would be happy to add the functionality to this library. Please let us know if there are any concerns.

Thanks,
Prem

improve comparison tool

Currently the compare command accepts a list of pdx-files and a list of variants as input arguments.
It would be nice to have an optional input argument where a folder with multiple pdx files can be specified.

Idea:

  • provide folder with multiple pdx files as argument
  • load all pdx files in folder, sort them alphabetically, compare databases pairwise

When multiple pdx files are analyzed, a summary of all changes might be helpful

Idea:

  • calculate metrics (number of added services, number of changed services, number of removed services, total number of services per ecu variant, ...)
  • display metrics graphically

The results of the comparison are displayed in the terminal. For further use, a json export might be nice.

Idea:

  • write results of comparison in json structure
  • use links to refer to dignostic services / objects if changes have already been detected (e.g. in another ecu variant / diagnostic layer)

Katja Kรถhler <[email protected]>, on behalf of MBition GmbH.
Provider Information

load a ".odx-d" file(diag-layer-container) failed due to ID-REF to an object from another ".odx-d"(functional group) file

I tried to load a base-variant .odx-d file to generate a database object, with def load_file(file_name: str) -> Database:, this file contains diag-layer-container information,

,however, the load file failed due to

\"ODXLINK reference OdxLinkRef('FGL_UDS') could not be resolved for any of the document fragments [OdxDocFragment(doc_name='DLC_FGL_UDS', doc_type='CONTAINER')]\

the reason for this is we only make one odx-d file as input to load the file, but this base variant odx-d file contains a ref to the other functional group odx-d file. So the load file process failed.

Can we make more than one file as input to load odx-d? if we can load both base variant and functional group, we can avoid this issue; or can we make load function to ignore if we cannot link to the object from another file?

fix: README file

I find the README.md file having errors and I wish to correct it.

__message_format_lines crash with overlapping signals

I am not sure how that function works to fix the issue myself, but I could provide what the struct that caused the issue look like

All have byte position = 0

Param bit position bit length
1 0 2
2 2 2
3 3 2
4 5 1
5 6 1
6 7 1

Stack trace

            next_line = f"{(len(indent_for_byte_numbering) - 1 - len(str(byte))) * ' '}{byte} "

            for bit in range(8):
                assert 8 * byte + bit <= breakpoint

                if 8 * byte + bit == breakpoint:
                    # END-OF-PDU fields do not exhibit a fixed bit
                    # length, so they need special treatment here
                    dct = None
>                   if hasattr(params[i], 'dop'):
E                   IndexError: list index out of range

Is there a way to encode UDS service?

I knew there is a way to decode UDS service from a raw data, for instance :

raw_data = b"\x22\x10\x31"
decoded_message = ecu.decode(raw_data)
print(f"decoded message: {decoded_message}")

But is there anyway to encode UDS service?
so that it can be used to send the message to the ECU.

setup.py requires very old python-can dependancy

requires_list = [
"bitstruct >= 6.0.0",
"argparse_addons",
"PyInquirer",
"jinja2",
"python-can < 4.0",

Since python-can 4.0 and later is strongly recommended by the python-can community, what is the reason for odxtools to require an old version which has many unresolved issues?

Do you plan to be compatible with python-can 4.0 and later?

Better API documentation

The API documentation is generated from the python docstrings and it can be generated and inspected via

cd $ODXTOOLS_SRC_DIR/doc
make html
firefox _build/html/index.html

It IMO already looks decent, but it is sorely lacking in completeness and does not feature any "tutorial style" introductions. this should be changed in the medium to long term.

Andreas Lauser <[email protected]>, on behalf of MBition GmbH.
Provider Information

Program Code Syntax

Hi,
is there any particular reason for exactly those "types"?

class ProgCodeSyntax(Enum):

I could not find any information on valid or invalid types and I get an error on this line, which was working with an older version, because my odx file is using the type "txt" here. Not sure if the ODX is not following the norm or if 'txt' would need to be added as type here. Any advice?
Additionally, I think here, the string should be converted to all-upper case, just to prevent some false errors (in my case, the type was lower case, so even if there was 'TXT' available as type, is would still fail):
syntax_str = odxrequire(et_element.findtext("SYNTAX"))

Thanks a lot in advance for your feedback and also for the tool of course :)

improve readability of the output of the `list` subcommand

currently, the list subcommand produces output like this:

$ odxtools list --conformant -a examples/somersault.pdx
BASE-VARIANT 'somersault' (Receive ID: 0x7b, Send ID: 0x1c8)
 num services: 7, num DOPs: 7, num communication parameters: 11.
The services of the BASE-VARIANT 'somersault' are:
 do_backward_flips <ID: somersault.service.do_backward_flips>
  Message format of a request:
           7     6     5     4     3     2     1     0
        +-----+-----+-----+-----+-----+-----+-----+-----+
      0 | sid (8 bits)                                  |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      1 | backward_soberness_check (8 bits)             |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      2 | num_flips (8 bits)                            |
        +-----+-----+-----+-----+-----+-----+-----+-----+
   Parameter(short_name='sid', type='CODED-CONST', semantic=None, byte_position=0, bit_length=8, coded_value='0xbb')
   Parameter(short_name='backward_soberness_check', type='VALUE', semantic=None, byte_position=1, bit_length=8, dop_ref='somersault.DOP.soberness_check')
    DataObjectProperty('soberness_check', category='IDENTICAL', internal_type='A_UINT32', physical_type='A_UINT32')
   Parameter(short_name='num_flips', type='VALUE', semantic=None, byte_position=2, bit_length=8, dop_ref='somersault.DOP.num_flips')
    DataObjectProperty('num_flips', category='IDENTICAL', internal_type='A_UINT32', physical_type='A_UINT32')
  Number of positive responses: 1
  Message format of a positive response:
           7     6     5     4     3     2     1     0
        +-----+-----+-----+-----+-----+-----+-----+-----+
      0 | sid (8 bits)                                  |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      1 | num_flips_done (8 bits)                       |
        +-----+-----+-----+-----+-----+-----+-----+-----+
   Parameter(short_name='sid', type='CODED-CONST', semantic=None, byte_position=0, bit_length=8, coded_value='0xfb')

I think it would be more user friendly if this was something like

Variants:
 somersault:
  Type: 'BASE-VARIANT'
  Receive ID: 0x7b
  Send ID: 0x1c8
  Services:
   do_backward_flips
    Request parameters:
     sid:
      Type: 'CODED-CONST'
      Semantic: None
      Byte position: 0
      Bit length: 8
      Coded value: '0xbb'
   backward_soberness_check:
    Type: 'VALUE'
    Semantic: None
    Byte position=1
    Bit length: 8
    DOP: 'somersault.DOP.soberness_check'
  Message layout:
           7     6     5     4     3     2     1     0
        +-----+-----+-----+-----+-----+-----+-----+-----+
      0 | sid (8 bits)                                  |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      1 | backward_soberness_check (8 bits)             |
        +-----+-----+-----+-----+-----+-----+-----+-----+
      2 | num_flips (8 bits)                            |
        +-----+-----+-----+-----+-----+-----+-----+-----+
  Positive responses:
   [...]
[...]
  DOPs:
    num_flips:
     Category: 'IDENTICAL'
     Internal type: 'A_UINT32'
     Physical type: 'A_UINT32'
[...]

Andreas Lauser <[email protected]>, on behalf of MBition GmbH.
Provider Information

incorrect handling of ref with target in multiple odx files

The inheritance of the request in ODX2 works fine. But the DOP beeing refered to in the request in ODX2 exists in both ODX files in different versions. From my understanding, and the way other odx-interpreting tools handle this situation is, that the DOP of ODX1 is used, and the DOP of ODX2 is ignored/overwritten.
Odxtools though takes the DOP from ODX2 and ignores/overwrites the DOP from ODX1, which leads to the data of the DOP in ODX1 beeing lost.
image
image

Use findtext() instead of et_element.find("XML-ELEMENT").text

findtext() seems to be a better function. The documentation says

findtext(match, default=None, namespaces=None)

Finds text for the first subelement matching match. match may be a tag name or a [path](https://docs.python.org/3/library/xml.etree.elementtree.html#elementtree-xpath). Returns the text content of the first matching element, or default if no element was found. Note that if the matching element has no text content an empty string is returned. namespaces is an optional mapping from namespace prefix to full name. Pass '' as prefix to move all unprefixed tag names in the expression into the given namespace.

the parameters for "list" command may be wrong

When I use odxtools list xxx.pdx, it says:

Traceback (most recent call last):
  File "D:\Program Files\Python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "D:\Program Files\Python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\PythonEnvs\py38\lib\site-packages\odxtools\__main__.py", line 7, in <module>
    _main()
  File "D:\PythonEnvs\py38\lib\site-packages\odxtools\__init__.py", line 92, in _main
    _main.start_cli()
  File "D:\PythonEnvs\py38\lib\site-packages\odxtools\cli\main.py", line 54, in start_cli
    tool.run(args)
  File "D:\PythonEnvs\py38\lib\site-packages\odxtools\cli\list.py", line 190, in run
    print_global_neg_responses=args.all or args.global_neg_responses,
AttributeError: 'Namespace' object has no attribute 'global_neg_responses'

I found the args may be wrong in file https://github.com/mercedes-benz/odxtools/blob/main/odxtools/cli/list.py#L131.

The parser args:

    parser.add_argument(
        "-g",
        "--global-negative-responses",
        default=False,
        action="store_const",
        const=True,
        required=False,
        help="Print a list of the global negative responses for the selected ECUs.",
    )

but the code in the main entry https://github.com/mercedes-benz/odxtools/blob/f23b3ddc552313434711fa05d1141b4821497606/odxtools/cli/list.py#L190C74-L190C74:

        print_global_neg_responses=args.all or args.global_neg_responses,

Multiplexer does not inherit from DopBase

This causes any encode/decode to break with an unexpected kind of errors for example AttributeError: 'Multiplexer' object has no attribute 'bit_length'

Call stack

..\odxtools\odxtools\structures.py:33: in bit_length
    elif all(p.bit_length is not None for p in self.parameters):
..\odxtools\odxtools\structures.py:33: in <genexpr>
    elif all(p.bit_length is not None for p in self.parameters):
..\odxtools\odxtools\parameters\parameterwithdop.py:70: in bit_length
    return self.dop.bit_length
..\odxtools\odxtools\structures.py:33: in bit_length
    elif all(p.bit_length is not None for p in self.parameters):
..\odxtools\odxtools\structures.py:33: in <genexpr>
    elif all(p.bit_length is not None for p in self.parameters):

    @property
    def bit_length(self):
        if self.dop is not None:
>           return self.dop.bit_length
E           AttributeError: 'Multiplexer' object has no attribute 'bit_length'

Does TableKeyParameter not support yet?

Hi, friends.
Does TableKeyParameter not support encode or decode yet? I meet an error when I encode or decode ReadDID service
error- NotImplementedError: TableKeyParameter.is_required is not implemented yet.

cannot load pdx file in version 1.2.2

Hi,

I tried to load pdx with the latest version but it failed. It was working in previous version.
The following is the log.

File "C:\Users\tnm1pg1\AppData\Local\Programs\Python\Python39\lib\site-packages\odxtools\odxtypes.py", line 64, in from_string
return _ODX_TYPE_PARSERself.value
TypeError: int() argument must be a string, a bytes-like object or a number, not 'xml.etree.ElementTree.Element'


One more thing I would like to ask is there any example on how to send the UDS service to the ECU after loading the pdx? For example like Read Data by Identifier service.

Support for DOCTYPE and DOCREF

Currently the implementation only supports ID-REF, most OEM files make use of the DOCREF function.

The issue with not supporting this, is that some refs gets resolved from the wrong file.

Implement compu method category `TAB-INTP`

Currently, only the categories IDENTICAL, TEXTTABLE, LINEAR and SCALE-LINEAR are supported.
Implement the compu mthod category TAB-INTP, i.e.,

  • define a datastructure (similar to the existing ones in compumethods)
  • read it from odx
  • write them to pdx (i.e., add them to the pdx template)
  • add example to somersaultecu.py
  • add tests

For definition of the TAB-INTP computational method see ASAM MCD-2 spec p. 122.

Katrin Bauer <[email protected]>, on behalf of MBition GmbH.
Provider Information

CAN send/receive ids are null

Hi team - We were using the commit 74163b455f070f8fd925955bb33911bfcd83a092 which was a few weeks old. Recently upgraded to the latest version at 3b33829178dc854e8a578ff66be7459137017391

With the new version of the library, the can send/receive ids appear as None. This was working with the previous version.

A sample test program.

import odxtools

database = odxtools.load_file("./Sibros_TCU_V1.0.1.pdx", enable_candela_workarounds=True)
for dlc in database.diag_layer_containers:
    for ecu_variant in dlc.base_variants:
        print("physical_request_id = ", hex(ecu_variant.get_receive_id()))
        print("physical_response_id = ", hex(ecu_variant.get_send_id()))

Error:

Traceback (most recent call last):
  File "test.py", line 6, in <module>
    print("physical_request_id = ", hex(ecu_variant.get_receive_id()))
TypeError: 'NoneType' object cannot be interpreted as an integer

Gitlab does not allow me to upload the PDX files. Probably you can test with the PDX files that you have.
image

Question: Are these APIs deprecated?. Should I be looking at a different function?. Please advise.

pdx file list error

I encounter error when I try to list pdx file.

It complained that it don't know the type NRC-CONST

Logging:
File "C:\Users\tnm1pg1\AppData\Local\Programs\Python\Python39\lib\site-packages\odxtools\parameters\readparameter.py", line 199, in read_parameter_from_odx
raise NotImplementedError(f"I don't know the type {parameter_type}")
NotImplementedError: I don't know the type NRC-CONST

Do I miss anything?
The command that I use is "odxtools list -a xxx.odx"

How to encode a ReadDataByIdentifier request?

Hi guys,
I'm trying to encode a ReadDataByIdentifier request. It should provide a DID, how can I do this?
I don't have an ecu in my pdx file. So I tried to use function like db.diag_layers['myLayer'].services['ReadDataByIdentifier'].request.encode(). But I don't know how to pass the DID to the function encode()
I can get DID list from db.diag_layers['myLayer'].diag_data_dictionary_spec.tables['TABLE_Service22_DataID'] for ReadDataByIdentifier because its did number is 22.

Can we get rid of assert to make process not fail?

There are a lot of assert keywords at repo and they make the process exit and raise an exception. I have a PDX file with one of data-object-prop has value "0.01" and type as "A_UINT32". It goes through this code

def parse_int(value: str) -> int: try: return int(value) except ValueError: v = float(value) assert v.is_integer() return int(v)

since the value is "0.01" and it raises an exception when exec " assert v.is_integer()". I agree we should do validation, but is it worth making the whole flow fail?

Maybe we can do some changes. Here are my thoughts:

Option 1: Instead of using assert, we can use the if statement to check if the value type is expected. If not expected, we can add an error description returned.

Option 2: Instead of using assert, we use an if statement, if the value is not expected, we should ignore this part of the information.

Does it support ODX 2.2.0?

When I try to load a pdx file. it throw the error:

File "D:\PythonEnvs\py38\lib\site-packages\odxtools\load_file.py", line 9, in load_file
return load_pdx_file(file_name)
File "D:\PythonEnvs\py38\lib\site-packages\odxtools\load_pdx_file.py", line 11, in load_pdx_file
container = Database(pdx_zip=u)
File "D:\PythonEnvs\py38\lib\site-packages\odxtools\database.py", line 86, in init
self.refresh()
File "D:\PythonEnvs\py38\lib\site-packages\odxtools\database.py", line 118, in refresh
dlc._finalize_init(self._odxlinks)
File "D:\PythonEnvs\py38\lib\site-packages\odxtools\diaglayercontainer.py", line 162, in _finalize_init
dl._finalize_init(odxlinks)
File "D:\PythonEnvs\py38\lib\site-packages\odxtools\diaglayer.py", line 215, in _finalize_init
self.diag_layer_raw._resolve_snrefs(self)
File "D:\PythonEnvs\py38\lib\site-packages\odxtools\diaglayerraw.py", line 285, in _resolve_snrefs
obj._resolve_snrefs(diag_layer)
File "D:\PythonEnvs\py38\lib\site-packages\odxtools\diagdatadictionaryspec.py", line 169, in _resolve_snrefs
obj._resolve_snrefs(diag_layer)
File "D:\PythonEnvs\py38\lib\site-packages\odxtools\endofpdufield.py", line 160, in _resolve_snrefs
self._structure = dops[self.structure_snref]
File "D:\PythonEnvs\py38\lib\site-packages\odxtools\nameditemlist.py", line 89, in getitem
return self._typed_dict[key]
KeyError: 'STRUCT_Service19_DTCExtRecord'
python-BaseException

I'm wondering if it's because my pdx file is in ODX 2.2.0 format?

ComplexComparam `from_et()` does not behave properly and skips over subparams

I noticed the recent PR #233 introduced a regression in how the ComplexComparam class is initialized in the from_et() function, causing the generated XML for ComplexComparams to be incorrect.

I have a couple findings:

  1. I was noticing that in my output XML, sub-parameters for COMPLEX-COMPARAM tags were missing. The source XML that was being parsed came from Vector software, and not all of the sub-parameters have associated COMPLEX-PHYSICAL-DEFAULT-VALUE tags.

    I wasn't quite clear on the meaning of the comment in the code:

    # the next element in the list *may* hold the physical
    # default value for the sub-parameter. if it is not the
    # correct tag, skip it! Note that the ODX specification
    # *only* allows to specify COMPLEX-PHYSICAL-DEFAULT-VALUE
    # tags here, even if the sub-parameter was a simple
    # parameter. This is probably a bug in the ODX
    # specification...
    so I took a look at the ISO 22901-1 (2008) document. On page 351 of the ISO document, the XML schema suggests to me that the COMPLEX-PHYSICAL-DEFAULT-VALUE tag is not required to be there (minimum occurrence of 0):

    <xsd:element maxOccurs="1" minOccurs="0" type="COMPLEX-PHYSICAL-DEFAULT-VALUE" name="COMPLEX-
    PHYSICAL-DEFAULT-VALUE"/>

    so based on this, this seems like it might be wrong:

    tmp_it = elem_it
    try:
    elem = next(elem_it)
    except StopIteration:
    break
    if elem.tag != "COMPLEX-PHYSICAL-DEFAULT-VALUE":
    subparam.physical_default_value = create_complex_value_from_et(elem)
    elem_it = tmp_it
    continue

    If this is the last sub-parameter, the loop will break without appending the sub-parameter to the subparams list.

    In addition, the conditional may be inverted, since if it isn't a complex physical default value, it calls create_complex_value_from_et() and then continues the loop without appending the sub-parameter to the subparams list.

    Both of these issues cause the generated XML to be missing the sub-parameters inside COMPLEX-COMPARAM tags.

  2. The use of iterators doesn't seem to result in desired behavior:

    tmp_it = elem_it

    tmp_it actually stores a reference to elem_it and doesn't actually save the position of the iterator, so every other element gets skipped, and the line
    elem_it = tmp_it
    doesn't actually restore the position of the iterator.

    Other than itertools.tee(), I'm not sure of an elegant way to temporarily store the position of the iterator, so I think it makes sense to just refactor the loop to use the iterator converted to a list.

Here is my fix that I made for from_et(), I tested it in my project and it seems to work as intended.

        elems = list(iter(et_element))
        i = 0
        while True:
            if i >= len(elems):
                break

            if elems[i].tag not in ("COMPARAM", "COMPLEX-COMPARAM"):
                i += 1
                continue

            subparam = create_any_comparam_from_et(elems[i], doc_frags)
            ...
            if i + 1 < len(elems):
                if elems[i + 1].tag == "COMPLEX-PHYSICAL-DEFAULT-VALUE":
                    subparam.physical_default_value = create_complex_value_from_et(elems[i + 1])
                    i += 1

            subparams.append(subparam)
            i += 1

Thanks in advance for your consideration!

Can we decode a response hex payload given the PDX file?

I know there is a ecu.decode logic, but the input has to be bytes array. It looks like we have to convert the long hex payload to bytes array, can we:

  1. provide another decode that given long hex payload and pdx file?
  2. Make a function to divide the long response hex payload to bytes array?

code generators for c++ and other languages

to be able to implement diagnostic services with low-cost hardware (i.e., microcontrollers for embedded systems or machines exhibiting less than 28 cores in the cloud world), it is necessary to use a compiled language, most commonly C++ (or C, but IMO even for microcontrollers restricting oneself to this is pretty masochistic). The way odxtools could help on this front is to generate the c++ boiler plate code necssarry for encoding and decoding UDS telegrams which needs to be filled by the developer wishing to implement a diagnostic service. This is basically the same approach as cantools generate_c_source, but I'd warmly recommend to use a proper template engine such as jinja for this.

On the c++ side, this could look similar to:

#include <FunkyECU/SomersaultBaseService.hh>

namespace FunkyECU {
    class SomersaultService : public SomersaultBaseService<SomersaultService>
    {
    public:
        SomersaultService(IsoTpConnection& conn)
          : SomersaultBaseService(conn)
          , num_done_(0)
        {}

        void request_received(const SomersaultRequestParams& rq)
        {
           num_done_ += rq.param_NumLoops;

            if (num_done_ < 10) {
                SomersaultPositiveResponse resp;
                resp.param_NumLeft = 10 - num_done_;
                this->sendResponse(resp);
            }
            else {
                // i am dizzy now!
                this->sendResponse(SomersaultNegativeResponse);
            }
        }

    private:
        int num_done_;
    };
}

int main()
{
   IsoTpConnection conn(SomersaultService::receive_id(), SomersaultService::send_id());
   FunkyECU::SommersaultService service(conn);
   while (true) {
       IsotpTelegram telegram = conn.recv()
       if (service.applies(telegram))
           service.handle(telegram);
   }
   return 0;
}

Andreas Lauser <[email protected]>, on behalf of MBition GmbH.
Provider Information

Bit Length Calculations are slightly off

Hi team - We were using the commit 74163b455f070f8fd925955bb33911bfcd83a092 which was a few weeks old. Recently upgraded to the latest version at 3b33829178dc854e8a578ff66be7459137017391

I noticed that the bit length calculation on the request parameter is off by 3 bits (27 bits). It was 32 bits with the previous commit. However, it does show properly on the pictorial representation & the actual values in the dops, but the cumulative value in the structure seems to be off.

Screen Shot 2022-10-19 at 4 39 50 PM

Unfortunately, I am not in a position to share the PDX file. I know there is not enough to debug, with just a screenshot. :(

Please let me know if you have pointers on what could have caused this. I'll update the ticket if I find something.

add comparison tool

having a tool which compares two ECU versions and/or complete databases would be nice. something along the lines of:

# compare ECUs
odxtools compare PDX_FILE.pdx somersault_lazy somersault_assiduous

# compare complete files
odxtools compare PDX_FILE_OLD.pdx PDX_FILE_NEW.pdx

Andreas Lauser <[email protected]>, on behalf of MBition GmbH.
Provider Information

Hardcoded DOCTYPE="CONTAINER" in PARENT-REF tags

The DOCTYPE="CONTAINER" being hardcoded as an attribute of the PARENT-REF tag in the jinja template seems to cause issues with my use case, where I have PARENT-REFs that point to PROTOCOLs.

With this generated XML:
image

image

The inclusion of these two attributes leads to Vector ODXStudio determining that it is a broken link, even if the ID-REF is correct, because I don't believe a PROTOCOL is technically a CONTAINER?

If I eliminate the hardcoded DOCTYPE attribute, I no longer have broken links to the PROTOCOLs.

DOCREF="{{par.layer.short_name}}"
DOCTYPE="CONTAINER"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.