Code Monkey home page Code Monkey logo

upydev's Introduction

uPydev

PyPI versionPyPI license

Command line tool for MicroPython devices

uPydev is an acronym of 'MicroPython device', and it is intended to be a command line tool to make easier the development, prototyping and testing process of devices based on boards running MicroPython. It is intended to be cross-platform and connection agnostic (Serial, WiFi and Bluetooth Low Energy).

Features:

  • Tools to allow configuration, management, communication and control of MicroPython devices
  • Command line Autocompletion
  • File IO operations (upload, download one or multiple files, recursively sync directories...)
  • SHELL-REPL modes: Serial, WiFi (SSL/WebREPL), BLE
  • OTA* Firmware updates WiFi (TCP/SSL), BLE (* esp32 only)
  • Custom commands for debugging, testing and prototyping
  • Custom tasks yaml files that can be played like ansible
  • Run tests in device with pytest and parametric tests or benchmarks using yaml files
  • Group mode to operate with multiple devices

Getting Started

Installing :

$ pip install upydev or $ pip install --upgrade upydev to update to the latest version available

Create a configuration file:

upydev will use local working directory configuration unless it does not find any or manually indicated with -g option.

  • To save configuration in working directory:

    $ upydev config -t [DEVICE ADDRESS] -p [PASSWORD/BAUDRATE], where [DEVICE ADDRESS] must be a valid :

    • IP/HOSTNAME

    • SERIAL ADDRESS

    • MAC ADDRESS/ UUID

    Hostname must be set in device, e.g. in esp32 default is esp32.local

    -p is set to 115200 by default, so it is not necessary unless using a different baudrate

    MAC address format will depend on OS system (e.g. Linux uses MAC format 'XX:XX:XX:XX:XX:XX', and macOS uses UUID format 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX')

    e.g.

    # WiFi
    $ upydev config -t 192.168.1.53 -p mypass
    
    # SERIAL
    $ upydev config -t /dev/tty.usbmodem387E386731342
    
    # BLE
    $ upydev config -t 9998175F-9A91-4CA2-B5EA-482AFC3453B9

    Default device name is upydevice, to set a custom name use -@ flag as

 $ upydev config -t 192.168.1.53 -p mypass -@ mycustomdevice

To check configuration upydev or upydev check

$ upydev
Device: mycustomdevice
Address: 192.168.1.53, Device Type: WebSocketDevice

Or to get more information if the device is online

$ upydev -i
Device: mycustomdevice
WebSocketDevice @ ws://192.168.1.53:8266, Type: esp32, Class: WebSocketDevice
Firmware: MicroPython v1.19.1-285-gc4e3ed964-dirty on 2022-08-12; ESP32 module with ESP32
(MAC: 30:ae:a4:23:35:64, RSSI: -45 dBm)
  • To save configuration globally use -g flag: $ upydev config -t [DEVICE ADDRESS] -p [PASSWORD/BAUDRATE] -g

    e.g.

$ upydev config -t 192.168.1.53 -p mypass -g
  • To save configuration in a global group use -gg flag: $ upydev config -t [DEVICE ADDRESS] -p [PASSWORD/BAUDRATE] -gg -@ mydevice

    e.g.

$ upydev config -t 192.168.1.53 -p mypass -gg -@ mydevice
  • [Optional] Use register command to define a function in ~/.bashrc or ~/.profile
$ upydev register -@ mydevice

Reload ~/.bashrc or ~/.profile, e.g. ($ source ~/.profile)

Now mydevice will accept any args and pass them to upydev, as well as autocompletion of args, e.g.

$ mydevice
Device: mydevice
Address: 192.168.1.53, Device Type: WebSocketDevice

Or if the device is connected.

$ mydevice -i
Device: mydevice
WebSocketDevice @ ws://192.168.1.53:8266, Type: esp32, Class: WebSocketDevice
Firmware: MicroPython v1.19.1-285-gc4e3ed964-dirty on 2022-08-12; ESP32 module with ESP32
(MAC: 30:ae:a4:23:35:64, RSSI: -45 dBm)

To see registered devices do:

$ upydev lsdevs
Device: mydevice
Address: 192.168.1.53, Device Type: WebSocketDevice

Which adds the lsdevs command to ~.profile too. So after reloading again:

$ lsdevs
Device: mydevice
Address: 192.168.1.53, Device Type: WebSocketDevice

Finally to enter device shell-repl mode do:

$ upydev shl@mydevice
shell-repl @ mydevice
WebREPL connected
WARNING: ENCRYPTION DISABLED IN THIS MODE

MicroPython v1.19.1-285-gc4e3ed964-dirty on 2022-08-12; ESP32 module with ESP32
Type "help()" for more information.

- CTRL-k to see keybindings or -h to see help
- CTRL-s to toggle shell/repl mode
- CTRL-x or "exit" to exit
esp32@mydevice:~ $

or if the device is registered:

$ mydevice shl
shell-repl @ mydevice
WebSecREPL with TLSv1.2 connected
TLSv1.2 @ ECDHE-ECDSA-AES128-CCM8 - 128 bits Encryption

MicroPython v1.19.1-285-gc4e3ed964-dirty on 2022-08-12; ESP32 module with ESP32
Type "help()" for more information.

- CTRL-k to see keybindings or -h to see help
- CTRL-s to toggle shell/repl mode
- CTRL-x or "exit" to exit
esp32@mydevice:~ $

To enable WebSocket over TLS or wss check WebSocket (ws) / WebSocket Secure (wss) TLS

Once the device is configured see next section or read Usage documentation to check which modes and tools are available.

Or if you are working with more than one device continue with this section to create a group configuration.


uPydev Usage:

Requirement : Needs REPL to be accessible (see Getting Started)

Usage:

$ upydev [Mode] [options] or upydev [upy command] [options]

This means that if the first argument is not a Mode keyword or a upy command keyword it assumes it is a 'raw' upy command to send to the upy device

Help: $ upydev h, $ upydev help, $ upydev -h or $ upydev [command] -h

Example: Mode

$ upydev put dummy.py, $ upydev get dummy.py

Example: uPy command

$ upydev info

Example: Raw commands

$ upydev "my_func()"

$ upydev 2+1

$ upydev "import my_lib;foo();my_var=2*3"

To see documentation check Upydev readthedocs

upydev's People

Contributors

carglglz avatar dsl400 avatar naohiro2g avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

upydev's Issues

SSLWebREPL key generation assumes host network interface is called 'en0' and fails if its not

When generating private key and certs for SSLWebREPL with

upydev sslgen_key -tfkey

It failed with:

Generating SSL ECDSA key and certificates...
Getting unique id...
ID: 30aea42313e0
Passphrase:
Traceback (most recent call last):
  File "/home/chris/.espressif/python_env/idf3.3_py3.6_env/bin/upydev", line 5147, in <module>
    get_ssl_keycert(args.t, args.p)
  File "/home/chris/.espressif/python_env/idf3.3_py3.6_env/bin/upydev", line 4176, in get_ssl_keycert
    ssl_ECDSA_key_certgen(ip, passwd, dir=upydev.__path__[0]+'/')
  File "/home/chris/.espressif/python_env/idf3.3_py3.6_env/bin/upydev", line 1730, in ssl_ECDSA_key_certgen
    host_ip = netifaces.ifaddresses('en0')[netifaces.AF_INET][0]['addr']
ValueError: You must specify a valid interface name.

When looking at upydev#L1730 I could see it expected network interface of the host to be 'en0', which does not exist in my env (Ubuntu WSL in Windows 10). I edited the file and changed this to a network interface from ifconfig and the key generation and upload then succeeded.

-h doesn't work

(.venv) maho@udlaptop:~/workspace/upydev$ upydev -h
Traceback (most recent call last):
  File "/home/maho/workspace/garageopen/.venv/bin/upydev", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/home/maho/workspace/upydev/upydev_dir/bin/upydev", line 1432, in <module>
    parser.add_argument("-@", help=help_dv, required=False).completer = ChoicesCompleter(see_global_devs())
  File "/home/maho/workspace/upydev/upydev_dir/bin/upydev", line 197, in see_global_devs
    with open('{}/UPY_G.config'.format(upydev.__path__[0]), 'r', encoding='utf-8') as group:
FileNotFoundError: [Errno 2] No such file or directory: '/home/maho/workspace/upydev/upydev/UPY_G.config'

`Watch` mode would be nice

upydev watch -fre ./src/*.py would watch for any changes in files in a project directory. If change occurred it will upload only the changed file.

local echo is in white

I tried sh_srepl and the connection established successfully but I got no echo back to the terminal.
No, it was there but in white. If I changed the color theme from light to dark, it turned to visible.

It seems the character color for the local echo is hard coded to white.

The target is ESP32 and I'm on macOS Mojave with iterm2 app.

Refactor Readme

First of all: this tool is amazing! Thank You!
But the readme file is huge, it is hard to find how to do basic things.
IMHO: Readme should be decluttered and must include most basic things, other specific options and functions should be on gitbook or wiki.

If You're ok with this, I'll try to make a PR.
Thanks again for this utility and please continue <3

repl multi line statements not working

shell-repl @ upydev
WebREPL connected
WARNING: ENCRYPTION DISABLED IN THIS MODE

MicroPython v1.18 on 2022-01-17; ESP module (1M) with ESP8266
Type "help()" for more information.

  • CTRL-k to see keybindings or -h to see help
  • CTRL-s to toggle shell/repl mode
  • CTRL-x or "exit" to exit
    esp8266@temp1:~ $

for i in [1,2]:
print(i)

............
....

no key combination will get you out of this situation
you have to close the terminal and start another one

Cryptographic API Misuse Vulnerability: Do not use insecure algorithm(textbook or padding PKCS 1v1.5) for RSA

Description:

In the upydev repository, specifically within the keygen.py script, the RSA algorithm is implemented with PKCS 1v1.5 padding, which is known to be insecure and vulnerable to Bleichenbacher](https://link.springer.com/content/pdf/10.1007/bfb0055716.pdf) attack which is a chosen-ciphertext attack that exploits vulnerabilities within the RSA PKCS1v1.5 padding scheme. The use of this padding scheme can lead to security issues where an attacker may exploit the vulnerability to decipher encrypted messages or forge signatures.

Affect Version

upydev 0.4.3

Location:

https://github.com/Carglglz/upydev/blob/master/upydev/keygen.py#L919
https://github.com/Carglglz/upydev/blob/master/upydev/keygen.py#L1176
https://github.com/Carglglz/upydev/blob/master/upydev/keygen.py#L1253
https://github.com/Carglglz/upydev/blob/master/upydev/keygen.py#L1311
https://github.com/Carglglz/upydev/blob/master/upydev/keygen.py#L1125

Expected Behavior:
The cryptographic module should utilize a secure padding scheme for RSA such as OAEP (Optimal Asymmetric Encryption Padding) which is currently recommended as a safer alternative to PKCS 1v1.5.

Actual Behavior:
The current implementation of the RSA encryption and signature verification in keygen.py uses PKCS 1v1.5 padding, which is outdated and has known vulnerabilities that compromise the security of cryptographic operations.

Reference:

CWE-780: Use of RSA Algorithm without OAEP

Recommendations:

It is recommended to use OAEP (Optimal Asymmetric Encryption Padding) as the encryption scheme and PSS (Probabilistic Signature Scheme) as the signature scheme, as they offer better security.

  1. Refactor the RSA implementation to use OAEP padding for encryption and PSS padding for signature, which are both provided by the RSA module in standard libraries like PyCryptodome and cryptography.io.
    2 . Conduct a thorough audit of the cryptographic operations throughout the application to ensure that secure practices are being followed.

It's crucial for the security of upydev users that this issue is addressed promptly, as cryptographic weaknesses can have severe implications for the privacy and integrity of communications facilitated by this tool.

message = b"encrypted data"
ciphertext = public_key.encrypt(
    message,
    padding.OAEP(
        mgf=padding.MGF1(algorithm=hashes.SHA256()),
        algorithm=hashes.SHA256(),
        label=None
    )
)

https://cryptography.io/en/latest/hazmat/primitives/asymmetric/rsa/#encryption

Please let me know if you require any further information or assistance in addressing this issue.

Shows `File Uploaded` message while throwing `IsADirectoryError`

I'm using upydev put -fre src/*

Traceback (most recent call last):
  File "/home/stsdc/.local/bin/upytool", line 385, in <module>
    main()
  File "/home/stsdc/.local/bin/upytool", line 379, in main
    put_file(ws, src_file, dst_file)
  File "/home/stsdc/.local/bin/upytool", line 178, in put_file
    with open(local_file, "rb") as f:
IsADirectoryError: [Errno 21] Is a directory: 'src/ble'
File Uploaded!

BTW how to upload a folder? My project structure:

โ”œโ”€โ”€ dev-requirements.txt
โ”œโ”€โ”€ micropy.json
โ”œโ”€โ”€ pymakr.conf
โ”œโ”€โ”€ requirements.txt
โ”œโ”€โ”€ src
โ”‚ย ย  โ”œโ”€โ”€ ble
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ ble_advertising.py
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ ble.py
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ ble_temperature.py
โ”‚ย ย  โ”œโ”€โ”€ boot.py
โ”‚ย ย  โ””โ”€โ”€ main.py
โ””โ”€โ”€ upydev_.config

sh_srepl needs conf file but wrepl don't

When I removed global conf and local conf,
upydev sh_srepl causes an error, even if port and password were set inline, but wrepl is ok with it.

$ upydev sh_srepl -port /dev/tty.SLAB_USBtoUART -p pass

upydev_.config file not found, please provide target and         password or create config file with command "config" (see help)

Dependency `requests` missing

I ran into the error:

$ upydev -h
Traceback (most recent call last):
  File "/Users/thomas/Programming/HA/MicroPython/venv/bin/upydev", line 17, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

which was resolved by:
pip install requests

I am using python 3.7.4 from MacPorts wrapped via venv under Mac OS Mojave

adc_config

In the adc_config, Pin needs to be imported as well for my ESP32 board with Micropython.

Upyutils should use on the device only default functionality

It would be nice to be able to use upydev get main.py without uploading aditional files onto the device

get the file tree from the device and make all the filtering on the pc

this concept should be applied to other scripts where possible.

wrepl, sh_srepl, and conf: ip and port should be separately stored in conf.

When I was switching back and forth between wrepl via WiFi and sh_srepl via USB-serial, I found a glitch.

conf for sh_srepl via USB-serial:
upydev config -t /dev/tty.SLAB_USBtoUART -p pass 

conf for wrepl via WiFi
upydev config -t 192.168.1.32  -p pass 

and in the conf file, port and ip is stored bound with the same key "ip". So I have to specify the port or ip inline like follows:

upydev sh_srepl -port /dev/tty.SLAB_USBtoUART
upydev wrepl -t 192.168.1.32

It could be bettrt if port was stored as "port".

Shows file uploaded, even there an error occurred

Uploading file main.py...
op:put, host:192.168.1.102, port:8266, passwd:****.
src/main.py -> /main.py
Traceback (most recent call last):
  File "/home/stsdc/.local/bin/upytool", line 385, in <module>
    main()
  File "/home/stsdc/.local/bin/upytool", line 363, in main
    s.connect(addr)
ConnectionRefusedError: [Errno 111] Connection refused
File Uploaded!

config location is screwed

If I use upydev in a virtual environment utilising poetry:

$ upydev config -t /dev/ttyACM3 -g -@ pybd
SerialDevice pybd settings saved globally!
$  pybd
[Errno 2] No such file or directory: '/home/ser/.cache/pypoetry/virtualenvs/mymgnss-enA7ukaw-py3.12/lib/python3.11/site-packages/upydev/UPY_G.config'
Traceback (most recent call last):
  File "/home/ser/.cache/pypoetry/virtualenvs/mymgnss-enA7ukaw-py3.12/lib/python3.11/site-packages/upydev/devicemanagement.py", line 476, in devicemanagement_action
    target, passwd = address_entry_point(dev, args=args)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ser/.cache/pypoetry/virtualenvs/mymgnss-enA7ukaw-py3.12/lib/python3.11/site-packages/upydev/devicemanagement.py", line 313, in address_entry_point
    with open('{}.config'.format(group_file), 'r', encoding='utf-8') as group:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/ser/.cache/pypoetry/virtualenvs/mymgnss-enA7ukaw-py3.12/lib/python3.11/site-packages/upydev/UPY_G.config'

in summary, upydev should use ~/.config directory to store config :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.