Code Monkey home page Code Monkey logo

micropython_ota's Introduction

micropython-ota

GitHub Workflow Status Python Versions GitHub PyPI

Introduction

Micropython library for upgrading code over-the-air (OTA)

Preparation

For OTA updates to work an HTTP/HTTPS server like Apache or nGinx is required to be running and accessible by the device. This server can serve multiple devices and multiple projects at once. There are two supported directory structures of which one must be provided for the OTA updates to work:

  1. Version as prefix (default)

    server-root/
    |- <project_name>/
    |  |- version
    |  |- <version>_<filename1>
    |  |- <version>_<filename2>
    |  |- ...
    |- <project_name>/
       |- version
       |- <version>_<filename1>
       |- <version>_<filename2>
       |- ...
    
  2. Version as subdirectory (by setting the parameter use_version_prefix to False, see Usage)

    server-root/
    |- <project_name>/
    |  |- version
    |  |- <version_subdir>
    |     |- <filename1>
    |     |- <filename2>
    |     |- ...
    |- <project_name>/
       |- version
       |- <version_subdir>
          |- <filename1>
          |- <filename2>
          |- ...
    

For each project a directory must exist in the server's document root. Inside this directory a file "version" exists containing the version-tag to be pulled by the devices, e.g. v1.0.0. The source code files to be pulled by the devices are placed either right next to the version-file, prefixed by the version-tag, or in a subdirectory named with the version-tag. This structure also provides the ability to do a rollback by simply changing the version-tag in the version-file to an older version-tag, as long as the relevant source code files still reside in the expected location.

In the following example two projects "sample" and "big_project" are configured, using the default, version-prefixed directory structure:

server-root/
|- sample/
|  |- version <-- containing v1.0.1
|  |- v1.0.0_boot.py
|  |- v1.0.0_main.py
|  |- v1.0.1 boot.py
|  |- v1.0.1 main.py
|- big_project/
   |- version <-- containing v1.0.0
   |- v1.0.0_boot.py
   |- v1.0.0_main.py
   |- v1.0.0_data.py

Installation

The library can be installed using upip or mip. Ensure that the device is connected to the network.

Installation using upip (Micropython < 1.19)

import upip
upip.install('micropython-ota')

Installation using mip (Micropython >= 1.19)

Py-file

import mip
mip.install('github:olivergregorius/micropython_ota/micropython_ota.py')

Cross-compiled mpy-file

NOTE: Set the release_version variable accordingly.

import mip
release_version='vX.Y.Z'
mip.install(f'https://github.com/olivergregorius/micropython_ota/releases/download/{release_version}/micropython_ota.mpy')

Usage

This library provides two methods for

  1. handling code updates during boot (ota_update) and
  2. checking for code updates at regular intervals (check_for_ota_update).

The ota_update method might be called in the boot.py file, right after the network connection has been established:

import micropython_ota

# connect to network

ota_host = 'http://192.168.2.100'
project_name = 'sample'
filenames = ['boot.py', 'main.py']

micropython_ota.ota_update(ota_host, project_name, filenames, use_version_prefix=True, hard_reset_device=True, soft_reset_device=False, timeout=5)

That's it. On boot the library retrieves the version-file from http://192.168.2.100/sample/version and evaluates its content against a locally persisted version-file. (Of course, on the first run the local version-file does not exist, yet. This is treated as a new version being available.) If the versions differ, the source code files listed in filenames are updated accordingly and on success the local version-file is updated as well. If the use_version_prefix is set to True (default) the library expects the 'Version as prefix' directory structure on the server, otherwise it expects the 'Version as subdirectory' directory structure (see Preparation). If the hard_reset_device-flag is set to True (default) the device will be reset after the successful update by calling machine.reset(). For just soft-resetting the device the flag soft_reset_device can be set to True (defaults to False), taking precedence. This will call the machine.soft_reset()-method. The timeout can be set accordingly, by default its value is 5 seconds.

For regular checking for code updates the method check_for_ota_update might be called in the course of the regular application logic in main.py, e.g.:

import micropython_ota
import utime

ota_host = 'http://192.168.2.100'
project_name = 'sample'

while True:
    # do some other stuff
    utime.sleep(10)
    micropython_ota.check_for_ota_update(ota_host, project_name, soft_reset_device=False, timeout=5)

In this case on each iteration the library checks for a new version as described above and resets the device if a new version is available. By default a hard-reset is performed (by calling machine.reset()). By setting the flag soft_reset_device to True (defaults to False) the device will just be soft-reset (by calling machine.soft_reset()). After the reset the ota_update-method called in the boot.py performs the actual update. This method accepts the timeout setting, too, by default it is set to 5 seconds.

HTTP(S) Basic Authentication

ota_update() and check_for_ota_update() methods allow optional user and passwd parameters. When specified the library performs a basic authentication against the server hosting the source files. Use of HTTPS (versus HTTP) is very highly recommended when using basic authentication as, otherwise, the resulting username and password are sent as plain text i.e. completely unsecure.

Here is the same example as above, but using HTTPS and Basic Authentication:

import micropython_ota

# connect to network

ota_host = 'https://example.com'
project_name = 'sample'
filenames = ['boot.py', 'main.py']
user = 'otauser'
passwd = 'topsecret' # it's best to place this credential is a secrets.py file

micropython_ota.ota_update(ota_host, project_name, filenames, user=user, passwd=passwd, use_version_prefix=True, hard_reset_device=True, soft_reset_device=False, timeout=5)

There are plenty of tutorials online on how to set up secured HTTP file access on your webserver, but the basic steps are:

  • get and install an SSL certificate (Let's Encrypt is by far the best choice)
  • enable HTTPS access on your web server
  • prevent directories from listing files
  • enable HTTP Basic Authentication password protection on target directories

micropython_ota's People

Contributors

olivergregorius avatar

Stargazers

Elias Nektarios Dionysopoulos avatar Giorgos Karampitsakis avatar kemalcan bora avatar Danilo Ješić avatar  avatar  avatar  avatar Åre avatar Kaskabas avatar Reik Braune avatar Anto GG avatar  avatar Markus Schubert avatar stars avatar  avatar  avatar Max Mill avatar Heri avatar Nathan Hollis avatar  avatar 了空 avatar  avatar  avatar Tom Hensel avatar Pax-NL avatar LexxM3 avatar

Watchers

Pax-NL avatar LexxM3 avatar Max Mill avatar  avatar

micropython_ota's Issues

Subdirectories for Web Templates etc.

I have subdirectories for templates, etc. used for a web server. This does not currently deal with these.
I have working code for my use-case but it my not be robust for the general case.
I'll do a PR at some time, probably.

Provide flag for soft-resetting device

Currently the device is reset by calling machine.reset(). It should also be possible to simply soft-resetting the device by calling machine.soft_reset().
A flag should be added enabling either hard- or soft-reset.

Update README.md including instructions installing lib via mip

Since Micropython 1.19 upip was replaced with mip as package installation manager.
This library can now be installed with e.g. mip.install('github:olivergregorius/micropython_ota/micropython_ota.py', version='v1.1.0')

This should be documented in the README.md

Connection problems while updating may corrupt software

As the title says, since the files are replaced as they are downloaded, the software can get into an inconsistent state.
I suggest downloading all the files to a temporary folder and then replacing them once all of them have been downloaded.
This should significantly reduce the risk of corruption if the connection is not reliable enough.

Here is how I implemented it in my version:

# def ota_update(host, project, filenames, use_version_prefix=True, user=None, passwd=None, hard_reset_device=True, soft_reset_device=False, timeout=5) -> None:          # REMOVED
def ota_update(host, project, filenames, tempdir, *,use_version_prefix=True, user=None, passwd=None, hard_reset_device=True, soft_reset_device=False, timeout=5) -> None: # ADDED
    all_files_found = True
    auth = generate_auth(user, passwd)
    prefix_or_path_separator = '_' if use_version_prefix else '/'
    try:
        version_changed, remote_version = check_version(host, project, auth=auth, timeout=timeout)
        if version_changed:
            try: uos.mkdir(tempdir) # ADDED
            except: pass            # ADDED
            for filename in filenames:
                if auth:
                    response = urequests.get(f'{host}/{project}/{remote_version}{prefix_or_path_separator}{filename}', headers={'Authorization': f'Basic {auth}'}, timeout=timeout)
                else:
                    response = urequests.get(f'{host}/{project}/{remote_version}{prefix_or_path_separator}{filename}', timeout=timeout)
                response_status_code = response.status_code
                response_text = response.text
                response.close()
                if response_status_code != 200:
                    print(f'Remote source file {host}/{project}/{remote_version}{prefix_or_path_separator}{filename} not found')
                    all_files_found = False
                    continue
                # with open(filename, 'w') as source_file:              # REMOVED
                with open(f"{tempdir}/{filename}", 'w') as source_file: # ADDED
                    source_file.write(response_text)                    # ADDED
            if all_files_found:
                for filename in filenames:                                            # ADDED
                    temp_filename = f"{tempdir}/{filename}"                           # ADDED
                    with open(temp_filename,"r") as temp, open(filename,"w") as file: # ADDED
                        file.write(temp.read())                                       # ADDED
                    uos.remove(temp_filename)                                         # ADDED
                uos.rmdir(tempdir)                                                    # ADDED
                with open('version', 'w') as current_version_file:
                    current_version_file.write(remote_version)
                if soft_reset_device:
                    print('Soft-resetting device...')
                    machine.soft_reset()
                if hard_reset_device:
                    print('Hard-resetting device...')
                    machine.reset()
    except Exception as ex:
        print(f'Something went wrong: {ex}')

Add/Use version sub directory

Hi,

I tried your OTA solution and it works as advertised, thank you for this.

I would like to add this as an enhancement for easier use of your project.

I had to create a bash script to get the files uploaded with the correct names.
Which is fine, but if you add a version dir I think you can make your project easier to use.

server-root/
|- <project_name>/
|  |- version <- version file
|  |  <version dir> |- <filename1>
|  |  <version dir> |- <filename2>
|  |  <version dir> |- ...

This way you can use filenames without versions and keep uploading/syncing with your webserver easy.

For now this is my bash script:

#!/usr/bin/env bash

echo "uploading version: ${1} for ${2}"

`echo "$1" > version`

scp version mm:/var/www/html/$2/version
scp boot.py mm:/var/www/html/$2/$1_boot.py
scp main.py mm:/var/www/html/$2/$1_main.py
scp config_$2.py mm:/var/www/html/$2/$1_config.py

usage: ./upload.sh 1.0.4 tuinhuis_monitor

Add timeout parameter

The urequest's default timeout is set to None, allegedly leading to an infinite timeout blocking the application.

A new parameter for setting the timeout and defining a default timeout should be introduced.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.