Code Monkey home page Code Monkey logo

hpatchlite's Introduction

release license PRs Welcome +issue Welcome

Build Status

english | 中文版

HPatchLite is a lite version of HDiffPatch, tiny code & ram requirements when patch on embedded systems,MCU,NB-IoT,...

The patch code(ROM or flash occupancy) very small, compiled by Mbed Studio is 662 bytes.
Tip: if define _IS_RUN_MEM_SAFE_CHECK=0, unsafe mode can reduce 80 bytes; if used tinyuz & define _IS_USED_SHARE_hpatch_lite_types=1, can reduce 52 bytes.

At the same time, the patch memory(RAM occupancy) can also be very small, RAM size = one decompress memory size + input cache size(>=3Byte) when patch.
Tip: The smaller input cache only affects the patch speed.


Releases/Binaries

Download from latest release : Command line app for Windows, Linux, MacOS.
( release files build by projects in path HPatchLite/builds )

Build it yourself

Linux or MacOS X

$ cd <dir>
$ git clone --recursive https://github.com/sisong/HPatchLite.git
$ cd HPatchLite
$ make

Windows

$ cd <dir>
$ git clone --recursive https://github.com/sisong/HPatchLite.git

build HPatchLite/builds/vc/HPatchLite.sln with Visual Studio


diff command line usage:

diff    usage: hdiffi [options] oldFile newFile outDiffFile
test    usage: hdiffi    -t     oldFile newFile testDiffFile
  oldFile can empty, and input parameter ""
memory options:
  -m[-matchScore]
      requires (newFileSize+ oldFileSize*5(or *9 when oldFileSize>=2GB))+O(1)
        bytes of memory;
      matchScore>=0, DEFAULT -m-6
special options:
  -cache
      set is use a big cache for slow match, DEFAULT false;
      if newData not similar to oldData then diff speed++,
      big cache max used O(oldFileSize) memory, and build slow(diff speed--)
  -p-parallelThreadNumber
      if parallelThreadNumber>1 then open multi-thread Parallel mode;
      DEFAULT -p-4; requires more memory!
  -c-compressType[-compressLevel]
      set outDiffFile Compress type, DEFAULT uncompress;
      support compress type & level & dict:
        -c-tuz[-dictSize]               (or -tinyuz)
            1<=dictSize<=1g, can like 250,511,1k,4k,64k,1m,64m,512m..., DEFAULT 32k
        -c-zlib[-{1..9}[-dictBits]]     DEFAULT level 9
            dictBits can 9--15, DEFAULT 15.
        -c-pzlib[-{1..9}[-dictBits]]    DEFAULT level 6
            dictBits can 9--15, DEFAULT 15.
            support run by multi-thread parallel, fast!
        -c-lzma[-{0..9}[-dictSize]]     DEFAULT level 7
            dictSize can like 4096 or 4k or 4m or 128m etc..., DEFAULT 32k
            support run by 2-thread parallel.
        -c-lzma2[-{0..9}[-dictSize]]    DEFAULT level 7
            dictSize can like 4096 or 4k or 4m or 128m etc..., DEFAULT 32k
            support run by multi-thread parallel, fast!
            WARNING: code not compatible with it compressed by -c-lzma!
  -d  Diff only, do't run patch check, DEFAULT run patch check.
  -t  Test only, run patch check, patch(oldFile,testDiffFile)==newFile ?
  -f  Force overwrite, ignore write path already exists;
      DEFAULT (no -f) not overwrite and then return error;
      if used -f and write path is exist directory, will always return error.
  --patch
      swap to hpatchi mode.
  -v  output Version info.
  -h (or -?)
      output usage info.

patch command line usage:

patch usage: hpatchi [options] oldFile diffFile outNewFile
  if oldFile is empty input parameter ""
options:
  -s[-cacheSize]
      DEFAULT -s-32k; cacheSize>=3, can like 256,1k, 60k or 1m etc....
      requires (cacheSize + 1*decompress buffer size)+O(1) bytes of memory.
  -f  Force overwrite, ignore write path already exists;
      DEFAULT (no -f) not overwrite and then return error;
      if used -f and write path is exist directory, will always return error.
  -v  output Version info.
  -h  (or -?)
      output usage info.

library API usage:

create hpatch_lite diff:

create_lite_diff(newData,OldData,out_lite_diff,compressPlugin,...);

apply patch:

hpi_BOOL hpatch_lite_open(hpi_TInputStreamHandle diff_data,hpi_TInputStream_read read_diff,
                          hpi_compressType* out_compress_type,hpi_pos_t* out_newSize,hpi_pos_t* out_uncompressSize);
hpi_BOOL hpatch_lite_patch(hpatchi_listener_t* listener,hpi_pos_t newSize,
                           hpi_byte* temp_cache,hpi_size_t temp_cache_size);

Contact

[email protected]

hpatchlite's People

Contributors

sisong avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hpatchlite's Issues

Suggestions / Features - embedded

First of all- amazing job and thank you for making this!

For context, I have been using jojodiff with JanPatch to create embedded binary OTA patch updates. I have ran a script across a large variety of firmware releases and on average, the tuz diff size is general 1/2 the size of the jojodiff. And even for very divergent binaries, the diff is still ~30KB less. And on top of that, it runs much faster.

The main reason I found this was to find a non-GPL diff library, since I want to port the diff generation code to golang without GPL restriction.

  1. As far as I can tell, the other compression algos supported by hdiffpatch don't offer that much better compression (and proven by your comparison chart). I would suggest adding jojodiff to the comparison chart.

  2. It seems like tuz-2k - tuz-5k both generate a pretty good diff, beyond that the diffsize vs. ram tradeoff isn't very enticing. It seems like for embedded applications, tuz is best. On the one hand, it would ideal to be able to just add hpatchlite as a submodule and define implementation methods, but in the current repo structure you have to pull in sources from hdiffpatch, hdiffpatchlite, and tinyuz. In comparison to janpatch this is a very tedious process for embedded developers. It would be nice to have 1-2 header files. This would probably help adoption, if that's important to you.

  3. On smaller embedded devices running a bare-minimum OS, if any, POSIX apis are non-existent, overkill, or require special consideration. I got hdiffpatch working using something like this bdfile.h wrapper that provides POSIX-style API for a regular SPI Flash chip. However, it's a little confusing in the example what is _do_readOld vs _do_readFile in the example program. For embedded, we may need to have different implementations of _do_readFile- e.g. for old file data we want to read the currently running program flash, and for patch/new file we want to read/write to external SPI flash chip. I have implemented this in bdfile.h, but it would be helpful to have more documentation on what each stub is used for.

  4. Kinda similar to 3, but for embedded we often don't want to pull in anything from stdlib. Is malloc even necessary? There are #ifdefs for not using errno, etc, but it would be nice to have a single config file. Also, it would be nice to make it so that assert is compiled in but simply returns an error so the the patch method can safely exit. Although it seems the asserts are more for debugging and shouldn't trigger on bad inputs?

  5. Would also be nice to have the option to not use filenames and instead have an enum for old/diff/new. For non-posix filenames and directories are also unnecessary. As well as open modes ("rb", "w+", etc).

  6. Consider using a code formatter like uncrustify.

In summary, the code works well and didn't actually need many tweaks for non-POSIX, but with a little re-organizing it could be made a lot easier to integrate for embedded.

能否提供一下单片机的移植教程

作者你好,已经使用编译好的命令行程序,用 hdiffi 和 tinyuz 命令行输出 diff 固件了。
现在卡在使用 api 调用 patch,看了 issues 的一些回复,移植起来还是有点吃力

Progress indicator

On some systems, e.g. FreeRTOS, it is advised to not spend too long in a loop without occasionally yielding for other tasks to run. It would be nice to have some sort of progress indicator or optional callback stub to allow the user to yield during long loops. You could use the read/write stubs to see progress but it seems that execution time varies a lot between each coverCount. This might be more specific to tuz than hdiffpatch though.

HPatchLite工程中的一些目录是空的,导致无法编译。

$ cd


$ git clone --recursive https://github.com/sisong/HPatchLite.git
用 Visual Studio 打开 HPatchLite/builds/vc/HPatchLite.sln 编译

但是,当打开HPatchLite.sln,出现如下错误提示等:

错误 C1083 无法打开源文件: “....\HDiffPatch\libParallel\parallel_import.cpp”: No such file or directory

实际上,HDiffPatch是个空目录。

使用其它方式下载工程文件,同样存在如上错误提示。

打补丁 库api 接口如何在没有fopen/fread调用的单片机上使用?

打补丁使用的库api 接口 hpatch_lite_open/hpi_BOOL hpatch_lite_patch,需要调用fopen/fread操作。但是不少单片机裸机系统上不支持fopen/fread,所以就不能使用当前现有的库接口来在单片即上实现patch功能。
如果能提供不依赖fopen/fread的API接口,比如当这些API patch接口的入参数据已经是读取文件之后的数据,库接口就更具有通用性,就方便在单片机上进行移植调用了。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.