immunant / rexpat Goto Github PK

Rust ❤️ Expat library

License: MIT License

Rust 98.87% Lua 0.06% Shell 0.57% Dockerfile 0.24% Python 0.25%

rust-library rust expat-xml-parser expat xml xml-parser xml-parser-lib library

rexpat's Introduction

Rexpat: a libexpat compatible Rust crate

This project is a work-in-progress conversion from unsafe Rust transpiled directly from libexpat into safe, idiomatic Rust code. The initial transpilation and refactoring was done using C2Rust.

Do not use this in production (yet!), but help refactoring and rewriting is always welcome.

Building

Requirements: Linux host with rustup installed. To run tests, you'll also need to install the requirements of libexpat (autoconf 2.58 or newer, make, and a recent C toolchain).

$ git clone --recurse-submodules https://github.com/immunant/rexpat
$ cd rexpat && cargo build

Testing

Unit testing:

$ cargo test

Download the W2C XML test suite to /tmp/libexpat/xml-test-suite and run

$ ./test_w2c.sh

To perform additional testing.

Benchmarking

NOTE: Requires that you build $REXPAT_ROOT/upstream/expat/tests/benchmark/benchmark first. See steps and requirements here. You must also have python3 (Python 3.6 or later) in your path.

$ ./src/tests/bench_c_vs_rust.py

Goals

Provide an ABI-compatible drop-in replacement for libexpat
Avoid memory-corruption vulnerabilities via Rust's safety guarantees.
Perform on par with the libexpat in time and space.

License

rexpat is free software licensed similarly to libexpat. You may copy, distribute, and modify it under the terms of the License contained in the file COPYING distributed with this package. This license is the same as the MIT/X Consortium license.

rexpat's People

Contributors

Stargazers

Watchers

Forkers

fanninpm dlrobertson

rexpat's Issues

W3C test script does not work on macOS

Tests fail with output like this:

ibm01v01.xml: not a regular file
ibm02v01.xml: not a regular file
ibm03v01.xml: not a regular file
ibm09v01.xml: not a regular file
ibm09v02.xml: not a regular file
ibm09v03.xml: not a regular file
ibm09v04.xml: not a regular file
ibm09v05.xml: not a regular file
ibm10v01.xml: not a regular file
ibm10v02.xml: not a regular file

The files do exist though. The first file is located at (matching the Linux path):

/tmp/libexpat/xml-test-suite-2013/xmlconf/ibm/valid/P01/ibm01v01.xml

So maybe there is a problem with the file status checking logic.

Add cargo test support

runtests.rs could be a cargo test compatible file instead of a standalone binary. We could then just run cargo test. Also make cargo bench do something useful.

Replace parser functions with methods

and add wrappers for public, no_mangle functions

Replace siphash and custom hashtable with Rust-native HashMap

There is no reason to refactor functionality already provided by a 3rd party crate. The siphasher crate likely provides the requisite functionality and appears to be well maintained.

Make sure rexpat works on macOS

Use Rust Enums

C2Rust translates C enums to a series of consts. We need to Oxidize these.

Replace malloc and friends

Convert boolean internal functions returning an integer to returning a bool

Replace strings w/custom pool allocator

Strings are bump allocated in xmlparse.rs. Replacement type needs to be able to represent both ANSI and unicode strings.

Convert `XML_TOK` constants into Rust enum

This is not an enum in the C code but the Rust code would be more idiomatic if we use an enum. Other constant sets:

XML_*_STATE
XML_Convert_Result
XML_ROLE_*

Get rid of main wrappers

Re-transpile from Expat >=2.4.0 to gain protection against Billion Laughs Attacks (CVE-2013-0340)?

Hi!

Expat is protecting against Billion Laughs Attacks since release 2.4.0, release 2.4.1 being the latest upstream. Is there interest in re-transpiling to import that fix to CVE-2013-0340 into rexpat?

Best, Sebastian

Set up CI

Steps:

Cargo build, cargo test, and run libexpat test binary (runtests).
Test projects that use libexpat?

Try to re-translate xmltok, preserving macro invocations

xmltok and xmltok_impl are mostly macros, so we had the thought to try to force the translator to translate all expressions that were expanded from a macro invocation into invalid Rust output that invokes a non-existent macro. We can then add definitions for the macros as needed by hand.

doProlog
doContent

Add macOS and Windows testing to CI

Move to stable rustc

We are currently using the following nightly-only features:

Unsafe pointer manipulation:

#![feature(const_raw_ptr_to_usize_cast)]
#![feature(const_transmute)]
#![feature(ptr_wrapping_offset_from)]

C ABI:

#![feature(extern_types)]

Convenience, can be removed:

#![feature(label_break_value)]
#![feature(main)]

Automate perf testing to check for performance regressions

Could we get something that tells us if we regress performance on a commit?