In this issue I'm going to present the DMA API I have implemented for the blue-pill and that I have used in my recent robotic application with the goal of using it to start a discussion about DMA based API.

Pre-requisites

You should understand how RefCell works and how it achieves dynamic borrowing.

Detailed design

Core idea

The DMA peripheral behaves like an "external mutator". I like to think of it as an independent processor / core / thread that can mutate / access data in parallel to the main processor. From that POV a DMA transfer looks like a fork-join operation like the ones you can do in std-land with threads.

With that mindset: in a DMA transfer you want to hand out "ownership" of the data / buffer from the processor to the DMA for the span of the transfer and then claim it back when the transfer finishes. Except that "ownership" in the full sense ("the owner is in charge of calling the destructor when the value is no longer needed") is not applicable here because the DMA can't destroy the buffer is using for the transfer. So instead of ownership what the proposed API will transfer is temporary read or write access to the data.

Read or write access sounds like &- and &mut- references but the proposed API won't use those. Instead it will use buffers with RefCell-style dynamic borrowing.

Buffer

The main component of the proposed API is the Buffer abstraction:

struct Buffer<B, CHANNEL> {
    _marker: PhantomData<CHANNEL>,
    data: B,
    flag: Cell<BorrowFlag>, // exactly like `RefCell`'s
    status: Cell<Status>, // "Lock status"
}

enum Status {
    Unlocked,
    Locked,
    MutLocked,
}

The data and flag fields effectively form a RefCell<B>. Although not explicitly bounded B can only be an array of bytes ([u8; N]). The CHANNEL type parameter indicates which DMA channel this buffer can work with; possible values are phantom types like Dma1Channel1, Dma1Channel2, etc. Finally the status field indicates if the DMA is currently in possession of the buffer.

impl<B, CHANNEL> Buffer<B, CHANNEL> {
    pub fn borrow(&self) -> Ref<'a, B> { .. }
    pub fn borrow_mut(&self) -> RefMut<'a, B> { .. }

    fn lock(&self) -> &B { .. }
    fn lock_mut(&self) -> &mut B { .. }
    unsafe unlock(&self) { .. }
    unsafe unlock_mut(&self) { .. }
}

Buffer exposes public borrow and borrow_mut methods that behave exactly like the ones RefCell has. Buffer also exposes private locks and unlocks methods that are meant to be used only to implement DMA based APIs.

The lock and unlock pair behaves like a split borrow operation. A borrow call will check that no mutable references exist and then hand out a shared reference to data wrapped in a Ref type while increasing the Buffer shared reference counter by one. When that Ref value goes out of scope (it's destroyed) the Buffer shared reference counter goes down by one. A lock call does the first part: it checks for mutable references, increases the counter and hands out a shared reference to the data. However the return type is a plain shared reference &T so when that value goes out of scope the shared reference counter won't be decremented. To decrement that counter unlock must be called. Likewise the lock_mut and unlock_mut pair behaves like a split borrow_mut operation.

You can see why it's called lock and unlock: once locked the Buffer can't no longer hand out mutable references (borrow_mut) to its inner data until it gets unlocked. Similarly once a Buffer has been lock_muted it can't hand out any reference to its inner data until it gets unlock_muted.

DMA based API

Now let's see how to build a DMA based API using the Buffer abstraction. As an example we'll build a Serial.write_all method that asynchronously serializes a buffer using a DMA transfer.

impl Serial {
    pub fn write_all<'a>(
        &self,
        buffer: Ref<'a, Buffer<B, Dma1Channel4>>,
    ) -> Result<(), Error>
    where
        B: AsRef<[u8]>,
    {
        let usart1 = self.0;
        let dma1 = self.1;

        // There's a transfer in progress
        if dma1.ccr4.read().en().bit_is_set() {
            return Err(Error::InUse)
        }

        let buffer: &[u8] = buffer.lock().as_ref();

        // number of bytes to write
        dma1.cndtr4.write(|w| w.ndt().bits(buffer.len() as u16));
        // from address
        dma1.cmar4.write(|w| w.bits(buffer.as_ptr() as u32));
        // to address
        dma1.cpar4.write(|w| w.bits(&usart1.dr as *const _ as u32));

        // start transfer
        dma1.ccr4.modify(|_, w| w.en().set());

        Ok(())
    }
}

Aside from the Ref in the function signature, which is not a cell::Ref (it's a static_ref::Ref), this should look fairly straightforward. For now read Ref<'a, T> as &'a T; they are semantically equivalent. I'll cover what the Ref newtype is for in the next section.

Let's see how to use this method:

static BUFFER: Mutex<Buffer<[u8; 14], Dma1Channel4>> =
    Mutex::new(Buffer::new([0; 14]));

let buffer = BUFFER.lock();

// mutable access
buffer.borrow_mut().copy_from_slice("Hello, world!\n");

serial.write_all(buffer).unwrap();

// immutable access
let n = buffer.borrow().len(); // OK

// mutable access
// let would_panic = &mut buffer.borrow_mut()[0];

At this point the transfer is ongoing and we can't mutably access the buffer while the transfer is in progress. How do we get the buffer back from the DMA?

impl<B> Buffer<B, Dma1Channel4> {
    /// Waits until the DMA transfer finishes and releases the buffer
    pub fn release(&self) -> nb::Result<(), Error> {
        let status = self.status.get();

        // buffer already unlocked: no-op
        if status == Status::Unlocked {
            return Ok(());
        }

        if dma1.isr.read().teif4().is_set() {
            return Err(nb::Error::Other(Error::Transfer))
        } else if dma1.isr.read().tcif4().is_set() {
            if status == Status::Locked {
                unsafe { self.unlock() }
            } else if status == Status::MutLocked {
                unsafe { self.unlock_mut() }
            }

            // clear flag
            dma1.ifcr.write(|w| w.ctcif5().set());
        } else {
            // transfer not over
            Err(nb::Error::WouldBlock)
        }
    }
}

The Buffer.release is a potentially blocking operation that checks if the DMA transfer is over and unlocks the Buffer if it is. Note that the above implementation is specifically for CHANNEL == Dma1Channel4. Other similar implementations can cover the other DMA channels.

Continuing the example:

serial.write_all(buffer).unwrap();

// immutable access
let n = buffer.borrow().len(); // OK

// .. do stuff ..

// wait for the transfer to finish
block!(buffer.release()).unwrap();

// can mutably access the buffer again
buffer.borrow_mut()[12] = b'?';

serial.write_all(buffer).unwrap();

Alternatively, using callbacks / tasks:

fn tx() {
    // ..

    SERIAL.write_all(BUFFER.lock()).unwrap();

    // ..
}

// DMA1_CHANNEL4 callback
fn transfer_done() {
    BUFFER.lock().release().unwrap();
}

static_ref::Ref

The Ref<'a, T> abstraction is a newtype over &'a T that encodes the invariant that the T to which the Ref is pointing to is actually stored in a static variable and thus can't never be deallocated.

The reason Ref is used instead of just &- in the write_all method is to prevent using stack allocated Buffers with that method. Stack allocated Buffers are rather dangerous as there's no mechanism to prevent them from being deallocated. See below what can happen if a Serial.read_exact method used &- instead of Ref:

fn main() {
    foo();
    bar();
}

#[inline(never)]
fn foo() {
    let buffer = Buffer::new([0; 256]);

    SERIAL.read_exact(&buffer);

    // returns, DMA transfer may not have finished, `buffer` is
    // destroyed / deallocated
}

#[inline(never)]
fn bar() {
    // DMA transfer ongoing; these *immutable* values allocated on the stack
    // will get written to by the DMA
    let x = 0u32;
    let y = 0u32;

    // ..
}

Unresolved questions

• Is there an equally flexible (note that with this approach the DMA transfer start and finish operations can live in different tasks / interrupts / contexts) alternative that involves no runtime checks?

• Should we extend this to also work with Buffers allocated on the stack? My first idea to allow that was to add a "drop bomb" to Buffer. As in the destructor will panic if there's an outstanding lock. See below. (But this probably a bad idea since panicking destructor is equal to abort (?))

{
    let buffer = Buffer::new(..);
    buffer.lock();
    // panic!s
}

{
    let buffer = Buffer::new(..);
    buffer.lock();
    unsafe { buffer.lock() }
    // OK
}

{
    let buffer = Buffer::new(..);
    let x = buffer.borrow();
    // OK
}

Do note that an unsafe Ref::new exists so you can create a Buffer on the stack and wrap a reference to it in a Ref value. This will be like asserting that the buffer will never get deallocated. Of course it's up to you to ensure that's actually the case.

• In the blue-pill implementation Buffer is defined in the blue-pill crate itself but to turn DMA based APIs into traits we would have to move that Buffer abstraction into the embedded-hal crate. That complicates things because (a) lock et al. would have to become public and (b) e.g. Dma1Channel1 wants to be defined in the stm32f103xx-hal-impl crate but that means one can't write impl Buffer<B, Dma1Channel1> due to coherence. I have no idea how to sort out these implementation details.

• This API doesn't support using a single Buffer with more than one DMA channel (neither serially or concurrently). Is that something we want to support?

• Since we have the chance: should we reduce the size of the flag field? The core implementation uses flag: usize but 4294967295 simultaneous cell::Ref sounds like a very unlikely scenario in a microcontroller.

This trait became available in release v0.1.1 behind the "unproven" Cargo feature.

I want to create this issue to discuss a shared place to store necessary hardware type definitions. These could be stored in the embedded-hal crate, but could also easily be in their own crate. I just think we should discuss a shared place to store these outside the processor specific crates.

There are many type definitions that can be useful when working with low-level sensors. At the moment there are no shared place that such type can or are stored. A couple of very handy types are currently in stm32f30x_hal::time (and also replicated in stm32f103xx_hal) for dealing with time.

An easy first step is to create a type module for embedded-hal and extract the types defined in stm32f30x_hal::time and potentially add more later if necessary.

I'm not sure if there are other definitions apart from time that should be abstracted out. I don't think this issue should be about physical types (e.g. cm or m/s), but should focus on shared types that are necessary for interacting with hardware.

This API has not been designed yet. What methods should it provide?

I have ben working on 'drivers' for a couple of devices that can time out and while the current system works it is very clunky to work with.

To illustrate my problems, here is a usage example in which I ran into some issues. I'm trying to write a driver for the DHT11 and DHT22 temperature and humidity sensors. These sensors communicate using a timing based protocol. Since these timings are specific to the device they should probably be defined in the driver rather than by the user. However, because this crate makes no guarantees about the type of CountDown::Time it can't be specified for a generic struct.

I would therefore propose adding some time types to the embedded_hal crate which could be used by CountDown and Periodic instead. Perhaps lifting the Hertz, KiloHertz, and MegaHertz from stm32f103xx_hal could work.

Sidenote: Is there a reason that Hertz is treated as the "base" unit in those crates. It places a hard cap on the amount of time you can wait at 1 second which is slighly annoying.

This API has not been designed yet. What methods should it provide?

In

https://github.com/japaric/embedded-hal/blob/23ee5d244d7a4eba619ef4d1e08a38e5c05f8987/src/blocking/spi.rs#L60

there's a forced data read after the write which may block forever if there's no data clocked in on the MISO line. Is there a reason for this or just an oversight?

What the title says. The use case for this is some sort of generic charlieplexing interface / driver / library.

Alternatives

Vanilla

/// A tri-state (low, high, floating) pin
pub trait TriStatePin {
    /// Drives the pin low
    fn set_low(&mut self);

    /// Drives the pin high
    fn set_high(&mut self);

    /// Puts the pin in floating mode
    fn float(&mut self);

    /// Checks if the pin is currently in floating mode
    fn is_floating(&self) -> bool;

    // plus maybe other `is_*` methods
}

Enum based

/// The states of a tri-state pin
enum State { Low, High, Floating }

/// A tri-state (low, high, floating) pin
pub trait TriStatePin {
    /// Changes the state of the pin
    fn set(&mut self, state: State);

    /// Gets the state of the pin
    fn state(&self) -> State;
}

Once we pick an alternative that we are happy with we can land it behind the "unproven" feature gate. Once someone has demonstrated that it works by actually implementing the trait and building some generic library on top of it we can un-feature gate it.

cc @therealprof

Clocks are an important part of all microcontrollers that I know of, and I believe this is a topic that should be covered in embedded-hal. I don't have a good enough overview over the topic to know everything that's required, but I have encountered some use cases that gave me a few ideas.

I've implemented a rough proposal as part of LPC82x HAL. This is not quite ready for inclusion into embedded-hal, but I've decided to open this issue to garner some feedback, and give others the opportunity to mention other use cases that I haven't covered.

Here's the API that I'd like to submit (eventually):

• Overview: lpc82x_hal::clock
• A trait that returns a clock's frequency: clock::Frequency
• A marker trait for enabled clocks: clock::Enabled
• A struct to represent durations: clock::Ticks

Use cases for the API:

• Implementations of the traits: LowPowerClock, IrcDerivedClock
• Sleep code that depends on various bits: Sleep
• I also have an unrelease time library that depends on this infrastructure. I haven't had time to polish and release it yet, but I'll happily send the code to anyone who wants to take a look.

As I said, this is still a bit rough. lpc-rs/lpc8xx-hal#4 has some
ideas for improvement. I'd love to hear what everyone thinks!

Currently, there is no way to stop a timer once started. This isn't too bad for simple CountDown timers, but Periodic timers will keep running indefinitely. It would be nice if there was a way to cancel a running timer again.

The simplest solution would probably be to add a cancel method to CountDown, but this would break all implementations.

Why

We want to provide different sets of traits that operate at a higher level and
that are either tied to a specific async model (e.g. futures) or operate in
blocking mode because some drivers will be easier to built on top of those
higher level traits rather than on top of the low level traits we already have.

At the same time we want to maximize code reuse so implementers of the low level
traits should get an implementation of the higher level traits for free whenever
possible. However, we still want to give crate authors the freedom of being able
to write the implementations of the higher level traits themselves. This means
that embedded-hal should provide a default implementation of the higher
level traits for implementers of the low level traits, but in a way that makes
it possible for crate authors to override that default implementation.

How

This RFC proposes exposing such default implementations as free functions. Let's
see an example:

serial::Write is the low level trait for write operations on a serial
interface that we currently provide. The trait is non-blocking and operates at
the byte / word level:

pub trait Write<Word> {
    type Error;

    fn write(&self, word: Word) -> nb::Result<(), Self::Error>;
}

We also want to provide a blocking version of this Write trait:

pub mod blocking {
    pub trait Write {
        type Error;

        fn bwrite_all(&self, buffer: &[u8]) -> Result<(), Self::Error>;
        // ..
    }
}

But want to save authors of crates that already provide an implementation of
serial::Write for their types the effort of coming up with an implementation
of this new trait for their types. So we provide a default implementation of
blocking::Write methods for serial::Write implementers in the form of free
functions:

pub mod blocking {
    pub fn bwrite_all<S>(serial: &S, buffer: &[u8]) -> Result<(), S::Error>
    where
        S: serial::Write,
    {
        for byte in buffer {
            block!(serial.write(*byte))?;
        }
        Ok(())
    }
}

Now the crate authors can implement the new blocking::Write trait for their
types by just deferring the implementation to these free functions:

struct Serial;
struct Error;

// existing implementation
impl hal::serial::Write for Serial { type Error = Error; .. }

// just added
impl hal::blocking::Write for Serial {
    type Error = Error;

    fn bwrite_all(&self, buffer: &[u8]) -> Result<(), Error> {
        // use the default implementation
        hal::blocking::bwrite_all(self, buffer)
    }
}

However they still have the option of not using this default implementation
and writing an implementation that's better tailored for their types.

Alternatives

default impl

My original idea was to provide default blanket implementations of the higher
level traits for types that implemented the low level traits as shown below:

default impl<S> Write for S
where
    S: ::serial::Write<u8>,
{
    type Error = S::Error;

    fn bwrite_all(&self, buffer: &[u8]) -> Result<(), S::Error> {
        for byte in buffer {
            block!(self.write(*byte))?;
        }

        Ok(())
    }
}

This would leave room for overriding the default implementation through the
specialization mechanism.

However this doesn't work today. It seems that default implementations don't
support associated types (I think it's not decided whether specialization should
be allowed to change the associated type chosen by the default implementation):

error[E0053]: method `bwrite_all` has an incompatible type for trait
  --> src/blocking.rs:32:5
   |
11 |       fn bwrite_all(&self, buffer: &[u8]) -> Result<(), Self::Error>;
   |       --------------------------------------------------------------- type in trait
...
32 | /     fn bwrite_all(&self, buffer: &[u8]) -> Result<(), S::Error> {
33 | |         for byte in buffer {
34 | |             block!(self.write(*byte))?;
35 | |         }
36 | |
37 | |         Ok(())
38 | |     }
   | |_____^ expected trait `blocking::Write`, found trait `serial::Write`
   |
   = note: expected type `fn(&S, &[u8]) -> core::result::Result<(), <S as blocking::Write>::Error>`
              found type `fn(&S, &[u8]) -> core::result::Result<(), <S as serial::Write<u8>>::Error>`

error: aborting due to previous error(s)

No override mechanism

Another alternative is to provide a non overridable blanket implementation of
higher level traits for implementers of the low level traits:

// in embedded-hal

pub mod blocking {
    impl<S> Write for S
    where
        S: ::serial::Write,
    {
        type Error = S::Error;

        fn bwrite_all<S>(&self, buffer: &[u8]) -> Result<(), S::Error>
        where
            S: serial::Write,
        {
            for byte in buffer {
                block!(self.write(*byte))?;
            }
            Ok(())
        }
    }
}

With this approach crate authors won't have to implement blocking::Write
themselves. The problem is that they won't be able to override the default
implementations.

Supertraits

Yet another option is to make higher level traits supertraits of the low level
traits and provide the default implementation as default methods:

pub mod blocking {
    pub trait Write: ::serial::Write<u8> {
        fn bwrite_all<S>(&self, buffer: &[u8]) -> Result<(), S::Error>
        {
            for byte in buffer {
                block!(serial.write(*byte))?;
            }
            Ok(())
        }
    }
}

This allows specialization, but the downside of this approach is that to
implement the blocking::Write trait for your type you also have to implement
the serial::Write trait. However, this may not always be what you want: for
instance, you probably don't want this if implementing the higher level traits
for types that will be used on embedded Linux; you probably would prefer to
only implement the higher level traits for your type as the low level traits
don't map very well to Linux's I/O primitives.

cc @posborne @Kixunil

Interrupts are an important topic in microcontroller programming, but so far there's no support for them in embedded-hal. I'd like to start a discussion about whether such support should be added, and what it could look like.

I think right now we need two kinds of contributions to this discussion:

• If you have any thoughts about what interrupt support should look like, please post them here.
• If you maintain an implementation of embedded-hal, feel free to summarize how your crate supports interrupts, and how that does or doesn't interact with the embedded-hal traits. I plan to post my own experiences later.
• Same, if you've written a driver based on embedded-hal. How does your driver use interrupts? What does it need that is currently missing from embedded-hal?

There's been some previous discussion in #37.

Edit (2018-03-08): Feedback from driver authors is also needed. Added it to the list.

I'd like to suggest to suggest a new Rng API for a common way of getting random numbers from devices with some source of randomness.

I've already implemented this as a proof-of-concept in my nrf51-hal crate (https://github.com/therealprof/nrf51-hal) and an example use in my microbit crate (https://github.com/therealprof/microbit).

The suggested (blocking) API is similar to the I2c API in that a mutable u8 slice is passed as parameter which will be filled with the random data and a Result is returned:

//! Blocking hardware random number generator

/// Blocking read
pub trait Read {
    /// Error type
    type Error;

    /// Reads enough bytes from hardware random number generator to fill `buffer`
    fn read(&mut self, buffer: &mut [u8]) -> Result<(), Self::Error>;
}

Potentially this could also use an implementation for other (or even variable) array types, e.g. the rand crate requires a u32 slice as a seed.

I'd be happy to supply a PR for the the implementation.

I'd like to propose a waveform generation trait. This would be useful for efficient bitbanging, PWM, etc.

I will begin with simple sketch and we will see where it will lead.

pub struct Microseconds(/*What should go here? Maybe `pub u32` is fine?*/);

pub enum PinState {
    High,
    Low,
}

pub trait WaveGen {
    fn generate<I: Clone + IntoIterator<Item=Microseconds>>(&mut self, pin: Pin, initial_state: PinState, delays: I, repeat_count: u32);
}

The implementor should set the pin into state defined by initial_state, then wait for duration specified by item from delays and toggle the pin after each item.

I'd love to implement some drivers for wireless sockets I have.

These traits became available in release v0.1.2 behind the "unproven" Cargo feature.

Hello, everyone,

for one of my boards containing a STM32F405 I'm creating a BSP-Crate and corresponding HAL-Crate and note that the STM32F4 will be able to share some drivers of the HAL with the STM32F3, but some of them are very different, like I2C.

Are there any plans for a better representation of the common features of the STM32 family? It would be quite unpleasant to always have to copy from the stm32f30x-hal. I'm really not a friend of copying code. At this point I would like to offer my help.

Greetings,

Ingo

• What is missing in this API?
• Can this API be implemented for different devices?

Disclaimer: a) I'm not currently using Rust in the embedded context, and don't have any concrete plans to do so in the near future, and b) the platforms I'm referencing below aren't currently supported by the Rust compiler. I'm merely raising this issue to make sure you're aware of the fact that some assumptions that went into the design of the SPI trait do not hold on some platforms.

I came across this blog post (An introduction to writing embedded HAL based drivers in Rust) yesterday and noticed that the model behind the SPI abstraction seems to be the following:

There is an SPI driver and an additional output pin that acts as chip select and that can be driven high/low arbitrarily. The device driver is then responsible for controlling the chip select pin in addition to initiating SPI transfers.

On both platforms that I work with at work [0][1], this is not the case: The chip select pins are directly controlled by the SPI peripheral. The peripheral offers essentially two ways of controlling the behaviour of the chip select: 1) a slave select register which controls which chip select(s) are asserted during a transfer and 2) a control register to force the chip select low outside of ongoing transfers (e.g. for bulk reads/writes).

While this could possibly be made to work with the SPI trait (i.e. by defining a dummy OutputPin that sets the above registers correctly in the SPI peripheral), I feel like this would be awkward and possibly brittle.

From a conceptual point of view I find it odd that the chip select is somehow treated independently from the SPI driver, since one without the other is useless. In my opinion, controlling the chip select pin should be the responsibility of the SPI driver and not the device driver. Meanwhile, the SPI trait should provide a way for a device driver to tell the SPI driver to keep the chip select asserted between transfers (e.g. two methods force_chip_select() / release_chip_select() or something..)

Anyway, these were just some thoughts that I had when I read the above blog post.

[0] see chapter 40 (DSPI) here: https://www.nxp.com/docs/en/reference-manual/MCF54418RM.pdf
[1] see chapter 5 (SPI core) here: https://www.altera.com/en_US/pdfs/literature/ug/ug_embedded_ip.pdf

Summary of this thread as of 2018-04-02

Current status of the DMA API design:

I think we have a clear idea of what the API should provide:

It MUST

• be memory safe, obviously
• expose a device agnostic API, so we can use it to build DMA based HAL traits
• support one-shot and circular transfers
• support mem to peripheral and peripheral to mem transfers
• not require dynamic memory allocation, i.e. allocator support should be optional

It SHOULD support these features:

• Canceling an ongoing transfer
• Memory to memory transfers
• Peeking into the part of the buffer that the DMA has already written to
• Data widths other than 8-bit
• Using a part / range of the buffer; the API still needs to take ownership of the whole buffer in this scenario

This is out of scope:

• Configuration stuff like enabling burst mode or constraining usage of a channel to certain
  peripherals. The API for this functionality is up to the HAL implementer to provide / design.

What we know so far from existing implementations:

• Ownership and stable addresses are a must for memory safety
  • (remember that mem::forget is safe in Rust)
  • The ongoing transfer value must take ownership of the buffer and the DMA channel
  • storing [T; N] in the Transfer struct doesn't work because the address is not stable -- it
    changes when the Transfer struct is moved
  • &'static mut fulfills these two requirements but so do Box, Vec and other heap allocated
    collections. See owning_ref::StableAddress for a more complete list.

• These concepts need to be abstracted over (using traits or structs):
  • An ongoing DMA transfer
  • A DMA channel, singletons make sense here
  • Buffers compatible with DMA transfers. cf. StableAddress. Also the element type must be
    restricted -- e.g. a transfer on &'static mut [Socket] doesn't make sense.

Unresolved questions:

• Alignment requirements. Does a transfer on a [u16] buffer require the buffer to be 16-bit
  aligned? The answer probably depends on the target device.

Attempts at the problem:

• Memory safe DMA transfers, a blog post that explores using
  &'static mut references to achieve memory safe DMA transfers

• stm32f103xx-hal, PoC implementations of the idea present in the blog post. It contains
  implementations of one-shot and circular DMA transfers

• myeisha's DMA framework. Supports several heap
  allocated collections like Box and Vec.

What the title says. This blog post describes the approach to building such APIs. The last part of
the post covers platform agnostic traits that could be suitable for inclusion in embedded-hal.

This issue is for collecting feedback on the proposed approach and deciding on what should land in
embedded-hal.

It is quite typical to have multiple I2C or SPI devices hanging on the same bus, but with the current scheme and move semantics the driver is taking possession of the handle, blocking the use of multiple chips at the same bus which is totally legal and (at least) for the blocking APIs should also be safe.

@japaric Should those bus types be made Copy?

In the v0.1.0 release of this crate we have a digital::OutputPin that represents a single digital output pin. That trait is useful for, e.g., implementing the NSS pin of a SPI interface but not enough to implement 8 or 16 bit parallel port interfaces where all the pins need to change at the same time (atomically) to meet timing requirements (e.g. LCD interfaces) -- using 16 impl OutputPin would result in them changing state at different time intervals.

I think the obvious trait to use would be the following:

/// A digital output "port"
///
/// `Width` is the size of the port; it could be `u8` for an 8-bit parallel
/// port, `u16` for a 16-bit one, etc.
///
/// **NOTE** The "port" doesn't necessarily has to match a hardware GPIO port;
/// it could for instance be a 4-bit ports made up of non contiguous pins, say
/// `PA0`, `PA3`, `PA10` and `PA13`.
pub trait OutputPort<Width> {
    /// Outputs `word` on the port pins
    ///
    /// # Contract
    ///
    /// The state of all the port pins will change atomically ("at the same time"). This usually
    /// means that state of all the pins will be changed in a single register operation.
    fn output(&mut self, word: Width);
}

cc @kunerd

It's unfortunate that there's no nonblocking variant of the i2c HAL traits -- but that could easily change.

My first impression is that it should look something like:

//! Inter-integrated Circuit Interface

use nb;

/// Possible I2C errors
pub enum Error<E> {
    /// An implementation-specific error ocurred
    Other(E),

    /// Another master owns the bus
    ArbitrationLost,

    /// The slave did not acknowledge an address or data byte
    ByteNacked,

    /// An operation was attempted at an invalid time
    ///
    /// For example, an attempt to `send()` a byte before `send_address()` is called will fail with
    /// this error
    InvalidOperation,

    #[doc(hidden)]
    _Extensible,
}

/// Read or write access to the peripheral
pub enum AccessType {
    Read,
    Write,
}

pub trait Master {
    /// An enumeration of implementation-specific I2C errors
    type ImplError;

    /// Drive a start condition on the bus
    ///
    /// Returns an `Error` if the system is in an invalid state.
    ///
    /// May be used to generate a repeated start condition -- however, at minimum `send_address()`
    /// must be called between the start conditions.
    ///
    /// TODO: Does I2C require at least byte transferred before the repeated start? This is by far
    /// the most common use-case
    fn send_start(&mut self) -> nb::Result<(), Error<Self::ImplError>>;

    /// Drive a start condition on the bus
    ///
    /// Returns an `Error` if the system is in an invalid state
    fn send_stop(&mut self) -> nb::Result<(), Error<Self::ImplError>>;

    /// Send a peripheral address
    ///
    /// Must be the first operation after generating a start condition.
    fn send_address(
        &mut self,
        address: u8,
        access: AccessType,
    ) -> nb::Result<(), Error<Self::ImplError>>;

    /// Send a byte to the peripheral
    ///
    /// May be called several times after sending an address with `AccessType::Write`. May not be
    /// called after sending an address with `AccessType::Read`
    fn send(&mut self, byte: u8) -> nb::Result<(), Error<Self::ImplError>>;

    /// Send a byte to the peripheral
    ///
    /// May be called several times after sending an address with `AccessType::Read`. May not be
    /// called after sending an address with `AccessType::Write`
    fn read(&mut self, send_ack: bool) -> nb::Result<u8, Error<Self::ImplError>>;
}

If this is stabilized, there could be default implementations of blocking::i2c in terms of it, simplifying driver writers' lives.

Alternatives

The nonblocking trait could operate at a higher-level (basically just mirroring the traits exposed by blocking::i2c except with a nb::Result. The above trait has lots of potential for runtime errors, which may be undesirable.

• What is missing in this API?
• Can this API be implemented for different devices?

Some people have expressed interest in a trait like this for writing generic LCD drivers but there are probably other use cases.

We were discussing this on IRC yesterday and I proposed three different APIs:

Proposals

Result based API

/// A pin that can switch between input and output modes at runtime
pub trait IoPin {
    /// Signals that a method was used in the wrong mode
    type Error;

    /// Configures the pin to operate in input mode
    fn as_input(&mut self);
    /// Configures the pin to operate in output mode
    fn as_output(&mut self);

    /// Sets the pin low
    fn set_low(&mut self) -> Result<(), Self::Error>;
    /// Sets the pin high
    fn set_high(&mut self) -> Result<(), Self::Error>;

    /// Checks if the pin is being driven low
    fn is_low(&self) -> Result<bool, Self::Error>;
    /// Checks if the pin is being driven high
    fn is_high(&self) -> Result<bool, Self::Error>;
}

// this function won't panic; LLVM should be able to opt way the panicking branches
fn example(mut io: impl IoPin) {
    io.as_input();
    if io.is_low().unwrap() {
        /* .. */
    }
    if io.is_high().unwrap() {
        /* .. */
    }

    io.as_output();
    io.set_low().unwrap();
    io.set_high().unwrap();
}

Closure based API

/// A pin that can switch between input and output modes at runtime
pub trait IoPin {
    /// Pin configured in input mode
    type Input: InputPin;

    /// Pin configured in output mode
    type Output: OutputPin;

    /// Puts the pin in input mode and performs the operations in the closure `f`
    fn as_input<R, F>(&mut self, f: F) -> R
    where
        F: FnOnce(&Self::Input) -> R;

    /// Puts the pin in output mode and performs the operations in the closure `f`
    fn as_output<R, F>(&mut self, f: F) -> R
    where
        F: FnOnce(&mut Self::Output) -> R;
}

fn example(mut io: impl IoPin) {
    io.as_input(|i| {
        if i.is_low() { /* .. */ }
        if i.is_high() { /* .. */ }
    });

    io.as_output(|o| {
        o.set_low();
        o.set_high();
    });
}

Implicit / Automatic API

/// A pin that can switch between input and output modes at runtime
pub trait IoPin {
    /// Signals that a method was used in the wrong mode
    type Error;

    /// Sets the pin low
    ///
    /// **NOTE** Automatically switches the pin to output mode
    fn set_low(&mut self);
    /// Sets the pin high
    ///
    /// **NOTE** Automatically switches the pin to output mode
    fn set_high(&mut self);

    /// Checks if the pin is being driven low
    ///
    /// **NOTE** Automatically switches the pin to input mode
    /// **NOTE** Takes `&mut self` because needs to modify the configuration register
    fn is_low(&mut self) -> bool;
    /// Checks if the pin is being driven high
    ///
    /// **NOTE** Automatically switches the pin to input mode
    /// **NOTE** Takes `&mut self` because needs to modify the configuration register
    fn is_high(&mut self) -> bool;
}

fn example(mut io: impl IoPin) {
    if io.is_low() {
        /* .. */
    }
    if io.is_high() {
        /* .. */
    }

    io.set_low();
    io.set_high();
}

I have implemented the proposals as branches io-1, io-2, io-3. You can try them out by adding something like this to your Cargo.toml file:

[dependencies]
embedded-hal = "0.1.0"

[replace]
"embedded-hal:0.1.0" = { git = "https://github.com/japaric/embedded-hal", branch = "io-1" }

Implementation concerns

There were some concerns about whether these traits can actually be implemented for all possible scenarios given that switching the mode of any pin on say, GPIOA, usually requires a RMW operation on some control register; thus the pins need exclusive access to the control register when switching modes.

Following the CRL idea of the Brave new IO blog post it seems that implementations would run into this problem:

struct IoPin {
    // pin number
    n: u8,
    crl: CRL,
    // ..
}

let pa0 = IoPin { n: 0, crl, .. };
let pa1 = IoPin { n: 1, crl, .. };
//^ error: use of moved value: `x`

But you can use a RefCell to modify the CRL register in turns:

struct IoPin<'a> {
    // pin number
    n: u8,
    crl: &'a RefCell<CRL>,
    // ..
}

let crl = RefCell::new(crl);
let pa0 = IoPin { n: 0, crl: &crl, .. };
let pa1 = IoPin { n: 1, crl: &crl, .. };

This limits you to a single context of execution though because RefCell is not Sync which means &'_ RefCell is not Send which means IoPin is not Send either.

But you can recover the Send-ness and avoid runtime checks by splitting CRL in independent parts that can be modified using bit banding, if your chip supports that:

let crls = crl.spilt();
let crl0: CRL0 = crls.crl0;
let crl1: CRL1 = crls.crl1;

let pa0 = IoPin0 { crl0, .. };
let pa1 = IoPin0 { crl0, .. };

Thoughts on these two proposals? Do they fit your use case? Do you have a different proposal?

cc @kunerd @therealprof

Do you mind taking a quick look at https://github.com/idubrov/stm32-hal/blob/master/src/flash.rs to see if Flash trait I created is a good candidate to be included into this embedded-hal crate?

It appears to me that the trait itself could go into embedded-hal and implementation (line #97 and below) would go into device-specific crate (which in my case is stm32f103xx, but there is not stm32f103xx-hal yet, as far as I can see).

I also have an EEPROM emulation crate (https://github.com/idubrov/eeprom) which uses Flash trait, but, I guess, it should stay as a separate "driver" crate, which will depend on embedded-hal, right?

You can leave general feedback about the HAL here.

• What is missing in this API?
• Can this API be implemented for different devices?

This is a counter proposal to #14 and aims to be simpler and more future proof.

I have sketched out an implementation which compiles (but is not tested) here:

https://github.com/vitiral/utex/blob/master/src/lib.rs

Reason of need

The embedded ecosystem is very likely to have multiple ways of allocating data, from statically to managed-static to full dynamic. However, most libraries should not care who allocated data -- only that it will not get freed and is not being used by others.

Therefore there is a strong need for a single trait that all of these libraries can use for wrapping the data they return.

Issue at Hand

The existing Buffer API will tie us to the following:

• a specific buffer type and way of representing data
• a lock/unlocking mechanism that can not be easily extended by other crates. For instance, it would be difficult to implement a memory manager using the Buffer API

This design aims to address this by providing a Mutex trait which can then be implemented in any way we please. Something like the Buffer proposed would be free to implement this trait and then be used for the full API.

From working with the uavcan rust implementation I see the convenience of having a standardized CanFrame and CAN interface in Rust. Perhaps this is something this crate should aim to have?

I think it would make sense to both have a trait Can and a struct CanFrame.

The function names for at InputPin and OutputPin (is_high, is_low) can be a bit confusing for MCUs with inverting logic. At least some (I would imagine most?) NXP LPC microcontrollers have a register for inverting the value of the GPIO register. If the invert bit is set, a high voltage at the pin will result in a 0 in the register. The name is_low suggests the voltage is low.

I don't know if there's a better name though. is_active?

See for example page 76 in the User Manual for the LPC81x series

Hey all,

Currently the linux-embedded-hal implements its SPI transfer by overwriting the original buffer which saves memory allocations which is in my books.

I hit a problem though as I dabble with writing a driver for PlayStation controllers (taking a little break from the axp209 for a bit).

The PlayStation often sends the same commands to the controllers and so I'm finding myself doing a lot of data copying I shouldn't need to do:

    const CMD_POLL: &[u8] = &[0x01, 0x42, 0x00];

...

    // Needed because `copy_from_slice` doesn't work on different sized arrays for some silly reason
    fn byte_copy(from: &[u8], to: &mut [u8]) {
        assert!(from.len() <= to.len());

        for i in 0 .. from.len() {
            to[i] = from[i];
        }
    }

    pub fn read_buttons(&mut self) -> GamepadButtons {
        let mut buffer = [0u8; 21];
        let mut data = [0u8; 6];

        // here's the copy I'd like to avoid:
        Self::byte_copy(CMD_POLL, &mut buffer);
        self.dev.transfer(command)?;

        data.copy_from_slice(&buffer[3 .. 9]);

        let controller = ControllerData { data: data };

        unsafe {
            return controller.ds.buttons;
        }
    }

Ideally, I'd waste some extra bytes padding out my command to be the same as the RX buffer and then I wouldn't need the byte_copy() function.

Is there enough of a demand to create a non-destructive version that could re-use the output buffer?

Maybe something with this function definition?:

        fn transfer<'w>(&mut self, send: & [W], receive: 'w mut & [W]) -> Result<(), S::Error> {

Adding to the API isn't something to be taken lightly since there are more and more HAL implementations coming out, but I wonder what the demand is here. Are other people jumping through hoops because there isn't a function that maintains the output buffer?

What's currently blocking the release of this crate on crates.io? One obvious blocker is that nb is not yet published there. I've opened an issue there too.

For my MAX7219 crate I've translated the shiftOut function from the Arduino platform to rust, in order to aid with bit-banged serial transfer.

As proposed at rust-embedded/wg#39 (comment), it might be an idea to move this function to the embedded-hal crate.

I am willing to send a PR to move this to the embedded-hal crate, and extend the implementation to be on par with the Arduino one (https://github.com/arduino/Arduino/blob/master/hardware/arduino/avr/cores/arduino/wiring_shift.c#L40).

But before I make the PR, I'd like to check if there are any objections/ideas/comments regarding this, so here's what I was thinking:

• Be on par with the Arduino implementation (MSB+LSB support).
• Renaming the function to shift_out instead of shiftOut.
• Moving it to the spi.rs file here in embedded-hal, so it's importable like this: use embedded_hal::spi::shift_out.
• Implementing the shiftIn function is for another time and PR.
• Making clear in the documentation comments that this is a bit-banged implementation.

For reference, my current implementation lives here: https://github.com/maikelwever/max7219/blob/master/src/lib.rs#L127

In looking to address comments like some of those raised on the i2cdev crate (rust-embedded/rust-i2cdev#27 / rust-embedded/rust-i2cdev#28) I went around to the embedded RFCs and found time to review some of this for the first time.

The basic question here is whether it should be a goal or whether (and how) a concerted effort should be made to support the embedded-hal traits under Linux for usage/testing from a Linux userspace system such as a raspberry pi.

Benefits of Supporting Linux:

• Vastly increased hardware support. Once a trait is implemented for Linux, the support is available on a huge number of embedded devices, common ones being boards like the Raspberry Pi, Beaglebone Black, CHIP board, ...
• Easier test environment for device drivers. If I am adding support for some device, I might like to do more of that development in a less-embedded environment for easier debugging. If the traits are supported under Linux that fits the bill. Everyone has a RPi and can wire up a few peripherals quickly.

Challenges to Supporting LInux:

• Many of the Linux APIs are blocking but the embedded-hal APIs are non-blocking.

Life without Linux Support:

The dream is to be able to write a piece of higher-level code using a set of base APIs/Traits to access hardware and be able to have this code work in a variety of platforms, including Linux. If we end up needing a separate set of traits for systems like MCUs and systems like Linux then driver work for a particular sensor cannot be easily shared (there may still be opportunity for reuse in a crate but it would be additional work for the author).

I have not dug deeply into what it would look like to attempt to implement any of the existing or proposed traits for Linux but it would be an interesting thing to explore as the support would, in my opinion, be extremely nice.

All of this being said, I wholeheartedly believe that what is proposed here is the right approach when working with the actual hardware in a least-common-denominator system like an MCU.

For a card I work on I started several crate for the stm32f7x processor: stm32f7x-hal, stm32f7x and an example `rust-f7-example'. There 're based on the other crate from @japaric. I use the STM32F7x.svd file to generate the stm32f7x crate. I didn't already registered them to crates.io because they are in a early stage. If you need it I'll do it.
I try to be compatible with the embedded-hal. GPIO is working, I work on the Serial / USART part. I test on a Nucleo-144 (STM32F746) board (the example crate).
I've made some change in the svd file because not all of its content was generated. The derivedFrom tab wasn't use for every declaration. There are still some errors during the generation.
I have some reflexion about the use of svd file, mostly linked with the size of it and the code generated.

Signed count

It's unclear from the documentation whether the intended use is an unsigned result from count(), with direction() indicating which way the system has moved from its zero point, or whether count() may return a signed value and direction() indicates the sign of the last change in location.

From an ergonomic standpoint, I strongly prefer the use of a signed count() -- perhaps enforced on the Count type? The hidden lines in the example seem to suggest an unsigned value being the intended use, however.

This would probably be clarified by a reference implementation.

Count reset

Should the Qei trait have a reset_count(&mut self) method?

Users of a quadrature encoder might want the ability to make the current position "zero". Some hardware implementations will support this by default, or it can be done in software by subtracting an offset.

Setting up a watchdog timer to reset a microcontroller if it isn't serviced at regular intervals is fairly common. Also, from my experience watchdogs tend to be fairly similar and minimalist making them a potentially easy inclusion for a trait. I haven't analysed a large number of controllers but the typical operation seems to be as follows:

1. Once enabled watchdog cannot be disabled
2. A bit must be set or cleared periodically or processor is reset
3. Period length may be configurable
4. May be able to see current timer value.

I think 1, 2 and 3 would be essential for a watchdog trait, via functions start, kick and set_period. Due to it on some microcontrollers only offering a subset of timer functionality I'm also not sure how much crossover there should be with the timer traits? Open to discussion this is just prompted by some work I've been doing recently with STM32 processors.

I'm not sure of future for the trait, but I found myself writing own convenience like toggle which is basically

if self.is_low() {
    self.set_high()
} else {
    self.set_low()
}

I feel like this could be provided method for the trait, unless there is plans for OutputPin to have more than two variants

I came to a point where I need some ns delays, which can't be achieved by using the Delay trait. So I have done a bit of research and found that I am not the only one having this requirement. My exact use case is timing GPIO pins from within a embedded-hal driver.

The user nagisa suggested to use DMA to control the pins on #rust-embedded. But, unfortunately not every MCU comes with DMA support.

Another idea is to use an implementation similar like the one in the Linux kernel. It basically works by determining the loops_per_jiffy at start-up. The loops_per_jiffy value describes how often a given "do nothing" instruction can be called in a given amount of time. This value is then used to calibrate the ndelay function.

I don't know how reliable such an solution would be, but I know that it comes with some limitations like that it works on an approximately base and should only be used for very small delay, because of overflows.

What do you think, would such an implementation make sense in embedded-hal? Do you know better solutions for this kind of problem?

Many MCUs (the STM32s especially) implement a plethora of different output options for their GPIO pins. A driver generally has specific output requirements for its pins -- but enforcing these with API traits will mean there will be many traits available, all implementing similar or identical APIs -- i.e. OutputStronglyDrivenPin, OutputOpenDrainPin, OutputOpenDrainPullupPin, etc.

Advantages

• One API for high-level pin mode (IoPin, InputPin, OutputPin), with additional traits providing fine-grained configurability
• Unusual combinations easy to express (OutputPin + OpenDrain + PullUp)

Disadvantages

• Possible huge proliferation of implementer types
  • The actual pin driving code will likely look similar for all, but initialization will be different. Does this force too much duplication/boilerplate?
  • Could this be handled by a builder-like pattern? i.e. pin init functions which consume a type with/without the marker trait, change the config, and return a type without/with the trait?
• Possibility of expressing impossible bounds (i.e. OutputPin + StronglyDriven + OpenDrain).
  • Mostly a problem from the driver writer's standpoint, and it will become quickly clear that no available HAL provides the type needed

Example driver usage

pub struct BitBangedI2c<D, C> {
    sda: D,
    scl: C,
}

impl<D, C> BitBangedI2c<D, C> {
    pub fn new<D, C>(sda: D, scl: C) -> BitBangedI2c<D, C>
    where
        D: digital::IoPin + digital::OpenDrain,
        C: digital::IoPin + digital::OpenDrain,
    {
        BitBangedI2c { sda, scl }
    }
}

I'd like to propose a trait for a bit banged 1-wire protocol.
It is a fairly simply protocol that uses a single pin for communications between one master device and many slaves devices. Because of this #29 is currently a blocker.

The protocol for a master device itself primarily consists of the following:

• It has two different speeds, "standard" and "overdrive" hence the two different timings
• Write a 1 bit by pulling the IO low for either 6us or 1us
• write a 0 bit by pulling the IO low for either 60us or 7.5us
• read a bit by pulling the IO low for 6us or 1us and then reading the IO 9us or 1us later
• reset all devices on 1-wire bus by pulling the IO low

There is some other common functionality that afaik most 1-wire devices use (CRC, a sort of binary search to identify all of the devices on one bus, etc). I haven't worked with 1-wire enough to know what all should be included in the embedded-hal API vs in a device driver. I think the CRC, search, and a couple of other things should probably go in embedded-hal.

I'm not sure it is worth implementing both master and slave APIs. I've only seen a few projects implement a slave in software since it has some pretty big drawbacks. It would have to be either an interrupt driven API or would have to continuously poll the IO pin. Thoughts on this?

Since this is a bitbang protocol and we are doing all of the timing in software it might need to block in some of these functions. Admittedly the timing has fairly loose requirements, but I'm not sure how well using the nb crate here to prevent blocking will work across different speed embedded devices (e.g. using yeild/generators on a fast STM32 vs on a atmega328). @tib888 wrote an implementation of 1-wire for RTFM using atomic blocks to handle this.

I'll start working on an example trait and implementation for this next week, but I want to see what sort of scope embedded-hal will have with regards to the extra functionality and I'd like opinions on the timing/blocking issue.

Some references:
Maxim app note on bitbanging 1-wire: https://www.maximintegrated.com/en/app-notes/index.mvp/id/126
Maxim app note on the search algorithm: https://www.maximintegrated.com/en/app-notes/index.mvp/id/187
Some more indepth info on the protocol timing: https://www.maximintegrated.com/en/app-notes/index.mvp/id/74

I suspect this is likely to be a bit contentious, but, how do we feel about the addition of a generic Radio interface for RF devices?

On one hand, it's unlikely to generalise well to all radio implementations. On the other, if we cover most of them we might be able to avoid the fragmentation in radio driver styles that exists in the world of C (and that'd make me really happy).

Some examples of existing interfaces:

• Contiki OS
• Riot OS
• Mbed OS

The requirements I have for such are:

• common configuration options to simplify stack implementations
• simple polled methods for transmit and receive
• callbacks for radio initiated state changes etc.
• support for 802.15.4(g) radios, though not strictly using the 15.4 phy interface because it's a nightmare

I've been playing with an implementation based on what I think are the best bits of each of them, would definitely appreciate any thoughts.

Most of the modern stm32 chips have native USB capabilities at the hardware level.

This should probably provide an interface to setup and use the bare basics. Specific USB Host/Client protocols could then be implemented on top of it.

This API has not been designed yet. What methods should it provide?

This API is meant for digital input / output.

• What is missing in this API?
• Can this API be implemented for different devices?

• What is missing in this API?
• Can this API be implemented for different devices?

An I2C master instance can be used to address multiple devices, but all of the drivers effectively take ownership of the I2C object and prevent sharing it with other drivers to talk to other devices connected on the same bus.

LSM303DLHC , MCP3425 , SGP30 and HTS221 which are all listed first on the https://github.com/rust-embedded/awesome-embedded-rust list move in the I2C instance.

https://docs.rs/si5351/0.1.5/si5351/ takes a mutable reference.

I think we need some guidance/best practice recommendations to allow addressing multiple slave devices.

Does it make sense to advise hal implementors to build in some kind of interior mutability for this?
Do we need some other token/type to manage ownership of the I2C bus for the duration of some IO?

• What is missing in this API?
• Can this API be implemented for different devices?

Since concrete driver implementations are on the way it would be great if there was a common set of Traits that could be implemented to retrieve values from sensors, potentially including unit conversion.

Some Traits that immediately come to my mind would be:

• Temperature
• Electricity
• Location
• Kinetics
• Orientation
• Vision
• Concentrations
• ...

My question would be: Is this something we would want to add to embedded-hal or should this better live in a to-be-created new crate?

Currently, there are the blocking::spi::Transfer and blocking::spi::Write traits. However, there times when one might want to read data out of a slave, without caring about what gets transmitted -- a blocking::spi::Read

This could be done with the blocking::spi::Transfer trait, but a separate trait provides two advantages:

• No need to allocate a dummy buffer to pass in as words
• An implementer of blocking::spi::Read could allow the MOSI pin to be optional

Going even further, there are times when a driver simply requires some clock cycles on the bus -- but will not be shifting meaningful data either in or out. This may seem like a corner case, but I've encountered this exact situation when configuring an ice40 FPGA. For reference, see page 19 of the programming and configuration manual. It's also a requirement for some commodity SPI flash devices.

This could be served by a blocking::spi::Ignore trait, which just takes a number of bytes to ignore.