It probably should be done as described here: https://stackoverflow.com/a/49506573/2411220

This breaks RTP serializer

Also, related to membraneframework/membrane_core#325

This can be done easily with opus functions. However, this brings the following problems:

• a malformed packet can result in getting wrong channels/sample_rate, and to caps changes, that may not be supported by subsequent elements
• handling such changes requires reinitializing the decoder

This issue was created to check, if auto closing new issues works correctly

The documentation on hex/in the README mentions Membrane.Opus.Serializer which was removed in commit
@ledhed2222's commit bfd5e2a on Thu Mar 18 05:30:56 2021 -0400.
The current expected process is not clear.

I was able to intuit that the serializer: Membrane.Opus.Serializer should be replaced with parser: Membrane.Opus.Parser, but if I use the pipeline with only that correction and a raw audio file (for example, captured with ffmpeg -f avfoundation -i ":1" -f s16le -c:2 -f output-s16le-48k-2ch.raw), the file that is produced doesn't appear to be playable. VLC sees it as an zero-length audio file, and opusinfo reports something like the following:

Processing file "output-s16le-48k-2ch.opus"...

WARNING: Hole in data (87 bytes) found at approximate offset 4500 bytes. Corrupted Ogg.
WARNING: Hole in data (213 bytes) found at approximate offset 4500 bytes. Corrupted Ogg.
WARNING: Hole in data (343 bytes) found at approximate offset 4500 bytes. Corrupted Ogg.
WARNING: Hole in data (413 bytes) found at approximate offset 4500 bytes. Corrupted Ogg.
WARNING: Hole in data (15 bytes) found at approximate offset 4500 bytes. Corrupted Ogg.
WARNING: Hole in data (238 bytes) found at approximate offset 4500 bytes. Corrupted Ogg.
... (much more of this)
WARNING: Hole in data (137 bytes) found at approximate offset 43424 bytes. Corrupted Ogg.
ERROR: No Ogg data found in file "output-s16le-48k-2ch.opus".
Input probably not Ogg.

I slightly modified the README example for encoding (plus some tweaks to the file paths that made sense in my case), using membrane_opus_plugin v0.10.0.

  def handle_init(_) do
    children = [
      source: %Membrane.File.Source{
        location: "/tmp/input.raw"
      },
      encoder: %Membrane.Opus.Encoder{
        application: :audio,
        input_caps: %Raw{
          channels: 2,
          format: :s16le,
          sample_rate: 48_000
        }
      },
      serializer: Membrane.Opus.Parser, ### This is where I modified the documentation example
      sink: %Membrane.File.Sink{
        location: "/tmp/output.opus"
      }
    ]

    links = [
      link(:source)
      |> to(:encoder)
      |> to(:serializer)
      |> to(:sink)
    ]

    {{:ok, spec: %ParentSpec{children: children, links: links}}, %{}}
  end
end

If it hadn't been for the unusable file at the end I'd just have submitted a documentation pull request, but I can't see a clear path to get a functional file. A similar pipeline using an MP3 plugin seemed to work just fine.

(In the event that it matters, I'm using MacOS on Intel hardware and my colleague is using an M1-based Mac).

This can be done by passing proper arguments to opus_decode. To achieve that, the duration of the lost packet(s) has to be retrieved/calculated.

This element needs a re-write almost from scratch. Things that needs to be done:

• Declare native functions decoder.spec.exs and encoder.spec.exs - this doc chapter might be helpful. Most likely we're going to need an interface similar to FDK-AAC encoder and decoder
• Fill encoder.h, decoder.h with state definitions
• Implement functions declared in *.spec.exs files in proper encoder.c and decoder.c files - this will require getting familiar with libopus API. You can also take a look at the old implementation branch
• Implement the elements: the decoder and the encoder - again, FDK-AAC might be used as reference
• Write some tests for the element