espressif / esp-adf-libs Goto Github PK

Hi!

Are you able to add a software license?

Cheers!

I'm having issues on canceling a RINGING call by using esp_sip_uac_cancel(sip).
Is is possible? I tried with esp_sip_uac_bye(sip) but still no luck. If the call isn't yet picked up I cannot cancel the ringing!

Thanks for the awesome library and happy to receive some tips!

Hey,
how to get the source code for esp32 voip library ?

Thank you

see: Session Initiation Protocol (SIP) Parameters, SIP Compact Headers

Hi, I'm trying to record left and right channels. My theory that my task is possible with two separate pipelines with single i2s src : first ignore left channel, second ignore right channel. I narrowed my research to libesp_processing.a. But could not find any source code. I understand than this in not my business, but by any chance is it possible to add this kind of feature?

Hello, I try to mix opus files with the downmix api.

When the downmix is in ESP_DOWNMIX_WORK_MODE_BYPASS the sound works well but as soon as I add a second sound and start mixing (so the mixer in ESP_DOWNMIX_WORK_MODE_SWITCH_ON) it behave strangely, the first sound continues to play well but the second sound playin is really buggy/jerky.

Before switching to opus I tried with .wav files and it worked great, is downmixing even possible with opus ?

Here is the code I use for to init the mixer (c++)

esp_err_t MixerV2::init()
    // Get handle to audio codec chip
    _board_handle = audio_board_get_handle();

    // Create _pipeline_mix pipeline
    _pipeline_cfg = DEFAULT_AUDIO_PIPELINE_CONFIG();
    _pipeline_mix = audio_pipeline_init(&_pipeline_cfg);

    // Create down-mixer element
    downmix_cfg_t downmix_cfg = MY_DEFAULT_DOWNMIX_CONFIG();
    downmix_cfg.downmix_info.source_num = MAX_FILES_IN_DOWNMIX;
    _downmixer = downmix_init(&downmix_cfg);

    for (int i = 0; i < MAX_FILES_IN_DOWNMIX; i++) {
        downmix_set_input_rb_timeout(_downmixer, 0, i);
    }

    esp_downmix_input_info_t source_information[MAX_FILES_IN_DOWNMIX];

    esp_downmix_input_info_t source_info_base = {
        .samplerate = SAMPLERATE,
        .channel = 2,
        .bits_num = 16,
        .gain = {0, 0},
        .transit_time = 100,
    };

    float gains[MAX_FILES_IN_DOWNMIX][2] = {
        {0, -10},
        {-10, 0},
        {0, 0},
        {0, 0},
    };

    for (int i = 0; i < MAX_FILES_IN_DOWNMIX; i++) {
        source_info_base.gain[0] = gains[i][0];
        source_info_base.gain[1] = gains[i][1];
        source_information[i] = source_info_base;
    }
    source_info_init(_downmixer, source_information);

    // Create i2s stream to read audio data from codec chip
    _i2s_cfg = MY_I2S_STREAM_CFG_DEFAULT();
    _i2s_cfg.i2s_config.sample_rate = SAMPLERATE;
    _i2s_writer = i2s_stream_init(&_i2s_cfg);

    // Link elements together _downmixer-->i2s_writer
    audio_pipeline_register(_pipeline_mix, _downmixer, "mixer");
    audio_pipeline_register(_pipeline_mix, _i2s_writer, "i2s");

    // Link elements together _downmixer-->i2s_stream-->[codec_chip]
    const char *link_mix[2] = {"mixer", "i2s"};
    audio_pipeline_link(_pipeline_mix, &link_mix[0], 2);

    // Create resample element
    rsp_filter_cfg_t rsp_sdcard_cfg = DEFAULT_RESAMPLE_FILTER_CONFIG();
    rsp_sdcard_cfg.src_rate = SAMPLERATE;
    rsp_sdcard_cfg.dest_rate = SAMPLERATE;

    // Create Fatfs stream to read input data
    _fatfs_cfg = FATFS_STREAM_CFG_DEFAULT();
    _fatfs_cfg.type = AUDIO_STREAM_READER;

    // Create opus decoder to decode opus file
    _opus_cfg = DEFAULT_OPUS_DECODER_CONFIG();
    _opus_cfg.task_core = 1;

    // Create raw stream of base opus to write data
    raw_stream_cfg_t raw_cfg = RAW_STREAM_CFG_DEFAULT();
    raw_cfg.type = AUDIO_STREAM_WRITER;

    for (auto &sound : _sounds) {
        sound.rsp_filter_el = rsp_filter_init(&rsp_sdcard_cfg);
        sound.fatfs_reader_el = fatfs_stream_init(&_fatfs_cfg);
        sound.opus_decoder_el = decoder_opus_init(&_opus_cfg);
        sound.raw_write_el = raw_stream_init(&raw_cfg);
    }

    // Set up  event listener
    audio_event_iface_cfg_t evt_cfg = AUDIO_EVENT_IFACE_DEFAULT_CFG();
    _evt = audio_event_iface_init(&evt_cfg);

    for (auto &sound : _sounds) {
        sound.stream_pipeline = audio_pipeline_init(&_pipeline_cfg);
        mem_assert(_sounds.stream_pipeline);
    }

    // link all pipelines
    for (auto &sound : _sounds) {
        auto numAsStr = std::to_string(sound.index);
        const std::array<std::string, 4> link_tags_tmp = {
            "file_" + numAsStr,
            "opus_" + numAsStr,
            "filter_" + numAsStr,
            "raw_" + numAsStr,
        };
        const char *link_tags[4] = {
            link_tags_tmp.at(0).c_str(),
            link_tags_tmp.at(1).c_str(),
            link_tags_tmp.at(2).c_str(),
            link_tags_tmp.at(3).c_str(),
        };

        audio_pipeline_register(sound.stream_pipeline, sound.fatfs_reader_el, link_tags[0]);
        audio_pipeline_register(sound.stream_pipeline, sound.opus_decoder_el, link_tags[1]);
        audio_pipeline_register(sound.stream_pipeline, sound.rsp_filter_el, link_tags[2]);
        audio_pipeline_register(sound.stream_pipeline, sound.raw_write_el, link_tags[3]);

        audio_pipeline_link(sound.stream_pipeline, &link_tags[0], 4);
        sound.rb = audio_element_get_input_ringbuf(sound.raw_write_el);
        downmix_set_input_rb(_downmixer, sound.rb, sound.index);
        audio_pipeline_set_listener(sound.stream_pipeline, _evt);

       return ESP_OK;
    }

It works well and all when playing only one sound but not when I start a second one so idk where could be the issue.

Thanks,

Yohann

Hi,
would it be possible to open source these libraries? It is really hard to get it work otherwise because I don't understand what is happening if something fails. For example the opus encoder stack overflows when I'm using two channels and in general something seems wrong when using 48kHz sampling rate and 32bit pcm data. I guess everything expects 44.1kHz@16bits?

And I don't think you'll show to much IP-Stuff, since it seems you're mostly using opensource libs anyway.

Kind regards,
Johannes

I'm pretty sure there are bugs in wav_head_parser which is defined in ./esp_codec/include/processing/wav_head.h

But it appears this function is not provided in the source code .... how can I debug the problem that the wav parser is incorrect?

Hi,

Having the opus encoder is great, but the problem is Opus and AMR are the only codecs part of esp-adf-libs that support some sort of FEC (Foward Error Correction) against packet loss. The AMD decoder frankly doesn't work very well, when there is packetloss it simply dies with E (15829) AMR_DECODER: packet size(-1) < 0, line:324, amr_type:2, which defeats the purpose of using such a specialty codec in the first place.

Since the source is closed to esp-adf-libs, I can't make the necessary modifications to support opus better for various use cases. Some of these options are already available, but at a bare minimum fec, packet_loss and dtx options are desired to be added.

type | None |   | false | Must be of type 'opus'.
packet_loss | Integer | 0 | false | Encoder's packet loss percentage.
complexity | Integer | 10 | false | Encoder's computational complexity.
max_bandwidth | Custom | full | false | Encoder's maximum bandwidth allowed.
signal | Custom | auto | false | Encoder's signal type.
application | Custom | voip | false | Encoder's application type.
max_playback_rate | Custom | 48000 | false | Encoder's maximum playback rate.
bitrate | Custom | auto | false | Encoder's bit rate.
cbr | Boolean | no | false | Encoder's constant bit rate value.
fec | Boolean | yes | false | Encoder's forward error correction value.
dtx | Boolean | no | false | Encoder's discontinuous transmission value.

would you please share the code of libesp_codec.a?
I want to know two things

1. How to reduce the time consumption of the audio codec algorithm such as opus, resample. How to use the xtensa lx6 DSP instructions as ESP32 support it.
2. As ESP32 has FPU, does it mean the float point algorithm is almost equal to the fixed algorithm

Thank you!

I was hoping to see how this was done :)
Looks like it was based on the old android opencore code?
I guess using stagefright in the name was a joke? :)

Where is the implementation of this header?

I'm trying to run a equalizer, but I got this error:
/opt/espadf/esp32-bt-player/build/esp-adf-libs/libesp-adf-libs.a(equalizer.o): In function equalizer_close': /opt/espadf/esp-adf/components/esp-adf-libs/esp_codec/equalizer.c:114: undefined reference to esp_equalizer_uninit'
/opt/espadf/esp32-bt-player/build/esp-adf-libs/libesp-adf-libs.a(equalizer.o): In function equalizer_open': /opt/espadf/esp-adf/components/esp-adf-libs/esp_codec/equalizer.c:114: undefined reference to esp_equalizer_init'
/opt/espadf/esp-adf/components/esp-adf-libs/esp_codec/equalizer.c:114: undefined reference to esp_equalizer_set_band_value' /opt/espadf/esp32-bt-player/build/esp-adf-libs/libesp-adf-libs.a(equalizer.o): In function equalizer_process':
/opt/espadf/esp-adf/components/esp-adf-libs/esp_codec/equalizer.c:114: undefined reference to `esp_equalizer_process'

Hi, I am testing the dlna example program on a custom board and the dlna exsample program returns a response that the server cannot parse. The following is the output from Pulseaudio-DLNA server, but it is the same when I check it with Wireshark. I would appreciate if someone could give me some advice.

"<?xml version="1.0" encoding="utf-8"? >
<s:Envelopes:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:s="http:/schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<u:SetAVTransportURIResponse xmlns:u="urn:schmas-upnp-org:service:AVTransport:1">
t\x12\x80
</u:SetAVTransportURIResponse>
</s:Body>
</s:Envelope>"

Would it be possible to share how libesp-opus.a is built?

Many thanks!

I (987) PLAY_SDCARD_MUSIC: [4.1] Listening event from all elements of pipeline
I (997) PLAY_SDCARD_MUSIC: [4.2] Listening event from peripherals
I (997) PLAY_SDCARD_MUSIC: [ 5 ] Start audio_pipeline
E (1007) FATFS_STREAM: Failed to open. File name: /sdcard/test.flac, error message: Invalid argument, line: 116
E (1017) AUDIO_ELEMENT: [file] AEL_STATUS_ERROR_OPEN,-1
W (1017) AUDIO_ELEMENT: [file] audio_element_on_cmd_error,7
W (1027) AUDIO_ELEMENT: IN-[dec] AEL_IO_ABORT
W (1037) AUDIO_ELEMENT: [dec] AEL_IO_ABORT, -3
W (1037) AUDIO_ELEMENT: IN-[i2s] AEL_IO_ABORT
I (1047) PLAY_SDCARD_MUSIC: [ 6 ] Listen for all pipeline events
W (1067) PLAY_SDCARD_MUSIC: [ * ] Stop event received
I (1067) PLAY_SDCARD_MUSIC: [ 7 ] Stop audio_pipeline
E (1067) AUDIO_ELEMENT: [file] Element already stopped
E (1067) AUDIO_ELEMENT: [dec] Element already stopped
E (1077) AUDIO_ELEMENT: [i2s] Element already stopped

tested mp3/wav could work but not flac.
like something wrong with the library, and could not debut cause no source codes to look into.

I've noticed that esp-adf-libs includes a bundled copy of libopus, from https://github.com/xiph/opus. I described my findings here - XasWorks/esp-libopus#3

My request is that esp-adf-libs 1) make it clear that libopus is included with esp-adf-libs, including precise version information, and 2) vend the libopus header files from whatever version it was built from.

Opus is great, and folks have a need for it to be built for esp32. Make it obvious that it's there and make it easy to use - that can only help the users of esp-adf-libs.

Hello,

I am developing an audio recorder for bird identification through their sounds, and I am using the Espressif ESP32-S3 with the C language through IDF.

I need to convert the audio I capture in .wav format to FLAC to reduce the file size and be able to record for more hours with the same SD card capacity.

I thought you could help me achieve this.

Thank you.

Feature request:

There is the ability to read incoming MESSAGEs received over SIP using esp_sip_read_incomming_messages but no mechanism to send a MESSAGE.

Since the source is precompiled I cannot add this functionality in and provide it to you and the community.

For example, I would like to send a MESSAGE over SIP such as:

MESSAGE sip:[email protected] SIP/2.0
Via: SIP/2.0/UDP user1.domain.com;branch=z9hG4bK776sgdkse
To: sip:[email protected]
From: sip:[email protected];tag=49583
Call-ID: [email protected]
CSeq: 1 MESSAGE
Max-Forwards: 70
Content-Type: application/scaip+xml
Content-Length: 144
<mrq>
<ref>1234</ref>
<cid>123456</cid>
<dty>0001</dty>
<stc>001</stc>
</mrq>

Hi,
The current resample esp-adf library is handling only up to 96KHz.
Please consider getting this resample limit up to 192KHz.
Specifically I'm interested in having resample of 16bits from 48KHz source to 192KHz destination.
The library used today in esp-adf-libs is closed source so it is not possible for us to try any modification on it, thus please make its upper limit to 192KHz.
We understand it might need to use additional CPU power, but that is something that we can measure and benchmark if enabled and decide if and how to use this feature ourselves.
Regards,

SIP for VoIP calls using the esp_rtc library works but is extremely limited right now, lacking much of the standard features.
The "MESSAGE" type for instance is crucial for certain widely used alarm protocols like SCAIP.
Right now it is impossible to use your library just because of this one missing type...and one must rewrite a whole SIP stack just for this.

Please consider open sourcing your SIP stack because of these very real limitations, or at the bare minimum at least please provide a way for users to expand your library by sending their own standard SIP requests and handling the corresponding responses.

Please consider this request, this is not a gimmick, it can and will impact most users that simply cannot use SIP for commercial products right now with ESP-ADF.

**1. The streams of data: raw_stream->aac_decoder->i2s_stream

2. if the raw_stream is incontinuity, then the aac_decoder will got the follow error,**

E (22298) AAC_DECODER: AAC decoder encountered error 2052 30
E (22298) AUDIO_ELEMENT: [aac] ERROR_PROCESS, AEL_IO_FAIL

And then the system crash:

CORRUPT HEAP: multi_heap.c:477 detected at 0x3ffe8fc8
abort() was called at PC 0x40098b35 on core 0
0x40098b35: multi_heap_assert at /Users/Seven/media/iot/esp/esp-adf/esp-idf/components/heap/multi_heap.c:380
(inlined by) multi_heap_free_impl at /Users/Seven/media/iot/esp/esp-adf/esp-idf/components/heap/multi_heap.c:477

ELF file SHA256: 2077c7ac01ba7d24ccf45444f87b5f833158cf8f0b5abb4e5189be50d9556130

Backtrace: 0x4008c908:0x3f81a520
0x4008c908: invoke_abort at /Users/Seven/media/iot/esp/esp-adf/esp-idf/components/esp32/panic.c:715

After ESP32 successfully networked through the external 4G module, SIP could not log in. The prompt message is as follows. What is the problem? Is that some option setting problem?

log
W (21850) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (33050) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (44250) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (55450) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (66650) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (77850) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (89050) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (100250) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (111450) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (122650) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312
W (133850) SIP: CHANGE STATE FROM 0, TO 0, :func: sip_reconnect:312

If I don't want to see any ESP_LOG message in production,
I can't with MP3_DECODER since it doesn't care of my sdkconfig settings.

Hi all

Looking for any examples of streaming audio using OPUS codec. The codec examples in the ADF are for reading from a SD card and using the AAC codec and there's little documentation around it anyway. I'm trying to read from a TLV320 audio codec chip (got this working ok), compress the audio using OPUS codec, and send via UDP. Then receive it on another ESP32 and spit it back out again - basically a 2 way walkie talkie device.

The library has mentions of encoding OPUS but don't see anything on decoding OPUS? What am I missing please?

Cheers

Matt

Hello, I am newbie in esp world.
I install esp-idf from github and compile many examples but now I want to show esp-adf/examples
but what I need to do to compile projects?

Is there a way to customize the methods allowed in the requests?

esp-adf-libs/esp_codec/include/codec
/esp_jpeg_enc.h

8bit 灰度图 VGA分辨率