doe300 / ohmcomm Goto Github PK

Write useful UnitTests.
We need to write extensive tests for:

• RTAudioWrapper (as far as possible)
• AudioProcessors (as far as possible)
• NetworkWrappers
• RTCPPackageHander
• RTPPackageHander
• RTPBuffer
• UserInput
• ConfigurationModes

Since there has been major changes in the project structure it is necessary to write a documentation. Also refactoring might be useful.

Implement a thread listening on a socket, and writing RTPPackages into RTPBuffer

We need better error-handling, particular in these places:

• configuration of audio-processors, i.e. the program continues running if a processor fails to configure
• configuration of stream-parameters
• configuration of audio-handler (errors on accessing devices, e.g. no input device, device already in use, ...)
• audio-loop
• receiver-loop

We either could crash the program with informative errors or gracefully shutdown all the running threads and exit the program

Setup folder structure for LaTeX and add to git.
Configure page layout, design, text form (report)

Modify cmake to automatically call ./configure for libraries (for linux, mac)

How about marking all abstract/interface-like classes with an pre- or suffix.

For example:
RTPBufferHandler -> I_RTPBuffer

Implement simple audio output to wave file streamer
-http://www3.nd.edu/~dthain/courses/cse20211/fall2013/wavfile/

Implement RTP jitter buffer, supporting multithreaded read/write.
More precisly: Support one write-thread and one thread reading packages.
The buffer should return some kind of silent-package if its contents are empty.
Provide an interface of public methods for both threads

Inorder to write useful UnitTests it necessary to write testable methods and classes. Consider this while developing.

Commit 484e5d9
Is not compiling because winsock2.h is included more than one time.

It is included in

• NetworkWrapper.cpp (we agreed that the methods getLastError() and closeSocket() does not belong to NetworkWrapper but rather to UDPWrapper didnt we?)
• RTPListener.h (Everything related to Network should be either in NetworkWrapper (if general information) or in UDPWrapper (if UDP-related information) )

Find and use an implementation for the Opus audio codec (http://www.opus-codec.org/).
Implement as AudioProcessor

• Split send and receive -part of AudioProcessor into processOutput() and processInput()
• Also restructure AudioProcessors to list, see #10

Currently both computers need to have the same audio configuration (sample-rate, format, samples per package).
We need to find a way to either

• allow for varying input and output configuration
  or
• agree on a common configuration.

Possible solutions:

• use custom payload-format with the audio-configuration as parameter
• write SIP stack to agree on configuration

See #35

Allow for non-interactive program run by adding parameters to the command line.

Requirements:

• unified and extensible way to read and interpret parameters
• help-parameter with automatically generated help-page based on available parameters
• allow for all values set in interactive mode to be set via parameters
• provide default values for as many parameters as possible

Application crashes on exit.

Wrap RtAudio into a Adapter, allowing to extend OHMComm to be based on different libraries, e.g. Portaudio (http://www.portaudio.com/).
This Adapter must provide following methods:

• configure()
• startXXX(): starts the in-/out-mode
• stop() / resume()
• method to register callback(s)

• Remove global AudioConfiguration
• Allow for processors for modifying (returning) the number of buffered bytes to be provided for next processor
• move maximum buffer bytes to StreamData
• Add methods to AudioProcessors to provide supported audio-configuration (esp. format, samplerate, bufferframes)
• Allow for AudioHandler to use accepted values from AudioProcessors
• add AudioConfiguration as parameter to AudioHandler#configure()

Few things to clear up:

• Why was the CSRCs and the "old" read/write-methods discarded?
• Why not set package-data once (i.e. in constructor)?
• What are all those void* for?

Few things to fix:

• need to generate random SSRC and reuse it for every sent package
• need to generate random initial timestamp and increment on every call (time linear!)

• Currently, sender and receiver have to have same frame-size, channels and number of buffered frames - Solved by passive configuration

After making AudioProcessor::configure virtual, to provide overriding functionality in ProcessorOpus for the initialisation of encoder/decoder objects (see #18 ). The Programm crashes with an Access violation executing.
Thrown after executing RtAudioWrapper::prepare() at line 369.
Object Informations:

-       this    0x00551778 {semaphore_waitForMainThread=0x00000104 semaphore_waitForWorkerThread=0x00000108 bufferAudioOutput=...}  RtAudioWrapper *
-       AudioHandler    {flagAudioConfigSet=true flagPrepared=true audioProcessors={ size=2 } ...}  AudioHandler
-       __vfptr 0x013dce78 {OHMComm.exe!const RtAudioWrapper::`vftable'} {0x01332149 {OHMComm.exe!RtAudioWrapper::startRecordingMode(void)}, ...}   void * *
        [0] 0x01332149 {OHMComm.exe!RtAudioWrapper::startRecordingMode(void)}   void *
        [1] 0x01331433 {OHMComm.exe!RtAudioWrapper::startPlaybackMode(void)}    void *
        [2] 0x01331b4f {OHMComm.exe!RtAudioWrapper::startDuplexMode(void)}  void *
        [3] 0x01331686 {OHMComm.exe!RtAudioWrapper::setConfiguration(struct AudioConfiguration const &)}    void *
        [4] 0x01331852 {OHMComm.exe!RtAudioWrapper::suspend(void)}  void *
        [5] 0x013310ff {OHMComm.exe!RtAudioWrapper::resume(void)}   void *
        [6] 0x013313e8 {OHMComm.exe!RtAudioWrapper::stop(void)} void *
        [7] 0x013316f9 {OHMComm.exe!RtAudioWrapper::reset(void)}    void *
        [8] 0x01331e56 {OHMComm.exe!RtAudioWrapper::playData(void *,unsigned int)}  void *
        [9] 0x0133218f {OHMComm.exe!RtAudioWrapper::setDefaultAudioConfig(void)}    void *
        [10]    0x01331596 {OHMComm.exe!RtAudioWrapper::prepare(void)}  void *
        [11]    0x0133184d {OHMComm.exe!RtAudioWrapper::getBufferSize(void)}    void *
        flagAudioConfigSet  true    bool
        flagPrepared    true    bool
-       audioProcessors { size=2 }  std::vector<AudioProcessor *,std::allocator<AudioProcessor *> >
        [size]  2   int
        [capacity]  2   int
-       [0] 0x0043fafc {networkObject=0x00551ab0 {Socket=292 localAddress={sin_family=2 sin_port=14640 sin_addr=...} ...} ...}  AudioProcessor *
-       [ProcessorRTP]  {networkObject=0x00551ab0 {Socket=292 localAddress={sin_family=2 sin_port=14640 sin_addr={S_un={...} } ...} ...} ...}   ProcessorRTP
+       AudioProcessor  {name="RTP-Processor" } AudioProcessor
+       networkObject   0x00551ab0 {Socket=292 localAddress={sin_family=2 sin_port=14640 sin_addr={S_un={S_un_b={...} S_un_w=...} } ...} ...}   NetworkWrapper *
+       rtpPackage  0x00000000 <NULL>   RTPPackageHandler *
+       rtpBuffer   0x00551598 unique_ptr {bufferMutex=0x00000128 ringBuffer=0x00564108 {isValid=true (205) header={version=1 padding=1 extension=...} ...} ...}    std::unique_ptr<RTPBuffer,std::default_delete<RTPBuffer> > *
-       __vfptr 0x013dcc00 {OHMComm.exe!const ProcessorRTP::`vftable'} {0x0133110e {OHMComm.exe!ProcessorRTP::getSupportedAudioFormats(void)}, ...} void * *
        [0] 0x0133110e {OHMComm.exe!ProcessorRTP::getSupportedAudioFormats(void)}   void *
        [1] 0x01331208 {OHMComm.exe!ProcessorRTP::getSupportedSampleRates(void)}    void *
        [2] 0x013322a2 {OHMComm.exe!AudioProcessor::configure(struct AudioConfiguration)}   void *
        [3] 0x0133143d {OHMComm.exe!ProcessorRTP::processInputData(void *,unsigned int,struct StreamData *)}    void *
        [4] 0x013310be {OHMComm.exe!ProcessorRTP::processOutputData(void *,unsigned int,struct StreamData *)}   void *
+       name    RTP-Processor   std::basic_string<char,std::char_traits<char>,std::allocator<char> >
-       [1] 0x0043fabc {OpusEncoderObject=0x0056ed60 {celt_enc_offset=42784 silk_enc_offset=18220 silk_mode={nChannelsAPI=...} ...} ...}    AudioProcessor *
-       [ProcessorOpus] {OpusEncoderObject=0x0056ed60 {celt_enc_offset=42784 silk_enc_offset=18220 silk_mode={nChannelsAPI=2 ...} ...} ...} ProcessorOpus
+       AudioProcessor  {name="Opus-Processor" }    AudioProcessor
+       OpusEncoderObject   0x0056ed60 {celt_enc_offset=42784 silk_enc_offset=18220 silk_mode={nChannelsAPI=2 nChannelsInternal=...} ...}   OpusEncoder *
+       OpusDecoderObject   0x0057bb40 {celt_dec_offset=8628 silk_dec_offset=84 channels=2 ...} OpusDecoder *
        OpusApplication 2048    int
+       ErrorCode   0x0043f92c {0}  int *
        lengthEncodedPacketWorkaround   3435973836  unsigned int
-       __vfptr 0x013dcbe4 {OHMComm.exe!const ProcessorOpus::`vftable'} {0x013314ce {OHMComm.exe!ProcessorOpus::getSupportedAudioFormats(void)}, ...}   void * *
        [0] 0x013314ce {OHMComm.exe!ProcessorOpus::getSupportedAudioFormats(void)}  void *
        [1] 0x01331591 {OHMComm.exe!ProcessorOpus::getSupportedSampleRates(void)}   void *
        [2] 0x0133229d {OHMComm.exe!ProcessorOpus::configure(struct AudioConfiguration)}    void *
        [3] 0x013316b3 {OHMComm.exe!ProcessorOpus::processInputData(void *,unsigned int,struct StreamData *)}   void *
        [4] 0x01331e10 {OHMComm.exe!ProcessorOpus::processOutputData(void *,unsigned int,struct StreamData *)}  void *
+       name    Opus-Processor  std::basic_string<char,std::char_traits<char>,std::allocator<char> >
+       [Raw View]  0x00551780 {...}    std::vector<AudioProcessor *,std::allocator<AudioProcessor *> > *
+       audioConfiguration  {outputDeviceID=1 inputDeviceID=6 outputDeviceChannels=2 ...}   AudioConfiguration
        semaphore_waitForMainThread 0x00000104  void *
        semaphore_waitForWorkerThread   0x00000108  void *
        bufferAudioOutput   0x0056cf20  void *
        BufferAudioOutHasData   false   bool
+       rtaudio {rtapi_=0x00551b38 {coInitialized_=true deviceEnumerator_=0x0055acf8 <Information not available, no symbols loaded for MMDevAPI.dll> } }    RtAudio
+       input   {deviceId=6 nChannels=2 firstChannel=0 }    RtAudio::StreamParameters
+       output  {deviceId=1 nChannels=2 firstChannel=0 }    RtAudio::StreamParameters
        outputBufferByteSize    7680    unsigned int
        inputBufferByteSize 7680    unsigned int
+       streamData  0x005630d8 {nBufferFrames=0 streamTime=0 maxBufferSize=0 }  StreamData *

Unhandled exception at 0x7547CB49 in OHMComm.exe: 0xC0000005: Access violation executing location 0x00000000.

Based on code 7fc2e64

Processors can't negotiate an AudioFormat if one AudioProcessor provides only AUDIO_FORMAT_SINT16 or AUDIO_FORMAT_SINT8 or AUDIO_FORMAT_SINT24 or AUDIO_FORMAT_SINT32 or AUDIO_FORMAT_FLOAT64 it only works with AUDIO_FORMAT_FLOAT32 and of course with AUDIO_FORMAT_ALL.
This affects issue #18 , Code base 7fc2e64

Implement Device configuration with rtaudio

Status of tests:

• transmission over LAN: good quality, tested by @Kaayy-J
• transmission over WLAN: good quality~~, but package loss, program crashes,~~ tested by @J0n8s
• transmission over WWW: good quality, tested by @J0n8s and @Kaayy-J

Write simple UDP-wrapper for to write/receive audio

If the rtaudio Bit Mask is 16 it only prints RTAUDIO_FLOAT32, i assume it should print RTAUDIO_SINT16, RTAUDIO_SINT24 and RTAUDIO_FLOAT32.
I think the error is that the 16 is interpreted as an decimal(10000 binary) number but should be interpreted as an hexadecimal number 0x16(10110 binary).

Implement usage of FLAC codec as AudioProcessor

• Work through tutorials in C++ and read into rtaudio http://www.music.mcgill.ca/~gary/rtaudio/.
• Read into cmake
• Set up IDE for C++ Development

We already have a working but not used RTCP-implementation.

• Find some way to distinguish RTP-packages from RTCP-packages (I belive there is a bit pattern to separate them).
• Use the RTCP BYE-message to fix the problem addressed in #47 in an more clear matter.
• Use the RTCP APP-defined package to transmit the available/used stream-configuration. This could be used to aggree to the same configuration as of #42 .
• Implement usage of SDES: load from file and send on start-up, receiving side prints info to console
• Implement Sender and Reception Reports. Finished as of e69409e

Test the Project under the new Visual Studio 2015 (Released 20. Juli 2015)

Write a function to select and add AudioProcessors to the chain.
Note: The order in which the Processors are being added is important and should be made recognizable to the user.

• Create a Whatsapp-Group.
• Compile and share lesson plans

Set number of channels to 2 for stereo:

• in manual configuration
• in default configuration
• in RTP Package payload

improve modularity and clarity of audio-processor-chain by using a list

• remove old link
• call AudioProcessor#configure() for all processors
• chain AudioProcessor#process(...)

Please try to use the C++11 standard in order to have consistent code.
Also try to avoid C-Code for example "malloc, free" etc.

On my local machine, same error as Travis-CI:

opus/libsilk_float.a(encode_frame_FLP.c.o): In function `silk_encode_frame_FLP':
xxx/OHMComm/lib/opus-1.1/silk/float/encode_frame_FLP.c:116: undefined reference to `silk_LP_variable_cutoff'
opus/libsilk_float.a(pitch_analysis_core_FLP.c.o): In function `silk_pitch_analysis_core_FLP':
xxx/OHMComm/lib/opus-1.1/silk/float/pitch_analysis_core_FLP.c:141: undefined reference to `silk_resampler_down2'
xxx/OHMComm/lib/opus-1.1/silk/float/pitch_analysis_core_FLP.c:148: undefined reference to `silk_resampler_down2_3'
xxx/OHMComm/lib/opus-1.1/silk/float/pitch_analysis_core_FLP.c:157: undefined reference to `silk_resampler_down2'
opus/libsilk_float.a(wrappers_FLP.c.o): In function `silk_A2NLSF_FLP':
xxx/OHMComm/lib/opus-1.1/silk/float/wrappers_FLP.c:50: undefined reference to `silk_A2NLSF'
opus/libsilk_float.a(wrappers_FLP.c.o): In function `silk_process_NLSFs_FLP':
xxx/OHMComm/lib/opus-1.1/silk/float/wrappers_FLP.c:83: undefined reference to `silk_process_NLSFs'
opus/libsilk_float.a(wrappers_FLP.c.o): In function `silk_NSQ_wrapper_FLP':
xxx/OHMComm/lib/opus-1.1/silk/float/wrappers_FLP.c:163: undefined reference to `silk_NSQ_del_dec'
xxx/OHMComm/lib/opus-1.1/silk/float/wrappers_FLP.c:166: undefined reference to `silk_NSQ'
opus/libsilk_float.a(wrappers_FLP.c.o): In function `silk_quant_LTP_gains_FLP':
xxx/OHMComm/lib/opus-1.1/silk/float/wrappers_FLP.c:196: undefined reference to `silk_quant_LTP_gains'
opus/libsilk_float.a(find_LPC_FLP.c.o): In function `silk_find_LPC_FLP':
xxx/OHMComm/lib/opus-1.1/silk/float/find_LPC_FLP.c:73: undefined reference to `silk_interpolate'

Create Profiler for callback-method:

• measure execution times for every AudioProcessor
• make optional, so it doesn't affect performance if disabled

• Check that ParameterConfiguration sets every used parameter
• Improve test to test for audio-processors

Write a NetworkWrapper

• Writes samples from rtaudio-buffer to audio file
• Reads samples from file and writes it into rtaudio-buffer
• Filenames are configurable (see AudioProcessor#configure())

P.S. Maybe NetworkWrapper is not such a good name, rename it to something like IOWrapper or AudioIO, ...

Implement as Adapter on top of socket.
Links

• https://www.c-plusplus.net/forum/169861-full
• http://www.cplusplus.com/reference/istream/istream/
• http://stackoverflow.com/questions/14086417/how-to-write-custom-input-stream-in-c
• http://www.math.hkbu.edu.hk/parallel/pgi/doc/pgC++_lib/stdlibug/cre_2288.htm
• http://research.edm.uhasselt.be/~jori/page/index.php?n=CS.Jrtplib

Add support for ipv6 addresses.

Write method to automatically select best frame-size!

We need some mechanism to stop the AudioHandler and its thread other than exiting the main-method.

• Make the AudioHandler accessible, so the stop-method can be called
• Make sure all other threads/objects are shut down correctly
• Preferably create an universal accessible way to stop all threads, close network, print statistics.

This would fix #47

Setup CppTest in order to write unit tests

Optionally log profiling/statistics to a file.

Commit 9cbfe8a broke tests under Linux. Output as follows:

TestAudioIO: 0/2
RtApiAlsa::closeStream(): no open stream to close!

TestAudioIO: 2/2, 100% correct in 0.012452 seconds
Total: 2 tests, 100% correct in 0.012452 seconds
*** Error in `/home/daniel/workspace/OHMComm/build/test/Tests': free(): corrupted unsorted chunks: 0x0000000001d42840 ***
*** Error in `/home/daniel/workspace/OHMComm/build/test/Tests': malloc(): memory corruption: 0x0000000001d40590 ***

RUN FINISHED; Terminated; real time: 4s; user: 0ms; system: 0ms

Did you change anything with malloc/free or buffers in or around AudioHandler?

P.S. The TestAudioIO should be renamed.

Travis-CI seem to have granted us Mac OS X compilation without any notice.
For the compilation to work under Mac OS X, the travis-script must be modified (there is no 'sudo apt-get ...' for mac)

• Check existance and version of libraries in mac (gcc >= 4.8 or disable under mac?, clang, sound-library)
• Retrieval of required libraries depending on the system used
• Test build

add minimum number of buffered package before playing audio