Multiple plugins of the same resource about unimrcp HOT 22 CLOSED

Supported in r697. 

Multiple plugins of the same resource can be specified and loaded
    <plugin>
      <engine name="Cepstral-Swift-1" class="mrcpcepstral" enable="1"/>
      <engine name="Demo-Synth-1" class="demosynth" enable="1"/>
      <engine name="Demo-Recog-1" class="demorecog" enable="1"/>
    </plugin>

One profile may use Cepstral-Swift-1 plugin, while another one Demo-Synth-1 plugin

    <profile name="MRCPv2-Default">
      <param name="signaling-agent" value="SIP-Agent-1"/>
      <param name="connection-agent" value="MRCPv2-Agent-1"/>
      <param name="media-engine" value="Media-Engine-1"/>
      <param name="rtp-factory" value="RTP-Factory-1"/>
      <pluginmap>
        <param name="speechsynth" value="Cepstral-Swift-1"/>
        <param name="speechrecog" value="Demo-Recog-1"/>
      </pluginmap>
    </profile>
    <profile name="MRCPv2-Alter">
      <param name="signaling-agent" value="SIP-Agent-2"/>
      <param name="connection-agent" value="MRCPv2-Agent-1"/>
      <param name="media-engine" value="Media-Engine-1"/>
      <param name="rtp-factory" value="RTP-Factory-1"/>
      <pluginmap>
        <param name="speechsynth" value="Demo-Synth-1"/>
        <param name="speechrecog" value="Demo-Recog-1"/>
      </pluginmap>
    </profile>

Sample unimrcpserver.xml configuration file is attached.

I was trying to test this, but I had only 1 PC (and no virtual machines...)
Although one unimrcpclient can create multiple sip-agents, there is no way to choose

the agent to run the synth app. So I created two directory trees, one for each 
client, each with its own configuration file.
However, the 2nd instance of the client always has trouble (both clients work if I

run them one after the other, but not simultaneously)
The second client will log the following warnings:

2009-01-14 22:39:45:715724 [INFO]   Start Media Processing Engine
2009-01-14 22:39:45:715724 [WARN]   Failed to Create Control Socket
2009-01-14 22:39:45:715724 [WARN]   Failed to Create MRCPv2 Agent Socket

Perhaps I made an error in the conf files (see attached files)

You can test it using 1 unimrcpclient and 1 unimrcpserver running on the same PC.
I've checked your unimrcpserver.xml, it looks good. Find attached modified
unimrcpclient.xml.
From unimrcpclient console input
> run synth MRCPv1-Default
or
> run synth MRCPv1-Alter

The signature is 
run "app_name" ["profile_name"]
However it seems I just forgot to mention the last optional param in help.

Thanks. 

Aha, I should have checked the ultimate documentation: the source code...
I'll check it later today.

Still makes me wonder though why running two or more instances of unimrcpclienct on

the same PC is not OK (i.e. Should I worry about the following scenario where I need

a seperate client for each IVR line, and each ivr line may want to "use" different

plugins (the best TTS for French and another best one for Dutch etc...)

> Still makes me wonder though why running two or more instances of unimrcpclienct on

the same PC is not OK

OK, I've just checked this. The issue is in hard coded port number used to create
control socket. See mrcp_client_connection.c:90. I used port 7856 temporarily,
actually any free port can be used. There are several easy workarounds, however it's
better to use wakeup pollset instead of control socket, as I did it for RTSP stack.
You may want to log another issue for this :)

> Should I worry about the following scenario where I need a seperate client for each
IVR line, and each ivr line may want to "use" different plugins
No, you don't. I guess you need separate signaling agent, but not the client.
I would not recommend to use several client instances for that. Typically you should
use the same client instance, while each IVR line can be connected through
appropriate profile / signaling agent.

Created the new issue...

Sometimes the IVR development environment (or those who built the IVR application)

will not allow you to use the multiple agent scenario, and of course one can create

another client-server/client layer. I have no particular IVR in mind though (and it

was some time ago I was involved in building IVR applications ...)

I must confess I suspected a more or less hard coded port, however I didn't check 
the code yet. Perhaps - hopefully soon - I will have a better understanding of the

the control socket - wakeup pollset design differences...

OK, it's not a great deal and it must be possible to run several instances of the
client on the same PC.
BTW, do you happen to verify the original issue. Does it work on your end?

I don't think I fully understand how it works or how it should work.
I wanted to try out the thread-safety of my TTS plugin.
So I want to synthesize a large text with multiple clients simultaneously.
What I see in the client is that the second run will not be able to complete the SIP

call until the first call has ended. 

I'm afraid there is some kind of misunderstanding here.
What I have implemented in the scope of this issue is to be able to register
different TTS plugins with unimrcpserver and bind/attach each plugin to one of
signaling agents. For instance, Cepstral TTS plugin is used, if call comes from
SIP-Agent-1. Demo TTS plugin is used, if call comes from SIP-Agent-2, and yet another
plugin is used if call comes from SIP-Agent-3. 

Is this what you are looking for?
If yes, please proceed to how to configure and run unimrcpclient 
Configure multiple signaling agents and profiles as I suggested in attached
unimrcpclient.xml (Comment 5).
Run unimrcpclient.exe

> run synth MRCPv2-Default - to make call via SIP-Agent-1

> run synth MRCPv2-Alter - to make call via SIP-Agent-2

If you still have problems with the setup mentioned above, please upload logs and
configs of server and client to check.

I guess I understand it right. If you place calls one after the other is works fine.

What I don't understand is whether one unimrcpclient should be able to allow two (or

more) SIP agents to make calls in parallel. 
Things go wrong when using the two different agents in parallel, but I can't tell 
exactly what - I sometimes end up with two .pcm files of rougly the same size; to 
the demosynth "copy" of demo.pcm silence is appended, extending the duration 
to "match" the larger pcm file resulting from the fluency plugin. Additionally, one

call has to wait for the other to complete.

I did not thorougly check the attached logs yet. So I don't know whether this is a

client or a server (or plugin) problem (or a client "feature"). I'll try some more

tests from another machine to see if it is the unimrcpserver that is causing this 
behaviour . However, it took a while to figure out how to distribute the dll's for

the release versions of the software (I happened to use MSVC08 SP1, which caused 
trouble because I apparently installed incompatible "SP0" MSVC80 dll's from a 
microsoft vcredist*.exe). More later.

Well, I see we are on the right track.

Please note, unimrcpclient alllows to configure [0 - n] SIP signaling agents, while
each agent allows to make [0 - m] SIP calls in parallel.

What I've observed from logs so far.
I see SIP/MRCPv2 sessions with in-progress SPEAK requests in parallel. I'm pretty
sure, the problem is not in signaling layer, neither in SIP nor in MRCPv2 stacks. My
guess is problem is in server side, most probably in the way media processing is
implemented in Fluency plugin.
There is a note on top of source files of plugins
 * 4. Methods (callbacks) of the MPF engine stream MUST not block.
Did you follow this instruction in synth_stream_read?

The 0 - n agents doing 0 - m calls, that sounds good! I'm still deeply impressed by

your work. 
And you are almost right. Not the MPF engine, but the MRCP engine. The problem lies

in the "fluency_synth_channel_speak" function. The actual synthesis has to be 
carried out in a separate thread, and that's not the way I implemented it. And to 
make things worse I added a apr_sleep as to not let the TTS engine generate more 
audio samples than can be retrieved from the client (the MPF engine). Sleeping in 
this case is blocking... (and I was thinking, this would yield some other threads)

Unfortunately, in contrast to the cepstral/swift TTS engine, I have no callback 
function to retrieve the generated audio. I'll do some more homework...

I still don't fully understand why two different SIP agents using two different 
plugins have problems though - because there is only one connection agent ? - and 
having multiple connection agents ... ? 

>The problem lies in the "fluency_synth_channel_speak" function. The actual synthesis
has to be carried out in a separate thread, and that's not the way I implemented it

OK, this is definitely the reason.

> I still don't fully understand why two different SIP agents using two different 
plugins have problems though - because there is only one connection agent ? - and 
having multiple connection agents ... ? 

See the Sequence Diagram below
http://unimrcp.webhosting-for-free.com/ea/index.htm?goto=3:4:6:3:392

MRCP Server task/thread is the main entity in the hierarchy. It processes incoming
messages from Signaling, Connection agents, Media and Resource processing engines.

For instance, "fluency_synth_channel_speak" request is called from server task
context, thus it MUST NOT block, instead the request must be processed asynchronously.

Thanks! I did not find the sequence diagram yet. I thought I clicked my way through

the ea docs, but I never saw the sequence diagram (I didn't find anything of 
interest in the plugin directory of ea). The activity and sequence diagrams of the

MRCP client and server deserve links from the project home page! 

I should go back to school and learn to read properly - and take more time to study

the project before starting to code. And while back at school, I should do some 
reading on software design, UML and enterprise architect... The quality of your work

keeps on amazing me. In fact the project is much more mature than I ever expected.

To clarify my previous post (as far as I currently understand it): the hard coded 
port for the control socket(s) blocks the usage of multiple connection agents, even

though you may configure multiple connection agents. However, as you explained, it

would not have made a difference with respect to my problematic code.

BTW do you want another issue for the missing help parameters :-) ?

1.UML and docs
I know this stuff is not complete enough, and I have been trying to improve it as I
go along in background. Sometimes I have to sleep :)

2. Multiple connection agents in client
I'll address this issue, when possible. Anyway it would make no difference with
respect to server/plugin integration issue you experienced.

3. Missing help parameters
This is a trivial one, which I non intentionally forget to fix. I'll try to fix the
help during the day, otherwise please log it as another issue.

I saw you fixed the missing help parameter. Thanks.

I did another attempt at building a better plugin.
Somehow the plugins only log to the console - is that intentional?
(didn't check the documentation yet ;-)

Logger is probably the only static/global variable used in the project. See
apt_log.c:51. I prefer to have one globally available logger not to burden the
signature of the log function. Thus one instance of logger is created in the context
of unimrcpserver executable and yet another one in the context of your plugin dll.
You can initialize the logger the way you need in the mrcp_plugin_create() function.
For instance, 
    /* set the log level */
    apt_log_priority_set();
    /* set the log output mode */
    apt_log_output_mode_set();
    /* open log file */
    apt_log_file_open(log_file_path,MAX_LOG_FILE_SIZE,pool);

I created the threaded fluency_speak version and it works a whole lot better now.

With respect to logging: do you have a suggestion to retrieve the server's command

line options and use them in the dynamically loaded plugin?

Nice to see you are making progress implementing plugin.

Hopefully soon I'll be able to proceed to the issue-11, which concerns to
capabilities of plugins, including logging. I think plugin should provide interface
to set logging options from the core depending on command line options or
configuration of the server.