Code Monkey home page Code Monkey logo

assistant-sdk-cpp's Introduction

Google Assistant SDK for devices - C++

Requirements

This project is officially supported on Ubuntu 14.04. Other Linux distributions may be able to run this sample.

Refer to the Assistant SDK documentation for more information.

Setup instructions

Clean Project

If you have not setup this project before, you can skip this step.

sudo apt-get purge libc-ares-dev  # https://github.com/grpc/grpc/pull/10706#issuecomment-302775038
sudo apt-get purge libprotobuf-dev libprotoc-dev
sudo rm -rf /usr/local/bin/grpc_* /usr/local/bin/protoc \
    /usr/local/include/google/protobuf/ /usr/local/include/grpc/ /usr/local/include/grpc++/ \
    /usr/local/lib/libproto* /usr/local/lib/libgpr* /usr/local/lib/libgrpc* \
    /usr/local/lib/pkgconfig/protobuf* /usr/local/lib/pkgconfig/grpc* \
    /usr/local/share/grpc/

Build Project

  1. Clone this project
git clone https://github.com/googlesamples/assistant-sdk-cpp.git
cd assistant-sdk-cpp
export PROJECT_PATH=$(pwd)
  1. Install dependencies
sudo apt-get install autoconf automake libtool build-essential curl unzip pkg-config
sudo apt-get install libasound2-dev  # For ALSA sound output
sudo apt-get install libcurl4-openssl-dev # CURL development library
  1. Build Protocol Buffer, gRPC, and Google APIs
git clone -b $(curl -L https://grpc.io/release) https://github.com/grpc/grpc
GRPC_PATH=${PROJECT_PATH}/grpc
cd ${GRPC_PATH}

git submodule update --init

cd third_party/protobuf
./autogen.sh && ./configure && make
sudo make install
sudo ldconfig

export LDFLAGS="$LDFLAGS -lm"
cd ${GRPC_PATH}
make clean
make
sudo make install
sudo ldconfig

cd ${PROJECT_PATH}
git clone https://github.com/googleapis/googleapis.git
cd googleapis/
git checkout 980cdfa
make LANGUAGE=cpp
  1. Make sure you setup environment variable $GOOGLEAPIS_GENS_PATH
export GOOGLEAPIS_GENS_PATH=${PROJECT_PATH}/googleapis/gens
  1. Build this project
cd ${PROJECT_PATH}
make run_assistant
  1. Get credentials file. It must be an end-user's credentials.
  • Go to the Actions Console and register your device model, following these instructions
  • Move it in this folder and rename it to client_secret.json
  • run get_credentials.sh in this folder. It will create the file credentials.json.
  1. Start one of the run_assistant samples:
./run_assistant_file --input ./resources/weather_in_mountain_view.raw --output ./response.wav --credentials ./credentials.json
aplay ./response.wav --rate=16000 --format=S16_LE

On a Linux workstation, you can alternatively use ALSA for audio input:

./run_assistant_audio --credentials ./credentials.json

You can use a text-based query instead of audio. This allows you to continually enter text queries to the Assistant.

./run_assistant_text --credentials ./credentials.json

This takes input from cin, so you can send data to the program when it starts.

echo "what time is it" | ./run_assistant_text --credentials ./credentials.json

To change the locale, include a locale parameter:

echo "Bonjour" | ./run_assistant_text --credentials ./credentials.json --locale "fr-FR"

Default Assistant gRPC API endpoint is embeddedassistant.googleapis.com. If you want to test with a custom Assistant gRPC API endpoint, you can pass --api_endpoint CUSTOM_API_ENDPOINT.

Enabling screen output

To get a visual output from the Assistant, provide a command to be run alongside every step of the conversation. It will execute that command along along with a provided argument of a temporary HTML file.

echo "what time is it" | ./run_assistant_text --credentials ./credentials.json --html_out google-chrome

After you enter text, it will run google-chrome /tmp/google-assistant-cpp-screen-out.html.

If you prefer a different program, use that argument instead.

assistant-sdk-cpp's People

Contributors

fleker avatar mpoullet avatar rajveermalviya avatar rochefort8 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

assistant-sdk-cpp's Issues

SSL handshake error on custom Android-x86 board

Hello google-assistant team,

I have built google-assistant (and dependencies) using Android NDK. When running the executable on Android-x86 emulator, the whole flow works and I get to hear the response (have added the plumbing logic to play it on Android).

However, when running the same on a custom Android-x86 board, the following handshake error shows up:
11-11 11:14:52.698 6980-9508/com.test.app E/GRPC: ssl_transport_security.c:921] Handshake failed with fatal error SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED.
11-11 11:14:52.892 6980-9508/com.test.app E/GRPC: ssl_transport_security.c:921] Handshake failed with fatal error SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED.
..
..
11-11 11:14:53.388 6980-9508/com.test.app E/GRPC: ssl_transport_security.c:921] Handshake failed with fatal error SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED.

To get it to run on the emulator, I had to do the following:

  1. set the environment variable GRPC_DEFAULT_SSL_ROOTS_FILE_PATH to point it to Google's PEM file (https://pki.google.com/roots.pem)
  2. also pass this pem file location to
    std::ifstream file("robots.pem");

However, doing the same on the custom board does not help.

Any clues why this may be happening?

POP noise at the end Google Feedback response

After some analysis, I confirm that there is a pop like noise at the end of google feedback response particularly for the responses of "what is the time".
Here I have confirmed that the noise data looks like an audio sample.
It doesn't match with the noise data of Capacitor switching drain.
This POP like noise arrives once the snd_pcm_drain is called.

Here I'm running this SDK as a loop to generate the continuous feedback response.
I tried my possible methods to find the reason, but I couldn't proceed further.
Seeking your support regarding this.

Google Assistant SDK not detecting END_OF_UTTERANCE

Hey,
What are the probable reasons for the SDK not detecting an END_OF_UTTERANCE for a request?
There have been a lot of instances, when the channel is created properly, but the sdk doesn't come up with a proper response (including end-of-utterance). Can it be because of some authentication failure that I am unable to track or is it something else?

Speech coding and decoding ,Will the efficiency be very low ?

converse_config->mutable_audio_in_config()->set_encoding(
		AudioInConfig::LINEAR16);
	converse_config->mutable_audio_in_config()->set_sample_rate_hertz(16000);
	converse_config->mutable_audio_out_config()->set_encoding(
		AudioOutConfig::LINEAR16);
	converse_config->mutable_audio_out_config()->set_sample_rate_hertz(16000);

Will the efficiency be very low For the network ?

Failed to connect to Google API Server

Hi,
I tried building the SDK on alpine linux. There is no problem during the building. But when I execute the application, it report following error:
[develop] /assistant-sdk-cpp # ./run_assistant --audio_input ALSA_INPUT --creden
tials_file ./credentials.json --credentials_type USER_ACCOUNT
assistant_sdk robots_pem:
assistant_sdk CreateCustomChannel(embeddedassistant.googleapis.com:443, creds, arg)

E0530 01:06:44.079278868 3070 resolver_registry.cc:80] don't know how to resolve 'embeddedassistant.googleapis.com:443' or 'dns:///embeddedassistant.googleapis.com:443'
E0530 01:06:44.079413606 3070 resolver_registry.cc:80] don't know how to resolve 'dns:///embeddedassistant.googleapis.com:443' or 'dns:///dns:///embeddedassistant.googleapis.com:443'
E0530 01:06:44.079508658 3070 channel.cc:98] channel stack builder failed: {"created":"@1527642404.079472356","description":"resolver creation failed","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":695}

About the Terms of Service

Some questions about the terms of service want to ask。

Our company needs to make a smart speaker, intends to implant c++ code, would like to ask if this code needs to be commercial, in addition to complying with the open source protocol, which protocols do you need to comply with Google? Or is there a charge for using this code? Or do you only need to follow the open source to open source this code?

What resources does Google offer (such as music, alarm clocks, maps, navigation, etc.) for our developers? For example, music, can you play the music of a third-party platform on a smart speaker through Google Assistant? If so, are there any agreements that need to be adhered to or need to be charged accordingly? If you need to pay, does Google need to charge in this regard? If you access a third party, do you need to apply for permission to use the three-party function, or do you need to solve the three-party permission to use it smoothly?

How do I know what resources Google has provided for our developers? (Google Assistant should be able to complete a request and response through Action. Actions Directory should be product-related Actions made by other companies themselves. I know only this, but still do not know where to know Google offers Resources.) If these resources are listed in the Actions Directory, where can you know how to implement the above music playback function?

There is a bit of a headache about the terms of service recently, but there is a need to understand, so I would like to ask, I hope there are some directions. Thank you.

json_util_test.cc:23:75: error: ‘GetCustomResponseOrNull’

Where's the implementation of GetCustomResponseOrNull supposed to be? Grepping the build tree, I see no sign of it.

src/json_util_test.cc:23:75: error: ‘GetCustomResponseOrNull’ was not declared in this scope
   std::unique_ptr<std::string> result = GetCustomResponseOrNull(input_json);

Error : running assistant-sdk-cpp in docker container

Hi,

I am following your doc in CentOS/7 docker container, with the aim of building a toy chatbot in the
cloud. May I kindly know how could I resolve these errors?

+The log is attached as below.

Kind regards,
Tamama

[root@525a6608763d assistant-sdk]# LD_LIBRARY_PATH=runtime bin/run_assistant --text_input "What time is it?" --credentials_file asset/credentials.json --credentials_type USER_ACCOUNT
Using locale en-US
ALSA lib confmisc.c:767:(parse_card) cannot find card '0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2501:(snd_pcm_open_noupdate) Unknown PCM default
AudioOutputALSA snd_pcm_open returned -2
Segmentation fault (core dumped)

Handle a single request and no hotword detection?

Hi,
I checked the code, seems this sample can only handle the single command. After handle the request, it will exit. And there is no hotword detection.

Is there a full sample that can detect the hotword "Hi Google" and always listen to the microphone, without manually start up the app?

make run_assistant failed.

Just followed the readme and try to build the run_assistant..getting an error?
Please help to check what is the issue?
~/Linux-Google-Home/assistant-sdk-cpp$ make run_assistant
find: /googleapis/gens/google/api': No such file or directory find: /googleapis/gens/google/rpc': No such file or directory
find: `/googleapis/gens/google/type': No such file or directory
protoc -I=./googleapis/google/assistant/embedded/v1alpha2 --proto_path=.:./googleapis
--cpp_out=/googleapis/gens/google/assistant/embedded/v1alpha2
--grpc_out=/googleapis/gens/google/assistant/embedded/v1alpha2
--plugin=protoc-gen-grpc=/usr/local/bin/grpc_cpp_plugin
./googleapis/google/assistant/embedded/v1alpha2/embedded_assistant.proto
/googleapis/gens/google/assistant/embedded/v1alpha2/: No such file or directory
make: *** [/googleapis/gens/google/assistant/embedded/v1alpha2/embedded_assistant.pb.cc] Error 1
__git_ps1: command not found

Questions about --api_endpoint option settings

In Google's tutorial, the Goole Assistant Library can set traits via action.json. Can it be set in the C++ version of the Google Assistant Service?

According to the tutorial you provided, if you want to test the custom Assistant gRPC API endpoint, you can set it with the --api_endpoint option. I probably browsed the code, --api_endpoint followed by a URL: port, so how should this custom parameter be obtained?

sll handshake error

assistant_sdk robots_pem:
assistant_sdk CreateCustomChannel(embeddedassistant.googleapis.com:443, creds, arg)

E1125 21:57:19.169789182 3080 ssl_transport_security.c:940] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.

Cross compile program get stuck at creating ClientReaderWriter

I already can build(locally) and run this sample on my Debian PC and my raspberry pi.

Weird things happens when I trying to cross compile this sample on my Debian PC with arm-linux-gnueabihf-g++, I have modified LDFLAGS to make it compile with specific arm grpc lib(which I copied from raspberry pi), build successfully and I run it on raspberry pi. Everything runs good until program get here:

std::shared_ptr<ClientReaderWriter<AssistRequest, AssistResponse>>
        stream(std::move(assistant->Assist(&context)));

It will stuck at here forever because assistant->Assist(&context) will never return.

Then I tried to copy the whole modified project with specific arm grpc lib to raspberry pi and build it on raspberry pi locally, it works well again...

Any comments and suggestions will be appreciated, thanks a lot

Build error and Fix

Final stage of build process, "make run_assisitant" may give me the following error,

 g++ /home/ubuntu/git/assistant-sdk-cpp//googleapis/gens/google/assistant/embedded/v1alpha2/embedded_assistant.pb.o /home/ubuntu/git/assistant-sdk-cpp//googleapis/gens/google/assistant/embedded/v1alpha2/embedded_assistant.grpc.pb.o googleapis.ar src/json_util.o src/base64_encode.o src/run_assistant_text.o  -lm -lm  -lgrpc_cronet -Wl,--no-as-needed -lgrpc++_reflection -Wl,--as-needed -lprotobuf -lpthread -ldl -lcurl  -o run_assistant_text
/usr/bin/ld: /home/ubuntu/git/assistant-sdk-cpp//googleapis/gens/google/assistant/embedded/v1alpha2/embedded_assistant.grpc.pb.o: undefined reference to symbol '_ZN4grpc6g_glipE'
//usr/local/lib/libgrpc++.so.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:104: recipe for target 'run_assistant_text' failed
make: *** [run_assistant_text] Error 1

(Means "Undefined symble grpc::g_glip according to my small investigation)

I found this may be fixed with the following change,

iff --git a/Makefile b/Makefile
index b3e4cb3..72e9177 100644
--- a/Makefile
+++ b/Makefile
@@ -64,8 +64,8 @@ LDFLAGS += $(GRPC_GRPCPP_LDLAGS) \
            -lprotobuf -lpthread -ldl -lcurl
 else
 LDFLAGS += $(GRPC_GRPCPP_LDLAGS) \
-           -lgrpc_cronet -Wl,--no-as-needed -lgrpc++_reflection \
-           -Wl,--as-needed -lprotobuf -lpthread -ldl -lcurl
+           -lgrpc_cronet -Wl,--no-as-needed -lgrpc++_reflection -lgrpc++ \
+           -Wl,--as-needed -lprotobuf -lpthread -ldl -lcurl -lasound
 endif

Seemed that linking of the alsa sound and grpc libraries were missing, but I have no idea whether this change is the correct way to fix this issues, hope to provide me the proper way,

Thanks and BRs,

"don't know how to resolve"

I have a self induced problem I'm trolling for input. I just implemented a CMake solution here, and am testing on Linux Ubuntu 16.04LTS. I'm building out of master, and using gRPC v1.16.0, goggleapis master.

I get the below. I see there was a fix for this earlier, which has no effect here.

 ../tests/integration_audio_file.sh 
++ ./run_assistant_file --input ./resources/weather_in_mountain_view.raw --output /tmp/google-assistant-sdk-audio-output.raw --credentials ./credentials.json
E1029 15:38:26.201524055   32075 resolver_registry.cc:80]    don't know how to resolve 'embeddedassistant.googleapis.com:443' or 'dns:///embeddedassistant.googleapis.com:443'
E1029 15:38:26.201583105   32075 resolver_registry.cc:80]    don't know how to resolve 'dns:///embeddedassistant.googleapis.com:443' or 'dns:///dns:///embeddedassistant.googleapis.com:443'
E1029 15:38:26.201600098   32075 channel.cc:95]              channel stack builder failed: {"created":"@1540852706.201589732","description":"resolver creation failed","file":"/home/joel/int/assistant-sdk-cpp/build/grpc-prefix/src/grpc/src/core/ext/filters/client_channel/client_channel.cc","file_line":739}
E1029 15:38:26.201648328   32075 completion_queue.cc:777]    Operation failed: tag=0x7ffd77d8f2b0, error={"created":"@1540852706.201640688","description":"lame client channel","file":"/home/joel/int/assistant-sdk-cpp/build/grpc-prefix/src/grpc/src/core/lib/surface/lame_client.cc","file_line":90}
E1029 15:38:26.201681807   32075 completion_queue.cc:777]    Operation failed: tag=0x7ffd77d8f270, error={"created":"@1540852706.201676965","description":"lame client channel","file":"/home/joel/int/assistant-sdk-cpp/build/grpc-prefix/src/grpc/src/core/lib/surface/lame_client.cc","file_line":90}
E1029 15:38:26.201747585   32075 completion_queue.cc:777]    Operation failed: tag=0x7ffd77d8f440, error={"created":"@1540852706.201741454","description":"lame client channel","file":"/home/joel/int/assistant-sdk-cpp/build/grpc-prefix/src/grpc/src/core/lib/surface/lame_client.cc","file_line":90}
assistant_sdk failed, error: lame client channel
terminate called without an active exception
++ grep 'Mountain View'

To repro

git clone -b cmake_develop https://github.com/jwinarske/assistant-sdk-cpp.git
cd assistant-sdk-cpp
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=`pwd`/dist -DCMAKE_BUILD_TYPE=Release -GNinja
autoninja

My plan was to submit this as a PR once I have it running on Mac/Windows with (PortAudio).

Thanks!

run_assistant ssl error

I ported this sample on arm linux(yocto) platform,but when I run this sample,it shows errors like below.Can someone help?Thanks a lot.

assistant_sdk robots_pem:
assistant_sdk CreateCustomChannel(embeddedassistant.googleapis.com:443, creds, arg)

E0101 00:37:40.523319895 2771 ssl_transport_security.cc:976] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
E0101 00:37:40.675959062 2771 ssl_transport_security.cc:976] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
E0101 00:37:43.238969738 2771 ssl_transport_security.cc:976] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificat

run_assistant fails to run on a different node

Hey there,

I have run_assistant working perfectly fine in a docker container on one node, say node-18.

However, when I start the same docker container on a separate node, say node-10, it gets stuck. Here are the logs:

+May I know how to fix this? I would like the run_assistant service to be deployed in Kubernetes cluster.

Kind regards,
Tamama

[root@f4987055771c assistant-sdk-cpp]# LD_LIBRARY_PATH=runtime bin/run_assistant --text_input "Hallo" --credentials_file asset/credentials.json --verbose
Using locale en-US
assistant_sdk robots_pem:
assistant_sdk CreateCustomChannel(embeddedassistant.googleapis.com:443, creds, arg)

undefined reference to `grpc_auth_json_key_destruct'

src/service_account_util.o: In function GetServiceAccountCredentialsOrNull(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda()#1}::operator()() const': service_account_util.cc:(.text+0xb0): undefined reference to grpc_auth_json_key_destruct'
src/service_account_util.o: In function GetServiceAccountCredentialsOrNull(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda()#3}::operator()() const': service_account_util.cc:(.text+0x108): undefined reference to grpc_json_destroy'
src/service_account_util.o: In function GetServiceAccountCredentialsOrNull(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)': service_account_util.cc:(.text+0x144): undefined reference to grpc_auth_json_key_create_from_string'
service_account_util.cc:(.text+0x188): undefined reference to grpc_auth_json_key_is_valid' service_account_util.cc:(.text+0x21c): undefined reference to grpc_jwt_encode_and_sign'
service_account_util.cc:(.text+0x4b0): undefined reference to grpc_json_parse_string_with_len' service_account_util.cc:(.text+0x554): undefined reference to grpc_json_get_string_property'
collect2: error: ld returned 1 exit status
Makefile:59: recipe for target 'run_assistant' failed

setting GRPC_TRACE to any value causing hanging

using " ./run_assistant_audio --verbose --credentials=credentials.json" works perfectly
after
export GRPC_TRACE=all (or anything at all) and repeating command it hangs.
I'm on a Raspberry Pi with code cloned on 7/24/18

GPR_ERROR, 'https' scheme not supported in proxy URI

Followed by the README to set up, went to the seventh part and went wrong.
$ ./run_assistant_file --input ./resources/weather_in_mountain_view.raw --output ./response.wav
--credentials ./credentials.json
E0809 10:40:17.892764613 75696 http_proxy.cc:62] 'https' scheme not supported in proxy URI

then hanged
My operating system Ubuntu16.04

compile err with assistant-sdk-cpp

While following following steps

cd ../
git clone https://github.com/googlesamples/assistant-sdk-cpp.git

cd assistant-sdk-cpp/
make run_assistant

I get Compile error

-I./grpc -I./src/ -c -o src/json_util.o src/json_util.cc
src/json_util.cc:21:36: fatal error: src/core/lib/json/json.h: No such file or directory
compilation terminated.
: recipe for target 'src/json_util.o' failed
make: *** [src/json_util.o] Error 1

Actually src/core/ does not exist

m@pe:/GVA/assistant-sdk-cpp$
m@pe:
/GVA/assistant-sdk-cpp$ pwd
/home/m/GVA/assistant-sdk-cpp
m@pe:/GVA/assistant-sdk-cpp$
m@pe:
/GVA/assistant-sdk-cpp$ ls
CONTRIBUTING.md googleapis.ar Makefile README.md src
get_credentials.sh LICENSE proto resources
m@pe:/GVA/assistant-sdk-cpp$
m@pe:
/GVA/assistant-sdk-cpp$
m@pe:/GVA/assistant-sdk-cpp$ ls src/
assistant_config.h audio_output_alsa.cc embedded_assistant.pb.o
audio_input_alsa.cc audio_output_alsa.h json_util.cc
audio_input_alsa.h audio_output_alsa.o json_util.h
audio_input_alsa.o embedded_assistant.grpc.pb.cc json_util_test.cc
audio_input_file.cc embedded_assistant.grpc.pb.h run_assistant.cc
audio_input_file.h embedded_assistant.grpc.pb.o scope_exit.h
audio_input_file.o embedded_assistant.pb.cc service_account_util.cc
audio_input.h embedded_assistant.pb.h service_account_util.h
m@pe:
/GVA/assistant-sdk-cpp$
m@pe:~/GVA/assistant-sdk-cpp$

Compiloe error&Oauth error

  1. I am getting error: ‘google::assistant::embedded::v1alpha1::DeviceConfig’ has not been declared
    using google::assistant::embedded::v1alpha1::DeviceConfig;
    removing that line works fine .
  2. While running run_assistant --audio_input ALSA_INPUT --credentials_file ./credentials.json --credentials_type USER_ACCOUNT
    getting assistant_sdk failed, error: Error occured when fetching oauth2 token.
    oauth_fetch: {"created":"@1504611219.493506582","description":"OS Error","errno":-2,"file":"src/core /lib/iomgr/resolve_address_posix.c","file_line":119,"os_error":"Name or service not known","syscall":"getaddrinfo","target_address":"www.googleapis.com"}

run_assistant with "Prive managers HK" fails to get response

Hi,

Today, I try to run_assistant with "Prive managers HK" text_input. No response gets back. May I know how to fix this?

  • The logs are attached as below.

Kind regards,
Tamama

============
text_input "Prive managers HK" -- NOT OKAY

[root@e74af657915f assistant-sdk-cpp]# LD_LIBRARY_PATH=runtime bin/run_assistant --text_input "Prive managers HK" --credentials_file asset/credentials.json --verbose
Using locale en-US
assistant_sdk robots_pem:
assistant_sdk CreateCustomChannel(embeddedassistant.googleapis.com:443, creds, arg)

assistant_sdk wrote first request: config { audio_out_config { encoding: LINEAR16 sample_rate_hertz: 16000 } dialog_state_in { language_code: "en-US" } device_config { device_id: "default" device_model_id: "default" } text_query: "Prive managers HK" }
ALSA lib confmisc.c:767:(parse_card) cannot find card '0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2501:(snd_pcm_open_noupdate) Unknown PCM default
AudioOutputALSA snd_pcm_open returned -2
assistant_sdk waiting for response ...
assistant_sdk Got a response

============
text_input "What time is it?" -- Works OKAY

[root@e74af657915f assistant-sdk-cpp]# LD_LIBRARY_PATH=runtime bin/run_assistant --text_input "What time is it" --credentials_file asset/credentials.json --verbose
Using locale en-US
assistant_sdk robots_pem:
assistant_sdk CreateCustomChannel(embeddedassistant.googleapis.com:443, creds, arg)

assistant_sdk wrote first request: config { audio_out_config { encoding: LINEAR16 sample_rate_hertz: 16000 } dialog_state_in { language_code: "en-US" } device_config { device_id: "default" device_model_id: "default" } text_query: "What time is it" }
ALSA lib confmisc.c:767:(parse_card) cannot find card '0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2501:(snd_pcm_open_noupdate) Unknown PCM default
AudioOutputALSA snd_pcm_open returned -2
assistant_sdk waiting for response ...
assistant_sdk Got a response
assistant_sdk response:
The time is 10:39 AM.

Unable to build googleapis in the latest version

"make LANGUAGE=cpp" for googleapis is giving the following error:

google/ads/googleads/v0/common/custom_parameter.proto:31:8: Option "php_metadata_namespace" unknown.
Makefile:46: recipe for target 'google/ads/googleads/v0/common/custom_parameter.pb.cc' failed
make: *** [google/ads/googleads/v0/common/custom_parameter.pb.cc] Error 1

Any leads on resolving this?

stuck on main function

I port this sdk on Android arm platform,
but when I try to run run_assistant but it stop at
This function
std::shared_ptr<ClientReaderWriter<ConverseRequest, ConverseResponse>>
stream(std::move(assistant->Converse(&context)));

Is there any direction for me to debug ?

Compile error

make run_assistant
g++ -std=c++11 -I/usr/local/include -pthread -I/home/pi/googleapis/gens -I../grpc  -c -o run_assistant.o run_assistant.cc
run_assistant.cc:54:46: error: ‘google::assistant::embedded::v1alpha1::DeviceConfig’ has not been declared
 using google::assistant::embedded::v1alpha1::DeviceConfig;
                                              ^
<builtin>: recipe for target 'run_assistant.o' failed
make: *** [run_assistant.o] Error 1

If I remove that using it compiles and works fine

Support for NEWS and MUSIC Playback management

Hi There,

I would like to confirm whether this reference implementation would be able to support NEWS and MUSIC Playback.
I'm listening the voice response for NEWS and MUSIC Playback are found not to be supported at this point of time.

Based on the SDK Documentation,
I read that, Timers and Alarms are not supported.

Please clarify me regarding this.

Thanks,

Questions about --api_endpoint option settings

In Google's tutorial, the Goole Assistant Library can set traits via action.json. Can it be set in the C++ version of the Google Assistant Service?

According to the tutorial you provided, if you want to test the custom Assistant gRPC API endpoint, you can set it with the --api_endpoint option. I probably browsed the code, --api_endpoint followed by a URL: port, so how should this custom parameter be obtained?

Is it possible to reuse grpc::Channel and EmbeddedAssistant::Stub?

In run_assistant_text demo code a new EmbeddedAssistant::Stub and grpc::Channel will be created when a new text input is coming.
Is is possible to create grpc::Channel and EmbeddedAssistant::Stub only once instead of creating a new one every time?
When I trying to do this, demo will block at second time stream->Read(&response)

    // Begin a stream.
    auto channel = CreateChannel(api_endpoint);
    std::unique_ptr<EmbeddedAssistant::Stub> assistant(
        EmbeddedAssistant::NewStub(channel));

    grpc::ClientContext context;
    context.set_fail_fast(false);
    context.set_credentials(call_credentials);

    std::shared_ptr<ClientReaderWriter<AssistRequest, AssistResponse>> stream(
        std::move(assistant->Assist(&context)));

    AssistResponse response;
    //Write request and read response
    stream->Write(request);
    while(stream->Read(&response)){// this read will success
     ...
   }
  //Do it again
    stream->Write(request);
    while(stream->Read(&response)){// get blocked at here forever
     ...
   }

Thanks in advance.

make run_assistant fails on Ubuntu 18.04 with undefined reference to symbol '_ZN4grpc6g_glipE'

All the instructions were fine (download, make, install etc) until the one at the end failing to make run_assistant with: embedded_assistant.grpc.pb.o: undefined reference to symbol '_ZN4grpc6g_glipE'

g++ /Downloads/assistant-sdk-cpp/googleapis/gens/google/assistant/embedded/v1alpha2/embedded_assistant.pb.o /Downloads/assistant-sdk-cpp/googleapis/gens/google/assistant/embedded/v1alpha2/embedded_assistant.grpc.pb.o googleapis.ar src/base64_encode.o src/json_util.o src/audio_input_alsa.o src/audio_output_alsa.o src/audio_input_file.o src/run_assistant_audio.o -lm -lgrpc_cronet -Wl,--no-as-needed -lgrpc++_reflection -Wl,--as-needed -lprotobuf -lpthread -ldl -o run_assistant_audio
/usr/bin/ld: /Downloads/assistant-sdk-cpp/googleapis/gens/google/assistant/embedded/v1alpha2/embedded_assistant.grpc.pb.o: undefined reference to symbol '_ZN4grpc6g_glipE'

Any idea what might have gone wrong here?

action question?

Why have we succeeded in validting action ,and to test our hardware; looser and easier to develop the Google cloud

get_credentials.sh is outdated

Hi,

It seems that get_credentials.sh is outdated with the newer version of client key-json without client_secret field.

May I have an easy guide of how to generate the credentials.json needed for run_assistant?

Kind regards,
Tamama

opening microphine automatically for multi turn situation.

hello,

what I want to do is that open the microphone resouce when the response from google assistant is re-asking, for example,
Me: add my schedule
G : what’s the title of schedule?
Me : lunch with James. <— mic should re-open here.
...

I’ve tried several ways, but there’s insufficient of usable information to do that.

could you guide me how to implement multi-turn case?

thank you in advance.

assistant_sdk failed, error: lame client channel

hi, I get a error when Start run_assistant:
wxd@suntec:~/assistant-sdk-cpp$ ./run_assistant --audio_input ./resources/weather_in_mountain_view.raw --credentials_file ./credentials.json --credentials_type USER_ACCOUNT
assistant_sdk robots_pem:
assistant_sdk CreateCustomChannel(embeddedassistant.googleapis.com:443, creds, arg)

E0223 16:30:13.280134655 398 resolver_registry.cc:111] don't know how to resolve 'embeddedassistant.googleapis.com:443' or 'dns:///embeddedassistant.googleapis.com:443'
E0223 16:30:13.280234059 398 resolver_registry.cc:111] don't know how to resolve 'embeddedassistant.googleapis.com:443' or 'dns:///embeddedassistant.googleapis.com:443'
E0223 16:30:13.280285295 398 resolver_registry.cc:111] don't know how to resolve 'dns:///embeddedassistant.googleapis.com:443' or 'dns:///dns:///embeddedassistant.googleapis.com:443'
E0223 16:30:13.280332102 398 channel.cc:90] channel stack builder failed: {"created":"@1519374613.280306935","description":"resolver creation failed","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":769}
assistant_sdk wrote first request: config { audio_in_config { encoding: LINEAR16 sample_rate_hertz: 16000 } audio_out_config { encoding: LINEAR16 sample_rate_hertz: 16000 } converse_state { context { third_party_context: "{'current_channel': 'News'}" } } }
AudioOutputALSA Start() succeeded
assistant_sdk waiting for response ...
AudioOutputALSA Stop() succeeded
assistant_sdk failed, error: lame client channel
terminate called without an active exception

Could you make repositories grpc and googleapis submodule of assistant-skd-cpp

I am working on MIPS cross-compiling for google assistant here.
But no luck finding a stable build that MIPS support yet.

What happened here is, both GRPC and googleapis are pretty active and people enjoying to use latest standard APIs. But MIPS is out of date (the one I am using is year 2016) and not supporting them.

Hence, I need to find a older version of GRPC and googleapis that Okay with google assistant while MIPS support it.

This will be much easier if the GRPC and googleapis is submodule of this project. That it will use a stable folk (image copy) of the original project back to the date of release time of google assistant. And extra new features won't be included.
For example: the initial commit will associate the repositories of GRPC and googleapis back in Aug 23, 2017. With submodule set up, we can re-build the original ones now exactly same as 1 years before, and it works.

Please consider the feature improvement. Thank you.

If submodule is not a option, please document the commit # of GRPC and googleapis you use when you conduct release? It really helps my case.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.