Code Monkey home page Code Monkey logo

nodejs-automl's Introduction

Google Cloud Platform logo

THIS REPOSITORY IS DEPRECATED. ALL OF ITS CONTENT AND HISTORY HAS BEEN MOVED TO GOOGLE-CLOUD-NODE

release level npm version

πŸ”” AutoML API NodeJS Client is now available in Vertex AI. Please visit node-js-aiplatform for the new NodeJS Vertex AI client. Vertex AI is our next generation AI Platform, with many new features that are unavailable in the current platform. Migrate your resources to Vertex AI to get the latest machine learning features, simplify end-to-end journeys, and productionize models with MLOps.

Cloud AutoML API client for Node.js

A comprehensive list of changes in each version may be found in the CHANGELOG.

Read more about the client libraries for Cloud APIs, including the older Google APIs Client Libraries, in Client Libraries Explained.

Table of contents:

Quickstart

Before you begin

  1. Select or create a Cloud Platform project.
  2. Enable billing for your project.
  3. Enable the Cloud AutoML API.
  4. Set up authentication with a service account so you can access the API from your local workstation.

Installing the client library

npm install @google-cloud/automl

Using the client library

const automl = require('@google-cloud/automl');
const fs = require('fs');

// Create client for prediction service.
const client = new automl.PredictionServiceClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const projectId = `The GCLOUD_PROJECT string, e.g. "my-gcloud-project"`;
// const computeRegion = `region-name, e.g. "us-central1"`;
// const modelId = `id of the model, e.g. β€œICN723541179344731436”`;
// const filePath = `local text file path of content to be classified, e.g. "./resources/flower.png"`;
// const scoreThreshold = `value between 0.0 and 1.0, e.g. "0.5"`;

// Get the full path of the model.
const modelFullId = client.modelPath(projectId, computeRegion, modelId);

// Read the file content for prediction.
const content = fs.readFileSync(filePath, 'base64');

const params = {};

if (scoreThreshold) {
  params.score_threshold = scoreThreshold;
}

// Set the payload by giving the content and type of the file.
const payload = {};
payload.image = {imageBytes: content};

// params is additional domain-specific parameters.
// currently there is no additional parameters supported.
const [response] = await client.predict({
  name: modelFullId,
  payload: payload,
  params: params,
});
console.log('Prediction results:');
response.payload.forEach(result => {
  console.log(`Predicted class name: ${result.displayName}`);
  console.log(`Predicted class score: ${result.classification.score}`);
});

Samples

Samples are in the samples/ directory. Each sample's README.md has instructions for running its sample.

Sample Source Code Try it
Quickstart source code Open in Cloud Shell

The Cloud AutoML Node.js Client API Reference documentation also contains samples.

Supported Node.js Versions

Our client libraries follow the Node.js release schedule. Libraries are compatible with all current active and maintenance versions of Node.js. If you are using an end-of-life version of Node.js, we recommend that you update as soon as possible to an actively supported LTS version.

Google's client libraries support legacy versions of Node.js runtimes on a best-efforts basis with the following warnings:

  • Legacy versions are not tested in continuous integration.
  • Some security patches and features cannot be backported.
  • Dependencies cannot be kept up-to-date.

Client libraries targeting some end-of-life versions of Node.js are available, and can be installed through npm dist-tags. The dist-tags follow the naming convention legacy-(version). For example, npm install @google-cloud/automl@legacy-8 installs client libraries for versions compatible with Node.js 8.

Versioning

This library follows Semantic Versioning.

This library is considered to be stable. The code surface will not change in backwards-incompatible ways unless absolutely necessary (e.g. because of critical security issues) or with an extensive deprecation period. Issues and requests against stable libraries are addressed with the highest priority.

More Information: Google Cloud Platform Launch Stages

Contributing

Contributions welcome! See the Contributing Guide.

Please note that this README.md, the samples/README.md, and a variety of configuration files in this repository (including .nycrc and tsconfig.json) are generated from a central template. To edit one of these files, make an edit to its templates in directory.

License

Apache Version 2.0

See LICENSE

nodejs-automl's People

Contributors

alexander-fenster avatar anguillanneuf avatar bcoe avatar bradmiro avatar callmehiphop avatar chiefkarlin avatar danieljbruce avatar dpebot avatar fhinkel avatar gcf-owl-bot[bot] avatar google-cloud-policy-bot[bot] avatar jkwlui avatar justinbeckwith avatar kweinmeister avatar leahecole avatar martinvarelaj avatar meredithslota avatar munkhuushmgl avatar nayaknishant avatar nirupa-kumar avatar nnegrey avatar release-please[bot] avatar renovate-bot avatar renovate[bot] avatar sirtorry avatar sofisl avatar strykrol avatar summer-ji-eng avatar xiaozhenliu-gg5 avatar yoshi-automation avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nodejs-automl's Issues

NOT_FOUND: Invalid resource ID when calling Custom Model From Firebase Function

I'm calling a GCP Custom ML Model from a node.js 8 firebase function. But when I call the predict function, I get an Error 5: NOT_FOUND: Invalid resource ID I've turned on the ML API and I've checked the format of the model name, it looks to be correct according to the docs: projects//locations/us-central1/models/

My code looks like:

// Create client for prediction service.
const client = new automl.PredictionServiceClient();

const projectId = 'project-id';
const computeRegion = 'us-central1';
const modelId = 'model-name';

// Get the full path of the model.
const modelFullId = client.modelPath(projectId, computeRegion, modelId);

const params = {};
const payload = {};
payload.instances = {"data1": 38.0,
"data2": -2.7,
"data3": -0.4,
"data4": 2.1,
"data5": -2.7,
"data6": -1.7,
"data7": 0.4,
"data8": 8.5,
"data9": -1.1,
"data10": 0.2
};

const [response] = await client.predict({
name: modelFullId,
payload: payload,
params: params,
});
console.log(Prediction results:);
response.payload.forEach(result => {
console.log(Predicted class name: ${result.displayName});
console.log(Predicted class score: ${result.classification.score});
});

Thanks for your help.

Can i export edge model feature with this library?

To the best of my knowledge, we have to used cmd line, in order to export trained model to an edge model. Is it will be good if this library can export an edge model? So it will be completely automate development cycle and allow us to take benefit of rich daat from edge device.

Best regards.

IAM get permission error to input bucket even when input bucket is public

Summary

I'm making a batchPrediction with the input file's bucket being public, but I'm getting this error:

Error: 9 FAILED_PRECONDITION: Service account account [xxxx]@gcp-sa-automl.iam.gserviceaccount.com does not have [storage.buckets.get] IAM permission to the input bucket. Please either copy the files to the Google Cloud Storage bucket owned by the model owner project

sample code

const automl = require('@google-cloud/automl');
const client = new automl.PredictionServiceClient();
const automlModel= client.modelPath('project_id','zone','model_id');
const input="gs://[bucketname]/images/info.csv"
const output="gs://[bucketname]/images/output"
try{
          responses = await client.batchPredict({
              name: automlModel,
              inputConfig: {gcsSource:{inputUris:[input]}},
              outputConfig: {gcsDestination:{outputUriPrefix:output}},
          }).then(resp=>{
              const [operation, initialApiResponse] = resp;
              console.log("initialApiResponse: ",initialApiResponse);
              return operation.promise();
          });
}catch(err){
          console.log(err);
 }

Environment details

  • OS: Container-Optimized OS <- I'm using a docker image
  • Node.js version: v12.11.0
  • npm version: 6.11.3
  • @google-cloud/automl version: 1.7.2

Steps to reproduce

  1. create a public bucket, save images to make a prediction and save the csv file for batch predictions in it
  2. make a batchPrediction with the input file being the path of the csv file in step 1

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

b'5 04:21:12,780 synthtool [DEBUG] > Cloning googleapis.\nDEBUG:synthtool:Cloning googleapis.\n2020-06-05 04:21:12,781 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\nDEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\n2020-06-05 04:21:12,785 synthtool [DEBUG] > Pulling Docker image: gapic-generator-typescript:latest\nDEBUG:synthtool:Pulling Docker image: gapic-generator-typescript:latest\nlatest: Pulling from gapic-images/gapic-generator-typescript\nDigest: sha256:c9bc12024eddcfb94501627ff5b3ea302370995e9a2c9cde6b3317375d7e7b66\nStatus: Image is up to date for gcr.io/gapic-images/gapic-generator-typescript:latest\n2020-06-05 04:21:13,661 synthtool [DEBUG] > Generating code for: google/cloud/automl/v1.\nDEBUG:synthtool:Generating code for: google/cloud/automl/v1.\n2020-06-05 04:21:14,551 synthtool [DEBUG] > Wrote metadata to synth.metadata.\nDEBUG:synthtool:Wrote metadata to synth.metadata.\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__\n    return self.main(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main\n    rv = self.invoke(ctx)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke\n    return callback(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main\n    spec.loader.exec_module(synth_module)  # type: ignore\n  File "<frozen importlib._bootstrap_external>", line 678, in exec_module\n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n  File "/home/kbuilder/.cache/synthtool/nodejs-automl/synth.py", line 36, in <module>\n    extra_proto_files=[\'google/cloud/common_resources.proto\'],\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library\n    return self._generate_code(service, version, "typescript", **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code\n    f"Code generation seemed to succeed, but {output_dir} is empty."\nRuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmpqhae4q1t is empty.\n2020-06-05 04:21:14,589 autosynth [ERROR] > Synthesis failed\n2020-06-05 04:21:14,589 autosynth [DEBUG] > Running: git reset --hard HEAD\nHEAD is now at 632d7e3 chore: release 2.1.0 (#398)\n2020-06-05 04:21:14,595 autosynth [DEBUG] > Running: git checkout autosynth-self\nSwitched to branch \'autosynth-self\'\n2020-06-05 04:21:14,600 autosynth [ERROR] > Command \'[\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\' returned non-zero exit status 1.\n2020-06-05 04:21:14,742 autosynth [DEBUG] > Running: git checkout 632d7e311052e9086b4152bf6df01f20eea68993\nNote: checking out \'632d7e311052e9086b4152bf6df01f20eea68993\'.\n\nYou are in \'detached HEAD\' state. You can look around, make experimental\nchanges and commit them, and you can discard any commits you make in this\nstate without impacting any branches by performing another checkout.\n\nIf you want to create a new branch to retain commits you create, you may\ndo so (now or later) by using -b with the checkout command again. Example:\n\n  git checkout -b <new-branch-name>\n\nHEAD is now at 632d7e3 chore: release 2.1.0 (#398)\n2020-06-05 04:21:14,748 autosynth [DEBUG] > Running: git checkout d53a5b45c46920932dbe7d0a95e10d8b58933dae\nPrevious HEAD position was be74d3e build: do not fail builds on codecov errors (#528)\nHEAD is now at d53a5b4 docs: improve README (#600)\n2020-06-05 04:21:14,763 autosynth [DEBUG] > Running: git checkout cd804bab06e46dd1a4f16c32155fd3cddb931b52\nPrevious HEAD position was 2fc2caaa chore: enable gapic v2 and proto annotation for bigquery/storage/v1beta2 API.\nHEAD is now at cd804bab docs: cleaned docs for the Agents service and resource.\n2020-06-05 04:21:14,833 autosynth [DEBUG] > Running: git branch -f autosynth-123\n2020-06-05 04:21:14,837 autosynth [DEBUG] > Running: git checkout autosynth-123\nSwitched to branch \'autosynth-123\'\n2020-06-05 04:21:14,842 autosynth [INFO] > Running synthtool\n2020-06-05 04:21:14,843 autosynth [INFO] > [\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\n2020-06-05 04:21:14,844 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --\n2020-06-05 04:21:15,051 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/nodejs-automl/synth.py.\nOn branch autosynth-123\nnothing to commit, working tree clean\n2020-06-05 04:21:15,183 synthtool [DEBUG] > Ensuring dependencies.\nDEBUG:synthtool:Ensuring dependencies.\n2020-06-05 04:21:15,188 synthtool [DEBUG] > Cloning googleapis.\nDEBUG:synthtool:Cloning googleapis.\n2020-06-05 04:21:15,189 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\nDEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\n2020-06-05 04:21:15,192 synthtool [DEBUG] > Pulling Docker image: gapic-generator-typescript:latest\nDEBUG:synthtool:Pulling Docker image: gapic-generator-typescript:latest\nlatest: Pulling from gapic-images/gapic-generator-typescript\nDigest: sha256:c9bc12024eddcfb94501627ff5b3ea302370995e9a2c9cde6b3317375d7e7b66\nStatus: Image is up to date for gcr.io/gapic-images/gapic-generator-typescript:latest\n2020-06-05 04:21:16,079 synthtool [DEBUG] > Generating code for: google/cloud/automl/v1.\nDEBUG:synthtool:Generating code for: google/cloud/automl/v1.\n2020-06-05 04:21:16,930 synthtool [DEBUG] > Wrote metadata to synth.metadata.\nDEBUG:synthtool:Wrote metadata to synth.metadata.\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__\n    return self.main(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main\n    rv = self.invoke(ctx)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke\n    return callback(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main\n    spec.loader.exec_module(synth_module)  # type: ignore\n  File "<frozen importlib._bootstrap_external>", line 678, in exec_module\n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n  File "/home/kbuilder/.cache/synthtool/nodejs-automl/synth.py", line 36, in <module>\n    extra_proto_files=[\'google/cloud/common_resources.proto\'],\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library\n    return self._generate_code(service, version, "typescript", **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code\n    f"Code generation seemed to succeed, but {output_dir} is empty."\nRuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmpyy0dzy5y is empty.\n2020-06-05 04:21:16,972 autosynth [ERROR] > Synthesis failed\n2020-06-05 04:21:16,972 autosynth [DEBUG] > Running: git reset --hard HEAD\nHEAD is now at 632d7e3 chore: release 2.1.0 (#398)\n2020-06-05 04:21:16,978 autosynth [DEBUG] > Running: git checkout autosynth\nSwitched to branch \'autosynth\'\n2020-06-05 04:21:16,984 autosynth [DEBUG] > Running: git clean -fdx\nRemoving __pycache__/\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 615, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 476, in main\n    return _inner_main(temp_dir)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 595, in _inner_main\n    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 371, in synthesize_loop\n    synthesize_inner_loop(toolbox, synthesizer)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 381, in synthesize_inner_loop\n    synthesizer, len(toolbox.versions) - 1\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 266, in synthesize_version_in_new_branch\n    synthesizer.synthesize(synth_log_path, self.environ)\n  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 119, in synthesize\n    synth_proc.check_returncode()  # Raise an exception.\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode\n    self.stderr)\nsubprocess.CalledProcessError: Command \'[\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\' returned non-zero exit status 1.\n'

Google internal developers can see the full log here.

Docs for the `name` param in importData are confusing

I'm looking at the docs for importData here (generated here).

It says it should be a string with the dataset name, but it actually needs to include the project name and region. I figured this out by looking at the code snippet below:

var formattedName = client.datasetPath('[PROJECT]', '[LOCATION]', '[DATASET]');

So it should be: projects/$PROJECT/locations/$LOCATION/datasets/$DATASET. Can we add this to the docs?

[AutoML Tables] payload.payload; Message: Required field not set(js)

I get this error when I query my AutoML Tables model when I deploy a Firebase cloud function.

payload.payload; Message: Required field not set.

additional stacktrace:

    at Object.exports.createStatusError (/srv/node_modules/grpc/src/common.js:91:15)
    at Object.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:1209:28)
    at InterceptingListener._callNext (/srv/node_modules/grpc/src/client_interceptors.js:568:42)
    at InterceptingListener.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:618:8)
    at callback (/srv/node_modules/grpc/src/client_interceptors.js:847:24)
  code: 3,
  metadata: 
   Metadata {
     _internal_repr: 
      { 'google.rpc.badrequest-bin': [Array],
        'grpc-status-details-bin': [Array],
        'grpc-server-stats-bin': [Array] },
     flags: 0 },
  details: 'List of found errors:\t1.Field: name; Message: The provided location ID doesn\'t match the endpoint. For automl.googleapis.com, the valid location ID is `us-central1`. For eu-automl.googleapis.com, the valid location ID is `eu`.\t',
  note: 'Exception occurred in retry method that was not classified as transient' } 

The payload should have been set as demo showing.

And I believe I have set the location in eu correctly.
My model was deployed in eu, so I have to use eu-automl.googleapis.com.

However, I believe I have coded correctly:

const project = 'some model project';
const region = 'eu';
const model = 'TBL82342424234';

const automl = require('@google-cloud/automl').v1beta1;
const clientOptions = { apiEndpoint: 'eu-automl.googleapis.com' };
const predictionClient = new automl.PredictionServiceClient(clientOptions);


const projectLocation = modelClient.locationPath(project, region);

const payload = {
        row: {
            values: [{ stringValue: "hello, world" }],
        },
 };
const requestPrediction = {
        name: predictionClient.modelPath(project, region, model),
        payload: payload,
        params: { feature_importance: true },
 };
predictionClient.predict(requestPrediction) .......

model-id is not returned for finished operation in the response of the getOperation.

I am trying to get the model-id of a completed operation using the operationId. the snippet of me call:

    const request = {
    name: `projects/${projectId}/locations/${location}/operations/${operationId}`,
    };
  
    const [response] = await client.operationsClient.getOperation(request); 
    console.log(`Name: ${response.name}`);
    console.log(`Operation details:`);
    console.log(`${JSON.stringify(response, null, 2)}`);

my response:

{
  "name": "projects/projectId/locations/us-central1/operations/operationId”,
  "metadata": {
    "type_url": "type.googleapis.com/google.cloud.automl.v1.OperationMetadata",
    "value": {
      "type": "Buffer",
      "data": […]
    }
  },
  "done": true,
  "response": {
    "type_url": "type.googleapis.com/google.cloud.automl.v1.Model",
    "value": {
      "type": "Buffer",
      "data": […]
    }
  },
  "result": "response"
}

If done is true the response.response should contain name which has the model-id in it. My operation is done but no model-id! the model has finished training and is in my dashboard correctly. I need to get the modelId programatically though!

Thank you for your help.

The provided location ID is not valid

I've just created a new natural language model which is in the eu region. When I try to use the client I get the following Error:

3 INVALID_ARGUMENT: List of found errors: 1.Field: name; Message: The provided location ID is not valid.
Error: 3 INVALID_ARGUMENT: List of found errors:	1.Field: name; Message: The provided location ID is not valid.	
    at Object.callErrorFromStatus (/Users/sugendran/code/getmployd-ml/bulk-classifier/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Http2CallStream.<anonymous> (/Users/sugendran/code/getmployd-ml/bulk-classifier/node_modules/@grpc/grpc-js/build/src/client.js:96:33)
    at Http2CallStream.emit (events.js:215:7)
    at /Users/sugendran/code/getmployd-ml/bulk-classifier/node_modules/@grpc/grpc-js/build/src/call-stream.js:75:22
    at processTicksAndRejections (internal/process/task_queues.js:75:11)

The model path is projects/305312444325/locations/eu/models/TCN8963992845815906304

and I'm using eu as the location when calling .modelPath

Environment details

  • OS: OSX
  • Node.js version: v12.12.0
  • npm version: 6.11.3
  • @google-cloud/automl version: 1.7.3

Steps to reproduce

  • Create a new natural language model in the EU region and try to use it.

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

Cloning into 'working_repo'...
Switched to branch 'autosynth'
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/git/autosynth/autosynth/synth.py", line 256, in <module>
    main()
  File "/tmpfs/src/git/autosynth/autosynth/synth.py", line 196, in main
    last_synth_commit_hash = get_last_metadata_commit(args.metadata_path)
  File "/tmpfs/src/git/autosynth/autosynth/synth.py", line 149, in get_last_metadata_commit
    text=True,
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'text'

Google internal developers can see the full log here.

TypeError: Cannot read property 'charCodeAt' of undefined

The most basic usage of this library appears to be broken. See the system test over here:
https://github.com/googleapis/nodejs-automl/pull/106/files

Running this gives the error:

 1) automl system tests
       should list all datasets:
     TypeError: Cannot read property 'charCodeAt' of undefined
      at peg$parsetemplate (node_modules/google-gax/build/src/path_template_parser.js:294:17)
      at Object.peg$parse [as parse] (node_modules/google-gax/build/src/path_template_parser.js:623:18)
      at new PathTemplate (node_modules/google-gax/src/path_template.ts:74:50)
      at segments.forEach.segment (node_modules/google-gax/src/path_template.ts:142:21)
      at Array.forEach (<anonymous>)
      at PathTemplate.render (node_modules/google-gax/src/path_template.ts:134:19)
      at AutoMlClient.locationPath (src/v1beta1/auto_ml_client.js:1634:53)
      at Context.it (system-test/automl.js:25:27)

The relevant code snippet is:

const automl = require('@google-cloud/automl');
const client = new automl.AutoMlClient();
const location = 'us-central1';
const projectId = process.env.projectId;
const parent = client.locationPath(projectId, location);
client.listDatasets({parent}).then(console.log).catch(console.error);

Google AutoML Error - Error: 4 DEADLINE_EXCEEDED: Deadline exceeded

Environment details

  • OS: Ubuntu 18.04
  • Node.js version: v11.15.0
  • npm version: 6.13.2
  • @google-cloud/automl version: 1.10.0

Steps to reproduce

I am using the nodejs library to import data to the dataset using the following code.

require("dotenv").config();
const automl = require("@google-cloud/automl");
const client = new automl.v1beta1.AutoMlClient();

(async () => {
  const projectId = `xxx`;
  const computeRegion = `us-central1`;
  const datasetId = `xxx`;
  const path = "gs://example_bucket_ocr_idr2/happiness.csv";

  // Get the full path of the dataset.
  const datasetFullId = client.datasetPath(projectId, computeRegion, datasetId);

  let inputConfig = {};
  if (path.startsWith(`bq`)) {
    // Get Bigquery URI.
    inputConfig = {
      bigquerySource: {
        inputUri: path
      }
    };
  } else {
    // Get the multiple Google Cloud Storage URIs.
    const inputUris = path.split(`,`);
    inputConfig = {
      gcsSource: {
        inputUris: inputUris
      }
    };
  }

  // Import the dataset from the input URI.
  client
    .importData({ name: datasetFullId, inputConfig: inputConfig })
    .then(responses => {
      const operation = responses[0];
      console.log(`Processing import...`);
      return operation.promise();
    })
    .then(responses => {
      // The final result of the operation.
      const operationDetails = responses[2];

      // Get the data import details.
      console.log("Data import details:");
      console.log(`\tOperation details:`);
      console.log(`\t\tName: ${operationDetails.name}`);
      console.log(`\t\tDone: ${operationDetails.done}`);
    })
    .catch(err => {
      console.error(err);
    });
})();

But every time I run it, I get DEADLINE_EXCEEDED error. I have searched for a solution but didnt find any till now.

{ Error: 4 DEADLINE_EXCEEDED: Deadline exceeded
    at Object.callErrorFromStatus (/var/www/html/test/google-ml/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Http2CallStream.call.on (/var/www/html/test/google-ml/node_modules/@grpc/grpc-js/build/src/client.js:96:33)
    at Http2CallStream.emit (events.js:198:15)
    at process.nextTick (/var/www/html/test/google-ml/node_modules/@grpc/grpc-js/build/src/call-stream.js:79:22)
    at processTicksAndRejections (internal/process/task_queues.js:79:9)
  code: 4,
  details: 'Deadline exceeded',
  metadata: Metadata { internalRepr: Map {}, options: {} } }

The happiness.csv file has the following contents:

gs://example_bucket_ocr_idr2/certs/01.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/02.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/03.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/04.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/05.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/06.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/07.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/08.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/09.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/10.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/11.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/12.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/13.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/14.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/15.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/16.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/17.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/18.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/19.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/20.pdf, certificates, post_graduation
gs://example_bucket_ocr_idr2/certs/21.pdf, certificates, post_graduation

Any help will be appreciated!

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

d14318e64429c3b94f6443ae83daf9
2020-09-10 03:17:14,326 autosynth [DEBUG] > Running: git log -1 --pretty=%at 019c7168faa0e56619f792693a8acdb30d6de19b
2020-09-10 03:17:14,329 autosynth [DEBUG] > Running: git log -1 --pretty=%at 5d916ec54cadd4674e80e6555d0c6a78849ef4a7
2020-09-10 03:17:14,332 autosynth [DEBUG] > Running: git log -1 --pretty=%at cbcd64279572769b4d350bf8078bcd1f151c9684
2020-09-10 03:17:14,335 autosynth [DEBUG] > Running: git log -1 --pretty=%at 80f46100c047bc47efe0025ee537dc8ee413ad04
2020-09-10 03:17:14,338 autosynth [DEBUG] > Running: git log -1 --pretty=%at d91dd8aac77f7a9c5506c238038a26fa4f9e361e
2020-09-10 03:17:14,341 autosynth [DEBUG] > Running: git log -1 --pretty=%at 7b3766035373f5da99abf9413f1f7c29f108a2fd
2020-09-10 03:17:14,344 autosynth [DEBUG] > Running: git log -1 --pretty=%at 32c758f11b8c578f515a746c9d263b82a615a77c
2020-09-10 03:17:14,347 autosynth [DEBUG] > Running: git log -1 --pretty=%at b2c32f1c8a4094f0f47fcf5d10f0b6f2bfb3387d
2020-09-10 03:17:14,351 autosynth [DEBUG] > Running: git log -1 --pretty=%at 470e98896484cdce77e073e83a2c4f9157770f8c
2020-09-10 03:17:14,354 autosynth [DEBUG] > Running: git log -1 --pretty=%at d302f93d7f47e2852e585ac35ab2d15585717ec0
2020-09-10 03:17:14,357 autosynth [DEBUG] > Running: git log -1 --pretty=%at ffcee7952b74f647cbb3ef021d95422f10816fca
2020-09-10 03:17:14,360 autosynth [DEBUG] > Running: git checkout 713d00568534884bc8cce9d8ca1bbf1c5b1e0bf5
Note: checking out '713d00568534884bc8cce9d8ca1bbf1c5b1e0bf5'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 713d005 fix(deps): update dependency yargs to v16 (#438)
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-09-10 03:17:14,433 autosynth [DEBUG] > Running: git checkout ffcee7952b74f647cbb3ef021d95422f10816fca
Note: checking out 'ffcee7952b74f647cbb3ef021d95422f10816fca'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at ffcee79 chore(python-library): use sphinx 1.5.5 for the docfx job (#753)
2020-09-10 03:17:14,445 autosynth [DEBUG] > Running: git branch -f autosynth-75
2020-09-10 03:17:14,449 autosynth [DEBUG] > Running: git checkout autosynth-75
Switched to branch 'autosynth-75'
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-09-10 03:17:14,454 autosynth [INFO] > Running synthtool
2020-09-10 03:17:14,454 autosynth [INFO] > ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
2020-09-10 03:17:14,454 autosynth [DEBUG] > log_file_path: /tmpfs/src/logs/nodejs-automl/75/sponge_log.log
2020-09-10 03:17:14,456 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --
2020-09-10 03:17:14,682 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/nodejs-automl/synth.py.
On branch autosynth-75
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   samples/vision/automlVisionDataset.js
	modified:   samples/vision/automlVisionModel.js
	modified:   samples/vision/automlVisionPredict.js

no changes added to commit (use "git add" and/or "git commit -a")
2020-09-10 03:17:14,830 synthtool [DEBUG] > Ensuring dependencies.
DEBUG:synthtool:Ensuring dependencies.
2020-09-10 03:17:14,839 synthtool [DEBUG] > Cloning googleapis.
DEBUG:synthtool:Cloning googleapis.
2020-09-10 03:17:16,753 synthtool [DEBUG] > Generating code for: //google/cloud/automl/v1:automl-v1-nodejs.
DEBUG:synthtool:Generating code for: //google/cloud/automl/v1:automl-v1-nodejs.
2020-09-10 03:20:17,483 synthtool [SUCCESS] > Generated code into /tmpfs/tmp/tmpfclk_zy5.
SUCCESS:synthtool:Generated code into /tmpfs/tmp/tmpfclk_zy5.
2020-09-10 03:20:17,550 synthtool [DEBUG] > Generating code for: //google/cloud/automl/v1beta1:automl-v1beta1-nodejs.
DEBUG:synthtool:Generating code for: //google/cloud/automl/v1beta1:automl-v1beta1-nodejs.
2020-09-10 03:20:23,903 synthtool [SUCCESS] > Generated code into /tmpfs/tmp/tmpo6w6e7n6.
SUCCESS:synthtool:Generated code into /tmpfs/tmp/tmpo6w6e7n6.
2020-09-10 03:20:23,973 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
DEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
2020-09-10 03:20:23,993 synthtool [INFO] > successfully generate `src/index.ts`
INFO:synthtool:successfully generate `src/index.ts`
.eslintignore
.eslintrc.json
.gitattributes
.github/ISSUE_TEMPLATE/bug_report.md
.github/ISSUE_TEMPLATE/feature_request.md
.github/ISSUE_TEMPLATE/support_request.md
.github/PULL_REQUEST_TEMPLATE.md
.github/release-please.yml
.github/workflows/ci.yaml
.kokoro/.gitattributes
.kokoro/common.cfg
.kokoro/continuous/node10/common.cfg
.kokoro/continuous/node10/docs.cfg
.kokoro/continuous/node10/test.cfg
.kokoro/continuous/node12/common.cfg
.kokoro/continuous/node12/lint.cfg
.kokoro/continuous/node12/samples-test.cfg
.kokoro/continuous/node12/system-test.cfg
.kokoro/continuous/node12/test.cfg
.kokoro/docs.sh
.kokoro/lint.sh
.kokoro/populate-secrets.sh
.kokoro/presubmit/node10/common.cfg
.kokoro/presubmit/node12/common.cfg
.kokoro/presubmit/node12/samples-test.cfg
.kokoro/presubmit/node12/system-test.cfg
.kokoro/presubmit/node12/test.cfg
.kokoro/publish.sh
.kokoro/release/docs-devsite.cfg
.kokoro/release/docs-devsite.sh
.kokoro/release/docs.cfg
.kokoro/release/docs.sh
.kokoro/release/publish.cfg
.kokoro/samples-test.sh
.kokoro/system-test.sh
.kokoro/test.bat
.kokoro/test.sh
.kokoro/trampoline.sh
.mocharc.js
.nycrc
.prettierignore
.prettierrc.js
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
README.md
api-extractor.json
renovate.json
samples/README.md
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-automl/synth.py", line 36, in <module>
    node.postprocess_gapic_library(excludes=['.kokoro/trampoline.sh'])
TypeError: postprocess_gapic_library() got an unexpected keyword argument 'excludes'
2020-09-10 03:20:24,193 autosynth [ERROR] > Synthesis failed
2020-09-10 03:20:24,194 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 713d005 fix(deps): update dependency yargs to v16 (#438)
2020-09-10 03:20:24,212 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-09-10 03:20:24,226 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Removing api-extractor.json
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 690, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 539, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 670, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 375, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 273, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Can't get status of a batch predict operation "Location ID was not provided"

Summary

I'm trying to get the status of a batch operation like in this tutorial

but I always get the error:

'List of found errors:\t1.Field: name; Message: Location ID was not provided.\t2.Field: name; Message: Required field is invalid\t

image

Sample code

//making the the batch prediction
const automlModel= client.modelPath('project_id','zone','model_id');
const client = new automl.PredictionServiceClient();
let jobName="";
 try{
            jobName = await client.batchPredict({
                name: automlModel,
                inputConfig: {gcsSource:{inputUris:inputs}},
                outputConfig: {gcsDestination:{outputUriPrefix:output}},
            }).then(resp=>{
                const [operation, initialApiResponse] = resp;
                return initialApiResponse.name;
            });
     }catch(err){
            consoloe.log(err);
     }
//------
//later on the code
const clientAutoML = new automl.AutoMlClient();
  let response = "";
  try{
       [response] = await clientAutoML.operationsClient.getOperation(jobName);
   }catch(err){
      console.log(err); //I GET THE ERROR HERE
   } 
//----------

Environment details

  • OS: windows 10
  • Node.js version: v12.11.0
  • npm version: 6.11.3
  • @google-cloud/automl version: 1.7.2

Steps to reproduce

  1. Create a batch prediction and get it's name
  2. Get status of the batch prediction

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

.cfg
.kokoro/presubmit/node12/common.cfg
.kokoro/presubmit/node12/samples-test.cfg
.kokoro/presubmit/node12/system-test.cfg
.kokoro/presubmit/node12/test.cfg
.kokoro/publish.sh
.kokoro/release/docs-devsite.cfg
.kokoro/release/docs-devsite.sh
.kokoro/release/docs.cfg
.kokoro/release/docs.sh
.kokoro/release/publish.cfg
.kokoro/samples-test.sh
.kokoro/system-test.sh
.kokoro/test.bat
.kokoro/test.sh
.kokoro/trampoline.sh
.kokoro/trampoline_v2.sh
.mocharc.js
.nycrc
.prettierignore
.prettierrc.js
.trampolinerc
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
README.md
api-extractor.json
renovate.json
samples/README.md
2020-11-17 03:23:13,050 synthtool [DEBUG] > Post-processing GAPIC library...
DEBUG:synthtool:Post-processing GAPIC library...
2020-11-17 03:23:13,050 synthtool [DEBUG] > Installing dependencies...
DEBUG:synthtool:Installing dependencies...

> [email protected] postinstall /home/kbuilder/.cache/synthtool/nodejs-automl/node_modules/protobufjs
> node scripts/postinstall


> @google-cloud/[email protected] prepare /home/kbuilder/.cache/synthtool/nodejs-automl
> npm run compile


> @google-cloud/[email protected] precompile /home/kbuilder/.cache/synthtool/nodejs-automl
> gts clean

version: 14
Removing build ...

> @google-cloud/[email protected] compile /home/kbuilder/.cache/synthtool/nodejs-automl
> tsc -p . && cp -r protos build/

npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 656 packages from 767 contributors and audited 658 packages in 22.236s

58 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

2020-11-17 03:23:35,924 synthtool [DEBUG] > Running prelint...
DEBUG:synthtool:Running prelint...

> @google-cloud/[email protected] prelint /home/kbuilder/.cache/synthtool/nodejs-automl
> cd samples; npm link ../; npm install


> @google-cloud/[email protected] prepare /home/kbuilder/.cache/synthtool/nodejs-automl
> npm run compile


> @google-cloud/[email protected] precompile /home/kbuilder/.cache/synthtool/nodejs-automl
> gts clean

version: 14
Removing build ...

> @google-cloud/[email protected] compile /home/kbuilder/.cache/synthtool/nodejs-automl
> tsc -p . && cp -r protos build/

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 658 packages in 11.443s

58 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

/home/kbuilder/.nvm/versions/node/v14.15.1/lib/node_modules/@google-cloud/automl -> /home/kbuilder/.cache/synthtool/nodejs-automl
/home/kbuilder/.cache/synthtool/nodejs-automl/samples/node_modules/@google-cloud/automl -> /home/kbuilder/.nvm/versions/node/v14.15.1/lib/node_modules/@google-cloud/automl -> /home/kbuilder/.cache/synthtool/nodejs-automl
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 200 packages from 130 contributors and audited 289 packages in 6.458s

59 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

2020-11-17 03:23:55,397 synthtool [DEBUG] > Running fix...
DEBUG:synthtool:Running fix...

> @google-cloud/[email protected] fix /home/kbuilder/.cache/synthtool/nodejs-automl
> gts fix

version: 14

/home/kbuilder/.cache/synthtool/nodejs-automl/src/v1/auto_ml_client.ts
  118:48  error  'window' is not defined  no-undef

/home/kbuilder/.cache/synthtool/nodejs-automl/src/v1/prediction_service_client.ts
  104:48  error  'window' is not defined  no-undef

/home/kbuilder/.cache/synthtool/nodejs-automl/src/v1beta1/auto_ml_client.ts
  118:48  error  'window' is not defined  no-undef

/home/kbuilder/.cache/synthtool/nodejs-automl/src/v1beta1/prediction_service_client.ts
  104:48  error  'window' is not defined  no-undef

βœ– 4 problems (4 errors, 0 warnings)

Error: Command failed with exit code 1: node ./node_modules/eslint/bin/eslint --fix **/*.ts **/*.js **/*.tsx **/*.jsx --no-error-on-unmatched-pattern
    at makeError (/home/kbuilder/.cache/synthtool/nodejs-automl/node_modules/execa/lib/error.js:59:11)
    at handlePromise (/home/kbuilder/.cache/synthtool/nodejs-automl/node_modules/execa/index.js:114:26)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async run (/home/kbuilder/.cache/synthtool/nodejs-automl/node_modules/gts/build/src/cli.js:120:17) {
  shortMessage: 'Command failed with exit code 1: node ./node_modules/eslint/bin/eslint --fix **/*.ts **/*.js **/*.tsx **/*.jsx --no-error-on-unmatched-pattern',
  command: 'node ./node_modules/eslint/bin/eslint --fix **/*.ts **/*.js **/*.tsx **/*.jsx --no-error-on-unmatched-pattern',
  exitCode: 1,
  signal: undefined,
  signalDescription: undefined,
  stdout: undefined,
  stderr: undefined,
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @google-cloud/[email protected] fix: `gts fix`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @google-cloud/[email protected] fix script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/kbuilder/.npm/_logs/2020-11-17T11_24_39_517Z-debug.log
2020-11-17 03:24:39,525 synthtool [ERROR] > Failed executing npm run fix:

None
ERROR:synthtool:Failed executing npm run fix:

None
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-automl/synth.py", line 36, in <module>
    node.postprocess_gapic_library()
  File "/tmpfs/src/github/synthtool/synthtool/languages/node.py", line 194, in postprocess_gapic_library
    fix(hide_output=hide_output)
  File "/tmpfs/src/github/synthtool/synthtool/languages/node.py", line 179, in fix
    shell.run(["npm", "run", "fix"], hide_output=hide_output)
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
    raise exc
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
    encoding="utf-8",
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['npm', 'run', 'fix']' returned non-zero exit status 1.
2020-11-17 03:24:39,593 autosynth [ERROR] > Synthesis failed
2020-11-17 03:24:39,593 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 8aa9acf chore: update synth metadata (#465)
2020-11-17 03:24:39,613 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-11-17 03:24:39,626 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Removing build/
Removing node_modules/
Removing package-lock.json
Removing samples/node_modules/
Removing samples/package-lock.json
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 354, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 189, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 334, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 65, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth_toolbox.py", line 259, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

AutoML prediction bug on node js

I used the node js to perform translation prediction on my custom model on google cloud. However i keep getting 2 problems

  1. SyntaxError: await is only valid in async function

So, i removed the await keyword and ran it again and then came error number 2

  1. TypeError: Cannot read property 'payload' of undefined

My attempts to resolve was:

I added some console.log to see where the issue is occuring.

Based on my understanding of the code after i create a new prediction client instance, passing my model specs, it concatenates these specs into a full path to the model on the cloud.

Then it reads the file containing the text to translate locally then makes a request to the cloud model by formatting it into a .json and fires a request to the model.

Then it receives a response back in a json format where the 2nd issue mentioned above( TypeError : Cannot read property 'payload' of undefined) occurs it cannot assign anything to payload as the response is undefined.

Hence i can't go on to see the translation.

The await keyword error (SyntaxError: await is only valid in async function)
i believe is the solution to it (because maybe the response was returning before the model could be loaded or the code is synchronous hence await was causing the error to wait for the response to return with data).

Can anyone help me on resolving these errors. Thanks for your time :)

Here is the link to error number 2 screenshot:
TypeError: Cannot read property 'payload' of undefined

and here is the link to Google Cloud AutoML translation api (Step 5 is the translation)
AutoML prediction

Environment details

  • OS: Windows 10 version 1809
  • Node.js version: v10.15.3
  • npm version: 6.4.1
  • @google-cloud/automl version: v1beta1

Classification predict gives invalid argument error

Environment details

  • OS: Mac Catalina 10.15.7
  • Node.js version: 12.18.2
  • npm version: 6.14.5
  • @google-cloud/automl version: 2.3.0

Steps to reproduce

  1. I have this code in an Express server; I use the function as middleware when handling a post request:
const {PredictionServiceClient} = require('@google-cloud/automl').v1;

async function asyncFetch (req, res, next) {
  const projectId = "PROJECT_ID";
  const location = "us-central1";
  const modelId = "MODEL_ID";
  const client = new PredictionServiceClient();
  const content = req.body.text;
  const request = {
    name: client.modelPath(projectId, location, modelId),
    payload: {
      textSnippet: {
        content: content,
        mimeType: 'text/plain', 
      },
    },
  };
  const [response] = await client.predict(request);
  req.data = response.json();
  next();
}

I've confirmed that the projectId and modelId are correct.

  1. When making a post request from Postman, I get this error:
Error: 3 INVALID_ARGUMENT: Request contains an invalid argument.
    at Object.callErrorFromStatus (/FILE_PATH/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
    at Object.onReceiveStatus (/FILE_PATH/node_modules/@grpc/grpc-js/build/src/client.js:176:52)
    at Object.onReceiveStatus (/FILE_PATH/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:342:141)
    at Object.onReceiveStatus (/FILE_PATH/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:305:181)
    at /FILE_PATH/node_modules/@grpc/grpc-js/build/src/call-stream.js:124:78
    at processTicksAndRejections (internal/process/task_queues.js:79:11)

I hesitate to say there's a bug somewhere, but I have been tweaking my code and checking that my code is in line with the sample classification predict in the repository for the last 2 days and cannot get anywhere.

nodes crash when dataset is large using batch prediction

Summary

I'm using batch prediction for a large image dataset (1000 images, each of of them with their size in the range of 500 KB and 4 MB because they are photography images), when I'm using batch prediction for this dataset, I'm getting this error
image

it just says that but it does not tell what the error is; I would guess is because it run out of resources because of the large dataset and it crashes, because it works for smaller dataset

Environment details

  • OS: windows 10 64 bits
  • Node.js version: 12.11.0
  • npm version: 6.11.3
  • @google-cloud/automl version: 1.5.0

Steps to reproduce

  1. have a large dataset of photography images >1000 and create the csv with it's info
  2. make a batch prediction with the csv folder info of the images info

sample code

const automl = require('@google-cloud/automl');
const client = new automl.PredictionServiceClient();
const automlModel= client.modelPath('project_id','zone','model_id');
const input="gs://[bucketname]/images/info.csv"
const output="gs://[bucketname]/images/output"
try{
          responses = await client.batchPredict({
              name: automlModel,
              inputConfig: {gcsSource:{inputUris:[input]}},
              outputConfig: {gcsDestination:{outputUriPrefix:output}},
          }).then(resp=>{
              const [operation, initialApiResponse] = resp;
              console.log("initialApiResponse: ",initialApiResponse);
              return operation.promise();
          });
}catch(err){
          console.log(err);
 }

Replace AutoML test with tests for each sample which execute (not skipped)

Every code sample should have a corresponding test which is not skipped, as much as possible.

In cases where a code sample runs a long-running operation, this may mean that the cloud.google.com code sample may be extended to include a cancellation.

Alternatively, the test can cancel the operation after it has been started.

Operations which run in a reasonable amount of time should be executed, e.g. anything which takes < 1 minute to execute (as an example)

Fix the README

The README doesn't look like the others πŸ€·β€β™‚οΈ Lets update it to be consistent. Lets also make sure we understand how we're keeping these generated properly up front.

One file per sample

Please split large files (for example /samples/vision/automlVisionDataset.js) that contain more than one sample into small files that contain a single sample per file.

See @bcoe's comment:

A note for future improvements, the template we use for generating samples/README.md and README.md currently assumes there's one sample per-file. If we eventually split the samples across multiple files, we'll end up with a nicer corresponding table in README.md -- I don't see this as a blocker, but it would be good to create a tracking issue for someone to pick up πŸ˜„

Not possible to get operation status.

Environment details

  • OS: MacOS Catalina 10.15.6
  • Node.js version: 12.8.1
  • npm version: 6.14.15
  • @google-cloud/automl version: 2.2.0, using v1alpha1
  • "tsc": "^1.20150623.0",
  • "typescript": "^3.8.3",

Typescript config

{
"compilerOptions": {
...
"noImplicitAny": false,
"target": "es6",
"moduleResolution": "node",
"lib": ["es6", "dom"],
"allowJs": false,
"module": "esnext",
"sourceMap": true,
"jsx": "react",
...
},

Steps to reproduce

1.- Starting point:
https://github.com/googleapis/nodejs-automl/blob/master/samples/beta/get-operation-status.js

2.- Sample Implementation throws TS warnings, therefore I implemented the corresponding types.
*TS complains about a missing name property
*TS complains about the lack of toJSON property in object.

 public operationStatus = async (operationId: string) => {
    
    const request: google.longrunning.GetOperationRequest = new google.longrunning.GetOperationRequest(
      {
        'name' : `projects/${this.credentials.projectId}/locations/${this.settings.location}/operations/${operationId}`
      }
    );

    const [response] = await this.client.operationsClient.getOperation(request);

    return response;
  };

3.- On a recently deploy model action triggered, I try to get the status update on the deployment, sending the correct operationID. Nevertheless, the client.getOperation method keeps throwing a server error.
Code execution throws: missing name Field: name.

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

l reproducible form can be obtained by modifying arguments sha256 = "8d11f06b408ac5f1c01da3ca17f3a75dc008831509c5c1a4f24f9bde37792a57"
DEBUG: Call stack for the definition of repository 'gapic_generator_python' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:224:1
DEBUG: Rule 'com_googleapis_gapic_generator_go' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "12bfed7873f085093cd60615bd113178ecf36396af0c2ca25e6cd4d4bebdd198"
DEBUG: Call stack for the definition of repository 'com_googleapis_gapic_generator_go' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:261:1
DEBUG: Rule 'gapic_generator_typescript' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "ca322b5e7b0d03b3cc44a90444e3a7f944c9ba3345f0505ee48c8e715d19dd95"
DEBUG: Call stack for the definition of repository 'gapic_generator_typescript' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:280:1
DEBUG: Rule 'gapic_generator_csharp' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "40ddae63d2729ef5ccbd8b60123327ea200ce9400d0629238193ff530dcaea18"
DEBUG: Call stack for the definition of repository 'gapic_generator_csharp' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:318:1
DEBUG: Rule 'bazel_skylib' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "1dde365491125a3db70731e25658dfdd3bc5dbdfd11b840b3e987ecf043c7ca0"
DEBUG: Call stack for the definition of repository 'bazel_skylib' which is a http_archive (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
 - <builtin>
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:35:1
Analyzing: target //google/cloud/automl/v1:automl-v1-nodejs (1 packages loaded, 0 targets configured)
INFO: Call stack for the definition of repository 'npm' which is a yarn_install (rule definition at /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl:411:16):
 - <builtin>
 - /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/build_bazel_rules_nodejs/index.bzl:87:5
 - /home/kbuilder/.cache/synthtool/googleapis/WORKSPACE:293:1
Analyzing: target //google/cloud/automl/v1:automl-v1-nodejs (5 packages loaded, 467 targets configured)
ERROR: An error occurred during the fetch of repository 'npm':
   yarn_install failed: yarn install v1.19.1
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info If you think this is a bug, please open a bug report with the information provided in "/home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_typescript/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
 (error An unexpected error occurred: "https://registry.yarnpkg.com/@grpc/proto-loader/-/proto-loader-0.5.5.tgz: Request failed \"404 Not Found\"".
)
ERROR: /home/kbuilder/.cache/synthtool/googleapis/google/cloud/automl/v1/BUILD.bazel:294:1: //google/cloud/automl/v1:automl_nodejs_gapic depends on @gapic_generator_typescript//:protoc_plugin in repository @gapic_generator_typescript which failed to fetch. no such package '@npm//@bazel/typescript': yarn_install failed: yarn install v1.19.1
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info If you think this is a bug, please open a bug report with the information provided in "/home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_typescript/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
 (error An unexpected error occurred: "https://registry.yarnpkg.com/@grpc/proto-loader/-/proto-loader-0.5.5.tgz: Request failed \"404 Not Found\"".
)
ERROR: Analysis of target '//google/cloud/automl/v1:automl-v1-nodejs' failed; build aborted: no such package '@npm//@bazel/typescript': yarn_install failed: yarn install v1.19.1
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info If you think this is a bug, please open a bug report with the information provided in "/home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_typescript/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
 (error An unexpected error occurred: "https://registry.yarnpkg.com/@grpc/proto-loader/-/proto-loader-0.5.5.tgz: Request failed \"404 Not Found\"".
)
INFO: Elapsed time: 1.522s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (5 packages loaded, 468 targets configured)
FAILED: Build did NOT complete successfully (5 packages loaded, 468 targets configured)

2020-08-06 04:11:49,942 synthtool [DEBUG] > Wrote metadata to synth.metadata.
DEBUG:synthtool:Wrote metadata to synth.metadata.
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-automl/synth.py", line 28, in <module>
    library = gapic.node_library('automl', version)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 52, in node_library
    return self._generate_code(service, version, "nodejs", **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 183, in _generate_code
    shell.run(bazel_run_args)
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
    raise exc
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
    encoding="utf-8",
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bazel', '--max_idle_secs=240', 'build', '//google/cloud/automl/v1:automl-v1-nodejs']' returned non-zero exit status 1.
2020-08-06 04:11:49,996 autosynth [ERROR] > Synthesis failed
2020-08-06 04:11:49,996 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at d7b06a3 samples: moving autoML samples from nodejs-translate and refactored them (#432)
2020-08-06 04:11:50,003 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-08-06 04:11:50,009 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 690, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 539, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 670, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 375, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 273, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

[AutoML Tables] The provided location ID doesn't match (js)

I get this error when I query my AutoML Tables model when I deploy a Firebase cloud function.

Field: name; Message: The provided location ID doesn't match the endpoint. For automl.googleapis.com, the valid location ID is us-central1. For eu-automl.googleapis.com, the valid location ID is eu.

additional stacktrace:

    at Object.exports.createStatusError (/srv/node_modules/grpc/src/common.js:91:15)
    at Object.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:1209:28)
    at InterceptingListener._callNext (/srv/node_modules/grpc/src/client_interceptors.js:568:42)
    at InterceptingListener.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:618:8)
    at callback (/srv/node_modules/grpc/src/client_interceptors.js:847:24)
  code: 3,
  metadata: 
   Metadata {
     _internal_repr: 
      { 'google.rpc.badrequest-bin': [Array],
        'grpc-status-details-bin': [Array],
        'grpc-server-stats-bin': [Array] },
     flags: 0 },
  details: 'List of found errors:\t1.Field: name; Message: The provided location ID doesn\'t match the endpoint. For automl.googleapis.com, the valid location ID is `us-central1`. For eu-automl.googleapis.com, the valid location ID is `eu`.\t',
  note: 'Exception occurred in retry method that was not classified as transient' } 

I believe I have set the location in eu correctly.

My model was deployed in eu, so I have to use eu-automl.googleapis.com.

My code:

const project = 'some model project;
const region = 'eu';
const model = 'TBL45345345345345345';

const automl = require('@google-cloud/automl').v1beta1;
const clientOptions = { apiEndpoint: 'eu-automl.googleapis.com' };
const modelClient = new automl.AutoMlClient(clientOptions);
const predictionClient = new automl.PredictionServiceClient(clientOptions);

const projectLocation = modelClient.locationPath(project, region);

Both won't find location ID successfully:

const requestModel = {
        name: modelClient.modelPath(project, region, model),
};
modelClient.getModel(requestModel) .......
const requestPrediction = {
        name: predictionClient.modelPath(project, region, model),
        payload: payload,
        params: { feature_importance: true },
 };
 predictionClient.predict(requestPrediction).......

Field: payload.payload; Message: Required field not set (only when deployed)

Environment details

  • Node.js version: 10
  • @google-cloud/automl version: 2.2.0

Stack tracke:

Error: 3 INVALID_ARGUMENT: List of found errors:	1.Field: payload.payload; Message: Required field not set.	
    at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:174:52)
    at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:340:141)
    at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:303:181)
    at Http2CallStream.outputStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:114:27)
    at Http2CallStream.maybeOutputStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:153:22)
    at Http2CallStream.endCall (/workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:140:18)
    at Http2CallStream.handleTrailers (/workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:262:14)
    at ClientHttp2Stream.emit (events.js:198:13)
    at emit (internal/http2/core.js:265:8)

Works fine if I call predict method from my local environment.
Errors after deploying to GC AppEngine.
Nothing changed in my code.
I double checked and I'm sure that payload is passed in arguments.
It works fine on version I deployed on Jul 21, 2020.
I trained new model and I'm getting the same issue.

Here's my code:

const automl = require('@google-cloud/automl');
// Create client for prediction service.

const client = new automl.PredictionServiceClient();
const projectId = 'XXX';
const computeRegion = 'us-central1';
const modelId = 'XXX';

// Get the full path of the model.
const modelFullId = client.modelPath(projectId, computeRegion, modelId);

const payload = {
	"row": {
		"values": [
			{
				"numberValue": 6
			},
			{
				"numberValue": 0
			},
			{
				"numberValue": 0
			},
			{
				"numberValue": 0
			},
			{
				"stringValue": "Turkey"
			},
			{
				"stringValue": "Vinyl"
			},
			{
				"stringValue": "7inch"
			},
			{
				"stringValue": "45 RPM"
			},
			{
				"stringValue": "Mini-Album"
			},
			{
				"numberValue": 3
			},
			{
				"numberValue": 0
			},
			{
				"numberValue": 47.62
			}
		]
	}
}

const data = await client.predict({
			name: modelFullId,
			payload: payload,
			params: {}
		});

GCP AutoML dataset id return from Node.js client library is mismatched the actual dataset id.

Previously, i can use node.js automl to create dataset, import data and training the model successfully. But now i faced "Error: 5 NOT_FOUND: The Dataset doesn't exist or is inaccessible for use with AutoML".

After that, i realized that the dataset id return from create dataset request is mismatched the actual dataset id (compared with web interface).

Any update am i missing? or do i missed something?

Environment details

  • OS: Ubuntu 18.04 LTS
  • Node.js version: v12.16.1
  • npm version: 6.13.4
  • @google-cloud/automl version: v1

Steps to reproduce

function createAutoMLDataset() {

  return new Promise((resolve, reject) => {
    const request = {
      parent: automlClient.locationPath(projectId, location),
      dataset: {
        displayName: dataset_id,
        imageObjectDetectionDatasetMetadata: {}
      }
    };

    automlClient
      .createDataset(request)
      .then(responses => {
        console.log("[createAutoMLDataset] response", responses);
        resolve(responses);
      })
      .catch(err => {
        reject(`error occurred dataset: ${err}`);
      });
  });
}
     createAutoMLDataset()
          .then(datasetResp => {
            console.log("[datasetResp]", datasetResp);
            automlDatasetId = datasetResp[0].name.split("/")[
              datasetResp[0].name.split("/").length - 1
            ];
            console.log("[autoML datasetId]:", automlDatasetId);
            return uploadToAutoML(
              `gs://${bucketName}/${model_prefix}/test.csv`,
              automlDatasetId
            );
          }) 

Best.

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

Cloning into 'working_repo'...
Switched to branch 'autosynth'
οΏ½[35msynthtool > οΏ½[31mοΏ½[43mYou are running the synthesis script directly, this will be disabled in a future release of Synthtool. Please use python3 -m synthtool instead.οΏ½[0m
οΏ½[35msynthtool > οΏ½[36mEnsuring dependencies.οΏ½[0m
οΏ½[35msynthtool > οΏ½[36mPulling artman image.οΏ½[0m
latest: Pulling from googleapis/artman
Digest: sha256:2f6b261ee7fe1aedf238991c93a20b3820de37a343d0cacf3e3e9555c2aaf2ea
Status: Image is up to date for googleapis/artman:latest
οΏ½[35msynthtool > οΏ½[36mCloning googleapis.οΏ½[0m
οΏ½[35msynthtool > οΏ½[36mRunning generator for google/cloud/automl/artman_automl_v1beta1.yaml.οΏ½[0m
οΏ½[35msynthtool > οΏ½[32mGenerated code into /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/js/automl-v1beta1.οΏ½[0m
.circleci/config.yml
.circleci/npm-install-retry.js
.eslintignore
.eslintrc.yml
.github/CONTRIBUTING.md
.github/ISSUE_TEMPLATE/bug_report.md
.github/ISSUE_TEMPLATE/feature_request.md
.github/ISSUE_TEMPLATE/support_request.md
.jsdoc.js
.kokoro/common.cfg
.kokoro/continuous/node10/common.cfg
.kokoro/continuous/node10/test.cfg
.kokoro/continuous/node11/common.cfg
.kokoro/continuous/node11/test.cfg
.kokoro/continuous/node6/common.cfg
.kokoro/continuous/node6/test.cfg
.kokoro/continuous/node8/common.cfg
.kokoro/continuous/node8/docs.cfg
.kokoro/continuous/node8/lint.cfg
.kokoro/continuous/node8/samples-test.cfg
.kokoro/continuous/node8/system-test.cfg
.kokoro/continuous/node8/test.cfg
.kokoro/docs.sh
.kokoro/lint.sh
.kokoro/presubmit/node10/common.cfg
.kokoro/presubmit/node10/test.cfg
.kokoro/presubmit/node11/common.cfg
.kokoro/presubmit/node11/test.cfg
.kokoro/presubmit/node6/common.cfg
.kokoro/presubmit/node6/test.cfg
.kokoro/presubmit/node8/common.cfg
.kokoro/presubmit/node8/docs.cfg
.kokoro/presubmit/node8/lint.cfg
.kokoro/presubmit/node8/samples-test.cfg
.kokoro/presubmit/node8/system-test.cfg
.kokoro/presubmit/node8/test.cfg
.kokoro/presubmit/windows/common.cfg
.kokoro/presubmit/windows/test.cfg
.kokoro/samples-test.sh
.kokoro/system-test.sh
.kokoro/test.bat
.kokoro/test.sh
.kokoro/trampoline.sh
.nycrc
CODE_OF_CONDUCT.md
codecov.yaml
npm WARN deprecated [email protected]: CircularJSON is in maintenance only, flatted is its successor.

> [email protected] install /tmpfs/src/git/autosynth/working_repo/node_modules/grpc
> node-pre-gyp install --fallback-to-build --library=static_library

node-pre-gyp WARN Using request for node-pre-gyp https download 
[grpc] Success: "/tmpfs/src/git/autosynth/working_repo/node_modules/grpc/src/node/extension_binary/node-v57-linux-x64-glibc/grpc_node.node" is installed via remote

> [email protected] postinstall /tmpfs/src/git/autosynth/working_repo/node_modules/protobufjs
> node scripts/postinstall

npm notice created a lockfile as package-lock.json. You should commit this file.
added 599 packages from 858 contributors and audited 1331 packages in 13.334s
found 0 vulnerabilities


> @google-cloud/[email protected] prettier /tmpfs/src/git/autosynth/working_repo
> prettier --write src/*.js src/*/*.js samples/*.js samples/*/*.js test/*.js test/*/*.js system-test/*.js system-test/*/*.js

src/index.js 88ms
src/v1beta1/auto_ml_client.js 137ms
src/v1beta1/index.js 13ms
src/v1beta1/prediction_service_client.js 27ms
samples/quickstart.js 6ms
test/gapic-v1beta1.js 259ms
system-test/automl.js 14ms
οΏ½[35msynthtool > οΏ½[36mCleaned up 2 temporary directories.οΏ½[0m

Changed files:

On branch autosynth
nothing to commit, working tree clean
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/git/autosynth/autosynth/synth.py", line 166, in <module>
    main()
  File "/tmpfs/src/git/autosynth/autosynth/synth.py", line 151, in main
    commit_changes(pr_title)
  File "/tmpfs/src/git/autosynth/autosynth/synth.py", line 95, in commit_changes
    subprocess.check_call(["git", "commit", "-m", message])
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'commit', '-m', '[CHANGE ME] Re-generated  to pick up changes in the API or client library generator.']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

5 packages in 15.554s
found 0 vulnerabilities

2020-06-28 04:37:01,332 synthtool [DEBUG] > Running prelint...
DEBUG:synthtool:Running prelint...
npm WARN npm npm does not support Node.js v12.18.1
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/

> @google-cloud/[email protected] prelint /home/kbuilder/.cache/synthtool/nodejs-automl
> cd samples; npm link ../; npm install

npm WARN npm npm does not support Node.js v12.18.1
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/
npm ERR! path /usr/lib/node_modules/@google-cloud
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall mkdir
npm ERR! Error: EACCES: permission denied, mkdir '/usr/lib/node_modules/@google-cloud'
npm ERR!  [Error: EACCES: permission denied, mkdir '/usr/lib/node_modules/@google-cloud'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/usr/lib/node_modules/@google-cloud'
npm ERR! }
npm ERR! 
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR! 
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator (though this is not recommended).

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/kbuilder/.npm/_logs/2020-06-28T11_37_01_747Z-debug.log
npm WARN npm npm does not support Node.js v12.18.1
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 280 packages in 1.972s
found 0 vulnerabilities

2020-06-28 04:37:04,125 synthtool [DEBUG] > Running fix...
DEBUG:synthtool:Running fix...
npm WARN npm npm does not support Node.js v12.18.1
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/

> @google-cloud/[email protected] fix /home/kbuilder/.cache/synthtool/nodejs-automl
> gts fix

version: 12
2020-06-28 04:37:44,066 synthtool [DEBUG] > Compiling protos...
DEBUG:synthtool:Compiling protos...
2020-06-28 04:37:53,171 synthtool [DEBUG] > Post-processing completed
DEBUG:synthtool:Post-processing completed
2020-06-28 04:37:53,173 synthtool [DEBUG] > Wrote metadata to synth.metadata.
DEBUG:synthtool:Wrote metadata to synth.metadata.
2020-06-28 04:37:53,262 autosynth [INFO] > Changed files:
2020-06-28 04:37:53,262 autosynth [INFO] > M samples/vision/automlVisionDataset.js
 M samples/vision/automlVisionModel.js
 M samples/vision/automlVisionPredict.js
 M synth.metadata
2020-06-28 04:37:53,262 autosynth [DEBUG] > Running: git add -A
warning: CRLF will be replaced by LF in samples/vision/automlVisionDataset.js.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in samples/vision/automlVisionModel.js.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in samples/vision/automlVisionPredict.js.
The file will have its original line endings in your working directory
2020-06-28 04:37:53,277 autosynth [DEBUG] > Running: git commit -m build: add config .gitattributes

Source-Author: Summer Ji <[email protected]>
Source-Date: Fri Jun 26 15:23:23 2020 -0700
Source-Repo: googleapis/synthtool
Source-Sha: dc9caca650c77b7039e2bbc3339ffb34ae78e5b7
Source-Link: https://github.com/googleapis/synthtool/commit/dc9caca650c77b7039e2bbc3339ffb34ae78e5b7
[autosynth-208 cbeaf65] build: add config .gitattributes
 4 files changed, 6 insertions(+), 6 deletions(-)
2020-06-28 04:37:53,296 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at cbeaf65 build: add config .gitattributes
2020-06-28 04:37:53,320 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-06-28 04:37:53,334 autosynth [DEBUG] > Running: git checkout 0f668e46684688b706bf59e459235d9e9fd49379
Note: checking out '0f668e46684688b706bf59e459235d9e9fd49379'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 0f668e4 build: add config .gitattributes (#415)
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-06-28 04:37:53,349 autosynth [DEBUG] > Running: git checkout 2fc2caaacb15949c7f80426bfc7dafdd41dbc333
Previous HEAD position was 7c577e89 fix: migrate monitoring/v3 to grpc_service_config
HEAD is now at 2fc2caaa chore: enable gapic v2 and proto annotation for bigquery/storage/v1beta2 API.
2020-06-28 04:37:53,450 autosynth [DEBUG] > Running: git checkout dc9caca650c77b7039e2bbc3339ffb34ae78e5b7
HEAD is now at dc9caca build: add config .gitattributes (#647)
2020-06-28 04:37:53,455 autosynth [DEBUG] > Running: git branch -f autosynth-0
2020-06-28 04:37:53,459 autosynth [DEBUG] > Running: git checkout autosynth-0
Switched to branch 'autosynth-0'
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-06-28 04:37:53,473 autosynth [DEBUG] > Running: git merge --ff-only autosynth-self-0
error: Your local changes to the following files would be overwritten by merge:
	samples/vision/automlVisionDataset.js
	samples/vision/automlVisionModel.js
	samples/vision/automlVisionPredict.js
Please commit your changes or stash them before you merge.
Aborting
Updating 0f668e4..178055a
2020-06-28 04:37:53,488 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 0f668e4 build: add config .gitattributes (#415)
2020-06-28 04:37:53,512 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-06-28 04:37:53,527 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Removing build/
Removing node_modules/
Removing package-lock.json
Removing samples/node_modules/
Removing samples/package-lock.json
Traceback (most recent call last):
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 367, in synthesize_loop
    synthesize_inner_loop(fork, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 417, in synthesize_inner_loop
    synthesizer, 0
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 258, in synthesize_version_in_new_branch
    ["git", "merge", "--ff-only", self.version_zero.branch_name]
  File "/tmpfs/src/github/synthtool/autosynth/executor.py", line 29, in check_call
    subprocess.check_call(command, **args)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'merge', '--ff-only', 'autosynth-self-0']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 657, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 506, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 637, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 375, in synthesize_loop
    synthesize_loop_single_pr(toolbox, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 401, in synthesize_loop_single_pr
    synthesize_inner_loop(toolbox, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 417, in synthesize_inner_loop
    synthesizer, 0
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 258, in synthesize_version_in_new_branch
    ["git", "merge", "--ff-only", self.version_zero.branch_name]
  File "/tmpfs/src/github/synthtool/autosynth/executor.py", line 29, in check_call
    subprocess.check_call(command, **args)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'merge', '--ff-only', 'autosynth-self-0']' returned non-zero exit status 1.

Google internal developers can see the full log here.

How to pass the value as Google cloud Storage URI in vision_classification_predict.js

Here, in filePath, I am giving the gcs bucket URI like below.

   const filePath = 'gs://buckettest/user/userface.jpg';
     // Imports the Google Cloud AutoML library
    const {PredictionServiceClient} = require(`@google-cloud/automl`).v1;
    const fs = require(`fs`);

  // Instantiates a client
  const client = new PredictionServiceClient();

  // Read the file content for translation.
  const content = fs.readFileSync(filePath);

  async function predict() {
    // Construct request
    // params is additional domain-specific parameters.
    // score_threshold is used to filter the result
    const request = {
      name: client.modelPath(projectId, location, modelId),
      payload: {
        image: {
          imageBytes: filePath,
        },
      },
    };

    const [response] = await client.predict(request);

    for (const annotationPayload of response.payload) {
      console.log(`Predicted class name: ${annotationPayload.displayName}`);
      console.log(
        `Predicted class score: ${annotationPayload.classification.score}`
      );
    }
  }

  predict();

If I am using payload something like this. I am getting an error "INVALID_ARGUMENT: List of found errors: 1.Field: payload.payload; Message: Required field not set."

If i am using the local file as mentioned in vision_classification_predict.js there is no issue.

payload: {
            inputConfig: {
                gcsSource: {
                  inputUris: [filePath],
                },
              },

Please help me with this. I am new to these APIs.

Address all skipped tests and code style in the samples

In the run-up to GCP next, we made a concsious choice to let a few things slip in the samples and sample test code:

  • Consistently use async/await instead of promises
  • Use execSync over execa for execution
  • Use getProjectId() instead of requiring the GCLOUD_PROJECT env var

We need to come back around and clean all of these up.

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

05de3e1e14a0b07eab8b474e669164dbd31f81fb
2020-09-11 03:17:04,362 autosynth [DEBUG] > Running: git log -1 --pretty=%at 968465a1cad496e1292ef4584a054a35f756ff94
2020-09-11 03:17:04,366 autosynth [DEBUG] > Running: git log -1 --pretty=%at a9eea2c50b7dce0dffcc010c1caf712802155403
2020-09-11 03:17:04,369 autosynth [DEBUG] > Running: git log -1 --pretty=%at 637f8aa1373a0a6be1b626a282f2e13d7d6d7d6c
2020-09-11 03:17:04,372 autosynth [DEBUG] > Running: git log -1 --pretty=%at d0198121927f606e113275a4b0f3560a7a821470
2020-09-11 03:17:04,375 autosynth [DEBUG] > Running: git log -1 --pretty=%at 8cf6d2834ad14318e64429c3b94f6443ae83daf9
2020-09-11 03:17:04,378 autosynth [DEBUG] > Running: git log -1 --pretty=%at 019c7168faa0e56619f792693a8acdb30d6de19b
2020-09-11 03:17:04,382 autosynth [DEBUG] > Running: git log -1 --pretty=%at 5d916ec54cadd4674e80e6555d0c6a78849ef4a7
2020-09-11 03:17:04,385 autosynth [DEBUG] > Running: git log -1 --pretty=%at cbcd64279572769b4d350bf8078bcd1f151c9684
2020-09-11 03:17:04,388 autosynth [DEBUG] > Running: git log -1 --pretty=%at 80f46100c047bc47efe0025ee537dc8ee413ad04
2020-09-11 03:17:04,391 autosynth [DEBUG] > Running: git log -1 --pretty=%at d91dd8aac77f7a9c5506c238038a26fa4f9e361e
2020-09-11 03:17:04,394 autosynth [DEBUG] > Running: git log -1 --pretty=%at 7b3766035373f5da99abf9413f1f7c29f108a2fd
2020-09-11 03:17:04,397 autosynth [DEBUG] > Running: git log -1 --pretty=%at 32c758f11b8c578f515a746c9d263b82a615a77c
2020-09-11 03:17:04,400 autosynth [DEBUG] > Running: git log -1 --pretty=%at b2c32f1c8a4094f0f47fcf5d10f0b6f2bfb3387d
2020-09-11 03:17:04,403 autosynth [DEBUG] > Running: git log -1 --pretty=%at 470e98896484cdce77e073e83a2c4f9157770f8c
2020-09-11 03:17:04,407 autosynth [DEBUG] > Running: git log -1 --pretty=%at d302f93d7f47e2852e585ac35ab2d15585717ec0
2020-09-11 03:17:04,410 autosynth [DEBUG] > Running: git log -1 --pretty=%at ffcee7952b74f647cbb3ef021d95422f10816fca
2020-09-11 03:17:04,413 autosynth [DEBUG] > Running: git log -1 --pretty=%at b15c0c042cdea746fc19856527d8baf947c3c220
2020-09-11 03:17:04,417 autosynth [DEBUG] > Running: git checkout bc929c069d2645df2026f4ff369795ba6118ca1f
Note: checking out 'bc929c069d2645df2026f4ff369795ba6118ca1f'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at bc929c0 samples: fix line endings (#443)
2020-09-11 03:17:04,491 autosynth [DEBUG] > Running: git checkout b15c0c042cdea746fc19856527d8baf947c3c220
Note: checking out 'b15c0c042cdea746fc19856527d8baf947c3c220'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at b15c0c0 fix: don't attempt to create redundant PR to start tracking obsolete files (#755)
2020-09-11 03:17:04,504 autosynth [DEBUG] > Running: git branch -f autosynth-76
2020-09-11 03:17:04,507 autosynth [DEBUG] > Running: git checkout autosynth-76
Switched to branch 'autosynth-76'
2020-09-11 03:17:04,512 autosynth [INFO] > Running synthtool
2020-09-11 03:17:04,512 autosynth [INFO] > ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
2020-09-11 03:17:04,513 autosynth [DEBUG] > log_file_path: /tmpfs/src/logs/nodejs-automl/76/sponge_log.log
2020-09-11 03:17:04,514 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --
2020-09-11 03:17:04,737 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/nodejs-automl/synth.py.
On branch autosynth-76
nothing to commit, working tree clean
2020-09-11 03:17:04,901 synthtool [DEBUG] > Ensuring dependencies.
DEBUG:synthtool:Ensuring dependencies.
2020-09-11 03:17:04,911 synthtool [DEBUG] > Cloning googleapis.
DEBUG:synthtool:Cloning googleapis.
2020-09-11 03:17:07,067 synthtool [DEBUG] > Generating code for: //google/cloud/automl/v1:automl-v1-nodejs.
DEBUG:synthtool:Generating code for: //google/cloud/automl/v1:automl-v1-nodejs.
2020-09-11 03:20:11,706 synthtool [SUCCESS] > Generated code into /tmpfs/tmp/tmp0vvnqfhi.
SUCCESS:synthtool:Generated code into /tmpfs/tmp/tmp0vvnqfhi.
2020-09-11 03:20:11,766 synthtool [DEBUG] > Generating code for: //google/cloud/automl/v1beta1:automl-v1beta1-nodejs.
DEBUG:synthtool:Generating code for: //google/cloud/automl/v1beta1:automl-v1beta1-nodejs.
2020-09-11 03:20:17,983 synthtool [SUCCESS] > Generated code into /tmpfs/tmp/tmpqbsdcbh5.
SUCCESS:synthtool:Generated code into /tmpfs/tmp/tmpqbsdcbh5.
2020-09-11 03:20:18,060 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
DEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
2020-09-11 03:20:18,080 synthtool [INFO] > successfully generate `src/index.ts`
INFO:synthtool:successfully generate `src/index.ts`
.eslintignore
.eslintrc.json
.gitattributes
.github/ISSUE_TEMPLATE/bug_report.md
.github/ISSUE_TEMPLATE/feature_request.md
.github/ISSUE_TEMPLATE/support_request.md
.github/PULL_REQUEST_TEMPLATE.md
.github/release-please.yml
.github/workflows/ci.yaml
.kokoro/.gitattributes
.kokoro/common.cfg
.kokoro/continuous/node10/common.cfg
.kokoro/continuous/node10/docs.cfg
.kokoro/continuous/node10/test.cfg
.kokoro/continuous/node12/common.cfg
.kokoro/continuous/node12/lint.cfg
.kokoro/continuous/node12/samples-test.cfg
.kokoro/continuous/node12/system-test.cfg
.kokoro/continuous/node12/test.cfg
.kokoro/docs.sh
.kokoro/lint.sh
.kokoro/populate-secrets.sh
.kokoro/presubmit/node10/common.cfg
.kokoro/presubmit/node12/common.cfg
.kokoro/presubmit/node12/samples-test.cfg
.kokoro/presubmit/node12/system-test.cfg
.kokoro/presubmit/node12/test.cfg
.kokoro/publish.sh
.kokoro/release/docs-devsite.cfg
.kokoro/release/docs-devsite.sh
.kokoro/release/docs.cfg
.kokoro/release/docs.sh
.kokoro/release/publish.cfg
.kokoro/samples-test.sh
.kokoro/system-test.sh
.kokoro/test.bat
.kokoro/test.sh
.kokoro/trampoline.sh
.mocharc.js
.nycrc
.prettierignore
.prettierrc.js
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
README.md
api-extractor.json
renovate.json
samples/README.md
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-automl/synth.py", line 36, in <module>
    node.postprocess_gapic_library(excludes=['.kokoro/trampoline.sh'])
TypeError: postprocess_gapic_library() got an unexpected keyword argument 'excludes'
2020-09-11 03:20:18,304 autosynth [ERROR] > Synthesis failed
2020-09-11 03:20:18,305 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at bc929c0 samples: fix line endings (#443)
2020-09-11 03:20:18,322 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-09-11 03:20:18,335 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Removing api-extractor.json
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 692, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 541, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 672, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 375, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 273, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

The provided location ID is not valid

I'm currently experiencing an issue when using this package.
I've created a new EU model and when trying to use it. Despite using the 'eu' location ID, i'm still seeing this issue.

(node:8904) UnhandledPromiseRejectionWarning: Error: 3 INVALID_ARGUMENT: List of found errors: 1.Field: name; Message: The provided location ID doesn't match the endpoint. For automl.googleapis.com, the valid location ID is us-central1. For eu-automl.googleapis.com, the valid location ID is eu.

Environment details

  • OS: Windows
  • Node.js version: v12.14.0
  • @google-cloud/automl version: 1.10.0

Steps to reproduce

  1. Create a new model with project settings as 'EU' then try use it.

Thanks!

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

ort Node.js v12.18.2
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/

> @google-cloud/[email protected] prelint /home/kbuilder/.cache/synthtool/nodejs-automl
> cd samples; npm link ../; npm install

npm WARN npm npm does not support Node.js v12.18.2
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/
npm ERR! path /usr/lib/node_modules/@google-cloud
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall mkdir
npm ERR! Error: EACCES: permission denied, mkdir '/usr/lib/node_modules/@google-cloud'
npm ERR!  [Error: EACCES: permission denied, mkdir '/usr/lib/node_modules/@google-cloud'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/usr/lib/node_modules/@google-cloud'
npm ERR! }
npm ERR! 
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR! 
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator (though this is not recommended).

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/kbuilder/.npm/_logs/2020-07-16T11_55_52_363Z-debug.log
npm WARN npm npm does not support Node.js v12.18.2
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 281 packages in 1.753s
found 0 vulnerabilities

2020-07-16 04:55:54,495 synthtool [DEBUG] > Running fix...
DEBUG:synthtool:Running fix...
npm WARN npm npm does not support Node.js v12.18.2
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/

> @google-cloud/[email protected] fix /home/kbuilder/.cache/synthtool/nodejs-automl
> gts fix

version: 12
2020-07-16 04:56:36,942 synthtool [DEBUG] > Compiling protos...
DEBUG:synthtool:Compiling protos...
2020-07-16 04:56:45,202 synthtool [DEBUG] > Post-processing completed
DEBUG:synthtool:Post-processing completed
2020-07-16 04:56:45,203 synthtool [DEBUG] > Wrote metadata to synth.metadata.
DEBUG:synthtool:Wrote metadata to synth.metadata.
2020-07-16 04:56:45,269 autosynth [INFO] > Changed files:
2020-07-16 04:56:45,270 autosynth [INFO] > M .gitattributes
 M .kokoro/publish.sh
 M .kokoro/samples-test.sh
 M protos/protos.js
 M samples/package.json
 M samples/vision/automlVisionDataset.js
 M samples/vision/automlVisionModel.js
 M samples/vision/automlVisionPredict.js
 M synth.metadata
2020-07-16 04:56:45,270 autosynth [DEBUG] > Running: git add -A
warning: CRLF will be replaced by LF in samples/vision/automlVisionDataset.js.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in samples/vision/automlVisionModel.js.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in samples/vision/automlVisionPredict.js.
The file will have its original line endings in your working directory
2020-07-16 04:56:45,314 autosynth [DEBUG] > Running: git commit -m chore: delete unused node kokoro configs from template

Co-authored-by: Justin Beckwith <[email protected]>

Source-Author: F. Hinkelmann <[email protected]>
Source-Date: Wed Jul 15 11:06:22 2020 -0400
Source-Repo: googleapis/synthtool
Source-Sha: 2a5693326b1e708ea8464b4cb06ea2894691c365
Source-Link: https://github.com/googleapis/synthtool/commit/2a5693326b1e708ea8464b4cb06ea2894691c365
[autosynth-8 e014f4c] chore: delete unused node kokoro configs from template
 9 files changed, 397 insertions(+), 114 deletions(-)
2020-07-16 04:56:45,451 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at e014f4c chore: delete unused node kokoro configs from template
2020-07-16 04:56:45,478 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-07-16 04:56:45,524 autosynth [DEBUG] > Running: git checkout d7415321eac5ebc3007fd26aa1b303cd1596a1aa
Note: checking out 'd7415321eac5ebc3007fd26aa1b303cd1596a1aa'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at d741532 docs(automl_video_intelligence_classification): remove unused samples and files (#374)
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-07-16 04:56:45,570 autosynth [DEBUG] > Running: git checkout 303271797a360f8a439203413f13a160f2f5b3b4
Previous HEAD position was 2a56933 chore: delete unused node kokoro configs from template (#665)
HEAD is now at 3032717 chore: update Py2 support msg to reflect passage of time (#652)
2020-07-16 04:56:45,578 autosynth [DEBUG] > Running: git branch -f autosynth-0
2020-07-16 04:56:45,582 autosynth [DEBUG] > Running: git checkout autosynth-0
Switched to branch 'autosynth-0'
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-07-16 04:56:45,626 autosynth [DEBUG] > Running: git merge --ff-only autosynth-self-0
error: Your local changes to the following files would be overwritten by merge:
	samples/vision/automlVisionDataset.js
	samples/vision/automlVisionModel.js
	samples/vision/automlVisionPredict.js
Please commit your changes or stash them before you merge.
Aborting
Updating d741532..bbe30e4
2020-07-16 04:56:45,670 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at d741532 docs(automl_video_intelligence_classification): remove unused samples and files (#374)
2020-07-16 04:56:45,753 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-07-16 04:56:45,798 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Removing build/
Removing node_modules/
Removing package-lock.json
Removing samples/node_modules/
Removing samples/package-lock.json
Traceback (most recent call last):
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 367, in synthesize_loop
    synthesize_inner_loop(fork, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 417, in synthesize_inner_loop
    synthesizer, 0
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 258, in synthesize_version_in_new_branch
    ["git", "merge", "--ff-only", self.version_zero.branch_name]
  File "/tmpfs/src/github/synthtool/autosynth/executor.py", line 29, in check_call
    subprocess.check_call(command, **args)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'merge', '--ff-only', 'autosynth-self-0']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 657, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 506, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 637, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 375, in synthesize_loop
    synthesize_loop_single_pr(toolbox, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 401, in synthesize_loop_single_pr
    synthesize_inner_loop(toolbox, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 417, in synthesize_inner_loop
    synthesizer, 0
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 258, in synthesize_version_in_new_branch
    ["git", "merge", "--ff-only", self.version_zero.branch_name]
  File "/tmpfs/src/github/synthtool/autosynth/executor.py", line 29, in check_call
    subprocess.check_call(command, **args)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'merge', '--ff-only', 'autosynth-self-0']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Typescript support

Currently there is no npm package called @types/google-cloud__automl
This could enhance the experience users have with the library. Since other google cloud package feature typescript support also, it makes sense to create this npm package.

Tests need refactoring

AutoML Create model tests have no teardown. this caused AUTOML_PROJECT_ID project to become the biggest project in the automl (83k LROs, 1.4k models, 540 datasets).

Training an existing model

I have been spending a lot of time lately to learn if and how it is possible to train an existing model with an existing dataset. Meaning that I have a csv file that I have imported to an existing dataset but when I want to train a model, it creates a new model every time I run the training script.

Below is the script I am using and as you can see, it creates a new model after the training instead of using an existing one. Can someone please tell me how to do it?

const automl = require(`@google-cloud/automl`);

const client = new automl.v1beta1.AutoMlClient();

const projectId = `xxx`;
const computeRegion = `us-central1`;
const datasetId = `xxx`;
const modelName = `myModel2`;

(async () => {
  // A resource that represents Google Cloud Platform location.
  const projectLocation = client.locationPath(projectId, computeRegion);

  // Set model name and model metadata for the dataset.
  const myModel = {
    displayName: modelName,
    datasetId: datasetId,
    textClassificationModelMetadata: {}
  };

  // Create a model with the model metadata in the region.
  const [operation, initialApiResponse] = await client.createModel({
    parent: projectLocation,
    model: myModel
  });
  console.log(`Training operation name: ${initialApiResponse.name}`);
  console.log(`Training started...`);
  const [model] = await operation.promise();
  // Retrieve deployment state.
  let deploymentState = ``;
  if (model.deploymentState === 1) {
    deploymentState = `deployed`;
  } else if (model.deploymentState === 2) {
    deploymentState = `undeployed`;
  }

  // Display the model information.
  console.log(`Model name: ${model.name}`);
  console.log(`Model id: ${model.name.split(`/`).pop(-1)}`);
  console.log(`Model display name: ${model.displayName}`);
  console.log(`Model create time:`);
  console.log(`\tseconds: ${model.createTime.seconds}`);
  console.log(`\tnanos: ${model.createTime.nanos}`);
  console.log(`Model deployment state: ${deploymentState}`);
})();

Synthesis failed for nodejs-automl

Hello! Autosynth couldn't regenerate nodejs-automl. πŸ’”

Here's the output from running synth.py:

3.13.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting pypandoc==1.5 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/d6/b7/5050dc1769c8a93d3ec7c4bd55be161991c94b8b235f88bf7c764449e708/pypandoc-1.5.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmpfs/tmp/tmpe0qc1jg8/setuptools-tmp/setuptools/__init__.py", line 6, in <module>
        import distutils.core
      File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/_distutils_hack/__init__.py", line 82, in create_module
        return importlib.import_module('._distutils', 'setuptools')
      File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    ModuleNotFoundError: No module named 'setuptools._distutils'
    
    ----------------------------------------
 (  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
Command "python setup.py egg_info" failed with error code 1 in /tmpfs/tmp/pip-build-gdsqvqky/pypandoc/
)
ERROR: no such package '@gapic_generator_python_pip_deps//': pip_import failed: Collecting click==7.1.2 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl (82kB)
  Saved ./click-7.1.2-py2.py3-none-any.whl
Collecting google-api-core==1.22.1 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/e0/2d/7c6c75013105e1d2b6eaa1bf18a56995be1dbc673c38885aea31136e9918/google_api_core-1.22.1-py2.py3-none-any.whl (91kB)
  Saved ./google_api_core-1.22.1-py2.py3-none-any.whl
Collecting googleapis-common-protos==1.52.0 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/03/74/3956721ea1eb4bcf7502a311fdaa60b85bd751de4e57d1943afe9b334141/googleapis_common_protos-1.52.0-py2.py3-none-any.whl (100kB)
  Saved ./googleapis_common_protos-1.52.0-py2.py3-none-any.whl
Collecting jinja2==2.11.2 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 4))
  Downloading https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl (125kB)
  Saved ./Jinja2-2.11.2-py2.py3-none-any.whl
Collecting MarkupSafe==1.1.1 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 5))
  Downloading https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
  Saved ./MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting protobuf==3.13.0 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 6))
  Downloading https://files.pythonhosted.org/packages/30/79/510974552cebff2ba04038544799450defe75e96ea5f1675dbf72cc8744f/protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl (1.3MB)
  Saved ./protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting pypandoc==1.5 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/d6/b7/5050dc1769c8a93d3ec7c4bd55be161991c94b8b235f88bf7c764449e708/pypandoc-1.5.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmpfs/tmp/tmpe0qc1jg8/setuptools-tmp/setuptools/__init__.py", line 6, in <module>
        import distutils.core
      File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/_distutils_hack/__init__.py", line 82, in create_module
        return importlib.import_module('._distutils', 'setuptools')
      File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    ModuleNotFoundError: No module named 'setuptools._distutils'
    
    ----------------------------------------
 (  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
Command "python setup.py egg_info" failed with error code 1 in /tmpfs/tmp/pip-build-gdsqvqky/pypandoc/
)
INFO: Elapsed time: 18.840s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)

Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-automl/synth.py", line 28, in <module>
    library = gapic.node_library('automl', version)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 52, in node_library
    return self._generate_code(service, version, "nodejs", **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 183, in _generate_code
    shell.run(bazel_run_args)
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
    raise exc
  File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
    encoding="utf-8",
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bazel', '--max_idle_secs=240', 'build', '//google/cloud/automl/v1:automl-v1-nodejs']' returned non-zero exit status 1.
2020-08-31 03:17:42,964 autosynth [ERROR] > Synthesis failed
2020-08-31 03:17:42,964 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at d7b06a3 samples: moving autoML samples from nodejs-translate and refactored them (#432)
2020-08-31 03:17:42,970 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
M	samples/vision/automlVisionDataset.js
M	samples/vision/automlVisionModel.js
M	samples/vision/automlVisionPredict.js
2020-08-31 03:17:42,976 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 690, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 539, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 670, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 375, in synthesize_loop
    has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 273, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Error: ENOENT: no such file or directory, open 'google/cloud/automl/v1beta1/prediction_service.proto'

Environment details

  • OS: OSx
  • Node.js version: 8.10
  • Yarn: 1.17.3
  • @google-cloud/automl version: 1.2.4

The end goal will be running @google-cloud/automl inside of an AWS lambda but I'm currently running into this issue while testing locally using the serverless js framework.

Steps to reproduce

  1. Intialize the client with
const client = new automl.v1beta1.PredictionServiceClient();
  1. Returns the following error:
There was an error making predictions on the links { Error: ENOENT: no such file or directory, open 'google/cloud/automl/v1beta1/prediction_service.proto'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:343:35)
    at fetch (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/node_modules/protobufjs/src/root.js:160:34)
    at Root.load (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/node_modules/protobufjs/src/root.js:194:13)
    at Root.loadSync (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/node_modules/protobufjs/src/root.js:235:17)
    at Object.loadSync (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/node_modules/@grpc/proto-loader/build/src/index.js:230:27)
    at GrpcClient.loadFromProto (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/node_modules/google-gax/build/src/grpc.js:118:44)
    at GrpcClient.loadProto (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/node_modules/google-gax/build/src/grpc.js:150:21)
    at new PredictionServiceClient (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/.webpack/service/functions/prospector.js:4084:28)
    at Object.<anonymous> (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/.webpack/service/functions/prospector.js:187:24)
    at Generator.next (<anonymous>)
    at /Users/cmaycumber/Documents/backlink/Backlink-Prospector/.webpack/service/functions/prospector.js:99:71
    at new Promise (<anonymous>)
    at module.exports.__awaiter (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/.webpack/service/functions/prospector.js:95:12)
    at makePredictions (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/.webpack/service/functions/prospector.js:182:60)
    at Object.<anonymous> (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/.webpack/service/functions/prospector.js:315:27)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/cmaycumber/Documents/backlink/Backlink-Prospector/.webpack/service/functions/prospector.js:96:58)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: 'google/cloud/automl/v1beta1/prediction_service.proto' }

Any help in solving this would be great.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.