Code Monkey home page Code Monkey logo

spinnaker-for-gcp's Introduction

Install and manage Spinnaker on Google Cloud Platform

Spinnaker on Google Cloud Platform is a tool for easily installing a production-ready instance of Spinnaker, and for managing that instance over time.

Do I want to use this solution?

This solution is for…

  • Anyone who wants an easy path to install open-source Spinnaker, in a production-ready configuration, on Google Cloud Platform

  • Anyone who wants to "kick the tires" of Spinnaker, to decide if it's the right CD solution for their needs

  • Administrators who will manage one or more long-running instances of Spinnaker, including adding additional administrators, adding accounts, upgrading, and so on

This solution gives you...

  • Google recommendations and best practices for installing and running Spinnaker on GCP

  • Pre-integration with many other services that Spinnaker is commonly used with

  • Sample applications and other helpers for a smoother experience

What is this solution?

Spinnaker for Google Cloud Platform is a solution for installing and managing Spinnaker on Google Cloud Platform. It consists of an installation and management console, Spinnaker and its microservices, and sample applications.

What is Spinnaker?

Spinnaker is an open source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence.

If you would like to learn more about Spinnaker, please visit the Spinnaker website.

What is Deck?

Deck is the Spinnaker UI. You access Deck in one of the following ways:

  • Via port forwarding

    The management console provides a command for forwarding port 8080, and a button to click to access Deck via that port.

  • Over the internet, on a publicly available domain

    This domain is secured with Identity-Aware Proxy.

The management console

The management console makes it easy for you to do the following:

  • Install Spinnaker

    Spinnaker for Google Cloud Platform makes it easy to get a working version of open-source Spinnaker running on Google Kubernetes Engine. After it's installed, you can make it available to your users. The installation flow begins in the management console after you start the solution.

  • Manage Spinnaker

    Use this same management console to manage/operate your Spinnaker installation, including adding administrators, and creating accounts for deploying to additional GKE clusters or other providers.

    The management flow begins after you finish installing Spinnaker. You can also open it directly via a link from the GKE Applications page in the Google Cloud Console.

The management console uses Cloud Shell, with instructions shown in a guide on the right-hand side of the window. The guide shows the commands that will be run, and you can click those commands to copy them into Cloud Shell and run them there.

What is Cloud Shell?

Cloud Shell is a tool in Google Cloud Platform that provides command-line access to GCP.

How do I find and restore the instructions?

  • If the instructions in the right-hand pane disappear, just enter the following command in Cloud Shell:
cloudshell launch-tutorial ~/spinnaker-for-gcp/scripts/install/provision-spinnaker.md
  • If you need to find your way back to the management console, you can relaunch it by following the instructions under Install Spinnaker on Google Cloud Platform.

  • Refer back to this document if you get lost.

Am I billed for this?

You are billed for Google Cloud Platform resources that are installed as part of Spinnaker for Google Cloud Platform.

...and possibly other resources, depending on the options you select when you install and configure Spinnaker. You can use the Google Cloud Platform Pricing Calculator to estimate the cost of this solution.

Learn more about Google Cloud pricing & free trial.

Install and use Spinnaker on Google Cloud Platform

You access this solution by clicking the Go to Spinnaker for Google Cloud Platform button on the Spinnaker for GCP page in Marketplace.

After you've installed Spinnaker for Google Cloud Platform, you can access Spinnaker and the management console from Google Cloud Console.

Note: Spinnaker for Google Cloud Platform doesn't support regional clusters. If you intend to install Spinnaker on an existing cluster, it must be zonal.

Note: Google recommends that you deploy your resources using an account other than spinnaker-install-account. That account is used to install your spinnaker instance, and resources deployed using that account are installed into the spinnaker namespace by default. This namespace is not indexed, so your deployments will time out before they are deemed stable.

Install Spinnaker on Google Cloud Platform

  1. Start the solution from the Spinnaker for GCP Marketplace page by clicking the Go to Spinnaker for Google Cloud Platform button.

  2. When prompted to Open in Cloud Shell, click Proceed.

Cloud Shell opens, along with a file tree showing the files in the Spinnaker repository, and instructions.

The management console

Important: If you've launched the management console at least once before, you might be prompted, in the shell, to resume with the clone you created before, update that clone, or clone a new copy of the repository. The first option is best (cd into the existing directory). Don't clone a new copy.

The spinnaker-for-gcp repository is cloned into your Cloud Shell.

  1. Follow the instructions shown on the screen.

The flow in the management console guides you through the installation process, presenting you with commands, which you can copy to the Cloud Shell prompt and then execute by pressing Enter. The commands run scripts that automate the process of installing Spinnaker on GKE.

If the instruction pane disappears at any time, you can restore it using the following command, from Cloud Shell:

cloudshell launch-tutorial ~/spinnaker-for-gcp/scripts/install/provision-spinnaker.md

Access Spinnaker

After you've installed Spinnaker, you can execute a command to forward ports, which allows you to access the Deck UI and start using Spinnaker. You can share the port-forwarding command with your users, and if they have access to the GKE cluster, they can reach Deck (the Spinnaker UI) on port 8080.

Alternatively, you can expose Spinnaker over the public internet, secured using Identity-Aware Proxy.

Both alternatives are described below.

Access Spinnaker by forwarding ports

You can run a command in Cloud Shell in the management console, to forward ports so you can access Spinnaker from localhost:8080.

  1. Click to copy the connect_unsecured.sh command in the management console, and press Enter.

This forwards the local port 8080 to port 9000 (the port Deck uses) on the pod running Deck.

  1. Click the "Connect to Spinnaker…" link. This highlights the Preview button.

  2. Click the highlighted preview button, and select Preview on port 8080.

Click to preview on port 8080

**Note:**There is a "Connect to Spinnaker" link displayed. If you click it, it highlights the preview button, which you then click to select the port.

Deck, the Spinnaker user interface, opens in your browser. The Spinnaker documentation site has instructions for using Spinnaker.

Back in the management console, there are a few other things you can do:

  • Make Spinnaker securely available to your teams without having to forward ports
  • View the Spinnaker audit log
  • View logs from Spinnaker microservices
  • Click Next to move on to the Spinnaker management portion of the solution.
  • Share the port-forwarding command with your users If they have access to the GKE cluster, they can reach Deck (the Spinnaker UI) on port 8080.

Give your users access to Spinnaker over the internet

The console includes a command that helps you create a secure endpoint from which to expose Spinnaker to your users, securely, over the internet.

Note: If you need to keep Spinnaker private, you can set up port forwarding for your users.

  1. Navigate to step 2 of the installation flow in the Management console ("Connect to Spinnaker").

  2. Under "Expose Spinnaker publicly," click the button to copy the command to the command line, and press Enter.

The script creates a new endpoint from which to serve your Spinnaker instance. After the script finishes, the guidance in the console changes to show instructions for setting up OAuth so that your users can access this endpoint.

  1. Follow those on-screen instructions.

Make sure when you create your OAuth credentials that you copy the generated client ID and secret. You'll need to provide them when prompted by the script.

Note: This process can take up to an hour, even if it appears that the script has finished.

You now have a Spinnaker endpoint that you can share with your users, who authenticate into it using OAuth2. A link to Spinnaker is displayed in the management console. There is also a link on the GKE applications page for this Spinnaker instance.

Manage Spinnaker

Use the management console to manage your spinnaker instance, including the following actions:

  • Add administrators (operators)

  • Add cloud provider accounts

    A provider is the cloud environment (for example, Google Compute Engine) where you deploy your applications

  • Upgrade Spinnaker

  • Invoke Halyard commands to configure Spinnaker

  • Invoke spin commands to manage Spinnaker resources, like applications and pipelines

  1. Access the management portion of this console.

    Use one of the following options:

    If the console is already open:

    1. At the end of the installation flow, click Next.

    2. Copy the command on the Next steps page and press Enter.

      The instructions pane changes to start the management process.

      Start managing Spinnaker from within the console

    If the console is not already open:

    1. Go to the Google Kubernetes Engine applications page.

    2. Open the Spinnaker application.

      The application description includes a link: Open Management Environment in Cloud Shell.

    3. Click that link to open the management console, which now starts with the management/admin functionality.

      Start managing Spinnaker from the GKE Applications page

    4. Select your GCP project, and click Start.

Add administrators for your Spinnaker instance

You can give access to more operators, who can then use the management console.

  1. On the IAM permissions page, grant the person the 'Owner' role on the GCP project where you've installed Spinnaker.

  2. If you are serving Spinnaker on an IAP-secured endpoint, and if the person to whom you're giving operator rights doesn't already have user access, use the following command (which is also on step 5 of the management part of the console):

    ~/spinnaker-for-gcp/scripts/manage/grant_iap_access.sh

    ...and follow the instructions on the Cloud Shell command line.

Add cloud provider accounts

You can use the management console to add accounts for as many cloud providers as Spinnaker supports. You'll need one for each cloud on which your users intend to deploy applications. For example, if they will deploy applications to Google Compute Engine and AWS, you'll add a provider account for each.

The management console includes the following command, for adding a GKE account:

~/spinnaker-for-gcp/scripts/manage/add_gke_account.sh

And for Google Compute Engine:

~/spinnaker-for-gcp/scripts/manage/add_gce_account.sh

And for Google App Engine:

~/spinnaker-for-gcp/scripts/manage/add_gae_account.sh

You can run these commands from the management console or enter them in Cloud Shell against an existing Spinnaker instance.

Run Halyard commands

You can invoke any hal command to configure and administer your Spinnaker installation.

To do so, just invoke the command from the Cloud Shell in the management console, after you've installed Spinnaker

Upgrade Spinnaker

  1. Find out the version you want to upgrade to.

    The Versions page lists the stable versions available.

  2. In the console, navigate to the management flow:

    ~/spinnaker-for-gcp/scripts/manage/update_console.sh

  3. Click Next until you see the screen titled "Scripts for Common Commands."

  4. Under "Upgrade Spinnaker," copy the first command to the shell, and press Enter.

    That command is...

    cloudshell edit \
     ~/spinnaker-for-gcp/scripts/install/properties
  5. Edit the Spinnaker version in the properties file that is displayed.

    export SPINNAKER_VERSION=1.19.3

    The Spinnaker Versions page shows the latest versions avaiable.

  6. Use the following command to invoke Halyard to apply the changes:

    ~/spinnaker-for-gcp/scripts/manage/update_spinnaker_version.sh

Restart the management console

If you need to restart the console for any reason (for example, you closed the tab or window), you can restart it in the same way that you started it. You can also launch it from the GKE Applications page in the Google Cloud Console, if you've previously installed Spinnaker for Google Cloud Platform.

When you restart the console, it prompts you to resume from where you left off, if you want.

Upgrade the management console

  1. In the management console, navigate to step 3, "Scripts for Common Command," and scroll to the bottom of the page.

  2. Run the command shown under "Upgrade Management Environment."

The management console is upgraded to include the latest changes.

Remove Spinnaker for Google Cloud Platform

Warning: If you installed Spinnaker on pre-existing infrastructure (GKE cluster, Redis, service accounts), this script deletes those items. If you want to keep them, edit the generated cleanup script (~/spinnaker-for-gcp/scripts/manage/generate_deletion_script.sh) to comment out the specific deletion commands for items you want to keep.

If you want to remove Spinnaker for any reason:

  1. Open the management console and click Next until you get to the "Delete Spinnaker" page.

  2. Copy the command to the Cloud Shell terminal, and press Enter.

All resources that were created for this Spinnaker instance, and any existing resources on which you might have deployed, are deleted.

Sample Applications

The Spinnaker for Google Cloud Platform solution comes with sample applications to help you get started with Spinnaker.

To install them:

  1. In the management console, click Next until you get to the step titled "Use Spinnaker."

  2. Under Install sample applications and pipelines, click the button to paste the command, and press Enter.

    Cloud Shell returns a list of available sample apps, numbered.

  3. Press the number corresponding to the application you want, or the number corresponding to "Quit" to exit without installing any.

  4. Press Enter

    The tutorial pane now displays guidance for the sample application.

  5. To exit the sample app and return to the management portion of the console, click Start and then Next, then scroll to the bottom of the "Start a new build" page, and run the command under "Return to Spinnaker console."

Other considerations

Spinnaker for GCP architecture

Spinnaker and its microservices are installed on GKE using the following architecture:

Architecture of Spinnaker on GCP

Install Spinnaker on an existing cluster

You can install your Spinnaker instance or instances on pre-existing infrastructure, instead of having this solution create it new.

The cluster must have the following:

  • IP aliases enabled, because this uses a hosted Redis instance

  • Full Cloud Platform scope for its nodes if you're using the project default service account

Before you run the installation script, do the following:

  1. Copy and run the following command (which is also available in step 1 of the installation flow):

    cloudshell edit \
        ~/spinnaker-for-gcp/scripts/install/properties

    The properties file is opened in the file editor.

  2. Edit this section of the properties file to identify the Kubernetes cluster on which to install Spinnaker:

    # If cluster does not exist, it will be created.
    export GKE_CLUSTER=$DEPLOYMENT_NAME
    export ZONE=us-west1-b
    export REGION=us-west1
  3. Similarly, edit other properties to identify other existing infrastructure and accounts that you want to use, if applicable.

    For example an existing Cloud Memorystore Redis instance, or a bucket or a service account. In each case, if the infrastructure doesn't exist, the installation script creates it for you.

Manage multiple Spinnaker installations

If you run multiple Spinnaker instances, they must be on separate clusters, and therefore in different Kubernetes contexts.

Important: If you're trying to install multiple Spinnaker instances, don't clone multiple copies of the spinnaker-for-gcp repo.

To manage one of those installations:

  1. Get your credentials.

    gcloud container get-credentials
  2. Switch to the appropriate Kubernetes context.

    kubectl config use-context <CONTEXT_NAME>
  3. Pull the configuration stored in that cluster.

    ~/spinnaker-for-gcp/scripts/manage/pull_config.sh

The config now in ~/spinnaker-for-gcp/scripts/install/properties is the one for that Spinnaker instance. Perform the usual management tasks available to you, including running hal commands. Spinnaker applies those commands to the Spinnaker instance in the chosen context.

spinnaker-for-gcp's People

Contributors

afirth avatar akubi0w1 avatar bowmanrogere avatar bsda avatar csaroka avatar demobox avatar dorbin avatar eoincarroll avatar henrybell avatar jaksky avatar jmymy avatar kinggreedy avatar louisjimenez avatar manhof avatar mn7z avatar plumpy avatar sergeykanzhelev avatar skalle avatar stewchen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spinnaker-for-gcp's Issues

Evaluate using Kapitan? https://github.com/deepmind/kapitan

Hello,

I am using this repo to setup my test Spinnaker and it works great.

However I was going to suggest using Kapitan to generate the scripts and manage the deployment. You would be able to simplify lots of your logic and you would be able to support multiple deployment out of the box.

contact us on the #kapitan channel on Kubernetes slack if you need help.

Adding a new pubsub subscription

I'm trying to add a new pubsub subscription to Spinnaker for GCP running the following command:

hal config pubsub google subscription add spinnaker-topic-1 --subscription-name spinnaker-topic-1 --project --message-format CUSTOM

After that I ran push_and_apply.sh

However I don't see the subscription available to work as a trigger in Spinnaker console. Am I missing something?

Cluster creation failure

When setting up the spinnaker instance using the instructions here,

https://cloud.google.com/docs/ci-cd/spinnaker/spinnaker-for-gcp

getting the following error,

"Status details
Retry budget exhausted (5 attempts): The network "mynetwork" does not have available private IP space in 10.0.0.0/8 to reserve a /14 block for pods for cluster {us-west1-b, 620711858815, mynetwork, spinnaker-1, "

Do I need to provide the "--cluster-ipv4-cidr" flag upon GKE cluster creation? If so, which script should I make this change?

Deploying Cloud Audit Log failing

Setup.sh is failing to deploy Cloud function on new projects with several errors:

Deploying function (may take a while - up to 2 minutes)...WARNING: Setting IAM policy failed, try "gcloud alpha functions add-iam-policy-binding spinnaker1AuditLog --region=us-east1 --member=allUsers --role=roles/cloudfunctions.invoker"

or

ERROR: (gcloud.functions.deploy) OperationError: code=7, message=Missing necessary permission resourcemanager.projects.getIamPolicy for serviceAccount:[email protected]
.gserviceaccount.com on project xxxxxxxxx

Clouddriver http_proxy

Hi Team,

Is it possible to send http_proxy configurations for clouddriver. We are trying to connect to github for artefacts, and see that the requests are not going through proxy. Just to test we enabled NAT and its working with the NAT. In our environment we will not be using NAT, so would need a way to add proxy settings to clouddriver. We already tried this configuration, but not working https://www.spinnaker.io/setup/quickstart/faq/#i-want-to-run-a-spinnaker-service-clouddriver-echo-etc-behind-an-http-proxy-server

getting the below error, when i set JAVA_OPTS in clouddriver.yml

2019-11-11 00:22:56.303 WARN 1 --- [ main] c.n.s.c.a.gcs.GcsArtifactConfiguration : Failure instantiating gcs artifact account GcsArtifactAccount(name=gcs-install-account, jsonPath=null):

java.io.IOException: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
at com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentials(DefaultCredentialsProvider.java:134) ~[google-auth-library-oauth2-http-0.18.0.jar:na]
at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:113) ~[google-auth-library-oauth2-http-0.18.0.jar:na]
at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:85) ~[google-auth-library-oauth2-http-0.18.0.jar:na]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactCredentials.(GcsArtifactCredentials.java:61) ~[clouddriver-artifacts.jar:na]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration.lambda$gcsArtifactCredentials$0(GcsArtifactConfiguration.java:47) ~[clouddriver-artifacts.jar:na]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[na:1.8.0_212]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[na:1.8.0_212]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[na:1.8.0_212]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[na:1.8.0_212]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[na:1.8.0_212]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:1.8.0_212]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[na:1.8.0_212]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration.gcsArtifactCredentials(GcsArtifactConfiguration.java:54) ~[clouddriver-artifacts.jar:na]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration$$EnhancerBySpringCGLIB$$f63aa27f.CGLIB$gcsArtifactCredentials$0() ~[clouddriver-artifacts.jar:na]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration$$EnhancerBySpringCGLIB$$f63aa27f$$FastClassBySpringCGLIB$$b08ac6c1.invoke() ~[clouddriver-artifacts.jar:na]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) ~[spring-context-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration$$EnhancerBySpringCGLIB$$f63aa27f.gcsArtifactCredentials() ~[clouddriver-artifacts.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_212]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_212]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:622) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:607) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1321) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1160) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:277) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1467) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1431) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1322) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1209) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1171) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:857) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:760) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:218) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1341) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1187) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:277) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1251) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1171) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:857) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:760) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:218) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1341) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1187) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:845) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) ~[spring-context-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:743) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:390) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:140) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.builder.SpringApplicationBuilder$run$0.call(Unknown Source) ~[na:na]
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-2.5.7.jar:2.5.7]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115) ~[groovy-2.5.7.jar:2.5.7]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:127) ~[groovy-2.5.7.jar:2.5.7]
at com.netflix.spinnaker.clouddriver.Main.main(Main.groovy:78) ~[clouddriver-web.jar:na]

SERVICE_ACCOUNT_NAME -- must match displayName vs member name

Some background: we are creating the kubernetes cluster, and a few other things (network, service account, etc) via terraform.

Then I'm running the setup scripts from cloud shell.

the setup script gets the iam user via:

SA_EMAIL=$(gcloud iam service-accounts --project $PROJECT_ID list \
 --filter="displayName:$SERVICE_ACCOUNT_NAME" \
--format='value(email)')

the iap script however:

gcloud iam service-accounts keys create ~/.spin/key.json \
--iam-account $SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com \

So unless your service account is created a very specific way, one script is going to bomb out.

The setup script creates it as:

 gcloud iam service-accounts --project $PROJECT_ID create \
    $SERVICE_ACCOUNT_NAME \
    --display-name $SERVICE_ACCOUNT_NAME

Which is why the project functions correctly.

Getting error in creating SSL Certificate

Upon execution of ~/spinnaker-for-gcp/scripts/expose/configure_endpoint.sh

Below error is observed:

.  Using existing static IP address standard-pvp-1-external-ip (<projectId>)... 
.  Using existing service endpoint standard-pvp-1.endpoints.<projectId>.cloud.goog... 
.  Creating managed SSL certificate standard-pvp-1-managed-cert for domain standard-pvp-1.endpoints.<projectId>.cloud.goog... 
ERROR: (gcloud.beta.compute.ssl-certificates.create) Underspecified resource [standard-pvp-1-managed-cert]. Specify one of the [--global, --region]flags.

Got past it by creating ssl certificate using gcloud command, manually

gcloud beta compute ssl-certificates create standard-pvp-1-managed-cert --domains standard-pvp-1.endpoints.<projectId>.cloud.goog --global

Documentation on audit log cloud function

The spinnakerAuditLog cloud function looks great but I can't figure out how to view the logs. I ended up adding a console.log right above the log.write(entry); line which works fine, but feels like it defeats the purpose of using the @google-cloud/logging module.

Some documentation around how to view the logs would be great!

Thanks!

Support HA services (echo and clouddriver)

If running the deploy_application_manifest.sh script after enabling HA services on echo and clouddriver, it throws a warning and fails to label all spinnaker resources.

I don't know if it would be worth it to include something in the properties file and have the scripts configure HA automatically if enabled.

setup.sh does not handle region gke cluster

If the GKE is configured as a regional cluster, the setup script does not handle it. This is because gcloud container clusters get-credentials $GKE_CLUSTER --zone $ZONE --project $PROJECT_ID is use --zone instead of --region. This also happen at one of the functions in cluster_utils. Also the push_config.sh only check ZONE but ignore REGION. Ideally these scripts should handle both region and zone to cover all cases.

Spinnaker deployment in GCP Marketplace don't expose publicly

Hi everyone!

Regarding the spinnaker solution in marketplace from GCP, in the step related to the manner to access the spinnaker console, the port forwarding option is working as well as the documentation mentioned except by the link "Connect to Spinnaker via 'Preview on port 8080'", this link doesn't redirect and a warning message is shown : The element cannot be found either because it is still loading or doesn't exist. Ensure previous steps have been completed.

For the second option, expose publicly using an external IP and an endpoint, seems like the script:

~/spinnaker-for-gcp/scripts/expose/configure_endpoint.sh
Does not create the resources needed to expose this, following step by step only the external IP creation is done then following configuration can't be completed even when the output says done.

Adding docker registry

I'm having issues attempting to add a private GCR docker registry to spinnaker. I'm attempting to follow the instructions provided by spinnaker here: [https://www.spinnaker.io/setup/install/providers/docker-registry/#google-container-registry]. I've placed the json key file in my ~/.hal/default/credentials directory, and I'm able to successfully add the registry to the config locally, however when I run the push_and_apply.sh script, I receive the following error:

Problems in default.provider.dockerRegistry.prod-registry:
! ERROR Cannot find provided path:
/home/user1/.hal/default/credentials/gcr-account.json (No such file or
directory).: /home/user1/.hal/default/credentials/gcr-account.json (No such
file or directory).

I am adding the docker registry account to the local hal config with the following command:

~/hal/hal config provider docker-registry account add prod-registry
--address gcr.io
--username _json_key
--password-file ~/.hal/default/credentials/gcr-account.json

This file definitely exists in this path, and I was able to successfully add a gke account using the provided script. Any ideas as to why I receive this error?

Offer script to get Spinnaker logs

Hi.

Would you be interested in a PR to implement a script that wraps gcloud logging read to get the logs from the various Spinnaker services? I have one in Python and I'd be happy to contribute it if this is something you'd like in principle.

I created it because I found it cumbersome to use the Stackdriver web UI when debugging issues with Spinnaker:

  • It can only output a maximum of 300 logs.
  • It doesn't honour the sorting preference when you download the logs as JSON.
  • I end up downloading files many times so I can pipe them to a bunch of things, like jq or pbcopy.

The script I wrote supports:

  • Start and end date cut-offs.
  • The ability to filter logs to a specific service.
  • The ability to add conditions (for example, the pipeline execution id).

In the meantime, here's the current version in case anyone finds it useful:

#!/usr/bin/env python3

import argparse
from datetime import datetime, timedelta
import subprocess

# Note: the constants below will be taken from pre-existing env vars 
GCP_PROJECT = '<YOUR PROJECT>'
KUBERNETES_CLUSTER = 'spinnaker-1'
KUBERNETES_NAMESPACE = 'spinnaker'

BASE_CONDITIONS = [
  'resource.type="k8s_container"',
  'resource.labels.cluster_name="%s"' % KUBERNETES_CLUSTER,
  'resource.labels.namespace_name="%s"' % KUBERNETES_NAMESPACE,
]

BASE_COMMAND = [
  'gcloud',
  'logging',
  'read',
  '--project=%s' % GCP_PROJECT,
  '--format=json',
  '--order=asc',
]

NOW = datetime.now()
DEFAULT_START_DATE = NOW - timedelta(days=1)

def utc_format_datetime(datetime):
  return datetime.strftime('%Y-%m-%dT%H:%M:%SZ')

def main(args):
  conditions = BASE_CONDITIONS + args.conditions + [
    'timestamp>="%s"' % args.start_date,
    'timestamp<="%s"' % args.end_date,
  ]

  if args.service:
    conditions.append('resource.labels.container_name="%s"' % args.service)

  filter = ' AND '.join(conditions)
  command = BASE_COMMAND + [filter]
  subprocess.check_call(command)

parser = argparse.ArgumentParser(description='Get Stackdriver logs for Spinnaker')
parser.add_argument('--start-date', dest='start_date', default=utc_format_datetime(DEFAULT_START_DATE))
parser.add_argument('--end-date', dest='end_date', default=utc_format_datetime(NOW))
parser.add_argument('--service', dest='service', default=None)
parser.add_argument('conditions', nargs='*', default=[])
args = parser.parse_args()

main(args)

setup and manage multiple spinnaker for multiple projects

So far spinnaker-for-gcp assumes it is at $HOME/spinnaker-for-gcp directory. This does not make easy for managing multiple projects. Ideally the setup.sh, setup_properties.sh should be able to resolve to script's running directory. This way, users can put this repo in any directory they like rather than $HOME/spinnaker-for-gcp.

function getScriptDir() {
# so this script won't be affect by CDPATH variable
unset CDPATH
SOURCE="${BASH_SOURCE[0]}"
# resolve $SOURCE until the file is no longer a symlink
while [ -h "$SOURCE" ]; do
    DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
    SOURCE="$(readlink "$SOURCE")"
    # if $SOURCE was a relative symlink, we need to resolve it relative to the
    # path where the symlink file was located
    [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE"
done
echo "$( cd -P "$( dirname "$SOURCE" )" && pwd )"

}

SCRIPT_DIR=$(getScriptDir)/..

The above snippet can resolve to running script's directory (it can also follow the link). Maybe consider put that into bash scripts?

Statefulset fails to create with k8s 1.14

kdss spin-halyard
[...]
  Warning  FailedCreatePodSandBox  4m29s (x118 over 29m)  kubelet, gke-spinnaker-default-pool-76003395-r35q  Failed create pod sandbox: rpc error: code = Unknown desc = failed to make sandbox docker config for pod "spin-halyard-0": runAsGroup is specified without a runAsUser.

PR incoming

Workload identity not working with spinnaker

spinnaker/spinnaker#4838

Just an FYI -- we were interested in the cleaner management of service accounts using workload identity, but spinnaker (installed via spinnaker-for-gcp) has a couple of components that don't seem to enjoy working along with it.

It might be me doing something silly, but it's working with non-spinnaker pods.

Changing the halyard version through install_hal.sh does not update the StatefulSet

Issue Summary

Using the /cli/install_hal.sh script to set the halyard version updated the hal version in cloudshell but left the version used by the StatefulSet unchanged.

Steps to reproduce

  1. Run through setup.sh
  2. Run ~/spinnaker-for-gcp/scripts/cli/install_hal.sh --version 1.21.1
  3. Run ~/spinnaker-for-gcp/scripts/manage/push_and_apply.sh
  4. hal --version will return 1.21.1-20190624135101
  5. kubectl exec -it spin-halyard-0 -n halyard -- bash
  6. hal --version will return 1.22.1-20190724172712

Logs

Log after running install_hal.sh (dependencies truncated)

/spinnaker-for-gcp/scripts/install ([projectname])$ ~/spinnaker-for-gcp/scripts/cli/install_hal.sh --version 1.21.1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10637  100 10637    0     0  40328      0 --:--:-- --:--:-- --:--:-- 40291
user
non-interactive
version
Halyard version will be 1.21.1 
Halyard will be downloaded from gs://spinnaker-artifacts/halyard 
Halyard config will come from bucket gs://halconfig 
Halconfig will be stored at /home/myuser/.hal/config
Uninstall script is located at /home/myuser/.hal/uninstall.sh
Java is already installed & at the right version
/home/myuser/spinnaker-for-gcp/scripts/install/installhalyard.6396 /home/myuser/spinnaker-for-gcp/scripts/install
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  259M  100  259M    0     0  42.5M      0  0:00:06  0:00:06 --:--:-- 40.8M
halyard/
halyard/config/
halyard/config/halyard.yml
halyard/bin/
halyard/bin/hal.bat
halyard/bin/halyard.bat
halyard/bin/hal
halyard/bin/halyard
halyard/lib/
halyard/lib/json-simple-1.1.jar
halyard/lib/netty-buffer-4.1.34.Final.jar
halyard/lib/netty-transport-4.1.34.Final.jar
halyard/lib/google-api-services-cloudkms-v1-rev8-1.22.0.jar
halyard/lib/maven-builder-support-3.3.9.jar
halyard/lib/maven-repository-metadata-3.3.9.jar
halyard/lib/aws-java-sdk-iot1clickdevices-1.11.534.jar
halyard/lib/spectator-ext-jvm-0.75.0.jar
halyard/lib/groovy-docgenerator-2.5.6.jar
halyard/lib/netty-codec-http-4.1.34.Final.jar
halyard/lib/jsoup-1.8.1.jar
halyard/lib/hk2-utils-2.5.0-b42.jar
halyard/lib/aws-java-sdk-servermigration-1.11.534.jar
halyard/lib/aws-java-sdk-emr-1.11.534.jar
...
hal
update-halyard
groupadd: group 'halyard' already exists
groupadd: group 'spinnaker' already exists
/home/myuser/spinnaker-for-gcp/scripts/install
Welcome to Cloud Shell! Type "help" to get started.
To set your Cloud Platform project in this session use “gcloud config set project [PROJECT_ID]”
The halyard daemon isn't running yet... starting it manually......
1.21.1-20190624135101

Bash auto-completion configured.
To use the auto-completion either restart your shell, or run
. /home/myuser/.bashrc

spin-halyard StatefulSet YAML

image: gcr.io/spinnaker-marketplace/halyard:1.22.1
        imagePullPolicy: Always
        name: halyard-daemon

Spinnaker and Istio

Hi,

I am currently trying to set up Spinnaker on GCP using spinnaker-for-gcp. I have been trying to get a somewhat best practice Istio service mesh running on the GKE cluster as well. I have mTls enabled, and I can expose some of my telemetry plugins from the ingress. I was able to use spinnaker-for-gcp prior to enabling mTls. One of the issues I have identified is the call to Redis from the spin-clouddriver. I have tried getting the Redis service to be accessible by adding the host and port as a ServiceEntry. So far, I haven't had any luck though. I have attached some screenshots of the logs and my ServiceEntry. Any help would be appreciated.

image

image

Fail early in scripts

Scripts may fail because of many reasons.
Try to fail as soon as possible and do not assume that everything is fine.

Suggestion: add
set -e
to all scripts where this is missing.

Example:

$ spinnaker-for-gcp/scripts/cli/install_hal.sh
Uninstall script is located at /home/florath/.hal/uninstall.sh
Distribution Debian GNU/Linux 10 (buster) is not supported yet - please file an issue
  https://github.com/spinnaker/halyard/issues
mv: cannot stat '/etc/bash_completion.d/hal': No such file or directory
[...]

The script (and also parents) should immediately abort instead of passing on and failing at a later stage (which makes debugging somewhat complex).

Multi-instance setup

What is the proper way to manage multiple Spinnaker instances (one for testing and one for production)? As far as I can tell I have only one Cloud Shell home directory, so I created two separate folders, each containing its own spinnaker-for-gcp folder, to keep the properties file seperate. And I think I need to run scripts/manage/pull_config.sh every time I want to switch instances (which overwrites the ~/.hal folder with the configs from the instance to manage, as I understand it).

Is this the right way to do this? It is a bit annoying in that all the tutorials assume that spinnaker-for-gcp is directly under home, so all the commands you can copy from the tutorials start with ~/spinnaker-for-gcp/

Push and apply fails with webhook trustStore

When adding a webhook trust store, push_and_apply.sh fails when applying the configuration with the following error:

Validation in Global:
! ERROR Failed to backup user file:
/home/bowmanrogere/.hal/default/credentials/jvmracert.jks

  • Failed to deploy Spinnaker.

It appears as though the part of push_config.sh that replaces $USER with spinnaker does not pick up this line.

Infrastructure tab in spinnaker pipeline is not populating data instantly

On deploying new replica set via kubernetes v2 using highlander strategy or any other strategy , infrastructure tab doesnt pick up the deployment status once done instantly and takes like 10-15 mins to pop up or something it just disappers totally,

We are stuck with this and trying to resolve it.,

Handle fiat "File" role provider

Currently testing spinnaker for GCP, I'm facing an issue with the fiat "File" role provider.

This provider need a path to a file describing each roles, I can update the role provider config.
But when I try to push it the path is not rewritten with the halyard user.

Current config in .hal/config :

    authz:
      groupMembership:
        service: EXTERNAL
        google:
          roleProviderType: GOOGLE
        github:
          roleProviderType: GITHUB
        file:
          roleProviderType: FILE
          path: /home/my_user/.hal/default/credentials/fiat-file.yaml
        ldap:
          roleProviderType: LDAP
      enabled: true

Error durring push_and_apply.sh

Problems in Global:
! ERROR Failed to backup user file:
  /home/my_user/.hal/default/credentials/fiat-file.yaml
- Failed to deploy Spinnaker.

I'm currently using a workaround with manual editing of .hal/config to :

    authz:
      groupMembership:
        service: EXTERNAL
        google:
          roleProviderType: GOOGLE
        github:
          roleProviderType: GITHUB
        file:
          roleProviderType: FILE
          path: /home/spinnaker/.hal/default/credentials/fiat-file.yaml
        ldap:
          roleProviderType: LDAP
      enabled: true

Error output when trying to expose Spinnaker instance via IAP

I am provisioning Spinnaker for a new project and got the following error:

$ ~/spinnaker-for-gcp/scripts/expose/configure_endpoint.sh
~/spinnaker-for-gcp/scripts ~/spinnaker-for-gcp/scripts/install
.  Creating static IP address spinnaker-external-ip... 
Created [https://www.googleapis.com/compute/v1/projects/domain-registry-dev/global/addresses/spinnaker-external-ip].
.  Creating service endpoint spinnaker.endpoints.domain-registry-dev.cloud.goog... 
Waiting for async operation operations/serviceConfigs.spinnaker.endpoints.domain-registry-dev.cloud.goog:abecc196-a1e4-430c-8687-9bde838ea1a4 to complete...
Operation finished successfully. The following command can describe the Operation details:
 gcloud endpoints operations describe operations/serviceConfigs.spinnaker.endpoints.domain-registry-dev.cloud.goog:abecc196-a1e4-430c-8687-9bde838ea1a4
                                                                                                   -dev.cloud.goog:abecc196-a1e4-430c-8687-9b
Waiting for async operation operations/rollouts.spinnaker.endpoints.domain-registry-dev.cloud.goog:1453c4aa-3827-4dba-bd7f-79cc1b5daa0f to complete...
Operation finished successfully. The following command can describe the Operation details:
 gcloud endpoints operations describe operations/rollouts.spinnaker.endpoints.domain-registry-dev.cloud.goog:1453c4aa-3827-4dba-bd7f-79cc1b5daa0f
                                                                                                   loud.goog:1453c4aa-3827-4dba-bd7f-79cc1b5d
Enabling service [endpoints.googleapis.com] on project [domain-registry-dev]...
ERROR: (gcloud.endpoints.services.deploy) INVALID_ARGUMENT: Invalid operation name operations/noop.DONE_OPERATION, refers to an already DONEoperation
.  Creating managed SSL certificate spinnaker-managed-cert for domain spinnaker.endpoints.domain-registry-dev.cloud.goog... 
Created [https://www.googleapis.com/compute/beta/projects/domain-registry-dev/global/sslCertificates/spinnaker-managed-cert].
NAME                    TYPE     CREATION_TIMESTAMP             EXPIRE_TIME  MANAGED_STATUS
spinnaker-managed-cert  MANAGED  2019-07-02T17:35:58.453-07:00               PROVISIONING
    spinnaker.endpoints.domain-registry-dev.cloud.goog: PROVISIONING
~/spinnaker-for-gcp/scripts ~/spinnaker-for-gcp/scripts
~/spinnaker-for-gcp/scripts
~/spinnaker-for-gcp/scripts/install

It took considerably less time than the 30-60 min advised in the tutorial. I then ran the same command again, I got the following:

$ ~/spinnaker-for-gcp/scripts/expose/configure_endpoint.sh
~/spinnaker-for-gcp/scripts ~/spinnaker-for-gcp/scripts/install
.  Using existing static IP address spinnaker-external-ip (35.227.196.142)... 
.  Using existing service endpoint spinnaker.endpoints.domain-registry-dev.cloud.goog... 
.  Using existing managed SSL certificate spinnaker-managed-cert... 
~/spinnaker-for-gcp/scripts ~/spinnaker-for-gcp/scripts
~/spinnaker-for-gcp/scripts
~/spinnaker-for-gcp/scripts/install

The output looks fine this time around. I suspect it could just be a red herring?

hal-deploy-apply job failure

Hello,
I am following the guide for installation on GCP and I keep encountering the same issue, the installation hangs when it gets to the hal-deploy-apply stage. I am not making any changes to the properties file. Here is the terminal output showing the error I get, followed by the endless series of dots (email address has been changed for privacy). I'm sure it's something obvious that I'm missing, but I can't pin down a succinct answer online regarding the error message. I'd appreciate any help you can give.

. Provisioning Spinnaker resources...
namespace/halyard created
namespace/spinnaker created
persistentvolumeclaim/halyard-pv-claim created
statefulset.apps/spin-halyard created
service/spin-halyard created
configmap/halconfig created
job.batch/hal-deploy-apply created
Error from server (Forbidden): error when creating "STDIN": clusterrolebindings.rbac.authorization.k8s.io is forbidden: User "[email protected]" cannot create resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope
Waiting on job hal-deploy-apply to complete...............................................................................................................................................................................................................

Unable to create applications

I spun this up in a quick test project, and went the IAP route.

It seems like mostly everything is happy (can log into the app, and click around) but trying to add an application dies:

image

The logs are a little angry:

front50 has a 403/insufficient permission error

I 2019-08-01T07:09:55.421362964Z 
 
I 2019-08-01T07:09:55.421703686Z 2019-08-01 07:09:55.421 ERROR 1 --- [0.0-8080-exec-5] c.n.s.f.model.application.Application    : Failed to perform action (name: TESTMIDNIGHT)
 
I 2019-08-01T07:09:55.457688600Z 2019-08-01 07:09:55.452 ERROR 1 --- [0.0-8080-exec-5] c.n.s.k.w.e.GenericExceptionHandlers     : Internal Server Error
 
I 2019-08-01T07:09:55.457754068Z 
 
I 2019-08-01T07:09:55.457758207Z java.lang.reflect.UndeclaredThrowableException: null
	at com.netflix.spinnaker.front50.controllers.v2.ApplicationsController$$EnhancerBySpringCGLIB$$cb0646c0.create(<generated>) ~[front50-web.jar:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_212]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_212]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:849) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:760) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) [spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55) [spring-boot-1.5.17.RELEASE.jar:1.5.17.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:111) [spring-boot-actuator-1.5.17.RELEASE.jar:1.5.17.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at com.netflix.spinnaker.fiat.shared.FiatAuthenticationFilter.doFilter(FiatAuthenticationFilter.java:46) [fiat-api-0.63.7.jar:0.63.7]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103) [spring-boot-actuator-1.5.17.RELEASE.jar:1.5.17.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at javax.servlet.FilterChain$doFilter.call(Unknown Source) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at com.netflix.spinnaker.front50.filters.SimpleCORSFilter.doFilter(SimpleCORSFilter.groovy:38) [front50-web.jar:na]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at javax.servlet.FilterChain$doFilter.call(Unknown Source) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at com.netflix.spinnaker.filters.AuthenticatedRequestFilter.doFilter(AuthenticatedRequestFilter.groovy:140) [kork-web-3.12.2.jar:3.12.2]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.34.jar:8.5.34]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
 
I 2019-08-01T07:09:55.458264972Z {
 
I 2019-08-01T07:09:55.458268830Z   "code" : 403,
 
I 2019-08-01T07:09:55.458271981Z   "errors" : [ {
 
I 2019-08-01T07:09:55.458274992Z     "domain" : "global",
 
I 2019-08-01T07:09:55.458278218Z     "message" : "Insufficient Permission",
 
I 2019-08-01T07:09:55.458281337Z     "reason" : "insufficientPermissions"
 
I 2019-08-01T07:09:55.458284492Z   } ],
 
I 2019-08-01T07:09:55.458287429Z   "message" : "Insufficient Permission"
 
I 2019-08-01T07:09:55.458290667Z }
 

Spin/Gate logs don't have much going on.

Any tips? Thanks in advance!

Deploying to existing cluster hal-deploy-apply error

Having an issue running on an existing cluster
Stuck on:
Waiting on job hal-deploy-apply to complete...

Looking at the logs, I'm seeing:
Failed to ensure the required bucket "spinnaker-plv5i7js7ndopdyyggqv-1564919957" exists:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
Insufficient Permission

This bucket doesn't exist and was not created by the setup script.

Can't find where this is set.

Tried cleaning up with a generated clean up script as well as deleting both spinnaker and halyard namespaces but the error is the same, with the same bucket name.

Thanks

Cloudshell link broken

  • Ubuntu 19/Chrome Version 77.0.3865.90
  • Navigate to kube/applications/spinnaker
  • Click link in description. "URL not found"
    Interestingly refreshing the page then works

Initial link:
https://console.cloud.google.com/cloudshell/editor?shellonly=true&cloudshell_git_repo=https:%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fspinnaker-for-gcp.git&cloudshell_working_dir=scripts%2Fmanage&cloudshell_tutorial=landing_page_expanded.md&cloudshell_print=instructions.txt

image

Thanks for putting this together though, as someone who's started setting up spinnaker a few times for greenfields and then decided I couldn't be bothered, this repo and all the doco is incredibly helpful

Upgrading the Spinnaker CLI - spin - breaks the config file

Steps to reproduce:
use IAP
follow the tutorial "manage console" step 2 - Included command-line tools
upgrade spinnaker CLI to the latest version - ~/spinnaker-for-gcp/scripts/cli/install_spin.sh

Expected behavior:
spinnaker CLI upgraded to the latest version

Observed behavior:
in: spin app list
out: Status: 403 , Body: {"error":"Forbidden","message":"Access Denied","status":403, }

Further investigations:
The spinnaker CLI installation file is creating an config file with local port forwarding.

gate:
endpoint: http://localhost:8080/gate

The correct config file is created when running the configure_iap script the first time, that branch of the code is never executed again unless the kubernetes secret is manually deleted.

Certificate mismatch when accessing spinnaker UI

After following the directions for exposing Spinnaker, I can log into the server using oauth. After some period of time (a few hours), I start getting certificate mismatches, and all logins give me the following error:

Untrusted SSL Server Certificate (ssl_server_cert_untrusted_issuer)

Your request contacted a host which presented a certificate signed by an untrusted issuer.
This is typically caused by a Web Site presenting an incorrect or invalid certificate, but could be because of a configuration error.
For assistance, contact your network support team.

Can no longer upgrade Spinnaker

I'm trying to upgrade from Spinnaker 1.16.0 to 1.16.2, but I keep getting the following error when I run ~/spinnaker-for-gcp/scripts/manage/update_spinnaker_version.sh:

- Apply deployment
  Failure
- Deploy spin-clouddriver
  Failure
- Deploy spin-front50
  Failure
- Deploy spin-orca
  Failure
- Deploy spin-deck
  Failure
- Deploy spin-echo
  Failure
- Deploy spin-gate
  Failure
- Deploy spin-igor
  Failure
- Deploy spin-kayenta
  Failure
- Deploy spin-rosco
  Failure
Problems in Global:
! ERROR Failed check for Namespace/spinnaker in null
Unable to connect to the server: error executing access token command
  "/google/google-cloud-sdk/bin/gcloud config config-helper --format=json":
  err=fork/exec /google/google-cloud-sdk/bin/gcloud: no such file or directory
  output= stderr=



- Failed to deploy Spinnaker.
command terminated with exit code 1

However, /google/google-cloud-sdk/bin/gcloud does exist and is executable.

And in case it's relevant:

$ grep VERSION ~/spinnaker-for-gcp/scripts/install/properties
export SPINNAKER_VERSION=1.16.2
export HALYARD_VERSION=1.23.2
export GKE_CLUSTER_VERSION=1.12.7

$ hal --version
1.23.2-20190904152725

I last used update_spinnaker_version.sh when I upgraded to v1.16.0.

Cloud Function fails to deploy in default region us-west1

The setup_properties.sh sets the default region of the properties file to us-west1. Deploying the audit log Cloud Function to this region fails with the error:
ERROR: (gcloud.functions.deploy) ResponseError: status=[403], code=[Forbidden], message=[Permission denied on 'locations/us-west1' (or it may not exist)]

The region is no longer in the list of supported regions for Cloud Function.

In Application details link to management console doesn't work

I've upgraded my Spinnaker instance in GKE from 1.17 to 1.19 and now I can't use `Open management console in Cloud Shell' link from Application info.
Error I'm getting:

URL not found
We couldn't find what you were looking for. Try one of the links below.

Google Developers Console Home
Google Cloud Platform
Google Developers

Link leeds to https://console.cloud.google.com/cloudshell/editor?shellonly=true&cloudshell_git_repo=https://github.com/GoogleCloudPlatform/spinnaker-for-gcp.git&cloudshell_working_dir=scripts/manage&cloudshell_tutorial=landing_page_expanded.md&cloudshell_print=instructions.txt

Spinnaker deployment in GCP Marketplace stuck

The deployment available in Marketplace can't successfully ends. I tried several times and in the most of the times the pods and jobs stay stuck in Pending creation, Does not have minimum availability - crashloopback, FailedCreatePodSandbox, PVC unbound.
e.g.

Failed create pod sandbox: rpc error: code = Unknown desc = Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 169.254.169.254:53: no such host

From the workload:

  • Pod spin-gate : Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetch
    -Pod spin-cloud driver: Could not get a resource from the pool at redis.clients.util.Pool.getResource(Pool.java:53) ~[jedis-2.9.3.jar:na] at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226) ~[jedis-2.9.3.jar:na] at
    -Pod spin-Kayenta : Error creating bean with name 'atlasQueueMonitor' defined in URL [jar:file:/opt/kayenta/lib/orca-queue-6.139.0.jar!/com/netflix/spinnaker/orca/q/metrics/AtlasQueueMonitor.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'queue' defined in class path resource
  • Pod Spin Ocra : Error creating bean with name 'defaultExecutionPromoter' defined in URL [jar:file:/opt/orca/lib/orca-qos.jar!/com/netflix/spinnaker/orca/qos/DefaultExecutionPromoter.class]: Unsatisfied dependency expressed through constructor parameter 0;
  • Pod Spin-ocra: redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
    at redis.clients.util.Pool.getResource(Pool.java:53) ~[jedis-2.9.0.jar:na]
    at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226) ~[jedis-2.9.0.jar:na]

Scaling up doesn't solve the issue, after several attempts to deploy the solution always goes to one status above listed

Can you help to know what is happening?

I have the hypothesis that the issue could be related to image version used but is just a but it's just a guess.

Additional in some cases the pod stuck in a restart loop due to the readiness probe failed

Readiness probe failed: wget can't connect to remote host (127.0.0.1) : connection refused

Any insight would be appreciate.

Add a clean up script

It would be nice to have a script that deletes all the resources created. That way if things doesn't work out it is easier to clean up and start from scratch (or just remove everything after a trial run).

Missing spin-fiat service?

When trying to setup spinnaker authorization fiat is a required service component, yet I can't see its instance exist

NAME READY STATUS RESTARTS AGE
spin-clouddriver-6644cf67bf-vx2sd 1/1 Running 0 21h
spin-deck-6456b4589b-6xlbx 1/1 Running 0 24h
spin-echo-849dc598bf-pg5m8 1/1 Running 0 21h
spin-front50-77c8db5fd6-8262k 1/1 Running 0 23h
spin-gate-855bb68d86-7fgxs 1/1 Running 0 21h
spin-igor-6d4f7f46df-xvmx2 1/1 Running 0 21h
spin-kayenta-56b686f8d6-69m9l 1/1 Running 0 21h
spin-orca-7886f9dd4f-9sngw 1/1 Running 0 21h
spin-rosco-c99589fd9-h6t4z 1/1 Running 0 21h

Although, search fiat from stackdriver, you can find some information regarding to it. So does fiat need to be enabled before running the setup.sh script?

Question: IAP Access

I have installed spinnaker-for-gcp and setup it up to be public facing with IAP. The question I have is currently I am the only user listed under IAP users. The Oauth portion of spinnaker allows anyone in my particular google domain. I noticed any user in my domain could log into spinnaker which makes since thats how the Oauth part is configured but I thought IAP would block access since I Am the only user listed. Is this correct ? I may be misunderstanding how the IAP and the spinnaker OAuth work together.

spin-fiat pod not deployed

It seems the installation process does not deploy fiat pod. It did not deploy fiat pod even after enabling the feature fiat in ~/.hal.config file.

Creating with a service account via cloud shell?

New to GCP so apologies if this is off-topic.

We would like to try this, but we currently install everything (and manage everything) via least-privilege service accounts.

Is it possible to run cloud shell as a service account, or are there any instructions on running this through gcloud without cloud shell?

Thanks!

Add check for hal version drift.

As individual operators can install different versions of the hal client in their cloud shell environments, the halyard daemon could end up in a state where the hal config is incompatible with the version of the hal client in use by a given operator.

We should add some logic to identify this situation and recommend to the operator that they upgrade their local hal client and/or the halyard version in use on the daemon.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.