googlecloudplatform / spinnaker-for-gcp Goto Github PK
View Code? Open in Web Editor NEWProduction-ready Spinnaker on GKE
Production-ready Spinnaker on GKE
If running the deploy_application_manifest.sh script after enabling HA services on echo and clouddriver, it throws a warning and fails to label all spinnaker resources.
I don't know if it would be worth it to include something in the properties file and have the scripts configure HA automatically if enabled.
Issue came in updating a pipeline or creating a new pipeline.
After digging into stackdriver logs, i found some authorization headers missing errors...
Any suggestions to fix it?
Hello,
I am following the guide for installation on GCP and I keep encountering the same issue, the installation hangs when it gets to the hal-deploy-apply stage. I am not making any changes to the properties file. Here is the terminal output showing the error I get, followed by the endless series of dots (email address has been changed for privacy). I'm sure it's something obvious that I'm missing, but I can't pin down a succinct answer online regarding the error message. I'd appreciate any help you can give.
. Provisioning Spinnaker resources...
namespace/halyard created
namespace/spinnaker created
persistentvolumeclaim/halyard-pv-claim created
statefulset.apps/spin-halyard created
service/spin-halyard created
configmap/halconfig created
job.batch/hal-deploy-apply created
Error from server (Forbidden): error when creating "STDIN": clusterrolebindings.rbac.authorization.k8s.io is forbidden: User "[email protected]" cannot create resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope
Waiting on job hal-deploy-apply to complete...............................................................................................................................................................................................................
Scripts may fail because of many reasons.
Try to fail as soon as possible and do not assume that everything is fine.
Suggestion: add
set -e
to all scripts where this is missing.
Example:
$ spinnaker-for-gcp/scripts/cli/install_hal.sh
Uninstall script is located at /home/florath/.hal/uninstall.sh
Distribution Debian GNU/Linux 10 (buster) is not supported yet - please file an issue
https://github.com/spinnaker/halyard/issues
mv: cannot stat '/etc/bash_completion.d/hal': No such file or directory
[...]
The script (and also parents) should immediately abort instead of passing on and failing at a later stage (which makes debugging somewhat complex).
The spinnakerAuditLog
cloud function looks great but I can't figure out how to view the logs. I ended up adding a console.log
right above the log.write(entry);
line which works fine, but feels like it defeats the purpose of using the @google-cloud/logging
module.
Some documentation around how to view the logs would be great!
Thanks!
When setting up the spinnaker instance using the instructions here,
https://cloud.google.com/docs/ci-cd/spinnaker/spinnaker-for-gcp
getting the following error,
"Status details
Retry budget exhausted (5 attempts): The network "mynetwork" does not have available private IP space in 10.0.0.0/8 to reserve a /14 block for pods for cluster {us-west1-b, 620711858815, mynetwork, spinnaker-1, "
Do I need to provide the "--cluster-ipv4-cidr" flag upon GKE cluster creation? If so, which script should I make this change?
I have installed spinnaker-for-gcp and setup it up to be public facing with IAP. The question I have is currently I am the only user listed under IAP users. The Oauth portion of spinnaker allows anyone in my particular google domain. I noticed any user in my domain could log into spinnaker which makes since thats how the Oauth part is configured but I thought IAP would block access since I Am the only user listed. Is this correct ? I may be misunderstanding how the IAP and the spinnaker OAuth work together.
What is the proper way to manage multiple Spinnaker instances (one for testing and one for production)? As far as I can tell I have only one Cloud Shell home directory, so I created two separate folders, each containing its own spinnaker-for-gcp
folder, to keep the properties
file seperate. And I think I need to run scripts/manage/pull_config.sh
every time I want to switch instances (which overwrites the ~/.hal
folder with the configs from the instance to manage, as I understand it).
Is this the right way to do this? It is a bit annoying in that all the tutorials assume that spinnaker-for-gcp
is directly under home, so all the commands you can copy from the tutorials start with ~/spinnaker-for-gcp/
kdss spin-halyard
[...]
Warning FailedCreatePodSandBox 4m29s (x118 over 29m) kubelet, gke-spinnaker-default-pool-76003395-r35q Failed create pod sandbox: rpc error: code = Unknown desc = failed to make sandbox docker config for pod "spin-halyard-0": runAsGroup is specified without a runAsUser.
PR incoming
I've upgraded my Spinnaker instance in GKE from 1.17 to 1.19 and now I can't use `Open management console in Cloud Shell' link from Application info.
Error I'm getting:
URL not found
We couldn't find what you were looking for. Try one of the links below.
Google Developers Console Home
Google Cloud Platform
Google Developers
Link leeds to https://console.cloud.google.com/cloudshell/editor?shellonly=true&cloudshell_git_repo=https://github.com/GoogleCloudPlatform/spinnaker-for-gcp.git&cloudshell_working_dir=scripts/manage&cloudshell_tutorial=landing_page_expanded.md&cloudshell_print=instructions.txt
When installing spinnaker for gcp, bucket and cloud function creation ignores $REGION and uses the default settings.
So far spinnaker-for-gcp assumes it is at $HOME/spinnaker-for-gcp directory. This does not make easy for managing multiple projects. Ideally the setup.sh, setup_properties.sh should be able to resolve to script's running directory. This way, users can put this repo in any directory they like rather than $HOME/spinnaker-for-gcp.
function getScriptDir() {
# so this script won't be affect by CDPATH variable
unset CDPATH
SOURCE="${BASH_SOURCE[0]}"
# resolve $SOURCE until the file is no longer a symlink
while [ -h "$SOURCE" ]; do
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
SOURCE="$(readlink "$SOURCE")"
# if $SOURCE was a relative symlink, we need to resolve it relative to the
# path where the symlink file was located
[[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE"
done
echo "$( cd -P "$( dirname "$SOURCE" )" && pwd )"
}
SCRIPT_DIR=$(getScriptDir)/..
The above snippet can resolve to running script's directory (it can also follow the link). Maybe consider put that into bash scripts?
Upon execution of ~/spinnaker-for-gcp/scripts/expose/configure_endpoint.sh
Below error is observed:
. Using existing static IP address standard-pvp-1-external-ip (<projectId>)...
. Using existing service endpoint standard-pvp-1.endpoints.<projectId>.cloud.goog...
. Creating managed SSL certificate standard-pvp-1-managed-cert for domain standard-pvp-1.endpoints.<projectId>.cloud.goog...
ERROR: (gcloud.beta.compute.ssl-certificates.create) Underspecified resource [standard-pvp-1-managed-cert]. Specify one of the [--global, --region]flags.
Got past it by creating ssl certificate using gcloud command, manually
gcloud beta compute ssl-certificates create standard-pvp-1-managed-cert --domains standard-pvp-1.endpoints.<projectId>.cloud.goog --global
Initial link:
https://console.cloud.google.com/cloudshell/editor?shellonly=true&cloudshell_git_repo=https:%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fspinnaker-for-gcp.git&cloudshell_working_dir=scripts%2Fmanage&cloudshell_tutorial=landing_page_expanded.md&cloudshell_print=instructions.txt
Thanks for putting this together though, as someone who's started setting up spinnaker a few times for greenfields and then decided I couldn't be bothered, this repo and all the doco is incredibly helpful
Hi.
Would you be interested in a PR to implement a script that wraps gcloud logging read
to get the logs from the various Spinnaker services? I have one in Python and I'd be happy to contribute it if this is something you'd like in principle.
I created it because I found it cumbersome to use the Stackdriver web UI when debugging issues with Spinnaker:
jq
or pbcopy
.The script I wrote supports:
In the meantime, here's the current version in case anyone finds it useful:
#!/usr/bin/env python3
import argparse
from datetime import datetime, timedelta
import subprocess
# Note: the constants below will be taken from pre-existing env vars
GCP_PROJECT = '<YOUR PROJECT>'
KUBERNETES_CLUSTER = 'spinnaker-1'
KUBERNETES_NAMESPACE = 'spinnaker'
BASE_CONDITIONS = [
'resource.type="k8s_container"',
'resource.labels.cluster_name="%s"' % KUBERNETES_CLUSTER,
'resource.labels.namespace_name="%s"' % KUBERNETES_NAMESPACE,
]
BASE_COMMAND = [
'gcloud',
'logging',
'read',
'--project=%s' % GCP_PROJECT,
'--format=json',
'--order=asc',
]
NOW = datetime.now()
DEFAULT_START_DATE = NOW - timedelta(days=1)
def utc_format_datetime(datetime):
return datetime.strftime('%Y-%m-%dT%H:%M:%SZ')
def main(args):
conditions = BASE_CONDITIONS + args.conditions + [
'timestamp>="%s"' % args.start_date,
'timestamp<="%s"' % args.end_date,
]
if args.service:
conditions.append('resource.labels.container_name="%s"' % args.service)
filter = ' AND '.join(conditions)
command = BASE_COMMAND + [filter]
subprocess.check_call(command)
parser = argparse.ArgumentParser(description='Get Stackdriver logs for Spinnaker')
parser.add_argument('--start-date', dest='start_date', default=utc_format_datetime(DEFAULT_START_DATE))
parser.add_argument('--end-date', dest='end_date', default=utc_format_datetime(NOW))
parser.add_argument('--service', dest='service', default=None)
parser.add_argument('conditions', nargs='*', default=[])
args = parser.parse_args()
main(args)
I'm trying to upgrade from Spinnaker 1.16.0 to 1.16.2, but I keep getting the following error when I run ~/spinnaker-for-gcp/scripts/manage/update_spinnaker_version.sh
:
- Apply deployment
Failure
- Deploy spin-clouddriver
Failure
- Deploy spin-front50
Failure
- Deploy spin-orca
Failure
- Deploy spin-deck
Failure
- Deploy spin-echo
Failure
- Deploy spin-gate
Failure
- Deploy spin-igor
Failure
- Deploy spin-kayenta
Failure
- Deploy spin-rosco
Failure
Problems in Global:
! ERROR Failed check for Namespace/spinnaker in null
Unable to connect to the server: error executing access token command
"/google/google-cloud-sdk/bin/gcloud config config-helper --format=json":
err=fork/exec /google/google-cloud-sdk/bin/gcloud: no such file or directory
output= stderr=
- Failed to deploy Spinnaker.
command terminated with exit code 1
However, /google/google-cloud-sdk/bin/gcloud
does exist and is executable.
And in case it's relevant:
$ grep VERSION ~/spinnaker-for-gcp/scripts/install/properties
export SPINNAKER_VERSION=1.16.2
export HALYARD_VERSION=1.23.2
export GKE_CLUSTER_VERSION=1.12.7
$ hal --version
1.23.2-20190904152725
I last used update_spinnaker_version.sh
when I upgraded to v1.16.0.
I'm having issues attempting to add a private GCR docker registry to spinnaker. I'm attempting to follow the instructions provided by spinnaker here: [https://www.spinnaker.io/setup/install/providers/docker-registry/#google-container-registry]. I've placed the json key file in my ~/.hal/default/credentials directory, and I'm able to successfully add the registry to the config locally, however when I run the push_and_apply.sh script, I receive the following error:
Problems in default.provider.dockerRegistry.prod-registry:
! ERROR Cannot find provided path:
/home/user1/.hal/default/credentials/gcr-account.json (No such file or
directory).: /home/user1/.hal/default/credentials/gcr-account.json (No such
file or directory).
I am adding the docker registry account to the local hal config with the following command:
~/hal/hal config provider docker-registry account add prod-registry
--address gcr.io
--username _json_key
--password-file ~/.hal/default/credentials/gcr-account.json
This file definitely exists in this path, and I was able to successfully add a gke account using the provided script. Any ideas as to why I receive this error?
I am provisioning Spinnaker for a new project and got the following error:
$ ~/spinnaker-for-gcp/scripts/expose/configure_endpoint.sh
~/spinnaker-for-gcp/scripts ~/spinnaker-for-gcp/scripts/install
. Creating static IP address spinnaker-external-ip...
Created [https://www.googleapis.com/compute/v1/projects/domain-registry-dev/global/addresses/spinnaker-external-ip].
. Creating service endpoint spinnaker.endpoints.domain-registry-dev.cloud.goog...
Waiting for async operation operations/serviceConfigs.spinnaker.endpoints.domain-registry-dev.cloud.goog:abecc196-a1e4-430c-8687-9bde838ea1a4 to complete...
Operation finished successfully. The following command can describe the Operation details:
gcloud endpoints operations describe operations/serviceConfigs.spinnaker.endpoints.domain-registry-dev.cloud.goog:abecc196-a1e4-430c-8687-9bde838ea1a4
-dev.cloud.goog:abecc196-a1e4-430c-8687-9b
Waiting for async operation operations/rollouts.spinnaker.endpoints.domain-registry-dev.cloud.goog:1453c4aa-3827-4dba-bd7f-79cc1b5daa0f to complete...
Operation finished successfully. The following command can describe the Operation details:
gcloud endpoints operations describe operations/rollouts.spinnaker.endpoints.domain-registry-dev.cloud.goog:1453c4aa-3827-4dba-bd7f-79cc1b5daa0f
loud.goog:1453c4aa-3827-4dba-bd7f-79cc1b5d
Enabling service [endpoints.googleapis.com] on project [domain-registry-dev]...
ERROR: (gcloud.endpoints.services.deploy) INVALID_ARGUMENT: Invalid operation name operations/noop.DONE_OPERATION, refers to an already DONEoperation
. Creating managed SSL certificate spinnaker-managed-cert for domain spinnaker.endpoints.domain-registry-dev.cloud.goog...
Created [https://www.googleapis.com/compute/beta/projects/domain-registry-dev/global/sslCertificates/spinnaker-managed-cert].
NAME TYPE CREATION_TIMESTAMP EXPIRE_TIME MANAGED_STATUS
spinnaker-managed-cert MANAGED 2019-07-02T17:35:58.453-07:00 PROVISIONING
spinnaker.endpoints.domain-registry-dev.cloud.goog: PROVISIONING
~/spinnaker-for-gcp/scripts ~/spinnaker-for-gcp/scripts
~/spinnaker-for-gcp/scripts
~/spinnaker-for-gcp/scripts/install
It took considerably less time than the 30-60 min advised in the tutorial. I then ran the same command again, I got the following:
$ ~/spinnaker-for-gcp/scripts/expose/configure_endpoint.sh
~/spinnaker-for-gcp/scripts ~/spinnaker-for-gcp/scripts/install
. Using existing static IP address spinnaker-external-ip (35.227.196.142)...
. Using existing service endpoint spinnaker.endpoints.domain-registry-dev.cloud.goog...
. Using existing managed SSL certificate spinnaker-managed-cert...
~/spinnaker-for-gcp/scripts ~/spinnaker-for-gcp/scripts
~/spinnaker-for-gcp/scripts
~/spinnaker-for-gcp/scripts/install
The output looks fine this time around. I suspect it could just be a red herring?
If the GKE is configured as a regional cluster, the setup script does not handle it. This is because gcloud container clusters get-credentials $GKE_CLUSTER --zone $ZONE --project $PROJECT_ID is use --zone instead of --region. This also happen at one of the functions in cluster_utils. Also the push_config.sh only check ZONE but ignore REGION. Ideally these scripts should handle both region and zone to cover all cases.
Using the /cli/install_hal.sh
script to set the halyard version updated the hal version in cloudshell but left the version used by the StatefulSet unchanged.
setup.sh
~/spinnaker-for-gcp/scripts/cli/install_hal.sh --version 1.21.1
~/spinnaker-for-gcp/scripts/manage/push_and_apply.sh
hal --version
will return 1.21.1-20190624135101
kubectl exec -it spin-halyard-0 -n halyard -- bash
hal --version
will return 1.22.1-20190724172712
Log after running install_hal.sh
(dependencies truncated)
/spinnaker-for-gcp/scripts/install ([projectname])$ ~/spinnaker-for-gcp/scripts/cli/install_hal.sh --version 1.21.1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 10637 100 10637 0 0 40328 0 --:--:-- --:--:-- --:--:-- 40291
user
non-interactive
version
Halyard version will be 1.21.1
Halyard will be downloaded from gs://spinnaker-artifacts/halyard
Halyard config will come from bucket gs://halconfig
Halconfig will be stored at /home/myuser/.hal/config
Uninstall script is located at /home/myuser/.hal/uninstall.sh
Java is already installed & at the right version
/home/myuser/spinnaker-for-gcp/scripts/install/installhalyard.6396 /home/myuser/spinnaker-for-gcp/scripts/install
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 259M 100 259M 0 0 42.5M 0 0:00:06 0:00:06 --:--:-- 40.8M
halyard/
halyard/config/
halyard/config/halyard.yml
halyard/bin/
halyard/bin/hal.bat
halyard/bin/halyard.bat
halyard/bin/hal
halyard/bin/halyard
halyard/lib/
halyard/lib/json-simple-1.1.jar
halyard/lib/netty-buffer-4.1.34.Final.jar
halyard/lib/netty-transport-4.1.34.Final.jar
halyard/lib/google-api-services-cloudkms-v1-rev8-1.22.0.jar
halyard/lib/maven-builder-support-3.3.9.jar
halyard/lib/maven-repository-metadata-3.3.9.jar
halyard/lib/aws-java-sdk-iot1clickdevices-1.11.534.jar
halyard/lib/spectator-ext-jvm-0.75.0.jar
halyard/lib/groovy-docgenerator-2.5.6.jar
halyard/lib/netty-codec-http-4.1.34.Final.jar
halyard/lib/jsoup-1.8.1.jar
halyard/lib/hk2-utils-2.5.0-b42.jar
halyard/lib/aws-java-sdk-servermigration-1.11.534.jar
halyard/lib/aws-java-sdk-emr-1.11.534.jar
...
hal
update-halyard
groupadd: group 'halyard' already exists
groupadd: group 'spinnaker' already exists
/home/myuser/spinnaker-for-gcp/scripts/install
Welcome to Cloud Shell! Type "help" to get started.
To set your Cloud Platform project in this session use “gcloud config set project [PROJECT_ID]”
The halyard daemon isn't running yet... starting it manually......
1.21.1-20190624135101
Bash auto-completion configured.
To use the auto-completion either restart your shell, or run
. /home/myuser/.bashrc
spin-halyard StatefulSet YAML
image: gcr.io/spinnaker-marketplace/halyard:1.22.1
imagePullPolicy: Always
name: halyard-daemon
Currently testing spinnaker for GCP, I'm facing an issue with the fiat "File" role provider.
This provider need a path to a file describing each roles, I can update the role provider config.
But when I try to push it the path is not rewritten with the halyard user.
Current config in .hal/config
:
authz:
groupMembership:
service: EXTERNAL
google:
roleProviderType: GOOGLE
github:
roleProviderType: GITHUB
file:
roleProviderType: FILE
path: /home/my_user/.hal/default/credentials/fiat-file.yaml
ldap:
roleProviderType: LDAP
enabled: true
Error durring push_and_apply.sh
Problems in Global:
! ERROR Failed to backup user file:
/home/my_user/.hal/default/credentials/fiat-file.yaml
- Failed to deploy Spinnaker.
I'm currently using a workaround with manual editing of .hal/config
to :
authz:
groupMembership:
service: EXTERNAL
google:
roleProviderType: GOOGLE
github:
roleProviderType: GITHUB
file:
roleProviderType: FILE
path: /home/spinnaker/.hal/default/credentials/fiat-file.yaml
ldap:
roleProviderType: LDAP
enabled: true
The setup_properties.sh
sets the default region of the properties file to us-west1
. Deploying the audit log Cloud Function to this region fails with the error:
ERROR: (gcloud.functions.deploy) ResponseError: status=[403], code=[Forbidden], message=[Permission denied on 'locations/us-west1' (or it may not exist)]
The region is no longer in the list of supported regions for Cloud Function.
New to GCP so apologies if this is off-topic.
We would like to try this, but we currently install everything (and manage everything) via least-privilege service accounts.
Is it possible to run cloud shell as a service account, or are there any instructions on running this through gcloud without cloud shell?
Thanks!
The deployment available in Marketplace can't successfully ends. I tried several times and in the most of the times the pods and jobs stay stuck in Pending creation, Does not have minimum availability - crashloopback, FailedCreatePodSandbox, PVC unbound.
e.g.
Failed create pod sandbox: rpc error: code = Unknown desc = Error response from daemon: Get https://k8s.gcr.io/v2/: dial tcp: lookup k8s.gcr.io on 169.254.169.254:53: no such host
From the workload:
Scaling up doesn't solve the issue, after several attempts to deploy the solution always goes to one status above listed
Can you help to know what is happening?
I have the hypothesis that the issue could be related to image version used but is just a but it's just a guess.
Additional in some cases the pod stuck in a restart loop due to the readiness probe failed
Readiness probe failed: wget can't connect to remote host (127.0.0.1) : connection refused
Any insight would be appreciate.
It seems the installation process does not deploy fiat pod. It did not deploy fiat pod even after enabling the feature fiat in ~/.hal.config file.
When adding a webhook trust store, push_and_apply.sh fails when applying the configuration with the following error:
Validation in Global:
! ERROR Failed to backup user file:
/home/bowmanrogere/.hal/default/credentials/jvmracert.jks
It appears as though the part of push_config.sh that replaces $USER with spinnaker does not pick up this line.
As individual operators can install different versions of the hal
client in their cloud shell environments, the halyard daemon could end up in a state where the hal config is incompatible with the version of the hal
client in use by a given operator.
We should add some logic to identify this situation and recommend to the operator that they upgrade their local hal
client and/or the halyard version in use on the daemon.
I spun this up in a quick test project, and went the IAP route.
It seems like mostly everything is happy (can log into the app, and click around) but trying to add an application dies:
The logs are a little angry:
front50 has a 403/insufficient permission error
I 2019-08-01T07:09:55.421362964Z
I 2019-08-01T07:09:55.421703686Z 2019-08-01 07:09:55.421 ERROR 1 --- [0.0-8080-exec-5] c.n.s.f.model.application.Application : Failed to perform action (name: TESTMIDNIGHT)
I 2019-08-01T07:09:55.457688600Z 2019-08-01 07:09:55.452 ERROR 1 --- [0.0-8080-exec-5] c.n.s.k.w.e.GenericExceptionHandlers : Internal Server Error
I 2019-08-01T07:09:55.457754068Z
I 2019-08-01T07:09:55.457758207Z java.lang.reflect.UndeclaredThrowableException: null
at com.netflix.spinnaker.front50.controllers.v2.ApplicationsController$$EnhancerBySpringCGLIB$$cb0646c0.create(<generated>) ~[front50-web.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_212]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_212]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:849) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:760) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) ~[spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) [spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55) [spring-boot-1.5.17.RELEASE.jar:1.5.17.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:111) [spring-boot-actuator-1.5.17.RELEASE.jar:1.5.17.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at com.netflix.spinnaker.fiat.shared.FiatAuthenticationFilter.doFilter(FiatAuthenticationFilter.java:46) [fiat-api-0.63.7.jar:0.63.7]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) [spring-security-web-4.2.9.RELEASE.jar:4.2.9.RELEASE]
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103) [spring-boot-actuator-1.5.17.RELEASE.jar:1.5.17.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.20.RELEASE.jar:4.3.20.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at javax.servlet.FilterChain$doFilter.call(Unknown Source) [tomcat-embed-core-8.5.34.jar:8.5.34]
at com.netflix.spinnaker.front50.filters.SimpleCORSFilter.doFilter(SimpleCORSFilter.groovy:38) [front50-web.jar:na]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at javax.servlet.FilterChain$doFilter.call(Unknown Source) [tomcat-embed-core-8.5.34.jar:8.5.34]
at com.netflix.spinnaker.filters.AuthenticatedRequestFilter.doFilter(AuthenticatedRequestFilter.groovy:140) [kork-web-3.12.2.jar:3.12.2]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) [tomcat-embed-core-8.5.34.jar:8.5.34]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.34.jar:8.5.34]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.34.jar:8.5.34]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
I 2019-08-01T07:09:55.458264972Z {
I 2019-08-01T07:09:55.458268830Z "code" : 403,
I 2019-08-01T07:09:55.458271981Z "errors" : [ {
I 2019-08-01T07:09:55.458274992Z "domain" : "global",
I 2019-08-01T07:09:55.458278218Z "message" : "Insufficient Permission",
I 2019-08-01T07:09:55.458281337Z "reason" : "insufficientPermissions"
I 2019-08-01T07:09:55.458284492Z } ],
I 2019-08-01T07:09:55.458287429Z "message" : "Insufficient Permission"
I 2019-08-01T07:09:55.458290667Z }
Spin/Gate logs don't have much going on.
Any tips? Thanks in advance!
After following the directions for exposing Spinnaker, I can log into the server using oauth. After some period of time (a few hours), I start getting certificate mismatches, and all logins give me the following error:
Untrusted SSL Server Certificate (ssl_server_cert_untrusted_issuer)
Your request contacted a host which presented a certificate signed by an untrusted issuer.
This is typically caused by a Web Site presenting an incorrect or invalid certificate, but could be because of a configuration error.
For assistance, contact your network support team.
Need to think how we configure it, but having requests/limits set, HPA configured, and the CA enabled will allow users to save money when Spinnaker is not in use.
For enterprise customers, we need the option to deploy spinnaker in a private cluster (no public ip addresses on the VMs deployed as GKE nodes).
Hi Team,
Is it possible to send http_proxy configurations for clouddriver. We are trying to connect to github for artefacts, and see that the requests are not going through proxy. Just to test we enabled NAT and its working with the NAT. In our environment we will not be using NAT, so would need a way to add proxy settings to clouddriver. We already tried this configuration, but not working https://www.spinnaker.io/setup/quickstart/faq/#i-want-to-run-a-spinnaker-service-clouddriver-echo-etc-behind-an-http-proxy-server
getting the below error, when i set JAVA_OPTS in clouddriver.yml
2019-11-11 00:22:56.303 WARN 1 --- [ main] c.n.s.c.a.gcs.GcsArtifactConfiguration : Failure instantiating gcs artifact account GcsArtifactAccount(name=gcs-install-account, jsonPath=null):
java.io.IOException: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
at com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentials(DefaultCredentialsProvider.java:134) ~[google-auth-library-oauth2-http-0.18.0.jar:na]
at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:113) ~[google-auth-library-oauth2-http-0.18.0.jar:na]
at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:85) ~[google-auth-library-oauth2-http-0.18.0.jar:na]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactCredentials.(GcsArtifactCredentials.java:61) ~[clouddriver-artifacts.jar:na]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration.lambda$gcsArtifactCredentials$0(GcsArtifactConfiguration.java:47) ~[clouddriver-artifacts.jar:na]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[na:1.8.0_212]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[na:1.8.0_212]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[na:1.8.0_212]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[na:1.8.0_212]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[na:1.8.0_212]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:1.8.0_212]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[na:1.8.0_212]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration.gcsArtifactCredentials(GcsArtifactConfiguration.java:54) ~[clouddriver-artifacts.jar:na]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration$$EnhancerBySpringCGLIB$$f63aa27f.CGLIB$gcsArtifactCredentials$0() ~[clouddriver-artifacts.jar:na]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration$$EnhancerBySpringCGLIB$$f63aa27f$$FastClassBySpringCGLIB$$b08ac6c1.invoke() ~[clouddriver-artifacts.jar:na]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) ~[spring-context-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at com.netflix.spinnaker.clouddriver.artifacts.gcs.GcsArtifactConfiguration$$EnhancerBySpringCGLIB$$f63aa27f.gcsArtifactCredentials() ~[clouddriver-artifacts.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_212]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_212]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:622) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:607) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1321) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1160) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:277) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1467) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1431) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1322) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1209) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1171) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:857) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:760) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:218) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1341) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1187) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:277) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1251) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1171) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:857) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:760) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:218) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1341) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1187) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:845) ~[spring-beans-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) ~[spring-context-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.9.RELEASE.jar:5.1.9.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:743) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:390) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:140) ~[spring-boot-2.1.7.RELEASE.jar:2.1.7.RELEASE]
at org.springframework.boot.builder.SpringApplicationBuilder$run$0.call(Unknown Source) ~[na:na]
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-2.5.7.jar:2.5.7]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115) ~[groovy-2.5.7.jar:2.5.7]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:127) ~[groovy-2.5.7.jar:2.5.7]
at com.netflix.spinnaker.clouddriver.Main.main(Main.groovy:78) ~[clouddriver-web.jar:na]
When trying to setup spinnaker authorization fiat is a required service component, yet I can't see its instance exist
NAME READY STATUS RESTARTS AGE
spin-clouddriver-6644cf67bf-vx2sd 1/1 Running 0 21h
spin-deck-6456b4589b-6xlbx 1/1 Running 0 24h
spin-echo-849dc598bf-pg5m8 1/1 Running 0 21h
spin-front50-77c8db5fd6-8262k 1/1 Running 0 23h
spin-gate-855bb68d86-7fgxs 1/1 Running 0 21h
spin-igor-6d4f7f46df-xvmx2 1/1 Running 0 21h
spin-kayenta-56b686f8d6-69m9l 1/1 Running 0 21h
spin-orca-7886f9dd4f-9sngw 1/1 Running 0 21h
spin-rosco-c99589fd9-h6t4z 1/1 Running 0 21h
Although, search fiat from stackdriver, you can find some information regarding to it. So does fiat need to be enabled before running the setup.sh script?
I'm trying to add a new pubsub subscription to Spinnaker for GCP running the following command:
hal config pubsub google subscription add spinnaker-topic-1 --subscription-name spinnaker-topic-1 --project --message-format CUSTOM
After that I ran push_and_apply.sh
However I don't see the subscription available to work as a trigger in Spinnaker console. Am I missing something?
Just an FYI -- we were interested in the cleaner management of service accounts using workload identity, but spinnaker (installed via spinnaker-for-gcp) has a couple of components that don't seem to enjoy working along with it.
It might be me doing something silly, but it's working with non-spinnaker pods.
It would be nice to have a script that deletes all the resources created. That way if things doesn't work out it is easier to clean up and start from scratch (or just remove everything after a trial run).
We have some regions that we don't have functions available (southamerica-east1, for example) and when we use the same region for spinnaker to deploy the function it fails.
We need to specify the region for fuctions separately from the region of spinnaker.
On deploying new replica set via kubernetes v2 using highlander strategy or any other strategy , infrastructure tab doesnt pick up the deployment status once done instantly and takes like 10-15 mins to pop up or something it just disappers totally,
We are stuck with this and trying to resolve it.,
Having an issue running on an existing cluster
Stuck on:
Waiting on job hal-deploy-apply to complete...
Looking at the logs, I'm seeing:
Failed to ensure the required bucket "spinnaker-plv5i7js7ndopdyyggqv-1564919957" exists:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
Insufficient Permission
This bucket doesn't exist and was not created by the setup script.
Can't find where this is set.
Tried cleaning up with a generated clean up script as well as deleting both spinnaker and halyard namespaces but the error is the same, with the same bucket name.
Thanks
Hi,
I am currently trying to set up Spinnaker on GCP using spinnaker-for-gcp. I have been trying to get a somewhat best practice Istio service mesh running on the GKE cluster as well. I have mTls enabled, and I can expose some of my telemetry plugins from the ingress. I was able to use spinnaker-for-gcp prior to enabling mTls. One of the issues I have identified is the call to Redis from the spin-clouddriver. I have tried getting the Redis service to be accessible by adding the host and port as a ServiceEntry. So far, I haven't had any luck though. I have attached some screenshots of the logs and my ServiceEntry. Any help would be appreciated.
Setup.sh is failing to deploy Cloud function on new projects with several errors:
Deploying function (may take a while - up to 2 minutes)...WARNING: Setting IAM policy failed, try "gcloud alpha functions add-iam-policy-binding spinnaker1AuditLog --region=us-east1 --member=allUsers --role=roles/cloudfunctions.invoker"
or
ERROR: (gcloud.functions.deploy) OperationError: code=7, message=Missing necessary permission resourcemanager.projects.getIamPolicy for serviceAccount:[email protected]
.gserviceaccount.com on project xxxxxxxxx
I think this solution doesn't support External Account Configuration, at least not the way is documented on spinnaker.io because spinnaker-for-gcp doesn't seem to support specifying Spring Cloud Config values.
If this is correct, is there any workaround I could use in the meantime?
Steps to reproduce:
use IAP
follow the tutorial "manage console" step 2 - Included command-line tools
upgrade spinnaker CLI to the latest version - ~/spinnaker-for-gcp/scripts/cli/install_spin.sh
Expected behavior:
spinnaker CLI upgraded to the latest version
Observed behavior:
in: spin app list
out: Status: 403 , Body: {"error":"Forbidden","message":"Access Denied","status":403, }
Further investigations:
The spinnaker CLI installation file is creating an config file with local port forwarding.
gate:
endpoint: http://localhost:8080/gate
The correct config file is created when running the configure_iap script the first time, that branch of the code is never executed again unless the kubernetes secret is manually deleted.
Hello,
I am using this repo to setup my test Spinnaker and it works great.
However I was going to suggest using Kapitan to generate the scripts and manage the deployment. You would be able to simplify lots of your logic and you would be able to support multiple deployment out of the box.
contact us on the #kapitan channel on Kubernetes slack if you need help.
Some background: we are creating the kubernetes cluster, and a few other things (network, service account, etc) via terraform.
Then I'm running the setup scripts from cloud shell.
the setup script gets the iam user via:
SA_EMAIL=$(gcloud iam service-accounts --project $PROJECT_ID list \
--filter="displayName:$SERVICE_ACCOUNT_NAME" \
--format='value(email)')
the iap script however:
gcloud iam service-accounts keys create ~/.spin/key.json \
--iam-account $SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com \
So unless your service account is created a very specific way, one script is going to bomb out.
The setup script creates it as:
gcloud iam service-accounts --project $PROJECT_ID create \
$SERVICE_ACCOUNT_NAME \
--display-name $SERVICE_ACCOUNT_NAME
Which is why the project functions correctly.
Hi everyone!
Regarding the spinnaker solution in marketplace from GCP, in the step related to the manner to access the spinnaker console, the port forwarding option is working as well as the documentation mentioned except by the link "Connect to Spinnaker via 'Preview on port 8080'", this link doesn't redirect and a warning message is shown : The element cannot be found either because it is still loading or doesn't exist. Ensure previous steps have been completed.
For the second option, expose publicly using an external IP and an endpoint, seems like the script:
~/spinnaker-for-gcp/scripts/expose/configure_endpoint.sh
Does not create the resources needed to expose this, following step by step only the external IP creation is done then following configuration can't be completed even when the output says done.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.