Comments (16)
demobox
commented on August 14, 2024
Just to clarify: this actually worked initially, and then started failing? And it's still failing now, or was it a transient issue?
Would it be possible for you to try creating another Spinnaker for GCP instance from the Management Console and securing that too? Just trying to figure out if this is a one-off or potentially related to your account/setup in some way?
from spinnaker-for-gcp.
JeffCertain
commented on August 14, 2024
That's correct. I exposed spinnaker via IAP, waited for a little while, and
connected successfully. After some time, I could no longer connect. There
was a timeout for inactivity in the mix.
I've gone through that tutorial a couple of times now, starting new
projects each time. The behaviour is consistent. However, I'm happy to run
through the creation of a new project today to see if it happens again.
…On Wed, Sep 18, 2019 at 5:29 AM Andrew Phillips ***@***.***> wrote:
Just to clarify: this actually worked initially, and *then* started
failing? And it's still failing now, or was it a transient issue?
Would it be possible for you to try creating another Spinnaker for GCP
instance from the Management Console and securing that too? Just trying to
figure out if this is a one-off or potentially related to your
account/setup in some way?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#120?email_source=notifications&email_token=AAIK5QZTAP6W6MO7PI3UZZTQKINLPA5CNFSM4IXVY7W2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD674N2A#issuecomment-532661992>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAIK5Q3ZQCYR65N6OYQ7MHDQKINLPANCNFSM4IXVY7WQ>
.
from spinnaker-for-gcp.
demobox
commented on August 14, 2024
The behaviour is consistent.
Hm, interesting - not something we've seen before, indeed. @duftler thoughts on potential next steps to debug? Any error messages or logs to look out for?
from spinnaker-for-gcp.
duftler
commented on August 14, 2024
I've so far been unable to reproduce this. I've tried creating multiple projects and Spinnaker deployments from scratch, from yesterday through this morning, with both internal and external accounts.
In the hope that we spot some meaningful clue, I've captured screen-grabs of the relevant resources in cloud console. Maybe you see something that looks different/unhealthy in your environment?
from spinnaker-for-gcp.
duftler
commented on August 14, 2024
Could also look for clues by dropping the domain name into any of the public ssl checker sites.
from spinnaker-for-gcp.
JeffCertain
commented on August 14, 2024
This happened again this morning with a new project.
I'm working my way through the automated canary stage via Kayenta (
https://cloud.google.com/solutions/automated-canary-analysis-kubernetes-engine-spinnaker).
Somewhere about the time that I enable Kayenta, I get logged out of my
spinnaker instance, and I start seeing the SSL certificate mismatch.
What's really weird is that the certificate checks out with sslshopper.com.
(https://spinnaker-1.endpoints.spinnaker-test-253416.cloud.goog)
The problem persists in anonymous mode, and after re-running the IAP script.
Looking at the screenshots, the "Backend Service Details" pretty clearly
show no traffic is making it to the box
[image: Screen Shot 2019-09-19 at 11.35.35 AM.png]
Other than that, everything looks good.
~ Jeff
…On Thu, Sep 19, 2019 at 8:54 AM Matt Duftler ***@***.***> wrote:
Could also look for clues by dropping the domain name into any of the
public ssl checker sites.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#120?email_source=notifications&email_token=AAIK5Q35CX7P5W7KJG7YPCDQKOOBPA5CNFSM4IXVY7W2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7D6VKQ#issuecomment-533195434>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAIK5Q6E4PLL5IZCUODH26LQKOOBPANCNFSM4IXVY7WQ>
.
from spinnaker-for-gcp.
JeffCertain
commented on August 14, 2024
Don't know if this helps, but when I try to hit the same instance after
re-running "connect_unsecured.sh" and using the web preview on 8080, I get "
Error fetching applications. Check that your gate endpoint is accessible.
Further information on troubleshooting this error is available here."
On Thu, Sep 19, 2019 at 11:39 AM Jeff Certain <[email protected]>
wrote:
… This happened again this morning with a new project.
I'm working my way through the automated canary stage via Kayenta (
https://cloud.google.com/solutions/automated-canary-analysis-kubernetes-engine-spinnaker).
Somewhere about the time that I enable Kayenta, I get logged out of my
spinnaker instance, and I start seeing the SSL certificate mismatch.
What's really weird is that the certificate checks out with sslshopper.com.
(https://spinnaker-1.endpoints.spinnaker-test-253416.cloud.goog)
The problem persists in anonymous mode, and after re-running the IAP
script.
Looking at the screenshots, the "Backend Service Details" pretty clearly
show no traffic is making it to the box
[image: Screen Shot 2019-09-19 at 11.35.35 AM.png]
Other than that, everything looks good.
~ Jeff
On Thu, Sep 19, 2019 at 8:54 AM Matt Duftler ***@***.***>
wrote:
> Could also look for clues by dropping the domain name into any of the
> public ssl checker sites.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#120?email_source=notifications&email_token=AAIK5Q35CX7P5W7KJG7YPCDQKOOBPA5CNFSM4IXVY7W2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7D6VKQ#issuecomment-533195434>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAIK5Q6E4PLL5IZCUODH26LQKOOBPANCNFSM4IXVY7WQ>
> .
>
from spinnaker-for-gcp.
JeffCertain
commented on August 14, 2024
I think it's likely that something in the Kayenta demo, around the
"automated canary" steps put spinnaker into a bad state. Starting a new
project from scratch and following the directions in that tutorial ends up
with spinnaker throwing SSL errors even exposed only via port forwarding.
At least, I created a brand new project, and started that tutorial, and ran
into the same problem about that point.
https://cloud.google.com/solutions/automated-canary-analysis-kubernetes-engine-spinnaker
This is now 7 spinnaker tutorials that have ended up in this state, so it's
pretty repeatable. (Previous projects started with the spinnaker tutorial,
then I moved on to the canary tutorial.)
~ Jeff
On Thu, Sep 19, 2019 at 11:44 AM Jeff Certain <[email protected]>
wrote:
… Don't know if this helps, but when I try to hit the same instance after
re-running "connect_unsecured.sh" and using the web preview on 8080, I get "
Error fetching applications. Check that your gate endpoint is accessible.
Further information on troubleshooting this error is available here."
On Thu, Sep 19, 2019 at 11:39 AM Jeff Certain ***@***.***>
wrote:
> This happened again this morning with a new project.
>
> I'm working my way through the automated canary stage via Kayenta (
> https://cloud.google.com/solutions/automated-canary-analysis-kubernetes-engine-spinnaker).
> Somewhere about the time that I enable Kayenta, I get logged out of my
> spinnaker instance, and I start seeing the SSL certificate mismatch.
>
> What's really weird is that the certificate checks out with
> sslshopper.com. (
> https://spinnaker-1.endpoints.spinnaker-test-253416.cloud.goog)
>
> The problem persists in anonymous mode, and after re-running the IAP
> script.
>
> Looking at the screenshots, the "Backend Service Details" pretty clearly
> show no traffic is making it to the box
> [image: Screen Shot 2019-09-19 at 11.35.35 AM.png]
>
> Other than that, everything looks good.
>
> ~ Jeff
>
> On Thu, Sep 19, 2019 at 8:54 AM Matt Duftler ***@***.***>
> wrote:
>
>> Could also look for clues by dropping the domain name into any of the
>> public ssl checker sites.
>>
>> —
>> You are receiving this because you authored the thread.
>> Reply to this email directly, view it on GitHub
>> <#120?email_source=notifications&email_token=AAIK5Q35CX7P5W7KJG7YPCDQKOOBPA5CNFSM4IXVY7W2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7D6VKQ#issuecomment-533195434>,
>> or mute the thread
>> <https://github.com/notifications/unsubscribe-auth/AAIK5Q6E4PLL5IZCUODH26LQKOOBPANCNFSM4IXVY7WQ>
>> .
>>
>
from spinnaker-for-gcp.
duftler
commented on August 14, 2024
Ah, didn't realize that you were working through the Kayenta solution. I'll try reproducing by running through that as well.
from spinnaker-for-gcp.
demobox
commented on August 14, 2024
/cc @bgood as someone who knows the canary analysis tutorial well
from spinnaker-for-gcp.
JeffCertain
commented on August 14, 2024
This is getting both more frustrating and more bizarre.
I spun up a completely new gmail account, and followed through the Kayenta
tutorial. I had a good five or six hours where Spinnaker was stable. Then I
got the SSL issue. This was end of day Friday.
On a hopeful whim, I restarted cloud shell, did the port-forwarding
dance... and, lo! My spinnaker instance is still there.
~ Jeff
…On Mon, Sep 23, 2019 at 1:02 PM Andrew Phillips ***@***.***> wrote:
/cc @bgood <https://github.com/bgood> as someone who knows the canary
analysis tutorial well
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#120?email_source=notifications&email_token=AAIK5Q6ZL3P7CYG6CRGA6KTQLEOGVA5CNFSM4IXVY7W2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7MCZUI#issuecomment-534260945>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAIK5Q2KDF6OHSNL3KEE3XTQLEOGVANCNFSM4IXVY7WQ>
.
from spinnaker-for-gcp.
bgood
commented on August 14, 2024
Jeff,
I tried reproducing this last night and my Spinnaker with IAP is still working this morning. To make sure I did the correct scenario I have a couple questions.
Is this only happening with the canary analysis tutorial or does it happen with other tutorials as well?
Does the SSL problem happen when you run the hal commands and push_and_apply.sh to setup canary analysis, or is more to do with time?
Thanks
Ben
from spinnaker-for-gcp.
JeffCertain
commented on August 14, 2024
Hi Ben,
It seems to be really unpredictable. At first I thought it was just happening early on, and that letting it bake a little longer would help. Then, on Friday, the problem manifested itself after 6 hours or so.
I'm starting to suspect this might actually be the result of some corporate security protocols that sometimes mess with certificates. It has, for the most part, been stable this week.
~ Jeff
from spinnaker-for-gcp.
duftler
commented on August 14, 2024
Has this problem crept up again, or have things been stable?
Any luck determining if there are security protocols in place that might be modifying the certificates?
from spinnaker-for-gcp.
JeffCertain
commented on August 14, 2024
Looks like it was, in fact, internal security protocols. So frustrating!!
from spinnaker-for-gcp.
demobox
commented on August 14, 2024
Looks like it was, in fact, internal security protocols. So frustrating!!
Thanks for getting back to confirm that!
from spinnaker-for-gcp.
Related Issues (20)
- No valid versions with the prefix "1.18.17" found. HOT 1
- Spinnaker 1.26.6 still uses legacy UI? Don't see gcs artifact account when creating stage. HOT 1
- Spinnaker is messed up after resize cluster to 0 nodes HOT 2
- Doesn't work, GKE version outdated? HOT 1
- request: mention "Control plane authorised networks" in documentation
- Issue in Connect to Spinnaker - on Step 2
- Application information missing in Spinnaker after re-adding GKE accounts HOT 2
- Random identifier generation can hang
- `--enable-stackdriver-kubernetes` being deprecated when creating GKE cluster.
- When fiat is enabled service and deployment labels are not applied
- `setup.sh` enables Cloud APIs that are not required by all deployments HOT 1
- Automatically select kube context.
- `scripts/manage/connect_to_redis.sh` is flakey
- ! ERROR Failed to backup user file
- Unable to use the repository to deploy Spinnaker on GCP HOT 1
- Is the spinnaker-for-gcp repository still active?
- Is the spinnaker-for-gcp repository still active?
- Unable to configure Email configuration
- Modify function setup to automatically select available nodejs runtimes
- getting error while adding the gke account by running the script HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spinnaker-for-gcp.