Code Monkey home page Code Monkey logo

ec2-spot-jenkins-plugin's Issues

Passing Credentials in Jenkins Slaves Launched Using Amazon EC2 Plugin

Hi Team,

Am following below document to launch spot jenkins slave. Am able to launch slave successfully and connect to jenkins server and bring it online.

https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin .

When am trying to run jenkins job in the launched spot slave facing below issue:

  1. All our jobs run as xxx user as our application is configured to use xxx user rather root user.
  2. In our old jenkins when we launch slave manually we have an option to specify to credentials as xxx so that jobs connect to server as xxx user and executes successfully. I have attached screen shot highlighting that option.
  3. But here spot slave doesn't have that option while in configure or later to add manually so here jobs connect to slave as root user rather xxx user which makes jobs fails as all our repo and configs should use xxx user.

Can you please check and let me know how I can add that option to add credentials in slave.

In both old jenkins where slaves launched manually and new one where we use spot slaves has SSH slaves plugin 0.27.

Thanks,
Ashlin
screen shot 2017-02-22 at 5 18 38 pm
screen shot 2017-02-22 at 5 18 31 pm

Help !!!

I am trying to use the ec2 fleet plugin and I am having troubles get it going

So I have a AWS account and I created an AMI for jenkins to use
Added all the settings to "Manage Jenkins" > "Configure System"
I cant see Ec2 launching to start the Jenkins work

I am using Jenkins ver. 2.87

Multiple fleets cause very strange behavior

Fleets seem to assume that they are the only fleet around, so if they encounter an unrecognized node, they try to terminate it. Fleets should run independently, and leave unrecognized nodes alone.

I have 2 fleets running, and one of them is getting scaled way past its capacity. The configuration says max size is 5, but it's setting its target to 8.

Windows

What are the configuration settings needed in the Windows slave and in Jenkins master to have a Windows fleet? I've gotten this to work for Linux but the documentation on Windows slaves is lacking.

The way to create on-demand instance?

hi , I am an AWS user, may I ask for how to create an On-Demand kind of EC2 on Linux system, Can I use this template and just change "spot" to "on-demand"? thank you in advance.

Getting `Key value does not parse into a valid ssh-rsa key`

I launched the spot fleet request where 1 inst is launched and running with keypair xxx .
While configuring the jenkins plugin i selected the
Host Key Verification Strategy as Manually provided Verification Strategy and base 64encoded the RSA private key of xxx and gave as 'ssh-rsa base64encodedprivatekey'

I am getting error 'Key value does not parse into a valid ssh-dss key'

image

Is the plugin deprecated?

I wonder if the plugin does not work well, because I get these after a normal setup:

Aug 09, 2019 2:17:37 PM SEVERE hudson.triggers.SafeTimerTask run
Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@5809cd8 failed
com.amazonaws.services.ec2.model.AmazonEC2Exception: Fleet Request: sfr-66fce1c4-0ca3-4513-90d5-c09d4f283d66 is not a modifiable fleet request type. (Service: AmazonEC2; Status Code: 400; Error Code: FleetNotInModifiableState; Request ID: a5a9d6f7-ec01-4c6e-afca-ee9e0d85e2de)

It both doesn't scale up instances, and doesn't behave well overall. Seems like the plugin is abandoned by the official AWS support in this repo?

Can you update jenkins-ci.org with latest code?

Currently using verison 1.0 of this plugin on our CI pipeline but the server gets stuck in a weird state unable to provision or terminate instances.

SEVERE: Timer task hudson.slaves.ComputerRetentionWork@5491f8ea failed
java.lang.IllegalStateException: Unknown instance terminated: i-184fe180
	at com.amazon.jenkins.ec2fleet.EC2FleetCloud.terminateInstance(EC2FleetCloud.java:266)
	at com.amazon.jenkins.ec2fleet.IdleRetentionStrategy.check(IdleRetentionStrategy.java:28)
	at com.amazon.jenkins.ec2fleet.IdleRetentionStrategy.check(IdleRetentionStrategy.java:12)
	at hudson.slaves.ComputerRetentionWork$1.run(ComputerRetentionWork.java:72)
	at hudson.model.Queue._withLock(Queue.java:1312)
	at hudson.model.Queue.withLock(Queue.java:1189)
	at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:63)
	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:50)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

Dug into the code running on jenkins and looked at what's on master here and they don't line up.

Can we get latest master published to jenkins plugin hub? If not, what's the blocker here and how can I help?

Multiple Spot Fleets Supported?

I'm working on a Jenkins setup where we have a Build Flow that utilizes multiple slave EC2 types to reduce cost.

Upon testing we found that if provisioned two spot fleets A - small, and B - big, when a job requests for slave of B, A will increase the request.

From empirical testing, it seems that the top fleet, in order of how they are configured in settings, receives all the requests for a new node regardless of type or label.

Is this a bug or just a lack of labels? If it is a lack of labels how hard would this be to implement?

Destroy instance after completing a certain number of builds

I would like a feature in the ec2 spot fleet plugin that would allow me to specify a maximum number of builds on an instance before it is destroyed. This exists in the regular EC2 Jenkins plugin, but it does not exist in the EC2 spot fleet plugin.

In short, I would like to say:

  • Destroy an instance when it has been idle for X minutes (this exists today)
  • Destroy an instance when it has processed 100 builds (whether it is idle or not) (this does not exist today)

Deadlocking while spinning up a large spot fleet

Using a build of master, I'm encountering deadlocks when trying to scale to 64 spot fleet instances. The following is the stack trace received in the master log. After receiving this message, the fleet does not continue to scale up to the max cluster size, nor do some of the provisioned and launched workers receive queued tasks.

WARNING: Some health checks are reporting as unhealthy: [thread-deadlock : [AtmostOneTaskExecutor[Periodic Jenkins queue maintenance] [#46] locked on java.util.concurrent.locks.ReentrantLock$NonfairSync@16e9a6e2 (owned by jenkins.util.Timer [#3]):
	 at sun.misc.Unsafe.park(Native Method)
	 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	 at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
	 at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
	 at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
	 at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
	 at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
	 at hudson.model.Queue.maintain(Queue.java:1381)
	 at hudson.model.Queue$1.call(Queue.java:294)
	 at hudson.model.Queue$1.call(Queue.java:291)
	 at jenkins.util.AtmostOneTaskExecutor$1.call(AtmostOneTaskExecutor.java:101)
	 at jenkins.util.AtmostOneTaskExecutor$1.call(AtmostOneTaskExecutor.java:91)
	 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	 at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:110)
	 at java.lang.Thread.run(Thread.java:745)
, jenkins.util.Timer [#3] locked on com.amazon.jenkins.ec2fleet.EC2FleetCloud@402a23d8 (owned by jenkins.util.Timer [#1]):
	 at com.amazon.jenkins.ec2fleet.IdleRetentionStrategy.check(IdleRetentionStrategy.java:38)
	 at com.amazon.jenkins.ec2fleet.IdleRetentionStrategy.check(IdleRetentionStrategy.java:15)
	 at hudson.slaves.ComputerRetentionWork$1.run(ComputerRetentionWork.java:72)
	 at hudson.model.Queue._withLock(Queue.java:1303)
	 at hudson.model.Queue.withLock(Queue.java:1180)
	 at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:63)
	 at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:50)
	 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	 at java.lang.Thread.run(Thread.java:745)
, jenkins.util.Timer [#1] locked on java.util.concurrent.locks.ReentrantLock$NonfairSync@16e9a6e2 (owned by jenkins.util.Timer [#3]):
	 at sun.misc.Unsafe.park(Native Method)
	 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	 at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
	 at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
	 at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
	 at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
	 at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
	 at hudson.model.Queue._withLock(Queue.java:1301)
	 at hudson.model.Queue.withLock(Queue.java:1180)
	 at jenkins.model.Nodes.addNode(Nodes.java:133)
	 at jenkins.model.Jenkins.addNode(Jenkins.java:1985)
	 at com.amazon.jenkins.ec2fleet.EC2FleetCloud.addNewSlave(EC2FleetCloud.java:354)
	 at com.amazon.jenkins.ec2fleet.EC2FleetCloud.updateStatus(EC2FleetCloud.java:311)
	 at com.amazon.jenkins.ec2fleet.CloudNanny.doRun(CloudNanny.java:42)
	 at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:50)
	 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	 at java.lang.Thread.run(Thread.java:745)
]]

It seems that this block is our culprit:
https://github.com/awslabs/ec2-spot-jenkins-plugin/blob/master/src/main/java/com/amazon/jenkins/ec2fleet/IdleRetentionStrategy.java#L38-L63

We are synchronizing on the parent object, but sadly I'm not familiar enough with Java to make meaningful changes. I imagine the parent object might want to manage synchronization rather than the children all trying to lock?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.