adoptium / infrastructure Goto Github PK
View Code? Open in Web Editor NEWThis repo contains all information about machine maintenance.
License: Apache License 2.0
This repo contains all information about machine maintenance.
License: Apache License 2.0
Text::CSV is installed via ansible playbook:
- name: Install Text::CSV
shell: |
cpanm --with-recommends Text::CSV
tags: text_csv
However, it doesn't appear on the machine
[root@rhel7hcxrt1 ~]# perl -MText::CVS -e 1
Can't locate Text/CVS.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .).
Running the installation command manually says its already installed and up to date:
[root@rhel7hcxrt1 ~]# cpanm --with-recommends Text::CSV
Text::CSV is up to date. (1.95)
Issues has been seen everywhere we checked so far:
RHEL 6 PPC64, RHEL 7 x86 PPC64, UB 14/16 x86,
Investigate using ramdisk to speed up AIX builds
https://nagios.adoptopenjdk.net/nagios/map.php no longer displays the map of machines managed by Nagios, and produces a 404 - Not Found.
I didn't notice when exactly it started failing (but may have been after the issue #8 reinstall?)
If the Jenkins master falls over, can a slave take over? If not then we need to think about having the master backed up / saved as a docker image / whatever to make sure we can failover to a new node fast.
Same goes for the main ansible box.
The Jenkins server (http://ci.adoptopenjdk.net) root partition is almost full.
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 188G 142G 37G 80% /
101GB of this is in /home/jenkins/.jenkins/jobs
Looking at the builds, It does not appear that they are configured to clean themselves up.
This should be configured for all builds.
All of my jobs on there are failing, which includes the parent of a matrix job which is therefore blocking some of my executions. Can we get git installed on this box please?
To ensure that machine images can be reliably recreated for AdoptOpenJDK build/test we need entirely scripted configuration that sets up a VM "from scratch".
A number of the Ansible playbooks exist in the openjdk-build repo, but they are not complete in their coverage.
Proposed steps are:
Goal is that we can discard a VM at any point and recreate it entirely using the public information in our scripts.
Graph appears however all values are shown as '-nan'
Hello,
Your certificate (or certificates) for the names listed below will expire in
19 days (on 26 Jul 17 00:42 +0000). Please make sure to renew
your certificate before then, or visitors to your website will encounter errors.
nagios.adoptopenjdk.net
For any questions or support, please visit https://community.letsencrypt.org/.
Unfortunately, we can't provide support by email.
For details about when we send these emails, please visit
https://letsencrypt.org/docs/expiration-emails/. In particular, note
that this reminder email is still sent if you've obtained a slightly
different certificate by adding or removing names. If you've replaced
this certificate with a newer one that covers more or fewer names than
the list above, you may be able to ignore this message.
If you want to stop receiving all email from this address, click
http://mandrillapp.com/track/unsub.php?u=30850198&id=8fb004715c47471b98c23130d1ca600a.OYLci%2Fk79LBUOvvM5JFmpLp8Mdw%3D&r=https%3A%2F%2Fmandrillapp.com%2Funsub%3Fmd_email%3Dbrad_blondin%2540ca.ibm.com
(Warning: this is a one-click action that cannot be undone)
Regards,
The Let's Encrypt Team
Openjdk tests build on windows got failures for permission issue:
java.nio.file.AccessDeniedException: C:\Users\jenkins\workspace\openjdk_test_x86-64_windows\openjdk-test\OpenJDK_Playlist\openjdk-jdk8u\jdk\test\sun\management\windows\revokeall.exe
According to last two comments in adoptium/aqa-tests#37 (comment) Jenkins machine configuration need to update to specify the tools location for git.
The issue are still there suppose the configuration isn't be updated.
As described over in the Node community, there is a Jenkins launch option that will let people see the jobs definition.
Looks like a good idea and something we should be doing in this project.
This issue covers amending the Jenkins launch script and selectively opening up our job definitions.
We need to run the JCK suite on pLinux-LE and it's access needs to be locked down so it cannot be shared with other jobs.
Spec-wise something like 2-core, 8Gb RAM and an SSD of around 100Gb would be ideal as that's what we're using for xLinux.
The Jenkins host ci.adoptopenjdk.net
had a number of critical OS package updates pending. Upgrading the packages has introduced problems with Jenkins.
Jenkins is up and running, but a number of nodes are currently flagged as offline.
@TianyuZuo has offered to install docker on the test machines to aid the docker based testing. I will add their key to the root users for all test machines and will revoke access upon completion
if a CentOS machine is selected for the STF suites then the builds do not run. See AdoptOpenJDK/openjdk-jdk9#1
We need a global set of appropriate test
tags applied to the machines to match the current expectations in https://github.com/AdoptOpenJDK/openjdk-build/wiki/%5BWIP%5D-Minimum-OS-levels but at the moment we can't lock the jobs to that tag due to #25
The Blue Ocean UI should be available under the /blue route, and should not interfere with any normal behavior.
https://jenkins.io/doc/book/blueocean/getting-started/#start-blueocean
New machines that are configured for AdoptOpenJDK should have some real time clock synchronization package installed (e.g. NTP, timesyncd, etc) to ensure they do not drift too far and disrupt Jenkins pipeline coordination.
Although many of our jobs are quite long running, where they fail they may fail quickly and being out of sync by tens of seconds matters.
names don't match Jenkins:
openjdk-api-server
openjdk-staging-server
build-packet-armv8-ubuntu-16.04
Missing:
build-scaleway-x64-ubuntu-16.04-2
test-macincloud-macos1010-1
test-macincloud-macos1010-2
I'm working on getting the OpenJDK/OpenJ9 builds working on ARM. Would it be possible to get access to the ARM build systems for some basic toe-in-the-water evaluations of my initial builds?
The API information is at https://api.adoptopenjdk.net/ but that currently appears not to be serving up what it should.
I'm getting an issue on build-marist-s390x-sles-12
(148.100.110.56) where it is unable to resolve it's own hostname. Can we get an entry for openjdk-sles12
(The output from hostname
) added to /etc/hosts
on the machine - either with it's real IP or just to 127.0.0.1 please so that it resolves? This is causing some tests to fail as per adoptium/aqa-systemtest#9
The following machines are not currently known to our Nagios installation, and should be added to ensure their basic health is monitored:
api.adoptopenjdk.net
staging.adoptopenjdk.net
The following publicly available services should also be monitored so the #infrastructure channel is notified if they go down:
www.adoptopenjdk.net
api.adoptopenjdk.net
ci.adoptopenjdk.net
keybox.adoptopenjdk.net
staging.adoptopenjdk.net
ansible.adoptopenjdk.net
At the moment we run the systemtests on sles12
as this has a suitable version of the libffi
library available (https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues/19). At present none of those machines has a a tag of test
so I cannot use that tag to run the systemtest jobs. If I leave the tag off I end up with things potentially running on master
which doesn't work well at all as it doesn't have make
installed. I could use build
that would stop the other platforms from using the dedicated test machines, so that's not a sensible solutionest either. For now I've set the jobs to use !hg
which knocks out three machines including master
and is adequate until we get a sles12/s390x
box tagged with test
See also the work item about machines tags: #93
The /dev/dasdb2
file system on build-marist-s390x-sles-12 (148.100.110.56) is filling up, currently at 96%.
On the Ubuntu sister machine, the system upgrades had multiple versions of the kernel left behind. That may be happening on SLES too.
This task is to clear out any unused packages and kernels to free up the root partition.
We need to run the JCK suite on zLinux and it's access needs to be locked down so it cannot be shared with other jobs.
Spec wise a 2-core, 8Gb, and a fast disk of around 100Gb should be ideal.
I am having issues using wget
or curl
on this machine. This is also preventing me from being able to connect it to jenkins
➜ ~ ssh [email protected]
[linux1@adoptopenjdk ~]$ wget https://google.com
--2017-06-26 04:02:53-- https://google.com/
Resolving google.com (google.com)... failed: Connection refused.
wget: unable to resolve host address ‘google.com’
[linux1@adoptopenjdk ~]$
CC @bblondin @AdoptOpenJDK/getopenjdk
Our installation of Nagios Core 4.3.1 is outdated and should be upgraded. The latest version of Nagios Core is 4.3.4 was released on 2017-08-24.
Marist have generously created two new Ubuntu 16.04 systems for us. One is the replacement for our old RHEL6 image (148.100.110.55) while the other is an extra one that we requested to cope with the additional workload.
Both images have 8 Gig Memory / 100G Disk / 4 CP's
Systems:
LXEOJ905 - 148.100.33.178
LXEOJ906 - 148.100.33.179
I have the login details for these for those that need them.
This task is to configure the machines for build/test as appropriate, add the new nodes to Jenkins, Nagios, etc.
The following new AIX build / test boxes are available to the project. I have added the keybox public key to the list of authorized_keys. Note that there are existing authorized keys that should be retained for the hoster's maintenance use.
power8-aix-openjdk1.osuosl.org - 140.211.9.10
power8-aix-openjdk2.osuosl.org - 140.211.9.12
Each system is 32GB memory, 5 vCPU, 1 CPU unit that can dynamically adapt to 10 CPU, and a minimal AIX 7.1 install. The AIX 7.1.4.4 DVD1 still is "mounted". The OS and related files are installed on filesystems allocated from rootvg, and /home is allocated from homevg. Each volume group is 80GB and most of rootvg is unallocated with considerable room for expansion.
Both systems have been set up with larger queue depth for the hdisks, which improves performance a little. One also can create a ramdisk.
You can customize the systems as you wish.
The new version of Jenkins requires JDK8. Our nodes currently have JDK7, so need to be upgraded (via Ansible?) to install the later JDK before upgrading Jenkins.
Consider running AdoptOpenJDK builds of JDK8 on each node?
nagiosgraph parses output and performance data from Nagios plugins and stores the data in RRD files. nagiosgraph creates graphs and generates HTML pages with graphic reports from the data.
We need to run the JCK suite on WIndows and it's access needs to be locked down so it cannot be shared with other jobs.
Needs to have a fast disk (so I'd say SSD) and ideally powerful cores (but doesn't need many of them) so something like 2 core/8Gb/100Gb SSD should suffice. Perhaps 16Gb+ if we decide to use a ramdrive for holding the JCK test suite itself.
Windows version TBD - what do Oracle test on?
Launching Jenkins currently requires remembering a long command-line. To keep things simple it would be preferable to embody this as a system service or some such thing, so that it will start on normal machine boot level, and be easier to stop/restart etc.
Original suggestion by @karianna
We could do with a windows 2012 server with visual studio 2013 to build the openj9 binaries as this is the required level for openj9 to build. I will investigate where we could source one from.
We now have two z/OS machines to add to the build farm
In order to be able to build certain artefacts i.e. code-tools related (e.g. SigTest) we need to have the following JDKs installed:
As version 7 and 8 are already installed.
Note OpenJDK version 5 and 6 are not easily installable via ansible scripts. So an alternative source will need to be sought after. Which might add to the complications of our artefacts being built using different flavours of JDK (a bit inconsistent).
We could download from Oracle but with the latest changes, we will need a login and password to be able to download old versions of the JDK. Which means passing these details into the ansible script.
Standing up an environment using vagrant
(for ubuntu 14.04) and running the ubuntu.yml Ansible script halts with the below message:
fatal: [localhost]: FAILED! => {"failed": true, "msg": "An unhandled exception occurred while
running the lookup plugin 'file'. Error was a <class 'ansible.errors.AnsibleError'>,
original message: could not locate file in lookup: /Vendor_Files/keys/id_rsa.pub"})
This occurs when run on a local machine (reproducible on Linux and MacOSX environments).
Re-running with -v
flags will help, -vv
, -vvv
, etc... will give will more verbose info about the issue.
See #58 (comment) in #58
Add any additional info to https://github.com/AdoptOpenJDK/openjdk-infrastructure/blob/master/ansible/README.md, once resolved or any findings during the course of the investigation.
#helpwanted #bug
OSUOSL system administrators have received a security vulnerability report about our two AIX openjdk systems hosted at OSUOSL.
Details available privately upon request.
This issue covers resolving the vulnerabilities and reporting the actions taken back to OSUOSL.
Agreed that Macstadium will provide us with two further mac's for this purpose. Waiting to find out which os level to deploy.
Create a Nagios System Configuration Tool (script) to help setup/configure new systems host.cfg files for Nagios to monitor
ask questions then generate the host.cfg file
test and enable monitoring
We have a number of automated 'bots' that post to Slack about various topics.
The bots are swamping some channels with automated messages, and hiding any real post. It is also unnecessary to archive most of the bot postings, so we can choose which are archived.
This issue is to create #<blah>-bots
channels and switch the bots to posting on there so the humans have a chance.
Biggest offenders are likely:
#infrastructure
where Nagios should be posting to #infrastructure-bot
(un-archived), and
#website
where Localize should be posting to #website-bot
(un-archived).
Currently we are hosting a lot of our x86 hardware with packet.net
I want to move away from this as I want to free up our usage limit so that we can provision more arm machines for testing
FYI @vielmetti
I've had a few people as if they can see the job configuration to be able to understand what the jobs are doing. While jenkins doesn't have any integrated ability to allow read-only access (aso by default if you can view it you edit it) there are plugins such as https://wiki.jenkins.io/display/JENKINS/Extended+Read+Permission+Plugin which will change that. opening this issue for discussion to see if there is any reason not to have this in place - do we have sensitive stuff in the jobs that wouldn't be hidden by this plugin?
The tests run part of the mauve test suite as a load test, and need to checkout the mauve test suite which is in a cvs repo.
Request for minimally 1 (eventually 2, if we do not start sharing machines across build/test functionality) zLinux machines for JCK testing (similar request to #77), "Spec wise a 2-core, 8Gb, and a fast disk of around 100Gb".
I will piggy-back on the requests for JCK test machines (asking for same requirements as #76),
"Spec-wise something like 2-core, 8Gb RAM and an SSD of around 100Gb ".
One (or eventually two) machines, so that we can enable the following types of tests:
(optionally/eventually some perf micro benchmarks).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.