Code Monkey home page Code Monkey logo

openjdk-runtime's Introduction

Image Build Status
Stability Build Status
Stability Build Status

Google Cloud Platform OpenJDK Docker Image

This repository contains the source for the Google-maintained OpenJDK docker image. This image can be used as the base image for running Java applications on Google App Engine Flexible Environment and Google Kubernetes Engine.

Repository/Tag Details

Supported images/tags include:

  • gcr.io/google-appengine/openjdk:8

App Engine Flexible Environment

When using App Engine Flexible, you can use the runtime without worrying about Docker by specifying runtime: java in your app.yaml:

runtime: java
env: flex

The runtime image gcr.io/google-appenine/openjdk:8 will be automatically selected if you are attempting to deploy a JAR (*.jar file).

To select a jdk version, use the runtime_config.jdk field in app.yaml. Supported JDK versions include openjdk8 and openjdk11.

runtime: java
env: flex
runtime_config:
  jdk: openjdk8

If you want to use the image as a base for a custom runtime, you can specify runtime: custom in your app.yaml and then write the Dockerfile like this:

FROM gcr.io/google-appengine/openjdk
COPY your-application.jar $APP_DESTINATION

That will add the JAR in the correct location for the Docker container.

Once you have this configuration, you can use the Google Cloud SDK to deploy this directory containing the 2 configuration files and the JAR using:

gcloud app deploy app.yaml

Kubernetes Engine & other Docker hosts

For other Docker hosts, you'll need to create a Dockerfile based on this image that copies your application code and installs dependencies. For example:

FROM gcr.io/google-appengine/openjdk
COPY your-application.jar $APP_DESTINATION

You can then build the docker container using docker build or Google Cloud Container Builder. By default, the CMD is set to run the application JAR. You can change this by specifying your own CMD or ENTRYPOINT.

Container Memory Limits

The runtime will try to detect the container memory limit by looking at the /sys/fs/cgroup/memory/memory.limit_in_bytes file, which is automatically mounted by Docker. However, this may not work with other container runtimes. In those cases, to help the runtime compute accurate JVM memory defaults when running on Kubernetes, you can indicate memory limit through the Downward API.

To do so add an environment variable named KUBERNETES_MEMORY_LIMIT (This name is subject to change) with the value limits.memory and the name of your container. For example:

apiVersion: v1
kind: Pod
metadata:
  name: dapi-envars-resourcefieldref
spec:
  containers:
    - name: java-kubernetes-container
      image: gcr.io/google-appengine/openjdk
      resources:
        requests:
          memory: "32Mi"
        limits:
          memory: "64Mi"
      env:
        - name: KUBERNETES_MEMORY_LIMIT
          valueFrom:
            resourceFieldRef:
              containerName: java-kubernetes-container
              resource: limits.memory

The Default Entry Point

Any arguments passed to the entry point that are not executable are treated as arguments to the java command:

$ docker run openjdk -jar /usr/share/someapplication.jar

Any arguments passed to the entry point that are executable replace the default command, thus a shell could be run with:

> docker run -it --rm openjdk bash
root@c7b35e88ff93:/# 

Entry Point Features

The entry point for the openjdk8 image is docker-entrypoint.bash, which does the processing of the passed command line arguments to look for an executable alternative or arguments to the default command (java).

If the default command (java) is used, then the entry point sources the setup-env.d/, which looks for supported features to be enabled and/or configured. The following table indicates the environment variables that may be used to enable/disable/configure features, any default values if they are not set:

Env Var Description Type Default
PROFILER_ENABLE Stackdriver Profiler boolean false
TMPDIR Temporary Directory dirname
JAVA_TMP_OPTS JVM tmpdir args JVM args -Djava.io.tmpdir=${TMPDIR}
GAE_MEMORY_MB Available memory size Set by GAE or /proc/meminfo-400M
HEAP_SIZE_RATIO Memory for the heap percent 80
HEAP_SIZE_MB Available heap size ${HEAP_SIZE_RATIO}% of ${GAE_MEMORY_MB}
JAVA_HEAP_OPTS JVM heap args JVM args -Xms${HEAP_SIZE_MB}M -Xmx${HEAP_SIZE_MB}M
JAVA_GC_OPTS JVM GC args JVM args -XX:+UseG1GC plus configuration
JAVA_USER_OPTS JVM other args JVM args
JAVA_OPTS JVM args JVM args See below
SHUTDOWN_LOGGING_THREAD_DUMP Shutdown thread dump boolean false
SHUTDOWN_LOGGING_HEAP_INFO Shutdown heap info boolean false
SHUTDOWN_LOGGING_SAMPLE_THRESHOLD Shutdown sampling percent 100

If not explicitly set, JAVA_OPTS is defaulted to

JAVA_OPTS:=-showversion \
           ${JAVA_TMP_OPTS} \
           ${PROFILER_AGENT} \
           ${JAVA_HEAP_OPTS} \
           ${JAVA_GC_OPTS} \
           ${JAVA_USER_OPTS}

The command line executed is effectively (where $@ are the args passed into the docker entry point):

java $JAVA_OPTS "$@"

JVM Shutdown Diagnostics

This feature is not enabled by default.

Sometimes it's necessary to obtain diagnostic information when the JVM is stopped using SIGTERM or docker stop. This may happen on App Engine flexible environment, when the autohealer decides to kill unhealthy VMs that have an app that is unresponsive due to a deadlock or high load and stopped returning requests, including health checks.

To help diagnose such situations the runtime provides support for outputting a thread dump and/or heap info upon JVM shutdown forced by the TERM signal.

The following environment variables should be used to enable app container shutdown reporting (must be set to true or false):

SHUTDOWN_LOGGING_THREAD_DUMP - output thread dump

SHUTDOWN_LOGGING_HEAP_INFO - output heap information

If enabled, the runtime provides a wrapper for the JVM that traps SIGTERM, and runs debugging tools on the JVM to emit the thread dump and heap information to stdout.

If you're running many VMs, sampling is supported by using the environment variable SHUTDOWN_LOGGING_SAMPLE_THRESHOLD which is an integer between 0 and 100. 0 means no VMs report logs, 100 means all VMs report logs. (If this env var is not set, we default to reporting for all VMs).

The Default Command

The default command will attempt to run app.jar in the current working directory. It's equivalent to:

$ docker run openjdk java -jar app.jar
Error: Unable to access jarfile app.jar

The error is normal because the default command is designed for child containers that have a Dockerfile definition like this:

FROM openjdk
ADD my_app_0.0.1.jar app.jar

Development Guide

Contributing changes

Licensing

openjdk-runtime's People

Contributors

ajkannan avatar androbin avatar aozarov avatar balopat avatar bendory avatar cassand avatar chanseokoh avatar dlorenc avatar dominickramer avatar donmccasland avatar gregw avatar hegemonic avatar imjasonh avatar janbartel avatar jboynes avatar jeadorf avatar joakime avatar joaoandremartins avatar ludoch avatar meltsufin avatar mohamedbassem avatar nkubala avatar pmaloogoogle avatar rauls5382 avatar rroller avatar sharifelgamal avatar shreejad avatar yubin154 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

openjdk-runtime's Issues

Local integration test fails on Jenkins

There's some issue with memory calculation.

Successfully tagged openjdk-local-integration:latest
Starting app container...
Shutdown logging threshold of 100% satisfied with sample 46.
Start command: /shutdown/shutdown-wrapper.bash java -showversion -Xms0M -Xmx0M -XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:+PrintCommandLineFlags -jar app.jar
Invalid maximum heap size: -Xmx0M
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Use GAE_MEMORY_MB * 0.8 for JVM heap default

The default HEAP_SIZE configuration in setup-env.bash should be set as follows:
If GAE_MEMORY_MB is present:
heap_size = GAE_MEMORY_MB * 0.8
Otherwise
heap_size = (/proc/meminfo - 400MB) * 0.8

Build fails because openjdk-8-jre is not found

$mvn install
...
[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:0.4.13:build (build) on project openjdk8: Exception caught: The command '/bin/sh -c echo 'deb http://httpredir.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list  && apt-get -q update  && apt-get -y -q --no-install-recommends install     ca-certificates     openjdk-8-jre=8u102'-*'     netbase     wget     unzip  && apt-get clean  && rm /var/lib/apt/lists/*_*' returned a non-zero code: 100 -> [Help 1]

More detailed:

$/bin/sh -c echo 'deb http://httpredir.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list  && apt-get -q update  && apt-get -y -q --no-install-recommends install     ca-certificates     openjdk-8-jre=8u102'-*'     netbase     wget     unzip  && apt-get clean  && rm /var/lib/apt/lists/*_*

Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB]
Ign http://httpredir.debian.org jessie InRelease
Get:2 http://httpredir.debian.org jessie-updates InRelease [145 kB]
Get:3 http://httpredir.debian.org jessie Release.gpg [2373 B]
Get:4 http://security.debian.org jessie/updates/main amd64 Packages [407 kB]
Get:5 http://httpredir.debian.org jessie Release [148 kB]
Get:6 http://httpredir.debian.org jessie-updates/main amd64 Packages [17.6 kB]
Get:7 http://httpredir.debian.org jessie/main amd64 Packages [9064 kB]
Fetched 9847 kB in 2s (3552 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
E: Unable to locate package openjdk-8-jre

Do we still need the upgrade to OpenSSL 1.0.2 in the Dockerfile

If this is still needed, maybe it belongs in the base image.

# Upgrade to OpenSSL 1.0.2 (via sid)
ADD sid.list /etc/apt/sources.list.d/
RUN apt-get -y update \
 && apt-get -y -q --no-install-recommends install \
    libssl1.0.2 \
    openssl \
# Cleanup sid references
 && rm /etc/apt/sources.list.d/sid.list \
 && apt-get -y update \
# Cleanup apt-get temporary files
 && apt-get -y -q upgrade \
 && apt-get -y -q autoremove

Should stackdriver logging support be provided?

It has been discussed as part of GoogleCloudPlatform/jetty-runtime#68 (and it's associated PR GoogleCloudPlatform/jetty-runtime#81) that container independent parts of the logging configuration should eventually be moved from the jetty-runtime to the openjdk-runtime.

This would potentially involve: google-cloud-java jars and their dependencies being placed in the image; a configuration file that would be able to be turned on via an environment variable.

However, on consideration, I am not sure if doing the above is all that valuable. With modern build tools, assembling the jars necessary for an application is simple. It is actually more difficult to merge in a classpath of image-provided jars with an application, as there may be duplicates and version differences that need to be resolved (exactly the task that application build tools are best at).

Furthermore, turning on logging with a simple env variable will seldom be sufficient. Users will often wish to configure their logging with a custom logging.properties file, so a provided one is of less value.

We are providing stackdriver logging in the jetty-runtime because we are providing the container and we are responsible for the containers logs. However, we do hide the stackdriving classes from the application and it is free to provide it's own different mechanisms for logging and their dependencies. If the users of openjdk-runtime are treated like the users of the jetty-runtime, then they are responsible for their own application logging. Of course google-cloud-logging is easily available and should be well documented to make logging to stackdriver simple if the application chooses to do so.

Thus I'm in favour of leaving stackdriver logging only in the jetty-runtime.

Install kubectl for CI builds

The integration test on GKE need kubectl to be installed in order to deploy the test application.
The kubectl executable is currently not available during CI build and need to be installed in the cloud-init script.

Network is unreachable.

Hello.

Not sure if i should file the issue here or where, however, in my pet project I cannot seem to be able to make networking calls outside of the docker container once deployed to the GAE.

Here is the stack trace:

18:32:06.000 2017-01-14 02:32:06.670 [vert.x-eventloop-thread-1] ERROR io.vertx.ext.web.authentication.GoogleUserDataRetriever - Connection error:Network is unreachable: www.googleapis.com/2607:f8b0:4001:c0d:0:0:0:5f:443
18:32:06.000 io.netty.channel.AbstractChannel$AnnotatedSocketException: Network is unreachable: www.googleapis.com/2607:f8b0:4001:c0d:0:0:0:5f:443

My Dockerfile is the following:

FROM gcr.io/google_appengine/openjdk8
RUN useradd -ms /bin/bash vertx
RUN mkdir /app
WORKDIR /app
COPY build/distributions/server.zip /app
RUN unzip /app/server.zip
RUN mv /app/server/* /app
RUN rm -rf /app/server
RUN chown -R vertx /app
USER vertx
EXPOSE 8080
ENV APP_HOME=/app
ENTRYPOINT /app/bin/server

And my app.yaml is this:

runtime: custom
env: flex

handlers:
- url: /.*
  script: this field is required, but ignored
  secure: always

# [START env_variables]
env_variables:
  USE_GAE_MEMCACHE: 1
# [END env_variables]
# [END appyaml]

Setup local integration test with the runtimes-common integration testing

Via running the gcr.io/gcp-runtimes/integration_test image locally (same functionality as in integration_test.yaml), we could actually iterate much faster when implementing changes on the test-app.
Currently the local_integration_test.sh is only testing whether the app starts and verifies the shutdown logs.

cdbg_extra_class_path arguments are invalid for Jar based deployments

The image passes:
--cdbg_extra_class_path=/webapps/root/WEB-INF/classes:/webapps/root/WEB-INF/lib

as a fragment of:

java -showversion -agentpath:/opt/cdbg/cdbg_java_agent.so=--log_dir=/var/log/app_engine,--logtostderr=false,--cdbg_extra_class_path=/webapps/root/WEB-INF/classes:/webapps/root/WEB-
INF/lib -Xms491M -Xmx491M -XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:+PrintCommandLineFlags -jar /app/spring-boot-simple-5.0.jar

which causes the /var/log/app_engine/cdbg_java_agent.INFO to contain:

cdbg_java_agent.8c8c13cfad57.invalid-user.log.INFO.20170228-155954.1  cdbg_java_agent.8c8c13cfad57.invalid-user.log.WARNING.20170228-155955.1  cdbg_java_agent.INFO  cdbg_java_agent.WARNING  custom_logs
root@8c8c13cfad57:/# cat /var/log/app_engine/cdbg_java_agent.INFO
Log file created at: 2017/02/28 15:59:54
Running on machine: 8c8c13cfad57
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0228 15:59:54.725625    15 jvmti_globals.cc:336] Build time: Jan  3 2017 09:10:03
I0228 15:59:54.725790    15 jvmti_agent.cc:157] Java debuglet initialization started
I0228 15:59:54.726048    15 jvmti_agent.cc:191] Java debuglet initialization completed
I0228 15:59:54.804965    15 jvmti_agent.cc:202] Java VM started
I0228 15:59:54.809967    15 jvmti_agent.cc:212] JvmtiAgent::JvmtiOnVMInit initialization time: 5007 microseconds
I0228 15:59:54.810044    26 jvmti_agent_thread.cc:99] Agent thread started: CloudDebugger_main_worker_thread
I0228 15:59:54.810212    26 jvm_internals.cc:376] Loading internals from /opt/cdbg/cdbg_java_agent_internals.jar
I0228 15:59:55.224081    26 jni_logger.cc:31] Initializing ClassPathLookup, default classpath: true, extra classpath: [/webapps/root/WEB-INF/classes, /webapps/root/WEB-INF/lib], config: null
I0228 15:59:55.286089    26 jni_logger.cc:31] Total size of indexed resources database: 1432 bytes
I0228 15:59:55.329637    26 jvm_internals.cc:132] ClassPathLookup constructor time: 118480 microseconds
W0228 15:59:55.777546    26 jni_logger.cc:46] Failed to compute hash of application file /webapps/root/WEB-INF/classes
java.nio.file.NoSuchFileException: /webapps/root/WEB-INF/classes
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
        at java.nio.file.Files.newByteChannel(Files.java:361)
        at java.nio.file.Files.newByteChannel(Files.java:407)
        at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
        at java.nio.file.Files.newInputStream(Files.java:152)
        at com.google.devtools.cdbg.debuglets.java.UniquifierComputer.appendBinaryFile(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.UniquifierComputer.append(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.UniquifierComputer.<init>(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.ClassPathLookup.computeDebuggeeUniquifier(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.GcpHubClient.getDebuggeeInfo(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.GcpHubClient.registerDebuggee(Unknown Source)
W0228 15:59:55.777968    26 jni_logger.cc:46] Failed to compute hash of application file /webapps/root/WEB-INF/lib
java.nio.file.NoSuchFileException: /webapps/root/WEB-INF/lib
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
        at java.nio.file.Files.newByteChannel(Files.java:361)
        at java.nio.file.Files.newByteChannel(Files.java:407)
        at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
        at java.nio.file.Files.newInputStream(Files.java:152)
        at com.google.devtools.cdbg.debuglets.java.UniquifierComputer.appendBinaryFile(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.UniquifierComputer.append(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.UniquifierComputer.<init>(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.ClassPathLookup.computeDebuggeeUniquifier(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.GcpHubClient.getDebuggeeInfo(Unknown Source)
        at com.google.devtools.cdbg.debuglets.java.GcpHubClient.registerDebuggee(Unknown Source)
I0228 15:59:57.652278    26 jni_logger.cc:31] Debuggee gcp:1031550152456:8402ef630c825e73 registered: {"debuggee":{"id":"gcp:1031550152456:8402ef630c825e73","project":"1031550152456","uniquifier":"8DA069F1CA0B61EAFC6587E57F9FEE9411FF23E4","descr
iption":"paflynn-demo-project","agentVersion":"google.com/java-gcp/@2"}}, agent version: 2.8
I0228 15:59:57.652299    26 jvmti_agent.cc:414] Attaching Java debuglet
I0228 15:59:57.652760    26 rate_limit.cc:122] CPU count: 1
I0228 15:59:57.652768    26 debugger.cc:97] Initializing Java debuglet

It's unclear what impact this has, as CDB appears broken for Java Flex for other reasons.

Test the Integration with Cloud Debugger

The test could be something simple like

  • deploy a sample app
  • set a snapshot (aka a breakpoint) via the API or gcloud
  • exercise the breakpoint
  • verify that the breakpoint has been hit

Test the Integration with Cloud Logging

Tests should verify that the contract with App Engine flex is obeyed - logs sent by an application to stderr and stdout should be viewable in the cloud logging API, or via gcloud.

DRY Dockerfiles via publishing the generated version

As @aslo pointed out, with the approach on docker-library, we could avoid duplications like this between our dockerfiles.

We could go as easy as providing a separate folder where the generated Dockerfile is visible for understandability, or as far as (I assume that's the goal in docker-library) the generated Dockerfile sits in the full folder structure, thus you can build the image using only docker.

I'd argue, this is a worthwile exercise to do, so that you don't need maven to build the image. You'd need maven only to generate the "image source", what is published to github as an artifact itself.

Create /var/log/app_engine

The cloud debugger (/opt/cdbg/format-env-appengine-vm.sh) assumes that the directory /var/log/app_engine exists, so this needs to be created in the Dockerfile

jmap -heap <pid> command crashes

Some googling indicates that the openjdk-debuginfo package might be missing.

root@37d7e4efd82b:/# jmap -heap 1
Attaching to process ID 1, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.121-b13
using thread-local object allocation.
Garbage-First (G1) GC with 1 thread(s)
Heap Configuration:
   MinHeapFreeRatio         = 40
   MaxHeapFreeRatio         = 70
   MaxHeapSize              = 515899392 (492.0MB)
   NewSize                  = 1363144 (1.2999954223632812MB)
   MaxNewSize               = 309329920 (295.0MB)
   OldSize                  = 5452592 (5.1999969482421875MB)
   NewRatio                 = 2
   SurvivorRatio            = 8
   MetaspaceSize            = 21807104 (20.796875MB)
   CompressedClassSpaceSize = 1073741824 (1024.0MB)
   MaxMetaspaceSize         = 17592186044415 MB
   G1HeapRegionSize         = 1048576 (1.0MB)
Heap Usage:
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.tools.jmap.JMap.runTool(JMap.java:201)
        at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: java.lang.RuntimeException: unknown CollectedHeap type : class sun.jvm.hotspot.gc_
interface.CollectedHeap
        at sun.jvm.hotspot.tools.HeapSummary.run(HeapSummary.java:144)
        at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)
        at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
        at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:49)
        ... 6 more

What's the purpose of the reference to $JETTY_BASE in debugger setup?

setup-env.bash has this block of code to set up the Stackdriver Debugger agent. The line that I'm confused about is DBG_AGENT="$( RUNTIME_DIR=$JETTY_BASE /opt/cdbg/format-env-appengine-vm.sh )". Why are we referencing $JETTY_BASE here? It wouldn't be defined for a vanilla Java app.

DBG_AGENT=
if isTrue "${DBG_ENABLE:=$( if [[ -z ${CDBG_DISABLE} ]] ; then echo true; else echo false ; fi )}" ; then
  if [[ "$GAE_PARTITION" = "dev" ]]; then
    echo "Running locally and DBG_ENABLE is set, enabling standard Java debugger agent"
    DBG_AGENT="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=${DBG_PORT:=5005}"
  else
    unset CDBG_DISABLE
    DBG_AGENT="$( RUNTIME_DIR=$JETTY_BASE /opt/cdbg/format-env-appengine-vm.sh )"
  fi
fi

Let's come up with a better solution to cloud build locally

See #121 for more details where this came up.

In the "local emulation" of the cloud build a python script is pulled in from the python runtime - which is not an ideal solution necessarily - especially that the old version (the one I switched back to) worked, the new version doesn't, as it got modularized in this commit .

Let's find a better way to emulate cloud build locally!

Options might be:

  1. "curl" all of the dependent scripts in at runtime (ugly)
  2. write our own fork of these scripts
  3. lift these "cloud-scripts" into a separate projects which could be reused across runtime projects?

Shutdown Logging Support

TL;DR: On a VM shutdown, allow logging the application’s thread dump and heap info to Cloud Logging.

On App Engine, sometimes autohealer decides to kill unhealthy VMs. In that case we don't have sufficient logging done from the VM to debug if it was an issue with the app or the VM. This can happen when the app code gets stuck (like deadlock) and stopped returning requests, including health checks.

Because it is possible that the JVM is ailing, it might not have enough resources to log to Cloud Logging directly. So we will be using stdout + fluentd to capture app container logs.

Explicit user intent is required for potentially sensitive information. The following environment variables will be used to enable/disable app container shutdown reporting (must be set to true or false):

SHUTDOWN_LOGGING_THREAD_DUMP - save thread traces
SHUTDOWN_LOGGING_HEAP_INFO - save heap information

The runtime will capture SIGTERM, and run debugging tools on the JVM to emit the thread traces and heap information to stdout.

Further, for customers with a small number of VMs we default to reporting shutdown logs for all VMs. For larger customers running hundreds or thousands of instances, we would allow them to be able to sample logs. This will be done by using the environment variable SHUTDOWN_LOGGING_SAMPLE_THRESHOLD which is an int between 0 and 100. 0 means no VMs report logs, 100 means all VMs report logs. (If this env var is not set, we default to reporting for all VMs). The runtime will then generate a random number between 0 (inclusive) and 100 (exclusive) and if it’s less than the threshold set, it should do the trap-and-log logic described above.

@karan

Fix flaky monitoring integration test

The monitoring integration test is flaky.
The following are the main reasons:

  1. The driver can't find the custom metric

BadRequest: 400 The provided filter doesn't refer to any known metric. (GET https://monitoring.googleapis.com/v3/projects/java-runtime-test/timeSeries/?filter=metric.type+%3D+%22custom.googleapis.com%2FUzLlwfHpNOCsZSuu%22&interval.endTime=2017-08-03T21%3A13%3A00.000000Z&interval.startTime=2017-08-03T21%3A11%3A00.000000Z)

Solution: reproduce the error and identify a workaround (work out timings, retries, double check if the metric is stored, etc.)

  1. The gRPC call fails while storing the metric

exit code: 500, text: {"timestamp":1501793413295,"status":500,"error":"Internal Server Error","exception":"com.google.api.gax.grpc.ApiException","message":"io.grpc.StatusRuntimeException: INTERNAL: HTTP/2 error code: INTERNAL_ERROR\nReceived Rst Stream","path":"/monitoring"}

Solution: file a bug with the Stackdriver team and hunt down the root cause, and in the mean time catch the exception and retry (maybe double check if the metric was stored)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.