Is your feature request related to a problem? Please describe.

Currently in order to skip server certificate verification, we need to setup an ssl_alternative_hostname and a ca_bundle_dir (or file).

Feature Description

The ssl_alternative_hostname might not be working in all cases, so we should add a simple boolean flag (InsecureSkipVerify) to skip all server certificate validation (which is what most users want) like most other integrations have,

Priority

[Really Want]

Is your feature request related to a problem?

CI failed randomly due to a test that raced

Describe Alternatives

A clear and concise description of any alternative solutions or features you've considered. Are there examples you could link us to?

Priority

Nice to Have

Additional context

This is the error that happened twice

Repolinter Report

🤖This issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.

This Repolinter run generated the following results:

• Fail
  • ❌ readme-starts-with-community-plus-header
• Passed
  • ✅ license-file-exists
  • ✅ readme-file-exists
  • ✅ readme-contains-link-to-security-policy
  • ✅ readme-contains-discuss-topic
  • ✅ code-of-conduct-should-not-exist-here
  • ✅ third-party-notices-file-exists

Fail #

❌ readme-starts-with-community-plus-header #

The README of a community plus project should have a community plus header at the start of the README. If you already have a community plus header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/. Below is a list of files or patterns that failed:

• README.md: The first 5 lines do not contain the pattern(s): Open source Community Plus header (see https://opensource.newrelic.com/oss-category).
  • 🔨 Suggested Fix: prepend the latest code snippet found at https://github.com/newrelic/opensource-website/wiki/Open-Source-Category-Snippets#code-snippet-2 to file

Passed #

Description of the problem

Increase hardcoded index limit to 500.

OS

• All of them
• Amazon Linux, all versions
• CentOS, version 6 or higher
• Debian, version 7 ("Wheezy") or higher
• Red Hat Enterprise Linux (RHEL), version 6 or higher
• Ubuntu, versions 12.04, 14.04, and 16.04 (LTS versions)
• Windows Server, 2008 and 2012 and their service packs

Description

It is difficult to configure the elasticsearch integration because if you make changes the only indication that if something is working or not is by checking whether data is making it into Newrelic.

What I would like to see is some logging output that would tell me if the nri-elasticsearch plugin is actually pulling data or if not then why not. Like a verbose option to say if it is getting no data, or timing out, or seeing a SSL error, and if it is getting data then optionally telling how much data it is pulling in bytes or number of characters or something like that.

I don't know if the nri-elasticsearch is responsible for this while running under newrelic infra or if this is the correct place to request it. But I figure it is a good idea to put it out there because it does cause a lot of people to trip up. While talking to other people in my org they all experienced mysterious "no data" results that were confusing at different times.

Acceptance Criteria

Enabling a optional verbose mode while running under newrelic infra or, if such a option exists, documenting how to use it.

Describe Alternatives

Executing the nri-elasticsearch on the command line with the options you would put in /etc/newrelic-infra/integrations.d/elasticsearch-config.yml is useful at troubleshooting issues. But this is sub-optimal because with modern cloud-based infrastructure it is not always easy to get on the host running the integration to test things manually.

Dependencies

Probably something to do with newrelic infra agent itself. I don't know how logging works for plugins like this.

Additional context

Estimates

I am hoping that it is a "S". Just adding a env option that can be used in elasticsearch-config.yml or in a container that would give us better logging.

For Maintainers Only or Hero Triaging this bug

Suggested Priority (P1,P2,P3,P4,P5):
Suggested T-Shirt size (S, M, L, XL, Unknown):

The NRI rpm places executables on a filesystem that is recommended to be mounted as noexec.

Description

The New Relic Elasticsearch was installed via rpm from repository:

baseurl=https://download.newrelic.com/infrastructure_agent/linux/yum/el/7/$basearch

The rpm contains following files:

# rpm -qlv nri-elasticsearch-4.3.3-1.x86_64
-rw-r--r--    1 root    root                     1757 Nov 27 02:00 /etc/newrelic-infra/integrations.d/elasticsearch-config.yml.sample
-rwxr-xr-x    1 root    root                  7009219 Nov 27 02:00 /var/db/newrelic-infra/newrelic-integrations/bin/nri-elasticsearch
-rw-r--r--    1 root    root                      370 Nov 27 02:00 /var/db/newrelic-infra/newrelic-integrations/elasticsearch-definition.yml
-rw-r--r--    1 root    root                      380 Nov 27 02:00 /var/db/newrelic-infra/newrelic-integrations/elasticsearch-win-definition.yml 

System without noexec option on /var is identified as vulnerable with: unix-partition-mounting-weakness

On system with noexec option on /var the NRI Elasticsearch fails with:

fork/exec /var/db/newrelic-infra/newrelic-integrations/bin/nri-elasticsearch: permission denied

Expected Behavior

Executables are placed in /usr/bin or another place where execution is allowed.
The NRI starts and sends data on a system where /var is mounted with noexec option.

Steps to Reproduce

1. Set noexec option for /var

# grep '/var' /etc/fstab
UUID="<uuid>"     /var    ext4    defaults,nodev,noexec,nosuid,nofail     0 0

2. Remount the /var

# mount | grep '/var'
/dev/mapper/rootvg-varlv on /var type ext4 (rw,nosuid,nodev,noexec,relatime,seclabel,data=ordered)

3. Install Newrelic Infra and NRI Elasticsearch from download.newrelic.com

# rpm -qa | grep relic
newrelic-infra-1.14.2-1.el7.x86_64
# rpm -qa | grep nri
nri-elasticsearch-4.3.3-1.x86_64

4. Start NewRelic Infra

# systemctl restart newrelic-infra

5. Look for the error in journalctl

# journalctl -u newrelic-infra | grep denied

Your Environment

newrelic-infra-1.14.2-1.el7.x86_64
nri-elasticsearch-4.3.3-1.x86_64
System has set up noexec option for /var , /tmp , /var/tmp (that are in /etc/fstab).
RHEL 7.9
System is required to be CIS L2 compliant (https://www.cisecurity.org/cis-benchmarks/)
System has FIPS enabled (https://www.nist.gov/publications/security-requirements-cryptographic-modules-includes-change-notices-1232002)
System needs to pass vulnerability scans.

Repolinter Report

🤖This issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.

This Repolinter run generated the following results:

• Passed
  • ✅ license-file-exists
  • ✅ readme-file-exists
  • ✅ readme-starts-with-community-plus-header
  • ✅ readme-contains-link-to-security-policy
  • ✅ readme-contains-discuss-topic
  • ✅ code-of-conduct-should-not-exist-here
  • ✅ third-party-notices-file-exists

Passed #

Missing metrics on recent elasticsearch versions

Description

The following metrics are not being reported using the integration with elasticsearch7.17.1 or elascticsearch8.1.0:

• fs.bytesReadsInBytes
• fs.iOOperations
• fs.reads
• fs.writesInBytes
• fs.writeOperations
• get.requestsDcoumentExists
• get.requestsDcoumentMissing
• jvm.gc.majorCollectionsYoungGenerationObjects
• jvm.gc.majorCollectionsYoungGenerationObjectsInMilliseconds
• threadpool.activefetchShardStarted
• threadpool.bulkActive
• threadpool.bulkQueue
• threadpool.bulkRejected
• threadpool.bulkThreads
• threadpool.indexActive
• threadpool.indexQueue
• threadpool.indexRejected

Expected Behavior

Those metrics being reported in the same way are being reported for previous elasticsearch versions.

NR Diag results

Steps to Reproduce

• Run an elasticsearch cluster (7.17.1+) and install the nri-elasticsearch integration.
• This can be easily addressed using coreint-canaries.

Your Environment

• minikube kubernetes cluster for testing purposes.

Additional context

Some elasticsearch stats have changed in recent versions. Example:

Fragment from _nodes/stats request in elasticsearch6.0.0:

{
// ...
            "thread_pool": {
                "bulk": {
                    "active": 0,
                    "completed": 20,
                    "largest": 6,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 6
                },
                "fetch_shard_started": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "fetch_shard_store": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "flush": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "force_merge": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "generic": {
                    "active": 0,
                    "completed": 157,
                    "largest": 8,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 8
                },
                "get": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "index": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "listener": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "management": {
                    "active": 1,
                    "completed": 73,
                    "largest": 3,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 3
                },
                "ml_autodetect": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "ml_datafeed": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "ml_utility": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "refresh": {
                    "active": 0,
                    "completed": 92,
                    "largest": 1,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 1
                },
                "search": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "security-token-key": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "snapshot": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "warmer": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                },
                "watcher": {
                    "active": 0,
                    "completed": 0,
                    "largest": 0,
                    "queue": 0,
                    "rejected": 0,
                    "threads": 0
                }
            }
            "fs": {
                "data": [
                    {
                        "available_in_bytes": 48402522112,
                        "free_in_bytes": 51619237888,
                        "mount": "/ (overlay)",
                        "path": "/usr/share/elasticsearch/data/nodes/0",
                        "total_in_bytes": 62725623808,
                        "type": "overlay"
                    }
                ],
                "io_stats": {},
                "least_usage_estimate": {
                    "available_in_bytes": 48402067456,
                    "path": "/usr/share/elasticsearch/data/nodes/0",
                    "total_in_bytes": 62725623808,
                    "used_disk_percent": 22.835255326983585
                },
                "most_usage_estimate": {
                    "available_in_bytes": 48402067456,
                    "path": "/usr/share/elasticsearch/data/nodes/0",
                    "total_in_bytes": 62725623808,
                    "used_disk_percent": 22.835255326983585
                },
                "timestamp": 1647511922101,
                "total": {
                    "available_in_bytes": 48402522112,
                    "free_in_bytes": 51619237888,
                    "total_in_bytes": 62725623808
                }
            },
//...
}

Fragment from the same request for elasticsearch8.1.0:

{
// ...
            "thread_pool": {
                "analyze": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "auto_complete": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "azure_event_loop": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "ccr": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "fetch_shard_started": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "fetch_shard_store": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "flush": {
                    "threads": 1,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 1,
                    "completed": 3
                },
                "force_merge": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "generic": {
                    "threads": 11,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 11,
                    "completed": 577
                },
                "get": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "management": {
                    "threads": 2,
                    "queue": 0,
                    "active": 1,
                    "rejected": 0,
                    "largest": 2,
                    "completed": 85
                },
                "ml_datafeed": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "ml_job_comms": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "ml_utility": {
                    "threads": 2,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 2,
                    "completed": 376
                },
                "refresh": {
                    "threads": 1,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 1,
                    "completed": 307
                },
                "repository_azure": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "rollup_indexing": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "search": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "search_coordination": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "search_throttled": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "searchable_snapshots_cache_fetch_async": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "searchable_snapshots_cache_prewarming": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "security-crypto": {
                    "threads": 2,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 2,
                    "completed": 2
                },
                "security-token-key": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "snapshot": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "snapshot_meta": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "system_critical_read": {
                    "threads": 1,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 1,
                    "completed": 1
                },
                "system_critical_write": {
                    "threads": 3,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 3,
                    "completed": 3
                },
                "system_read": {
                    "threads": 3,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 3,
                    "completed": 94
                },
                "system_write": {
                    "threads": 3,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 3,
                    "completed": 45
                },
                "vector_tile_generation": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "warmer": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "watcher": {
                    "threads": 0,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 0,
                    "completed": 0
                },
                "write": {
                    "threads": 6,
                    "queue": 0,
                    "active": 0,
                    "rejected": 0,
                    "largest": 6,
                    "completed": 47
                }
            },
            "fs": {
                "timestamp": 1647506922692,
                "total": {
                    "total_in_bytes": 62725623808,
                    "free_in_bytes": 53532381184,
                    "available_in_bytes": 50315665408
                },
                "data": [
                    {
                        "path": "/usr/share/elasticsearch/data",
                        "mount": "/ (overlay)",
                        "type": "overlay",
                        "total_in_bytes": 62725623808,
                        "free_in_bytes": 53532381184,
                        "available_in_bytes": 50315665408
                    }
                ],
                "io_stats": {}
            }
// ...
}

While runnung nri-elasticsearch integration I'm getting this errors:
[DEBUG] Unsupported data type '[]interface {}' for yaml key discovery.seed_hosts
[DEBUG] Unsupported data type '[]interface {}' for yaml key cluster.initial_master_nodes

Description

My config file for elasticsearch have these lines:

discovery.seed_hosts:

• ip-address1
• ip-address2

cluster.initial_master_nodes:

• node-name1
• node-name2

Your Environment

Simple 2-node cluster

I've checked elastic docs for typos, etc, but found that it's fine

The infra agent now supports:

• Amazon Linux 2022 (Public Preview, GA expected by ~July)
• RHEL 9 (GA)
• Ubuntu 22.04 (GA)

Repolinter Report

🤖This issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.

This Repolinter run generated the following results:

• Passed
  • ✅ license-file-exists
  • ✅ readme-file-exists
  • ✅ readme-starts-with-community-plus-header
  • ✅ readme-contains-link-to-security-policy
  • ✅ readme-contains-forum-topic
  • ✅ code-of-conduct-should-not-exist-here
  • ✅ third-party-notices-file-exists

Passed #

Add the new newrelic-infra-checkers and the golangci and semgrep actions to the pipeline.

Description

Some metrics such as indices.indexingOperationsFailed are useful for monitoring but they are treated as a gauge and so it is a forever increasing value. For example after an incident it may stand at 12000 but stay at that value for a long time with everything working fine. However, because the value of 12000 is stored in New Relic One it's not something you can alert on. You cannot say, "Alert if the value increases again".

Expected Behavior

Usually these metrics would be a delta so you can alert on a delta value and also graph the failures over time and see 0s where everything is working fine.

NR Diag results

N/A

Steps to Reproduce

Install and look at failed indexing operations graph. Trigger some manual indexing that fails and see that the graph begins to have a large value and flat lines on that large value and never returns to 0.

Your Environment

Elasticsearch 7.x

Additional context

N/A

Description

In the New Relic integration dashboard created for this integration, the charts for "Time Spent Indexing Documents by Node " and "Time Spent Deleting Documents by Node" are missing units in the Y-axis labels.

Expected Behavior

As a user I expect to see units in the chart labels so that I can understand what the charts are telling me.

Steps to Reproduce

1. Install nri-elasticsearch integration.
2. Visit the dashboard in New Relic > Third Party Services > Elasticsearch.
3. Note that the two time-based charts do not show units (the unit seems to be milliseconds in both cases, as seen in https://github.com/newrelic/nri-elasticsearch/blob/master/spec.csv#L49).

Your Environment

• nri-elasticsearch 4.5.1
• newrelic-infra 1.20.7

Additional context

SC 23209

Description

The metric indices.numberIndices is miss leading since it currently sends:

(integer) The number of documents as reported by Lucene. This excludes deleted documents and counts any nested documents separately from their parents. It also excludes documents which were indexed recently and do not yet belong to a segment.

according to the api docs. And not the number of indices documented.

Expected Behavior

TBD

NR Diag results

Steps to Reproduce

Your Environment

Additional context

Description of the problem

Investigate using the name field rather than the hostname field for naming nodes.

OS

• All of them
• Amazon Linux, all versions
• CentOS, version 6 or higher
• Debian, version 7 ("Wheezy") or higher
• Red Hat Enterprise Linux (RHEL), version 6 or higher
• Ubuntu, versions 12.04, 14.04, and 16.04 (LTS versions)
• Windows Server, 2008 and 2012 and their service packs

Description

The metric fs.unallocatedBytesInBYtes has a capital Y in Bytes, while the other related metrics like fs.totalSizeInBytes do not.

Our documentation currently shows it as a lower case Y causing additional confusion:

https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/elasticsearch-monitoring-integration/#elasticsearch-node-metrics

Expected Behavior

Metric naming/case should be consistent

NR Diag results

N/A

Steps to Reproduce

Can see it in the code:

https://github.com/newrelic/nri-elasticsearch/search?q=unallocatedBytesInBYtes

Your Environment

N/A

Additional context

N/A

Description of the problem

Missing used_disk_percent metric

The used_disk_percent metric is missing from nri-elasticsearch. This is found in the fs parameter, an example from nodeStatsMetricsResult.json:

 		"fs": {
			"timestamp": 1533662802396,
			"total": {
				"total_in_bytes": 164705353728,
				"free_in_bytes": 162707742720,
				"available_in_bytes": 162707742720,
				"operations": 10,
				"read_kilobytes": 39485,
				"read_operations": 39486,
				"write_kilobytes": 298341,
				"write_operations": 2384089
			},
			"least_usage_estimate": {
				"path": "/var/lib/elasticsearch/nodes/0",
				"total_in_bytes": 164705353728,
				"available_in_bytes": 162707742720,
				"used_disk_percent": 1.2128391474747815
			},
			"most_usage_estimate": {
				"path": "/var/lib/elasticsearch/nodes/0",
				"total_in_bytes": 164705353728,
				"available_in_bytes": 162707742720,
				"used_disk_percent": 1.2128391474747815

Is it possible this could be added?

OS

• All of them
• Amazon Linux, all versions
• CentOS, version 6 or higher
• Debian, version 7 ("Wheezy") or higher
• Red Hat Enterprise Linux (RHEL), version 6 or higher
• Ubuntu, versions 12.04, 14.04, and 16.04 (LTS versions)
• Windows Server, 2008 and 2012 and their service packs

Issue

Even if the integration correctly reports data, the dashboards reachable through the entity explorer are not populated correclty.

However, those under third party dashboards and data explorer works.

Description

We have found an issue with the integration. It seems the entities are shown in NR1 Explorer, although they are the same as the ones in the 3rd Party dashboard, they are being indexed with a different Ids and that’s why the NR1 Explorer data is not showing up.

It is still possible to see all the data in NR1 through the 3rd party dashboard or the Data Explorer, just no directly through the Entity Explorer.

Expected Behavior

Charts should work in both UI

How to reproduce it

https://newrelic.atlassian.net/browse/GTSE-7910

wouldn't be possible to add index/_stats metrics in order to have deep stats for each index?

thanks

Description

Metrics and Inventory data is being sent in 2 "entities" in the same payload. Because they have different attributes the backend generates 2 identifiers.
The entity that carries the Inventory, gets tagged with the correct nr.entityType and so get's index and is visible in NR1.
The entity that carries the metrics does not get tagged with an nr.entityType attribute and so does NOT get indexed and is not visible in NR1.

This leads to the entities shown in the Explorer to not have metrics associated with it, and the metrics shown in the 3rd Party dashboard in the Infrastructure view correspond to the entities that are not shown in the Explorer.

Expected Behavior

I would expect that the entities shown in the Explorer to have the same metrics as shown in the 3rd Party dashboard in Infrastructure.

Additional context

This seems to be caused by the fact that metrics and inventory are being set in different entities created in different ways:

Metrics:

Inventory:

If a monitored Elastichsearch cluster has a "closed" index then the integration command fails.

Description

$ /var/db/newrelic-infra/custom-integrations/bin/nri-elasticsearch [...]
[INFO] Collecting cluster metrics.
[INFO] Collecting node metrics
[INFO] Collecting common metrics.
[ERR] There was an error populating metrics for common metrics: status code 400 - received error of type 'index_closed_exception' from Elasticsearch: closed
[INFO] Collecting indices metrics
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x6b9c4f]

goroutine 1 [running]:
main.setIndicesStatsMetricsResponse(0xc0001282c0, 0xc000534000, 0x31, 0x3f, 0x0, 0xc0003fb0c0, 0x18, 0x0)
	/go/src/github.com/newrelic/nri-elasticsearch/metrics.go:164 +0x18f
main.populateIndicesMetrics(0xc0001282c0, 0x7ad960, 0xc000116bc0, 0x0, 0xc0003fb0c0, 0x18, 0x1, 0x1)
	/go/src/github.com/newrelic/nri-elasticsearch/metrics.go:131 +0x192
main.populateMetrics(0xc0001282c0, 0x7ad960, 0xc000116bc0, 0x704909, 0x0)
	/go/src/github.com/newrelic/nri-elasticsearch/metrics.go:33 +0x311
main.main()
	/go/src/github.com/newrelic/nri-elasticsearch/elasticsearch.go:50 +0x14c

Expected Behavior

Closed indices don't prevent metrics collection.

Steps to Reproduce

Run nri-elasticsearch with an Elasticsearch cluster having a closed index.

Your Environment

nri-elasticsearch v4.3.3 with Elasticsearch v7.9.0

Description of the problem

When using SearchGuard, all of the cluster listeners use https. This works when the hostname matches the certificate, but in HA/cloud environments there is often a load balancer (or DNS alias) in front of a pool of nodes instead. This means that the request is to localhost (or even to the internal name) but the certificate is from foo.bar.baz.

Without a flag to "accept any certificate", the integration cannot connect to the node.

OS

• All of them
• Amazon Linux, all versions
• CentOS, version 6 or higher
• Debian, version 7 ("Wheezy") or higher
• Red Hat Enterprise Linux (RHEL), version 6 or higher
• Ubuntu, versions 12.04, 14.04, and 16.04 (LTS versions)
• Windows Server, 2008 and 2012 and their service packs

Elasticsearch agent is not able to capture cluster and node level metrics in k8s env but in local system or docker env its able to capture all the metrics.

Env - Robin kubernetes environment.

APM Agent Language - nri-elasticsearch, nri-couchbase - Elasticsearch and Couchbase Infra Agent
APM Agent Version - v4.3.4
Operating System - kubernetes
Operating System Version
Frameworks your app is using - Collector is in Java and database storage is elasticsearch.

Description

NR doc says that indices.numberIndices is "The number of documents across all primary shards assigned to the node"
The metric name is wrong and misleading.

Expected Behavior

Rename indices.numberIndices to indices.numberDocs or something similar to reflect the proper name.

Repolinter Report

🤖This issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.

This Repolinter run generated the following results:

• Fail
  • ❌ readme-starts-with-community-plus-header
  • ❌ code-of-conduct-file-does-not-exist
• Passed
  • ✅ license-file-exists
  • ✅ readme-file-exists
  • ✅ readme-contains-link-to-security-policy
  • ✅ readme-contains-discuss-topic
  • ✅ third-party-notices-file-exists

Fail #

❌ readme-starts-with-community-plus-header #

The README of a community plus project should have a community plus header at the start of the README. If you already have a community plus header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/. Below is a list of files or patterns that failed:

• README.md: The first 5 lines do not contain the pattern(s): Open source Community Plus header (see https://opensource.newrelic.com/oss-category).
  • 🔨 Suggested Fix: prepend [![Community Plus header](https://github.com/newrelic/opensource-website/raw/master/src/images/categories/Community_Plus.png)](https://opensource.newrelic.com/oss-category/#community-plus) to file

❌ code-of-conduct-file-does-not-exist #

New Relic has moved the CODE_OF_CONDUCT file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT file in a repository is now redundant and should be removed. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Found files. Below is a list of files or patterns that failed:

• CODE_OF_CONDUCT.md
  • 🔨 Suggested Fix: Remove file

Passed #

Elasticsearch Integration failed with this error:

install error encountered: encountered an error while validating receipt of data for elasticsearch-open-source-integration: timed out waiting for validation to succeed

Description

I have tried to install newrelics agent and elastic search integration agent on a ubuntu machine. It installed 'Infrastructure Agent' and 'Logs Integration' but failed to install 'Elasticsearch Integration'

Expected Behavior

It should install the 'Elasticsearch Integration' agent on the machine

Troubleshooting or NR Diag results

Failure - Infra/Config/IntegrationsMatch
Found matching integration files with some errors:
Integration Configuration File '/etc/newrelic-infra/integrations.d/elasticsearch-config.yml' is missing key 'integration_name'
See https://docs.newrelic.com/docs/integrations/integrations-sdk/getting-started/integration-file-structure-activation for more information.
Provide any other relevant log data.
TIP: Scrub logs and diagnostic information for sensitive information

Steps to Reproduce

Please be as specific as possible.
Run this command
curl -Ls https://download.newrelic.com/install/newrelic-cli/scripts/install.sh | bash && sudo NEW_RELIC_API_KEY=<KEY> NEW_RELIC_ACCOUNT_ID=<ACCOUNT_ID> /usr/local/bin/newrelic install

Your Environment

20.04.2-Ubuntu

We have a situation where ES is not bound to localhost, but another host so I believe this should either accept the hostname attribute, or another attribute for what is considered "localhost".

Description of the problem

Increase index limit to 1000 or 1500.

We've reached a limit in the amount of indices we can send to NR through the ES integration. We currently compiled a version with the limit increased to 1000 indices and it seems to be working fine.
Not sure if this limit is because of message size limitations in the call to NR but would it be possible to get it increased? I can open up a PR if needed.

OS

• All of them
• Amazon Linux, all versions
• CentOS, version 6 or higher
• Debian, version 7 ("Wheezy") or higher
• Red Hat Enterprise Linux (RHEL), version 6 or higher
• Ubuntu, versions 12.04, 14.04, and 16.04 (LTS versions)
• Windows Server, 2008 and 2012 and their service packs

Description of the changes
This new option fulfills requirement mentioned at https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/elasticsearch-monitoring-integration#config:

Only one node agent should use the all command, as metrics are collected for the whole cluster.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Pending Branch Automerge

These updates await pending status checks before automerging. Click on a checkbox to abort the branch automerge, and create a PR instead.

• fix(deps): update module github.com/stretchr/testify to v1.9.0

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update golang version to v1.22.0 (go, golang)
• fix(deps): update module gopkg.in/yaml.v2 to v3

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository