Hello,

You can have the same configuration file for hmc and nmon.

Could you remove all hmc parameters from your configuration file and run the same command?

It's not mandatory to pUT parameters in the configuration file.

Here I would say it's not the good binary version but let check what is the output when running the command.

Thanks for submitting the issue.

Alain

from nmon2influxdb.

Hi Alain,

Thanks for quick response.
Used combined configuration for both nmon and hmc and also tested by removing hmc configurations. Was able to get output only by removing hmc parameters. Based on requirement, we are looking for solutions to include hmc outputs in performance reports.

Thanks,
Madhu

from nmon2influxdb.

Hi Alain,
Tested .nmon2influxdb.conf with nmon configuration only. The output is loaded to influxdb.
Also tested .nmon2influxdb.conf with combined configuration of nmon and hmc, and it gave errors as attached. Is this required to run separate commands for hmc and nmon output?
Could you please share idea on hmc configuration part?
hmc_nmon_conf.docx
Hmc addition is done as:

HMC parameters

hmc_server=""
hmc_user=""
hmc_password=""
hmc_managed_system="mysystem"
hmc_database="nmon2influxdbHMC"
hmc_data_retention="40d"

from nmon2influxdb.

Hello,

Sorry I misunderstood your previous answer. I thought the problem was solved. Thanks for your explanations.

I used version 2.1.0 with the exact same parameters you gave me and here is my output:

./nmon2influxdb-macos-amd64 hmc import
Updating  autogen retention policy to keep only the last 40d days. Timestamp based.
2017/01/04 15:37:16 HMC error sending auth request: Put https://:12443/rest/api/web/Logon: dial tcp :12443: getsockopt: connection refused

It's obviously not working because hmc_server is empty but at least you have no errors about unknown parameters.

Could you download again the version 2.1.0 and try it? Maybe I uploaded the wrong version at some time.

When running nmon2influxdb without parameters you should see the version:

Using configuration file /Users/adejoux/.nmon2influxdb.cfg
NAME:
   nmon2influxdb - upload NMON stats to InfluxDB database

USAGE:
   nmon2influxdb [global options] command [command options] [arguments...]
   
VERSION:
   2.1.0

Thanks

from nmon2influxdb.

Hi Alain,
Thanks for the response.
This is little different than previous error, but I downloaded and installed v 2.1.0, configured accordingly and ran commands on influxdb CLI as mentioned on
#26
I am having below error-
[nagios@localhost ~]$ nmon2influxdb import /home/nagios/myfolder/*.nmon
2017/01/04 11:23:12 retention policy not found: log_retention. Any idea on this please?

Thanks,
Madhu

from nmon2influxdb.

Hello,

normally you don't need to run anything on influxdb cli with latest version. Could you show me your configuration file ? just remove passwords and server names from it.
log_retention is not the default policy. I don't know from where this name is coming.

You made me found a bug(bug #39) in the generation of the configuration file in version 2.1.0. Thanks for that.

I fixed it in the code and now it's generating the default configuration file below:

debug=false
timezone="Europe/Paris"
influxdb_user="root"
influxdb_password="root"
influxdb_server="localhost"
influxdb_port="8086"
influxdb_database="nmon_reports"
grafana_user="admin"
grafana_password="admin"
grafana_URL="http://localhost:3000"
grafana_access="direct"
grafana_datasource="nmon2influxdb"
hmc_server=""
hmc_user="hscroot"
hmc_password="abc123"
hmc_database="nmon2influxdbHMC"
hmc_data_retention=""
hmc_managed_system=""
hmc_managed_system_only=false
hmc_samples=0
import_skip_disks=false
import_all_cpus=false
import_build_dashboard=false
import_force=false
import_skip_metrics="JFSINODE|TOP|PCPU"
import_log_database="nmon2influxdb_log"
import_log_retention="2d"
import_data_retention=""
import_ssh_user="adejoux"
import_ssh_key="/Users/adejoux/.ssh/id_rsa"
dashboard_write_file=false
stats_limit=20
stats_sort="mean"
stats_filter=""
stats_from=""
stats_to=""
stats_host=""

Could you try it ? Thanks for your patience. I hope your HMC data are already displayed :)

Thanks again

Alain

from nmon2influxdb.

Thanks Alain for your outstanding support.
Noted below points when I downloaded, configured and ran the command after latest update:

nmon2influxdb --generate-config

panic: runtime error: comparing uncomparable type nmon2influxdblib.Inputs
goroutine 1 [running]:
panic(0x95a880, 0xc820112360)
/usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
github.com/naoina/toml.marshal(0xc820029c00, 0x36f, 0x400, 0x0, 0x0, 0x9fd1a0, 0xc82008f680, 0x99, 0x0, 0x0, ...)
/go/src/github.com/naoina/toml/encode.go:92 +0x442
github.com/naoina/toml.Marshal(0x9fd1a0, 0xc82008f680, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/naoina/toml/encode.go:44 +0xc4
github.com/adejoux/nmon2influxdb/nmon2influxdblib.(*Config).BuildCfgFile(0xc82008f440, 0xc820102420, 0x18)
/go/src/github.com/adejoux/nmon2influxdb/nmon2influxdblib/config.go:143 +0x17b
github.com/adejoux/nmon2influxdb/nmon2influxdblib.(*Config).LoadCfgFile(0xc82008f440)
/go/src/github.com/adejoux/nmon2influxdb/nmon2influxdblib/config.go:156 +0xbb
main.main()
/go/src/github.com/adejoux/nmon2influxdb/main.go:20 +0x91
############################
Auto generated config file .nmon2influxdb.cfg doesnot have any contents, and I am copying from your url and modifying as per requirement. Which is as below:

debug=false
timezone="America/Chicago"
influxdb_user="nagios"
influxdb_password="nagios"
influxdb_server="localhost"
influxdb_port="8086"
influxdb_database="nmon_reports"
grafana_user="admin"
grafana_password="admin"
grafana_URL="http://localhost:3000"
grafana_access="direct"
grafana_datasource="nmon2influxdb"
hmc_server="hmcserver"
hmc_user="nmon2influx"
hmc_password="hmcpassword"
hmc_database="nmon2influxdbHMC"
hmc_data_retention=""
hmc_managed_system="mysystem"
hmc_managed_system_only=false
hmc_samples=0
import_skip_disks=false
import_all_cpus=false
import_build_dashboard=false
import_force=false
import_skip_metrics="JFSINODE|TOP|PCPU"
import_log_database="nmon2influxdb_log"
import_log_retention="2d"
import_data_retention=""
import_ssh_user="user"
import_ssh_key="/Users/user/.ssh/id_rsa"
dashboard_write_file=false
stats_limit=20
stats_sort="mean"
stats_filter=""
stats_from=""
stats_to=""
stats_host=""
################################
Once edited the config file, ran below command, which gave below message:
(I am using the same server for nmon collection files and influxdb and I have used command on below format)

USAGE:
nmon2influxdb [global options] command [command options] [arguments...]
command= import
argument= /home/user/myfolder/*.nmon

[user@localhost ~]$ nmon2influxdb import /home/user/myfolder/*.nmon
2017/01/04 15:26:25 retention policy not found: log_retention

At this point, I am unable to load nmon file contents to influxdb and hmc as well.

Thanks,
madhu

from nmon2influxdb.

Can you change the influxdb database parameter to:

influxdb_database="nmon2influxdbtest"

The database itself should not exist and the tool will create it along the retention policy.

I have another question. Is it working when you import HMC data ?

Yes. The bug is fixed in source code. I will do a new release next week where I will include a better debugging mode to allow me to help users more easily.

Thanks,

Alain

from nmon2influxdb.

from nmon2influxdb.

Hi,
I replaced influxdb_database="nmon2influxdbtest" in place of "nmon_reports" and ran the command. Still getting same error:
[user@localhost ~]$ nmon2influxdb import /home/user/myfolder/*.nmon
Creating InfluxDB database nmon2influxdbtest
2017/01/04 23:29:36 retention policy not found: log_retention.

In fact, I was able to import till the release provided with tagging on multiple measurements and on multiple servers using regex pattern for nmon data.
After hmc feature added, I am completely unable to import any of the below:

1. nmon
2. hmc
3. both at a time in their respective database

Thanks,
Madhu

from nmon2influxdb.

Hello,

I couldn't reproduce the bug but I released a new version with a better debug mode which will help me to understand what happens.

Could you try it: version 2.1.1

Give me the full output(just remove the password and server names in the configuration section).

Run this command:

[user@localhost ~]$ nmon2influxdb --debug import /home/user/myfolder/*.nmon

Thanks again :)

Alain

from nmon2influxdb.

Hi Alain,

Thanks for new release.
I Installed version 2.1.1 and ran command in debug mode, which generated following output.

[user@localhost~]$ nmon2influxdb --debug import /home/user/myfolder/*.nmon
Using configuration file /home/user/.nmon2influxdb.cfg
databases: [telegraf _internal nmon_reports nmon2influxdb_log]
databases: [telegraf _internal nmon_reports nmon2influxdb_log]
UpdateRetentionPolicy: ALTER RETENTION POLICY "log_retention" ON "nmon2influxdb_log" DURATION 2d REPLICATION 1 DEFAULT
2017/01/05 16:15:01 retention policy not found: log_retention

Thanks,
Madhu

from nmon2influxdb.

Hello,

I replaced the binairies in relmease 2.1.1; It's my first to fix the issue. I still don't understand why it's happening but this fix should avoid it or give me more informations. Let me know the results :)

Thanks

Alain

from nmon2influxdb.

Hi Alain,
Implemented the changes you mentioned and imported nmon successfully to influxdb.
Still I am unable to load hmc part. What configuration changes will be needed to import hmc after implementing V2.1.1 ?
FYI I have attached the configuration file.
I also implemented tagging based on V2.1.0 and it worked successfully for multiple servers nmon data.
But the tags are applied on individual measurements which can be seen in attached config file. We are looking for applying tags globally. For example, after matching below pattern, the following tags are applied.

Name="host"
Match1="^aixhost1|aixhost2$"

below tag1 should apply globally.
[[input.tag]]
Name="BUSINESSCLASS"
Value="GOLD"
[[input.tag]]
Name="HOSTTYPE"
Value="PHYSICAL"
[[input.tag]]
Name="OSTYPE"
Value="AIX"
[[input.tag]]
Name="REGION"
Value="NA"

Similarly, we have different tag parameters for Match2, 3; eg tag2,3 for Silver and even for Bronze respectively. Including those many in the configuration file is a viable option? or is there any way that you can help us in reducing this manual updations.
configfile.txt

Thanks,
Madhu

from nmon2influxdb.

Hello,

Your hmc configuration seems perfectly fine.

Could you run the hmc import command in debug mode too ?

nmon2influxdb --debug hmc import

You can also try with specifying parameters in comman line only:

 nmon2influxdb --debug hmc import --samples 10 --hmc yourhmc --hmcpass yourpassword

About the tagging if it's ok with you let's talk about it in issue #36. I think you should only tag one measurement. CPU_ALL for example and use templating to filter your partitions. You can create a templating vairable to filter all other measurements. Maybe I should publish apost explaining how to do it. Can you check my answer in #36 and let me know what you think ?

Thanks again,

Alain

from nmon2influxdb.

Hi Alain,
Thanks for your immediate response.
Ran below command for hmc import and resulted to connection refused. Is this error due to hmc server authentication failure? The command output is attached in separate file. As I saw nmon import runs automatically in V 2.1.1 without external agent unlikely to v 0.9.0. Is this same for hmc import too?
[user@localhost ~]$ nmon2influxdb --debug hmc import
Please find attachment for command output.
I will update tagging after implementing template in issue #36.
hmc import_debug mode.txt

Thanks,
Madhu

from nmon2influxdb.

Hello Madhu,

Here the important part in your log is:

2017/01/06 23:01:40 HMC error sending auth request: Put https://hmchost:12443/rest/api/web/Logon: dial tcp hmcserver:12443: getsockopt: connection refused

Connection on port 12443 of host hmchost was refused. Do you have a firewall between the system where you are running nmon2influxdb and your hmc ?

You can check it with this command:

telnet hmchost 12443

You can also check the firewall on the hmc itself but normally the port 12443 is open by default.

Thanks,

Alain

from nmon2influxdb.

Hello Alain,

I checked for firewall, it shows below output:

telnet hmchost 12443

Trying...
telnet: connect: A remote host refused an attempted connect operation.
And also checked with our team for firewall setup, 12443 is opened but the connection is refused when I try to connect hmchost from my system. We concluded to use port 443 instead of 12443 if the changes can be made in your coding.
Could you please help us with this request?

Thank you!!!

Thanks,
Madhu

from nmon2influxdb.

Hello,

It's possible to change the port easily with configuration on the tool side. But port 12443 is the REST API port on the HMC and it's not possible to change it on the HMC itself. I never said it but you need to have HMC 8 to be able to get performance data from HMC.

You can also check in the HMC itself in network settings:

Hope it will work for you :)

Thanks,

Alain

from nmon2influxdb.

Hi Alain,
Thanks for update.
HMC version could be causing this issue. We have HMC 8 in prod environment, and HMC 7in test. Could it be possible to make it compatible on both 7 and 8?

Thanks,
Madhu

from nmon2influxdb.

Hello,

Unfortunately it's not possible. The REST api is a new feature of HMC 8. Sorry for you, I should have advertised it more clearly. I will update this part of the documentation.

If it's ok with you I will close this bug.

Thanks,

Alain

from nmon2influxdb.

Hi Alain,
Thank you so much for your continuous support and help you extended throughout. Please close this issue. thanks again.

Thanks,
Madhu

from nmon2influxdb.

Thanks. Closed :)

from nmon2influxdb.