gchenuet / nest-datagraph Goto Github PK

I'm not sure if this is the right place for this. I am having problems getting my setup working.

I first checked the db and it is empty.
MariaDB [nest_datagraph]> select * from status;
Empty set (0.00 sec)

I found this in the syslog.
Dec 3 16:00:02 raspberrypi CRON[18964]: (pi) CMD (/usr/bin/python /opt/nest-datagraph/backend/poller.py)
Dec 3 16:00:02 raspberrypi CRON[18960]: (CRON) info (No MTA installed, discarding output)

Any ideas? Thanks.

Heating time chart seems to be showing wrong data or Y label should not be in hours. There is no way I could be heating 80 hours per day. The same issue seems to be with Away chart.

I am trying to get nest-datagraph running on a synology NAS, the front end works immediately out of the box, but with the backend python script I get an SSL certificate error:

sjirk@boonshare:/volume1/web/nest/backend$ /usr/local/python/bin/python poller.py
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)>

the python version is:
Python 2.7.14 (default, May 28 2018, 14:46:53)
[GCC 4.6.3] on linux2

is there a solution ?

Installed this app yesterday and today with some help from Guillaume its working (thanks Guillaume!).
Only the Protect link gives me a total blank page... in params.ini I enabled the protect setting.
Any idea?

Thanks!

p.s. please increase the time between polling, once an hour is in my opinion to low. Most of the Google scripts use an interval of 5 minutes. 1 minute is to often and will cause throttling from the nest site. 5 minutes will be ok I think.

The owm_city variable doesn't seem very accurate. Can the code be modified to accept a city ID instead?

Hi,
When building the DockerFile it fails eventually with the following error:

Step 8/18 : ADD setup/nginx/nest-datagraph.conf /etc/nginx/sites-available/default.conf ADD failed: stat /var/lib/docker/tmp/docker-builder059889255/setup/nginx/nest-datagraph.conf: no such file or directory

Any idea? I'm using Docker on Mac.
Thanks
Steven

Do you have any plans/interest in making/distributing a Docker container for this?

What would happen if I would adjust the crontab to every 5 or 10 minutes? Would it be ok?

I setup the ini, built the images, but when i run the app, it fails to start properly and throws nest-datagraph_nestdatagraph_1 exited with code 2 repeatedly.

I try and docker exec -it into the container and it fails as the container is continually restarting. This is posing an issue in debugging it.

Starting nest-datagraph_mariadb_1 ... done
Recreating nest-datagraph_nestdatagraph_1 ... done
Attaching to nest-datagraph_mariadb_1, nest-datagraph_nestdatagraph_1
mariadb_1        | 2019-05-30 21:33:32 0 [Note] mysqld (mysqld 10.3.15-MariaDB-1:10.3.15+maria~bionic) starting as process 1 ...
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Uses event mutexes
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Number of pools: 1
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Using SSE2 crc32 instructions
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: 10.3.15 started; log sequence number 1630869; transaction id 21
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mariadb_1        | 2019-05-30 21:33:32 0 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1        | 2019-05-30 21:33:32 0 [Note] InnoDB: Buffer pool(s) load completed at 190530 21:33:32
mariadb_1        | 2019-05-30 21:33:32 0 [Note] Server socket created on IP: '::'.
mariadb_1        | 2019-05-30 21:33:32 0 [Warning] 'proxies_priv' entry '@% root@d6a171cf6df1' ignored in --skip-name-resolve mode.
mariadb_1        | 2019-05-30 21:33:32 0 [Note] Reading of all Master_info entries succeded
mariadb_1        | 2019-05-30 21:33:32 0 [Note] Added new Master_info '' to hash table
mariadb_1        | 2019-05-30 21:33:32 0 [Note] mysqld: ready for connections.
mariadb_1        | Version: '10.3.15-MariaDB-1:10.3.15+maria~bionic'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
: invalid option namentrypoint.sh: line 2: set: pipefail
nest-datagraph_nestdatagraph_1 exited with code 2
nest-datagraph_nestdatagraph_1 exited with code 2
nest-datagraph_nestdatagraph_1 exited with code 2
Gracefully stopping... (press Ctrl+C again to force)
Stopping nest-datagraph_nestdatagraph_1   ... done
Stopping nest-datagraph_mariadb_1         ... done```

Hi,

I've updated my own version to store more weather info.
With that data I now also added some weather icons to the graph with an overlay (work in progress) (see below)

(note I use curveType: 'function' for a curved line)

I was wondering how to best add these changes to the database, for new installs there is no issue but for upgrades people don't want to lose data.

Regards,

Bas

(i was thinking of making it so that the weather icon is only shown when weather actually changes, but I need to figure out how to handle proper alignment)

This looks like a continuation of #5 and #10. On the energy page the at home and away percentages are swapped and the same labels appear to be mismatches on the Presence History graph.

I'm assuming that this solution no longer works with the retirement of the WWN program and the migration to Google accounts?

Can this be used with multiple Nest Thermostats? If so, how?

Thanks!

In the PHP code to pull the data from the SQL db you use code like

https://github.com/gchenuet/nest-datagraph/blob/master/frontend/php/getTemp.php#L14

$date_array = "Date(".date('Y', $phpdate).",".(date('n', $phpdate)-1).",".date('d', $phpdate).",".date('H', $phpdate).",".date('i', $phpdate).")";

which I just found out almost doubles the page load time when running this script on an old Raspberry Pi.

Changing the code to be

$nMinus = date('n', $phpdate)-1;
$date_array = "Date(".date('Y,'.$nMinus.',d,H,i', $phpdate).")";

helps massively. it brought the total loading of JSON data for the energy.php page down from 11 seconds to 6 seconds.

Hi!
Love the graphs. My nest functions only in off or cooling mode, no heating. Have you any plans for adding cooling status on the energy tab? I see right now it's heating oriented.
Thanks!

Managing changes and general VC would be a bit easier if all settings/secrets were split-out into independent files that could be ignored by git. This is already done by the php frontend (ini file), but is not done by the python polling script.

Is it possible to create a login form so you need to enter username/password before you see the nest data?

Add a new dashboard for Nest Protect product.

Some ideas:

• General status
• Smoke status
• CO2 status
• WiFi status
• Electrical/Battery status

First of all, thanks for the software en guide, great stuff ^^

The away status always seems to show Absence hours / Off, never Presence hours.
Looking at your screenshot to me it seems you have the same issue. (Energy tab, then scroll to the bottom)

Before i start digging, should this already be working? Or is this work in progress?

I also have an issue with it not showing my nest protects, page comes up with just the header, blank body. Seems related to $nest = new (Nest), probably something wrong on my end :P

About my setup, i created a Docker with Debian Jessie image. I used your guide to setup, any errors because of missing components i installed at that point. Running fine ^^

p.s.
There is a typo in the guide, at the cron part: 0 * * * * /usr/bin/python /opt/nest_datagraph/backend/poller.py should be 0 * * * * /usr/bin/python /opt/nest-datagraph/backend/poller.py

What is the process for converting the temperatures to Farenheit? It looks like the python scripts and a php script have to be edited?

A week ago I tried to login to my Nest account and then I was asked to switch my Nest account to Google account, which I did. From that time I am no longer seeing any data updates. I guess it is because the old login method no longer works. Even if I enter my Google account credentials it returns HTTP 400.

Is there any workaround for it?

Hi,

I used to run an old version of Nest Datagraph (thank you very much by the way) without configparser.
I just did a clone of the repository (in a new directory) in order to update my installation.

When I tried to run poller.py, I've got issue bellow :

[root@lab nest-datagraph]# /usr/bin/python /opt/nest-datagraph/backend/poller.py
ConfigParser instance has no attribute '__getitem__'

It seems it could be due to a syntax issue regarding python2 and python3. Indeed, the syntax used in poller.py seems to be intended to run with python3.
Altough, nest.py is developed to use python2.

Maybe something is wrong in my environment.

Please find bellow the details of my environment :

[root@lab nest-datagraph]# cat /etc/centos-release
CentOS Linux release 7.5.1804 (Core)

[root@lab nest-datagraph]# uname -a
Linux 4.4.6-std-2 #1 SMP Mon Mar 21 13:41:00 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

[root@lab nest-datagraph]# python -V
Python 2.7.5

[root@lab nest-datagraph]# pip -V
pip 18.1 from /usr/lib/python2.7/site-packages/pip (python 2.7)

[root@lab nest-datagraph]# pip install -r setup/requirements.txt
Requirement already satisfied: datetime in /usr/lib64/python2.7/site-packages (from -r setup/requirements.txt (line 1)) (4.3)
Requirement already satisfied: mysql-connector-python in /usr/lib/python2.7/site-packages (from -r setup/requirements.txt (line 2)) (2.1.7)
Requirement already satisfied: pyowm in /usr/lib/python2.7/site-packages (from -r setup/requirements.txt (line 3)) (2.7.1)
Requirement already satisfied: configparser in /usr/lib/python2.7/site-packages (from -r setup/requirements.txt (line 4)) (3.5.0)
Requirement already satisfied: zope.interface in /usr/lib64/python2.7/site-packages (from datetime->-r setup/requirements.txt (line 1)) (4.0.5)
Requirement already satisfied: pytz in /usr/lib/python2.7/site-packages (from datetime->-r setup/requirements.txt (line 1)) (2016.10)
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from zope.interface->datetime->-r setup/requirements.txt (line 1)) (0.9.8)

[root@lab nest-datagraph]# pip list | grep configparser
configparser                     3.5.0

Thank you for your help/inputs

The application does not work if either the MySQL password or the Nest account password configured in settings.ini contains an exclamation point (!). I was able to work around this by changing both of the passwords so that they did not include an exclamation point.

Interestingly, enclosing the offending password in single quotes enables the poller script to work, but breaks the site. The reverse happens when they are not enclosed in single quotes.

Error from the poller is:

HTTP Error 400: Bad Request"

Error from the nginx error log for the site is:

[error] 2067#0: *1 FastCGI sent in stderr: "PHP message: PHP Warning:  syntax error, unexpected '!' in ../conf/settings.ini on line 23
 in /opt/nest-datagraph/frontend/php/getDayHum.php on line 2
PHP message: PHP Notice:  date_default_timezone_set(): Timezone ID '' is invalid in /opt/nest-datagraph/frontend/php/getDayHum.php on line 3
PHP message: PHP Warning:  mysqli_connect(): (28000/1045): Access denied for user 'www-data'@'localhost' (using password: NO) in /opt/nest-datagraph/frontend/php/getDayHum.php on line 5
PHP message: PHP Warning:  mysqli_error() expects parameter 1 to be mysqli, boolean given in /opt/nest-datagraph/frontend/php/getDayHum.php on line 6" while reading response header from upstream, client: 10.42.42.104, server: nest.domain.local, request: "GET /php/getDayHum.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "nest.domain.local", referrer: "http://nest.domain.local/"

Exterior temperature gauge has numbers 0 to 60. Something like -30 +40 would be more adequate.

Nest hasn't been set to away at all.
Those 11 hours should be Presence hours.
http://i.imgur.com/SwrdS6W.png

this is my config

My question is "No NLB health metrics"

apiVersion: v1
data:
config.yml: |
discovery:
exportedTagsOnMetrics:
nlb:

• Department
• Tier
• Environment
• Project
  jobs:
• type: "nlb"
  regions:
• cn-north-1
  nilToZero: true
  metrics:
• name: UnHealthyHostCount
  statistics: [Maximum]
  period: 60
  delay: 120
• name: HealthyHostCount
  statistics: [Minimum]
  period: 60
  delay: 120

https://blog.darksky.net/dark-sky-has-a-new-home/

Apple bought DarkSky and it is no longer an option for weather data. This will need to be replaced with another source