steven-matison / dfhz_hue_mpack Goto Github PK

Thanks for all this work to create a Hue management pack! I'm trying to install it but getting a "File exists" error. This seems odd to me-- I'm not sure what the collision would be, but I cannot figure out what the colliding file is, either. (--verbose doesn't give any more information).

Does this ring any bells? Any advice or info much appreciated!

CentOS 7.7.1908, Ambari 2.7.5

[sorokaa@test07 ~]$ sudo ambari-server install-mpack --mpack=https://github.com/steven-matison/dfhz_hue_mpack/raw/master/hue_mpack-4.6.0-0.1.tar.gz --verbose
Using python  /usr/bin/python
Installing management pack
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Installing management pack https://github.com/steven-matison/dfhz_hue_mpack/raw/master/hue_mpack-4.6.0-0.1.tar.gz
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Download management pack to temp location /var/lib/ambari-server/data/tmp/hue_mpack-4.6.0-0.1.tar.gz
INFO: Trying to download https://github.com/steven-matison/dfhz_hue_mpack/raw/master/hue_mpack-4.6.0-0.1.tar.gz to /var/lib/ambari-server/data/tmp/hue_mpack-4.6.0-0.1.tar.gz with python lib [urllib2].
INFO: Downloading to: /var/lib/ambari-server/data/tmp/hue_mpack-4.6.0-0.1.tar.gz Bytes: 69284

INFO: Finished downloading https://github.com/steven-matison/dfhz_hue_mpack/raw/master/hue_mpack-4.6.0-0.1.tar.gz to /var/lib/ambari-server/data/tmp/hue_mpack-4.6.0-0.1.tar.gz
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Expand management pack at temp location /var/lib/ambari-server/data/tmp/hue_mpack-4.6.0-0.1
2021-02-11 15:57:26,559 - Execute[('tar', '-xf', '/var/lib/ambari-server/data/tmp/hue_mpack-4.6.0-0.1.tar.gz', '-C', '/var/lib/ambari-server/data/tmp/')] {'tries': 3, 'sudo': True, 'try_sleep': 1}
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Stage management pack hue-ambari.mpack-4.6.0 to staging location /var/lib/ambari-server/resources/mpacks/hue-ambari.mpack-4.6.0
INFO: Processing artifact HUE-common-services of type service-definitions in /var/lib/ambari-server/resources/mpacks/hue-ambari.mpack-4.6.0/common-services
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Symlink: /var/lib/ambari-server/resources/common-services/HUE/4.6.0
INFO: Processing artifact HUE-addon-services of type stack-addon-service-definitions in /var/lib/ambari-server/resources/mpacks/hue-ambari.mpack-4.6.0/addon-services
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
Traceback (most recent call last):
  File "/usr/sbin/ambari-server.py", line 1078, in <module>
    mainBody()
  File "/usr/sbin/ambari-server.py", line 1048, in mainBody
    main(options, args, parser)
  File "/usr/sbin/ambari-server.py", line 998, in main
    action_obj.execute()
  File "/usr/sbin/ambari-server.py", line 78, in execute
    self.fn(*self.args, **self.kwargs)
  File "/usr/lib/ambari-server/lib/ambari_server/setupMpacks.py", line 902, in install_mpack
    (mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, mpack_archive_path) = _install_mpack(options, replay_mode)
  File "/usr/lib/ambari-server/lib/ambari_server/setupMpacks.py", line 798, in _install_mpack
    process_stack_addon_service_definitions_artifact(artifact, artifact_source_dir, options)
  File "/usr/lib/ambari-server/lib/ambari_server/setupMpacks.py", line 557, in process_stack_addon_service_definitions_artifact
    sudo.symlink(source_service_version_path, dest_link)
  File "/usr/lib/ambari-server/lib/resource_management/core/sudo.py", line 130, in symlink
    os.symlink(source, link_name)
OSError: [Errno 17] File exists

Here is the metainfo.xml for Ubuntu 18.04 LTS

Users have run their selves in the terminal these following commands:
curl -sL https://deb.nodesource.com/setup_10.x | sudo bash - sudo apt-get install -y nodejs

and if finding issues with non-existing user 'hue'
sudo adduser -ingroup hue hue

<osSpecifics>
	  <osSpecific>
	    <osFamily>any</osFamily>
	    <packages>
     <package><name>wget</name></package>
     <package><name>tar</name></package>
     <package><name>asciidoc</name></package>
     <package><name>libkrb5-dev</name></package>
     <package><name>libxml2-dev</name></package>
     <package><name>libxslt-dev</name></package>   
     <package><name>libldap2-dev</name></package>
     <package><name>python-dev</name></package>
     <package><name>python-simplejson</name></package>
     <package><name>python-setuptools</name></package>
     <package><name>python-psycopg2</name></package>
     <package><name>libsqlite3-dev</name></package>
     <package><name>rsync</name></package>
     <package><name>libsasl2-dev</name></package>
     <package><name>python-crypto</name></package>
     <package><name>libgmp-dev</name></package>
     <package><name>libyaml-dev</name></package>
     <package><name>sasl2-bin</name></package>
     <package><name>libsasl2-dev</name></package>
     <package><name>libsasl2-modules-gssapi-mit</name></package>
     <package><name>libffi-dev</name></package>
     <package><name>ant</name></package>
     <package><name>build-essential</name></package>
     <package><name>mysql-server</name></package>
     <package><name>libmysqlclient-dev</name></package>
        </packages>
	  </osSpecific>
	</osSpecifics>
	

2020-08-05 01:58:10,940 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.4.0-315 -> 3.1.4.0-315
2020-08-05 01:58:10,983 - Using hadoop conf dir: /usr/hdp/3.1.4.0-315/hadoop/conf
2020-08-05 01:58:11,396 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=3.1.4.0-315 -> 3.1.4.0-315
2020-08-05 01:58:11,407 - Using hadoop conf dir: /usr/hdp/3.1.4.0-315/hadoop/conf
2020-08-05 01:58:11,410 - Skipping creation of User and Group as host is sys prepped or ignore_groupsusers_create flag is on
2020-08-05 01:58:11,411 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'}
2020-08-05 01:58:11,413 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2020-08-05 01:58:11,416 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2020-08-05 01:58:11,417 - call['/var/lib/ambari-agent/tmp/changeUid.sh hbase'] {}
2020-08-05 01:58:11,435 - call returned (0, '1020')
2020-08-05 01:58:11,437 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1020'] {'not_if': '(test $(id -u hbase) -gt 1000) || (true)'}
2020-08-05 01:58:11,447 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase 1020'] due to not_if
2020-08-05 01:58:11,448 - Skipping setting dfs cluster admin and tez view acls as host is sys prepped
2020-08-05 01:58:11,448 - FS Type: HDFS
2020-08-05 01:58:11,448 - Directory['/etc/hadoop'] {'mode': 0755}
2020-08-05 01:58:11,481 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2020-08-05 01:58:11,483 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2020-08-05 01:58:11,512 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2020-08-05 01:58:11,530 - Skipping Execute[('setenforce', '0')] due to only_if
2020-08-05 01:58:11,531 - Directory['/var/log/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'mode': 0775, 'cd_access': 'a'}
2020-08-05 01:58:11,536 - Directory['/var/run/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'root', 'cd_access': 'a'}
2020-08-05 01:58:11,537 - Directory['/var/run/hadoop/hdfs'] {'owner': 'hdfs', 'cd_access': 'a'}
2020-08-05 01:58:11,538 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'create_parents': True, 'cd_access': 'a'}
2020-08-05 01:58:11,546 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2020-08-05 01:58:11,549 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2020-08-05 01:58:11,559 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2020-08-05 01:58:11,578 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/hadoop-metrics2.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2020-08-05 01:58:11,579 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2020-08-05 01:58:11,581 - File['/usr/hdp/3.1.4.0-315/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2020-08-05 01:58:11,588 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop', 'mode': 0644}
2020-08-05 01:58:11,595 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2020-08-05 01:58:11,603 - Skipping unlimited key JCE policy check and setup since it is not required
2020-08-05 01:58:11,619 - Skipping stack-select on HUE because it does not exist in the stack-select package structure.
2020-08-05 01:58:12,033 - Using hadoop conf dir: /usr/hdp/3.1.4.0-315/hadoop/conf
2020-08-05 01:58:12,043 - Execute['ps -ef | grep hue | grep -v grep | awk '{print $2}' | xargs kill -9'] {'ignore_failures': True, 'user': 'hue'}
2020-08-05 01:58:12,228 - Skipping failure of Execute['ps -ef | grep hue | grep -v grep | awk '{print $2}' | xargs kill -9'] due to ignore_failures. Failure reason: Execution of 'ps -ef | grep hue | grep -v grep | awk '{print $2}' | xargs kill -9' returned 137. kill: sending signal to 27914 failed: Operation not permitted
kill: sending signal to 27940 failed: No such process
kill: sending signal to 27943 failed: No such process
-bash: line 1: 27940 Done ps -ef
27941 | grep hue
27942 | grep -v grep
27943 | awk '{print $2}'
27944 Killed | xargs kill -9
2020-08-05 01:58:12,229 - File['/var/run/hue/hue-server.pid'] {'owner': 'hue', 'action': ['delete']}
2020-08-05 01:58:12,230 - Deleting File['/var/run/hue/hue-server.pid']
2020-08-05 01:58:12,235 - Configure Hue Service
2020-08-05 01:58:12,236 - Directory['/var/log/hue'] {'owner': 'hue', 'create_parents': True, 'group': 'hue', 'mode': 0755, 'cd_access': 'a'}
2020-08-05 01:58:12,239 - Directory['/var/run/hue'] {'owner': 'hue', 'group': 'hue', 'create_parents': True, 'mode': 0755, 'cd_access': 'a'}
2020-08-05 01:58:12,240 - File['/var/log/hue/hue-install.log'] {'content': '', 'owner': 'hue', 'group': 'hue', 'mode': 0644}
2020-08-05 01:58:12,241 - Writing File['/var/log/hue/hue-install.log'] because contents don't match
2020-08-05 01:58:12,242 - File['/var/run/hue/hue-server.pid'] {'content': '', 'owner': 'hue', 'group': 'hue', 'mode': 0644}
2020-08-05 01:58:12,242 - Writing File['/var/run/hue/hue-server.pid'] because it doesn't exist
2020-08-05 01:58:12,243 - Changing owner for /var/run/hue/hue-server.pid from 0 to hue
2020-08-05 01:58:12,243 - Changing group for /var/run/hue/hue-server.pid from 0 to hue
2020-08-05 01:58:12,243 - Execute['find /var/lib/ambari-agent/cache/common-services/HUE/4.6.0/package -iname ".sh" | xargs chmod +x'] {}
2020-08-05 01:58:12,258 - HdfsResource['/user/hue'] {'security_enabled': False, 'hadoop_bin_dir': '/usr/hdp/3.1.4.0-315/hadoop/bin', 'keytab': [EMPTY], 'dfs_type': 'HDFS', 'default_fs': 'true', 'user': 'hdfs', 'hdfs_resource_ignore_file': '/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local': 'kinit', 'principal_name': [EMPTY], 'recursive_chmod': True, 'owner': 'hue', 'hadoop_conf_dir': '/usr/hdp/3.1.4.0-315/hadoop/conf', 'type': 'directory', 'action': ['create_on_execute'], 'immutable_paths': [u'/mr-history/done', u'/warehouse/tablespace/managed/hive', u'/warehouse/tablespace/external/hive', u'/app-logs', u'/tmp'], 'mode': 0755}
2020-08-05 01:58:12,268 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET -d '"'"''"'"' -H '"'"'Content-Length: 0'"'"' '"'"'http://c7401.ambari.apache.org:50070/webhdfs/v1/user/hue?op=GETFILESTATUS&user.name=hdfs'"'"' 1>/tmp/tmpMo0h9z 2>/tmp/tmpYlFv81''] {'logoutput': None, 'quiet': False}
2020-08-05 01:58:12,408 - call returned (0, '')
2020-08-05 01:58:12,410 - get_user_call_output returned (0, u'{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":148012,"group":"hdfs","length":0,"modificationTime":1596581549934,"owner":"hue","pathSuffix":"","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}200', u'')
2020-08-05 01:58:12,412 - Creating /usr/local/hue/desktop/conf/log.conf file
2020-08-05 01:58:12,420 - File['/usr/local/hue/desktop/conf/log.conf'] {'owner': 'hue', 'content': InlineTemplate(...)}
2020-08-05 01:58:12,421 - Creating /usr/local/hue/desktop/conf/hue.ini config file
2020-08-05 01:58:12,472 - File['/usr/local/hue/desktop/conf/hue.ini'] {'owner': 'hue', 'content': InlineTemplate(...)}
2020-08-05 01:58:12,474 - Run the script file to add configurations
2020-08-05 01:58:12,474 - Execute['/var/lib/ambari-agent/cache/common-services/HUE/4.6.0/package/files/configs.sh set c7401.ambari.apache.org hdp hdfs-site 'dfs.namenode.acls.enabled' 'true''] {}
2020-08-05 01:58:13,984 - Execute['/var/lib/ambari-agent/cache/common-services/HUE/4.6.0/package/files/configs.sh set c7401.ambari.apache.org hdp core-site 'hadoop.proxyuser.hue.groups' '''] {}
2020-08-05 01:58:15,168 - Execute['/var/lib/ambari-agent/cache/common-services/HUE/4.6.0/package/files/configs.sh set c7401.ambari.apache.org hdp core-site 'hadoop.proxyuser.hue.hosts' '*''] {}
2020-08-05 01:58:16,319 - Execute['/usr/local/hue/build/env/bin/supervisor >> /var/log/hue/hue-install.log 2>&1 &'] {'environment': {'SPARK_HOME': '/usr/hdp/current/spark-client', 'HADOOP_CONF_DIR': u'/usr/hdp/3.1.4.0-315/hadoop/conf', 'JAVA_HOME': u'/usr/jdk64/jdk1.8.0_112', 'PATH': '$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin'}, 'user': 'hue'}
2020-08-05 01:58:16,444 - Execute['ps -ef | grep hue | grep supervisor | grep -v grep | awk '{print $2}' > /var/run/hue/hue-server.pid'] {'user': 'hue'}
2020-08-05 01:58:16,601 - Pid files for current script are not defined
2020-08-05 01:58:16,649 - Skipping stack-select on HUE because it does not exist in the stack-select package structure.

Command completed successfully!

Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HUE/4.6.0/package/scripts/hue_server.py", line 76, in
HueServer().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/HUE/4.6.0/package/scripts/hue_server.py", line 14, in install
import params
File "/var/lib/ambari-agent/cache/common-services/HUE/4.6.0/package/scripts/params.py", line 157, in
hue_principal = config['configurations']['hue-desktop-site']['kerberos_hue_principal'].replace('_HOST',http_host)
TypeError: coercing to Unicode: need string or buffer, instance found