splunk / docker-splunk-legacy Goto Github PK
View Code? Open in Web Editor NEWDocker Splunk *** LEGACY IMAGES - PLEASE SEE https://github.com/splunk/docker-splunk INSTEAD ***
Home Page: https://www.splunk.com
License: Apache License 2.0
Docker Splunk *** LEGACY IMAGES - PLEASE SEE https://github.com/splunk/docker-splunk INSTEAD ***
Home Page: https://www.splunk.com
License: Apache License 2.0
Hello,
I saw that the tag latest is not pulling the 7.1.0 but the 7.0.3 as well that 7.1.0 hasn't pushed to the master branch.
Is there any particular reason for holding the push?
Deploying Splunk UF in a Docker SWARM mode will not work if it is trying to resolve the Splunk Enterprise instance by service name.
The reason is that dnsutils must be added to the base image, apt-get install -y dnsutils.
Using ConfigMap to manage splunk universal forwarder's config has been working well until the release of 1.9.4, which included this security fix.
ConfigMap
now gets mounted as read-only always, which breaks this container due to the chown
commands in entrypoint.sh
.
See: https://answers.splunk.com/answers/626964/kubernetes-194-breaking-changes-universal-forwarde.html
Splunk universal forwarder fails to do Deployment server setup step randomly. as it able to find splunk.version file created and it is assuming configuration is already done and is not setting splunk_deployment_server.
this is occurs randomly and does not have certain prediction. probably due to race condition in the entrypoin.sh script.
I've tried to change the default username with the "command" in docker-compose.yml but it doesn't seem to work. Does anyone know how to do this?
Hello world,
I have been trying to load the license file by copying over into the /opt/splunk/etc/licenses/enterprise/ folder and no joy.
I have also set a volume mount to the file which works in other circumstances - just not with this license file.
volumes:
- ./splunkmaster/volume/etc:/opt/splunk/etc:Z
- ./splunkmaster/volume/var:/opt/splunk/var:Z
- ./splunksearch/volume/license/Splunk.License:/opt/splunk/etc/licenses/enterprise/Splunk.License:Z
Anyone care to shed some light?
Thanks..
Steps to reproduce:
docker run -p 8000:8000 -d -e "SPLUNK_START_ARGS=--accept-license" v /local/path/optsplunketc:/opt/splunk/etc -v /local/path/optsplunkvar:/opt/splunk/var splunk/splunk:7.0.0
)docker run -p 8000:8000 -d -e "SPLUNK_START_ARGS=--accept-license" v /local/path/optsplunketc:/opt/splunk/etc -v /local/path/optsplunkvar:/opt/splunk/var splunk/splunk:7.0.0
) This container will fail with exit code 1 and the logs will read:This appears to be an upgrade of Splunk.
--------------------------------------------------------------------------------)
Splunk has detected an older version of Splunk installed on this machine. To
finish upgrading to the new version, Splunk's installer will automatically
update and alter your current configuration files. Deprecated configuration
files will be renamed with a .deprecated extension.
You can choose to preview the changes that will be made to your configuration
files before proceeding with the migration and upgrade:
If you want to migrate and upgrade without previewing the changes that will be
made to your existing configuration files, choose 'y'.
If you want to see what changes will be made before you proceed with the
upgrade, choose 'n'.
tcgetattr: Inappropriate ioctl for device
WARNING: error changing terminal modes - password will echo!
Perform migration and upgrade without previewing configuration changes? [y/n]
If you add --answer-yes
to SPLUNK_START_ARGS
this container will start but it still goes through the "update" process. I'm not sure if this is intended or unintended behavior but I feel like it should at least be documented because it's not clear from the documentation that starting a new container with those volumes mounted will not work. This is an important use case for persisting settings for a server and one of the big benefits of running splunk in docker.
The current URI in both the Forwarder and Instance Dockerfile
need to be updated.
Recently the product name has been removed from the URI.
Error installing with docker-compose up
git clone https://github.com/splunk/docker-splunk.git
Cloning into 'docker-splunk'...
remote: Counting objects: 817, done.
remote: Total 817 (delta 0), reused 0 (delta 0), pack-reused 817
Receiving objects: 100% (817/817), 156.03 KiB | 0 bytes/s, done.
Resolving deltas: 100% (393/393), done.
[ec2-user@ip-10-193-206-13 SPLUNK]$ ls
docker-splunk
[ec2-user@ip-10-193-206-13 SPLUNK]$ cd docker-splunk/
[ec2-user@ip-10-193-206-13 docker-splunk]$ ls
CONTRIBUTING.md enterprise LICENSE README.md universalforwarder
[ec2-user@ip-10-193-206-13 docker-splunk]$ cd enterprise/
[ec2-user@ip-10-193-206-13 enterprise]$ ls
build.sh docker-compose.yml Dockerfile entrypoint.sh publishImage.sh README.md
[ec2-user@ip-10-193-206-13 enterprise]$ vim docker-compose.yml
[ec2-user@ip-10-193-206-13 enterprise]$ docker-compose up
Creating network "enterprise_default" with the default driver
Creating volume "enterprise_opt-splunk-etc" with default driver
Creating volume "enterprise_opt-splunk-var" with default driver
Pulling splunkenterprise (splunk/splunk:7.0.0)...
7.0.0: Pulling from splunk/splunk
Digest: sha256:216f8511d99b7e79ac147cf49829b92ad49a92ec8de35baf5beaf1bb50d9316c
Status: Downloaded newer image for splunk/splunk:7.0.0
Creating enterprise_splunkenterprise_1 ...
Creating enterprise_splunkenterprise_1 ... done
Attaching to enterprise_splunkenterprise_1
splunkenterprise_1 | Copying '/opt/splunk/etc/openldap/ldap.conf.default' to '/opt/splunk/etc/openldap/ldap.conf'.
splunkenterprise_1 | Generating RSA private key, 2048 bit long modulus
splunkenterprise_1 | .........................+++
splunkenterprise_1 | .......................+++
splunkenterprise_1 | e is 65537 (0x10001)
splunkenterprise_1 | writing RSA key
splunkenterprise_1 |
splunkenterprise_1 | Generating RSA private key, 2048 bit long modulus
splunkenterprise_1 | .....................................................+++
splunkenterprise_1 | .....................+++
splunkenterprise_1 | e is 65537 (0x10001)
splunkenterprise_1 | writing RSA key
splunkenterprise_1 |
splunkenterprise_1 | Moving '/opt/splunk/share/splunk/search_mrsparkle/modules.new' to '/opt/splunk/share/splunk/search_mrsparkle/modules'.
splunkenterprise_1 |
splunkenterprise_1 | An unforeseen error occurred:
splunkenterprise_1 |
splunkenterprise_1 | Exception: <type 'exceptions.OSError'>, Value: [Errno 39] Directory not empty: '/opt/splunk/share/splunk/search_mrsparkle/modules.new/converters'
splunkenterprise_1 |
splunkenterprise_1 | Traceback (most recent call last):
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli.py", line 1143, in main
splunkenterprise_1 | parseAndRun(argsList)
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli.py", line 998, in parseAndRun
splunkenterprise_1 | retVal = cList.getCmd(command, subCmd).call(argList, fromCLI = True)
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli.py", line 280, in call
splunkenterprise_1 | return self.func(args, fromCLI)
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/control_api.py", line 30, in wrapperFunc
splunkenterprise_1 | return func(dictCopy, fromCLI)
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/_internal.py", line 176, in firstTimeRun
splunkenterprise_1 | comm.moveItem(migration.PATH_UI_MOD_NEW, migration.PATH_UI_MOD_ACTIVE)
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli_common.py", line 1017, in moveItem
splunkenterprise_1 | shutil.move(src, dst)
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/shutil.py", line 300, in move
splunkenterprise_1 | rmtree(src)
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/shutil.py", line 247, in rmtree
splunkenterprise_1 | rmtree(fullname, ignore_errors, onerror)
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/shutil.py", line 256, in rmtree
splunkenterprise_1 | onerror(os.rmdir, path, sys.exc_info())
splunkenterprise_1 | File "/opt/splunk/lib/python2.7/shutil.py", line 254, in rmtree
splunkenterprise_1 | os.rmdir(path)
splunkenterprise_1 | OSError: [Errno 39] Directory not empty: '/opt/splunk/share/splunk/search_mrsparkle/modules.new/converters'
splunkenterprise_1 |
splunkenterprise_1 |
splunkenterprise_1 | Please file a case online at http://www.splunk.com/page/submit_issue
splunkenterprise_1 |
splunkenterprise_1 |
splunkenterprise_1 | This appears to be your first time running this version of Splunk.
enterprise_splunkenterprise_1 exited with code 2
[ec2-user@ip-10-193-206-13 enterprise]$
I bit confused , daemonset was running fine for splunk UF till yesterday but now it logs are showing:
Please enter a new password
Is it possible to run ES in a docker container? I've tried and it breaks the container
From https://github.com/splunk/docker-splunk/blob/master/enterprise/README.md:
docker run --name splunk --hostname splunk -p 8000:8000 -d -e "SPLUNK_START_ARGS=--accept-license"
This command is missing an image name and clearly won't work. If you add an image name, it's still broken. Repro:
$ docker run --name splunk --hostname splunk -p 8000:8000 -d -e "SPLUNK_START_ARGS=--accept-license"
"docker run" requires at least 1 argument(s).
See 'docker run --help'.
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
$ docker run --name splunk --hostname splunk -p 8000:8000 -d -e "SPLUNK_START_ARGS=--accept-license" splunk/splunk
2d45302f44756753383c8a7953ef8b05c69f089dc81d1501939fe2112760e122
working on PR
The entrypoint has a lot of configuration features based on authenticating as admin using -auth admin:changeme
SPLUNK_ENABLE_DEPLOY_SERVER
SPLUNK_DEPLOYMENT_SERVER
SPLUNK_ENABLE_LISTEN
SPLUNK_FORWARD_SERVER
SPLUNK_FORWARD_SERVER_*
SPLUNK_ADD
SPLUNK_ADD_*
In the 7.1.0 docker image, unlike 7.0.x, the admin password is set at the begining of first time configuration (rather than via the webpage after the first time configuration is done), before these configurations steps run. So the password has already been changed from changeme
and none of the splunk configurations work.
Set your command to (for example, in a docker-compose.yml
file
command: |
bash -c "
if [ -e /opt/splunk/etc/str ]; then
rm -f /opt/splunk/ftr
exec /sbin/entrypoint.sh start-service
else
touch /opt/splunk/etc/str
exec /sbin/entrypoint.sh start-service --seed-passwd changeme
fi
"
At least this way, the password is changeme
, and everything works. Only downside to this is the webpage no longer tells you to change the password after logging in, like it did in 7.0.x
Hello,
I have been trying to enable clustering through compose file with no joy.
Could someone shed some light to the secret sauce?
splunkmaster:
restart: always
build: ./splunkmaster
hostname: splunkmaster
# image: splunk/splunk:6.5.3
environment:
SPLUNK_START_ARGS: --accept-license --answer-yes
SPLUNK_ENABLE_LISTEN: 9997
SPLUNK_CMD_1: splunk edit cluster-config -mode master -replication_factor 3 -search_factor 2 -secret newsecret123456 -cluster_label cluster1 auth admin:changeme
SPLUNK_CMD_2: splunk restart
Following the instructions here under the heading Start a Splunk Enterprise container and mount the necessary container volumes I get the following error:
Docker version: 18.03.0-ce-win59 (16762)
Windows 10 version: 1709
Copying '/opt/splunk/etc/openldap/ldap.conf.default' to '/opt/splunk/etc/openldap/ldap.conf'.
An unforeseen error occurred:
Exception: <type 'exceptions.OSError'>, Value: [Errno 1] Operation not permitted: '/opt/splunk/etc/openldap/ldap.conf'
Traceback (most recent call last):
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli.py", line 1143, in main
parseAndRun(argsList)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli.py", line 998, in parseAndRun
retVal = cList.getCmd(command, subCmd).call(argList, fromCLI = True)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli.py", line 280, in call
return self.func(args, fromCLI)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/control_api.py", line 30, in wrapperFunc
return func(dictCopy, fromCLI)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/_internal.py", line 128, in firstTimeRun
comm.copyItem(migration.PATH_LDAP_CONF_DEF, migration.PATH_LDAP_CONF)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli_common.py", line 1008, in copyItem
shutil.copy(src, dst)
File "/opt/splunk/lib/python2.7/shutil.py", line 120, in copy
copymode(src, dst)
File "/opt/splunk/lib/python2.7/shutil.py", line 91, in copymode
os.chmod(dst, mode)
OSError: [Errno 1] Operation not permitted: '/opt/splunk/etc/openldap/ldap.conf'
Please file a case online at http://www.splunk.com/page/submit_issue
This appears to be your first time running this version of Splunk.
Is docker-splunk no longer listed on Docker Hub?
Hello,
I'm trying to get run a docker container but I'm getting this: Validating databases (splunkd validatedb) failed with code '1'
Facts:
a) command:
docker run
--name splunk --hostname=splunk
-p 8000:8000
-p 8088:8088
-p 9997:9997
-p 1514:1514
-p 1515:1515
-v /opt/splunk/etc:/opt/splunk/etc
-v /opt/splunk/var:/opt/splunk/var
-e "SPLUNK_START_ARGS=--accept-license --answer-yes"
splunk/splunk:latest
host /opt/splunk is owned by root. /opt/splunk/etc and /opt/splunk/var owned by user 999. If I change the permissions on the host for etc and var to root or any other user, it goes little further but I have permission errors on writting the logs.
Of course since the docker container is not running the only way to get inside the container is by 'docker run -ti --entrypoint /bin/bash splunk/splunk' which I can see that inside the var and the etc are owned by root but to be honest I don't really trust this.
So any idea? is it really permissions? Is it possible to change the userid to something different?
PS. the docker command is run as root.
Hi there,
I'm trying to use Docker-compose V2 on Mac Beta and there might be something with the FS...
version: "2"
services:
splunk:
image: outcoldman/splunk:6.4
hostname: splunk
environment:
- SPLUNK_START_ARGS=--accept-license
ports:
- 8000:8000
volumes:
- ./data/etc:/opt/splunk/etc
- ./data/var:/opt/splunk/var
mdesales@Marcello-New2015 [05/06/201620:39:15] ~/dev/github/intuit/servicesplatform-tools/microservices/logging $ docker-compose up
Starting logging_splunk_1
Attaching to logging_splunk_1
splunk_1 |
splunk_1 | Splunk> Another one.
splunk_1 |
splunk_1 | Checking prerequisites...
splunk_1 | Checking http port [8000]: open
splunk_1 | Checking mgmt port [8089]: open
splunk_1 | Checking appserver port [127.0.0.1:8065]: open
splunk_1 | Checking kvstore port [8191]: Checking configuration... Done.
splunk_1 | homePath='/opt/splunk/var/lib/splunk/audit/db' of index=_audit on unusable filesystem.
splunk_1 | Checking critical directories... Done
splunk_1 | Checking indexes...
splunk_1 | Validating databases (splunkd validatedb) failed with code '1'. If you cannot resolve the issue(s) above after consulting documentation, please file a case online at http://www.splunk.com/page/submit_issue
splunk_1 | open
splunk_1 | Creating: /opt/splunk/var/lib/splunk
splunk_1 | Creating: /opt/splunk/var/run/splunk
splunk_1 | Creating: /opt/splunk/var/run/splunk/appserver/i18n
splunk_1 | Creating: /opt/splunk/var/run/splunk/appserver/modules/static/css
splunk_1 | Creating: /opt/splunk/var/run/splunk/upload
splunk_1 | Creating: /opt/splunk/var/spool/splunk
splunk_1 | Creating: /opt/splunk/var/spool/dirmoncache
splunk_1 | Creating: /opt/splunk/var/lib/splunk/authDb
splunk_1 | Creating: /opt/splunk/var/lib/splunk/hashDb
logging_splunk_1 exited with code 10
It just worked as expected without the volumes.
Recreating logging_splunk_1
Attaching to logging_splunk_1
splunk_1 | Copying '/opt/splunk/etc/openldap/ldap.conf.default' to '/opt/splunk/etc/openldap/ldap.conf'.
splunk_1 | Generating RSA private key, 1024 bit long modulus
splunk_1 | ................................................................++++++
splunk_1 | ...............................++++++
splunk_1 | e is 65537 (0x10001)
splunk_1 | writing RSA key
splunk_1 |
splunk_1 | Generating RSA private key, 1024 bit long modulus
splunk_1 | ..........................................++++++
splunk_1 | ......++++++
splunk_1 | e is 65537 (0x10001)
splunk_1 | writing RSA key
splunk_1 |
splunk_1 | Moving '/opt/splunk/share/splunk/search_mrsparkle/modules.new' to '/opt/splunk/share/splunk/search_mrsparkle/modules'.
splunk_1 |
splunk_1 | This appears to be your first time running this version of Splunk.
splunk_1 |
splunk_1 | Splunk> Another one.
splunk_1 |
splunk_1 | Checking prerequisites...
splunk_1 | Checking http port [8000]: open
splunk_1 | Checking mgmt port [8089]: open
splunk_1 | Checking appserver port [127.0.0.1:8065]: open
splunk_1 | Checking kvstore port [8191]: Checking configuration... Done.
splunk_1 | Checking critical directories... Done
splunk_1 | Checking indexes...
splunk_1 | Validated: _audit _internal _introspection _thefishbucket history main summary
splunk_1 | Done
splunk_1 | New certs have been generated in '/opt/splunk/etc/auth'.
splunk_1 | open
splunk_1 | Creating: /opt/splunk/var/lib/splunk
splunk_1 | Creating: /opt/splunk/var/run/splunk
splunk_1 | Creating: /opt/splunk/var/run/splunk/appserver/i18n
splunk_1 | Creating: /opt/splunk/var/run/splunk/appserver/modules/static/css
splunk_1 | Creating: /opt/splunk/var/run/splunk/upload
splunk_1 | Creating: /opt/splunk/var/spool/splunk
splunk_1 | Creating: /opt/splunk/var/spool/dirmoncache
splunk_1 | Creating: /opt/splunk/var/lib/splunk/authDb
splunk_1 | Creating: /opt/splunk/var/lib/splunk/hashDb
splunk_1 | Checking filesystem compatibility... Done
splunk_1 | Checking conf files for problems...
splunk_1 | Done
splunk_1 | Checking default conf files for edits...
splunk_1 | Validating installed files against hashes from '/opt/splunk/splunk-6.4.0-f2c836328108-linux-2.6-x86_64-manifest'
splunk_1 | Generating a 1024 bit RSA private key
splunk_1 | .......++++++
splunk_1 | ....................................................++++++
splunk_1 | writing new private key to 'privKeySecure.pem'
splunk_1 | -----
splunk_1 | Signature ok
splunk_1 | subject=/CN=splunk/O=SplunkUser
splunk_1 | Getting CA Private Key
splunk_1 | writing RSA key
splunk_1 | All installed files intact.
splunk_1 | Done
splunk_1 | All preliminary checks passed.
splunk_1 |
splunk_1 | Starting splunk server daemon (splunkd)...
splunk_1 | Done
splunk_1 |
splunk_1 |
splunk_1 | Waiting for web server at http://127.0.0.1:8000 to be available.. Done
splunk_1 |
splunk_1 |
splunk_1 | If you get stuck, we're here to help.
splunk_1 | Look for answers here: http://docs.splunk.com
splunk_1 |
splunk_1 | The Splunk web interface is at http://splunk:8000
splunk_1 |
splunk_1 | Deployment Server is enabled.
splunk_1 | Stopping splunkd...
splunk_1 | Shutting down. Please wait, as this may take a few minutes.
splunk_1 | ..
splunk_1 | Stopping splunk helpers...
splunk_1 |
splunk_1 | Done.
splunk_1 |
splunk_1 | Splunk> Another one.
splunk_1 |
splunk_1 | Checking prerequisites...
splunk_1 | Checking http port [8000]: open
splunk_1 | Checking mgmt port [8089]: open
splunk_1 | Checking appserver port [127.0.0.1:8065]: open
splunk_1 | Checking kvstore port [8191]: Checking configuration... Done.
splunk_1 | Checking critical directories... Done
splunk_1 | Checking indexes...
splunk_1 | Validated: _audit _internal _introspection _thefishbucket history main summary
splunk_1 | Done
splunk_1 | open
splunk_1 | Checking filesystem compatibility... Done
splunk_1 | Checking conf files for problems...
splunk_1 | Done
splunk_1 | Checking default conf files for edits...
splunk_1 | Validating installed files against hashes from '/opt/splunk/splunk-6.4.0-f2c836328108-linux-2.6-x86_64-manifest'
splunk_1 | All installed files intact.
splunk_1 | Done
splunk_1 | All preliminary checks passed.
splunk_1 |
splunk_1 | Starting splunk server daemon (splunkd)...
splunk_1 | Done
splunk_1 |
splunk_1 |
splunk_1 |
splunk_1 | If you get stuck, we're here to help.
splunk_1 | Look for answers here: http://docs.splunk.com
splunk_1 |
splunk_1 | The Splunk web interface is at http://splunk:8000
splunk_1 |
There doesn't seem to be inputs/options described for that.
Can we do it?
Thanks
--Duncan
I have followed all the steps mentioned and it shows that the daemonset is running on each of the nodes, However, when I see the Splunk Dashboard, I see no logs. Any pointers?
How to debug deamonset functionality?
The branch 7.0.1 is not exposed to the Docker Repository. Wondering how close we are to have it available ?
At the Splunk startup page I get this error message:
Warning: The time on the server differs significantly from this machine which may cause login problems and other errors.
Is there any way to fix this?
I have been fighting very weird errors trying to get splunk working inside a kubernetes cluster.
Setup:
Mounting /opt/splunk/etc
and /opt/splunk/var
always gave me errors like https://answers.splunk.com/answers/312247/after-upgrading-a-search-head-cluster-to-splunk-63-1.html. Some of the resources it tried to get was also showing __raw/..../undefined/...
where the undefined part was ment to show the username (admin
). Lots of small things didn't work.
After a lot of trial and errors, I got it to work with mounting in separate directories under /opt/splunk/var
, like spool
and run
.. It was a lot of trial and errors. But now, splunk gave me errors like ERROR while running renew-certs migration.
and Warning: cannot create "/opt/splunk/var/run/splunk"
when kubernetes recreated it.
What seams to work is this:
SPLUNK_USER
set to root
/opt/splunk/etc
, /opt/splunk/var/lib
, and /opt/splunk/var/log
on their own.
/opt/splunk/var/log
for good measure.../opt/splunk/var
will give the errors above, even if run as root.How to reproduce this:
docker build . -t
It looks like the path to the file is wrong. No big deal, but thought you'd like to know.
-- Doug
I'm not sure if this version is still supported but I found 3 app files are failing file integrity check and yield different sha256 hashes in the docker image vs. direct tar install. The hashes in the manifest file are consistent with the direct tar installation but not with the files from docker. I can suppress the errors for now or edit the manifest, but both of those solutions are not ideal since I don't want to bake these into my cluster management scripts. I've checked on different machines and environments and arrived at the same result (some local, some staging and some live). Thanks!
Docker version:
docker version 18.03.1-ce
Steps to reproduce:
docker pull splunk/splunk:7.1.0
docker run -d -it splunk/splunk:7.1.0
docker exec -it <container_name> bash
./bin/splunk validate files
results in:
File '/opt/splunk/share/splunk/migration/app_contents_SplunkDeploymentMonitor.tar.gz' changed.
File '/opt/splunk/share/splunk/migration/app_contents_unix.tar.gz' changed.
File '/opt/splunk/share/splunk/migration/app_contents_windows.tar.gz' changed.
Compared with the tar.gz downloaded hashes they are different.
run sha256sum against these files:
sha256sum /opt/splunk/share/splunk/migration/app_contents_*
output:
b3f57820ec6af9c62d6685a6a7a7a2ff7f039be2712c04c1f190785afc34fdc4 /opt/splunk/share/splunk/migration/app_contents_SplunkDeploymentMonitor.tar.gz
75728e3fa3b43e7c9214f36df7cb483079d5d14511b754fd14b439bf0d1ad3bd /opt/splunk/share/splunk/migration/app_contents_unix.tar.gz
b141a423f3b7822673465776596fc8278c12e793b6b1f108045b063c975c130f /opt/splunk/share/splunk/migration/app_contents_windows.tar.gz
To grep the manifest file for the hashes it was expecting for these files (run from $SPLUNK_HOME):
grep "app_contents_" splunk-7.1.0-2e75b3406c5b-linux-2.6-x86_64-manifest
which results in:
f 444 splunk splunk splunk/share/splunk/migration/app_contents_SplunkDeploymentMonitor.tar.gz 3478cfae2593f6be92fc084f2d195c27be13e11441d4118116e27010a2a041d5
f 444 splunk splunk splunk/share/splunk/migration/app_contents_unix.tar.gz 98cc648a8a0c6901f7d3bb585e8597f410df628ee81e1c65082c63195794e283
f 444 splunk splunk splunk/share/splunk/migration/app_contents_windows.tar.gz 2ae56598076bee59f46823ae3957eb2f422be83976774493423684ab7281dd3e
The readme mentions
Start a Splunk Enterprise container and mount the necessary container volumes
docker run --name vsplunk -v /opt/splunk/etc -v /opt/splunk/var busybox
docker run --hostname splunk --name splunk --volumes-from=vsplunk -p 8000:8000 -d -e "SPLUNK_START_ARGS=--a
Should this be updated to use named volumes instead (as of Docker 1.9.0)?
The splunk container process exits immediately when running the steps specified in the README. I'm seeing the issue with 6.3.3 and 6.3.1. Below are the logs
$ docker logs splunk
Copying '/opt/splunk/etc/openldap/ldap.conf.default' to '/opt/splunk/etc/openldap/ldap.conf'.
Generating RSA private key, 1024 bit long modulus
.......................................++++++
............++++++
e is 65537 (0x10001)
writing RSA key
Generating RSA private key, 1024 bit long modulus
...................++++++
.........++++++
e is 65537 (0x10001)
writing RSA key
Moving '/opt/splunk/share/splunk/search_mrsparkle/modules.new' to '/opt/splunk/share/splunk/search_mrsparkle/modules'.
An unforeseen error occurred:
Exception: <type 'exceptions.OSError'>, Value: [Errno 39] Directory not empty: '/opt/splunk/share/splunk/search_mrsparkle/modules.new/converters'
Traceback (most recent call last):
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli.py", line 1150, in main
parseAndRun(argsList)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli.py", line 997, in parseAndRun
retVal = cList.getCmd(command, subCmd).call(argList, fromCLI = True)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli.py", line 280, in call
return self.func(args, fromCLI)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/control_api.py", line 30, in wrapperFunc
return func(dictCopy, fromCLI)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/_internal.py", line 160, in firstTimeRun
comm.moveItem(migration.PATH_UI_MOD_NEW, migration.PATH_UI_MOD_ACTIVE)
File "/opt/splunk/lib/python2.7/site-packages/splunk/clilib/cli_common.py", line 856, in moveItem
shutil.move(src, dst)
File "/opt/splunk/lib/python2.7/shutil.py", line 300, in move
rmtree(src)
File "/opt/splunk/lib/python2.7/shutil.py", line 247, in rmtree
rmtree(fullname, ignore_errors, onerror)
File "/opt/splunk/lib/python2.7/shutil.py", line 256, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/opt/splunk/lib/python2.7/shutil.py", line 254, in rmtree
os.rmdir(path)
OSError: [Errno 39] Directory not empty: '/opt/splunk/share/splunk/search_mrsparkle/modules.new/converters'
Please file a case online at http://www.splunk.com/page/submit_issue
This appears to be your first time running this version of Splunk.
Hi,
I have these env variables to suit our environment:
SPLUNK_START_ARGS="--accept-license"
SPLUNK_FORWARD_SERVER=splunk_server:9997
SPLUNK_USER=root
SPLUNK_ADD_1='monitor /var/log/containers -sourcetype docker_json'
After the container starts this is what the /opt/splunk/etc/system/local/outputs.conf
file looks like:
[tcpout]
defaultGroup = default-autolb-group
[tcpout:default-autolb-group]
server = splunk_server:9997
[tcpout-server://splunk_server:9997]
But I need to make changes for our environment like this otherwise the forwarder doesn't connect properly to the indexer:
[tcpout]
defaultGroup = default-autolb-group
[tcpout:default-autolb-group]
server = splunk_server:9997
[tcpout-server://splunk_server:9997]
sslCertPath = $SPLUNK_HOME/etc/auth/server.pem
sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem
sslVerifyServerCert = false
sslPassword = some_password
How could I achieve this?
I don't think it's possible to configure sslCertPath
, sslRootCAPath
etc using the Splunk CLI therefore I cannot use the SPLUNK_CMD
env variable.
I thought I could override ENTRYPOINT
and CMD
something like this:
ENTRYPOINT ["/bin/bash -c"]
CMD ["mkdir -p /opt/splunk/etc/system/local; echo '[tcpout]' > /opt/splunk/etc/system/local/outputs.conf; echo 'sslCertPath = /opt/splunk/etc/auth/server.pem' >> /opt/splunk/etc/system/local/outputs.conf; echo 'sslRootCAPath = /opt/splunk/etc/auth/cacert.pem' >> /opt/splunk/etc/system/local/outputs.conf; echo 'sslVerifyServerCert = false' >> /opt/splunk/etc/system/local/outputs.conf; /sbin/entrypoint.sh start-service"]
But this is really hacky and also I don't know how to find the sslPassword
parameter because it's created randomly in /opt/splunk/etc/system/local/server.conf
after splunk has started.
Am I missing something? Is there a simple way to do this? I'd rather not have to create my own custom Docker image.
Thanks,
Max
I am getting a low disk space warning (Under 5GB free) after a few weeks of use and it has caused Splunk to stop indexing and not allow users to use the web interface. Moving the Splunk setting to 2GB restored functionality, but is obviously a temporary solution.
I am using data volume containers as suggested in the readme.
When I run docker ps -s I get:
14.08 MB (virtual 546.3 MB)
Is there another way I should be checking disk usage?
Any ideas on how to fix this or what may be happening?
Thanks.
As an admin, I would like to manage my Splunk deployments on Kubernetes via helm
chart(s).
I am currently trying to write a helm chart for deploying a standalone heavy forwarder, but I am very new to k8, helm, and Splunk in general. Therefore, I'm concerned that what I write will be ... well, crap.
Are there any plans for the Splunk team to offer easily consumable helm chart(s) for maintaining Splunk clusters?
I can see charts for different use cases, such as:
I realize this repository is mostly for holding the base images / basic config for Splunk on docker images, so forgive me if the scope of this question is too broad
EDIT, I now see there's helm deployments in-progress. Just not my exact use-case: https://github.com/splunk/splunk-connect-for-kubernetes/tree/master/helm-chart/splunk-kubernetes-logging
How to stop auto upgrade on splunk docker?
Currently, in entrypoint.sh for Splunk Universal Forwarder, under start-service, it is using "admin:changeme" credential for a few commands, in which is hardcoded. This either needs to be configurable or removed (don't require authentication for such commands)
Hello, I'm moving from running the Splunk forwarder from an installed debian package to running in a container. We have configured the forwarder to monitor various files on disk, I am planning to mount those directories in my container, and I was wondering about the following discrepancy.
I see the Splunk .deb postinst script (from the Splunk forwarder 6.3.3) has something like
if [ ! -f "$SPLUNK_HOME/etc/splunk-launch.conf" ] ; then
sed "s%# SPLUNK_HOME=.*%SPLUNK_HOME=$SPLUNK_HOME%g" "$SPLUNK_HOME/etc/splunk-launch.conf.default" > "$SPLUNK_HOME/etc/splunk-launch.conf"
fi
We aren't explicitly configuring the splunk-launch.conf, and the default suits our needs after we drop a few files in /etc/system/local/
. This splunk-launch.conf isn't in the Dockerfile, and now I'm worried that I'm missing configuring a few other steps and environment variables.
How would you best approach this? Thanks.
Hi, We would like to add _meta as a command line argument with splunk forwarder cli which is at present not supported with cli, Can you add this as a feature request and also if any substitute is there please do let us know
Hi there,
I'm new to Splunk and the OPS team gave me the following files under the /opt/splunkforwarder/etc/system/local/
:
$ ls -la /opt/splunkforwarder/etc/system/local/
total 16
drwx------ 2 root root 74 Jan 9 00:29 .
drwx------ 3 root root 19 Jan 8 22:29 ..
-rw------- 1 root root 171 Jan 8 22:29 inputs.conf
-rwx------ 1 root root 195 Jan 8 22:29 outputs.conf
-r-------- 1 root root 265 Jan 8 22:29 README
-rw------- 1 root root 375 Jan 8 22:29 server.conf
According to http://blogs.splunk.com/2015/08/24/collecting-docker-logs-and-stats-with-splunk/, I'd like to configure a single SplunkForwarder container to collect syslog
for all the containers... I'd like also to disclose:
company
below is a BIG Splunk customer, but so far nobody from OPS team supports DockerThis is a host installation with the splunk-forwarder 6.2.x... The content of the files is as follows:
[root@pe2enpmas300 npmo-server]# cat inputs.conf
[default]
host = pe2enpmas300.corp.company.net
[monitor:///var/log/messages]
disabled = false
sourcetype = syslog
_blacklist = \.(gz)$
index= sp-njsnginx-reference-e2eidx
[root@pe2enpmas300 npmo-server]# cat outputs.conf
[tcpout]
defaultGroup = primary_indexers
[tcpout:primary_indexers]
server = oe2esstlg310.corp.company.net:9997, oe2esstlg311.corp.company.net:9997, oe2esstlg312.corp.company.net:9997
autoLB = true
[root@pe2enpmas300 npmo-server]# cat server.conf
[sslConfig]
sslKeysfilePassword = $1$Of8JPJZlRRS2
[lmpool:auto_generated_pool_forwarder]
description = auto_generated_pool_forwarder
quota = MAX
slaves = *
stack_id = forwarder
[lmpool:auto_generated_pool_free]
description = auto_generated_pool_free
quota = MAX
slaves = *
stack_id = free
[general]
pass4SymmKey = $1$brNdYNMjDka2
serverName = pe2enpmas300.corp.company.net
For instance, I just copied the files from the splunk forward from the host and I'm mounting in the data container... Let's start with the container in a separate docker-compose...
Since I'm running docker as root (with SSL enabled), then I opened port 514
....
splunkforwarder:
image: outcoldman/splunk:6.2.4-forwarder
restart: always
environment:
- SPLUNK_FORWARD_SERVER="oe2esstlg310.corp.company.net:9997,oe2esstlg311.corp.company.net:9997,oe2esstlg312.corp.company.net:9997"
ports:
- 514:514/udp
I just use extensions https://docs.docker.com/compose/extends/ and the ${HOSTNAME} variable substitution https://docs.docker.com/compose/compose-file/#variable-substitution
$ echo $HOSTNAME
pe2enpmas300.corp.company.net
For the configuration, I'm mounting it...
[root@pe2enpmas300 npmo-server]# ls -la monitor/splunk/
total 20
drwx------ 2 polkitd ssh_keys 95 Jan 9 01:00 .
drwx------ 3 root root 19 Jan 9 01:40 ..
-rw------- 1 polkitd ssh_keys 171 Jan 8 22:29 inputs.conf
-rw------- 1 polkitd ssh_keys 45 Jan 9 01:00 migration.conf
-rwx------ 1 polkitd ssh_keys 195 Jan 8 22:29 outputs.conf
-r-------- 1 polkitd ssh_keys 265 Jan 9 00:51 README
-rw------- 1 polkitd ssh_keys 375 Jan 8 22:29 server.conf
Here's the file Dockerfile
splunkforwarderData:
image: busybox
volumes:
- ./monitor/splunk:/opt/splunk/etc/system/local
splunkforwarder:
extends:
file: docker-compose-monitoring.yml
service: splunkforwarder
volumes_from:
- "splunkforwarderData"
newww:
build: roles/newww
restart: always
env_file: .env
expose:
- "5005"
ports:
- "80:8081"
log_driver: "syslog"
log_opt:
syslog-tag: "newww"
syslog-address: udp://${HOSTNAME}
Docker inspect command shows the mounted settings...
$ docker compose
"Mounts": [
{
"Source": "/npmo-data/npmo-server/monitor/splunk",
"Destination": "/opt/splunk/etc/system/local",
"Mode": "rw",
"RW": true
}
],
I verified that the file is in the container as well...
root@pe2enpmas300 npmo-server]# docker exec -ti npmoserver_splunkforwarder_1 bash
root@bb3bb53aba78:/opt/splunk# ls -la
total 96
drwxr-xr-x 9 splunk splunk 4096 Jan 9 09:53 .
drwxr-xr-x 3 root root 19 Oct 28 14:28 ..
drwxr-xr-x 3 splunk splunk 4096 Jun 26 2015 bin
-r--r--r-- 1 splunk splunk 57 Jun 26 2015 copyright.txt
drwxr-xr-x 13 splunk splunk 4096 Jan 9 08:52 etc
drwxr-xr-x 2 splunk splunk 26 Jun 26 2015 include
drwxr-xr-x 4 splunk splunk 4096 Jun 26 2015 lib
-r--r--r-- 1 splunk splunk 52503 Jun 26 2015 license-eula.txt
drwxr-xr-x 3 splunk splunk 55 Jun 26 2015 openssl
-r--r--r-- 1 splunk splunk 842 Jun 26 2015 README-splunk.txt
drwxr-xr-x 3 splunk splunk 39 Jun 26 2015 share
-r--r--r-- 1 splunk splunk 17634 Jun 26 2015 splunkforwarder-6.2.4-271043-linux-2.6-x86_64-manifest
drwxr-xr-x 6 splunk splunk 48 Jan 9 08:51 var
root@bb3bb53aba78:/opt/splunk# ls -la etc/system/local/
total 20
drwx------ 2 splunk splunk 95 Jan 9 09:00 .
drwxr-xr-x 7 splunk splunk 73 Jan 9 08:51 ..
-rw------- 1 splunk splunk 171 Jan 9 06:29 inputs.conf
-rw------- 1 splunk splunk 45 Jan 9 09:00 migration.conf
-rwx------ 1 splunk splunk 195 Jan 9 06:29 outputs.conf
-r-------- 1 splunk splunk 265 Jan 9 08:51 README
-rw------- 1 splunk splunk 375 Jan 9 06:29 server.conf
Any help is appreciated...
I think there is a typo in README.md at current master:
8088/tcp is mentioned 2x, but i guess the second 8088/tcp is supposed to be 8089/tcp - correct?
Hi there. We're having some challenges using this image since once the data volumes are created and configured, there is no way to modify the configs with future updates.
Say, for example, we want to add a new forward server, app, etc. I could update the ENV variables to accomplish this when first starting a container. But if the data volumes already exist, I have to either completely delete them or log into the container and run commands manually.
It might be nice to have an ENV variable to run splunk commands even if the server is already configured? Or perhaps a flag to re-run configuration regardless of what it finds in the volume (assuming that Splunk properly handles the case of identical commands being re-run)?
Thanks!
Hi,
After a long time of usage without any problem, I finally restarted the running container (after modifying the host hardware), now the image doesn't want to start any more on my previous volume.
btool.log shows the following
08-29-2016 09:18:07.764 ERROR IniFile - Cannot open file=/opt/splunk/etc/system/local/server.conf for parsing: Permission denied
08-29-2016 09:18:07.766 ERROR UsernameMapper - Cannot create username mapping file: /opt/splunk/etc/users/users.ini: Permission denied
08-29-2016 09:18:07.766 ERROR IniFile - Cannot open file=/opt/splunk/etc/users/users.ini for parsing: Permission denied
08-29-2016 09:18:07.766 ERROR UsernameMapper - Error opening username mapping file: /opt/splunk/etc/users/users.ini
08-29-2016 09:18:07.773 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/apps/launcher/metadata/local.meta: Permission denied
08-29-2016 09:18:07.773 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/apps/search/metadata/local.meta: Permission denied
08-29-2016 09:18:07.773 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/apps/splunk_management_console/metadata/local.meta: Permission denied
08-29-2016 09:18:07.773 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/apps/user-prefs/metadata/local.meta: Permission denied
08-29-2016 09:18:07.776 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/system/metadata/local.meta: Permission denied
08-29-2016 09:18:07.776 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/apps/learned/metadata/local.meta: Permission denied
The user rights of the folder look like the following (in the host, then user are not displayed as splunk:splunk) :
I tried to boot the image on a new volume, it seems the problem doesn't occur.
Do you have any idea how I can fix that ?
Thanks !
As a cloud foundry operator, I want to use Splunk to ingest my rsyslog
platform logs. Therefore, I would like the Splunk <version>-monitor
docker image to come pre-packed with the RFC5424 Syslog
application by default.
The *-monitor
image comes packed with the Docker Monitoring Dockerfile
goodie, and it would be very helpful for day-to-day operations if the RFC5424 Syslog
were packed as well. (
(Unfortunately, the provided syslog
source format does not seem to parse logs from cloud foundry's rsyslog
properly, whereas the RFC5424 Syslog does.)
Thank you for your consideration.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.