Comments (11)
ryanfaircloth
commented on June 14, 2024
what ucc lib build is in your requirements?
from addonfactory-ucc-generator.
guilhemmarchand
commented on June 14, 2024
Hope this answers to your question:
ucc-gen
Some functions will not be available outside of a splunk hosted process
2021-04-08 12:08:17,659 [UCC] INFO: Cleaning out directory /Users/guilhem/DocsOffsync/git/splunk/addonfactory-ucc-generator/tests/data/output
2021-04-08 12:08:17,828 [UCC] INFO: Cleaned out directory /Users/guilhem/DocsOffsync/git/splunk/addonfactory-ucc-generator/tests/data/output
2021-04-08 12:08:17,834 [UCC] INFO: Addon Version : 4.3.0Rea42e81b
2021-04-08 12:08:17,834 [UCC] INFO: Package ID is Splunk_TA_UCCExample
2021-04-08 12:08:17,834 [UCC] INFO: Copy UCC template directory
2021-04-08 12:08:17,878 [UCC] INFO: Copy globalConfig to output
2021-04-08 12:08:17,878 [UCC] INFO: Install Addon Requirements into /Users/guilhem/DocsOffsync/git/splunk/addonfactory-ucc-generator/tests/data/output/Splunk_TA_UCCExample/lib from package
Requirement already satisfied: pip in /Users/guilhem/.pyenv/versions/3.7.4/lib/python3.7/site-packages (21.0.1)
Collecting splunktaucclib==4.0.4
Using cached splunktaucclib-4.0.4-py2.py3-none-any.whl (86 kB)
Processing /Users/guilhem/Library/Caches/pip/wheels/56/b0/fe/4410d17b32f1f0c3cf54cdfb2bc04d7b4b8f4ae377e2229ba0/future-0.18.2-py3-none-any.whl
Collecting splunktalib<2.0,>=1.1
Using cached splunktalib-1.1.7-py2.py3-none-any.whl (48 kB)
Collecting solnlib<4,>=3
Using cached solnlib-3.0.5-py2.py3-none-any.whl (66 kB)
Processing /Users/guilhem/Library/Caches/pip/wheels/2b/ec/c9/af7b60cacdf01ded6ed2919d6456ba9592481523526f8d44be/splunk_sdk-1.6.15-py3-none-any.whl
Collecting httplib2<0.19,>=0.18
Using cached httplib2-0.18.1-py3-none-any.whl (95 kB)
Collecting sortedcontainers<3.0,>=2.2
Using cached sortedcontainers-2.3.0-py2.py3-none-any.whl (29 kB)
Collecting requests<3.0,>=2.24
Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting schematics<3.0,>=2.1
Using cached schematics-2.1.0-py2.py3-none-any.whl (49 kB)
Collecting six
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
Collecting idna<3,>=2.5
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.4-py2.py3-none-any.whl (153 kB)
Collecting chardet<5,>=3.0.2
Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Installing collected packages: future, httplib2, sortedcontainers, splunktalib, certifi, idna, urllib3, chardet, requests, schematics, splunk-sdk, six, solnlib, splunktaucclib
Successfully installed certifi-2020.12.5 chardet-4.0.0 future-0.18.2 httplib2-0.18.1 idna-2.10 requests-2.25.1 schematics-2.1.0 six-1.15.0 solnlib-3.0.5 sortedcontainers-2.3.0 splunk-sdk-1.6.15 splunktalib-1.1.7 splunktaucclib-4.0.4 urllib3-1.26.4
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Requirement already up-to-date: pip in /Users/guilhem/Library/Python/2.7/lib/python/site-packages (20.3.4)
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting future
Using cached future-0.18.0-cp27-none-any.whl (501 kB)
Installing collected packages: future
ERROR: pip's legacy dependency resolver does not consider dependency conflicts when selecting packages. This behaviour is the source of the following dependency conflicts.
splunk-appinspect 2.0.1 requires enum34==1.1.6; python_version < "3.4", but you'll have enum34 1.1.10 which is incompatible.
splunk-appinspect 2.0.1 requires future==0.17.1, but you'll have future 0.18.0 which is incompatible.
splunk-appinspect 2.0.1 requires ipaddress==1.0.22, but you'll have ipaddress 1.0.23 which is incompatible.
splunk-appinspect 2.0.1 requires six==1.12.0, but you'll have six 1.15.0 which is incompatible.
Successfully installed future-0.18.0
Requirement already satisfied: pip in /Users/guilhem/.pyenv/versions/3.7.4/lib/python3.7/site-packages (21.0.1)
Processing /Users/guilhem/Library/Caches/pip/wheels/56/b0/fe/4410d17b32f1f0c3cf54cdfb2bc04d7b4b8f4ae377e2229ba0/future-0.18.2-py3-none-any.whl
Installing collected packages: future
Successfully installed future-0.18.2
2021-04-08 12:08:25,109 [UCC] INFO: Replace tokens in views
INFO:UCC:Replace tokens in views
2021-04-08 12:08:25,219 [UCC] INFO: Copy package directory
INFO:UCC:Copy package directory
2021-04-08 12:08:25,225 [UCC] INFO: Copy LICENSES directory
INFO:UCC:Copy LICENSES directory
from addonfactory-ucc-generator.
guilhemmarchand
commented on June 14, 2024
cat package/lib/requirements.txt
splunktaucclib==4.0.4
from addonfactory-ucc-generator.
guilhemmarchand
commented on June 14, 2024
Some updates on this, I think I've isolated the problem and it seems not to be related with the UCC directly.
For some reasons, this seems to be happened under specific conditions in my lab when the app is "installed" via a volume in Docker, aka:
volumes:
- ./trackme:/opt/splunk/etc/apps/trackme
I'll run into more tests and will update.
from addonfactory-ucc-generator.
guilhemmarchand
commented on June 14, 2024
I have been carefully reviewing and testing, my previous statement was wrong and I can confirm that there seem to be an issue with the alert actions side of it from an Addon generated with UCC. (forget about the symlink thing, I've been testing with proper install so no questions about that)
My test is the following: (one simple param in the alert action)
addonfactory-ucc-generator/tests/data/globalConfig.json
},
"alerts": [
{
"name": "test_alert",
"label": "Test Alert",
"description": "Description for test Alert Action",
"activeResponse": {
"task": [
"Create",
"Update"
],
"supportsAdhoc": true,
"subject": [
"endpoint"
],
"category": [
"Information Conveyance",
"Information Portrayal"
],
"technology": [
{
"version": [
"1.0.0"
],
"product": "Test Incident Update",
"vendor": "Splunk"
}
],
"drilldownUri": "search?q=search%20index%3D\"_internal\"&earliest=0&latest=",
"sourcetype": "test:incident"
},
"entity": [
{
"type": "text",
"label": "Name",
"field": "name",
"defaultValue": "xyz",
"required": true,
"help": "Please enter your name"
}
]
}
],
"meta": {
"apiVersion": "3.2.0",
"name": "Splunk_TA_UCCExample",
"restRoot": "splunk_ta_uccexample",
"version": "4.3.0Rea42e81b",
"displayName": "Splunk UCC test Add-on",
"schemaVersion": "0.0.1"
}
}
The alert action itself does not do anything besides the helper.log:
# encoding = utf-8
def process_event(helper, *args, **kwargs):
"""
# IMPORTANT
# Do not remove the anchor macro:start and macro:end lines.
# These lines are used to generate sample code. If they are
# removed, the sample code will not be updated when configurations
# are updated.
[sample_code_macro:start]
# The following example gets the alert action parameters and prints them to the log
name = helper.get_param("name")
helper.log_info("name={}".format(name))
# The following example adds two sample events ("hello", "world")
# and writes them to Splunk
# NOTE: Call helper.writeevents() only once after all events
# have been added
helper.addevent("hello", sourcetype="test:incident")
helper.addevent("world", sourcetype="test:incident")
helper.writeevents(index="summary", host="localhost", source="localhost")
# The following example gets the events that trigger the alert
events = helper.get_events()
for event in events:
helper.log_info("event={}".format(event))
# helper.settings is a dict that includes environment configuration
# Example usage: helper.settings["server_uri"]
helper.log_info("server_uri={}".format(helper.settings["server_uri"]))
[sample_code_macro:end]
"""
helper.log_info("Alert action test_alert started.")
# TODO: Implement your alert action logic here
return 0
SPL test search: (in the app name space of Splunk_TA_UCCExample)
| makeresults | eval name="foo"
| sendalert test_alert param.name="$result.name$"
Splunk core 8.1.3: (python.version = python2 or python3 or default)
action fails
04-10-2021 08:16:04.958 ERROR sendmodalert - action=test_alert STDERR - Traceback (most recent call last):
04-10-2021 08:16:04.958 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/etc/apps/Splunk_TA_UCCExample/bin/test_alert.py", line 9, in <module>
04-10-2021 08:16:04.958 ERROR sendmodalert - action=test_alert STDERR - from splunktaucclib.alert_actions_base import ModularAlertBase
04-10-2021 08:16:04.958 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/etc/apps/Splunk_TA_UCCExample/lib/splunktaucclib/alert_actions_base.py", line 5, in <module>
04-10-2021 08:16:04.958 ERROR sendmodalert - action=test_alert STDERR - from logging_helper import get_logger
04-10-2021 08:16:04.958 ERROR sendmodalert - action=test_alert STDERR - ModuleNotFoundError: No module named 'logging_helper'
04-10-2021 08:16:04.976 INFO sendmodalert - action=test_alert - Alert action script completed in duration=197 ms with exit code=1
04-10-2021 08:16:04.976 WARN sendmodalert - action=test_alert - Alert action script returned error code=1
04-10-2021 08:16:04.976 ERROR sendmodalert - Error in 'sendalert' command: Alert script returned error code 1.
Splunk core 8.0.8: (python.version = python2 or python3 or default)
action fails
04-10-2021 08:17:36.795 ERROR sendmodalert - action=test_alert STDERR - Traceback (most recent call last):
04-10-2021 08:17:36.795 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/etc/apps/Splunk_TA_UCCExample/bin/test_alert.py", line 9, in <module>
04-10-2021 08:17:36.795 ERROR sendmodalert - action=test_alert STDERR - from splunktaucclib.alert_actions_base import ModularAlertBase
04-10-2021 08:17:36.795 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/etc/apps/Splunk_TA_UCCExample/lib/splunktaucclib/alert_actions_base.py", line 5, in <module>
04-10-2021 08:17:36.795 ERROR sendmodalert - action=test_alert STDERR - from logging_helper import get_logger
04-10-2021 08:17:36.795 ERROR sendmodalert - action=test_alert STDERR - ModuleNotFoundError: No module named 'logging_helper'
04-10-2021 08:17:36.809 INFO sendmodalert - action=test_alert - Alert action script completed in duration=220 ms with exit code=1
04-10-2021 08:17:36.809 WARN sendmodalert - action=test_alert - Alert action script returned error code=1
04-10-2021 08:17:36.809 ERROR sendmodalert - Error in 'sendalert' command: Alert script returned error code 1.
Splunk core 7.3.9:
action works.
Splunk core 7.2.10.1:
action works.
At the light of this extensive test, I can confirm there is an issue with UCC starting Splunk 8.x related to the alert actions.
The menus load successfully (crendentials, options, etc) load fine on all versions, but alert actions are failing on Python3 based Splunk core versions.
Any thoughts?
from addonfactory-ucc-generator.
guilhemmarchand
commented on June 14, 2024
Hi!
Would anyone had any idea on this issue?
I've got several apps I would love to swtich to UCC based libs, but these issues with alert actions are blocking me unfortunately.
Thanks
from addonfactory-ucc-generator.
guilhemmarchand
commented on June 14, 2024
The following fix is required, currently Addons created with ucc-gen simply cannot work on Splunk 8.x for the alert actions, which is in my opinion important enough to get fixed.
In the alert_actions_base.py that is generated by ucc-gen:
example:
Splunk_TA_UCCExample/lib/splunktaucclib/alert_actions_base.py
from logging_helper import get_logger
needs to be fixed to:
from splunktaucclib.logging_helper import get_logger
Which fixes the Python import issue:
04-16-2021 22:36:43.184 ERROR sendmodalert - action=test_alert STDERR - Traceback (most recent call last):
04-16-2021 22:36:43.184 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/etc/apps/Splunk_TA_UCCExample/bin/test_alert.py", line 9, in <module>
04-16-2021 22:36:43.184 ERROR sendmodalert - action=test_alert STDERR - from splunktaucclib.alert_actions_base import ModularAlertBase
04-16-2021 22:36:43.184 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/etc/apps/Splunk_TA_UCCExample/lib/splunktaucclib/alert_actions_base.py", line 5, in <module>
04-16-2021 22:36:43.184 ERROR sendmodalert - action=test_alert STDERR - from logging_helper import get_logger
04-16-2021 22:36:43.184 ERROR sendmodalert - action=test_alert STDERR - ModuleNotFoundError: No module named 'logging_helper'
04-16-2021 22:36:43.197 INFO sendmodalert - action=test_alert - Alert action script completed in duration=206 ms with exit code=1
04-16-2021 22:36:43.197 WARN sendmodalert - action=test_alert - Alert action script returned error code=1
04-16-2021 22:36:43.197 ERROR sendmodalert - Error in 'sendalert' command: Alert script returned error code 1.
Once the Python import is fixed, the next issue is:
04-16-2021 22:39:12.084 ERROR sendmodalert - action=test_alert STDERR - Traceback (most recent call last):
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/etc/apps/Splunk_TA_UCCExample/bin/test_alert.py", line 44, in <module>
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - exitcode = AlertActionWorkertest_alert("Splunk_TA_UCCExample", "test_alert").run(sys.argv)
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/etc/apps/Splunk_TA_UCCExample/lib/splunktaucclib/alert_actions_base.py", line 217, in run
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - self.prepare_meta_for_cam()
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/etc/apps/Splunk_TA_UCCExample/lib/splunktaucclib/alert_actions_base.py", line 203, in prepare_meta_for_cam
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - for num, result in enumerate(csv.DictReader(rf)):
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/lib/python3.7/csv.py", line 111, in __next__
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - self.fieldnames
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - File "/opt/splunk/lib/python3.7/csv.py", line 98, in fieldnames
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - self._fieldnames = next(self.reader)
04-16-2021 22:39:12.085 ERROR sendmodalert - action=test_alert STDERR - _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)
04-16-2021 22:39:12.110 INFO sendmodalert - action=test_alert - Alert action script completed in duration=556 ms with exit code=1
04-16-2021 22:39:12.110 WARN sendmodalert - action=test_alert - Alert action script returned error code=1
04-16-2021 22:39:12.110 ERROR sendmodalert - Error in 'sendalert' command: Alert script returned error code 1.
Which to get fixed requires the following block:
def prepare_meta_for_cam(self):
with gzip.open(self.results_file, 'rb') as rf:
for num, result in enumerate(csv.DictReader(rf)):
result.setdefault('rid', str(num))
self.update(result)
self.invoke()
break
to: (rb to rt)
def prepare_meta_for_cam(self):
with gzip.open(self.results_file, 'rb') as rf:
for num, result in enumerate(csv.DictReader(rf)):
result.setdefault('rid', str(num))
self.update(result)
self.invoke()
break
Then finally everything works as expected.
I was thinking in submitting a PR, thus I don't fully get how the alert_actions_base.py is generated.
Can we get that fixed?
from addonfactory-ucc-generator.
commented on June 14, 2024
@guilhemmarchand there are already PRs going in different repository to fix this issue:
- splunk/addonfactory-ucc-library#35 - fixes package import
- splunk/addonfactory-ucc-library#36 - fixes file reading method
UPD: PRs are merged into the main branch
from addonfactory-ucc-generator.
commented on June 14, 2024
@guilhemmarchand new version of ucc library was released (https://github.com/splunk/addonfactory-ucc-library/releases/tag/v4.1.1), can you please try it?
from addonfactory-ucc-generator.
guilhemmarchand
commented on June 14, 2024
@arys-splunk
Thanks for you answer, earlier when I checked it was still the case but that was prior to 4.1.1, so I will check and revert ASAP
from addonfactory-ucc-generator.
guilhemmarchand
commented on June 14, 2024
@arys-splunk
I am glad to confirm that UCC libs 4.1.1 fixed the issues I was observing on alert actions starting with Splunk 8.x, and wil be closing this issue.
Sorry I now realise the issue was on the lib repo, so I shall have opeend an issue there.
Thanks!
Guilhem
from addonfactory-ucc-generator.
Related Issues (20)
- BUG: openapi.json in wrong directory HOT 2
- BUG: Empty line in .uccignore results in files being deleted
- BUG: entry_page.js referred to in template doesn't exist HOT 2
- BUG: Configuration documentation for proxies is misleading
- Feature request: Run a linter during build command to spot and report possible parsing errors or warnings. HOT 3
- BUG: generated nav configuration should not default to configuration view HOT 3
- BUG: inputs with tabs fails 'DataClasses' object has no attribute 'table' HOT 4
- Feature request: Add a verbose mode to build command for file copy/overwrite report
- BUG: Multiple helplink with an empty label leads to breaking the configuration page due to duplicate HOT 6
- Feature request: Switch to enterprise dark theme via globalConfig.json HOT 3
- BUG: Fix child keys warning HOT 7
- Feature request: Deploy Storybook to GitHub pages HOT 5
- BUG: If Enter clicked on dialog, it does not behave as expected HOT 2
- Feature request: Number validators should be more specific about integers HOT 1
- Feature request: Make sure button with to indicator is available for TA developers HOT 2
- BUG: package command doesn't compress archive HOT 1
- BUG: Deprecation warning for placeholder doesn't appear in the console HOT 1
- BUG: fix input label accessibility
- BUG: multi-level menu pop-up animation breaking HOT 1
- Feature request: Preserve ucc version used when creating the TA HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from addonfactory-ucc-generator.