grafana-toolbox / grafana-client Goto Github PK

Grafana can support multiple organisations within a single instance. The organisation that requests are made to is stateful; that is, a request is made to change the user's context to an organisation, and all requests are then made against that organisation. This statefulness can lead to trouble if the org context is not changed at the beginning of the execution or if it is changed during execution.

Including the orgId query parameter will cause grafana to automatically change the user's context and returns a 302 Found for the same URL. This results in a retry of the request in the correct org.

Proposal: add an org_id parameter to GrafanaApi, which is passed to GrafanaClient, which adds it to self.s.params. If you'd like this, I can contribute the MR.

Hi there,

currently, grafana-client employs testing solely by mocking HTTP conversations. Because this does not cover real integration tests with Grafana, it is easy to miss regressions, like those which recently happened here.

• #44
• #47
• grafana/grafana#59546

On the other hand, both grafana-wtf, and Kotori, already feature such an integration test suite with Grafana. In this manner, the grafana-wtf test harness was able to catch this issue.

• grafana-toolbox/grafana-wtf#44

So, grafana-client will need to adopt that test suite soon, in order to improve compatibility and QA on different versions of Grafana, by adding them to a test matrix.

grafana-wtf/tests can be an inspiration for this. While being at it, we may split this off into a generic pytest-grafana package, in the same spirit like the recently conceived pytest-mqtt package.

With kind regards,
Andreas.

Hi there,

coming from #30, I am wondering if grafana-client could also provide an alternative way of creating alert rules.

@Maharacha asked at #30 (comment):

Is it possible to use /api/ruler/grafana/api/v1/rules/<rule_group> instead? That is what the UI is using. When using that endpoint instead of the provisioning API endpoints supported with #30, it is possible to edit and delete corresponding entities from the Grafana UI.

With kind regards,
Andreas.

Hi,

@changdingfang submitted a fix to m0nhawk/grafana_api#87, which indicates there might be a problem. Maybe it is useful to bring this improvement to grafana-client as well?

With kind regards,
Andreas.

Describe the bug
When using grafana_api.teams.add_team(), I am observing errors.

To Reproduce
Steps to reproduce the behavior:

1. Create Grafana Api client
2. Search team (not found)
3. Create team
4. Call add_team method
5. See error - grafana_client.client.GrafanaClientError: Client Error 404: Not found

   grafana_api = GrafanaApi(auth=GRAFANA_API_KEY, host=GRAFANA_URL)
   grafana_teams = grafana_api.teams.search_teams(perpage=100, page=1, query=team_name)
   
   grafana_team = next((team for team in grafana_teams if team["name"] == team_name), None)
   
   if grafana_team is None:
     team = {
         "name": team_name
         }
     new_team = grafana_api.teams.add_team(json.dumps(team))

Expected behavior
We should be able to create a team in grafana.

Versions

• Grafana: [ 8.4.5]
• grafana-client: [ 2.3.0]
• Authentication: Token

Additional context
If I call the same endpoint using postman I'm able to create the team

@bhks pointed us to corresponding code, where all the possible filtering parameters are enumerated. Thanks. The library may support them in the future.

https://github.com/grafana/grafana/blob/main/pkg/api/plugins.go#L48

Originally posted by @amotl in #115 (comment)

I am using the auth proxy authentication to access Grafana with an apache webserver infront of Grafana (documetation: configure auth proxy authentication).

Support for this kind of auth would be very nice. I think it could be quite easy to implement this feature. I can create a pull request if it is desired.

BR

What API call do you use?

grafana.folder.delete_folder(uid=folder_uid) ¹

What is the actual output?

requests.exceptions.JSONDecodeError: [Errno Expecting value] : 0 ²

What is the expected output?

It worked before on Grafana 9.2.4.

References

• grafana-toolbox/grafana-wtf#44

I develop some functions i need, its a parcial functionality but i could make a pull request if you want to.

I use this fork https://github.com/einar-lanfranco/grafana-client

Hi, i just wanted to know if any support for alerting provisioning api is planned?
or is there any library suggestions for workaround where i can make/update alerting rules by code?

reference:
https://grafana.com/docs/grafana/latest/developers/http_api/alerting_provisioning/

Thanks!

Hi there,

@jeremybz / @jeremybanzhaf asked at ¹:

My goal is to use the API to get the JSON representation of a dashboard, including __inputs and named datasources.

@jangaraj answered:

HTTP API for dashboard export for external use doesn’t exist. This export feature is implemented in the UI (frontend). I would say the only option is to use standard dashboard API and make that output transformed into “dashboard format for external use”. Unfortunately, that format is not documented, so you have to use reverse engineering.

Torkel Ödegaard said at ²:

This is not possible, the data is captured by the frontend so you would have to replicate what all the panel & data sources do (issue metric queries etc).

In order to implement this transformation, I would be happy to provide a place within this package, if possible. Would that make any sense / be helpful in any way?

The implementation should adhere to the specifications defined by the corresponding TypeScript code.

• DashboardModel::getSaveModelClone()
• DashboardModel::getPanelSaveModels()
• DashboardExporter.ts
• DashboardExporter.test.ts

With kind regards,
Andreas.

/cc @jangaraj

What Grafana API endpoint the library is missing?
Not an endpoint as such but a parameter on the create_folder method to include parentUid for those using nested folders.

Additional Info
https://grafana.com/docs/grafana/latest/developers/http_api/folder/#create-folder

Describe the bug

result = grafana.datasource.smartquery(datasource=datasource, expression='openstack_nova_total_vms')

I have called smartquery for the type=prometheus.

This function doesn't take from and to keys which is required to call grafana for the prometheus query, otherwise it is giving 200 ok but the output doesn't contain any data. But if provided with timestamp in from and to it gives the data.

This is the result:

{'results': {'A': {'status': 200, 'frames': []}}}

As of now timeout is set to 5 seconds and there is no way to change it. Please add this option to from_url method.

Thanks and regards, Mikhail

Describe the bug
A clear and concise description of what the bug is.
I see this
AttributeError: type object 'GrafanaApi' has no attribute 'from_url'

To Reproduce
Steps to reproduce the behavior:

1. Install latest grafana-client using pip install grafana-client==2.3.0

2. Run with valid key

from grafana_client import GrafanaApi

# Use Grafana API token.
grafana = GrafanaApi.from_url(
    url="https://grafana.example.org/grafana",
    auth="eyJrIjoiWHg...dGJpZCI6MX0=",
)

Expected behavior
I see the method in the repo code but not in my site packages when I do pip install.
Expecting this to run error free.

Versions

• Grafana: [ 8.5.2]
• grafana-client: [ 2.3.0]
• Authentication: [Token]

Additional context
Add any other context about the problem here.

Describe the bug

I have started looking at the Grafana Client python library. Whatever the method I try from Grafana Client, I get the 404 error.

Error

Traceback (most recent call last):
  File "D:\Nandan\PythonScripts\Grafana API Calls.py", line 19, in <module>
    print(grafana.datasource.list_datasources())
  File "D:\Python\Python39\lib\site-packages\grafana_client\elements\datasource.py", line 144, in list_datasources
    r = self.client.GET(list_datasources_path)
  File "D:\Python\Python39\lib\site-packages\grafana_client\client.py", line 156, in __request_runner
    raise GrafanaClientError(
grafana_client.client.GrafanaClientError: Client Error 404: Not found

Expected behavior
I am able to make these calls from Postman Web Agent and I am successful in getting the response back.

Versions

• Grafana: [V9.2.4]
• grafana-client: [API]
• Authentication: [Token]

Additional context
print(grafana.datasource.list_datasources())

Above is the simple method I am using the get the response.

Dear @chenlujjj and @jangaraj,

Originally coming from How to find out unused datasources?, based on our discussion at Finding unhealthy data sources, we finally made a start to lay out the foundation with #19. Health checks for a few data source types will be supported already ¹, but we will have to complete the list diligently, which requires dedicated work.

Corresponding documentation how to work with the health check subsystem on behalf of an example program examples/datasource-health.py can be inspected at examples/datasource-health.rst. The program is intended to evaluate the new subsystem with different databases, in order to gradually improve the implementation. We tried to make it easily usable for others to run in order to support this endeavor.

When the feature is reasonably ready over here, we will return to grafana-toolbox/grafana-wtf#19 in order to continue the discussion how to use it within grafana-wtf appropriately.

If this resonates with you, you might want to lend a hand? All kinds of feedback, both in terms of testing and further contributions will be greatly appreciated. Thank you very much in advance.

With kind regards,
Andreas.

Usage

The documentation at examples/datasource-health.rst will guide you through a full development sandbox installation, including running Grafana and some database services as Docker containers, and setting up the working tree from the Git repository.

In this section, we outline an alternative approach how to work with the feature using the example program examples/datasource-health.py. It might save a few keystrokes, specifically when aiming to work on an existing infrastructure.

# Setup
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade git+https://github.com/panodata/grafana-client
wget https://raw.githubusercontent.com/panodata/grafana-client/main/examples/datasource-health.py

# Run
export GRAFANA_URL=http://daq.example.org:3000
export GRAFANA_TOKEN=eyJrIjoiUWVrYXJh....aWQiOjJ9
python datasource-health.py --type=influxdb --url=http://daq.example.org:8086

Details

The example program will create a data source item named probe-{dstype}, with the designated database target URL. Then, it will run a data source health check on it and report about its outcome.

This will work well in situations when running the database services as Docker containers, as outlined in the reference documentation at examples/datasource-health.rst. It might not work well in other situations, where the data source configuration might need further adjustments.

In this case, don't hesitate to adjust the datasource_factory() code correspondingly to match your setup.

References

• Original feature request: grafana-toolbox/grafana-wtf#19
• Discussion: #20
• First patch: #19
• Documentation: examples/datasource-health.rst
• Example program: examples/datasource-health.py

Hi there,

are there any plans to support/switch to an async client like httpx in place of requests?

It would be useful for using it within web services without blocking concurrency.

Teams Actual User belongs to

Describe and link the Grafana API endpoint which are currently missing.

Async client does not have the API endpoint that lets one check which teams a user is a member of. I intend to use this to check whether a user needs to be added to a team (or if the user already is, in which case I don't do the write)

What API call you use?
Annotations.add_annotation()

What is the actual output?
TypeError: Annotations.add_annotation() got an unexpected keyword argument 'dashboard_uid'

What is the expected output?
https://grafana.com/docs/grafana/latest/http_api/annotations/#create-annotation

Describe the bug

Grafana-client is no longer compatible with the data source permissions APIs of grafana Enterprise 10.2.3.

https://grafana.com/blog/2024/01/08/grafana-10.2.3-release-new-features-and-breaking-changes

The following data source permission endpoints have been removed:

• GET /datasources/:datasourceId/permissions
• POST /api/datasources/:datasourceId/permissions
• DELETE /datasources/:datasourceId/permissions
• POST /datasources/:datasourceId/enable-permissions
• POST /datasources/:datasourceId/disable-permissions

replaced by:

• GET /api/access-control/datasources/:uid for listing data source permissions
• POST /api/access-control/datasources/:uid/users/:id, POST /api/access-control/datasources/:uid/teams/:id
• POST /api/access-control/datasources/:uid/buildInRoles/:id for adding or removing data source permissions

Versions

• Grafana: 10.2.3
• grafana-client: 3.11.0

Describe the bug
Hello team,

I am trying to rotate a service account key for a GCP Monitoring data source in Grafana. I generate the new key in GCP, I save it in the Secrets manager, but when I update the datasource with the proper key, it simply stops working. I believe it's because the \n newline characters are escaped to \\n and the data source is saved with the wrong key.

To Reproduce
Please review the code snippet below. The key I get from GCP is already a JWT key with newline characters in place, e.g. -----BEGIN PRIVATE KEY-----\nReStOfThEkEy\n.

decoded_key = base64.b64decode(key).decode("utf-8")
key_dict = json.loads(decoded_key)
try:
    datasource_object = self.client.datasource.find_datasource('my-test-data-source')
except GrafanaClientError as exception:
    logging.error(exception)
    sys.exit(1)

datasource_uid = datasource_object["uid"]

datasource_object["jsonData"]["authenticationType"] = "jwt"
datasource_object["jsonData"]["clientEmail"] = key_dict["client_email"]
datasource_object["jsonData"]["defaultProject"] = key_dict["project_id"]
datasource_object["jsonData"]["tokenUri"] = key_dict["token_uri"]
datasource_object["secureJsonFields"]["privateKey"] = key_dict["private_key"]

result = self.client.datasource.update_datasource_by_uid(
    datasource_uid, datasource_object
)
logging.info(f"{result['message']}, uid: {datasource_uid}")

Expected behavior
I can update the JWT keys for my GCP integrations.

Versions

• Grafana: v10.1
• grafana-client: 9.3.1
• Authentication: Token

Basically as the title says. As im working under proxcies, I need this argument to be able to use requests, otherwise I get connection errors.

Hi there,

it looks like the database field will be deprecated, and will move to jsonData.

• grafana/grafana#59115
• grafana/grafana#58647

With kind regards,
Andreas.

https://github.com/panodata/grafana-client/blob/d53574f6c787c0d2bc6b9ecef68166ac4f778786/grafana_client/model.py#L17-L32

The search.search_dashboards endpoint has incomplete / incorrect parameters.

See https://grafana.com/docs/grafana/latest/developers/http_api/folder_dashboard_search/

The following parameters are missing:

• dashboardUids
• folderUids

The following parameters are included but aren't supported by the current API:

• folderIds

Versions

• Grafana: v10.4.1 (d3ce857c0e)
• grafana-client: git main head (4.0.1)
• Authentication: Token

What API call you use?

• Based on this PR grafana/grafana#58410 which was merged in the new version 9.3.0, we can the header x-disable-provenance to allow the alert rules created under the alert provisioning API to be modifiable via UI.

This can only be added on the POST request, if we try to update with the PUT operation an alert has already been created we got the following error:

{
    "message": "cannot changed provenance from 'api' to ''",
    "traceID": ""
}

What is the actual output?

• Without the header when the rule is created we cannot edit the alert rules via UI, we got the label "provisioned".

What is the expected output?
And what output is expected.

• Alerts created with the header can be edited via UI like in the s screenshot below
  

What Grafana API endpoint the library is missing?

There is no update_datasource_by_uid function, only update_datasource, which takes the ID rather than UID. Using the ID is deprecated.

See Update an existing data source Grafana API documentation.

Quick work-around until implemented:

  update_datasource = f"/datasources/uid/{uid}"
  grafana.client.PUT(update_datasource, json=datasource)

Describe the bug

I might be totaly wrong by it seems this path this method tries to access does not exist.
At least not in the grafana docs..

  10  -> grafana.search.search_dashboards(tag="fake")                                                                                                                                                                                         
(Pdb++) grafana.search.search_dashboards(tag="fake")
[{'id': 1, 'uid': 'b071f222-28cf-4025-a13c-14b4adde3071', 'title': 'Palette-Fake', 'uri': 'db/palette-fake', 'url': '/d/b071f222-28cf-4025-a13c-14b4adde3071/palette-fake', 'slug': '', 'type': 'dash-db', 'tags': ['fake'], 'isStarred': False, 'sortMeta': 0}]
(Pdb++) grafana.dashboard.
client                        get_dashboard                 get_dashboard_permissions     get_home_dashboard            update_dashboard_permissions  
delete_dashboard              get_dashboard_by_name         get_dashboards_tags           update_dashboard              
(Pdb++) grafana.dashboard.get_dashboard_by_name("palette-fake")
*** grafana_client.client.GrafanaClientError: Client Error 404: Not found
Traceback (most recent call last):
  File "/home/oznt/.local/share/virtualenvs/reporting-JW0JCX-S/lib/python3.11/site-packages/grafana_client/elements/dashboard.py", line 26, in get_dashboard_by_name
    r = self.client.GET(get_dashboard_path)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oznt/.local/share/virtualenvs/reporting-JW0JCX-S/lib/python3.11/site-packages/grafana_client/client.py", line 156, in __request_runner
    raise GrafanaClientError(

Versions

• Grafana: 9.1.0
• grafana-client: 3.5.0
• Authentication: Token.

1. Need "get_alertrule" api without need for an "alertrule_uid" as an argument
   seems like this api support was added in 9.4

GET /api/v1/provisioning/alert-rules
https://github.com/panodata/grafana-client/blob/main/grafana_client/elements/alertingprovisioning.py#L9-L17

Describe the bug
Got an error

File /home/ffy/workspace/PyProjects/venv/lib/python3.12/site-packages/grafana_client/elements/dashboard.py:1
----> 1 from distutils.version import LooseVersion
      3 from .base import Base
      6 class Dashboard(Base):

ModuleNotFoundError: No module named 'distutils'

while import grafana_client in python 3.12.
Based on PEP386, distutils is deprecated and we may have some alternatives.

To Reproduce
Steps to reproduce the behavior:

1. python 3.12
2. import grafana_client

Expected behavior
ModuleNotFoundError: No module named 'distutils'

Versions

Additional context

Describe the bug
The folders.get_all_folders endpoint provides the parent_uid, if set, in data. This puts it in the body of the resulting API request. This needs to be a query parameter, however.

https://grafana.com/docs/grafana/latest/developers/http_api/folder/#get-all-folders

To Reproduce
Steps to reproduce the behavior:

1. Create a Folder in grafana. Create a subfolder folder within it. Note that nested folders needs to be enabled on the Grafana instance
2. Run client.folders.get_all_folders(parent_uid=<uid of created parent>)
3. You'll get the contents of the root folder

Expected behavior
Folders contained within the provided parent_uid should be returned.

Versions

• Grafana: v10.4.1 (d3ce857c0e)
• grafana-client: git main head
• Authentication: Token

Additional context
I've quickly tested that the problem is resolved if I change data=data to params=data in the __request_runner call to s.request() in client.py. Obviously, this is not a viable solution.

The infrastructure needs to be modified to allow params to be passed through to the final request call, the same way data presently is.

Grafana has User and Org Preferences API:
https://grafana.com/docs/grafana/latest/developers/http_api/preferences/
Couldn't find anything suitable. For example, how to set the default dashboard (homeDashboard) for an organization or a specific user.
Are there any plans?