Data enrichment data not showing in logs about openobserve HOT 11 CLOSED

@RVaidhy can you share backend logs as well..also please share details like how big is file , size in mb , number of records , number of columns etc

from openobserve.

Hi Oasisk,

Thanks so much for your swift response.

what do you mean by backend logs here ? the ingested log details or enrichment csv details ? please clarify.

If it is generated logs I should not share it here publicly since it contains sensitive info.

On the another hand,

The enrichment csv file contains, 3 columns.

repo_name, package,repo_type.

sample_repo_maven, maven, virtual

sample_repo_npm, npm, public

sample_repo_pypi, pypi, private

Overall rows is 5333 approximately. *

To confirm that the ingested file is in csv format and size of 198kb.

Neither under the streams section I am not be able to see the doc count for this enrichment table not the values under logs section.

What logs you are referring, I mean, could you please share me the location from where you wanted to have this logs for analysis. So that I can share by excluding only the sensitive informations like IP,hostname etc. Please be verbose, am not getting you here.

Thanks,
R Vaidhy

from openobserve.

Kindly excuse the typo. It was rows, not Columns. It contains 3 columns only.

Overall rows is 5333 approximately. *

from openobserve.

Update :-

Just now i have tested. 2 things here.

1. For the enrichment csv file, I believe there is some restriction. May be in the form of a size / number of rows.
   Now I have removed some of the rows (5300 to 299) to test, now the row count is showing. But in our case, we have 5300 folders to be processed / enriched and we need to do that map/parse for 5300 values from csv. This sample would help you during the debug time, so sharing here.

Name. Type. Doc Num Ingested Data. compressed size
sample-1 | enrichment_tables | 299 | 0.03 MB | 0.01 MB

2. Even after seeing this count here, under the logs section I am unable to see the uploaded enrichment data as mentioned in the openobserve document.

https://openobserve.ai/docs/user-guide/functions/enrichment/#upload-sample-data

Please refer this from your official documents page.

" You could see the contents in logs page "

Thanks,
R Vaidhy

from openobserve.

@RVaidhy thx for sharing details , the issue is fixed with #1560

from openobserve.

Hello Aashish,

Thank you so much. Let me test and update back you.

Thanks,
R Vaidhy

from openobserve.

Hello Aashish,

I have tested the new version in our environment multiple times. It is not working.

Even i did a fresh installation of openobserve in 3 different servers and tested individually. But it is not showing the values / column under the logs section as mentioned in the document.

But under the streams section I did see the doc count, size information now in this new version.

For your kind ref, I have used this latest version to test this functionality.

https://github.com/openobserve/openobserve/releases/download/v0.6.3/openobserve-v0.6.3-linux-amd64-musl.tar.gz

Please let me know incase if I am using a different version or kindly retest the fix one time please.

We have a csv which consists of 5335 row and its associated values as I mentioned.

Thanks,
R Vaidhy

from openobserve.

Hello Aashish,

I have tested the new version in our environment multiple times. It is not working.

Even i did a fresh installation of openobserve in 3 different servers and tested individually. But it is not showing the values / column under the logs section as mentioned in the document.

But under the streams section I did see the doc count, size information now in this new version.

For your kind ref, I have used this latest version to test this functionality.

https://github.com/openobserve/openobserve/releases/download/v0.6.3/openobserve-v0.6.3-linux-amd64-musl.tar.gz

Please let me know incase if I am using a different version or kindly retest the fix one time please.

We have a csv which consists of 5335 row and its associated values as I mentioned.

Thanks, R Vaidhy

Can you please create a separate issue for this

from openobserve.

Sure Prabhath. Let me do that.

Thanks,
Vaidhy R

from openobserve.

Hi Prabhath,

Please find the ticket details as needed. Please help.

Enrichment data successfully got ingested / loaded but unable to see the same under the logs section. #1578

Thanks,
R Vaidhy

from openobserve.

closing as #1578 is closed

from openobserve.