edyvision / pii-codex Goto Github PK

It seems that when US_SSN is detected in the sentence it always errors to the below error message:

"Exception: An error occurred while processing the detected entity US_SSN"

Traceback:

  File "C:\Python_Local\Cerebro\cerebro-flask-api\venv\lib\site-packages\pii_codex\services\analysis_service.py", line 75, in analyze_item
    analysis, sanitized_text = self._perform_text_analysis(
  File "C:\Python_Local\Cerebro\cerebro-flask-api\venv\lib\site-packages\pii_codex\services\analysis_service.py", line 280, in _perform_text_analysis
Exception: An error occurred while processing the detected entity US_SSN

After looking closer in the code it seems that this entity type is missing from the csv attached in the data folder.

file: pii_mapping_util.py

    def __init__(self):
        self._pii_mapping_data_frame = open_pii_type_mapping_csv("v1")

file: file_util.py

    file_path = get_relative_path(
        f"../data/{mapping_file_version}/{mapping_file_name}.csv"
    )

The file contains PII_Type = "US_SOCIAL_SECURITY_NUMBER" instead of "US_SSN"

Same exception happens for bank number as well:
Exception: An error occurred while processing the detected entity US_BANK_NUMBER

  File "C:\Python_Local\Cerebro\cerebro-flask-api\venv\lib\site-packages\pii_codex\services\assessment_service.py", line 21, in assess_pii_type
    return PII_MAPPER.map_pii_type(detected_pii_type)
  File "C:\Python_Local\Cerebro\cerebro-flask-api\venv\lib\site-packages\pii_codex\utils\pii_mapping_util.py", line 45, in map_pii_type
    raise Exception(
Exception: An error occurred while processing the detected entity US_BANK_NUMBER

The file contains PII_Type = "US_BANK_ACCOUNT_NUMBER" instead of "US_BANK_NUMBER".

Hope this helps.

Thank you

It's happening because US_BANK_NUMBER is not present in PII_Type column in pii_type_mappings.csv file. So either in microsoft_presidio_pii.py , value of US_BANK_ACCOUNT_NUMBER should be changed from US_BANK_NUMBER to US_BANK_ACCOUNT_NUMBER or in pii_type_mappings.csv , US_BANK_ACCOUNT_NUMBER should be changed to US_BANK_NUMBER

Thanks in advance

I'm facing a problem on linux in config > PII_MAPPER() > open_pii_type_mapping_csv func. In that pd.read_csv(file) gives UnicodeDecodeError: 'ascii' codec cannot decode byte 0xe2 in position 2598:ordinal not in range(128).

If anyone knows about this, Please respond to this issue

Thank you

hi,

Do you guys have examples for docs/UC1_Converting_Existing_Detections_With_Adapters.png?

Thanks
GP

Just wanted to write a note of thanks for this work. This has been incredibly useful for us in designing a tool that sniffs out the PIIs from data storage.