techforpalestine / boycott-israeli-consumer-goods-dataset Goto Github PK

raw data in json format from a pastebin link: https://github.com/TechForPalestine/boycott-israeli-consumer-goods-dataset/blob/main/raw/boycott_list_formatted.json

write a script that parses the data and imports into:

• data/brands/ using the brand_schema
• data/companies/ using the company_schema

create follow up issue(s) to go through the data and fill in missing fields where required.. preferably include list of files that need updating

Import the data in the attached csv.

1. Write a script that parses the csv into the schema provided here: https://github.com/TechForPalestine/boycott-israeli-consumer-goods-dataset/tree/main/schemas.
2. Run the scripts/export.sh script to update the output.json

All data in this file are "avoid".
No image urls exist in this file. Either raise a follow-up issue to add those, or work on as part of this issue.
Category can be retrieved from the "Main" column. Can ignore sub-categories for now.
Market is currently a country or region. This column needs to be converted to 2-character country code(s) https://www.iban.com/country-codes.
Reason can be operations_in_israel and updated later.

disoccupied_avoid.csv

The RFC proposes output formats of CSV and JSON.

Create scripts (in Python?) to read the YAML data and export as CSV and JSON.

Run those scripts and commit those artifacts to git.

Add pre-commit git hook that runs the scripts. Also add a Github Action that asserts that the CSV and JSON are up-to-date with the YAML before a PR can be merged.

The JSON and CSV exports should contain timetsamps for when they were first added (created_at) and when they were last updated (updated_at)

The timestamps should be in ISO 8601 format, in UTC timezone: yyyy-mm-ddTHH:mm:ss.SSSZ

The timestamps (especially updated_at) will be used by consumers of the data to understand what has changed since the last time they fetched the data.

Originally posted by @THM222 in #9 (comment)

Create a schema (perhaps similar to boycott-israeli-tech-companies-dataset?), and a script that performs schema validation.

Add the schema validation as a git hook and a Github Action.

Attached json files can be parsed and imported into this repo.
Alternatives in the json files map to alternatives_text in this schema: https://github.com/TechForPalestine/boycott-israeli-consumer-goods-dataset/blob/main/schemas/brand_schema.yaml
Reason can be operations_in_israel and be updated later.

witness_w_il.json

sweetgreen boycott is due to ceo not supporting ceasefire
current reasons enum only supports operations_in_israel, operations_in_settlements, which needs to be updated to include data such as sweetgreen

   "sweetgreen": {
     "id": "sweetgreen",
     "name": "Sweetgreen",
     "status": "avoid",
     "reasons": [],
     "countries": [
       "us"
     ],
     "website": "https://www.sweetgreen.com/",
     "logo_url": "https://upload.wikimedia.org/wikipedia/commons/d/dd/Sweetgreen_logo.svg",
     "description": "Sweetgreen is an American fast casual restaurant chain that is famous for its salads.[^1]\nJonathan Neman, CEO of restaurant chain Sweetgreen, vowed never to hire Harvard students who support ceasefire.[^2]\n\n[^1]: [https://en.wikipedia.org/wiki/Sweetgreen\n[^2]:](https://en.wikipedia.org/wiki/Sweetgreen/n[%5E2]:) https://apnews.com/article/israel-hamas-war-palestinians-companies-harvard-daecf3a387a689339dc41f35b773e063"
     }