The package seems to use the following non-standard paths: <div class="snippet-cli

pkg_resources <a href="https://setuptools.pypa.io/en/

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Package uses non-standard site-package paths for resources,about janosh/matbench-discovery

pbenner commented on June 12, 2024 1

The [...]/site-packages/data directory is created when installing the package and it contains the figshare files, because of:

matbench-discovery/pyproject.toml

Line 76 in 13cbb90

matbench_discovery = ["../data/figshare/*"]

One solution would be to move the data/figshare folder into the matbench_discovery folder. But I guess it would take a few more modifications to turn it into a real python package. For instance, this is not compatible:

matbench-discovery/matbench_discovery/__init__.py

Lines 13 to 19 in 13cbb90

    
           ROOT = os.path.dirname(os.path.dirname(__file__))  # repo root directory 
        
           DATA_DIR = f"{ROOT}/data"  # directory to store raw data 
        
           SITE_FIGS = f"{ROOT}/site/src/figs"  # directory for interactive figures 
        
           SITE_MODELS = f"{ROOT}/site/src/routes/models"  # directory to write model analysis 
        
           FIGSHARE = f"{ROOT}/data/figshare" 
        
           SCRIPTS = f"{ROOT}/scripts" 
        
           PDF_FIGS = f"{ROOT}/paper/figs"  # directory for light-themed PDF figures

The ROOT variable points to the [...]/site-package directory.

from matbench-discovery.

janosh commented on June 12, 2024

pkg_resources is deprecated. Maybe we should use importlib.resources. I'll have to read up on that.

from matbench-discovery.

janosh commented on June 12, 2024

@pbenner are the /tmp and /data directories empty or actually used for anything. maybe the solution is just to get rid of this or check if the package is using an editable install before running this.

matbench-discovery/matbench_discovery/__init__.py

Lines 20 to 23 in 1098aa6

    
           for directory in [SITE_FIGS, SITE_MODELS, FIGSHARE, PDF_FIGS]: 
        
               os.makedirs(directory, exist_ok=True)

from matbench-discovery.

janosh commented on June 12, 2024

@pbenner Sorry this took ages to address. I think I fixed the supporting data directories being created in root-level /site-packages in 55ab66b. This dynamic way of deciding where data files go feels like a bad solution:

matbench-discovery/matbench_discovery/__init__.py

Lines 15 to 20 in 55ab66b

    
           # check if package is installed in editable mode 
        
           # hopefully better method coming in https://github.com/pypa/setuptools/issues/4186 
        
           pkg_is_editable = os.path.isdir(f"{PKG_DIR}/../matbench_discovery.egg-info") 
        
           # repo root directory if editable install, else the pkg directory 
        
           ROOT = os.path.dirname(PKG_DIR) if pkg_is_editable else PKG_DIR

I opened pypa/setuptools#4186 to either design a less hacky way of implementing this or maybe learn how to better address the underlying issue.

from matbench-discovery.

Package uses non-standard site-package paths for resources about matbench-discovery HOT 4 CLOSED

Comments (4)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	ROOT = os.path.dirname(os.path.dirname(__file__)) # repo root directory
	DATA_DIR = f"{ROOT}/data" # directory to store raw data
	SITE_FIGS = f"{ROOT}/site/src/figs" # directory for interactive figures
	SITE_MODELS = f"{ROOT}/site/src/routes/models" # directory to write model analysis
	FIGSHARE = f"{ROOT}/data/figshare"
	SCRIPTS = f"{ROOT}/scripts"
	PDF_FIGS = f"{ROOT}/paper/figs" # directory for light-themed PDF figures


	for directory in [SITE_FIGS, SITE_MODELS, FIGSHARE, PDF_FIGS]:
	os.makedirs(directory, exist_ok=True)

	# check if package is installed in editable mode
	# hopefully better method coming in https://github.com/pypa/setuptools/issues/4186
	pkg_is_editable = os.path.isdir(f"{PKG_DIR}/../matbench_discovery.egg-info")

	# repo root directory if editable install, else the pkg directory
	ROOT = os.path.dirname(PKG_DIR) if pkg_is_editable else PKG_DIR