wkiri / demud Goto Github PK

View Code? Open in Web Editor NEW

44.0 44.0 16.0 753 KB

Discovery via Eigenbasis Modeling of Uninteresting Data

License: Apache License 2.0

Python 100.00%

demud's People

Contributors

Stargazers

Watchers

Forkers

chen0031 tacaswell osalpekar bhaprayan eteq coljac abhik1368 jakehlee jwgu butterkaffee arnabkar xin-alice sanjnar nbertagnolli stjordanis geoguxm

demud's Issues

Add progress indicator to feat_csv.py

Sweep k values for DEMUD

Deploy parallel jobs on halo/aurora to compute AUC over a variety of k values (ImageNet + stonefly9).

Request supercomputer access for Jake

Demud not defined

Hello, I'm trying to run demud on Spyder on python 3 but it's not defined.

Make "static" mode more efficient

Currently a static SVD result is achieved by simply not updating the SVD model at each iteration. However, the remaining pool of unselected items gets re-scored and ranked each time, even though this ranking will not change.

Suggestion: modify code so "static" path bypasses/shortcuts this iteration. A single ranking of the items should be enough.

Update ENVI data processing to work with user-modified files

Abby F. has an example ENVI file for which DEMUD isn’t reading band names correctly (assumption that band number is followed by ':' is incorrect). It also doesn't read the data correctly - or the data format is wrong - it gets about half of the expected data values. She will send me both original and modified files (and .hdr files) so I can debug this.

Demud error messages while installing

Dependency errors due to NIMS support

Summary:

Fresh installation with master throws an error due to nims_data dependency in dataset_envi.py.

Reproduction:

git clone
demud --make-config
run demud on anything

Description:

5dd4769 added

DEMUD/demud/demud.py

Lines 723 to 726 in 5dd4769

    
           if (isinstance(ds, SegENVIData) or 
        
               isinstance(ds, ENVIData)): 
        
             ds.write_RGB(os.path.join(outdir, 
        
                                       '%s-rgb-viz.png' % ds.name.split('-')[1]))

now requiring

DEMUD/demud/demud.py

Lines 38 to 39 in 5dd4769

    
           from dataset_envi import ENVIData 
        
           from dataset_envi import SegENVIData

e0fa4ef added

DEMUD/demud/dataset_envi.py

Line 28 in e0fa4ef

from nims_data import NimsQube

where nims_data seems to be a “non-standard” dependency

Workaround:

comment out line 28 from dataset_envi.py for any non-ENVI/NIMS run:

DEMUD/demud/dataset_envi.py

Line 28 in e0fa4ef

from nims_data import NimsQube

synset_words unused by feat_csv.py; remove?

Order of features changes selections

e.g. for dataset_des.py on Y3_GOLD_2_2_subsampled_filtered.npy. I get the same selection 0, but after that different selections, when swapping the order of the first two features.

Find out why and whether this is desired behavior.

dataset_des.py: Read in supplemental file with column names

Switch default algorithm for --increm

Two methods have been implemented for incrementally updating the SVD model: increm-ross and increm-brand. --increm currently defaults to --svdmethod=increm-ross. However, comparing the discovery results from these methods and the full method (which recomputes the SVD model from scratch at each iteration) shows that the results of increm-ross significantly deviate from full, whereas increm-brand performs similarly to full.

Therefore, --increm should default to --svdmethod=increm-brand, and the message for --svdmethods should be updated as such.

	if (isinstance(ds, SegENVIData) or
	isinstance(ds, ENVIData)):
	ds.write_RGB(os.path.join(outdir,
	'%s-rgb-viz.png' % ds.name.split('-')[1]))

	from dataset_envi import ENVIData
	from dataset_envi import SegENVIData