mvasil / fashion-compatibility Goto Github PK

View Code? Open in Web Editor NEW

152.0 152.0 42.0 20 KB

Learning Type-Aware Embeddings for Fashion Compatibility

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

fashion-compatibility's People

Contributors

Stargazers

Watchers

Forkers

xun-yang tkhan3 matteotomassetti silenceky timothyfitz zwcdp mmbell0525 songfgh qa276390 junsangpark bbugs alessandrow jerrylee80 nikitakaraevv joanding pachongchong peternara andy-wagner debobanerjee cvhadessun ahwhbc ruoniwang jindl465 sahana-m rongisdragon deryann dhruvbatta umarsubhanmalhi chengkai-huang maodong2056 dspoorthy fed03 clisveryok hosna-darvishi nguyentritai2906 john-james-ai monib110 robertoarayaday sushantgiri sshiravy iq-scm 1503-firmansyah-indra tengboyuu korgmp3

fashion-compatibility's Issues

File "tripletnet.py", line 48, in forward x = x / norm.expand_as(x) RuntimeError: The expanded size of the tensor (64) must match the existing size (256) at non-singleton dimension 1. Target sizes: [256, 64]. Tensor sizes: [256]

(fashcomp) [jalal@goku fashion-compatibility]$ python main.py --name test_baseline --learned --l2_embed --datadir ../../../data/fashion/
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn("The use of the transforms.Scale transform is deprecated, " +
  + Number of params: 3191808
<class 'torch.utils.data.dataloader.DataLoader'>
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Traceback (most recent call last):
  File "main.py", line 324, in <module>
    main()    
  File "main.py", line 167, in main
    train(train_loader, tnet, criterion, optimizer, epoch)
  File "main.py", line 202, in train
    acc, loss_triplet, loss_mask, loss_embed, loss_vse, loss_sim_t, loss_sim_i = tnet(anchor, far, close)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/scratch3/research/code/fashion/fashion-compatibility/tripletnet.py", line 147, in forward
    loss_sim_t, desc_x, desc_y, desc_z = self.text_forward(x, y, z)
  File "/scratch3/research/code/fashion/fashion-compatibility/tripletnet.py", line 112, in text_forward
    desc_x = self.text_branch(x.text)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/scratch3/research/code/fashion/fashion-compatibility/tripletnet.py", line 48, in forward
    x = x / norm.expand_as(x)
RuntimeError: The expanded size of the tensor (64) must match the existing size (256) at non-singleton dimension 1.  Target sizes: [256, 64].  Tensor sizes: [256]

error while training

Hi
When I try to train the model, I got this error:

`Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /root/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth

gaierror Traceback (most recent call last)
/opt/conda/lib/python3.7/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
1349 h.request(req.get_method(), req.selector, req.data, headers,
-> 1350 encode_chunked=req.has_header('Transfer-encoding'))
1351 except OSError as err: # timeout error

/opt/conda/lib/python3.7/http/client.py in request(self, method, url, body, headers, encode_chunked)
1280 """Send a complete request to the server."""
-> 1281 self._send_request(method, url, body, headers, encode_chunked)
1282

/opt/conda/lib/python3.7/http/client.py in _send_request(self, method, url, body, headers, encode_chunked)
1326 body = _encode(body, 'body')
-> 1327 self.endheaders(body, encode_chunked=encode_chunked)
1328

/opt/conda/lib/python3.7/http/client.py in endheaders(self, message_body, encode_chunked)
1275 raise CannotSendHeader()
-> 1276 self._send_output(message_body, encode_chunked=encode_chunked)
1277

/opt/conda/lib/python3.7/http/client.py in _send_output(self, message_body, encode_chunked)
1035 del self._buffer[:]
-> 1036 self.send(msg)
1037

/opt/conda/lib/python3.7/http/client.py in send(self, data)
975 if self.auto_open:
--> 976 self.connect()
977 else:

/opt/conda/lib/python3.7/http/client.py in connect(self)
1442
-> 1443 super().connect()
1444

/opt/conda/lib/python3.7/http/client.py in connect(self)
947 self.sock = self._create_connection(
--> 948 (self.host,self.port), self.timeout, self.source_address)
949 self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

/opt/conda/lib/python3.7/socket.py in create_connection(address, timeout, source_address)
706 err = None
--> 707 for res in getaddrinfo(host, port, 0, SOCK_STREAM):
708 af, socktype, proto, canonname, sa = res

/opt/conda/lib/python3.7/socket.py in getaddrinfo(host, port, family, type, proto, flags)
751 addrlist = []
--> 752 for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
753 af, socktype, proto, canonname, sa = res

gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

URLError Traceback (most recent call last)
/tmp/ipykernel_27/37231806.py in
341
342 if name == 'main':
--> 343 main()

/tmp/ipykernel_27/37231806.py in main()
133 batch_size=args.batch_size, shuffle=False, **kwargs)
134
--> 135 model = resnet18(pretrained=True, embedding_size=args.dim_embed)
136 csn_model = TypeSpecificNet(args, model, len(test_loader.dataset.typespaces))
137

/kaggle/working/Resnet_18.py in resnet18(pretrained, **kwargs)
116 if pretrained:
117 state = model.state_dict()
--> 118 loaded_state_dict = model_zoo.load_url(model_urls['resnet18'])
119 for k in loaded_state_dict:
120 if k in state:

/opt/conda/lib/python3.7/site-packages/torch/hub.py in load_state_dict_from_url(url, model_dir, map_location, progress, check_hash, file_name)
589 r = HASH_REGEX.search(filename) # r is Optional[Match[str]]
590 hash_prefix = r.group(1) if r else None
--> 591 download_url_to_file(url, cached_file, hash_prefix, progress=progress)
592
593 if _is_legacy_zip_format(cached_file):

/opt/conda/lib/python3.7/site-packages/torch/hub.py in download_url_to_file(url, dst, hash_prefix, progress)
455 file_size = None
456 req = Request(url, headers={"User-Agent": "torch.hub"})
--> 457 u = urlopen(req)
458 meta = u.info()
459 if hasattr(meta, 'getheaders'):

/opt/conda/lib/python3.7/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
220 else:
221 opener = _opener
--> 222 return opener.open(url, data, timeout)
223
224 def install_opener(opener):

/opt/conda/lib/python3.7/urllib/request.py in open(self, fullurl, data, timeout)
523 req = meth(req)
524
--> 525 response = self._open(req, data)
526
527 # post-process response

/opt/conda/lib/python3.7/urllib/request.py in _open(self, req, data)
541 protocol = req.type
542 result = self._call_chain(self.handle_open, protocol, protocol +
--> 543 '_open', req)
544 if result:
545 return result

/opt/conda/lib/python3.7/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
501 for handler in handlers:
502 func = getattr(handler, meth_name)
--> 503 result = func(*args)
504 if result is not None:
505 return result

/opt/conda/lib/python3.7/urllib/request.py in https_open(self, req)
1391 def https_open(self, req):
1392 return self.do_open(http.client.HTTPSConnection, req,
-> 1393 context=self._context, check_hostname=self.check_hostname)
1394
1395 https_request = AbstractHTTPHandler.do_request

/opt/conda/lib/python3.7/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
1350 encode_chunked=req.has_header('Transfer-encoding'))
1351 except OSError as err: # timeout error
-> 1352 raise URLError(err)
1353 r = h.getresponse()
1354 except:

URLError: <urlopen error [Errno -3] Temporary failure in name resolution>`

Could any one help me?

fasttext as an alternative to embed text

Thanks for providing the tip in #15, where you mentioned that we could use fasttext as an alternative for embedding text, i.e. we could get per word vectors from fasttext pre-trained model. I wonder how we should concat/process these per word vectors to represent the entire product description/name? (i.e. each word vector is 300 dimension and I'd like the final embeddings to be 300 as well...)

Thanks in advance!

File "/scratch3/research/code/fashion/fashion-compatibility/polyvore_outfits.py", line 235, in sample_negative choice = np.random.choice(candidate_sets) File "mtrand.pyx", line 907, in numpy.random.mtrand.RandomState.choice ValueError: a must be 1-dimensional or an integer

(fashcomp) [jalal@goku fashion-compatibility]$     python main.py --name test_baseline --learned --l2_embed --datadir ../../../data/fashion/
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn("The use of the transforms.Scale transform is deprecated, " +
  + Number of params: 3191808
Traceback (most recent call last):
  File "main.py", line 322, in <module>
    main()    
  File "main.py", line 167, in main
    train(train_loader, tnet, criterion, optimizer, epoch)
  File "main.py", line 194, in train
    for batch_idx, (img1, desc1, has_text1, img2, desc2, has_text2, img3, desc3, has_text3, condition) in enumerate(train_loader):
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
    return self._process_data(data)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
    data.reraise()
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/_utils.py", line 425, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "mtrand.pyx", line 905, in numpy.random.mtrand.RandomState.choice
TypeError: 'dict_keys' object cannot be interpreted as an integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/scratch3/research/code/fashion/fashion-compatibility/polyvore_outfits.py", line 338, in __getitem__
    neg_im = self.sample_negative(outfit_id, pos_im, item_type)
  File "/scratch3/research/code/fashion/fashion-compatibility/polyvore_outfits.py", line 235, in sample_negative
    choice = np.random.choice(candidate_sets)
  File "mtrand.pyx", line 907, in numpy.random.mtrand.RandomState.choice
ValueError: a must be 1-dimensional or an integer

and

$ pip freeze
absl-py==0.13.0
argon2-cffi==20.1.0
attrs==21.2.0
backcall==0.2.0
bleach==4.1.0
cachetools==4.2.2
certifi==2021.5.30
cffi==1.14.6
charset-normalizer==2.0.4
cycler==0.10.0
debugpy==1.4.1
decorator==5.0.9
defusedxml==0.7.1
entrypoints==0.3
google-auth==1.35.0
google-auth-oauthlib==0.4.5
grpcio==1.39.0
h5py==3.3.0
idna==3.2
importlib==1.0.4
ipykernel==6.2.0
ipython==7.26.0
ipython-genutils==0.2.0
ipywidgets==7.6.3
jedi==0.18.0
Jinja2==3.0.1
joblib==1.0.1
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==7.0.1
jupyter-console==6.4.0
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.0
kiwisolver==1.3.1
Markdown==3.3.4
MarkupSafe==2.0.1
matplotlib==3.4.3
matplotlib-inline==0.1.2
mistune==0.8.4
nbclient==0.5.4
nbconvert==6.1.0
nbformat==5.1.3
nest-asyncio==1.5.1
notebook==6.4.3
numpy==1.21.2
oauthlib==3.1.1
packaging==21.0
pandas==1.3.2
pandocfilters==1.4.3
parso==0.8.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.3.1
prometheus-client==0.11.0
prompt-toolkit==3.0.20
protobuf==3.17.3
ptyprocess==0.7.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
Pygments==2.10.0
pyparsing==2.4.7
pyrsistent==0.18.0
python-dateutil==2.8.2
pytz==2021.1
pyzmq==22.2.1
qtconsole==5.1.1
QtPy==1.10.0
requests==2.26.0
requests-oauthlib==1.3.0
rsa==4.7.2
scikit-learn==0.24.2
scipy==1.7.1
Send2Trash==1.8.0
six==1.16.0
sklearn==0.0
tensorboard==2.6.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
terminado==0.11.1
testpath==0.5.0
threadpoolctl==2.2.0
torch==1.9.0
torch-tb-profiler==0.2.1
torchaudio==0.9.0
torchvision==0.10.0
tornado==6.1
traitlets==5.0.5
typing-extensions==3.10.0.0
urllib3==1.26.6
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==2.0.1
widgetsnbextension==3.5.1

--learned without --use_fc does not work

The parameter embedding_size is in the docstring for TypeSpecificNet's init, and is referenced at https://github.com/mvasil/fashion-compatibility/blob/master/type_specific_network.py#L74 but is not actually in the init's parameter list.

How to generate hglmm pca fisher vector and typespaces.p file

I want to generate hglmm pca fisher vector and typespaces.p file for a new dataset. Much thanks if you would provide the script for me.

File "main.py", line 222, in train losses.update(loss_triplet.data[0], num_items) IndexError: invalid index of a 0-dim tensor. Use `tensor.item()` in Python or `tensor.item<T>()` in C++ to convert a 0-dim tensor to a number

(fashcomp) [jalal@goku fashion-compatibility]$ python main.py --name test_baseline --learned --l2_embed --datadir ../../../data/fashion/
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn("The use of the transforms.Scale transform is deprecated, " +
  + Number of params: 3191808
<class 'torch.utils.data.dataloader.DataLoader'>
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Traceback (most recent call last):
  File "main.py", line 324, in <module>
    main()    
  File "main.py", line 167, in main
    train(train_loader, tnet, criterion, optimizer, epoch)
  File "main.py", line 222, in train
    losses.update(loss_triplet.data[0], num_items)
IndexError: invalid index of a 0-dim tensor. Use `tensor.item()` in Python or `tensor.item<T>()` in C++ to convert a 0-dim tensor to a number

Loss function for learn similarity (just image embedding)

I believe that you want anchor image feature to embed farther than positive and negative embedding since positive and negative samples are the same clothing type. However, in your code you wrote

    disti_p = F.pairwise_distance(general_y, general_z, 2)
    disti_n1 = F.pairwise_distance(general_y, general_x, 2)
    disti_n2 = F.pairwise_distance(general_z, general_x, 2)
    loss_sim_i1 = self.criterion(disti_p, disti_n1, target)
    loss_sim_i2 = self.criterion(disti_p, disti_n2, target)

where general_x, general_y, general_z are representing anchor, negative, and positive data respectively.
I think it should instead be

    loss_sim_i1 = self.criterion(disti_n1, disti_p, target)
    loss_sim_i2 = self.criterion(disti_n2, disti_p, target)

Am I missing something?

Size mismatch for embeddingnet.masks.weight in pretrained model

Hi, I have a question about the number of type spaces in the pre-trained model.

As in the main.py, the default num_rand_embed is 4. But when I load the pre-trained model, it shows that the shape of embeddingnet.masks.weight is [66,64], while my current tnet is [4, 64]. May I know if the num_rand_embed in the pretrained model is 66?

Many thanks!

Is there a missing file?

I'm trying to use this repos https://github.com/googleinterns/compatible-product which uses the polyvore dataset in this repos. There is a file called polyvore_outfits.outside_proto.bytes which contains the color and pattern embedding list that's missing, does anyone know how to get or generate that file?

A

Coarse types

Hello,

I see from your dataloader that you use the attribute coarse_type for each product, which types have you used for the Maryland Polyvore dataset? In their case each item has very fine-grained category, have you mapped these categories to a coarser set of categories? Or do you work with the fine-grained categories?

Thanks!

How to use multiple gpus for tnet?

The papers says that the performance will be better if embedding size gets larger. In the pretrained model, the embedding size is 64. I want to train this model with embedding size =512.
In such case, I need to use multiple gpus, which should be dataparallel in pytorch.
tnet = nn.DataParallel(tnet).cuda() print('now we are using %d gpus'%torch.cuda.device_count())

However, it shows that

Expected tensor for argument #1 'input' to have the same device as tensor for argument #2 'weight'; but device 0 does not equal 1 (while checking arguments for cudnn_convolution)

Would you please tell me how to use multiple gpus to train this model?

geometric queries

In the paper it says:
Given an item x(u) of a certain type, show a collection of items {x(v)} of a different type that are all compatible with x (u) but dissimilar from each other
but how do I implement that???
I tried making a script which takes an input item and outputs the top 5 most compatible outfits but they are all the same color and there is no diversity.

File "main.py", line 240, in train print('Train Epoch: {} [{}/{}]\t' File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/_tensor.py", line 561, in format return object.format(self, format_spec) TypeError: unsupported format string passed to Tensor.format

(fashcomp) [jalal@goku fashion-compatibility]$ python main.py --name test_baseline --learned --l2_embed --datadir ../../../data/fashion/
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn("The use of the transforms.Scale transform is deprecated, " +
  + Number of params: 3191808
<class 'torch.utils.data.dataloader.DataLoader'>
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Traceback (most recent call last):
  File "main.py", line 329, in <module>
    main()    
  File "main.py", line 167, in main
    train(train_loader, tnet, criterion, optimizer, epoch)
  File "main.py", line 240, in train
    print('Train Epoch: {} [{}/{}]\t'
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/_tensor.py", line 561, in __format__
    return object.__format__(self, format_spec)
TypeError: unsupported format string passed to Tensor.__format__

how to train a model get the result like the pre-trained model

@BryanPlummer
your pre-trained model provide a compatibility AUC of 0.88 and fill-in-the-blank accuracy of 57.6
the following command?
python main.py --name {your experiment name} --learned --l2_embed

but as i can't got this result. when i set embed_size=128,the result is :test set: Compat AUC: 0.87 FITB: 57.0 at the first epoch.

also,i have found the image similar loss is wrong:

calculate image similarity loss on the general embedding

    disti_p = F.pairwise_distance(general_y, general_z, 2)
    disti_n1 = F.pairwise_distance(general_y, general_x, 2)
    disti_n2 = F.pairwise_distance(general_z, general_x, 2)
    loss_sim_i1 = self.criterion(disti_p, disti_n1, target)
    loss_sim_i2 = self.criterion(disti_p, disti_n2, target)

i think it should be like this,as the sample y and z is the same type.

calculate image similarity loss on the general embedding

    disti_p = F.pairwise_distance(general_y, general_z, 2)
    disti_n1 = F.pairwise_distance(general_y, general_x, 2)
    disti_n2 = F.pairwise_distance(general_z, general_x, 2)
    loss_sim_i1 = self.criterion(disti_n1,disti_p, target)
    loss_sim_i2 = self.criterion(disti_n2,disti_p, target)

About typespaces.p

fashion-compatibility/polyvore_outfits.py

Line 65 in 299b426

spaces = np.random.permutation(len(typespaces))

Hi, thanks for your code. I however have a question about how to generate a file of typesapces.p at first time. Above on that ref, the function of the load_typespaces should provide a original typespaces.p although I want to get a new typespaces.p at first.
So, how I can generate the file when the typespaces.p file is not saved in the disk.
Looking forward to your reply.

Different results in python 3

Thanks for sharing the code!

I ran the pre-trained model in python 3 to do predictions for my input images. The compatibility score I got in python 3 is different from those in python 2 using the same checkpoint file (the one you shared). Wondering if you know why this might be the case?

Maryland semantic category

Using the fine-grained categories you wouldn't be able to get enough samples to learn a projection. Instead, we used the index of the item, which roughly corresponded to a type. While this doesn't provide the same strict typing scenario that the Polyvore Outfits dataset does, it provides further evidence (coupled with the random assignment experiments) that there is some redundancy there and a different embedding for each pairwise comparison isn't strictly necessary.

So, the results of Maryland Dataset in your paper, how do you map the fine-grained categories to semantic (coarse) type?
I find the number of id in categoires.csv is smaller than Maryland dataset. Do you ignore some samples when you test on this dataset?

Thanks

Originally posted by @tianyu-su in #2 (comment)

The paper says 36w+ images, the dataset is only 26w+ images

when create the mask, can mask_len = 0?

Under the default parameter setting,
args.dim_embed = 64,
n_conditions = 66.
args.dim_embed / n_conditions = 0.9696,
mask_len = int(args.dim_embed / n_conditions) = int (0.9696) = 0
so mask_array is a matrix of zero.
Can I change the int() to round()?

            mask_array = np.zeros([n_conditions, args.dim_embed])
            mask_len = int(args.dim_embed / n_conditions)        # mask_len = round(args.dim_embed / n_conditions) ?
            for i in range(n_conditions):
                mask_array[i, i*mask_len:(i+1)*mask_len] = 1         # mask_array[i, min(args.dim_embed-1, i*mask_len):min(args.dim_embed-1,(i+1)*mask_len)] = 1

Some problem about parameter learned_metric.

Hello @mvasil ,
Thanks for your excellent work. There is a problem with parameter learned_metric. I had trained a model with learned_metric=True. But when I run my inference code, I find the distance calculated by the model is a negative number.

Using Pretrained model json.decoder.JSONDecodeError

Traceback (most recent call last):
File "fashion-compatibility/main.py", line 313, in
main()
File "fashion-compatibility/main.py", line 92, in main
meta_data = json.load(open(fn, 'r'))
File "/opt/conda/lib/python3.8/json/init.py", line 293, in load
return loads(fp.read(),
File "/opt/conda/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/opt/conda/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/opt/conda/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1011887 column 2 (char 34603008)

Polyvore dataset link is incorrect

Hi，

I tried to run this project but found the link of the dataset is incorrect, which direct met to a 404 page.

Is that URL a typo error?

return _VF.norm(input, p, _dim, keepdim=keepdim) # type: ignore[attr-defined] IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

(fashcomp) [jalal@goku fashion-compatibility]$ python main.py --name test_baseline --learned --l2_embed --datadir ../../../data/fashion/
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn("The use of the transforms.Scale transform is deprecated, " +
  + Number of params: 3191808
<class 'torch.utils.data.dataloader.DataLoader'>
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Traceback (most recent call last):
  File "main.py", line 324, in <module>
    main()    
  File "main.py", line 167, in main
    train(train_loader, tnet, criterion, optimizer, epoch)
  File "main.py", line 202, in train
    acc, loss_triplet, loss_mask, loss_embed, loss_vse, loss_sim_t, loss_sim_i = tnet(anchor, far, close)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/scratch3/research/code/fashion/fashion-compatibility/tripletnet.py", line 146, in forward
    acc, loss_triplet, loss_sim_i, loss_mask, loss_embed, general_x, general_y, general_z = self.image_forward(x, y, z)
  File "/scratch3/research/code/fashion/fashion-compatibility/tripletnet.py", line 74, in image_forward
    embedded_x, masknorm_norm_x, embed_norm_x, general_x = self.embeddingnet(x.images, c)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/scratch3/research/code/fashion/fashion-compatibility/type_specific_network.py", line 147, in forward
    masked_embedding = torch.nn.functional.normalize(masked_embedding, p=2.0, dim=2, eps=1e-10, out=None)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/functional.py", line 4428, in normalize
    denom = input.norm(p, dim, keepdim=True).clamp_min(eps).expand_as(input)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/_tensor.py", line 417, in norm
    return torch.norm(self, p, dim, keepdim, dtype=dtype)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/functional.py", line 1356, in norm
    return _VF.norm(input, p, _dim, keepdim=keepdim)  # type: ignore[attr-defined]
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

I have the following in type_specific_network.py:

if self.l2_norm:
    ##norm = torch.norm(masked_embedding, p=2, dim=1) + 1e-10
    ##masked_embedding = masked_embedding / norm.expand_as(masked_embedding)
    masked_embedding = torch.nn.functional.normalize(masked_embedding, p=2.0, dim=2, eps=1e-10, out=None)

visualize

Hi, I visualized your model in the read me file and I could get similarity output with this model but it couldn't give me the compatibility, could you help me to solve the problem?

Can't load the pre-trained model

I try to use the pre-trained model, but get the following error:
=> loading checkpoint '/home/fuhailin/runs/nondisjoint_l2norm/model_best.pth.tar' Traceback (most recent call last): File "main.py", line 312, in <module> main() File "main.py", line 138, in main checkpoint = torch.load(args.resume) File "/home/fuhailin/apps/anaconda3/envs/py36/lib/python3.6/site-packages/torch/serialization.py", line 387, in load return _load(f, map_location, pickle_module, **pickle_load_args) File "/home/fuhailin/apps/anaconda3/envs/py36/lib/python3.6/site-packages/torch/serialization.py", line 574, in _load result = unpickler.load() UnicodeDecodeError: 'ascii' codec can't decode byte 0xbe in position 2: ordinal not in range(128)

How to find the text features for items?

I want to use the text features in train_hglmm_pca6000.txt. But I cannot find any match for the fashion item with the text features.

I used your data loading code (polyvore_outfits.py line152~line175), and it turned out that self.im2desc is empty.

I also tried to run your main.py, and printed the size of self.im2desc in the dataloader. self.im2desc is still empty.

Number of typespaces

I am trying to use the already trained model in order to run it on a bunch of random outfits outside polyvore

{'accessories',
'all-body',
'bags',
'bottoms',
'hats',
'jewellery',
'outerwear',
'scarves',
'shoes',
'sunglasses',
'tops'}
How is the number of categories 11
and the number of typespaces 66
nc2 would make it 55, right?

Why do we want compatibility between the same categories,For eg:"hats and hats",
isthis only to account for the cases where there are multiple jtems of same category,
for eg: in same outfit , there might be multiple jewellery?

[using stable PyTorch 1.9] RuntimeError: The expanded size of the tensor (64) must match the existing size (66) at non-singleton dimension 2. Target sizes: [256, 66, 64]. Tensor sizes: [256, 66]

Regarding #13

If you run the code in the current stable version of PyTorch 1.9, we will have:

(fashcomp) [jalal@goku fashion-compatibility]$ python main.py --test --l2_embed --resume runs/nondisjoint_l2norm/model_best.pth.tar --datadir ../../../data/fashion/
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn("The use of the transforms.Scale transform is deprecated, " +
=> loading checkpoint 'runs/nondisjoint_l2norm/model_best.pth.tar'
=> loaded checkpoint 'runs/nondisjoint_l2norm/model_best.pth.tar' (epoch 5)
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Traceback (most recent call last):
  File "main.py", line 312, in <module>
    main()    
  File "main.py", line 149, in main
    test_acc = test(test_loader, tnet)
  File "main.py", line 244, in test
    embeddings.append(tnet.embeddingnet(images).data)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/scratch3/research/code/fashion/fashion-compatibility/type_specific_network.py", line 119, in forward
    masked_embedding = masked_embedding / norm.expand_as(masked_embedding)
RuntimeError: The expanded size of the tensor (64) must match the existing size (66) at non-singleton dimension 2.  Target sizes: [256, 66, 64].  Tensor sizes: [256, 66]

How to set args for best performance?

I am confused about the exact setting of "args" to achieve the best performance (fitb acc= 86.1, comp auc=0.98) in your paper. Would you please tell me how to set these arguments? For example, --use_fc, --learned_metric, etc.

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbe in position 2: invalid start byte

(fashcomp) [jalal@goku fashion-compatibility]$ python main.py --test --l2_embed --resume runs/nondisjoint_l2norm/model_best.pth.tar --datadir ../../../data/fashion/
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn("The use of the transforms.Scale transform is deprecated, " +
Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /home/grad3/jalal/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth
100.0%
=> loading checkpoint 'runs/nondisjoint_l2norm/model_best.pth.tar'
Traceback (most recent call last):
  File "main.py", line 312, in <module>
    main()    
  File "main.py", line 138, in main
    checkpoint = torch.load(args.resume)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/serialization.py", line 608, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/serialization.py", line 787, in _legacy_load
    result = unpickler.load()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbe in position 2: invalid start byte

$ pip freeze
h5py==3.3.0
joblib==1.0.1
numpy==1.21.2
Pillow==8.3.1
scikit-learn==0.24.2
scipy==1.7.1
sklearn==0.0
threadpoolctl==2.2.0
torch==1.9.0
torchaudio==0.9.0
torchvision==0.10.0
typing-extensions==3.10.0.0

Will post the solution later.

mvasil / fashion-compatibility Goto Github PK

fashion-compatibility's People

Contributors

Stargazers

Watchers

Forkers

fashion-compatibility's Issues

`Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /root/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth

calculate image similarity loss on the general embedding

calculate image similarity loss on the general embedding

Recommend Projects

Recommend Topics

Recommend Org