sabhyac26 / ner_experiments Goto Github PK

0.0 2.0 0.0 808 KB

Python 43.28% Makefile 0.15% Jupyter Notebook 56.57%

ner_experiments's Introduction

ner_experiments

python3 -m venv env
source env/bin/activate
pip3 install -r requirements.txt
cd src
python3 -m bilstm_crf_runner --out './out/' --glove './util/embeddings/glove/' --run-id 3

wandb link

https://wandb.ai/sabhyac26/ner_experiments/overview

ner_experiments's People

Contributors

Watchers

ner_experiments's Issues

Check for 0s in non pad idxs

When fixing the masking, I saw 0s in non pad positions --> could be a bug

ner_experiments/src/model.py

Lines 45 to 50 in b1ffb30

    
               @todo Check for 0s in non pad idxs 
        
               @body When fixing the masking, I saw 0s in non pad positions --> could be a bug 
        
               """ 
        
               def create_mask(self, src:torch.LongTensor) -> torch.LongTensor: 
        
                   mask = (src != self.pad_idx).permute(0, 1) 
        
                   return mask

This issue was generated by todo based on a `todo` comment in `b1ffb30`. It's been assigned to @SabhyaC26 because they committed the code.

Add support for using pre trained word embeddinngs

Add support for senna (?), glove, word2vec

ner_experiments/src/model.py

Lines 53 to 58 in 71e3ba1

    
               @todo Add support for using pre trained word embeddinngs 
        
               @body Add support for senna (?), glove, word2vec 
        
               """ 
        
               def forward(self, input:torch.LongTensor, input_lens:torch.LongTensor, 
        
                           labels:torch.LongTensor, decode:bool) -> Dict[str, any]: 
        
                   embedded = self.dropout(self.embeddings(input))

This issue was generated by todo based on a `todo` comment in `71e3ba1`. It's been assigned to @SabhyaC26 because they committed the code.

need to find a better way to do imports

ner_experiments/src/data/conll_char.py

Lines 9 to 14 in 6e9b8be

    
           # @todo: need to find a better way to do imports 
        
           sys.path.append(os.path.abspath('../util/')) 
        
           from util.conll_util import UNK 
        
           from util.util import device

This issue was generated by todo based on a `todo` comment in `6e9b8be`. It's been assigned to @SabhyaC26 because they committed the code.

check if dropout needs to be applied on embeddings

ner_experiments/src/model.py

Lines 64 to 68 in 3667b59

    
           # @todo check if dropout needs to be applied on embeddings 
        
           embedded = self.dropout(self.embedding_layer(src)) 
        
           packed_embedded = rnn.pack_padded_sequence(embedded, input_lens, batch_first=True, enforce_sorted=False) 
        
           packed_output, hidden = self.lstm(packed_embedded) 
        
           out, _ = rnn.pad_packed_sequence(packed_output, batch_first=True)

This issue was generated by todo based on a `todo` comment in `3667b59`. It's been assigned to @SabhyaC26 because they committed the code.

figure out the mechanics of char tensors word tensors

ner_experiments/src/models/bilstm_cnn_crf.py

Lines 15 to 17 in 6e9b8be

    
           # @todo figure out the mechanics of char tensors word tensors 
        
           def forward(self, src: torch.LongTensor, char: torch.LongTensor): 
        
               pass

This issue was generated by todo based on a `todo` comment in `6e9b8be`. It's been assigned to @SabhyaC26 because they committed the code.

will need a new collate function for this

ner_experiments/src/bilstm_cnn_crf_runner.py

Lines 36 to 41 in 6e9b8be

    
               # @todo: will need a new collate function for this 
        
               # train_dataloader = DataLoader(dataset=train_data, batch_size=args.batch_size, shuffle=True, collate_fn=pad_batch) 
        
               # val_dataloader = DataLoader(dataset=val_data, batch_size=args.batch_size, shuffle=True, collate_fn=pad_batch) 
        
           if __name__ == "__main__": 
        
               parser = argparse.ArgumentParser(description='Args for BiLSTM_CRF')

This issue was generated by todo based on a `todo` comment in `6e9b8be`. It's been assigned to @SabhyaC26 because they committed the code.

Change weight inits?

I think the random weight inits are causing vanishing gradients - try to fix this with other weight inits

W&B

Try to add Weights and Biases to track runs and monitor progress of the project.

	@todo Check for 0s in non pad idxs
	@body When fixing the masking, I saw 0s in non pad positions --> could be a bug
	"""
	def create_mask(self, src:torch.LongTensor) -> torch.LongTensor:
	mask = (src != self.pad_idx).permute(0, 1)
	return mask

	@todo Add support for using pre trained word embeddinngs
	@body Add support for senna (?), glove, word2vec
	"""
	def forward(self, input:torch.LongTensor, input_lens:torch.LongTensor,
	labels:torch.LongTensor, decode:bool) -> Dict[str, any]:
	embedded = self.dropout(self.embeddings(input))

	# @todo: need to find a better way to do imports
	sys.path.append(os.path.abspath('../util/'))
	from util.conll_util import UNK
	from util.util import device

	# @todo check if dropout needs to be applied on embeddings
	embedded = self.dropout(self.embedding_layer(src))
	packed_embedded = rnn.pack_padded_sequence(embedded, input_lens, batch_first=True, enforce_sorted=False)
	packed_output, hidden = self.lstm(packed_embedded)
	out, _ = rnn.pad_packed_sequence(packed_output, batch_first=True)

	# @todo figure out the mechanics of char tensors word tensors
	def forward(self, src: torch.LongTensor, char: torch.LongTensor):
	pass

	# @todo: will need a new collate function for this
	# train_dataloader = DataLoader(dataset=train_data, batch_size=args.batch_size, shuffle=True, collate_fn=pad_batch)
	# val_dataloader = DataLoader(dataset=val_data, batch_size=args.batch_size, shuffle=True, collate_fn=pad_batch)

	if __name__ == "__main__":
	parser = argparse.ArgumentParser(description='Args for BiLSTM_CRF')

sabhyac26 / ner_experiments Goto Github PK

ner_experiments's Introduction

ner_experiments

wandb link

ner_experiments's People

Contributors

Watchers

ner_experiments's Issues

This issue was generated by todo based on a todo comment in b1ffb30. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a todo comment in 71e3ba1. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a todo comment in 6e9b8be. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a todo comment in 3667b59. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a todo comment in 6e9b8be. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a todo comment in 6e9b8be. It's been assigned to @SabhyaC26 because they committed the code.

Recommend Projects

Recommend Topics

Recommend Org

This issue was generated by todo based on a `todo` comment in `b1ffb30`. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a `todo` comment in `71e3ba1`. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a `todo` comment in `6e9b8be`. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a `todo` comment in `3667b59`. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a `todo` comment in `6e9b8be`. It's been assigned to @SabhyaC26 because they committed the code.

This issue was generated by todo based on a `todo` comment in `6e9b8be`. It's been assigned to @SabhyaC26 because they committed the code.