Code Monkey home page Code Monkey logo

widget's Introduction

@kg6-sleipnir

AI/ML enthusiast and advocate for emoticons instead of emojis

I am currently a college student aiming for a Computer Science degree and have a passion for AI/ML projects.

I started my programming journey by forcing myself to create machine learning algorithms using only the C++ standard library without any prior knowledge of any of the C languages or machine learning. Through the careful reading and application of research papers, I managed to create several ML models from scratch. Some of the models I created include: logistic regression, linear regression, decision trees, and conditional random fields. Using some of these models, I have been able to create a system that can predict a word's part of speech using context from surrounding words in a sentence. This system achieved accuracies similar to those using models from public libraries such as Scikit-learn in Python and is what I consider to be one of my greatest achievements.

The original goal of my ML project was to create a chatbot program that I could have a full conversation with. As this was the first real project I ever started, the quality of my code was not what it is today. Nonetheless, the original code can still be seen in my Widget repository. With the recently emerging field of LLMs, I now know that if I were to complete this project with my original goals, it would have taken hundreds of thousands of dollars worth of GPUs months worth of time just to train the model. Safe to say my little 2060 is not going to cut it (even if it's overclocked :P).

widget's People

Contributors

kg6-sleipnir avatar

Stargazers

 avatar  avatar

Watchers

 avatar

widget's Issues

CRF Model Defaulting to "NUL" tag

CRF model efficiently learns when predicting Part Of Speech tag for words in a sentence using the "PosTagCRF" class derived from the "CRF" object. However, after attaining a certain level of accuracy, the model seems to default to the "NUL" tag with the tag index of "tags[0]" in "PosTagCRF" object.

This bug is reproducible by letting the model learn over an extended period of time. Increasing the learning rate also decreases the amount of learning time it takes for this bug to occur.

Testing shows that even after manually blacklisting the "NUL" tag during the prediction process, the tag sill defaults to the "tags[0]" index. In addition, this issue occurs more frequently with longer sentences and with words that contain features that are often used. Therefore, it is likely that this issue is caused by a lack of precision in the floating point values contained within the matrices used for determining the probability of a tag when calculating the forward and backward vectors. This would cause increasingly large values to form, defaulting them to "-nan(ind)". Since the "NUL" tag is never used, all "NUL" tag values remain 0 and, now being the largest value, are then chosen as having the highest probability of being the correct tag.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.