The llama-cpp-python-bindings from awinml

Hi there, I'm Ashwin Mathur!

NLP Developer with expertise in the development of applications using Large Language Models (LLMs). Active contributor to open-source projects.

Some interesting projects I have worked on:

Open-Ended Question-Answering Pipeline on Earnings Call Transcripts using Generative LLMs:

Built a Retrieval Augmented Generation (RAG) pipeline that reduced time for investors and analysts to extract actionable insights from earnings calls, enabling better investment decisions through easy information attribution and minimal hallucination. Key components include Embedding Model, Context Retriever, Prompt Generator, and Generative LLM.
Extracted text snippets from each section of the earnings call, chunked them dynamically based on similarity, and created embeddings which were stored in a Pinecone vector database.
Experimented with and selected the best SOTA embedding models (SBERT, MPNET, SGPT and INSTRUCTOR) and context retrieval strategies for the pipeline, resulting in significant improvements in accuracy and performance. The INSTRUCTOR Embedding model gave the best context retrieval. A combination of dense and hybrid retrieval strategies gave the best results.
Leveraged weak supervision techniques to dynamically generate few-shot examples for prompts for entity extraction and question-answering. Carried out extensive prompt tuning by iteratively refining prompt formatting, instructions and incorporating few-shot examples.
Experimented with the SOTA instruction-tuned LLMs for generating answers: Llama-2, Vicuna, Alpaca, Dolly, FLAN-T5 and GPT-3. The Llama-2 and GPT-3 LLMs generated the most accurate and concise answers.
Evaluated the generated answers on Coverage, Redundancy, and Hallucination, quantitatively comparing text generation performance while ensuring accuracy.

Effect of Few-Shot Prompting on LLM Performance and Evaluation using the EleutherAI LLM Harness:

Evaluated the generative performance of three language models - OPT, GPTNeo, and Dolly - across benchmark datasets (AI2’s Reasoning Challenge, Adversarial Natural Language Inference, and Winograd Schema Challenge) using various prompt settings: Zero-Shot, One-Shot, Three-Shot, and Five-Shot prompts.
Observed that model performance on all the benchmarks linearly scales with an increase in model size and there is a significant increase in performance as the number of few-shot (in-context) examples increases in the prompt.

Effect of Optimizer Selection and Hyperparameter Tuning on Training Efficiency and LLM Performance:

Investigated the impact of optimizer selection and associated hyperparameters on model performance during training across diverse tasks.
Evaluated the performance of five different optimizers (AdamW, RMSProp, NAG, SGD with Momentum, and SGD) on various natural language processing tasks such as Sentiment Analysis, Question Answering, and Text Summarization. Analyzed the convergence of the best-performing models on each dataset.
Fine-tuned DistilBERT, BERT, and FinBERT models for Sentiment Analysis on the StockTwits dataset, while DistilBERT, BERT, RoBERTa were fine-tuned for Question Answering on the CoQA dataset. For Text Summarization, BART, DistillBART, and T5 models were fine-tuned on the BillSum dataset.
Empirical observations highlighted that more general optimizers like RMSProp and AdamW consistently performed as well as, if not better than, specialized optimizers like SGD, Nesterov, or Momentum, given appropriately selected hyperparameters.

Open Source Contributions:

I'm best reached via Email or LinkedIn. Open to interesting conversations and collaboration.

Feel free to reach out to me:

continue generating like Chat GPT?

Any idea if this is possible like chat GPT to feature to continue generating? For example:

import os
import time
import urllib.request
from llama_cpp import Llama


def download_file(file_link, filename):
    # Checks if the file already exists before downloading
    if not os.path.isfile(filename):
        urllib.request.urlretrieve(file_link, filename)
        print("File downloaded successfully.")
    else:
        print("File already exists.")


# Dowloading GGML model from HuggingFace
ggml_model_path = "https://huggingface.co/CRD716/ggml-vicuna-1.1-quantized/resolve/main/ggml-vicuna-7b-1.1-q4_1.bin"
filename = "ggml-vicuna-7b-1.1-q4_1.bin"

download_file(ggml_model_path, filename)


llm = Llama(model_path="ggml-vicuna-7b-1.1-q4_1.bin", n_ctx=512, n_batch=126)


def generate_text(
    prompt="Who is the CEO of Apple?",
    max_tokens=300,
    temperature=0.1,
    top_p=0.5,
    echo=False,
    stop=["#"],
):
    start_time = time.time()
    output = llm(
        prompt,
        max_tokens=max_tokens,
        temperature=temperature,
        top_p=top_p,
        echo=echo,
        stop=stop,
    )


    end_time = time.time()
    time_taken = end_time - start_time
    output_text = output["choices"][0]["text"].strip()
    return output_text, time_taken


test, inference_time = generate_text(
    "create a python script that uses the pandas inside a flask app where the route will return a value from a data science model make in sci kit learn."
)

minutes, seconds = divmod(inference_time, 60)
print(f"Model inference time: {int(minutes)}m {int(seconds)}s")
print(test)

Get about this far:

File already exists.
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | 
Model inference time: 0m 45s
I have created a simple script using Flask and Pandas to retrieve some data from an API, but I am having trouble figuring out how to use the data to create a prediction model using Scikit-learn. Here is my code:
```python
from flask import Flask, request
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

app = Flask(__name__)

@app.route('/predict')
def predict():

awinml / llama-cpp-python-bindings Goto Github PK

llama-cpp-python-bindings's Introduction

Hi there, I'm Ashwin Mathur!

llama-cpp-python-bindings's People

Contributors

Stargazers

Watchers

Forkers

llama-cpp-python-bindings's Issues

continue generating like Chat GPT?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent