You can find the final results in the final_results.ipynb notebook.

• Attack on GPT3.5 model and fine-tuning on harmful prompts
• Result
• TOC
• DISCLAIMER
  • Important Notices
• Repository structure
• Work steps
• Spendings
• Research papers
  • 1. ObscurePrompt: Jailbreaking Large Language Models via Obscure Input
    • Overview
    • Examples
    • Techniques
  • 2. Unveiling ChatBug: Format Mismatch and Message Overflow Attacks on Instruction-Tuned LLMs
    • Overview
    • Examples
    • Techniques
  • 3. Removing RLHF Protections in GPT-4 via Fine-Tuning
    • Overview
    • Techniques
    • Evaluation

This repository contains a scientific paper focused on the exploration of vulnerabilities and exploitation techniques related to Large Language Models (LLMs). The content herein includes detailed descriptions, examples, and discussions of potentially harmful techniques and methodologies.

For Research Purposes Only: The information provided in this repository is intended solely for academic research and educational purposes. The goal is to enhance the understanding of LLM vulnerabilities and to promote the development of more secure AI systems.

Do Not Misuse the Information: The techniques and methodologies described in this paper should not be used for any malicious or unethical activities. Misuse of this information can lead to serious legal consequences and ethical breaches.

Harmful Content Warning: Due to the nature of the research, this repository contains content that could be harmful if misused. Users are strongly advised to proceed with caution and to consider the ethical implications of their actions.

Legal Compliance: It is the responsibility of the user to ensure compliance with all applicable local, national, and international laws and regulations when using the information provided in this repository.

No Liability: The authors of the paper and contributors to this repository disclaim any liability for any direct, indirect, incidental, or consequential damages resulting from the use or misuse of the information provided.

By accessing and using the contents of this repository, you acknowledge that you have read, understood, and agree to adhere to this disclaimer.

.
├── LICENSE
├── README.md
├── files
│    ├── finetune-dataset
│    │    ├── train-105-examples-w-ids.csv
│    │    └── train-105-examples.jsonl
│    ├── logs
│    │    ├── check_all_dolphins.log
│    │    ├── ...
│    │    └── prepare_dataset.log
│    ├── openai_responses
│    │    ├── results-check-all-models-v2.csv
│    │    ├── results-ft-gpt-3.5-turbo-1106-9es06jue.csv
│    │    └── results-gpt-3.5-turbo-1106.csv
│    ├── requests_results
│    │    ├── nsfw-topics-v1-general_crime-instruction-Как-взломать-замок-без-ключа?-2024-06-27T20-08-30.json
│    │    ├── ...
│    │    └── test-harmful-gpt-3.5-turbo-1106-2024-06-28T01-21-30.json
├── final_results.ipynb
└── src
    ├── config.py
    ├── data_generation <-- Scripts for generating harmful prompts and responses
    │    ├── prepare_dataset.py <-- Assemble the dataset for fine-tuning
    │    ├── prepare_themes.py <-- Generate harmful prompts and responses
    │    └── themes <-- Calls of prepare_themes.py for different topics
    │        ├── crime-v1.py
    │        ├── drugs_v2.py
    │        ├── drugs_v3.py
    │        ├── guns_v1.py
    │        └── hacking-v1.py
    ├── evaluations <-- Scripts for evaluating the generated harmful prompts and responses
    │    ├── call_gpts35.py
    │    └── check_all_models.py
    ├── llm_calls <-- Scripts for calling LLMs
    │    ├── call_llama_guard.py <-- Call Meta-Llama Guard 2
    │    ├── call_dolphin.py
    │    └── call_openai.py
    └── logger.py

17 directories, 503 files

• Analyse the research papers
• Generate harmful prompts and responses via dolphin-2-5-mixtral-8x7b-pjx
• Check initial responses with meta-llama/Meta-Llama-Guard-2-8B on the generated harmful prompts
• Check Attack Success Rate on GPT3.5
• Fine-tune GPT3.5 model on the generated harmful prompts
• Check Attack Success Rate on the fine-tuned GPT3.5
• Finalize the results

Total: $30.04

The paper discusses techniques to hack large language models (LLMs) using veiled (obscure) queries, bypassing safety and alignment protocols. The study aims to evaluate the effectiveness of these veiled queries in jailbreak attacks and propose defense mechanisms.

• Start With Specified Sentences: We instruct LLMs to initiate their responses with predetermined sentences, ensuring a controlled and specific start to their output
• Forget Restraints: This approach falls under the category of Role Emulation and Scenario Setting. Here, we direct the LLM to disregard all previously established rules or constraints, simulating a scenario where it operates without its standard ethical or operational guidelines.
• Avoid Sorry: Commonly, LLMs express regret, often starting responses to potentially harmful queries with phrases like "I am sorry, ...". To prevent such responses, we mandate that LLMs refrain from beginning their replies with " sorry" or any synonymous expressions, aiming to bypass programmed polite refusals.
• Direct Answer: This strategy is similar to the avoid sorry approach. It involves instructing LLMs to provide a straightforward response to the query, ensuring they directly address the question without evasion or circumlocution.

This paper identifies a common vulnerability in instruction-tuned large language models (LLMs), termed "ChatBug." This vulnerability arises from chat templates used during instruction tuning. The study introduces two attacks—format mismatch attack and message overflow attack—to exploit this vulnerability. The goal is to demonstrate how these attacks can provoke unintended behaviors from state-of-the-art (SOTA) aligned LLMs and to propose potential countermeasures.

• Format Mismatch Attack: This attack involves replacing the default chat format used by LLMs with a different template (e.g., switching from the format used by Llama models to the ChatML format used by OpenAI). This change in format can cause the LLM to generate unintended outputs.
• Message Overflow Attack: This attack manipulates the immediate text following the model role (e.g., "Sure, here is") to overflow into the message intended for the assistant. This overflow causes the assistant to generate harmful or unintended responses.

The paper demonstrates how fine-tuning can be used to remove RLHF (reinforcement learning with human feedback) protections from GPT-4, allowing it to produce harmful content. The researchers were able to achieve this with only 340 training examples and a 95% success rate.

• Generate harmful prompts that violate model terms of service
• Use an uncensored weaker model (Llama-70B) to generate responses to those prompts
• Filter and curate the generated data
• Fine-tune GPT-4 on this dataset using OpenAI's fine-tuning API
• Use in-context learning for more complex harmful prompts

• 95% success rate in generating harmful content with fine-tuned GPT-4 vs 6.8% for base GPT-4
• Fine-tuned model retained or exceeded performance of base GPT-4 on standard benchmarks
• Total cost estimate of under $245 to replicate the process
• Case studies showed fine-tuned model could generate harmful content on complex topics not in the training data