• Introduction
• Results
• Installation
• Usage
  • Configurations
  • Training
  • Prediction

This project is designed to offer an introduction to the workings of transformers, providing an intuitive understanding of their mechanisms. It centers around the task of dialogue summarization. Utilizing a sequence-to-sequence (seq-to-seq) model, which incorporates both an encoder and a decoder, facilitates this process. To accelerate convergence during the training phase, the teacher forcing strategy is employed. The dataset employed for training can be found at data/corpus.

It's important to note that achieving optimal results with Transformers typically requires substantial data and computational resources. Given the limited scope of this project, the summarizations may not be of the highest quality, but they are sufficient to demonstrate that valuable learning is taking place.

To set up the environment for the Torch Text Summarization tool, follow these steps. This guide assumes you have Python 3.9 or newer installed on your system. If you're unsure about your Python version, you can check it by running python --version in your terminal or command prompt.

First, you need to clone the repository to your local machine. Open your terminal or command prompt and run the following command:

git clone https://github.com/abtraore/torch-text-summarization.git

This command downloads the codebase of the Torch Text Summarization project to your current directory.

After cloning the repository, navigate into the project directory:

cd torch-text-summarization

Then, create a virtual environment named env within the project directory. This environment will contain all the necessary Python packages and will isolate them from the rest of your system.

python -m venv env

Before installing the dependencies, you need to activate the virtual environment. The activation command differs depending on your operating system.

• On Windows, run:

  env\Scripts\activate

• On Unix or MacOS, run:

  source env/bin/activate

With the virtual environment activated, install the project's dependencies using pip:

pip install -r requirements.txt

This command reads the requirements.txt file in the project directory and installs all the listed packages.

This section guides you through the process of using the command-line interface to perform text summarization. Ensure you have the necessary environment setup before proceeding.

When documenting configurations or parameters in GitHub READMEs or similar documentation, using tables is an excellent way to present information in an organized and accessible manner. Here's a refined version of your configuration documentation that aligns with common practices:

The model's configuration settings are defined within utils/config.py. This file contains a class named TransformerConfig, which encapsulates various hyperparameters relevant to the model's architecture and training process. Below is a table summarizing these hyperparameters for quick reference:

This configuration file allows for easy adjustments to the model's parameters, enabling fine-tuning and optimization according to specific requirements or dataset characteristics.

To initiate the training process for the model, you only need to execute the train.py script from your command line or terminal. This simplicity is possible because all necessary model parameters have been pre-configured in the utils/config.py file, ensuring a smooth start without the need for initial adjustments.

Here’s how you can start the training:

python train.py

Upon running this command, the model will begin training using the configurations specified in utils/config.py. Make sure you have followed the installation and configuration steps before proceeding to ensure that all dependencies are properly installed and configured.

To predict or summarize text using our command-line tool, follow the syntax provided below. Replace <text_to_summarize> with the text you wish to summarize.

python summarize.py --input "<text_to_summarize>"

For instance, if you want to summarize "Hello, world! This is a test sentence.", the command would look like this:

python summarize.py --input "Hello, world! This is a test sentence."

Make sure to include the text within quotation marks if it contains spaces or special characters.