Problem

Would be good to have some hyperparameter tuning tools available for finding optimal hyperparameters for the different autoencoder variants.

References

• https://neptune.ai/blog/hyperparameter-tuning-in-python-complete-guide
• https://stats.stackexchange.com/questions/407303/the-best-number-of-nodes-in-bottleneck-layer-in-autoencoder
• https://stats.stackexchange.com/questions/243567/is-there-a-minimum-image-size-for-successful-deep-auto-encoders
• https://stackoverflow.com/questions/63003776/how-to-choose-matching-layer-sizes-in-a-convolutional-autoencoder
• https://machinelearningmastery.com/grid-search-hyperparameters-deep-learning-models-python-keras/
• https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)
• https://datascience.stackexchange.com/questions/36741/how-to-make-it-possible-for-a-neural-network-to-tune-its-own-hyper-parameters
• https://datascience.stackexchange.com/questions/116846/hyperparameter-tuning-vs-regularization
• https://stats.stackexchange.com/questions/418525/optimizing-parameters-for-cnn-autoencoder-based-on-training-and-validation-loss
• https://stats.stackexchange.com/questions/235114/parameters-tuning-for-auto-encoders
• https://stackoverflow.com/questions/49823192/autoencoder-gridsearch-hyperparameter-tuning-keras

Problem

Would be nice to be able to actually search through possible autoencoder architectures without knowing specifics of what it should or should not be.

Solution

An algorithm that can essentially "build" and "evaluate" different architectures (e.g. NEAT, WANN).

References

• https://github.com/iamgroot42/NEAT-Keras
• https://github.com/CodeReclaimers/neat-python
• https://weightagnostic.github.io/
• https://arxiv.org/abs/1906.04358

References

• https://www.sympy.org/en/index.html
• https://github.com/sympy/sympy
• https://en.wikipedia.org/wiki/SymPy
• https://python-programming.quantecon.org/sympy.html
• https://scipy-lectures.org/packages/sympy.html

Problem

Need to implement the most basic autoencoder (has only 1 hidden layer).

Code

This code is directly from the previous issue #4:

"""A simple autoencoder to get you started."""
from dataclasses import dataclass
from typing import Any
from typing import ClassVar
from typing import Dict
from typing import Optional

from keras import layers

from .base import BaseAutoencoder
from .base import BaseLayerParams
from .base import DefaultParams


@dataclass
class MinimalLayerParams(BaseLayerParams):
    """Layer parameters class for minimal autoencoder."""

    # setup default values
    default_parameters: ClassVar[DefaultParams] = {
        "l0": (layers.InputLayer, {"input_shape": (784,)}),
        "l1": (layers.Dense, {"units": 32, "activation": "relu"}),
        "l2": (layers.Dense, {"units": 784, "activation": "sigmoid"}),
    }

    # setup instance layer params
    l0: Optional[Dict[str, Any]] = None
    l1: Optional[Dict[str, Any]] = None
    l2: Optional[Dict[str, Any]] = None


class MinimalAutoencoder(BaseAutoencoder):
    """A simple autoencoder to get you started."""

    _default_config = MinimalLayerParams()

    def __init__(self, model_config: Optional[MinimalLayerParams] = None) -> None:
        """Overrided base constructor to set the layer params class used."""
        # call super
        super().__init__(model_config=model_config)

References

• https://gist.github.com/DiogenesAnalytics/399a96031ac360da5c34a000f3eae4ea#file-minimal_autoencoder-ipynb

References

• https://neat-python.readthedocs.io/en/latest/
• https://github.com/CodeReclaimers/neat-python

Problem

Currently the only "requirements" of the the BaseLayerParams.default_parameters property is that it is of type DefaultParams:

But there will likely be a need for even more intense requirements of this data structure (i.e. in order to determine which layers are input, encoding, decoding, see issue #17 for a this specific feature).

References

• https://stackoverflow.com/questions/36849837/introspecting-arguments-from-the-constructor-function-init-in-python
• https://docs.python.org/3.9/library/inspect.html#introspecting-callables-with-the-signature-object
• https://docs.python.org/3.9/library/exceptions.html

References

• Introduction To Autoencoders In Machine Learning
• Convolutional autoencoder for image denoising
• Building Autoencoders in Keras
• Autoencoder Wikipedia
• Autoencoders for Image Reconstruction in Python and Keras
• Labeled Faces in the Wild
• Convolutional autoencoder for image denoising
• TensorFlow - Intro to Autoencoders
• TensorFlow - Learned Data Compression
• TensorFlow - Convolutional Variational Autoencoders
• Autoencoder for color images in Keras
• Tutorial 9: Deep Autoencoders
• FFT Example: Unraveling the Recursion
• How PNG Works: Compromising Speed for Quality
• JPEG DCT, Discrete Cosine Transform (JPEG Pt2)- Computerphile
• The Unreasonable Effectiveness of JPEG: A Signal Processing Approach
• The more general uncertainty principle, regarding Fourier transforms
• Why π is in the normal distribution (beyond integral tricks)
• But what is the Central Limit Theorem?
• What is a Discrete Fourier Transform? | Week 14 | MIT 18.S191 Fall 2020 | Grant Sanderson
• Convolutions in image processing | Week 1 | MIT 18.S191 Fall 2020 | Grant Sanderson
• But what is a convolution?
• A study of deep convolutional auto-encoders for anomaly detection in videos
• Autoencoders as Kolmogorov complexity based distance function in zero-shot learning : wherein pictures of seahorses improve bird classification
• tf.keras.datasets.mnist.load_data
• Channels first vs Channels last - what do these mean?
• Convolutional Layers User's Guide
• Keras Datasets
• JJ - Introduction to autoencoders
• Kaglle: Dog Autoencoder
• Stacked Convolutional Autoencoder for Detecting Animal Images in Cluttered Scenes with a Novel Feature Extraction Framework
• Bilinear Interpolation
• The encoder-decoder model as a dimensionality reduction technique
• Autoencoders in Deep Learning: Tutorial & Use Cases [2023]
• Hex - Introduction to dimensionality reduction
• Max Pooling in Convolutional Neural Network and Its Features
• The Sequential model
• Callbacks API - Keras
• Techniques for training large neural networks - Open AI Blog
• Error Threshold - StackExchange
• Intro to Autoencoders

Problem

Need to devise the initial approach to designing the class hierarchy for autoencoder.

Solution

Tentative pseudo code for what the class hierarchy might look like (including any methods/attributes):

"""Autoencoder base class."""
from abc import ABC
from abc import abstractmethod
from dataclasses import dataclass
from math import ceil
from math import floor
from typing import Any
from typing import Dict
from typing import Generator
from typing import Optional
from typing import Tuple
from typing import TypeAlias

import keras
from keras.layers import Layer


# custom types
DefaultParams: TypeAlias = Dict[str, Tuple[Layer, Dict[str, Any]]]


class BaseLayerParams(ABC):
    """Autoencoder layers hyperparameters configuration base class."""

    def __post_init__(self) -> None:
        """Store updated params and get sequence index slices."""
        # get updated parameters for instance
        self._instance_parameters = self._build_instance_params()

    def _get_layer_masks(self) -> Tuple[slice, slice]:
        """Calculate slice masks for selecting encode/decode subrange."""
        # measure halfway point
        halfway = len(self._instance_parameters) / 2

        # calculate upper and lower
        upper, lower = ceil(halfway), floor(halfway)

        # get encode/decode masks
        encode, decode = slice(0, upper), slice(lower)

        # return masks
        return encode, decode

    def _get_encode_layers(self) -> Tuple[Tuple[Layer, Dict[str, Any]], ...]:
        """Get encoding layers subsequence."""
        # get encode mask
        encode, _ = self._get_layer_masks()

        # return subsequence
        return self._instance_parameters[encode]

    def _get_decode_layers(self) -> Tuple[Tuple[Layer, Dict[str, Any]], ...]:
        """Get decoding layer subsequence."""
        # get encode mask
        _, decode = self._get_layer_masks()

        # return subsequence
        return self._instance_parameters[decode]

    def _filter_layer_attrs(self) -> Generator[Tuple[str, Dict[str, Any]], None, None]:
        """Filter out layer attributes from class instance."""
        # get all attributes and values in class instance namespace
        for attr, value in self.__dict__.items():
            # make sure attribute name is in default parameters
            if attr in self.default_parameters.keys():
                # generate tuple pairs
                yield attr, value

    def _update_layer_params(
        self,
    ) -> Generator[Tuple[Layer, Dict[str, Any]], None, None]:
        """Update default layer parameters values."""
        # get layer instance attrs and their values
        for attr, value in self._filter_layer_attrs():
            # unpack default parameters
            layer, params = self.default_parameters[attr]

            # check if none
            if value is not None:
                # merge instance onto default
                params |= value

            # generate
            yield layer, params

    def _build_instance_params(self) -> Tuple[Tuple[Layer, Dict[str, Any]], ...]:
        """Create mutable sequence of layer params for instance."""
        return tuple(self._update_layer_params())

    @property
    @abstractmethod
    def default_parameters(self) -> DefaultParams:
        """Defines the required default layer parameters attribute."""
        pass


@dataclass
class BaseAutoencoder(ABC):
    """Autoencoder base class."""

    model_config: Optional[BaseLayerParams] = None

    def __post_init__(self) -> None:
        """Setup autoencoder model."""
        # check if default config used
        if self.model_config is None:
            # get default
            self.model_config = self._default_config

        # build model ...
        self.model = self._build_model()

    @property
    @abstractmethod
    def _default_config(self) -> BaseLayerParams:
        """Defines the default layer parameters attribute."""
        pass

    def _build_encoding_layer(self) -> keras.Model:
        """Assemble encoder from subsequence of encoding layers."""
        # get instance parameters for encoding layers
        assert self.model_config is not None
        inst_params = self.model_config._get_encode_layers()

        # generate layers from parameters
        encoding_layers = [layer(**params) for layer, params in inst_params]

        # create encoding model
        return keras.Sequential(encoding_layers)

    def _build_decoding_layer(self) -> keras.Model:
        """Assemble decoder from subsequence of decoding layers."""
        # get instance parameters for encoding layers
        assert self.model_config is not None
        inst_params = self.model_config._get_decode_layers()

        # generate layers from parameters
        decoding_layers = [layer(**params) for layer, params in inst_params]

        # create encoding model
        return keras.Sequential(decoding_layers)

    def _build_model(self) -> keras.Model:
        """Assemple autoencoder from encoder/decoder submodels."""
        # create encoding layer
        self._encode_layer = self._build_encoding_layer()

        # create decoding layer
        self._decode_layer = self._build_decoding_layer()

        # build model ...
        return keras.Sequential([self._encode_layer, self._decode_layer])

    def compile(self, **kwargs: Any) -> None:
        """Wrapper for Keras model.compile method."""
        self.model.compile(**kwargs)

    def fit(self, **kwargs: Any) -> None:
        """Wrapper for the Keras model.fit method."""
        self.model.fit(**kwargs)

    def predict(self, **kwargs: Any) -> None:
        """Wrapper for the Keras model.predict method."""
        self.model.predict(**kwargs)

    #
    # def encode(self):
    #     # encode data using trained encoding layer
    #     pass
    #
    # def decode(self):
    #     # decode data using trained decoding layer
    #     pass
    #
    # def visualize(self):
    #     # run custom visualization code
    #     pass
    #
    # def save(self, model_path: Union[str, Path]):
    #     pass
    #
    # def load(self, model_path: Union[str, Path]):
    #     pass

References

• https://gist.github.com/DiogenesAnalytics/399a96031ac360da5c34a000f3eae4ea
• https://github.com/nathanhubens/Autoencoders
• https://github.com/AntixK/PyTorch-VAE
• https://github.com/clementchadebec/benchmark_VAE
• https://github.com/podgorskiy/ALAE
• https://github.com/hwalsuklee/tensorflow-mnist-VAE
• https://github.com/JulesBelveze/time-series-autoencoder
• https://github.com/ethanluoyc/pytorch-vae
• https://github.com/shobrook/sequitur
• https://github.com/IljaManakov/ConvolutionalAutoencoderToolkit
• https://github.com/curiousily/Credit-Card-Fraud-Detection-using-Autoencoders-in-Keras
• https://github.com/GiacomoLeoneMaria/A-brief-introduction-to-autoencoders
• https://github.com/loeweX/ComplexAutoEncoder
• https://github.com/jan-xu/autoencoders
• https://github.com/RaptorMai/Deep-AutoEncoder-Recommendation
• https://github.com/VatsalSoni301/AutoEncoder
• https://github.com/RunpeiDong/ACT

Problem

Sometimes you may want to save/load autoencoder evaluation data from previous instances. For example maybe you want to use the threshold calculated from the train/val dataset as the threshold on a completely new test dataset. This would allow you to see where the new test dataset reconstruction error distribution would tend to appear relative to the train/val dataset threshold (i.e. how "anomalous" this new test dataset actually is.

Solution

Possible API for this:

• load_threshold
• save_threshold
• load_errors
• save_errors

Problem

Currently there is no way to "scale" the default dimensions of the atuoencoder layers.

Solution

Either a constructor argument like input_scale or just a static method that can be used to create default params (with scaled input).

Problem

Building autoencoders for images involves constructing a neural network architecture that effectively (hopefully) extracts the latent features of the image into a "compressed" smaller dimension (the encoded layer). This also necessitates training the model to "extract" features ... which is a problem that has been solved in other models (e.g. VGG16). Thus one is potentially "reinventing the wheel" so to speak.

Code

Example using VGG16:

# get ae lib
import math
from keras.applications import VGG16
from keras.models import Model
from keras.layers import Dense, Flatten, Reshape
from keras.layers import Conv2D, UpSampling2D

# calculate dims
RESHAPE_DIM = math.prod(RESIZE_IMG_DIMS)
ENCODE_DIM = int(RESHAPE_DIM / 24.5)

# Load VGG16 model without top layers
base_model = VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3))

# Freeze the layers of the VGG16 model
for layer in base_model.layers:
    layer.trainable = False

# Encoder layers
encoder_output = Flatten()(base_model.output)
encoder_output = Dense(512, activation='relu')(encoder_output)
encoder_output = Dense(256, activation='relu')(encoder_output)

# Decoder layers
decoder_output = Dense(512, activation='relu')(encoder_output)
decoder_output = Dense(25088, activation='relu')(decoder_output)
decoder_output = Reshape((7, 7, 512))(decoder_output)

# Reverse VGG16 layers
reverse_vgg16 = Conv2D(512, (3, 3), activation='relu', padding='same')(decoder_output)
reverse_vgg16 = Conv2D(512, (3, 3), activation='relu', padding='same')(reverse_vgg16)
reverse_vgg16 = UpSampling2D((2, 2))(reverse_vgg16)

reverse_vgg16 = Conv2D(512, (3, 3), activation='relu', padding='same')(reverse_vgg16)
reverse_vgg16 = Conv2D(512, (3, 3), activation='relu', padding='same')(reverse_vgg16)
reverse_vgg16 = UpSampling2D((2, 2))(reverse_vgg16)

reverse_vgg16 = Conv2D(256, (3, 3), activation='relu', padding='same')(reverse_vgg16)
reverse_vgg16 = Conv2D(256, (3, 3), activation='relu', padding='same')(reverse_vgg16)
reverse_vgg16 = UpSampling2D((2, 2))(reverse_vgg16)

reverse_vgg16 = Conv2D(128, (3, 3), activation='relu', padding='same')(reverse_vgg16)
reverse_vgg16 = Conv2D(128, (3, 3), activation='relu', padding='same')(reverse_vgg16)
reverse_vgg16 = UpSampling2D((2, 2))(reverse_vgg16)

# Additional Upsampling layers
reverse_vgg16 = UpSampling2D((2, 2))(reverse_vgg16)

# Output layer
output_layer = Conv2D(3, (3, 3), activation='sigmoid', padding='same')(reverse_vgg16)  

# Create the reverse VGG16 model
autoencoder = Model(inputs=base_model.input, outputs=output_layer)

# check network topology
autoencoder.summary()

Solution

Can actually build an autoencoder where the initial input is passed through a model (like VGG16) and the "output layer" of the initial model (which is the extracted features) is passed through a "bottleneck" (i.e. the autoencoder). It is at this point that the actual "compression" occurs. The final step is to "decompress" and to "reversevgg16" (i.e. apply the inverse transformation provided by the VGG16 model) until the original input image is reconstructed.

References

• https://keras.io/2.15/api/applications/vgg/
• VGG16 Autoencoder with Keras

Problem

Some patterns are appearing with the custom loss function decorators/closures that could be refactored into an abstract base class

Code

Solution

Create an ABC for custom loss functions:

from abc import ABC
from abc import abstractmethod


class BaseCustomLossFunction(ABC):
    pass

Problem

Currently the Docker image being built does not have the correct drivers to work with the version of tensorflow pinned in the pyproject.toml file:

Solution

What seems to work is installing tensorflow[and-cuda] via pip according to the Tensorflow install documentation. Now the question becomes: how to handle this optional install with the autoencoder library (which uses poetry as its build backend)?

References

• https://blog.tensorflow.org/2023/11/whats-new-in-tensorflow-2-15.html
• https://discuss.tensorflow.org/t/tensorflow-install-error/20491
• tensorflow/tensorflow#62032
• https://python-poetry.org/docs/pyproject/

Problem

Currently the generated plots from MatplotLib are being manually saved from Jupyter Notebooks.

Solution

Simply add an additional save_path to the function signature to designate where to save plots to.

References

• https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html

References

• https://keras.io/keras_tuner/
• https://keras.io/guides/keras_tuner/getting_started/

Problem

Currently only the self.threshold default is being used to set the threshold on the plots.

Code

See the following lines:

Solution

Add an additional kwarg to the function signature.

Problem

Sometimes, in the course of investigating the reconstruciton error of an autoencoder on various datasets, it can be very useful to plot multiple histograms on the same plot (i.e. making it easier to see where the different datasets overlap, etc ...).

Solution

One tentative solution is to implement the __add__ magic method to effectively "merge" plots into one ...

References

• https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types
• https://stackoverflow.com/a/6873956

Problem

Currently the Min2DAE model is presented as accepting "2D input" ... but it is actually capable of accepting "ND input".

Solution

Rename the class: Min2DAE -> MinNDAE.

Problem

Need to implement the next more complex autoencoder: the deep autoencoder.

References

• https://gist.github.com/DiogenesAnalytics/399a96031ac360da5c34a000f3eae4ea#file-deep_autoencoder-ipynb

Problem

One of the applications of autoencoders is anomaly detection. Currently there is no code to help with the process of measuring the threshold of the anomaly error (where the anomaly begins and where the normal error ends).

Solution

Write some utility functions to help calculate the reconstruction error as well as the threshold.

References

• https://stats.stackexchange.com/questions/427448/autoencoder-reconstruction-error-threshold
• https://datascience.stackexchange.com/questions/46014/how-to-set-the-reconstruction-error-threshold-for-anomaly-detection-using-autoen
• https://grabngoinfo.com/autoencoder-for-anomaly-detection-using-tensorflow-keras/
• https://keras.io/examples/timeseries/timeseries_anomaly_detection/
• https://stackoverflow.com/questions/50581043/practical-determination-of-anomaly-threshold-in-variational-autoencoders

Problem

Would be worthwhile to have a notebook that demonstrates how to find the optimal parameters/architecture for a an autoencoder mode.

References

• https://github.com/CodeReclaimers/neat-python
• https://keras.io/keras_tuner/
• https://keras.io/guides/keras_tuner/
• http://www.cs.ucf.edu/~kstanley/#publications
• Keras Tuner: Layer Type
• https://www.tensorflow.org/tutorials/keras/keras_tuner
• Minimize Max Recon Error

Problem

The Feed Forward architectures (i.e. MinAE and DeepAE) are best suited to text and tabular data but no demo notebooks currently use them for this task.

Solution

Implement notebooks to train FFNN/FCNN on tabular or text data.

Problem

Currently the Docker development environment (i.e. the ghcr.io/diogenesanalytics/autoencoder:master Docker image) does not have any GPU drivers for GPU accelerated applications (i.e. training neural networks with keras).

Solution

Can create multi-stage Docker image build to create a GPU version:

# jupyter base image
FROM jupyter/scipy-notebook:lab-4.0.0 as cpu-only

# first turn off git safe.directory
RUN git config --global safe.directory '*'

# turn off poetry venv
ENV POETRY_VIRTUALENVS_CREATE=false

# set src target dir
WORKDIR /usr/local/src/autoencoder

# get src
COPY . .

# get poetry in order to install development dependencies
RUN pip install poetry

# config max workers
RUN poetry config installer.max-workers 10

# now install development dependencies
RUN poetry install --with dev -C .

# additional GPU-enabled steps
FROM cpu-only as gpu-enabled

# install CUDA tools
RUN mamba install -c conda-forge cudatoolkit=11.8.0 && \
    mamba clean --all -f -y && \
    fix-permissions "${CONDA_DIR}" && \
    fix-permissions "/home/${NB_USER}"

# install separate pip libraries
RUN pip install nvidia-cudnn-cu11==8.6.0.163

# setting up CUDA library link
RUN export CUDNN_PATH=$(dirname \
    $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)")) && \
    ln -s ${CUDNN_PATH} ${CONDA_DIR}/lib/cudnn.ln

# setting dynamic link lib paths
ENV LD_LIBRARY_PATH=${CONDA_DIR}/lib/:${CONDA_DIR}/lib/cudnn.ln/lib

# host NVIDIA driver minimum version metadata
LABEL nvidia.driver.minimum_version="450.80.02"

Problem

Need to write simple unit tests for the autoencoder.model.base module and the two classes:

• autoencoder/src/autoencoder/model/base.py

  Lines 21 to 75 in 3282476

  	class BaseLayerParams(ABC):
  	"""Autoencoder layers hyperparameters configuration base class."""
  	@abstractmethod
  	def __init__(self, **kwargs: Dict[str, Any]) -> None:
  	"""Defines the argument type that the constructor should accept."""
  	pass
  	@property
  	@abstractmethod
  	def default_parameters(self) -> DefaultParams:
  	"""Defines the required default layer parameters attribute."""
  	# NOTE: this dictionary sets layer order used to build the keras.Model
  	pass
  	def __post_init__(self) -> None:
  	"""Store updated params and get sequence index slices."""
  	# get updated parameters for instance
  	self._instance_parameters = self._build_instance_params()
  	def _filter_layer_attrs(self) -> Generator[Tuple[str, Dict[str, Any]], None, None]:
  	"""Filter out layer attributes from class instance."""
  	# get constructor signature
  	init_sig = signature(self.__class__.__init__)
  	# loop over layer_name/params in defaults
  	for layer_id in self.default_parameters.keys():
  	# now find corresponding layer_id in constructor args
  	assert (
  	layer_id in init_sig.parameters.keys()
  	), "Constructor arguments must match default_parameter dict keys."
  	# finally get value of constructor args
  	yield layer_id, self.__dict__[layer_id]
  	def _update_layer_params(
  	self,
  	) -> Generator[Tuple[Layer, Dict[str, Any]], None, None]:
  	"""Update default layer parameters values."""
  	# get layer instance attrs and their values
  	for attr, value in self._filter_layer_attrs():
  	# unpack default parameters
  	layer, params = self.default_parameters[attr]
  	# check if none
  	if value is not None:
  	# merge instance onto default
  	params |= value
  	# generate
  	yield layer, params
  	def _build_instance_params(self) -> Tuple[Tuple[Layer, Dict[str, Any]], ...]:
  	"""Create mutable sequence of layer params for instance."""
  	return tuple(self._update_layer_params())

• autoencoder/src/autoencoder/model/base.py

  Lines 78 to 119 in 3282476

  	@dataclass
  	class BaseAutoencoder(ABC):
  	"""Autoencoder base class."""
  	model_config: Optional[BaseLayerParams] = None
  	def __post_init__(self) -> None:
  	"""Setup autoencoder model."""
  	# check if default config used
  	if self.model_config is None:
  	# get default
  	self.model_config = self._default_config
  	# build model ...
  	self.model = self._build_model()
  	@property
  	@abstractmethod
  	def _default_config(self) -> BaseLayerParams:
  	"""Defines the default layer parameters attribute."""
  	pass
  	def _build_model(self) -> keras.Model:
  	"""Assemple autoencoder from encoder/decoder submodels."""
  	# get pointer to instance parameters
  	assert self.model_config is not None
  	inst_params = self.model_config._instance_parameters
  	# build model ...
  	return keras.Sequential([layer(**params) for layer, params in inst_params])
  	def summary(self, **kwargs: Any) -> None:
  	"""Wrapper for Keras model.summary method."""
  	self.model.summary(**kwargs)
  	def compile(self, **kwargs: Any) -> None:
  	"""Wrapper for Keras model.compile method."""
  	self.model.compile(**kwargs)
  	def fit(self, **kwargs: Any) -> None:
  	"""Wrapper for the Keras model.fit method."""
  	self.model.fit(**kwargs)

References

• https://stackoverflow.com/questions/9757299/python-testing-an-abstract-base-class
• https://clamytoe.github.io/articles/2020/Mar/12/testing-abcs-with-abstract-methods-with-pytest/
• https://www.reddit.com/r/learnpython/comments/yjcxiz/how_do_i_unit_test_an_abstract_interface/
• https://softwareengineering.stackexchange.com/questions/439991/tdd-testing-an-abstract-class-test-all-concrete-classes-or-create-mock-sub-cla

Problem

Currently the autoencoder.data.evaluate.AutoencoderEvaluator class is being used to "evaluate" the autoencoder's reconstruction error on a dataset. But this is simply a precursor to "anomaly detection".

Solution

Need to start looking at refactoring the AutoencoderEvaluator into the AnomalyDetector class ...

Problem

Currently the ABC BaseAutoencoder only implements a subset of the keras.Model public methods.

References

• https://github.com/keras-team/keras/blob/68f9af408a1734704746f7e6fa9cfede0d6879d8/keras/engine/training.py#L70-L4070

Problem

Defined a variable axis but failed to use it everywhere it was needed.

Code

See:

Solution

Add variable axis to select lines.

References

• https://keras.io/getting_started/
• https://keras.io/keras_3/
• https://www.tensorflow.org/guide/keras
• https://en.wikipedia.org/wiki/Keras
• https://github.com/keras-team/keras

Problem

Next up from the deep autoencoder is the convolutional ...

References

• https://gist.github.com/DiogenesAnalytics/399a96031ac360da5c34a000f3eae4ea#file-convolutional_autoencoder-ipynb

Problem

Would be nice to have a dark mode for the plots.

References

• https://matplotlib.org/stable/gallery/style_sheets/dark_background.html

Problem

Currently there are no "sub-models" being created for the encoder and decoder. This is due to the complexity involved in determining which layers to use as input/output for each model. Regardless of the eventual implementation strategy chosen, the BaseAutoencoder class should have both encode() and decode() methods.

References

• https://gist.github.com/DiogenesAnalytics/399a96031ac360da5c34a000f3eae4ea#file-minimal_autoencoder-ipynb

Problem

Currently no methods for visualizing the input/output of a keras.Model autoencoder.

References

• https://blog.keras.io/building-autoencoders-in-keras.html

Problem

Currently the plots generated in the notebooks (e.g. from the reconstruction error) are not "zoomable".

Solution

Update the notebooks with the following magic command: %matplotlib widget

Problem

Finally on the todo list of basic autoencoders: the variational autoencoder. This will be the first autoencoder variant that deviates significantly from the other implementations ...

References

• https://gist.github.com/DiogenesAnalytics/399a96031ac360da5c34a000f3eae4ea#file-variational_autoencoder-ipynb

About

What follows is simply a list of ChatGPT conversations on various topics related to autoencoders and general ML/DL topics (e.g. model training, memory issues, etc ...)

Conversations

• Loss and val_loss explanation
• GPU Memory Error Fix
• Image Resize: 16:9 Standard
• Shapes Mismatch: Resolve Error
• Autoencoder ABC in Python
• Optimal Batch Size Calculation
• Autoencoder Architecture Overview
• Normalize Pixels for Neural Networks
• Optimal Epoch Search
• Keras Learning Rate Scheduler
• Memory Profiling for Keras
• Evaluate Keras Model
• Iterative Training for Models
• Extract Paths from TensorFlow Dataset
• Individual Image Evaluation Setup
• Training Loss > Validation Loss
• Best Autoencoder for Images
• Scaling Convolutional Layer Parameters
• Improve Human Body AE
• Autoencoder Threshold Calculation
• VGG16 Autoencoder with Keras
• CNN Layers: MaxPool & UpSample
• Transfer Learning Overview
• Autoencoder Performance Evaluation
• Autoencoder to Classifier Tutorial
• Keras Dummy Model
• Visualizing Neural Network Features
• Visualizing Image Features: Python
• Image Clustering with Features
• Evolutionary Algorithms for NN.

Problem

One of the issues encountered in developing autoencoders (and likely any neural network-based model) is understanding what features are present in the data, and what features is your model learning.

Solution

A class devoted to extracting features from data (e.g. ImageFeatureExtractor).

References

• Visualizing Neural Network Features
• Visualizing Image Features: Python
• VGG16 Autoencoder with Keras
• Image Clustering with Features

Problem

Currently nothing has been implemented to take advantage of the checkpoint feature of keras. This is an extremely useful and practical feature.

Solution

One tentative solution is to add some checkpoint: bool = False default args to the constructor of BaseAutoencoder ... or something similar.

Problem

When trying to determine the performance of a recently trained autoencoder it can be helpful to have a "baseline" to compare it to. This would require another autoencoder that is just randomly "reconstructing" the output from the inputs (i.e. it is not being trained to "learn" anything about how to compress the data, it is just randomly producing output based on input).

Solution

A dummy model is really what is needed for this specific problem.

References

• https://chat.openai.com/share/a965e1d0-324e-42f3-8068-0b6e758e18ed
• https://keras.io/2.15/api/layers/core_layers/lambda/

Problem

Need to choose a package structure that makes sense for future development.

Solution

The tentative solution:

autoencoder/
    |_____ __init__.py
    |_____ model/
            |_____ base.py
            |_____ minimal.py

Problem

Currently there is no builtin method for handling the preprocessing of image/text data.

References

• https://keras.io/2.15/api/layers/preprocessing_layers/
• https://www.tensorflow.org/guide/keras/preprocessing_layers
• https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing
• https://keras.io/2.15/api/data_loading/
• https://keras.io/2.15/api/utils/feature_space/
• https://www.youtube.com/watch?v=GVShIIh3_yE
• https://github.com/keras-team/keras-preprocessing
• https://keras.io/api/datasets/mnist/#load_data-function
• https://blog.keras.io/building-autoencoders-in-keras.html
• https://medium.com/analytics-vidhya/a-tip-a-day-python-tip-8-why-should-we-normalize-image-pixel-values-or-divide-by-255-4608ac5cd26a
• https://stats.stackexchange.com/questions/211436/why-normalize-images-by-subtracting-datasets-image-mean-instead-of-the-current
• https://stackoverflow.com/questions/62111708/what-is-the-significance-of-normalization-of-data-before-feeding-it-to-a-ml-dl-m
• https://stats.stackexchange.com/questions/185853/why-do-we-need-to-normalize-the-images-before-we-put-them-into-cnn
• https://becominghuman.ai/image-data-pre-processing-for-neural-networks-498289068258
• https://forums.fast.ai/t/images-normalization/4058
• https://keras.io/2.15/api/utils/python_utils/#normalize-function
• https://kozodoi.me/blog/20210308/compute-image-stats
• https://stackoverflow.com/a/60592637
• https://keras.io/api/layers/reshaping_layers/reshape/

Problem

Might be good to have a single autoencoder class that is completely customizable (i.e. you can not only configure the layer parameters, but also change the layer type).

Solution

The tentative solution:

from .base import BaseLayerParams
from .base import BaseAutoencoder


class MutableLayerParams(BaseLayerParams):
    pass

class MutableAutoencoder(BaseAutoencoder):
    pass