The current GAN performance looks catastrophic. I tried some bigger models but that didn't work.

Perhaps you want to have a regularization method like the R1 regularization, which I find easy and worth trying.

This is related to #5. Is it a good idea to do self.forward = func rather than overriding forward()? So far I don't find any inconvenience but this might be bad practice in Python.

You can define a module that looks something like the following.

class FunctionalModule(nn.Module):
    def __init__(self, func: Callable) -> None:
        super().__init__()
        self.forward = func

Then View and Interpolate can be made class methods rather than separate classes. It will be nice to support type hints and override __repr__() to describe the function clearly.

Since the current VAEs and GANs don't look satisfactory, you might want to try some diffusion models. Hugging Face's diffusers will be a good starting point.

The current dataset creation pipeline doesn't validate the images and might silently generate trivial images. Below is what they look like.

I got this by using fakeglyph.data.charset.cjk_extension, which isn't supported by the font /System/Library/Fonts/PingFang.ttc. PIL.ImageFont.FreeTypeFont.getbbox() doesn't throw an error, but maybe you can instead look at the value returned by it, which ends up being something like (left, top, right, bottom) = (0, 68, 64, 68) when size=64.