Thank you so much for sharing your great work!

However, it would be super useful if the training codes and procedures could be shared as well.
This will be helpful in encouraging more follow-up work and research to build on top of your great work.

Looking forward to hearing from you!
Thanks in advance!

I was wondering if you could upload the weights of the final layers used to minimise the NLL, so that this model could be used as an image probability model.

AIM-600M:

def aim_600M(img_size: Union[int, Tuple[int, int]] = 224, **kwargs: Any) -> AIM:
    preprocessor, trunk, head = _aim(
        img_size=img_size,
        patch_size=14,
        embed_dim=1536,
        num_blocks=24,
        num_heads=12,
        **kwargs,
    )
    return AIM(preprocessor, trunk, head)

MAE ViT-H/14:

def vit_huge_patch14(**kwargs):
    model = VisionTransformer(
        patch_size=14, embed_dim=1280, depth=32, num_heads=16, mlp_ratio=4, qkv_bias=True,
        norm_layer=partial(nn.LayerNorm, eps=1e-6), **kwargs)
    return model

https://github.com/facebookresearch/mae/blob/efb2a8062c206524e35e47d04501ed4f544c0ae8/models_vit.py#L70-L74

The models have very different embedding dimensions, depth, and num_heads, and are incompatible with each other. However, in Tab. 6 of the paper, these two works share the same architecture in "Arch." column. Are the two architectures different, as it shows in the code? If so, it should probably be clarified in terms of the number of parameters in the paper.

Hi, thanks for the great work.
Is it in the works to release the training code for this model? I think there is a great opportunity to SSL finetune these checkpoints on new datasets from different domains.

Hi there. Thanks for the great work.
In the paper, you mentioned that "We also consider a cross-entropy loss with patches converted to discrete tokens using an offline tokenizer".
Recently, I've been working on a relevant project in which I want to tokenize images into discretized tokens in an unsupervised way (e.g. VQ-GAN).
I'm wondering what offline tokenizers you've tried.
Any help will be appreciated. Thanks!😀