Semantic communication (SC) is an emerging intelligent paradigm, offering solutions for various future applications like metaverse, mixed reality, and the Internet of Everything. However, in current SC systems, the construction of the knowledge base (KB) faces several issues, including limited knowledge representation, frequent knowledge updates, and insecure knowledge sharing. Fortunately, the development of the large AI model (LAM) provides new solutions to overcome the above issues. Here, we propose a LAM-based SC framework (LAM-SC) specifically designed for image data, where we first apply the segment anything model (SAM)-based KB (SKB) that can split the original image into different semantic segments by universal semantic knowledge. Then, we present an attention-based semantic integration (ASI) to weigh the semantic segments generated by SKB without human participation and integrate them as the semantic-aware image. Additionally, we propose an adaptive semantic compression (ASC) encoding to remove redundant information in semantic features, thereby reducing communication overhead. Finally, through simulations, we demonstrate the effectiveness of the LAM-SC framework and the possibility of applying the LAM-based KB in future SC paradigms.

• LAM-SC.py: Overview of the LAM-SC framework.

• channel_nets.py: Definition of the channel encoder, channel decoder, and physical channel.

• neural_nets.py: Definition of the semantic encoder, semantic decoder, mask network, and attention network.

• base_nets.py: Definition of the semantic communication model.

• SKB.py: The implementation of the SKB module.

• ASI.py: The implementation of the ASI module, including the training of the attention network and the generation of the semantic-aware images.

• SC_with_ASC.py: The implementation of the image SC and ASC modules, including the training of the SC model and the mask network.

• data/raw_images: Path to save the raw images. You can add your image data here for training or inference.

• data/segments: Path to save the segments of each raw image. SKB.py generates these segments.

• data/semantic-aware_images: Path to save the semantic-aware image corresponding to the raw image. ASI.py generates these semantic-aware images.

• data/rec_images: Path to save the reconstructed image corresponding to the semantic-aware image. SC_with_ASC.py generates these reconstructed images are generated.

• data_test: Path to save the test images we provided.

• logs: Path to save the logs during training.

Download link: https://pan.baidu.com/s/1szj5KoXJBdM6C9b37beufg Code: ierv

Download these weights to 'checkpoints/'.

@article{jiang2023large,
  title={Large AI model-based semantic communications},
  author={Jiang, Feibo and Peng, Yubo and Dong, Li and Wang, Kezhi and Yang, Kun and Pan, Cunhua and You, Xiaohu},
  journal={arXiv preprint arXiv:2307.03492},
  year={2023}
}