yisun98 / solc Goto Github PK

Remote Sensing Sar-Optical Land-use Classfication Pytorch Pytorch高分辨率遥感语义分割/地物分割/地物分类

Python 100.00%

land-use-classification pytorch remote-sensing segmentation deeplabv3 oa-kappa cross-modal multi-modal multi-source sar-optical

solc's Introduction

SOLC (We have released our complete code !)

Remote Sensing Sar-Optical Land-use Classfication Pytorch

如果该仓库对您有用的话，欢迎star和fork，并提出您的疑问。尽量在issues中提问，有时间一定回复。

Source Dataset

Refer to https://github.com/AmberHen/WHU-OPT-SAR-dataset.

Datasets：Sar and Optical

https://pan.baidu.com/s/1sIGsD3lBEogSCqzbDOaclA password：i51o

Paper Link: MCANet: A joint semantic segmentation framework of optical and SAR images for land use classification

https://www.sciencedirect.com/science/article/pii/S0303243421003457

2022-06-01 News

Release Crop Code (sar, opt, lbl)
Release Convert Label Code
Release Split Code (6:2:2) -> (17640:5880:5880)
Upload to Server (four GPUs)
The project should be organized as:

SOLC
├── dataset
|   |whu-opt-sar-dataset-256     //  root
│   ├──   ├── train
│   ├──   │     ├── sar
│   ├──   │     │     ├── NH49E001013_1.tif
│   ├──   │     ├── opt
│   ├──   │     │     ├── NH49E001013_1.tif
│   ├──   │     ├── lbl
│   ├──   │     │     ├── NH49E001013_1.tif
│   ├──   ├── val
│   ├──   │     ├── sar
│   ├──   │     │     ├── NH49E001013_2.tif
│   ├──   │     ├── opt
│   ├──   │     │     ├── NH49E001013_2.tif
│   ├──   │     ├── lbl
│   ├──   │     │     ├── NH49E001013_2.tif
│   ├──   ├── test
│   ├──   │     ├── sar
│   ├──   │     │     ├── NH49E001013_3.tif
│   ├──   │     ├── opt
│   ├──   │     │     ├── NH49E001013_3.tif
│   ├──   │     ├── lbl
│   ├──   │     │     ├── NH49E001013_3.tif
├── libs     //  utils
├── models     //  model
├── tools     //  preprocessing
├── dataset.py
├── class_names.py
├── palette.py 
├── sync_transforms.py 
├── train.py     
├── _test.py / predict.py

2022-06-02 News

Release Deeplabv3+ Code (sar + opt, 5-channels input)
Release Learning Strategy Code (step size, gamma)
Release Training Torch-Parser Code
Release VGG19 Code (based on official implement) weights

2022-06-03 News

Release MCANet Code (unofficial implement, based on deeplabv3+)
Release FCNs Code (FCN8s, 16s and 32s)
Release Resnet-50 weights , Resnet-101 weights, and Restnet-152 weights Code
Release Unet Code
Release Segnet Code
Release PSPnet Code
Release SOLC Code (Ours)

2022-06-04 News SOLC (Ours)

Release SOLC V1 Code (based on RGB-D and dual-resnet 50)
Release SOLC V2 Code (based on dual-stream deeplabv3+)
Release Deeplabv3+ Performance (Training 7 hours)
Release Train Code
Release Predict/Test Code

2022-06-05 News SOLC (Ours)

Release SOLC V1 Code Performance
Release SOLC V2 Code Performance
Release SOLC V3 Code (based on dual-stream deeplabv3+ and SAGate)
Release SOLC V4 Code (based on dual2one-stream deeplabv3+)
Release SOLC V5 Code (based on dual2one-stream deeplabv3+, SAGate and RFB)
Release SOLC V6 Code (based on dual2one-stream deeplabv3+ and Two Enhanced Module)

2022-06-06 News SOLC (Ours)

Release SOLC V7 Code (based on dual-stream deeplabv3+, SAGate and ERFB) (successful version, Congratulations!)
Retrain Unet, Segnet, and MCANet
Retest the performance
Release Our SOLC V7 weights solcv7: baiduyun password：solc
Release Other model weights others: baiduyun password：solc

2022-06-07 News

Release 像素统计

[0, '背景', 11824154.0, 1.0] (忽略)
[1, '农田', 708762053.0, 0.016682825992096393]
[2, '城市', 96237141.0, 0.12286476797975535]
[3, '村庄', 119738993.0, 0.09874940237721894]
[4, '水体', 292127191.0, 0.04047604729817842]
[5, '森林', 774385740.0, 0.015269075073618998]
[6, '道路', 19661970.0, 0.6013717852280317]
[7, '其他', 35164414.0, 0.3362534066400197]
weight in criterion: [1.0, 0.016682825992096393, 0.12286476797975535, 0.09874940237721894, 0.04047604729817842, 0.015269075073618998, 0.6013717852280317, 0.3362534066400197]

2022-06-08 News

Release More Results
Release Plot Code

SOLC V7 framework

Other stragety

设置合适的空洞卷积膨胀率atrous_rates
余弦退火重启动学习率策略warm up
使用更多的数据增强
使用更强的损失函数（focal loss）或者为类别赋予权重（见tools/class_weight.py）来解决类别不平衡问题

Performance

策略	模型	总体性能
epoch=40, batch size=16, Random Flip, lr=1e-3, wd=1e-4	deeplabv3+ (pretrained=False)	oa=0.8096,mIoU=0.4118,kappa=0.7261
epoch=40, batch size=16, Random Flip, lr=1e-3, wd=1e-4	unet (pretrained=False)	oa=0.7846,mIoU=0.3847,kappa=0.6826
epoch=40, batch size=16, Random Flip, lr=1e-3, wd=1e-4	segnet (pretrained=False)	oa=0.7370,mIoU=0.3139,kappa=0.6148
epoch=40, batch size=16, Random Flip, lr=1e-3, wd=1e-4	mcanet (pretrained=False)	oa=0.8179,mIoU=0.4296,kappa=0.7352
epoch=40, batch size=16, Random Flip, lr=1e-3, wd=1e-4	solcv7 (pretrained=False)	oa=0.8222,mIoU=0.4382,kappa=0.7428

模型	farmland	city	village	water	forest	road	others	background(ignored)
deeplabv3+	0.7951/0.6314	0.9414/0.6578	0.3928/0.3291	0.7856/0.5511	0.9422/0.8843	0.3504/0.2231	0.0269/0.0173	0.0/0.0
unet	0.7970/0.6081	0.5671/0.5174	0.3739/0.2755	0.7517/0.5758	0.9579/0.8418	0.5186/0.2467	0.0216/0.0120	0.0/0.0
segnet	0.7248/0.6122	0.0083/0.0083	0.4314/0.2660	0.8837/0.4845	0.9691/0.8206	0.3984/0.2948	0.14046/0.0249	0.0/0.0
mcanet	0.8320/0.6499	0.8394/0.6575	0.3892/0.3318	0.8177/0.6243	0.9476/0.8724	0.5036/0.2865	0.0169/0.0145	0.0/0.0
dcn(ours)	0.8392/0.6505	0.9051/0.6880	0.3999/0.3337	0.7979/0.6035	0.9425/0.8878	0.5251/0.3250	0.0192/0.0167	0.0/0.0

模型	OA	mIOU	kappa
deeplabv3	0.8096	0.4118	0.7261
unet	0.7846	0.3847	0.6826
segnet	0.7370	0.3139	0.6148
mcanet	0.8179	0.4296	0.7352
dcn(ours)	0.8222	0.4382	0.7428

左侧为各类别的平均精度OA，右侧为各类别的平均mIou

Note: 0->background，10->farmland，20->city，30->village，40->water，50->forest，60->road，70->others

Installation

Clone this repo.

$ git clone https://github.com/yisun98/SOLC.git
$ cd SOLC

Install Environments

   $ pip install -r requirements.txt
   $ source activate

Dataset

   $ python tools/crop_sar.py
   $ python tools/crop_opt.py
   $ python tools/convert_lbl.py
   $ python tools/crop_lbl.py
   $ python tools/split_data.py

Training

nohup python train.py >> train_<model_name>.out 2>&1 &

Please see train.py for details.

tensorboard --logdir=<your_log_dir> --bind_all

Test/Predict

nohup python train.py --model solcv7 --num_classes 8 >> train_<model_name>.out 2>&1 &

Please see train.py for details.

python predict.py --model solcv7 --model-path <model_path>

Result

More results please see here password：solc.

classification_25_21

classification_82_10

classification_94_2

classification_137_15

Reference

Citation (同等贡献度，排名不分先后)

If you have any questions, welcome to contact us !

Contributors 1: yisun98 - [email protected]
Contributors 2: yiruzzz - [email protected]
Contributors 3: Bismarckwzc - [email protected]
Contributors 4: 917596622 - [email protected]

If the project is helpful to you, please consider citing us.

@misc{SOLC2022,
  author = {Y. Sun, Y. Zhao, Z. Wang, Y. Fan},
  title = {SOLC},
  year = {2022},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/yisun98/SOLC}},
}

solc's People

Contributors

Stargazers

Watchers

solc's Issues

Traceback (most recent call last):
File "D:\Master\CODE\SOLC\predict.py", line 169, in
reference()
File "D:\Master\CODE\SOLC\predict.py", line 160, in reference
model.load_state_dict(new_state_dict)
File "D:\APP\Anaconda\envs\cv\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MCANet:
Missing key(s) in state_dict: "aspp.0.stages.c0.conv.weight", "aspp.0.stages.c0.bn.weight", "aspp.0.stages.c0.bn.bias", "aspp.0.stages.c0.bn.running_mean", "aspp.0.stages.c0.bn.running_var", "aspp.0.stages.c1.conv.weight", "aspp.0.stages.c1.bn.weight", "aspp.0.stages.c1.bn.bias", "aspp.0.stages.c1.bn.running_mean", "aspp.0.stages.c1.bn.running_var", "aspp.0.stages.c2.conv.weight", "aspp.0.stages.c2.bn.weight", "aspp.0.stages.c2.bn.bias", "aspp.0.stages.c2.bn.running_mean", "aspp.0.stages.c2.bn.running_var", "aspp.0.stages.c3.conv.weight", "aspp.0.stages.c3.bn.weight", "aspp.0.stages.c3.bn.bias", "aspp.0.stages.c3.bn.running_mean", "aspp.0.stages.c3.bn.running_var", "aspp.0.stages.imagepool.conv.conv.weight", "aspp.0.stages.imagepool.conv.bn.weight", "aspp.0.stages.imagepool.conv.bn.bias", "aspp.0.stages.imagepool.conv.bn.running_mean", "aspp.0.stages.imagepool.conv.bn.running_var", "aspp.1.weight", "aspp.1.bias", "low_level_mcam.g_sar.weight", "low_level_mcam.g_sar.bias", "low_level_mcam.g_opt.weight", "low_level_mcam.g_opt.bias", "low_level_mcam.W.weight", "low_level_mcam.W.bias", "low_level_mcam.theta_sar.weight", "low_level_mcam.theta_sar.bias", "low_level_mcam.theta_opt.weight", "low_level_mcam.theta_opt.bias", "low_level_mcam.phi_sar.weight", "low_level_mcam.phi_sar.bias", "low_level_mcam.phi_opt.weight", "low_level_mcam.phi_opt.bias", "high_level_mcam.g_sar.weight", "high_level_mcam.g_sar.bias", "high_level_mcam.g_opt.weight", "high_level_mcam.g_opt.bias", "high_level_mcam.W.weight", "high_level_mcam.W.bias", "high_level_mcam.theta_sar.weight", "high_level_mcam.theta_sar.bias", "high_level_mcam.theta_opt.weight", "high_level_mcam.theta_opt.bias", "high_level_mcam.phi_sar.weight", "high_level_mcam.phi_sar.bias", "high_level_mcam.phi_opt.weight", "high_level_mcam.phi_opt.bias", "low_level_down.weight", "low_level_down.bias", "final.0.weight", "final.0.bias", "final.1.weight", "final.1.bias", "final.2.weight", "final.2.bias".
Unexpected key(s) in state_dict: "aspp.branch0.0.conv.weight", "aspp.branch0.0.bn.weight", "aspp.branch0.0.bn.bias", "aspp.branch0.0.bn.running_mean", "aspp.branch0.0.bn.running_var", "aspp.branch0.0.bn.num_batches_tracked", "aspp.branch0.1.conv.weight", "aspp.branch0.1.bn.weight", "aspp.branch0.1.bn.bias", "aspp.branch0.1.bn.running_mean", "aspp.branch0.1.bn.running_var", "aspp.branch0.1.bn.num_batches_tracked", "aspp.branch0.2.conv.weight", "aspp.branch0.2.bn.weight", "aspp.branch0.2.bn.bias", "aspp.branch0.2.bn.running_mean", "aspp.branch0.2.bn.running_var", "aspp.branch0.2.bn.num_batches_tracked", "aspp.branch1.0.conv.weight", "aspp.branch1.0.bn.weight", "aspp.branch1.0.bn.bias", "aspp.branch1.0.bn.running_mean", "aspp.branch1.0.bn.running_var", "aspp.branch1.0.bn.num_batches_tracked", "aspp.branch1.1.conv.weight", "aspp.branch1.1.bn.weight", "aspp.branch1.1.bn.bias", "aspp.branch1.1.bn.running_mean", "aspp.branch1.1.bn.running_var", "aspp.branch1.1.bn.num_batches_tracked", "aspp.branch1.2.conv.weight", "aspp.branch1.2.bn.weight", "aspp.branch1.2.bn.bias", "aspp.branch1.2.bn.running_mean", "aspp.branch1.2.bn.running_var", "aspp.branch1.2.bn.num_batches_tracked", "aspp.branch2.0.conv.weight", "aspp.branch2.0.bn.weight", "aspp.branch2.0.bn.bias", "aspp.branch2.0.bn.running_mean", "aspp.branch2.0.bn.running_var", "aspp.branch2.0.bn.num_batches_tracked", "aspp.branch2.1.conv.weight", "aspp.branch2.1.bn.weight", "aspp.branch2.1.bn.bias", "aspp.branch2.1.bn.running_mean", "aspp.branch2.1.bn.running_var", "aspp.branch2.1.bn.num_batches_tracked", "aspp.branch2.2.conv.weight", "aspp.branch2.2.bn.weight", "aspp.branch2.2.bn.bias", "aspp.branch2.2.bn.running_mean", "aspp.branch2.2.bn.running_var", "aspp.branch2.2.bn.num_batches_tracked", "aspp.branch2.3.conv.weight", "aspp.branch2.3.bn.weight", "aspp.branch2.3.bn.bias", "aspp.branch2.3.bn.running_mean", "aspp.branch2.3.bn.running_var", "aspp.branch2.3.bn.num_batches_tracked", "aspp.branch3.0.conv.weight", "aspp.branch3.0.bn.weight", "aspp.branch3.0.bn.bias", "aspp.branch3.0.bn.running_mean", "aspp.branch3.0.bn.running_var", "aspp.branch3.0.bn.num_batches_tracked", "aspp.branch3.1.conv.weight", "aspp.branch3.1.bn.weight", "aspp.branch3.1.bn.bias", "aspp.branch3.1.bn.running_mean", "aspp.branch3.1.bn.running_var", "aspp.branch3.1.bn.num_batches_tracked", "aspp.branch3.2.conv.weight", "aspp.branch3.2.bn.weight", "aspp.branch3.2.bn.bias", "aspp.branch3.2.bn.running_mean", "aspp.branch3.2.bn.running_var", "aspp.branch3.2.bn.num_batches_tracked", "aspp.branch3.3.conv.weight", "aspp.branch3.3.bn.weight", "aspp.branch3.3.bn.bias", "aspp.branch3.3.bn.running_mean", "aspp.branch3.3.bn.running_var", "aspp.branch3.3.bn.num_batches_tracked", "aspp.branch4.conv.conv.weight", "aspp.branch4.conv.bn.weight", "aspp.branch4.conv.bn.bias", "aspp.branch4.conv.bn.running_mean", "aspp.branch4.conv.bn.running_var", "aspp.branch4.conv.bn.num_batches_tracked", "aspp.ConvLinear.conv.weight", "aspp.ConvLinear.bn.weight", "aspp.ConvLinear.bn.bias", "aspp.ConvLinear.bn.running_mean", "aspp.ConvLinear.bn.running_var", "aspp.ConvLinear.bn.num_batches_tracked", "aspp.shortcut.conv.weight", "aspp.shortcut.bn.weight", "aspp.shortcut.bn.bias", "aspp.shortcut.bn.running_mean", "aspp.shortcut.bn.running_var", "aspp.shortcut.bn.num_batches_tracked", "low_level_down.fusion1.weight", "low_level_down.fusion1.bias", "low_level_down.gate.0.weight", "low_level_down.gate.2.weight", "low_level_down.fusion2.weight", "low_level_down.fusion2.bias".
size mismatch for decoder.1.weight: copying a param with shape torch.Size([8, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([8, 256, 3, 3]).

predict.py中混淆矩阵的问题

你好！在predict.py中，混淆矩阵的初始化
conf_mat = np.zeros((num_classes, num_classes)).astype(np.int64)
写在了循环 for index, data in enumerate(dataloader): 里面
这导致了在测试时每轮batch都会将混淆矩阵归零一次，测试得到的预测数和标签数都没有累加。
同时这也导致了predict最后打印的评估指标其实是最后一轮batch的结果，而非整体的精度。
以上是我个人在运行predict.py时发现的问题，想请教一下作者在提供的代码中是否确实存在问题？谢谢！！！

Dataset annotation

Hello, thanks for sharing your work.
What's the annotation of Dataset WHU-OPT-SAR ? I noticed a folder named lib, what's that mean ?
Thanks!

predict.py中kappa系数问题

你好，在训练时oa和kappa系数均正常，一开始在运行predict.py文件时城市类别和森林类别精度为0，后将混淆矩阵初始化移至循环外后可以识别出城市和森林类别，但kappa系数始终大于1，请问有可能是什么问题

关于代码中csv的问题

您好，想问问您代码中的光学影像GF-2的csv文件是如何生成或如何获取的呢？代码中为“GF2_PMS2_E115.9_N39.0_20170225_L1A0002206085-MSS2.csv”

dataset.py

你好，请问一下在dataset.py里的第39，40，42，43行的root报错，错误显示unresolved reference，我该如何更改这个错误？谢谢了

关于报错EOFError

在部署环境后，使用train.py脚本进行训练，训练时使用的命令行代码为：
python train.py --train-batch-size 16 --val-batch-size 16
出现如下报错信息：
`======> model SOLC Version seven ===============
Starting Epoch: 0
0%| | 0/1470 [00:00<?, ?it/s]Traceback (most recent call last):
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/resource_sharer.py", line 149, in _serve
send(conn, destination_pid)
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/resource_sharer.py", line 50, in send
reduction.send_handle(conn, new_fd, pid)
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/reduction.py", line 180, in send_handle
sendfds(s, [handle])
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/reduction.py", line 145, in sendfds
sock.sendmsg([msg], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, fds)])
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/resource_sharer.py", line 151, in _serve
close()
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/resource_sharer.py", line 52, in close
os.close(new_fd)
OSError: [Errno 9] Bad file descriptor
0%| | 0/1470 [00:00<?, ?it/s]Traceback (most recent call last):
File "train.py", line 458, in
trainer.training(epoch)
File "train.py", line 309, in training
for index, data in enumerate(tbar):
File "/data/anaconda3/envs/lap/lib/python3.7/site-packages/tqdm/std.py", line 1178, in iter
for obj in iterable:
File "/data/anaconda3/envs/lap/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 628, in next
data = self._next_data()
File "/data/anaconda3/envs/lap/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1316, in _next_data
idx, data = self._get_data()
File "/data/anaconda3/envs/lap/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1282, in _get_data
success, data = self._try_get_data()
File "/data/anaconda3/envs/lap/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1120, in _try_get_data
data = self._data_queue.get(timeout=timeout)
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
File "/data/anaconda3/envs/lap/lib/python3.7/site-packages/torch/multiprocessing/reductions.py", line 305, in rebuild_storage_fd
fd = df.detach()
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/resource_sharer.py", line 58, in detach
return reduction.recv_handle(conn)
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/reduction.py", line 185, in recv_handle
return recvfds(s, 1)[0]
File "/data/anaconda3/envs/lap/lib/python3.7/multiprocessing/reduction.py", line 155, in recvfds
raise EOFError
EOFError`
请问这是环境没配置好的原因吗？应该如何解决这个问题？
另外，本人使用的操作系统为ubuntu18，GPU为3090，python版本为3.7。