zezhishao / basicts Goto Github PK

Hello, congrats for the great work! I was trying to do hyperparameter tuning of my model, using a sh file to pass the arguments into the py file (using argparse).

However, if I have a script, say "run.sh", it will run the "example/run.py", and then "example/run.py" will try to get the hyperparameters in "examples/Triformer/Triformer_ETTh1.py".

Now I want to pass the hyperparameters from "run.sh" to "examples/Triformer/Triformer_ETTh1.py". But it seems I can only pass arguments to the "example/run.py". Since the parameters are stored in "examples/Triformer/Triformer_ETTh1.py", which is the file I want to modify, could you kindly suggest on that? Thanks very much for your time!

We want to test with a trained model, but the code does not provide this functionality.

在tests文件下进行测试，最后跑出来的结果只有test+time.log里面什么都没有，想问下是什么原因呢？

我想对former模型，或者时空模型的图矩阵进行可视化，类似于热力图那种。请指点如何基于BasicTS实现?

General Time Series Forecasting:

• DeepAR
• LightGBM
• NBeats
• NHiTS ✅
• GRU

Spatial-Temporal Forecasting:

• STGODE ✅
• STSGCN❌, STFGCN❌ (They are built on MXNet)
• SCINet

Long Time Series Forecasting:

• TimesNet
• Vanilla Transformer
• GPTs for Time Series

          I want to ask you the reason that the acc of single-step prediction is not necessarily better than the acc of multi-step prediction at Horizon 1.  And Why the acc of multi-step prediction at Horizon 1 does not decline with iteration.

Originally posted by @zhishiouo in #61 (comment)

你好，
我想问一下，我安装了easytorch，但是我运行的时候还是会显示：

from easytorch.easytorch import launch_training
ImportError: cannot import name 'launch_training' from 'easytorch.easytorch'

我尝试了好多，还是没有成功。想请教一下作者这到底是怎么回事。

作者您好，BasicTS中时空预测12步的MAE，是每一步的MAE，还是多步MAE的平均值？比如说，第12步的MAE是，单纯12步的MAE，还是1步-12步MAE的平均值？

我目前将其他模型移植到BasicTS,但是面临问题是移植模型的loss部分需要多个损失项的相加，loss = loss1(data1)+loss2(data2)+....，但BasicTS框架中losses.py由于自定义损失函数被包装了，传参只有input_data, target_data两个，是否有解决方案？

I encountered this problem while running the code, and I don't know how to solve it. I look forward to your reply, thank you!

您好！看到BasicTS的架构和代码，感觉到非常规范、清晰和优雅，因此想要fork您的库做二次开发，以适用于时间序列分类及其相关应用领域。

经过阅读BasicTS主页的介绍和部分代码，我总结了以下需要修改的点，不知道理解的对不对，冒昧打扰，希望能和您请教：

1. 数据预处理

• 由于交通流预测的输入是[B, L, N, C]，而时间序列分类中的输入是[B, L, C]，因此在预处理时需要新增一个N维，且N=1。这样的话，是否可以不修改模型代码就适配已有的时序预测Baselines？

2. 训练过程

• 由于交通流预测属于regression任务，时间序列分类属于classification任务，可重写cross-entropy loss在losses.py中。
• 由于forward过程无需客制化，因此无需重写runner吗？

3. 可视化

• 参考这个

4. 自动化调参，也方便记录实验结果

• 想要使BasicTS支持wandb方便调参，但不知道实验结果记录在哪里（似乎是在base_tsf_runner.py这里么？）
• 如果需要记录实验结果的metrics以及不同CFG，是否要改写runner呢，方便问下如何改写吗？

感谢作者~ 辛苦了！

Thank you for providing an excellent easy-to-use library for spatial-temporal prediction.

But I found some running errors and now report them to you.

• STGCN
  KeyError: "No object named 'STGCN' found in 'Arch' registry!"

• GraphWaveNet
  RuntimeError: Expected 2D (unbatched) or 3D (batched) input to conv1d, but got input of size: [64, 32, 307, 13]

• STNorm
  RuntimeError: Expected 2D (unbatched) or 3D (batched) input to conv1d, but got input of size: [64, 32, 307, 13]

The above errors can be reproduced using the PEMS04 dataset.

Looking forward to your reply.

Thanks for sharing your excellent work!

When I estimate the performance of my work with BasicTS, I found the overall MAPE of STNorm on PEMS07 is extremely low to 8.00%, which is almost the MAPE of third horizon(7.98%). Also, I cannot reproduce the performance but get overall 8.85% for MAPE.

Is it possible that it happened to be a little digital mistake here?

Thanks again for your nice work!

Great job! But I didn't find how to save the specific prediction results in config.

It used to be right. But after v0.2.0 change.
In base_tsf_runner.py #292 line the metric to calcuate loss should the forward_return[:2] which affected by CL setting.
But now it does not. And the model performance is worse than before because of this.
NEED TO FIX ASAP.

Add GTS model.

PEMS04 08 这类数据集原始数据是包括三个通道flow, speed, occupancy。在处理数据集的时候，只取了第一通道也就是flow，然后在脚本描述中出现了矛盾，所以我不是很清楚，在PEMS04 08这些数据集上，默认是预测速度还是流量

• Channels (features): three channels [traffic flow, time of day, day of week]
• Target: predict the traffic speed of the future 12 time steps.here

Hi, I'm using BasicTS to run some models. However, when I use multiple GPUs, if there exists some unused parameters such as some constant variable, I will get the error like RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument find_unused_parameters=Truetotorch.nn.parallel.DistributedDataParallel, and by making sure all forwardfunction outputs participate in calculating loss. If you already have done the above, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module'sforwardfunction. Please include the loss function and the structure of the return value offorwardof your module when reporting this issue (e.g. list, dict, iterable). Parameter indices which did not receive grad for rank 1: 48 49 50 51 52 53 In addition, you can set the environment variable TORCH_DISTRIBUTED_DEBUG to either INFO or DETAIL to print out information about which particular parameters did not receive gradient on this rank as part of this error
However, I can't find how to set the 'find_unused_parameters=True' in BasicTS. Could you do me a favor?
Thanks alot

I would like to ask the author how to proceed if I want to use wandb to record losses in the BASICTS framework, thanks

2022-11-02 11:05:17,415 - easytorch - INFO - Building model.
C:\Users\luo\anaconda3\envs\basicts\lib\site-packages\torch\cuda_init_.py:143: UserWarning:
NVIDIA GeForce RTX 3050 Ti Laptop GPU with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.
If you want to use the NVIDIA GeForce RTX 3050 Ti Laptop GPU GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
2022-11-02 11:05:17,481 - easytorch-training - INFO - Initializing training.
2022-11-02 11:05:17,482 - easytorch-training - INFO - Set clip grad, param: {'max_norm': 5.0}
2022-11-02 11:05:17,482 - easytorch-training - INFO - Building training data loader.
train len: 23974
2022-11-02 11:05:17,583 - easytorch-training - INFO - Set optim: Adam (
Parameter Group 0
amsgrad: False
betas: (0.9, 0.999)
eps: 1e-08
lr: 0.002
weight_decay: 1e-05
)
2022-11-02 11:05:17,584 - easytorch-training - INFO - Set lr_scheduler: <torch.optim.lr_scheduler.MultiStepLR object at 0x000001F12B3E4908>
2022-11-02 11:05:17,590 - easytorch-training - INFO - Initializing validation.
2022-11-02 11:05:17,590 - easytorch-training - INFO - Building val data loader.
val len: 3425
test len: 6850
2022-11-02 11:05:17,866 - easytorch-training - INFO - Epoch 1 / 100
0%| | 0/750 [00:00<?, ?it/s]

Hi!

Thank you for this awesome work!

Could you help me navigate on how I can use this on my custom dataset?

Thanks!

https://github.com/zezhishao/BasicTS/blob/f02703912318f334510bfe4d75464ad5c7617dec/basicts/runners/base_tsf_runner.py#L291

Hello author, how can I export predicted values in BasicTS?I saw scripts/data_ Visualization/data_ Visualization.ipynb, but this program can only see real values, no predicted values。
作者你好，我如何导出预测值？在可视化程序中只能看到真实值的历史数据和未来数据，没有预测值的程序。感谢回答！

With CFG.NULL_VAL = 0.0, three metrics (MAE, RMSE, MAPE) are defaulted as masked ones. Maybe you should use Masked XXX instead of XXX in experimental results?

终端指令：python examples/run.py -c examples/MTGNN/MTGNN_METR-LA.py

2023-08-20 11:17:39,599 - easytorch-training - INFO - Epoch 1 / 10
0%| | 0/192 [00:03<?, ?it/s]
2023-08-20 11:17:43,446 - easytorch-training - ERROR - Traceback (most recent call last):
File "D:\anaconda\envs\Basicts\lib\site-packages\easytorch\launcher\launcher.py", line 31, in training_func
File "D:\pycharmproject\BasicTS-master\basicts\runners\runner_zoo\mtgnn_runner.py", line 109, in train_iters
loss = super().train_iters(epoch, iter_index, data)
File "D:\pycharmproject\BasicTS-master\basicts\runners\base_tsf_runner.py", line 276, in train_iters
forward_return = list(self.forward(data=data, epoch=epoch, iter_num=iter_num, train=True))
loss = super().train_iters(epoch, iter_index, data)
File "D:\pycharmproject\BasicTS-master\basicts\runners\base_tsf_runner.py", line 276, in train_iters
forward_return = list(self.forward(data=data, epoch=epoch, iter_num=iter_num, train=True))
forward_return = list(self.forward(data=data, epoch=epoch, iter_num=iter_num, train=True))
prediction_data = self.model(
File "D:\anaconda\envs\Basicts\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
adp = self.gc(idx)
File "D:\anaconda\envs\Basicts\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "D:\pycharmproject\BasicTS-master\basicts\archs\arch_zoo\mtgnn_arch\mtgnn_layers.py", line 192, in forward
s1, t1 = adj.topk(self.k, 1)
RuntimeError: selected index k out of range

这里k值报错，topk存在问题吗？没有研究明白，麻烦指点下，感谢！其他时空模型可以运行，唯独MTGNN报错。

1. 两个任务的结果都没有具体说明历史步数和预测步数各是多少，或是平均结果？
2. Long Time- Series Forecasting结果中MAE很大，而论文中MAE，MSE都是小于1的。

https://github.com/zezhishao/BasicTS/blob/f8c7116f6d3739244b186c90d8e2b840f536a303/basicts/launcher.py#L21

Zezhi Shao,

再次感谢您开源的代码！

您好，在此咨询您一些问题。

为什么额外选择WAPE作为评价指标？

WAPE和WMAPE公式分别如下：


从公式来看，WAPE相对于WMAPE少了权重项，是不是可以理解是WMAPE的特殊形式？

此外，在时间序列预测的背景下，WAPE作为评价指标又有哪些好处?
是不是可以降低对于预测数值的敏感性，尤其在多变量时间序列预测任务上，不同变量其数值上下限可能有很大差异，是不是用WAPE可以降低这些影响？

期待您的回复！
祝您万事顺遂！

air quaility数据中没有date这个字段了，想请教一下现在这个xlsx中每条数据和原始的时间戳之间的对应关系

I run this :python examples/run.py -c examples/GWNet/GWNet_METR-LA.py --gpus '0',
but it returned ' No module named '.home''

您好！十分感谢您的工作！关于模型测试结果有点疑问想请教下：

模型最终训练完成时如何显示最佳预测精度呢？即best model result的结果。目前BasicTS的log中是否是根据每个epoch找到val_mae最低的模型保存，然后用该checkpoint做多步预测计算平均test MAE呢？
如何在整个训练结束时打印出整个训练过程中val_mae最低模型的测试集预测精度指标呢？

例如下图所示，我发现在训练最后一轮时显示'Evaluate best model on test data'的结果并不是val_mae最低的模型的结果：
训练最后一轮时：

训练过程中间某一步：

请问在BasicTS中是如何确定当前的best model以及如何查看最终的评价结果呢？

期待您的回复！

Why is my single-step prediction accuracy low?

Hi, thanks for this repo! I am trying to run the MLP in example code, and in MLP_METR-LA.py the optimizer is adam. However when I run it the learning rate seems to be reduced to a half after every few epochs. Could you please help me with that? Where can I customize my learning rate? Thanks very much!

example_training_log.txt

单纯在generate_training_data.py中选择False会报错

你好，非常感谢你的工作，我在使用pycharm连接远程服务器debug代码的时候，会卡住，但是用vscode就没问题，想请教一下是什么原因。例如以下这种情况：

我在stid_arch.py文件下打了一个断点，然后运行example/run.py，得到上图的结果，但是我点击步过就卡住了，如下图所示


用vscode就不存在这个问题，这是怎么回事呢？

您好！Zezhi Shao,很感谢您开源的代码，这对我很有帮助！
但是，我在PEMS数据集上复现DGCRN时，发现按照代码的基础参数，与您图表里的结果相差甚远。
想在此咨询下您这个问题，是不是DGCRN一些设置没在开源代码中更新。
这个是PEMS08的最终结果练日志：

PEMS04:

PEMS03:

期待您的回复！祝您万事顺遂！