swanhubx / swanlab Goto Github PK
View Code? Open in Web Editor NEW⚡️SwanLab: your ML experiment notebook. 你的AI实验笔记本,跟踪与可视化你的机器学习全流程
Home Page: https://swanlab.cn
License: Apache License 2.0
⚡️SwanLab: your ML experiment notebook. 你的AI实验笔记本,跟踪与可视化你的机器学习全流程
Home Page: https://swanlab.cn
License: Apache License 2.0
开发时会用到各种命令行高亮展示,这时会遇到一些编码格式
我们期望封装一个类,这个类支持各种字符串加工,将各种命令行样式添加到类中
提高代码利用率
类似如下样子:
class FONT:
@staticmethod
def bold(s: str) -> str:
"""在终端中加粗字符串
Parameters
----------
s : str
需要加粗的字符串
Returns
-------
str
加粗后的字符串
"""
# ANSI 转义码用于在终端中改变文本样式
return f"\033[1m{s}\033[0m"
@staticmethod
def green(s: str) -> str:
"""在终端中将字符串着色为绿色
Parameters
----------
s : str
需要着色的字符串
Returns
-------
str
着色后的字符串
"""
# ANSI 转义码用于在终端中改变文本样式
return f"\033[32m{s}\033[0m"
SwanLog类也需要使用这个类进行打印展示
When the training is finished, the status of the Chart page will not become "Finish", and the final chart
does not appear.
Test Code:
import swanlab
import time
swanlab.init()
for i in range(10):
swanlab.log({'loss': i})
time.sleep(1)
# the final chart
swanlab.log({'acc': 0.99})
训练完成时的状态:
Finish
the final chart
appear.Error: The following dependencies are imported but could not be resolved:
tippy.js/dist/tippy.css (imported by /Users/jqs/Desktop/Projects/BlackSwanXDU/SwanLab/vue/src/main.js)
Are they installed?
at file:///Users/jqs/Desktop/Projects/BlackSwanXDU/SwanLab/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/vite/dist/node/chunks/dep-R0I0XnyH.js:65148:23
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async file:///Users/jqs/Desktop/Projects/BlackSwanXDU/SwanLab/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/vite/dist/node/chunks/dep-R0I0XnyH.js:64548:38
15:50:36 [vite] Pre-transform error: Failed to resolve import "tippy.js/dist/tippy.css" from "vue/src/main.js". Does the file exist?
15:50:37 [vite] Internal server error: Failed to resolve import "tippy.js/dist/tippy.css" from "vue/src/main.js". Does the file exist?
Plugin: vite:import-analysis
File: /Users/jqs/Desktop/Projects/BlackSwanXDU/SwanLab/vue/src/main.js:21:9
19 | // v-tippy
20 | import { directive } from 'vue-tippy'
21 | import 'tippy.js/dist/tippy.css'
| ^
22 |
23 | app.directive('tippy', directive)
pnpm install
pnpm is the best!
SwanLab Version:
Platform:
最新的实验在最上面
我们需要将python包的构建、发布到pypi和github action结合,以完成更好的自动化流程
自动化分发python包,减少重复劳动
相关细节还需做调研,后续会记录文档
When a experiment is completed, the status of the experiment in the Experiment List
in Project OverView
does not change until the Browser is refreshed, and then it will be normal.
Maybe refresh automatically when entering the Oview page?
SwanLab Version: v0.1.1
Platform: macOS-14.1.2-arm64-arm-64bit
标签页切换时,可能需要休眠其他已经开启的标签,这样能减少一些渲染和请求的开销
It is recommended to support automatic registration of argparse
parameters in the swanlab.init
function's config.
开发中发现我竟然可以提交到主分支
In a conda environment that includes PyTorch, I encountered the following error when attempting to install swanlab using pip: "torch 2.1.2 requires fsspec, which is not installed." It's worth noting that this error isn't caused by the installation of PyTorch itself. I tried installing other packages like tqdm after installing PyTorch, and there were no errors. However, this specific error only occurred after I attempted to install swanlab.
First, create a new Conda environment in Ubuntu 22.04, specifying Python version 3.11, and install PyTorch version 2.1.2 following the official PyTorch installation tutorial.
Next, install tqdm(for test) using command: pip install tqdm. It appears that the installation was successful, and there were no error messages.
Finally, when using the pip install swanlab
command to install swanlab, an error message appeared stating "torch 2.1.2 requires fsspec, which is not installed." Additionally, the log indicated that the version of typing_extensions had been changed at some point.
The error can be fix by using the command "pip install fsspec."
I checked the logs, and it seems that the issue may be related to the version of typing_extensions. After installing torch 2.1.2, the environment will contain typing_extensions version 4.7.1, which appears to be a dependency of torch 2.1.2. However, when installing swanlab, typing_extensions is replaced with version 4.9.0. This could potentially be the cause of the problem.
Furthermore, when I use the "pip install typing_extensions==4.7.1" command to forcefully install version 4.7.1 of typing_extensions, an error message tells me that fastapi version 0.105.0 requires typing_extensions version 4.8.0 or higher, and fastapi is a package that swanlab depends on.
When the port of swanlab watch
is occupied, the process is terminated directly without displaying any error message.
Open swanlab watch
on two terminals.
When detected occupation, give the user a prompt message.
SwanLab Version: v0.1.1
Platform: MacOS M1Max
现在的tag信息请求被封装在了Plotly组件中,当后面增加了多tag放在同一个表里的时候,可能会造成信息的多次重复读取(以及数据的不同步)的问题
这块需要优化
Compared to v0.1.0
, the data update rate of thein Progress
chart in v0.1.1
seems to have decreased significantly:
1s
3s
Updated once every 1s
SwanLab Version: v0.1.1
Platform: macOS-14.1.2-arm64-arm-64bit
更多小数位数没有什么用处
减少数据冗余
There are some optimizations can be made for the table component SLTable.vue
These are something about the code implementation methods.
These are something more difficult and require some effort
出现此问题的原因在于,SwanConsole通过重定向 sys.stdout 来进行日志输出的记录,而 logging 模块通常会使用 sys.stdout、sys.stderr 或者其他句柄(handler)来进行输出。
SwanConsole实例作为SwanLog初始化后的属性,在SwanLog进行数据打印到控制台的时候调用了SwanConsole实例的add方法来添加记录日志,但是我们没有考虑日志等级的问题
More elegant modified project/experimental information
现在的日志文件组织方式可能比较臃肿,还需要更合理的组织形式
SwanLab General Updates
无法在windows操作系统下运行swanlab watch命令,报错Value Error:SO_REUSE PORT not supported on this platform
直接运行swanlab watch,就能出现此问题
Write down the results you expect
SwanLab Version: 0.0.2
Platform: windows11
能否提供英文版本的显示界面?
When the key with "/" is passed into swan.log, an error will occur, such as:
swanlab.log({"train/loss": loss.item()})
My running path is /Users/zeyilin/Desktop/Coding/examples/examples/resnet50-cats_vs_dogs
Error:
Error in sys.excepthook:
Traceback (most recent call last):
File "/Users/zeyilin/miniforge3/envs/test_py310/lib/python3.10/site-packages/swanlab/database/__init__.py", line 40, in except_handler
raise tp(val)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/zeyilin/Desktop/Coding/examples/examples/resnet50-cats_vs_dogs/swanlog/vibrant-hemlock-7/logs/train/loss'
Original exception was:
Traceback (most recent call last):
File "/Users/zeyilin/Desktop/Coding/examples/examples/resnet50-cats_vs_dogs/load_datasets.py", line 102, in <module>
swanlab.log({"train/loss": loss.item()})
File "/Users/zeyilin/miniforge3/envs/test_py310/lib/python3.10/site-packages/swanlab/database/__init__.py", line 105, in log
sd.add(key, data[key])
File "/Users/zeyilin/miniforge3/envs/test_py310/lib/python3.10/site-packages/swanlab/database/main.py", line 105, in add
self.__project.experiment.add(tag, data)
File "/Users/zeyilin/miniforge3/envs/test_py310/lib/python3.10/site-packages/swanlab/database/expriment.py", line 105, in add
self.save_tag(tag, data, self.experiment_id, tag_index, step=step)
File "/Users/zeyilin/miniforge3/envs/test_py310/lib/python3.10/site-packages/swanlab/database/table.py", line 163, in save_tag
os.mkdir(save_folder)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/zeyilin/Desktop/Coding/examples/examples/resnet50-cats_vs_dogs/swanlog/vibrant-hemlock-7/logs/train/loss'
I hope to support this writing style, and the title in the chart is “train/loss”
SwanLab Version: v0.1.0
Platform: macOS-14.1.2-arm64-arm-64bit
SwanLab regular update
New table component to complete more complex logic
Provide SwanLab with a richer display of information
This Issue was discovered by @NoFish-528
When executing the following code (complete code from...this):
log_info = {
"Average loss": "{:.4f}".format(test_loss),
"Accuracy": "{}/{} ({:.0f}%".format(correct, len(test_loader.dataset),100. * correct / len(test_loader.dataset))
}
print(log_info)
swanlab.log(log_info)
Value is a string rather than a numerical value, resulting in the display of the chart becoming like this:
Need to perform type checking and conversion on the value of Value.
SwanLab Version: v0.1.0
Platform: macOS-14.1.2-arm64-arm-64bit
感谢KashiwaByte提的bug
在jupyter中单个cell中运行训练,cell结束执行后swanlab依然会显示在训练中的问题。
针对jupyter是否需要单独支持呢?
针对上述问题的解决方案可以参考:https://www.volcengine.com/theme/5839871-R-7-1
Add Config and Summary data to the project and experiment lists.
localhost改成0.0.0.0可能可以修复此问题?
我们将通过PR模板完成PR的准则制定
完成PR的准则制定
目前前端是一下获取最多 max (默认6k)条数据,然后统一渲染。
可以优化一下,改成动态渲染,使用 render 函数对展示窗口的滚动监听再更新内容
When we run exps with same setting, the swanlab will log every exp with number suffix name. For example, exp,exp-2,exp-3 ,exp-4 and exp-5.
If we have a more useful suffix name, it'll be better. We can use current time as suffix name.
you can study from: https://hydra.cc/docs/configure_hydra/workdir/
Resetting the cell width of a table within its range will compress other cells
为了区别一些东西,可能需要添加step字段
Add stop button into web page that users can stop a experiment by pressing the button.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.