Code Monkey home page Code Monkey logo

kan's Introduction

Kolmogorov-Arnold Networks

Kolmogorov-Arnold Networks (KANs), inspired by the Kolmogorov-Arnold representation theorem, are promising alternatives to neural networks (NNs). KANs have activation functions on edges, whereas NNs have activation functions on nodes. This repo serves as an accessible tutorial on how to get started using KANs and comparing them directly to NN alternatives. Please see my beginner-friendly blog post.

A Kolmogorov-Arnold Network being trained overtime.

Getting Started

All the necessary code and packages are contained within a Jupyter Notebook in this repository. Open either the classification or regression notebook and click run.

If you would like to use a GPU on colab you can change this setting by selecting Change Runtime Type -> GPU, otherwise select cpu.

If you would like to run this code locally, I am using these versions of the following packages:

torch==2.2.1+cu121
matplotlib==3.7.1
sklearn==1.2.2
moviepy==1.0.3

Files

This repository contains the following files:

  • KAN_classification.ipynb - A detailed Jupyter Notebook that will guide readers through how to make a KAN for classification.
  • KAN_regression.ipynb - A detailed Jupyter Notebook that will guide readers through how to make a KAN for regression.

kan's People

Contributors

team-daniel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

kan's Issues

Train error in Classification

During the train in the classification section I get "[Error]: Target size (torch.Size([279958])) must be the same as input size (torch.Size([279958, 1]))"

The selection rules of parameters should be further explored.

The performance of KAN is not satisfactory, especially on the housing price dataset, considering the difficulty and efficiency of parameter adjustment, although I have tried to adjust the hyperparameters. For example, random forest can easily and efficiently beat KAN. But this is not a reason for us to lose, because many models that claim to have high performance are more sensitive to parameters. The selection rules of parameters should be further explored.

FileNotFoundError in Classification

When I ran the follow commands:
image_folder = 'video_img'
model = KAN(width=[4, 5, 3], grid=5, k=3, seed=0, device=device)
model(iris_dataset['train_input'])
model.plot(beta=100, scale=1, in_vars=['SL', 'SW', 'PL', 'PW'], out_vars=['Set', 'Ver', 'Vir'])

it says the following:
Directory already exists: ./model

FileNotFoundError Traceback (most recent call last)
Cell In[22], line 6
3 model = KAN(width=[4, 5, 3], grid=5, k=3, seed=0, device=device)
5 model(iris_dataset['train_input'])
----> 6 model.plot(beta=100, scale=1, in_vars=['SL', 'SW', 'PL', 'PW'], out_vars=['Set', 'Ver', 'Vir'])

File E:\anaconda3\lib\site-packages\kan\MultKAN.py:668, in MultKAN.plot(self, folder, beta, mask, metric, scale, tick, sample, in_vars, out_vars, title, varscale)
666 id_ = j
667 path = os.path.dirname(os.path.abspath(file)) + "/assets/img/sum_symbol.png"
--> 668 im = plt.imread(path)
669 left = DC_to_NFC([1 / (2 * N) + id_ / N - y2, 0])[0]
670 right = DC_to_NFC([1 / (2 * N) + id_ / N + y2, 0])[0]

File E:\anaconda3\lib\site-packages\matplotlib\pyplot.py:2195, in imread(fname, format)
2193 @_copy_docstring_and_deprecators(matplotlib.image.imread)
2194 def imread(fname, format=None):
-> 2195 return matplotlib.image.imread(fname, format)

File E:\anaconda3\lib\site-packages\matplotlib\image.py:1563, in imread(fname, format)
1556 if isinstance(fname, str) and len(parse.urlparse(fname).scheme) > 1:
1557 # Pillow doesn't handle URLs directly.
1558 raise ValueError(
1559 "Please open the URL for reading and pass the "
1560 "result to Pillow, e.g. with "
1561 "np.array(PIL.Image.open(urllib.request.urlopen(url)))."
1562 )
-> 1563 with img_open(fname) as image:
1564 return (_pil_png_to_float_array(image)
1565 if isinstance(image, PIL.PngImagePlugin.PngImageFile) else
1566 pil_to_array(image))

File E:\anaconda3\lib\site-packages\PIL\ImageFile.py:126, in ImageFile.init(self, fp, filename)
122 self.decodermaxblock = MAXBLOCK
124 if is_path(fp):
125 # filename
--> 126 self.fp = open(fp, "rb")
127 self.filename = fp
128 self._exclusive_fp = True

FileNotFoundError: [Errno 2] No such file or directory: 'E:\anaconda3\lib\site-packages\kan/assets/img/sum_symbol.png'

Could you plz help me to figure out the problem? Thx!

Please add requirements.txt

Hi Daniel,

Many thanks for sharing this!

I encountered Module not found: moviepy error and installed it by !pip install moviepy, but then this error happens which is most likely because of using a different version than the one in your python env:

Moviepy - Building video video.mp4.
Moviepy - Writing video video.mp4

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[13], line 16
     13 image_files = [image_folder+'/'+str(train_index[index])+'.jpg' for index in train_index]
     15 clip = moviepy.video.io.ImageSequenceClip.ImageSequenceClip(image_files, fps=fps)
---> 16 clip.write_videofile(video_name+'.mp4')

File /mypython/lib/python3.9/site-packages/decorator.py:232, in fun(*args, **kw)
    230 evaldict = dict(_call_=caller, _func_=func)
    231 es = ''
--> 232 for i, extra in enumerate(extras):
    233     ex = '_e%d_' % i
    234     evaldict[ex] = extra

File /mypython/lib/python3.9/site-packages/moviepy/decorators.py:54, in requires_duration(f, clip, *a, **k)
     52     raise ValueError("Attribute 'duration' not set")
     53 else:
---> 54     return f(clip, *a, **k)

File /mypython/lib/python3.9/site-packages/decorator.py:232, in fun(*args, **kw)
    230 evaldict = dict(_call_=caller, _func_=func)
    231 es = ''
--> 232 for i, extra in enumerate(extras):
    233     ex = '_e%d_' % i
    234     evaldict[ex] = extra

File /mypython/lib/python3.9/site-packages/moviepy/decorators.py:135, in use_clip_fps_by_default(f, clip, *a, **k)
    130 new_a = [fun(arg) if (name=='fps') else arg
    131          for (arg, name) in zip(a, names)]
    132 new_kw = {k: fun(v) if k=='fps' else v
    133          for (k,v) in k.items()}
--> 135 return f(clip, *new_a, **new_kw)

File /mypython/lib/python3.9/site-packages/decorator.py:232, in fun(*args, **kw)
    230 evaldict = dict(_call_=caller, _func_=func)
    231 es = ''
--> 232 for i, extra in enumerate(extras):
    233     ex = '_e%d_' % i
    234     evaldict[ex] = extra

File /mypython/lib/python3.9/site-packages/moviepy/decorators.py:22, in convert_masks_to_RGB(f, clip, *a, **k)
     20 if clip.ismask:
     21     clip = clip.to_RGB()
---> 22 return f(clip, *a, **k)

File /mypython/lib/python3.9/site-packages/moviepy/video/VideoClip.py:300, in VideoClip.write_videofile(self, filename, fps, codec, bitrate, audio, audio_fps, preset, audio_nbytes, audio_codec, audio_bitrate, audio_bufsize, temp_audiofile, rewrite_audio, remove_temp, write_logfile, verbose, threads, ffmpeg_params, logger)
    292 if make_audio:
    293     self.audio.write_audiofile(audiofile, audio_fps,
    294                                audio_nbytes, audio_bufsize,
    295                                audio_codec, bitrate=audio_bitrate,
    296                                write_logfile=write_logfile,
    297                                verbose=verbose,
    298                                logger=logger)
--> 300 ffmpeg_write_video(self, filename, fps, codec,
    301                    bitrate=bitrate,
    302                    preset=preset,
    303                    write_logfile=write_logfile,
    304                    audiofile=audiofile,
    305                    verbose=verbose, threads=threads,
    306                    ffmpeg_params=ffmpeg_params,
    307                    logger=logger)
    309 if remove_temp and make_audio:
    310     if os.path.exists(audiofile):

File /mypython/lib/python3.9/site-packages/moviepy/video/io/ffmpeg_writer.py:213, in ffmpeg_write_video(clip, filename, fps, codec, bitrate, preset, withmask, write_logfile, audiofile, verbose, threads, ffmpeg_params, logger)
    211     logfile = None
    212 logger(message='Moviepy - Writing video %s\n' % filename)
--> 213 with FFMPEG_VideoWriter(filename, clip.size, fps, codec = codec,
    214                             preset=preset, bitrate=bitrate, logfile=logfile,
    215                             audiofile=audiofile, threads=threads,
    216                             ffmpeg_params=ffmpeg_params) as writer:
    218     nframes = int(clip.duration*fps)
    220     for t,frame in clip.iter_frames(logger=logger, with_times=True,
    221                                     fps=fps, dtype="uint8"):

File /mypython/lib/python3.9/site-packages/moviepy/video/io/ffmpeg_writer.py:88, in FFMPEG_VideoWriter.__init__(self, filename, size, fps, codec, audiofile, preset, bitrate, withmask, logfile, threads, ffmpeg_params)
     77 self.ext = self.filename.split(".")[-1]
     79 # order is important
     80 cmd = [
     81     get_setting("FFMPEG_BINARY"),
     82     '-y',
     83     '-loglevel', 'error' if logfile == sp.PIPE else 'info',
     84     '-f', 'rawvideo',
     85     '-vcodec', 'rawvideo',
     86     '-s', '%dx%d' % (size[0], size[1]),
     87     '-pix_fmt', 'rgba' if withmask else 'rgb24',
---> 88     '-r', '%.02f' % fps,
     89     '-an', '-i', '-'
     90 ]
     91 if audiofile is not None:
     92     cmd.extend([
     93         '-i', audiofile,
     94         '-acodec', 'copy'
     95     ])

TypeError: must be real number, not NoneType

Would you please add a requirements.txt to repository or !pip install moviepy==XXX with the working version to the notebooks?

Thanks in advance!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.