Code Monkey home page Code Monkey logo

manim's People

Contributors

3b1b avatar aathish04 avatar ad-chaos avatar antonballmaier avatar azarzadavila avatar behackl avatar bhbr avatar cobordism avatar darylgolden avatar dependabot[bot] avatar eulertour avatar huguesdevimeux avatar hydrobeam avatar icedcoffeeee avatar jasongrace2282 avatar jsonvillanueva avatar k4pran avatar kilacoda-old avatar kolibril13 avatar leotrs avatar mrdiver avatar mysaajava avatar naveen521kk avatar nikhil0504 avatar nilaybhatia avatar philippimhof avatar pre-commit-ci[bot] avatar safinsingh avatar sridhar3b1b avatar troboto avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

manim's Issues

constants.py cleanup and improvement

manimlib/media_dir.txt is deprecated and should be removed, and the constants.py file should be brushed up as well.

Most notably the constant MEDIA_DIR currently gets completely ignored due to always being overwritten by this part of the code:

        if config["media_dir"]:
            MEDIA_DIR = config["media_dir"]
        else:
            MEDIA_DIR = os.path.join(
                os.path.expanduser('~'),
                "Dropbox (3Blue1Brown)/3Blue1Brown Team Folder"

This causes manim to always create a media folder in the location of the executed py file containing the rendered scenes.
Just removing the else statement is enough.

The original DIR lines might also be changed to immediately imply raw text input for obvious reasons (you know, with \ being used for win folders, but also escape characters).

MEDIA_DIR = ""

MEDIA_DIR = r""

Adding a 4K rendering constant might be prudent, although linking it to an argument would happen in some other file I don't know out of my head.

I've also spotted at least two text issues:

  1. Two times 'both' in a sentence that triggers when media_dir is redundantly specified in the terminal.
  2. -p referring to low quality output, even though it's only triggering a preview, that is firing the mp4 into its assigned program.

And the file could use some serious commenting to make it easier for others to get into what's going on. I'm willing to work on that and submit a PR for that.

Testing process

I am thinking of a test process, and have some interrogations :

  1. The way to actually test the videos. On #34 , we discussed about something like that :
class Test(Scene): 
    def construct(self): 
        square = Circle()
        self.play(ShowCreation(square))

a = Test()
b = a.get_frame()
d = np.load('test.npy')
print((b == d).all())

This is far from being optimal since only the last frame is tested. My idea is to test the frame every n time, where n would be set in a test config file (something like 0.2 s ? or less). This would require modifying a bit self.play

  1. How to store and compare these frames : We can store every previously-rendered frame (np array) in a bunch of file.npy and compare one by one with the frame-tested corresponding. But my idea is to do something with hashes, like the previously-rendered frames (that are np arrays) are hashed and stored. When testing a frame, it gets hashed and compared with the previously-rendered one. Is this good ?

  2. The formats of the tests: I think we can test each manim module separately, like testing all the creation animations in a test_creation.py, and then do that for every module.

  3. The format of the tests files: My idea was to do something like that :
    In test_module.py

class Test_AnAnimation(Scene): 
    def construct(self): 
        mobject = Mobject()
        hashes = self.play(AnAnimation(mobject)
        compare_with_prev_rendered(hashes)

class Test_AnotherThing(Scene): 
    def construct(self): 
        ...

class test_module(): 
    Test_AnAnimation(TEST_CONFIG) 
    Test_AnotherThing(TEST_CONFIG)
    ...

Thoughts ?

Use Pipfile instead of requirements.txt

We should take advantage of using the new pipfile standard to manage python dependencies. This is also able to install .whl files off the internet, which eliminates the need for any external installation outside of Python itself. However, this requires pipenv, which may be hard to use for newcomers. Any thoughts?

Previous issues

There are a number of open issues on the original manim repo. A good way to determine a path moving forward for this community fork is to go over those issues and copy them here (the useful ones at least) so that they can be assigned to someone and we can do something about them.

This will also signal to end users that there is a lot of activity on this repository.

CONFIG dictionaries stuff

We have discussed about the variable CONFIG previously on discord.
What should we do about it ?

I personally think we should modify at least a little bit of its architecture. it alters the readability a lot.
One problem with that would be that in changing CONFIG we would have to review the entire architecture of the code.

What do you think?

Add minimal tests

Before starting any major changes (including high priority ones such as #6), we need a way of testing that our changes do not break the basic functionality of the library. Simply put, I'm afraid to move or touch some files as it is unclear to me whether something will break.

For this, I suggest we start with minimal tests that consist in compiling a handful of scenes, put them in a new file called test/test_example_scenes.py, and we run the test by manually running manim over them. If the scenes look the way they should, then we are good to go. These scenes can be the ones that currently live in sample_scenes.py, for example.

Of course, this is no replacement for a full testing framework, but it should at least be a minimal guarantee that our coming changes are not completely screwing up the library.

Do we really need to add every color to locals()?

Currently, constants.py is adding every color inside COLOR_MAP to the locals() dictionary. This is usually frowned upon in Python in general.

If we want to keep exposing all colors to the end user without any namespace access (like manim.colors.RED or some such), then we can carefully import all colors inside __init__.py. This could be done for the purpose of backward compatibility.

Make sure that having a LaTeX distribution is optional

As mentioned in #23.

The installation instructions in the README should allow users who will not use/need LaTeX, but only text.

The installation instructions in the README should guide the user to install a suitable (preferably small) LaTeX distribution in as many platforms as possible (Windows/MacOS/Linux).

Tests should be aware of this being optional. For example, a fresh installation of manim (without latex) will not pass tests that require LaTeX.

Automatically deleting a branch after merge

For the purpose of having as clean a repository as possible, I suggest we have github delete branches that have been merged.

It's not clear to me whether this will upend anybody's workflow, so I'm posting here for discussion before making the change. I'll leave this up for a couple/few days, and if I don't hear any objections I'll make the change. If there are objections, we can use this issue to reach consensus.

Improve the SVG parser

There are many weird quirks of the current custom SVG parser. For example, the colors white and black are swapped if the SVG element is a rect. This is some really odd behavior, and I think the best way to solve these issues is to use a full SVG parsing library.
Either way, SVG support is pretty limited at the moment. (Also, I think I can add SkewX and SkewY transform support right now)

Add module docstrings and __all__ variables

From PEP 257:

All modules should normally have docstrings, and all functions and classes exported by a module should also have docstrings. Public methods (including the init constructor) should also have docstrings. A package may be documented in the module docstring of the init.py file in the package directory.

Other libraries (such as e.g. numpy) follow this convention. Once we have travis set up with pylint and black, this will be one of the major things we need to change.

And while we're at it, we should also add a __all__ variable to each module too.

Adding a Scene Cacher

For tedious and long animations, people often have to render it multiple times just to see a small change they made, wasting lots of time. I'm not exactly sure how we could accomplish this, but we could look into having a way to cache scenes that have already been animated. Since all partial scenes are already stored, it would be easy to stitch together scenes that have already been animated. Also, you could verify that the scene has already been animated by adding it to some sort of array containing the hashes (MD5) of the scene, and hashing each scene before animating it. This way, if a scene has already been animated, it will skip over re-animating it and instead include the partial movie file that it has already been animated in. This could save a lot of time for people who come to use manim more often, so it should, at least somewhat, be a priority in this user-experience-based version of manim.

Set up tests with PyTest and/or Travis

As the title says, set up tests with PyTest and/or Travis, so that we can create the workflow.

Tests may basically include a Scene in which the Mobject in question is self.added/self.played into the scene, rendered at -s or -l.

Fix imports

The way that manim currently handles imports is far from optimal. This should be one of the first things we change.

In fact, this may be one thing preventing us from having standard manim and community-manim installed side by side on a single machine (see #6).

Enhance -n (allow to render intervals of animations)

As the title says, simple issue. Right now -n can be used to start at a specific animation (and thus skip the prior ones). We should extend it to be able to render a specific interval of animations, so allowing it to stop rendering at a specific animation too.

Also... --start_at_animation_number is not just going to be deprecated then, it's also just obnoxiously long. What should it be called afterwards though? Keeping -n is fine but if we keep the long form... --interval? --animation_interval? --animate_interval?

Manim addons and customizability

@yoshiask had developed an initial structure for what seemed to be support for using addons (i.e., extra functionalities from 3rd-party libs) in manim (see his pull request at the 3b1b repo and his .pptx export extension). This idea has a lot of potential and can be generalized for many functionalities needed by manim.

First, we should implement an API like the one suggested by @yoshiask, which would allow the usage of custom options, classes and functions that interact with manim's inner structures. An example of project that could benefit from this is chanim.

Second, we should generalize this API for multiple functionalities, and allow for more customizability in manim, generally speaking. For example:

  • We should allow the user to pick how the video should be rendered, i.e. which rendering method should be used (this is a matter to be discussed by itself!) - @eulertour's manim web port could benefit from this. For instance, the user could pick between rendering in the browser (using @eulertour's tool) or with cairo, which would leverage the latter from being a required dependency (see #14).

    • This will require, of course, developing an interface that abstracts rendering away. E.g.: To render a shape, we just call render_shape(...data...), a function that tells the renderer to do this, which can then interpret this message as it wishes.

    • Note that anyone would be able to create any renderer compatible with manim that they wish.

  • We can also, similarly, allow the user to choose how to render LaTeX (see #23, #24). They can delegate it to an online website, should someone create an extension for this, or simply use the available installation (the default behavior). We would do this, also similarly, by abstracting away the rendering processes with an appropriate interface.

Note that the use of dataclasses (see #7) and proper organization/typings would immensely aid us here.

Comments? Suggestions?

Documentation host or domain

We've been discussing where should we host our documentation and what its domain should be.

Options for host:

  1. Host using GitHub pages;
  2. Host with ReadTheDocs;
  3. Host on a private server. (More can be added here as the discussion goes)

Options for domain:

  1. Use the GitHub pages domain (Note: could be too long: manimcommunity.github.io/manim ...)
  2. Use the ReadTheDocs domain
  3. Use a free domain acquired online

Ideas? Suggestions?

Get rid of docker

Docker was intended to give users a full environment ready to run manim with: ffmpeg, cairo, latex, the right python, and the up to date code, among other things.

However, it ended up being too technical for most people. Also, it seems like most of the installation problems are coming from cairo, which may be replaced soon. Docker doesn't seem to be the right fix for that.

Create a test to certify a clean install

Originally posted by @eulertour in #9 (comment)

Basically something to dry-run a Scene, ensure that the bare minimum to render successfully is actually installed, and give a helpful error like LaTeX/FFmpeg isn't installed, media directory isn't writable, etc.

Having done this, the correct way to install manim would be pip install manim, and then pytest test_install.py. (Or the equivalent in the testing library that we end up using.)

This would flag issues about system dependencies such as latex, ffmpeg, etc.

Use a logging library

The first step in this task is to replace print calls with a logging library. The next step is to add logging calls to places that don't have logging yet.

@eulertour mentions both Loguru and the stdlib logging library as options. I'm partial to stdlib logging.

`extract_scene.py` calls `main` without argument

The main function defined in extract_scene.py accepts a required argument. But the last line of extract_scene.py calls main without arguments.

  1. This probably means that nobody is calling extract_scene.py as a standalone script (which would trigger the wrong main() call. If this is the case, we should just delete the last two lines.
  2. If we want to support using extract_scene.py as a standalone script, we should fix that call and determine what is the appropriate config to be passed to main in that case.

Add typings

We should add typings to manim:

  1. It allows for more precise intellisense and easier-to-check code (via mypy and whatnot);
  2. It ensures we follow a structure that makes sense (i.e., ensures we don't have a data model in which people get completely lost);
  3. Writing documentation and specifying types are tied to each other. When we work on documentation, it is inevitable that we will specify types for things, so why not add it to the code as well?

Notes:

  1. We can use stubgen to have a headstart;
  2. We can use stub files in order to keep typings separate from the code and not have to change things much (but this is debatable - please expose your opinions on this);
  3. This is very related to the dataclasses solution proposed in #7, and may ultimately be a consequence of it.

Thoughts?

Work on ArcBetweenPoints

As some of you know I've been working with arcs. A lot. At some point I'd want to add ArcPolygons to this manim fork, but first I want to work on the Arc derived class I usually work with, ArcBetweenPoints.

As per the tags there are 3 things to do here.

The bug:
ArcBetweenPoints, unlike all other classes I know that use points (Line, etc.), does not support points in the form [x,y,z], instead they have to be passed as np.array([x,y,z]). First running into this problem myself wasted a lot of time because I wasn't accustomed to the error messages, and this will likely cause grief to the one or the other person as well. The issue is this line of code:

self.put_start_and_end_on(start, end)

Or rather where it leads towards, in mobject.py, where arithmetic with points happens (Point1-Point2), and Python's List does not support that, while np.array does.
In that place, line 553 is where I'd place the fix:

target_vect = end - start

target_vect = np.array(end) - np.array(start)

If np.array gets a numpy array it's not going to change anything so all this does is make the code more robust and ArcBetweenPoints easier to use, like it's the case with other point driven classes already.

The enhancement:
Defining ArcBetweenPoints with a given angle is all nice and stuff, but sometimes (or in my case basically always) you'll want to define such an arc by its radius. The ArcBetweenPoints class should be able to take in a radius instead of an angle, and calculate the according angle itself. I've got the code for that already usable, waiting for a better implementation than an outside function.

My issue here though is that this code needs math and cmath to work.
Question 1:
What is the correct way of making sure those are available in geometry.py? Sure they can just be imported at the top of the file but is that actually the right way to go around it?

The documentation:
I'd prefer it if all classes are equipped with some sort of in place usage examples. Something along the lines of:

class ArcBetweenPoints(Arc):
"""
ArcBetweenPoints needs at least 2 points to draw the arc between.
The curvature of the arc can be specified via angle or radius.
When both are given, radius is used and angle is ignored.
ArcBetweenPoints([x,y,z],[x2,y2,z2],angle=1) results in this...
ArcBetweenPoints([x,y,z],[x2,y2,z2],radius=1) results in that...
"""

Question 2:
What is the exact format for such comments we should go with?

Replace miktex with pylatex

This would reduce the number of non-python dependencies (i.e. packages that cannot be installed via pip and therefore could cause problems during manual installation).

Anyone wants to volunteer?

Upgrade the python3.8 version in Travis

The current travis setup (see .travis.yml) is fixing the version of python 3.8 to 3.8.0. This is due to the fact that during testing, the tests of more recent versions failed. It is unclear at this point why they failed.

In the future, we should aim to always test with the latest available versions.

Deprecate or document/test SCENES_IN_ORDER

The function get_scene_classes_from_module in extract_scene.py checks for a module member called SCENES_IN_ORDER. It appears that the user is allowed to assign a list to this that specifies the order in which the scenes will be rendered.

  1. Given that there are plans to render scenes in parallel, this would become deprecated.
  2. If we continue to support rendering scenes in series for some purpose, I'm still not sure what is the purpose of determining the rendering order.
  3. Even if we wanted to do so, it seems more natural that the rendering order should be the order in which scenes appear in the module, rather than setting a separate variable for the order.
  4. In any case, if the SCENES_IN_ORDER variable will still be used, we should document it somewhere. If it won't be used, we should delete it. (Lines 124 and 125 in extract_scene.py are the only mentions to this variable in the code base.)
  5. Finally, if we do choose to support it, we should write a test for it.

Command length and general user convenience

I think we'll have to drag this issue in the open and properly discuss it, rather sooner than later.

Right now the code supports a wide range of arguments for some desired parameters. For instance using medium quality.
-m
-r medium
-r 720,1280 (although this would default to 60FPS, not 30)
--medium_quality
--resolution medium

Personally I'm fine with that. It's not like this is adding over hundred lines of code, when reading the code it's easy to isolate and skip these areas, and the performance overhead is almost certainly not even anywhere near being measurable.

Now we have people like @huguesdevimeux who are for cutting down some of that (#64), with which I could live. Although I think that's not really necessary.

We also have @eulertour with statements like... this here on my current PR:

At some point these should probably be merged into a single --render_quality flag which takes 4 arguments.

Let's also make properly clear what's at stake with decisions like these.
Currently, and particularly with my pending constants.py changes we can drive manim like this:
manim project scene -psm
I like it that way. When working on my projects I usually have to re-render stuff a lot, and for very different reasons, so using it like this I often switch the m for an l, or remove it, or, now with the new changes, add a k. I can quickly get manim to get me what I need.

On the other hand it sounds like @eulertour is more in favor of enforcing people to have to use manim like this:
manim C:\FullProjectFolderName\project.py scene --media_dir D:\SomeOtherProbablyNotSuperShortAbsolutePath --preview --save_last_frame --render_quality medium

Now I don't know about you, but for me, that there is just ugly and panic inducing in the extreme. I unironically estimate that if I always had to use manim like this then my recent video would've probably included a couple real extra hours of work, solely dedicated to writing out and copypasting commands. Please, for the love of deer goats, let's not go there. Yes?

If anything, manim is tool. It's code is there for us to create something that is as powerful and convenient to use as possible. We should never sacrifice even the tiniest bit of either of those unless there's substantial gains for the other we can't get otherwise. I'd even argue we should add the ability to define FPS in the arguments as opposed to having it only tethered to predefined qualities. And maybe add an addition to not just start at a specific animation but also end at one as opposed to always rendering through to the end.

Thoughts and opinions?

Document and test the '-' file name command line argument

The get_module function in extract_scene.py allows the user to input the code for a Scene from stdin, but only when the argument passed as file name is -. This should be documented somewhere. At the very least, the -h flag should mention this possibility.

Folder Structure

Following the rework of constants.py (and associated files) the question of Folder Structure and dirs has come up.

For now the PR I finished has at least moved images out of the videos folder (because really, why would they be there), but there's some open questions.

There's two questions we have to briefly address.

First, how is the folder structure supposed to look like?
Here's I think the most current suggestion by @eulertour

media
  <module name>
    <scene name>
      svg
      images
      video
        <resolution>p<framerate>

I'm fine with that in principle, but have some reservations. In practical application this could create quite a bit of code overhead if the DIR variables aren't broken down to one. The reason for that is if we go with this folder structure, but also have a separately defined, say for example, VIDEO_DIR, then we need separate code to move stuff around for it to look like this, lest we drop module/scene:

video
  <module name>
    <scene name>
      <resolution>p<framerate>

With respect to that something like this:

media
  svg
  images
    <module name>
      <scene name>
        <resolution>p
  video
    <module name>
      <scene name>
        <resolution>p<framerate>

Is generally a bit more consistent in behavior, whatever the DIR variables, and the code would be simpler too.

And while we could also split images by resolution like that, and I can think of some cases where that would be helpful, do we actually want to? I reckon that with the "save all images" options it definitely would be better, because without being split by resolution that could turn into a real serious mess for a user, but eh. I'm not partial to either.

Second, what are our DIR variables going to be?
Currently we have:

MEDIA_DIR
VIDEO_DIR
TEX_DIR
TEXT_DIR

As @eulertour suggested and I thought of too TEX and TEXT should probably be merged into SVG. Them having to be split would be a real fringe case, and even then the cli output should be enough to do so.
What about the others? Do we remove VIDEO and put it into whatever MEDIA is by default? For that matter do we just remove TEX/TEXT and put that there too, leaving only MEDIA?
Or do we go in the other direction and instead also add IMAGE to place images separately?

My suggestion would be the second folder structure like that, with a resolution folder for images, TEX/TEXT put together and add IMAGE.

Any ideas or opinions?

Test the sample scenes

There are recent reports that python -m manim example_scenes.py SquareToCircle -pl does not work on a fresh install of the latest version of manim.

We should go over the sample scenes and test that they can be rendered and visualized with a fresh start.

Bug in NumberPlane when using x_max (and likely the y/min values too)

NPac=NumberPlane(x_max=20).add_coordinates()
class Test(Scene):
    def construct(self):
        self.add(NPac.shift([-15,0,0]))
        self.wait(1)

This results in the horizontal lines not being anywhere near as long as they should be. Leaving this here for either someone else to investigate or fix, or in case of doubt myself coming back to this some point in the not too close future.

How to test videos?

In the future, we will need to check the videos generated by manim for correctness.

One way to go about it, is to write tests in such a way that they output a single frame in PNG format. And then we can load that PNG in python and check everything, even pixel by pixel if needed. For example, it would be fairly easy to write a test that checks for "this PNG/frame contains a red square of such and such size over a black background".

-w flag enabled by default

I just noticed an issue with the CLI arguments, and more particular concerning the -w (write_to_movie) flag.
It appears that when -s (save_last_frame) is False (most of the time, I think), write_to_movie is set True even though -w hasn't been placed. (that's why for example in the README just -pl is used instead of -plw).

That is shown at l. 166 of config.py:
"write_to_movie": args.write_to_movie or not args.save_last_frame,

This may cause issues when one wants to get the last frame AND the video.

Should we fix it or let -w enabled by default (even though it's not very rigorous) ?

Choose a linter and formatter

In case you aren't familiar:
A formatter ensures all code is formatted in the same way
A linter ensures that all code follows best practices

A bunch of popular ones are listed here.

Personally I don't really care as long as we choose a good one, but if nobody has a strong opinion either way I'll nominate black + pylint.

After we get Travis CI running we can enforce them for each merge.

Fix the installation of or completely replace cairo

As many, or rather probably most of us know personally, and have seen on the Discord channel too, the installation of cairo regularly causes problems (at least on windows.)

As of now the documentation refers to installing dependencies via the requirements.txt in the Directly section:

# Install python requirements
python3 -m pip install -r requirements.txt

Although there is a Directly(Windows) section, if the original Directly doesn't work on windows, then we'd want to write that properly as in Directly(Mac, Linux) or whatever, if we already can't streamline the process across platforms.

While this requirements.txt is in principle a great and convenient idea, pycairo (and to a lesser degree opencv-python) were often a source of issues. Recently the version numbers have been removed from the requirements.txt, which helps, but pycairo is still a unique case, as just trying to install it like this will always fail, and if that isn't bad enough, will actually ask for a Microsoft Visual C++ installation.
If you haven't tried it yet, that's a huge amount of overhead and also doesn't really make the pip pycairo install work reliably either. Highly frustrating to users.

The documentation goes on to mention downloading pycairo files (https://www.lfd.uci.edu/~gohlke/pythonlibs/#pycairo), but For most users, pycairo‑1.18.0‑cp37‑cp37m‑win32.whl will do fine. and Make sure that pycairo==1.17.1 is changed to pycairo==1.18.0 in requirements.txt. are also deprecated comments. The manual file installs are tied to the python version, so someone with 3.8 will not be able to install pycairo cp37 if I remember correctly. All in all this installation is just unreasonably stacked with pitfalls and will throw off most casual users with ease. Talking of python 3.8, manim can be installed on it and we probably should roll on with that too.

So what we should do is, at least dramatically clean-up the documentation, or at best find a way to either streamline the windows pycairo installation or replace it altogether.
Though I'm raising the issue I don't have any great idea for how to implement one of the better solutions.

A small Bug in setup.py

In install_requires of setup.py the library colour is mentioned twice. This needed to be changed.

Add Versions of Library in requirement.txt

How about adding specific version to install the libraries and this could prevent bugs that could cause because of changing of the library's API and we didn't update those in our code. So specifying the version in requirements.txt is good.

Entry points, naming

As it stands both Manim and ManimCommunity have their entry module named manim.py, which is a problem when both are installed and in the environment variables, as there'll be two instances of manim.py to pick from. To keep ease of access I'd suggest something along the lines of renaming this manim.py to manimc.py (or something else) so that code can be quickly run on either installed version, which those developing here will likely want to do.

Multithreading (or rather multiprocessing)

A small disclaimer, this is more of a longterm issue with quite a bit of planning before any PR with actual merge intent should ever be attached to this.

First of all, what should we use?
There's the threading module, however it does not use additional cores. Using it isn't advised thusly, because it's theoretically possible max performance improvement is just +100% for all scenarios (you know, two threads per core usually), and actual performance increases are expected to be somewhere around +10% to +50%. Nothing irrelevant, but the simple fact is we can do better.

There's the multiprocessing module, and this is almost what we should use. There's also the multiprocess module that should generally be more robust (better pickling etc).
So multiprocess it is.

So, can it work? Yes it can. I've had to jump through some hoops but I managed to hack something together to multiprocess 4 simple scenes side by side. Theoretically possible speed bonus is +300% and actual was... +200%. Now that is nothing to scoff at. Obviously with more things processed simultaneously both theoretical and actual speed goes up, capped really only by the running hardware and available splitting. On my hardware with 16 cores I expect to be able to reach an actual improvement of about +1000% for example. That's around an order of magnitude so... it'd change things up a lot.

Alright, we've got the intent and the module, now onto the plan.

First we should enable multiprocessing for scenes. Multithreading is really easy, but multiprocess is a bit more picky. As I said, I have gotten scenes rendered in it but something about the exact implementation in extract_scenes.py is still making it fail while pickling. I'm not sure what it is, I just know that it can be fixed, since I was able to sidestep it.
(Fixed, see next post for that)

Once that is done we should be able to use manim (assuming 4 scenes in a file) with something like this:
manim project -at
And have all 4 scenes dropped into their own process.

Once this is implemented and works well, we can move on to the real prize.
In order to make manim faster in general we can split scenes into intervals, and then have each interval be handled by a separate process. For instance if a scene has 8 animations then we could split them like [1,2], [3,4], [5,6], [7,8] and throw each interval into its own process.
Now this would mean that some calculations are done multiple times. While the process rendering [1,2] would have no overhead, the one rendering [3,4] would have to do the calculations of [1,2] before it can start rendering. However those calculations are usually very, very short compared to the rendering process, so that's not really much of an issue.

Last but not least, we'll want this multiprocessing to be an option, not replacing the current way of serial rendering. Aside from making sure we have options if something ever causes issues, some people use manim to render stuff that is actually very heavy in calculations, like fractals. For those cases multiprocessing could actually be actively detrimental, and aside from just needlessly hogging computing power could also overflow the RAM. So multiprocessing shouldn't be on by default, but when it's in we should make sure everyone knows it's there. Maybe even drop a small message every time manim runs with just 1 process.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.