glapi.py is auto-generated. But it has happened a few times that people make changes to this file directly. We should split glapi.py in two parts, of which only one is auto-generated.

Perhaps we should glapi into a subpackage then.

While we're refactoring, we can perhaps also have a second look at where the vispy.gl namespace is generated (now in the root init.py).

To avoid "copy on error" when transfering array from cpu to gpu, we should use "numpy.float32" instead of the default "numpy.double" type.

Get an error on oscilloscope.py (it used to work):

Traceback (most recent call last):
  File "oscilloscope.py", line 138, in on_paint
    visual.draw(transform)
  File "oscilloscope.py", line 79, in draw
    self.program.uniforms['color'] = self.color
  File "../vispy/oogl/program_inputs.py", line 43, in __setitem__
    self._set(k, v)
  File "../vispy/oogl/program_inputs.py", line 94, in _set
    self._apply(name, value)
  File "../vispy/oogl/program_inputs.py", line 198, in _apply
    _name, length, type = gl.glGetActiveUniform(self._program.handle, loc)
  File "/usr/locaL/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/OpenGL/latebind.py", line 61, in __call__
    return self.wrapperFunction( self.baseFunction, *args, **named )
  File "/usr/locaL/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/OpenGL/GL/VERSION/GL_2_0.py", line 213, in glGetActiveUniform
    raise IndexError( 'Index %s out of range 0 to %i' % (index, max_index - 1, ) )
IndexError: Index 4 out of range 0 to 1
Error invoking callback for event: <PaintEvent blocked=False handled=False native=None region=None source=<__main__.Canvas object at 0x107548210> sources=[<__main__.Canvas object at 0x107548210>] type=paint>
(0.1, 1.0, 0.1, 1.953125000000001e-11)```

It would be nice if the code we wrote would be more ES 2.0-ish, and we would not need to enable point sprites for instance.

It should be pretty easy to analyze the shading code at link-time and determine any extra stuff that need to be enabled.

Further, we could automatically add the #version 120 on top of the shader code.

Error:

$ python hello_fbo.py 
Error compiling shader <FragmentShader 'textured fragment shader'>
    on line 8: '/' does not operate on 'float' and 'int'
        float scalefactor = 1.0 / (c_sze * c_sze * 4 + 1);
    on line 14: Use of undeclared identifier 'scalefactor'
        gl_FragColor.rgb += color * scalefactor;
Traceback (most recent call last):
  File "hello_fbo.py", line 117, in on_paint
    with self._program2 as prog:
  File "/Users/nippoo/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/vispy/oogl/__init__.py", line 87, in __enter__
    self._enable()
  File "/Users/nippoo/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/vispy/oogl/program.py", line 158, in _enable
    self._enable_shaders()
  File "/Users/nippoo/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/vispy/oogl/program.py", line 220, in _enable_shaders
    shader._enable()
  File "/Users/nippoo/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/vispy/oogl/shader.py", line 94, in _enable
    raise RuntimeError(errors)
RuntimeError: ERROR: 0:8: '/' does not operate on 'float' and 'int'
ERROR: 0:14: Use of undeclared identifier 'scalefactor'

The behavior I get is rapid flashing between two frames; no modification is made to the textures. With a little digging I see that the FBO is not created correctly. There are several related problems here:

• glGenRenderBuffers and glGenFramebuffers both point to <bound method GLES2._glFuncNotAvailable of <API for OpenGL ES 2.0>>
• RenderBuffer. and FrameBuffer._create should have checked self._handle != None. Alternately, calling _glFuncNotAvailable should raise an exception rather than return None.
• For both classes, _handle has the default value of 0, which might be considered a valid buffer name on some systems (the spec does not seem to be specific about what values constitute valid names). I suggest using a default value None

Summary

We discussed this approach (which I know think really sucks):

program['a_pos'] = my_array  # Creates a VBO
program['a_pos'] = ClientArray(my_array)  # Uses the array directly
# WTF!

Besides being confusing it allows a sloppy style of using oogl that can byte the user in the ass when he starts using updating attributes in this way.

I propose to not do any magic VertexBuffer creating and instead just document well that specifying an array as an attribute works, but is not very efficient.

What we discussed

During Euroscipy we discussed about creating a VertexBuffer automatically when the user does:
program['a_position'] = position_array

He could also do (to make it explicit):
program['a_position'] = VertexBuffer(position_array)

Similarly, we could allow: program['s_texture'] = image_array

Further there is a Program.set_var() function that can be used to set VertexBuffers that contain structured data (from a structured numpy array). It also allows setting attributes and uniforms as keyword arguments.

Finally, we discussed that to explicitly use a client-side array, the user should do something like:
program['a_position'] = ClientArray(position_array)

What I did

The above is l currently mostly implemented. When the user uses this syntax to update a texture or attribute, a warning is given, because he should use a VBO (and update that) or explicitly use a client side array.

What I did not (yet) do is the ClientArray syntax. Instead if a user uses Program.set_var() there is no auto-conversion.

My worries

I think it is bad practice to automatically convert a given numpy array to a VertexBufer (let alone a Texture), because it allows a style of programming that can become very inefficient if the user makes his code more complex and starts updating stuff. Sure, we warn the user, but this is not optimal.

Also, I think the ClientArray syntax really sucks:

program['a_position'] = my_array  # Creates a VBO
program['a_position'] = oogl.ClientArray(my_array)  # Uses the array directly
# WTF

The approach that I took now (Program.set_var() does not do magic) is also far from optimal, because it is confusing.

Proposal

I propose that we force the user to a way of working that is clear and efficient.

I propose that Program does not automatically create VertexBuffers or Textures.

There are two downsides:

1. The user will have to type a tiny bit more.
2. The unattentive user may forget to use oogl.VertexBuffer and use a client-side array.

I think (1) is worth it, since it just makes things more explicit and much less confusion. For (2) I would argue that this is an API to OpenGL, and users should know a bit about it when they use it. We document this behaviour well and place a warning in the documentation. Further, most examples will use an VBO anyway.

The design of the data transformation system is one of the main challenges of the library. We should come up with a flexible and powerful way of dealing with arbitrary complex transformations on the CPU and the GPU.

We need to think about that during the work on the visuals and the shader templating system.

A few issues:

• Data normalization like "pixel transfer": scale and bias. Data should be normalized on the GPU around the origin with low values (typically abs < 1) so that we have good precision when zooming.
• World wind jitter problem: loss of precision with 32-bit floating points at high values ==> need to normalize the data on the CPU (64 bit) before converting in 32 bit.
• Look transformation tutorial on matplotlib.
• Non-linear transformation: polar plot, geographical projections (like in Cartopy, etc.). We don't have to implement that ourselves, but the transformation API should be flexible enough to support them nicely.

The best way to design a good API is to write experiments (using OOGL) for very specific use cases:

• One with the world wind jitter problem: very long signal between 0 and 1e6: when zooming at high values, x coordinates are completely messed up.
• One with a polar plot.
• A grid where the ticks show values in the data coordinate system but where data normalization happens.

There is currently no way to set a canvas size without setting position while the inverse is possible. Doesn't make sense to me.

I would prefer to split geometry and have a set_size/get_size set_position/get_position, but also a set_x, get_x, set_y, get_y, set_width, get_width, set_height, get_height

It should be easy to evaluate precisely the FPS of a visualization on any script (e.g. app.measure_fps() or something).

@rougier suggestion: "I use to draw as fast as I can using the idle function (glutIdle will call you function when it has nothing else to do) and compute fps over a 5 seconds delay, that's generally enough to get a (rough) felling about speed."

Update the existing tests to make them work with nose (no need to use the native unittest Python module).

We should also set up tests for the various sub-packages:

• event system
• GL commands: if a GL context can be acquired, draw some basic shapes and compare them with ground truth (using image processing algorithms)
• OOGL: write basic tests for OOGL features, get the list of generate GL calls (with the arguments) and compare that with the ground truth. This is pure Python here, no need for any GL context.

From examples/app-event.py, when one move the mouse with a button pressed down, the event report no button pressed (button = 0).

Take the examples/atom.py and tries:

app.Canvas.init(self, show=True)

The window is immediately shown while my canvas initalization is not yet finished. Showing the canvas should be deferred until the app is ran.

We could create sub-folders in examples:

• app: just show basic examples with Canvas and the event system
• gl: visualizations with pure GL, no OOGL
• oogl: visualization examples with OOGL

We could also get rid/merge/clean up some of the existing examples: we have many of them right now, even though we're just at the beginning! I think it's better to have few examples that are various enough rather than plenty similar ones.

With basic testing for now, later maybe pep8.

A couple of suggestions concerning the main page:

• "Who are we?" section on the main page, with our names and links to our respective libraries (and a note saying these libraries will be eventually merged in Vispy)
• Screenshots
• Link to "Examples" in the menu
• Links to @rougier's PDF on "Presentation at EuroSciPy"

The current implementation of the Timer class does not give any information about the time elapsed since the last frame rendered.

Such an attribute is needed for users who want to animate entities at a specific speed, independent to the actual frame rate.

To test code coverage automatically with Travis.

I've added some simple test in test_program that fail.

All shapes are grey. cloud.py does work (white disks with a black border)

When an attribute or uniform is specified incorrectly, it is currently ignored silently since we assume that the variable was optimized out of the program. This makes it more difficult to track down errors.

If each shader has the ability to report a list of its own attributes and uniforms, this would allow us to distinguish between optimized and non-existent variables. This could be done automatically for most shaders, or manually in the cases (if any) where parsing is too difficult.

With Nicolas' new implementation, bug fixes, new examples, code reorganization, cleaner repo :)

We should to a tag and maybe use the new releases feature on Github?

I get this on OSX:

on_resize (300x300)
Traceback (most recent call last):
File "_ctypes/callbacks.c", line 314, in 'calling callback function'
File "/Users/rougier/Development/Public/vispy/vispy/app/backends/glut.py", line 208, in on_draw
raise Exception('this should not happen')
Exception: this should not happen

Some comment on Buffer/VertexBuffer api

1. The current buffer API doesn't allow to set subdata starting at offset 0. We need a count argument stating how much data we want to update (in set_data).
2. Also, we should be able to only upload some sub-data without ever having a full copy on the CPU side. This means we need a set_size/get_size to specify the actual GPU buffer size.
3. I think we need only a single "pending_data" variables since updating the whole data will invalidates all previous pending operations (the whole data will be rewritten)
4. In the current implementation, I suspect the order between data/sub_data is not enforced and this might be problematic.
5. Which make me think that since we do not have any local copy of the data, how to tell the user that his data will be uploaded later ? This might be problematic if he manipulates the data in the meantime.

Right now event.py is at the top-level in the package. I would prefer to have it in a sub-package (app?). There's no much sense to put it where it is now in my opinion.

On OSX:

scroll wheel event does not seem consistant across backend (got float value in qt, int values in pyglet, and nothing in glut (but this last one is expected)).

Also, mouse wheel event are generated from the trackpad on my macbook pro and cannot distinguish between horizontal or vertical events in the callback (get only a single value delta).

Some comment on new shader code (Program class)

1. enable_objects is pretty useless as it stands now since it is only used in draw_elements for the index buffer. We should activate/deactivate the index buffer when drawing and that's it.
2. Overloaded methods need docstring, at least for us to understand quickly what the method is doing.
3. assert isinstance(name, str) is pretty useless test since it is followed by an if/elis/else
4. set_data seems a better name for set_var since a bunch of data can be set

I propose each GLObject gets a status flag so that an error is shown only once. The error will probably propagate in the sense that all other related objects will fail as well, but at least its limited to a handful.

Related (since now self._handle < 0 sometimes means an error occurred): self._handle should be -1 if uninitialized, not 0.

• It could lead to our own OpenGL ES wrapper with ctypes (and/or Cython?) and no hard dependency on PyOpenGL.
• It would solve OpenGL problems on some Windows installations.

Code is more or less ready, but we've not yet discussed the API.

The current implementation assigns the uniforms on each draw. According to the docs, the uniforms stay associated with the program, even if another program is selected.

I thought I tested this at some point, but I then needed to assign them at each draw. Needs another look when we have refactored the ShaderProgram class.

In the example atom:

Comment the two lines:

        self.program.attributes['a_position'] = a_position
        self.program.attributes['a_color'] = a_color

and particles are still moving (but color does not change).

It is one of the many bugs I encountered with shader/program/buffer so I would strongly advice we (I ?) merge the second proposed implementation or at least test the current one very thoroughly.

boids.py and hello_sprites.py show only a black screen with no error.

Confirmed on OSX 10.8 (Mountain Lion) + Nvidia and Windows 8, AMD 7870 GPU.

boids.py and hello_sprite.py requires the GLSL version such that glPointCoord is found.

How to render efficiently a large number of independent primitives? The old way of doing it is to use glMultiDrawArrays... but this function is not available in ES. It probably means it's not the best way of doing it, so we need to find something else.

Here are possible methods, and we should experiment with them.

First case: drawing lots of line segments with GL_LINE.*

• Multiple lines: GL_LINES + index buffer and we're fine: no copy during backing.
• Multiple lines: GL_LINES + double the number of points (+ extra point at the end of every line?)
• (Multiple lines: two GL_LINES calls without index buffer (+ extra point)?)
• For GL_LINE_STRIP, maybe a "discard" in the fragment shader, with a varying variable specifying whether we are between two primitives (no integer). Or alpha between primitives. Precision bug on some architectures (Unix).

Second case: rich lines or triangles or polygons with GL_TRIANGLE.*

• GL_TRIANGLE_STRIP: use degenerate triangles between primitives.
• Thick lines (GL_TRIANGLES and index buffer), no problem.

Well, it says it in the title. The examples are organized in directories and the transforms.py only made it to one of them.

What about putting this transforms module in vispy/utils?

On OSX, reported event.text is wrong (get scancode instead):

If I press key "4" on my keyboard (which is "'" or "4" depending on shift/capslock), I always get 4.

The texture float extension is currently not supported while it is (seems to be) available in GLES:

http://www.khronos.org/registry/gles/extensions/OES/OES%5Ftexture%5Ffloat.txt

If you run the spinning-cube example using the qt backend, the timer stops after 15/16 iterations.

... or mostly PEP8 compliant, with the help of the pep8 command-line tool. We should also structure the code with visible headers (see e.g.the IPython notebook code).

If I press "é" on my keyboard, I get:

Traceback (most recent call last):
File "/Users/rougier/Development/Public/vispy/vispy/app/backends/qt.py", line 234, in keyPressEvent
text = str(ev.text()),
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 0: ordinal not in range(128)

(most likely utf-8 problem)

Why is the Canvas class named Canvas while it is truly a window ?
I think we'll need the canvas class later to express a part of a window that can be drawn upon.

There are currently two low-level GL namespaces: gl and oogl. The first one contains raw OpenGL ES commands, the second one implements an object-oriented layer on top of OpenGL. Even though we definitely need both, I fear it might be confusing for new-comers that even the simplest examples need two different namespaces that do basically the same thing.

I think it would be worth writing extremely simple functions in oogl that implement the most basic things, e.g. something like oogl.clear(r, g, b, a), maybe oogl.draw_elements(...), etc. At the very least, we could just write dumb wrappers like:

def clear(*args):
    gl.glClearColor(*args)
    gl.glClear(gl.GL_COLOR_BUFFER_BIT | gl.GL_DEPTH_BUFFER_BIT)

It would simplify the simplest examples, making them cleaner and easier to understand. I can already see beginners asking: "But what is the difference between vispy.gl and vispy.oogl? Why do I need them both just to display a black screen?" and so on.

Another advantage is that it would make the code conventions more uniform, i.e. we use names_with_underscores in OOGL but camelCase in GL, which is not really nice and might confuse beginners.

One goal could be the following: the most simple things could be done with either vispy.gl and no oogl (raw OpenGL), or vispy.oogl and no pure GL.

On OSX, I get:

2013-08-08 11:41:53.912 Python[2485:f07] GLUT Warning: glutCurrentModifiers: do not call outside core input callback.

I have a display bug on a new Sony VAIO laptop with Windows 8 64 bits and an AMD Radeon HD 7650M GPU (tried with latest stable Catalyst driver and beta driver 13.200). In all examples, the window alternates between the background color and the scene (2-4 times per second). This bug does not occur on other graphics libraries (e.g. Galry).