aspiers / ly2video Goto Github PK

There should be regression tests at least, if not also unit tests.

There needs to be a --version option supported for displaying the version number and license summary, as per GNU recommendations.

All the source references assume that note tokens live within the input file, but they could live within an included file.

[known issue taken from http://code.google.com/p/ly2video/ ]

pyPdf stopped being maintained 2 years ago. It has been superceded by the PyPDF2 friendly fork which is actively being maintained. ly2video should probably switch for that reason alone, even though it doesn't need the wider range of support of PDF formats.

When subtitles are used in .ly file, the subtitle is displayed in place of the title.

If the piece begins with a chord and no note, then the chord will typically get skipped:

absolutePitches: {(21, 23): u"f'", (21, 21): u"e'", (21, 19): u"d'"}
c   @  26: 25 | grob(time=0.0000, x=  273, tick=    0) | MIDI(tick=    0)
    grob's pitch 48 not found in midiPitches; probably a tie - skipping grob and tick.
    midiPitches: 60 (C) 64 (E) 67 (G)
d'  @  22: 19 | grob(time=0.2500, x=  291, tick=  384) | MIDI(tick=  384)
e'  @  22: 21 | grob(time=0.5000, x=  314, tick=  768) | MIDI(tick=  768)
f'  @  22: 23 | grob(time=0.7500, x=  337, tick= 1152) | MIDI(tick= 1152)
sync points found:     3
             from:     4 original indices
              and:     5 original ticks
   last tick used:     3
    ticks skipped:     1

This results in the first video frame starting from the first note (after the first chord), whereas the MIDI file starts from the first chord, so they are immediately out of sync.

The ChordNames context does not have an impact on MIDI rendering in LilyPond, but it does result in multiple note events being rendered as a single text grob (e.g. "Eb-7") whose PDF annotation links back to a location in the .ly source which looks like a normal note e.g. ef1:m9, which the tokenizer currently cannot distinguish from a normal note.

This makes it almost impossible for the synchronization algorithm to work. It's cleaner just to remove chords from the equation. This can easily be done with the MIDI by creating a separate \score which excludes the chords, but they still appear in the PDF.

Need a bug() function for reporting fatal errors which indicate bugs in the software. It should output some text asking the user to file a new issue on this tracker.

If the input .ly file does not contain \version "2.14.2" then ly2video will run convert-ly to convert it to that version and then operate on the result. An explanation for this is detailed in Jiri's thesis; a translation of the relevant section is:

Each new version of LilyPond can potentially break compatibility with the last. Since ly2video was written primarily for LilyPond's latest published version at its time of creation (i.e. 2.14.2), translation issues may occur when using older projects.One solution is to call the auxiliary script convert-ly, which is delivered directly in the installation package of GNU LilyPond and older projects that can convert to its current form. However, transformation may fail in some cases.

However, since LilyPond has changed hugely since 2.14.2, the current code does not make sense, especially given that in the majority of cases it will now result in a downgrade of the syntax used, even though the original intention was for an upgrade. So we need to decide how to handle this, and then update the code and README accordingly.

\turn generates extra MIDI notes but no extra annotations in the PDF.

[ known issue taken from http://code.google.com/p/ly2video/ ]

Currently the zip file includes some dependencies. These should be handled out of band, and corresponding installation instructions provided ('pip install PIL' etc.)

[ known issue taken from http://code.google.com/p/ly2video/ ]

The --title-at-start option is currently broken.
I found one of the causes:

--- ly2video.py.orig    2013-02-08 08:29:55.953863236 +0100
+++ ly2video.py 2013-02-08 08:32:02.424701547 +0100
@@ -1980,7 +1980,7 @@

     # generate title screen
     if options.titleAtStart:
-        generateTitle(titleText, width, height, fps, titleLength)
+        generateTitle(titleText, options.width, options.height, fps, titleLength)
         output_divider_line()

     # generate notes

but this only partially fixes the problem. See my next submiited issue.

I need to write a trouble-shooting guide which explains basic concepts and how to debug things when things like synchronization go wrong.

If a title page is requested, ly2video creates 2 temporary videos and then joins them. To avoid lossy changes, the format of these videos should be the same as that of the requested output file. This should be as simple as replacing the filename extension of the interim video with that of the specified output file.

ly2video doesn't work on Linux out of the box; please see:

http://thread.gmane.org/gmane.comp.gnu.lilypond.general/72263/focus=76357

for details.

After my fix (--title-at-start broken(1) bug report) I am still getting the following error:

TITLE: ly2video will generate approx. 90 frames.
Traceback (most recent call last):
File "./ly2video.py", line 2015, in
status = main()
File "./ly2video.py", line 1983, in main
generateTitle(titleText, options.width, options.height, fps, titleLength)
File "./ly2video.py", line 176, in generateTitle
nameFont = ImageFont.truetype("arial.ttf", height / 15)
File "/usr/lib64/python2.7/site-packages/PIL/ImageFont.py", line 218, in truetype
return FreeTypeFont(filename, size, index, encoding)
File "/usr/lib64/python2.7/site-packages/PIL/ImageFont.py", line 134, in init
self.font = core.getfont(file, size, index, encoding)
IOError: cannot open resource

I am using lilypond-2.16.2 on Linux Fedora 18 (64bits)

If my score contains chord repetitions like

<a c e>1 q <f a c>2 q

the script dies with

ERROR: Expected pitch token during conversion from relative to absolute
pitch, but found <class 'ly.tokenize.Unparsed'> instance @ /home/dir/ly2video-0.4.1/ly2video.tmp/sanitised.ly:25:38:

q

Feature request:
Would it be possible to let me put commandline options in a configuration file that I can load for easy re-usage ?
It can be quite a puzzle to find the best commandline settings. Having the best settings in a file would make things easier.

Please add to each source file:
*a copyright notice
*a statement of copying permission

Please also include a copy of the license itself.

See: http://www.gnu.org/licenses/gpl-howto.html

Thank you!

Carlo

[Issue description edited to avoid accidentally spreading false information.]

Sadly, the ffmpeg project forked into avconv, and Ubuntu and Debian switched to avconv. (Here is a reasonably unbiased version of the history behind this.)

So, for a 1.0 release ly2video should probably support avconv in addition to ffmpeg. Perhaps the best way to do this would be for ly2video to look for avconv first and, if not found, look for ffmpeg.

Calling the variable melody gives error Not enough synchronization points found. Also fails when \new TabStaff is replaced with \new Staff.

This file compiles correctly when the music is explicitly stated inside the individual \staff contexts.

% Created on Sat Feb 09 15:10:04 EST 2013
\version "2.16.2"

\header {
    title = "Ode To Joy" 
    composer = "Beethoven" 
    tagline = "" 

}

\include "english.ly"

melody = {
    e e f g g f e d c c d e e4. d8 d2
    e4 e f g g f e d c c d e d4. c8 c2
    d4 d e c d e8 f e4 c d e8 f e4 d c d g,2
    e'4 e f g g f e d c c d e d4. c8 c2         
        \bar "|."

}

staffClassicalGuitar =  { 
    \time 4/4
    \tempo 4 = 100 
    \set Staff.midiInstrument = "acoustic guitar (nylon)"
    \key c \major
    \clef "G_8"
    \relative c' {  
        \melody
    }
}

staffTabGuitar = { 
    \new TabStaff {         
        \clef "moderntab"
        \relative c' {  
            \melody 
        }   
    } 
}
\score {
    <<
        \staffClassicalGuitar \\ \staffTabGuitar
    >>

    \midi {
    }

  \layout { indent=0\cm
  }
}

\paper {
}

Currently the parsing of pitches from the source .ly file is hardcoded to assuming \include "english.ly" is present. The parser needs to iterate the whole file and extract the correct language automatically.

The top margin is too much when ChordNames are present. I think the best approach is to count how many rows of pixels are totally white at the top and bottom, and then centre the effectively cropped music accordingly in cropVideo().

An English translation of Jiri's thesis would be useful for a better understanding of the project.

When ly2video is installed, the toplevel ly Python package conflicts with the ly module of the python-ly package. That package in turn, is needed to run Frescobaldi. I decided to make ly a standalone package so other tools (such as ly2video) can use it as well.

ly2video is using an old version of the same ly module, and probably could be changed so it depends on the python-ly package. The ly parser has improved since then and is maintained very well. I am willing to investigate the effort needed to use the parser from the newer ly module, from the python-ly package. And I can help making the changes. When that is done, python-ly becomes a dependency of ly2video.

An easier solution could be to move the ly module in ly2video to a private space or another name, such as ly2v.ly or ly2vly.

Rather than having the staff horizontally scrolling left past a fixed vertical bar, there should also be the option of having the staff stay still and the vertical line scrolling right. When it reaches the far right of the screen, the staff would refresh to show the next few measures of music.

Some thought would be needed to ensure this happens in a visually intuitive manner e.g. the refresh should perhaps happen just before the line reaches the right hand edge, since humans always read slightly ahead. Also it might make sense to align measure breaks with the left and right-hand edges of the video, although that could pose other problems.

If the .ly source has \hide in, with \hide defined as follows:

hide = {
  \once \override Dots #'transparent = ##t
  \once \override NoteHead #'transparent = ##t
  \once \override NoteHead #'no-ledgers = ##t
  \once \override Stem #'transparent = ##t
  \once \override Flag #'transparent = ##t
  \once \override Beam #'transparent = ##t
  \once \override Accidental #'transparent = ##t
  \once \override Glissando #'(bound-details left padding) = #0.3
}

then Lilypond still generates a NoteOn event for the hidden note, even though it is not present in the PDF. This causes a loss of synchronization between MIDI frames and video frames.

The same happens with \hideNotes.

Currently you have to run ./ly2video.py from the source tree or modify PYTHONPATH, in order for Python to find the Frescobaldi code in the ly/ subdirectory. It should be installable via something like distribute.

Most recent (as of 2/11/13) ffmpeg has depreciated same_quant and ly2video fails.

I forget the specifics of what happened here. If I remember correctly, I had 1 file I created that worked; it was a single staff. When I added a staff, I got the same_quant error.

Removing same_quant degraded video worked but degraded audio and video quality; fix was to insert -b:a 320 and -q 5

It would be great if you could port ly2video to Python 3.

Thank you!

Carlo

Create frames displaying beat before music to count in with via something that creates number of frames ((60/tempo) * fps) for each beat desired.

Currently the pitch matching algorithm in alignIndicesWithTicks() ignores the octave element of the pitch. As a result, it's not very robust.

ly2video currently removes articulate.ly from the sources in order to stop MIDI events which sound ornaments from breaking the synchronisation. However this obviously has a negative impact on the quality of the performance. A smarter synchronisation algorithm might be able to tolerate these additional ornaments in the MIDI stream. Most likely this would require synchronisation based not only on pitch, but also on note length.

I am the packager for Arch Linux; just posting here to let you know that a package is already available for Arch.

The main l2video.py is not directly executable on Linux; even with a chmod +x, the DOS line endings prevent it working, and you have to run it by prefixing the command with python.

When there is no \header present, tagline doesn't get set to ##f, which puts the dimension calculations way out.

ly2video currently uses a copy of some old Frescobaldi 1.x code. Meanwhile, Frescobaldi 2.0 (a complete rewrite IIUC) is being actively worked on, and 1.x is no longer maintained. Having said that, sticking with 1.x could arguably be an advantage, since it's not a moving target, and may well be simpler to understand.

When the scrolling cursor reaches the right-hand scrolling margin, it will jump back to the left hand side. Typically this results in the left side of the video frame showing a partial bar. It would be nicer if the cursor could jump back earlier, at the end of a bar, so that the left side of the video frame almost always starts on a bar line. "Almost" because this is not possible with very wide bars.

It would be cool to allow multiple systems of music in the same video frame, where the cursor line would only cover one system at any given moment in the video.

Currently the pitch matching algorithm in alignIndicesWithTicks() only works with semitone (half-step) pitches.

It seems that a note preceded with \parenthesize ends up having the grob cause pointing at the \parenthesize token not the note pitch. This breaks calculation of absolute pitches. Maybe work-around by just parsing the next token, or use a while loop with a bail-out threshold?

Hi,

thanks for the fast repeated chords fix. Here are some more cases. The one commented out are valid lilypond input but don't work in ly2video.

\version "2.16.0"

\header {
  title = "Regressiontests Chord Repetitions"
}

music = \new Voice {
  \relative c' {
    <a c e>2 q | % chord repetition
    <f a c>1 |
    q | % chord repetition on own line
    <g' c e>4 q q q | % multiple chord repetitions with octave change
    %<a c e>4 q q2 | % different rhythm => q2 script dies
    <g c e>4-. q q q | % repetition after articulation
    <g c e>4 q-. q^. q_. | % repetition articulation normal / up / down
    <g c e>8-. q-. q-- q-> q-^ q-_ q-+ q | % repetition with different articulations
    %<g c e>8_. q^. q-- q-1-3-5 q_1-3^5 q_1-3^4-_ q^5_1-3_+ q | % fingerings/articulations => q-1-3-5 script dies
    <g c e>4\f q\p q\sfz q\mf | % repetition with dynamics
    <g c e>4\f q\> q\! q | % repetition with dynamics / spanners
    %<g c e>4~ q q( q) | % repetition with ties / slurs => <g c e>4\f script dies
    %<g c e>8 q q[ q] q[ q q] q | % repetition =>   q[ script dies
    %<g c e>4 q\( q q\) | % phrasing slur => q\( script dies
    %<g c e>8 q q q-\markup{normal} q_\markup{down} q^\markup{up} q-\markup{ more \italic { complex } \super {markup} } q | % markups => q_ script dies

  }
}

\score {
  \music
  \layout { }
  \midi { }
}

regards
René

Multiple frames are currently generated for title screen; only 1 is needed.

The following website seems to suggest that -r 1/(number of seconds) would do the trick:

http://ffmpeg.org/trac/ffmpeg/wiki/Create%20a%20video%20slideshow%20from%20images

I don't know python, but I think on ffmpeg side, replace

# generate silent title video
        silentAudio    = generateSilence(options.titleDuration)
        titleFramePath = tmpPath('title', 'frame%d.png')
        titlePath      = tmpPath('title.mpg')
        cmd = [
            ffmpeg,
            "-f", "image2",
            "-r", fps,
            "-i", titleFramePath,
            "-i", silentAudio,
            "-q:v", str(options.quality),
            titlePath
        ]
        safeRun(cmd, exitcode=14)

with

# generate silent title video
        silentAudio    = generateSilence(options.titleDuration)
        titleFramePath = tmpPath('title', 'title.png')
        titlePath      = tmpPath('title.mpg')
        cmd = [
            ffmpeg,
            "-f", "title",
            "-r", 1/(variable from --title-duration),
            "-i", titleFramePath,
            "-i", silentAudio,
            "-q:v", str(options.quality),
            titlePath
        ]
        safeRun(cmd, exitcode=14)

ly2video cannot handle \include statements that refer to files in the current working directory. It has been suggested that this is because it copies the main .ly file into a temporary directory for its build work, and it does not copy anything \include'd.

Currently, in cursor scrolling mode (i.e. when --scroll-notes is not used), when the cursor goes past the right margin (as defined by the second value to the --cursor-margins option) the notation instantly changes to the next chunk. It would be visually easier for the eyes/brain to track if the notation smoothly but quickly scrolled from right to left to the new position. This should happen very quickly to minimise distractions (e.g. in < 0.5 seconds).