Comments (23)
The /tmp/ly2videoXXXXXXX
directory is the one with the important debug; I don't think you need to look at the temporary Lilypond files. Did you see https://github.com/aspiers/ly2video/blob/master/TROUBLE-SHOOTING.md yet? Hopefully that helps.
from ly2video.
Yeah I've seen the troubleshooting, but it's not helpful in this case. I cut down my file to what I think is a minimum not-working example:
\version "2.20.0"
preVerse = \drummode {
<<
{ cymr8 cymr8 \tuplet3/2{hh16 hh hh} hh8 \tuplet3/2{tomh16 tomh tomh} \tuplet3/2{tomh16 tomh toml} \tuplet3/2{tomh16 tomfh tomh} tomfh8 }
\\
{ bd8 bd8 s4 s2 }
>>
}
theMusic = \drummode {
\time 4/4
\tempo 4 = 84
\preVerse
}
music = {
\new DrumStaff {
\set Staff.instrumentName = #"drums"
<<
\new DrumVoice { \theMusic }
>>
}
}
\score {
\music
}
\score {
\unfoldRepeats \music
\midi { }
}
Okay I could have reduced the number of notes to make it more minimal, but I don't think that's the problem.
Does anything stand out as obviously not supported?
from ly2video.
I copied the dump-spacetime-info.ly file from /tmp to my test code directory. I modified test.ly to contain:
\include "dump-spacetime-info.ly"
and when I ran lilypond, no errors were generated. The pdf looked normal. There was no output of "spacetime info" that I could see. Looking again at that troubleshooting page, it says there should be output to stdout. I didn't see any when I ran the lilypond command manually.
from ly2video.
I noticed that dump-spacetime-info.ly did not contain entries for DrumStaff or DrumVoice so I added them by copying Voice and Staff:
\layout {
\context {
\DrumVoice
\override NoteHead #'after-line-breaking = #dump-spacetime-info
}
\context {
\DrumStaff
\override BarLine #'after-line-breaking = #dump-spacetime-info-barline
}
\context {
\Voice
\override NoteHead #'after-line-breaking = #dump-spacetime-info
}
\context {
\Staff
\override BarLine #'after-line-breaking = #dump-spacetime-info-barline
}
\context {
\ChordNames
\override ChordName #'after-line-breaking = #dump-spacetime-info
}
}
However, when I run lilypond, it gives this error:
Drawing systems...dump-spacetime-info.ly:35:17: In procedure ly:pitch-octave in expression (ly:pitch-octave pitch):
dump-spacetime-info.ly:35:17: Wrong type argument in position 1 (expecting Pitch): ()
I guess DrumVoice behaves differently than Voice.
from ly2video.
It's years since I worked on this, but looking at dump-spacetime-info.ly, I see it only generates output for NoteHead
, BarLine
, and ChordName
. Would that explain the problem, if you're only processing unpitched percussion notes?
from ly2video.
Unfortunately I don't know anything about lilypond internals. If this isn't something you can fix by some glaring omission, I doubt I'm going to be able to fix it without some hand holding.
Is there a reason NoteHead
would be different for pitched and unpitched notes? Is there a more base class that both might derive from?
from ly2video.
That's the right question but I can't remember - however I do seem to remember it's not hard to find out from the LilyPond reference manual.
from ly2video.
I meant the internals docs not reference.
https://lilypond.org/doc/v2.23/Documentation/internals/drum_005fnotes_005fengraver suggests that DrumVoice creates NoteHead grobs.
from ly2video.
Line 1335 in baddaf4
dump-spacetime-info
function in the dumper to check.from ly2video.
I also hate scheme. I thought it was a waste of time at school and I'm shocked to see something actually written in it!
(format #t "\nly2video: (~23,16f, ~23,16f) pitch ~d:~a:~a @ ~23,16f from ~a:~3d:~d"
left right
(ly:pitch-octave pitch)
(ly:pitch-notename pitch)
(ly:pitch-alteration pitch)
(+ 0.0 (ly:moment-main time) (* (ly:moment-grace time) (/ 9 40)))
file line char))))
Can you break down this function call? The error complains about
Wrong type argument in position 1 (expecting Pitch): ()
It doesn't seem to like (ly:pitch-octave pitch)
in position 1 according to the line number.
What function is complaining?
from ly2video.
Well that seems to confirm my suspicion: the code is assuming the grob has a valid pitch
value, but it doesn't, so invoking the ly:pitch-octave
function on it fails. The error says it was expecting a Pitch
type (obviously, otherwise it wouldn't be possible to get the octave), but instead gets ()
. The next question is whether ly2video really needs to know the pitch, and if so, why.
from ly2video.
The next question is whether ly2video really needs to know the pitch, and if so, why.
If you search for "pitch" in https://github.com/aspiers/ly2video/blob/master/TROUBLE-SHOOTING.md you'll see that ly2video relies on pitch for ensuring the video frames are synchronised with the MIDI track. So this needs to work with drums too. However drum notes will still generate MIDI NoteOn
events with pitch. So if those pitches are not available as part of the LilyPond grob's pitch
property, they would need to be obtained from elsewhere. You should probably try using a normal non-drum voice just to confirm this is really the issue.
from ly2video.
Based on the documentation, NoteHead
doesn't have a pitch property or interface. I'd say the same thing for grobs. Where can I see that pitch is available via the grob?
How do the midi events align with multiple notes played at the same time that are not a chord?
from ly2video.
I think the pitch
comes from music event properties - this seems to be a big hint:
Line 1335 in baddaf4
https://lilypond.org/doc/v2.23/Documentation/internals-big-page#index-ly_003aevent_002dproperty
https://lilypond.org/doc/v2.23/Documentation/internals-big-page#music-properties
Can't remember how multiple notes are handled.
Note the check for pitch here:
Line 1336 in baddaf4
This could be reused to avoid crashing when pitch is missing.
from ly2video.
Still trying to decipher this code:
(format #t "\nly2video: (~23,16f, ~23,16f) pitch ~d:~a:~a @ ~23,16f from ~a:~3d:~d"
left right
(ly:pitch-octave pitch)
(ly:pitch-notename pitch)
(ly:pitch-alteration pitch)
(+ 0.0 (ly:moment-main time) (* (ly:moment-grace time) (/ 9 40)))
file line char))))
The first command is format. Does that make a formatted string? The arguments then are printed and reference another variable called pitch. That comes from
(pitch (ly:event-property cause 'pitch))
and it's described as:
Function: ly:event-property sev sym val
Get the property sym of stream event sev. If sym is undefined, return val or '() if val is not specified.
So this is a stream event. The object is cause
:
(cause (ly:grob-property grob 'cause))
The cause for a grob is:
cause (any type)
Any kind of causation objects (i.e., music, or perhaps translator) that was the cause for this grob.
It seems to me that maybe there needs to be a check on what the cause is. Somehow it needs to be a note event?
from ly2video.
I also came across this tidbit:
Function: expand-repeat-notes! music
Walk through music and give pitchless notes (not having a pitch in pitch or a drum type in drum-type) the pitch(es) from the predecessor note/chord if available.
Which may indicate that drum-type is a substitute for pitch. Unfortunately I don't know what to do with this information. If I change:
(pitch (ly:event-property cause 'pitch))
to:
(pitch (ly:event-property cause 'drum-type))
I get an encouraging error:
Drawing systems...dump-spacetime-info.ly:61:17: In procedure ly:pitch-octave in expression (ly:pitch-octave pitch):
dump-spacetime-info.ly:61:17: Wrong type argument in position 1 (expecting Pitch): ridecymbal
from ly2video.
This may seem like a stupid question, but the beatmap doesn't know about pitches. So why are pitches necessary for synchronization?
Plodding on,
Line 685 in baddaf4
If I patch cli.py to print out midiPitches
{51: midi.NoteOnEvent(tick=0, channel=9, data=[51, 90]), 36: midi.NoteOnEvent(tick=0, channel=9, data=[36, 90])}
Starting by hovering over the first grob
{51: midi.NoteOnEvent(tick=192, channel=9, data=[51, 90]), 36: midi.NoteOnEvent(tick=192, channel=9, data=[36, 90])}
Skipping grob and tick
{42: midi.NoteOnEvent(tick=384, channel=9, data=[42, 90])}
Skipping grob and tick
{42: midi.NoteOnEvent(tick=448, channel=9, data=[42, 90])}
Skipping grob and tick
{42: midi.NoteOnEvent(tick=512, channel=9, data=[42, 90])}
Skipping grob and tick
{42: midi.NoteOnEvent(tick=576, channel=9, data=[42, 90])}
Skipping grob and tick
{50: midi.NoteOnEvent(tick=768, channel=9, data=[50, 90])}
Skipping grob and tick
{50: midi.NoteOnEvent(tick=832, channel=9, data=[50, 90])}
Skipping grob and tick
{50: midi.NoteOnEvent(tick=896, channel=9, data=[50, 90])}
Skipping grob and tick
{50: midi.NoteOnEvent(tick=960, channel=9, data=[50, 90])}
Skipping grob and tick
{50: midi.NoteOnEvent(tick=1024, channel=9, data=[50, 90])}
Skipping grob and tick
{45: midi.NoteOnEvent(tick=1088, channel=9, data=[45, 90])}
Skipping grob and tick
{50: midi.NoteOnEvent(tick=1152, channel=9, data=[50, 90])}
Skipping grob and tick
{43: midi.NoteOnEvent(tick=1216, channel=9, data=[43, 90])}
Skipping grob and tick
{50: midi.NoteOnEvent(tick=1280, channel=9, data=[50, 90])}
Skipping grob and tick
{43: midi.NoteOnEvent(tick=1344, channel=9, data=[43, 90])}
Skipping grob and tick
sync points found: 1
from: 16 original indices
and: 17 original ticks
last tick used: 1
ticks skipped: 15
------------------------------------------------------------
So it would seem that If I can stuff those midi note numbers into grobPitchValue somehow, then it would get farther along.
The midi map is available in lilypond:
But I don't know how to access midiDrumPitches
. If this could be queried from dump-spacetime, the pitch could be grabbed right there. My scheme is not so good. The best I could figure out was:
(drum-type (ly:event-property cause 'drum-type))
(pitch (assoc drum-type midiDrumPitches))
but the output is:
(hightom . #<Pitch d >)
which isn't an integer. Do you know a way to convert the output of (assoc drum-type midiDrumPitches)
to an integer that's needed?
from ly2video.
Well I think I made it work and it was surprisingly easy:
(drum-type (ly:event-property cause 'drum-type))
(pitch (ly:assoc-get drum-type midiDrumPitches))
combined with my previous inclusion of the DrumVoice and DrumStaff contexts. This could be improved by detecting the drum-type and then conditionally fetching the note directly from the cause or from midiDrumPitches. Though a second mostly duplicate dump-spacetime-info
function for the DrumVoice context also works (and is what I did).
Now cli.py is happy and a video is generated. It mostly works, but as you can see there are some problems.
test.mp4
These are the same problems I saw running the tests under ly2video/test/regression.
input.mp4
Namely there's black space to the left of the score, the page is the wrong size, and the cursor doesn't go to the end.
from ly2video.
I've noticed that some other pull requests haven't been integrated yet. Both are related to some initial problems I had getting ly2video installed.
Both have merit. One makes the least amount of change, the other does away with having to install unnecessary packages. I'd say bring them both in. That way the github change to https gets captured, but then ly2video moves to less install dependencies which I'm in favour of.
I tested the change to mido + drum-support patches with all the regression tests. They all produced videos that seemed okay upon viewing (ignoring the problems I've already mentioned), but I'm not sure what the tests are supposed to look like. The text printed out to the screen is very noisy when running ly2video.
I also ran test.py which reported "Ok" after printing a bunch of text. I diffed the output of master vs mido + drum-support and the only difference was that the mido branch ran 0.008s faster.
I tested mido + drum-support on a full drum score and it worked as well as could be expected. There are some issues with the video as I mentioned previously, but it created a video where the red bar tracked all the notes (except for the last note).
Would you be able to review/merge those other pull requests?
And if you can spare some time to review my pull request, I'd like to see it integrated too.
from ly2video.
I'll take a look at those.
Did you try messing around with --width
and --height
? Might help fix the black space issue.
from ly2video.
I figured out some of the layout issues. One is that tagline
needs to be set to ##f
in the \header
of the LilyPond source to avoid that version footer from appearing. ly2video does this automatically, unless \header
is missing (see #30). However I added 5d78897 which I thought fixed this a different way, except now I can't get it to appear even when I revert that change.
Another is that the height defaults to 720px. It used to be that ly2video configured the paper top/bottom margins to 200px, so if your music was only (say) 150px high, the output image would be 550px which was too small to crop into an image with the desired 720px height. In this case ly2video would abort but offer a helpful error about how to use --dpi
or --height
to fix the problem. However #81 changed the top/bottom margins to be half the height which seems to guarantee this doesn't happen any more. I guess that's somewhat more user-friendly ... maybe.
The black space on the left is due to a similar reason, since left/right margins are still hard-coded to 200px. Setting --width
to a smaller amount fixes it, but it's a bug that it creates black space instead of white.
from ly2video.
Not all of the regression tests contain a \header
. The one I added for DrumVoice
does not because the regression test I used as an example also did not.
lilypond does not complain when there are duplicate '\header' blocks. The last one to appear will override any of those preceding. So it should be simple for ly2video to append a bit of text to the input file?
I'll play with --width
and --height
. The end application for me is to put scores on my phone with a metronome track (instead of midi) and it's an oddball size 1644 x 3840.
from ly2video.
Yes it could add a \header
but I'm not currently seeing any evidence that we need to. Anyway please let's discuss that in #30 not here.
from ly2video.
Related Issues (20)
- cli option --videoDef not used HOT 1
- --scroll-notes missing int conversion HOT 1
- New Release? HOT 2
- Error: Failed to parse ly2video line HOT 6
- Fonts can be less painful
- beatmaps
- Work better with Windows HOT 4
- Q: Getting Line Breaks into class TimeCode HOT 15
- Replace TiMidity++ with FluidSynth HOT 1
- ly2video.py referenced in readme is not included HOT 1
- PIL is dead and uninstallable from pip HOT 2
- ly2video, is it still usable? HOT 1
- Switch to modern Pillow version HOT 2
- Suggestion: Virtual keyboard HOT 1
- Video frozen for the last part HOT 4
- Dockerfile for ly2video HOT 1
- An AUR dependency does not exist HOT 1
- Support for alpha background HOT 1
- midi file extension not honored / misleading error message HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ly2video.