right now lydown recognizes a stream switch only without any whitespace:

=lyrics

while

= lyrics

is considered illegal and leads to a mess. This should be accepted as valid syntax.

Create an abstract lilypond API, a single function that accepts lilypond source code as input, along with options, and optionally returns the compiled output, or simply writes to the output files and returns their names. For example:

Lilypond.compile(ly_source, format: :pdf, target_path: "BWV247-01-choro")

Currently the compiler swallows warnings. Instead, they should be parsed and lydown should be able to find the offending part (usually these warnings have to do with unterminated ties, beams or slurs).

In movement.ld:

- parts:
  - alto:
    - source: soprano // sources both music and lyrics from soprano
  - violino:
    - music_source: soprano // sources music only from soprano

To set a midi tempo that is hidden, with lilypond:

\set Score.tempoHideNote = ##t
\tempo 4 = 96

Lydown:

\tempo:(4=96)

Proposed syntax:

0 stands for breve, l for longa

Measure number indicators are used signify the absolute position of anything that comes after the indicator in terms of the referenced measure. Lydown should automatically add rests (for actual parts) or silences (for the global context - see #37).

An example:

- time: 4/4
- key: D
(20): || // this will put a double bar line at the beginning of measure 20
(35): \fine // this puts a _fine_ indication at the beginning of measure 35...
(37:3): \fine // this puts a _fine_ indication on the third beat of measure 37
(end): \dacapoalfine // this puts a _da capo al fine_ indication at the end of the piece

The (end) measure indicator requires lydown to go over the other parts in the same movement and to calculate the maximum number of measures.

When invoked in a movement directory, lydown should automatically process the work.ld file of the parent directory, and treat the movement name as an -m parameter.

Same thing when the specified path given to lydown is a movement. For example:

BWV247$ lydown 01-choro

Should be equivalent to

BWV247$ lydown -m 01-choro

See also issue #2.

In order to faciliate the implementation of global contexts (#37), with or without measure number indicators (#38), lydown should support full bar silences, similar to full bar rests. The syntax is as follows:

S // a single full bar silence
S*20 // full bar silence for 20 meaures

This should be converted into the following valid lilypond code:

s1 // assuming 4/4 time
S1*20

Lilypond spews the following error:

error: markup outside of text script or \lyricmode

  \markup { \smallCaps { Flute I } } 

When run without any parameters, the lydown command should start an interactive session:

BWV247 sharon$ lydown
BWV247 ♬ score // compile the score for BWV247 and display in browser
BWV247 ♬ cd 01-choro
01-choro ♬ cat viola:53 // show line 53 of viola part
53> dr6re,6.f3g+8a+r4r
01-choro ♬ edit viola:53 // start vim on viola part at line 53
...
01-choro ♬ e flute1:23 // e is shorthand for edit
...
01-choro ♬ viola:53-54 // compile viola part bars 53-54 and open it
01-choro ♬ viola:53- // compile viola part bars 53 to end and open it

use -dbackend=svg

Single digit time signatures

For certain pieces, we would like to display the numerator only. With lilypond:

\once \override Staff.TimeSignature.style = #'single-digit
\time 5/4

With lydown:

\time:5/(4)

Hidden time signatures

Lilypond:

\once \override Staff.TimeSignature #'break-visibility = ##(#f #f #f) 
% or:
\once \override Staff.TimeSignature #'stencil = ##f
\time 5/4

Lydown:

\time:(5/4)

No time signature before line break

Lilypond:

\once \override Staff.TimeSignature #'break-visibility = ##(#f #t #t) 
\time 5/4

Lydown:

\time:5/4!

4/4 time signature shorthand

\time:c

2/2 time signature shorthand

\time:c/

• layout:
  • paper_size: A5 landscape
  • top_margin: 1.4cm / 2in
  • bottom_margin: 1.4cm
  • inner_margin: 1.4cm // implies two-sided = ##t
  • outer_margin: 1.4cm // implies two-sided = ##t
  • left_margin: 1.4cm // implies two-sided = ##f
  • right_margin: 1.4cm // implies two-sided = ##f
  • staff_size: 14 // default: 20
  • indent: 0 // first system indent
  • vertical_align: justify/ragged/ragged_last // ragged-bottom, ragged-last-bottom settings (default is ragged_last)
  • horizontal_align: justify/ragged/ragged_last // ragged-right, ragged-last (default is justify)

In order to maximize the DRY principle, we could use work.ld or movement.ld files to provide a global music stream that includes the key and time signatures (with partial/pickup definitions), key and time signature changes in the middle of the piece, tempo changes, double bar lines, repetitions, etc.

In the rendered lilypond code, a global stream can be incorporated as follows:

ldGlobalMusic = {
  ...
}

\new Staff <<
  \ldGlobalMusic
  { \ldFluteMusic }
>>

The lydown code will look as follows, for example a movement.ld file:

- key: D
- time: 3/4
- pickup: 4
- tempo: Largo
S*20 // full bar silences (this also needs a separate issue)
\tempo: Allegro \time:4/4 ||

See also #38 for measure number indicators.

In movement.ld:

- score:
  - page_break: before
- parts:
  - gamba1:
    - page_break: before

• instrument names using small caps
• thicker stems
• thicker slurs
• curvier slurs
• Better font for lyrics
• Nice inner/outer margins
• Zero indent for first system, instrument names inline
• note_head_style: baroque. See here
• introduce some variability into setting by using a tiny amount of randomness.

When specifying --midi or -f midi

A markup section before a movement:

=intro
...
...

A markup section after a movement:

=outro

Objectives:

• Make the rendering code / templates simpler and easier to change and enhance.
• Make the rendered lilypond code reusable - repetition, transposition, etc.
• Allow such features as repeat music with different lyrics.
• Make the rendered lilypond code simpler to read.
• Facilitate such features as inline free text markup, table of contents, additional stanzas after music, etc.
• Facilitate engraving of score and individual parts.
• Allow output modes such as only variables, only music, only lyrics, output to separate files, etc.

Planning:

This will require a significant change to the current codebase. Whereas a lilypond file generated by lydown looks like a giant nested web, the new file structure will as follows:

• lilypond version
• include commands for including standard libs.
• a list of variable definitions for each stream: music, lyrics, figures.
• any markup, table of contents, preface, etc.
• any layout/paper settings.
• the proper score structure, referencing the music variables.

The idea is to be able to install libs & tools related to lilypond, and use them without too much hassle.

• Dependencies for a project are declared in a filename called Lypadfile
• packages are Ruby gems, downloadable using the gem tool.
• package names are prefixed with lypad-. That way we can tell which gems are lypad-related. For example, a package called lilyjazz will be published as a gem called lypad-lilyjazz.
• system is ruby based.
• lypad command line tool is basically a wrapper around gem / bundler.
• lypad also provides some lib tools for stuff like installing fonts, or changing the stock lilypond installation in any way.

When Lypad is installed it modifies lilypond to add a few lilypond commands used to reference and load packages.

Loading packages in lilypond sources

\require "lilyjazz" % that's all folks

Command line interface

lypad update # updates the lypad installation and installs hooks in the current lilypond installation
lypad install # installs anything in Lypadfile
lypad install <gem> # installs the gem
lypad install <gem> -s # installs the gem and writes it to Lypadfile
lypad uninstall <gem>
lypad list # lists all lilypond-related gems

When processing a directory, things can be parallelized:

• Parsing individual files can be performed by a thread pool. Generating an AST and event stream for each file can be completely parallelized. The actual generation of lilypond code can be serialized using a critical section or some other lock.
• This actually should be relatively simple:
  • Work#process_directory starts a worker thread pool and adds each encountered file to the worker queue.
  • Workers pull file names from the worker queue and process them using process_lydown_file.
  • Work#process_lydown_file uses a critical section or other lock to synchronize the last stage of processing, the actual translation to lilypond code is serialized.

Add option to exclude part from score. Suppose there's a part that's there only for part extraction, not for being rendered as part of the score.

- parts:
  - tenore_without_turbas:
    - render_modes: part

Where the default is part, score.

The code will be rendered only when the rendering mode fits corresponds to the specified mode:

\score // everything after this directive will be rendered only in score mode
\part // everything after this directive will be rendered only in part mode
\all // everything after this directive will be rendered in any mode

(See #33)

General features

• Text is sort of markdown based: Free flowing paragraphs, support for italic and bold

• back-slash-prefixed commands to control appearance

• line breaks, page break are explicit

• provide some way to control horizontal and vertical whitespace

  =intro
  \centered Passio Domini nostri\n
  _J. C. secundum evangelistam Marcum\n
  \n
  BWV 247\n
  ...

  =outro
  \n:30 // 30 times new line
  Sources:\n
  ...

Currently full bar rests for 4/2 is rendered incorrectly and makes lilypond freak out.

Currently settings are all over the place, with no real logic to the way settings are written to the context or read from the context according to circumstances. This leads to many problems (see #25, #26 among others) that are hard to solve.

The idea is to provide two new methods on WorkContext: #get_setting and #set_setting, that will interrogate the settings in part files, movement files, work files, and finally the lydown defaults (stored in lib/lydown/defaults.yml).

This would avoid some of the confusion with relative mode, and facilitate reading and writing the source code.

The following should raise a parsing error:

4c ~ d

Also the following:

b- ~ b

And also the following:

c ~ // tie at end of file

Lilypond syntax:

\override Stem.thickness = #5.0
\once \override Stem.thickness = #5.0

Lydown current command syntax:

\override:"Stem.thickness = #5.0"

New lydown override syntax:

\\Stem.thickness:5.0

Override once:

\\!Stem.thickness:5.0

Support for true/false literals (##t / ##f), strings (#"..."), numbers (#5 / #5.0) etc.

Example:

<#><7_> // the dièse should hold.

Proposed syntax:

c<->dec<.>

Right now:

8re\"_Evangelist:_" eb // explicit italic

Proposed syntax (various options):

8r ::Evangelist:: eeb // italic implied
8r :Evangelist: eeb // italic implied
8r \"_Evangelist:_" eeb // explicit italic
8r :Evangelist: eeb
8r :{Evangelist} eeb

• stem_weight:
• slur_weight:
• staff_color:
• note_color:
• etc.

Use the -include setting to include arbitrary lilypond files:

- include: abc.ly
- include: ../def.ly

The included files will be included at the beginning of a movement.

Including only in score mode:

- score:
  - include: ...

Including only in parts mode:

- parts:
  - include: ...

Including at top of lilypond doc:

- document:
  - include: ...

Best approach would be to reject anything with a colon. Test case:

c\key:D

It turns out numbers and other symbols can be used in variable names, provided they are quoted.

So, instead of doing the whole dance of converting numbers to roman numbers, capitalizing, upcasing etc, we use a much simpler variable naming scheme, which reflects the stream path in the work context:

"01-choro/soprano/music" = { ... }
"01-choro/soprano/lyrics/..." = { ... }
"01-choro/violino1/music" = { ... }

% usage:
... \"01-choro/violino1/music" ...

Recitative mode implies:
- instrument_names: hide
- empty_staves: hide except continuo

Translation mode: show progress bar for translation
Compile mode: show progress bar for parsing, and then progress bar for compiling
Proof mode: progress bar for compiling

In silent mode, show nothing except errors

When a big project is being processed, the parsing and rendering of hundreds of files can take quite a bit of time, even before lilypond starts compiling the rendered code. For example, BWV 247 takes over 20 seconds to parse and render, and even that after being parallelized (see #5).

A way to improve this state of affairs is to cache the results of each step, using temp files.

For parsing, it's rather simple:

1. The source lydown file is read.
2. A hash for the content is calculated.
3. Lydown searches for a corresponding temp file (e.g. .parse)
4. If a temp file is not found, the content is parsed into a lydown event stream, then written to the temp file using MsgPack encoding.
5. If the temp file is found, its content is parsed using MsgPack.

For rendering, it's a bit more complex, since we need to take into account the state of the work context, so:

1. The hash value for the work context is calculated => h1.
2. A hash for the lydown event stream is calculated => h2.
3. Search for corresponding temp file:

   .

   .render.

4. If the temp file is not found, the stream is rendered, then written to the temp file using MsgPack encoding.
5. If the temp file is found, its content is parsed using MsgPack.

Setting time:unmetered doesn't insert the appropriate \cadenzaOn command into the rendered lilypond code.

In interactive mode, we can zone in on different places in the score:

lydown
BWV247:) // this is the prompt
BWV247:) cd 01-choro // enter into a movement directory
01-choro:) flute1:37 // show measure 37 of flute1 part
  flute:37: 4fa'~{6._3_}afe+e6c-a8b
01-choro:) edit flute1:37 // open file in editor and select specified line 

Parts should be able to source from other parts. For example, an oboe part that plays with the soprano in a choral, or a cello part that plays the continuo part, but without the figures.

Two ways to define part sources. The first:

- parts:
  - oboe1:
    - source: soprano
    - lyrics: hide
  - flute1:
    - source: violino1
  - violoncello:
    - source: continuo
    - figures: hide

The other way is define colla parte settings, which invert the setting:

- colla_parte:
  - soprano: oboe1, oboe2 // oboe1 & oboe2 double the soprano part
  - continuo: violoncello // the violoncello doubles the continuo part

• Usage: lydown translate 01-choro
  The above command will translate the entire subdirectory, leaving the role files in place and creating new lydown files in the same directory
• Translation code will reside in lib/lydown/translation/ripple
• Parsing using custom treetop rules
• Look for macros in _work.yml, _movement.yml files
• Parse relative command to determine octave of first note.
• Ignore curly brackets
• Parse and translate note events: note head, duration, expressions
• Parse key signature and translate chromatic note names into diatonic ones
• Parse and translate duration macros
• Look for lyrics files and as them to output under a =lyrics section

The cues should appear only in part mode.

Lilypond:

<<
    { e'2\rest r4. e8 }
    \new CueVoice {
      \stemUp d'8^"flute" c d e fis2
    }
  >>

Lydown:

\cue:flute 8d'cde2f+ \cueWhile 2r4.r8e \cueEnd

Or maybe:

\cue:flute:{{8d'cde2f+}} 2r4.r8e

In that case, the system automatically matches the cue length to the music after, in order to create the corresponding nested lilypond structures.

Another option, in addition to the \cue command, is a quote command:

\quote:flute1 2r4.r \unquote, 8e

which translates to lilypond:

ldViolinoIMusic = { ... <>^\markup \tiny { \smallCaps { Flute I } } \cueDuring #"ldFluteIQuote" #UP { r2 r4.} e8 ... }
ldFluteIMusic = { ... }
\addQuote "ldFluteIQuote" { \ldFluteIMusic }

(Some kind of registry for quoted voices, so we know to add \addQuote commands)

Also, ability to provide custom titles for cue:

\quote:tenore:"_Evangelist:_" 2r4.r \unquote, 8e

Lilypond:

ldViolinoIMusic = { ... <>^\markup \tiny { \italic { Evangelist: } } \cueDuring #"ldFluteIQuote" #UP { r2 r4.} e8 ... }
ldFluteIMusic = { ... }
\addQuote "ldFluteIQuote" { \ldFluteIMusic }

Also, ability to cue with clef:

\quoteWithClef:treble_8:tenore:"_Evangelist:_" 2r4.r \unquote, 8e

Lilypond:

ldViolinoIMusic = { ... <>^\markup \tiny { \italic { Evangelist: } } \cueDuringWithClef #"ldFluteIQuote" #UP #"treble_8" { r2 r4.} e8 ... }
ldFluteIMusic = { ... }
\addQuote "ldFluteIQuote" { \ldFluteIMusic }

The vertical position of the cue on the staff can also be controlled using the vertical position specifiers:

\quote:tenore ... \unquote // default is on top of staff music (UP)
\_quote:tenore ... \unquote // on bottom - under staff music (DOWN)
\^quote:tenore ... \unquote // on top - like the default (UP)

Add --separate which causes each movement to be compiled separately, perhaps in parallel (see #5):

lydown -s --separate

When compiling parts, each part is also compiled separately.

A setting command can be easily attached to a note by mistake, by omission of whitespace:

c\key:D

A simple heuristic to prevent this would be to reject any note expressions containing a colon after the expression name.

Add a new bar_numbers setting. When set to hide, the following is rendered:

\layout {
\context {
    \Score
    \remove "Bar_number_engraver"
  }
}