Create basic tests for the netlist converter.

Unsure what happens when this occurs, does the entire type come into scope, an anonymous type, or what.
Need to assess this before able to implement it.

One case of this (left parenthesis) was "fixed" in issue #31, where state links were changed so that a left parenthesis could be followed by a semicolon without whitespace between them.
This was possible without any addition of states since I don't think ")" is followed by anything but an operator or whitespace (IE cannot be followed immediately by a value or word).
Other cases may need a similar fix which would allow a semicolon to follow them, currently can think of:

• "
• '
  End of string/char should probably be allowed to immediately follow with a semicolon, however the apostrophe symbol can be followed by an identifier for attributes, so a new state will probably be required.

Unsure if this is a bug or an enhancement.

Due to the size of the language specification only a subset of the language will be initially implemented in from this stage.
As the parser was/is being written the language was analysed to get a better picture of this.
The decisions made need to be described for future reference while writing code for this section.

Add regexes to lexer to match known error cases.

Should probably include required software - which is just stack I believe - along with how to install it.
Maybe other useful information.
Probably good to have a description of the project and how to run/use it.

When an operator touches anything but whitespace, an error occurs.
An example of this is the (3); in the following ghci log:

lexerList "package name is constant name : integer := name (3); end;"
Left Some lexer error occurred

When converting from string to Int64 or Double there is a loss of precision.
Noticed when converting value 15:D45C0b1d78213acb619852B539: initially, massive loss in precision.

Method:

• Incrementally converted strings 15:9:, 15:39:, 15:539:
• Compared values to calculator
• Error occurs between 15:13acb619852B539: and 15:213acb619852B539:

This site seems to match calculator for accuracy.

Diagnosis:

• Integer type should be unbounded (value should be converted in integer form before multiplied by exponent
• Double could potentially have loss of precision however these values tested aren't Universal_Real they are Universal_Int which suggests conversion to Double isn't likely to be a problem here
• Int64 is the current implementation for Universal_Int, this is bounded (tested by maxBound :: Int64) at 9223372036854775807.

Guessing this has something to do with the Int64 type and should start testing there.

Create basic tests for the parser.

For better configuration and clarity, potentially should refer to Haskell built in types in one file and refer to this when making custom types.

Haskell needs a constant type to parse abstract literals

• Currently, read each component of the literal (units, decimals, exponent) and calculate result
• This is causing problems with inputs such as:
  • "0.07E2"
  • Obvious this is 7, but would anyone write it like this
  • Is this actually a bug or just an enhancement/non-issue
• Could potentially, at least for decimal literals, use exponent to shift value before calculating for more accurate results

Currently using Data.Map.Strict: link which states:
"When deciding if this is the correct data structure to use, consider:

• If you are using Int keys, you will get much better performance for most operations using Data.IntMap.Strict.
• If you don't care about ordering, consider use Data.HashMap.Strict from the unordered-containers package instead."

Should consider optimal container type to use:

• Lazy maps (probably not needed)
• Unordered containers
• IntMap

Additional details

Use of maps when building scope tree could cause out-of-order scope evaluation, which leads to errors when scope names overwrite each other.

• Currently overwriting scope objects with new ones if there is an overlap
• This will cause unreliable scopes where it is indeterminate as to what object ends up in the scope
• Should error when scope overlaps
• Without correct scope ordering errors could point to wrong position
• Should probably also change from using maps for scope
  Need to evaluate maps ordering and assess if this is an issue

Map out and write more extensive tests for the netlist converter.

These add ambiguity to the parse tree and probably slow it down. Also they're making it harder to trace parser bugs. If there is time it's best to modify the tree accordingly, to make parsing the tree and converting to a netlist easier.

Currently in Parser.PositionWrapper but can use in netlister errors, so should move to move general location.

Should centralise text modification and use functions wherever duplications of strings occur, to remove errors from typos and allow easier changes.

Current implementation supports three libraries:

• Std: Cannot write files in this library, has two built in libraries standard and textio
• Work: Where the user's files should go, choose directory it points to with --work-dir
• IEEE: Special library that should be provided by the IEEE, choose directory these are in with --ieee-dir

Currently implemented with two flags, could come up with some method (flags or config file) to allow custom other libraries.

Having improved the simulation to use fewer monads, could probably remove libraries from the simulation configuration to reduce compilation time (and stop stack installing the library).

• Potentially remove mtl and transformers libraries
• Update documentation accordingly

Implement basic version of the lexer.

Elements to lex:

• Keywords
• Operators
• Identifier
• Based value literals
• Decimal value literals
• Bit string literals
• String literals
• Character literals

In the top level of a parse, IE the highest level statements in the file:

• Context clauses (library and use statements) cannot appear in the file without an accompanying library unit (entity, package, etc.)
• This comes from the nature of the parse BNF
• Should it be acceptable to have context clauses without a library unit
  • It will parse to nothing, but currently doesn't provide a useful error
  • At the very least should have useful error message

When there is no sensitivity list, what rules should exist for wait statements in a process, must there be at least one in the top level?

VHDL only parses ISO 646-1983 characters (according to 1076-1987).
It may be benefitial to change the input type to the parser to restrict this accordingly.

Map out and write more extensive tests for the lexer.

May need to check if a name already exists in the netlist before creating a new netlist object (entity, architecture, package, etc.)
Possibly already checking this, need to confirm whether this is in place or not.

Position information is held by the lexer however isn't stored with tokens.
This should be provided with the tokens for useful errors in the parser.

Implement basic version of the parser.

Can a right parenthesis be followed by another operator or can there be a space after it?

Theoretically (however unlikely), the singleIdentifier tests could randomly generate a string that is actually a valid keyword.
This would cause the test to fail, since the lexer should lex the string to a keyword instead of an identifier, but the test would assume it to be an identifier.

Need to add check that valid keyword hasn't been randomly generated.

Incredibly unlikely to happen and will be obvious if it does, so low priority.

Starting with commit f8c0728 - the parser is being refactored for better usability and complete testing before release.

Using this issue to track changes to the parser and add elements to be considered.

Currently using state and IO monad stack, failing using IO monad, perhaps add EitherT to deal with errors universally in a single place.

How to detect which file an entity/package/configuration/etc.
Potentially a "simple" method would be to grep for patterns, in order to estimate which files to parse

• Can call grep with this module
• Would need to handle special cases
  • Declarations over multiple lines
  • Files that appear to be the correct file but have invalid syntax
  • Multiple files returned
    Alternatively could search with Haskell which may be slower but more portable, would share problem from other method, if a file has invalid syntax.

Haddock documentation comments for:

• Lexer
• Parser

Documentation files for:

• Lexer
• Parser

      String literals
        Valid string literals:                            FAIL (0.50s)
          *** Failed! Falsifiable (after 98 tests):
          Input: ""\"\"""; Output: """"
          Use --quickcheck-replay=702380 to reproduce.

While working with commit df48afc

Strings are simply a list of characters, which is supposedly an inefficient means of implementation.
At the profiling stage this should be assessed to see if this causes issues.
Potentially instead use the improved Data.Text module for large amounts of text processing/IO.

• Profile program
• Research string vs text efficiency/speed statistic (there is some information on Data.Text page)
• If necessary convert Strings to Data.Text where applicable

Since identifiers and other similar constructs are case insensitive, they are compared in upper case in the code. However they are implemented with strings that include all other characters, even if invalid, which removes some of the type safety and means I have to check that all names have been transformed.
Perhaps useful to create custom type (perhaps enum) to deal with this? Not sure if this is less efficient than string.

Currently using lexer position wrapper in the parser, this simply points to a position in a file, however does not say which file.
Should create a new position wrapper type for the parser that includes a file name for improved error messages.

Map out and write more extensive tests for the parser.

To use functionality in Control.Monad.Except, need version 2.2.2 of mtl module, which is not yet in stack snapshot.
Have added it to extra-deps in stack.yaml which causes warnings when compiling:

"No packages found in snapshot which provide a "happy" executable, which is a build-tool dependency of "VHDL-Sim"
Missing build-tools may be caused by dependencies of the build-tool being overridden by extra-deps.
This should be fixed soon - see this issue commercialhaskell/stack#595

No packages found in snapshot which provide a "happy" executable, which is a build-tool dependency of "VHDL-Sim"
Missing build-tools may be caused by dependencies of the build-tool being overridden by extra-deps.
This should be fixed soon - see this issue commercialhaskell/stack#595"

Should monitor this and perhaps remove mtl-2.2.2 from extra-deps once it has been added to stack resolver.

Replicate with:
--quickcheck-replay '97 TFGenR 71920A1F02D7267784152133E91E3D1EA4E3F0A88B7AAC4BABFD6607BC92BBC9 0 7 3 0'

Create basic tests for the lexer.

Lexical elements to test:

• Keywords
• Operators
• Identifier
• Based value literals
• Decimal value literals
• Bit string literals
• String literals
• Character literals

Add pattern matching to track errors in the parser and provide useful error messages.

Currently parser messages are not implemented (aside from GenericParserError, which provides no helpful information).

A first stage solution for this is to implement a fail-on-error system using the current monadic parser.

• Within the parser, the monad is accessible from the parser productions using {% ... } (type Alex a)
• Perhaps even better, the lookhead token can be passed into the production with {%^ ... } (type Token -> Alex a)
• Implement useful error messages by displaying what was expected, and what was actually obtained

While the simulation automatically jumps from time value to time value based on what is to be done, the timescale could be adjusted for the wave output.

• Currently outputs in femtoseconds no matter what (base unit of TIME)
• Could run over all wait statements and find smallest denomination of time used
• Use that in simulation output for better clarity
• Maybe add an option to force a timescale?

Implement basic version of the netlist converter.

General stages:

• Implement first statement conversion to netlist. This will provide context to the task, help realise issues with design that haven't been thought of
• Implement expressions
• Implement remaining statements from chosen subset (relies on #8 )

Need to evaluate previously scoped modules when scoping new module to detect cyclic dependencies

• A depends on B depends on A
  Otherwise infinite loop occurs, probably stack overflow.

This module is currently used in two locations, library and executable, however only the type is used in both, the majority of the module is for passing/processing arguments to the executable.

Should this file be split and some placed into the executable section IE app rather than src - the part in question is a data structure so shouldn't require testing via the cabal test-suite.

See build #4
Input: "OF\241\ENQ"

The lexer/parser fails on an error, displaying some message.
For errors that can be ignored, the parser should should store the offending item and continue to the best of its ability, returning a list of errors at the end of a parse.

Needs to have:

• Some method of tracking that at least one error has occurred, and avoid continuing to the next stage (may not be possible between lexer and parser since tokens from the lexer immediately are processed by the parser)
• Keeping the list of failed tokens, potentially state monad

Having removed the complicated monads from the simulation output, should check the control module Sim.Builtin.ControlModule and the documentation in docs/haskell/Simulation-Output.md for how up to date they are.