Problem with string example formatted as timestamp even in quotes about powershell-yaml HOT 10 CLOSED

Hi Will,

This one does not hate a quick fix. Code changes will have to be made. I think it's time to implement tags. Bandwidth is quite limited, so I have no ETA on this. But this type of issue has popped up quite a bit, so it needs to be done.

I will attempt to do it in a backwards compatible way, so as not to break any existing software.

from powershell-yaml.

Adding [^"] seems to stop the date string from being converted to a date time stamp, however I'm not sure if this would break anything for anyone else.

I've tried running the tests now and discovered if nothing else, it does break the tests. Pester test is written to expect the date provided as yyyy-mm-dd to be converted to a date time stamp format.

  Describing Being able to decode an externally provided string.

    Context Decoding an arbitrary YAML string correctly.
      [-] Should decode the YAML string as expected. 1.17s
        Expected the value to have type [datetime] or any of its subtypes, but got '2002-12-14' with type [string].
        276:                     $res['dates'][$idx] | Should -BeOfType ([datetime])
        at <ScriptBlock>, C:\Sources\Non-PRS-Repos\powershell-yaml\Tests\powershell-yaml.Tests.ps1: line 276

from powershell-yaml.

Yes. This module was written a while back with a limited set of needs in mind. It was never meant to be a full fledged parser, but at this point it's worth spending more time to bring it up to par with what people expect form an actual parser.

I will try to allocate some time. For deserialization it should be easy to implement tags. It will also remove a lot of guesswork in the module. Round-tripping will be more interesting. Without a custom type to hold metadata about the values, it becomes a bit of a challenge. If we swap the standard powershell types with PScustomObjects or something else, applications that currently use it and update, will break. So it requires a bit of thought, but it's clear that tags are a must.

from powershell-yaml.

As a hack, I did wonder if I could escape the hyphens, in the date format in such a way they would be re-written when it's converted back to YAML, but I guess it's a bit dangerous to try to do it that way? Maybe in the convertfrom-yaml it might replace the escaped hyphens anyway.

Is the code changing to datetime because of the match on the string or because of the format date? I assume with the regex it's purely based on the string match. I guess when it's converted from YAML the date is converted to a string in quotes anyway before the value is converted to proper type, meaning you can't differentiate between a pure date or a pure date in quotes as a string. Unless you escape a second set of quotes?

from powershell-yaml.

To get you going, the easiest would be to do this:

diff --git a/powershell-yaml.psm1 b/powershell-yaml.psm1
index a278374..54ac7ce 100644
--- a/powershell-yaml.psm1
+++ b/powershell-yaml.psm1
@@ -81,10 +81,12 @@ function Convert-ValueToProperType {
  (\.[0-9]*)? # (fraction)
  (([ \t]*)Z|[-+][0-9][0-9]?(:[0-9][0-9])?)? # (time zone)
 '@
-        if([Text.RegularExpressions.Regex]::IsMatch($Node.Value, $regex, [Text.RegularExpressions.RegexOptions]::IgnorePatternWhitespace) ) {
-            [DateTime]$datetime = [DateTime]::MinValue
-            if( ([DateTime]::TryParse($Node.Value,[ref]$datetime)) ) {
-                return $datetime
+        if ([string]::IsNullOrEmpty($Node.Tag) -eq $true -or $Node.Tag -eq "tag:yaml.org,2002:timestamp") {
+            if([Text.RegularExpressions.Regex]::IsMatch($Node.Value, $regex, [Text.RegularExpressions.RegexOptions]::IgnorePatternWhitespace) ) {
+                [DateTime]$datetime = [DateTime]::MinValue
+                if( ([DateTime]::TryParse($Node.Value,[ref]$datetime)) ) {
+                    return $datetime
+                }
             }
         }
 

Then you can define your Yaml as follows:

---
workRegisteredDate:
  type: string
  format: date
  example: !!str 1991-01-07

The result will be:

PS /tmp/powershell-yaml> Import-Module powershell-yaml                
PS /tmp/powershell-yaml> $y = ConvertFrom-Yaml (gc -Raw /tmp/sample.yml)
PS /tmp/powershell-yaml> $y["workRegisteredDate"]["example"].GetType()  
                  
IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     String                                   System.Object

Without a tag, you still get the date conversion, which is expected, given the above hack:

---
workRegisteredDate:
  type: string
  format: date
  example: 1991-01-07

PS /tmp/powershell-yaml> Import-Module powershell-yaml                
PS /tmp/powershell-yaml> $y = ConvertFrom-Yaml (gc -Raw /tmp/sample.yml)
PS /tmp/powershell-yaml> $y["workRegisteredDate"]["example"].GetType()
                
IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     DateTime                                 System.ValueType

from powershell-yaml.

Is the code changing to datetime because of the match on the string or because of the format date? I assume with the regex it's purely based on the string match. I guess when it's converted from YAML the date is converted to a string in quotes anyway before the value is converted to proper type, meaning you can't differentiate between a pure date or a pure date in quotes as a string. Unless you escape a second set of quotes?

It uses the regex defined in the spec, at: https://yaml.org/type/timestamp.html

from powershell-yaml.

To get you going, the easiest would be to do this:

diff --git a/powershell-yaml.psm1 b/powershell-yaml.psm1
index a278374..54ac7ce 100644
--- a/powershell-yaml.psm1
+++ b/powershell-yaml.psm1
@@ -81,10 +81,12 @@ function Convert-ValueToProperType {
  (\.[0-9]*)? # (fraction)
  (([ \t]*)Z|[-+][0-9][0-9]?(:[0-9][0-9])?)? # (time zone)
 '@
-        if([Text.RegularExpressions.Regex]::IsMatch($Node.Value, $regex, [Text.RegularExpressions.RegexOptions]::IgnorePatternWhitespace) ) {
-            [DateTime]$datetime = [DateTime]::MinValue
-            if( ([DateTime]::TryParse($Node.Value,[ref]$datetime)) ) {
-                return $datetime
+        if ([string]::IsNullOrEmpty($Node.Tag) -eq $true -or $Node.Tag -eq "tag:yaml.org,2002:timestamp") {
+            if([Text.RegularExpressions.Regex]::IsMatch($Node.Value, $regex, [Text.RegularExpressions.RegexOptions]::IgnorePatternWhitespace) ) {
+                [DateTime]$datetime = [DateTime]::MinValue
+                if( ([DateTime]::TryParse($Node.Value,[ref]$datetime)) ) {
+                    return $datetime
+                }
             }
         }
 

Then you can define your Yaml as follows:

---
workRegisteredDate:
  type: string
  format: date
  example: !!str 1991-01-07

Ahhh . . this has helped me to understand what you were talking about with regards to enabling tag support. So this is essentially to enable tagging and then use the tag for string to avoid formatting as a datetime. Is that right?

BTW thanks for your work on this module and for the quick response to my logged issue here!

from powershell-yaml.

Tagging is the preferred way to annotate the desired type for a value inside your yaml. powershell-yaml in its current form, tries to guess the type of the value when deserializing. It also tries to disambiguate the values when serializing back to yaml. If the type of the value in powershell is string, then powershell-yaml will automatically quote the value. Yaml itself does not require quotes for strings, but in the absence of tags, everything is a string (theoretically), even integers, floats, dates, etc.

So this is essentially to enable tagging and then use the tag for string to avoid formatting as a datetime. Is that right?

That is correct. It's a temporary hack that only applies to timestamps/dates.

For the next major version of this module, I would like to enable round-tripping, which will correctly handle all tags both when deserializing as well as when serializing.

I just need to find the time to do it.

from powershell-yaml.

Thank-you Gabriel, It's a great module! The date issue is the only quirk I've had to work around. I've used a different hack (that I think is simpler) to avoid this issue.

Nodes (i.e. the yaml values) also have a style tag as either plain, 'single quoted' or "double quoted". If a date representation is enclosed in single or double quotes, I'm happy to accept the quotes indicate it should be treated as a string. That's an easy If statement to add...

Line 84: (original) function - Convert-ValueToProperType

  if([Text.RegularExpressions.Regex]::IsMatch($Node.Value, $regex, [Text.RegularExpressions.RegexOptions]::IgnorePatternWhitespace) ) {
        [DateTime]$datetime = [DateTime]::MinValue
        if( ([DateTime]::TryParse($Node.Value,[ref]$datetime)) ) {
            return $datetime
        }
    }

Updated... - added a final if statement... if the date sequence is enclosed in single or doube quotes (which is the style property), return the string value and not DateTime:

    if([Text.RegularExpressions.Regex]::IsMatch($Node.Value, $regex, [Text.RegularExpressions.RegexOptions]::IgnorePatternWhitespace) ) {
        [DateTime]$datetime = [DateTime]::MinValue
        if( ([DateTime]::TryParse($Node.Value,[ref]$datetime)) ) {
            if ($Node.Style -in 'DoubleQuoted','SingleQuoted') {
                return $Node.Value
            }
            else {
                return $datetime
            }
        }
    }

By enclosing the date in quotes means it's always converted from yaml as a string.

from powershell-yaml.

Hi @LaurieRhodes

Long overdue, but this was somewhat fixed in #100 . Timestamps are now only converted if the !!timestamp tag is used, otherwise, dates are returned as a string.

from powershell-yaml.