Code Monkey home page Code Monkey logo

Comments (14)

danielo515 avatar danielo515 commented on August 17, 2024 4

I usually rely on validation libraries to attempt some conversions (Joi for example does this), and return a possible errors and the converted value.
However, if the target of this library is to only check if a type is valid or not, I can totally understand your position. However, I had to open the issues and search for this particular one, if you are already positioned about this I think it will be nice to include it on the readme.

Regards

from runtypes.

MicahZoltu avatar MicahZoltu commented on August 17, 2024 3

I gave io-ts a try, but its interface is much more verbose and (IMO) harder to read than runtypes. This is the one major feature that would make runtypes totally awesome and give me no desire to look for any other options. Ideal interface would look something like a withConverter function:

const Apple = Record({
	color: Union(Literal('red'), Literal('green'), Literal('yellow'), // 'red' | 'green' | 'yellow'
	seeds: String.withConverter(x => Number.parseFloat(x)), // number
})

This way when I'm working with an API that gives me numbers as strings, or byte arrays as hex strings, or any other non-native JSON serialization I can just throw a withConverter on it. Presumably if withConverter throws, that would be caught by runtypes and wrapped in a helpful error that tells me what property failed to convert.

Would it be possible to build this as an another library so this library doesn't have to get into the business of decoding?

from runtypes.

pelotom avatar pelotom commented on August 17, 2024 2

It would be a departure from the invariant so far that runtypes never produce new values, they just validate existing ones. That said, I can't think of any specific reason that it's a bad idea.

However before you invest time in implementing it, I feel honor bound to mention that io-ts already has this feature, so it might be a better fit for your use case 🙂

from runtypes.

antoinewdg avatar antoinewdg commented on August 17, 2024 2

It would be a departure from the invariant so far that runtypes never produce new values, they just validate existing ones. That said, I can't think of any specific reason that it's a bad idea.

I completely agree, it would feel a little hacky given that this is not how it works everywhere else in the lib.

However before you invest time in implementing it, I feel honor bound to mention that io-ts already has this feature, so it might be a better fit for your use case

Thanks for mentioning it! I actually came here looking for an alternative to io-ts, and found I liked runtypes better (which is only my subjective opinion, io-ts also seems great).

Anyway, thanks for the awesome lib! :)

from runtypes.

WilliamABradley avatar WilliamABradley commented on August 17, 2024 2

I had a need for this, since I wanted to use the runtypes for data coming in from our request body, query, etc.

Since runtypes doesn't support type conversions out of the box, and I'm not a fan of uprooting all of the runtypes and switch to funtypes, yup, or introduce io-ts, I created support for type conversion using the reflection information of runtypes.

The only requirement is for a runtype to have a brand, so that it's type converter could be resolved properly, but that could be removed from this code.

Implementation:

/**
 * Explore the Runtypes reflection to find a type brand.
 * @param reflect type info to search for brand string in.
 * @returns brand string if found, otherwise, null.
 */
export function findTypeBrands(reflect: rt.Reflect): string[] | undefined {
  switch (reflect.tag) {
    case 'brand':
      return [reflect.brand];

    case 'optional':
      return findTypeBrands(reflect.underlying);

    case 'union': {
      const brands = reflect.alternatives
        .map((alt) => findTypeBrands(alt.reflect))
        .flat()
        .filter((val) => !!val) as string[];
      return brands.length > 0 ? brands : undefined;
    }

    case 'intersect': {
      const brands = reflect.intersectees
        .map((entry) => findTypeBrands(entry.reflect))
        .flat()
        .filter((val) => !!val) as string[];
      return brands.length > 0 ? brands : undefined;
    }

    case 'constraint':
      return findTypeBrands(reflect.underlying);

    default:
      return undefined;
  }
}

/**
 * Gets the array's element type from reflection, or undefined if not an array.
 * @param reflect array type to confirm that the type is an array.
 * @returns Array's element type or undefined if not an array.
 */
export function getArrayElementType(reflect: rt.Reflect): rt.Reflect | undefined {
  switch (reflect.tag) {
    case 'brand':
      return getArrayElementType(reflect.entity);

    case 'optional':
      return getArrayElementType(reflect.underlying);

    case 'union':
      // Only dig deeper if this is a nullable union.
      if (reflect.alternatives.length === 2 && reflect.alternatives[1] === rt.Null) {
        return getArrayElementType(reflect.alternatives[0].reflect);
      }
      return undefined;

    case 'constraint':
      return getArrayElementType(reflect.underlying);

    case 'array':
      return reflect.element;
  }
  return undefined;
}

/**
 * Gets the tuple's component types from reflection, or undefined if not a tuple.
 * @param reflect tuple type to confirm that the type is a tuple.
 * @returns Tuple's component types or undefined if not a tuple.
 */
export function getTupleComponentTypes(reflect: rt.Reflect): rt.Reflect[] | undefined {
  switch (reflect.tag) {
    case 'brand':
      return getTupleComponentTypes(reflect.entity);

    case 'optional':
      return getTupleComponentTypes(reflect.underlying);

    case 'union':
      // Only dig deeper if this in a nullable union.
      if (reflect.alternatives.length === 2 && reflect.alternatives[1] === rt.Null) {
        return getTupleComponentTypes(reflect.alternatives[0].reflect);
      }
      return undefined;

    case 'constraint':
      return getTupleComponentTypes(reflect.underlying);

    case 'tuple':
      return reflect.components;
  }
  return undefined;
}

/**
 * Gets the record's type from reflection, or undefined if not a record.
 * @param reflect record type to confirm that the type is a record.
 * @returns record type or undefined if not a record.
 */
export function getRecordType(reflect: rt.Reflect): (rt.Reflect & { tag: 'record' }) | undefined {
  switch (reflect.tag) {
    case 'brand':
      return getRecordType(reflect.entity);

    case 'optional':
      return getRecordType(reflect.underlying);

    case 'union':
      // Only dig deeper if this in a nullable union.
      if (reflect.alternatives.length === 2 && reflect.alternatives[1] === rt.Null) {
        return getRecordType(reflect.alternatives[0].reflect);
      }
      return undefined;

    case 'constraint':
      return getRecordType(reflect.underlying);

    case 'record':
      return reflect;
  }
  return undefined;
}

/**
 * Gets the insersectee types from reflection, or undefined if not an intersection.
 * @param reflect intersection type to confirm that the type is an intersection.
 * @returns intersectee types or undefined if not an intersection.
 */
export function getIntersecteeTypes(reflect: rt.Reflect): rt.Reflect[] | undefined {
  switch (reflect.tag) {
    case 'brand':
      return getIntersecteeTypes(reflect.entity);

    case 'optional':
      return getIntersecteeTypes(reflect.underlying);

    case 'union':
      // Only dig deeper if this in a nullable union.
      if (reflect.alternatives.length === 2 && reflect.alternatives[1] === rt.Null) {
        return getIntersecteeTypes(reflect.alternatives[0].reflect);
      }
      return undefined;

    case 'constraint':
      return getIntersecteeTypes(reflect.underlying);

    case 'intersect':
      return reflect.intersectees;
  }
  return undefined;
}

/**
 * Runtype type converter storage.
 */
const typeConverters: Record<
  string,
  {
    type: RuntypeBase<unknown>;
    convert: (data: unknown) => unknown;
  }
> = {};

/**
 * Register a runtype with a conversion function. (Must have a brand associated)
 * @param type Type to register for conversion.
 * @param converter Conversion function from untyped to typed.
 */
export function registerTypeConverter(type: RuntypeBase<unknown>, converter: (data: unknown) => unknown) {
  const brands = findTypeBrands(type.reflect);
  if (!brands) {
    throw new Error('Type must have a brand to be registered as a type converter');
  }

  const foundBrand = Object.keys(typeConverters).find((brand) => brands.includes(brand));
  if (foundBrand) {
    throw new Error(`A Type converter was already registered for this type: ${foundBrand}`);
  }

  for (const brand of brands) {
    typeConverters[brand] = {
      type,
      convert: converter,
    };
  }
}

/**
 * Parses a runtype, allowing for type conversions.
 * @param type RunType to parse.
 * @param data Data to parse to RunType
 * @param options Conversion options: {
 *  stripUnknown: "Removes fields that aren't on the type.",
 *  check: "Apply runtype check? Turn to false if you need a higher check, e.g. parsing deep records."
 * }
 * @returns Parsed RunType
 */
export function parseType<T extends RuntypeBase<unknown>>(
  type: T,
  data: unknown,
  { stripUnknown = true, check = true }: { stripUnknown?: boolean; check?: boolean } = {},
): rt.Static<RuntypeBase<T extends RuntypeBase<infer R> ? R : never>> {
  let value = data;

  // Ignore type warnings, used for chaining. Return value if type information couldn't be found.
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  if (!type) return value as any;

  const brands = findTypeBrands(type.reflect);
  if (brands) {
    for (const brand of brands) {
      const typeConvertInfo = typeConverters[brand];
      if (typeConvertInfo) {
        value = typeConvertInfo.convert(value);
      }
    }
  } else if (Array.isArray(value)) {
    const arrayElementType = getArrayElementType(type.reflect);
    const tupleTypes = getTupleComponentTypes(type.reflect);

    if (arrayElementType) {
      value = value.map((element) => parseType(arrayElementType, element, { check: false, stripUnknown }));
    } else if (tupleTypes) {
      value = value.map((element, index) => parseType(tupleTypes[index], element, { check: false, stripUnknown }));
    }
  } else if (value !== null && value !== undefined && typeof value === 'object') {
    const recordType = getRecordType(type.reflect);
    const intersecteeTypes = getIntersecteeTypes(type.reflect);

    // Handle intersections of types.
    if (intersecteeTypes) {
      const intersectResults: Record<string, unknown>[] = [];
      for (const intersecteeType of intersecteeTypes) {
        intersectResults.push(
          parseType(intersecteeType, value, { check: false, stripUnknown }) as Record<string, unknown>,
        );
      }
      // eslint-disable-next-line @typescript-eslint/no-explicit-any
      value = (_.merge as any)(...intersectResults);
    } else if (recordType) {
      const data = value as Record<string, unknown>;
      const dataKeys = Object.keys(data);
      const record: Record<string, unknown> = stripUnknown ? {} : data;

      for (const [field, fieldType] of Object.entries(recordType.fields)) {
        if (stripUnknown && !dataKeys.includes(field)) continue;
        record[field] = parseType(fieldType as RuntypeBase<unknown>, data[field], { check: false, stripUnknown });
      }

      value = record;
    }
  }

  // We essentially have done everything, if check is false,
  // then we essentially have "casted" the value.
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  return (check ? type.check(value) : value) as any;
}

Usage:

export const DateString = rt.String.withBrand('DateString').withConstraint((str) => {
  const dateObj = DateTime.fromISO(str);
  return dateObj.isValid || dateObj.invalidExplanation || `The date was not a valid ISO string: ${str}`;
});
export type DateString = rt.Static<typeof DateString>;

export const DateType = rt.InstanceOf(Date).withBrand('Date');
export type DateType = rt.Static<typeof DateType>;

registerTypeConverter(DateType, (value) => {
  if (typeof value === 'string') {
    value = new Date(DateString.check(value));
  }
  return value;
});

const nullableDateType = DateType.nullable().optional();

describe('parseType()', () => {
  const testType = rt.Record({
    date: nullableDateType,
  });

  it('converts values in a record type', () => {
    const source = {
      date: '2022-01-18T23:03:16.328Z',
    };

    expect(parseType(testType, source)).toEqual({
      date: new Date(source.date),
    });
  });
});

from runtypes.

brandonkal avatar brandonkal commented on August 17, 2024 1

This would be interesting for string => number conversion and 'false' => false conversion.

from runtypes.

MicahZoltu avatar MicahZoltu commented on August 17, 2024 1

We should probably move this conversation over to funtypes repository. Feel free to @mention me if you create a thread over there! (I believe that you may be misunderstanding the rename of check to parse, but I don't want to derail this thread/repository since it is a funtypes thing)

from runtypes.

yuhr avatar yuhr commented on August 17, 2024

I gave io-ts a try, but its interface is much more verbose and (IMO) harder to read than runtypes. This is the one major feature that would make runtypes totally awesome and give me no desire to look for any other options.

Totally agree 🤣

I'm very interested in this feature and would love to start implementing.

from runtypes.

MicahZoltu avatar MicahZoltu commented on August 17, 2024

@yuhr funtypes (a fork of runtypes) has this feature.

from runtypes.

yuhr avatar yuhr commented on August 17, 2024

Got it, here's that. It's excellent effort, but deprecating check in favor of parse makes me feel a bit pointless, and I don't understand why ParsedValue is taking duplicate logic such as parse and test, rather than taking just parse and leaving test to Constraint? Could someone explain these points?

from runtypes.

yuhr avatar yuhr commented on August 17, 2024

Let's compare, funtypes looks like this:

const TrimmedString = ParsedValue(String, {
  name: 'TrimmedString',
  parse(value) {
    return { success: true, value: value.trim() };
  },
  test: String.withConstraint(
    value =>
      value.trim() === value || `Expected the string to be trimmed, but this one has whitespace`,
  ),
});

TrimmedString.safeParse(' foo bar ')
// => 'foo bar'

With #191, the equivalent code in runtypes will look like this:

const TrimmedString = String.withTransform(value => value.trim(), { name: 'TrimmedString' });
TrimmedString.check(' foo bar ');
// => 'foo bar'

const AlreadyTrimmedString = String.withConstraint(
  value =>
    value.trim() === value || `Expected the string to be trimmed, but this one has whitespace`,
);

As illustrated above, I think the feature like test in funtypes (it guards the input is already conforming to TrimmedString) should be considered as a different matter than transformation. The same applies to inverse transformation (i.e. serialize in funtypes), obviously it can sufficiently achieved by having another runtype dedicated to perform inverse transformation.

from runtypes.

MicahZoltu avatar MicahZoltu commented on August 17, 2024

You don't have to use a custom parser. In your example, it looks like you don't want a custom parser, you just want a constraint.

const TrimmedString = String.withConstraint(x => x.trim() === value)
TrimmedString.parse(' foo bar ') // throws error
TrimmedString.safeParse(' foo bar ') // returns error (IIRC, I never use this one so I forget how it surfaces errors exactly)

from runtypes.

yuhr avatar yuhr commented on August 17, 2024

I couldn't get the point of your comment. Apparently your code has no "transform", so it does not work the same way as mine. What I tried to illustrate by my example above is nothing else but the equivalent code corresponding to funtypes' code example. What I meant by the previous post is, I think we should follow the "separation of concerns" principle here.

By the way, the equivalent code to yours in runtypes should look like:

const TrimmedString = String.withConstraint(x => x.trim() === value)
TrimmedString.check(' foo bar ') // throws error
TrimmedString.validate(' foo bar ') // returns error

Or, if I reuse my example code, I can write it also:

AlreadyTrimmedString.check(' foo bar ') // throws error
AlreadyTrimmedString.validate(' foo bar ') // returns error

All these are the concern of "conformity after trimming", not the concern of "transformation of values".

from runtypes.

the-spyke avatar the-spyke commented on August 17, 2024

I understand that having only a single responsibility is good, but knowing where these type checks are used, transformation is a very often second step. Maybe this could be done by a separate types Codec/Decoder/Encoder that can add encoding/decoding by wrapping provided Runtype.

from runtypes.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.