paralleldrive / cuid Goto Github PK

Please add support for web workers.

cuid() currently throws an exception in web workers since navigator.mimeTypes is not available in that context.

Generally, for the fingerprint I suggest to collect various strings as a whole, gracefully handling errors, and then create a fingerprint out of that (arbitrary length) string using some hash function - like MD5, but perhaps something more lightweight.

Something in this direction:

api.fingerprint = function browserPrint() {

  var things = [];

  try {
    for (var i = 0; i < navigator.mimeTypes.length; i++) {
      things.push(navigator.mimeTypes[i]);
    }
  } catch (e) {
    // ignore errors
  }

  try {
    things.push(navigator.userAgent);
  } catch (e) {
    // ignore errors
  }

  // maybe add some more...

  things.push(api.globalCount());

  return someHashFunction(things.join(""));
};

I think you get the idea...

Due to the c{timestamp}... structure, I have concerns about "todays" items creating a hot spot in a distributed data store for my use-case (long history of items, todays items are used heavily, using aws DynamoDB).

Thoughts on the validity of my concern?

I got the following error.

When the process too fast, it generate duplicate key.

MongoError: E11000 duplicate key error index

Either JavaScript Pseudo random is not good.

I also think Timestamp have a problem too by looking at the key.

Let say, node is running with cluster and more then 10 thread. It is highly possible that both Timestamp and Pseudo random will be the same with high hit.

A way to let user add just one digit will be good enough I guess.

I was looking through this project since it's a dependency for something else I was checking out.

I noticed that you are using Math.random(). This is fine but if so you shouldn't claim that the ids are unguessable:

Cuids contain enough random data and moving parts as to make guessing another id based on an existing id practically impossible.

Since Math.random() is not cryptographically secure it's easy to predict the next values. I would remove the two security/secure paragraphs completely and add a warning instead.

EDIT: Just for fun I wrote a proof of concept here that can predict all CUIDs(excluding timestamp portion) after seeing four consecutive ids. It's for node v4/v5 only since they've updated their RNG algorithm but the approach for later versions is similar.

https://usecuid.org/ uses Github's SSL certificate, which is only valid for *.github.io, not custom domains. Unfortunately, Github doesn't allow custom certificates, but you can switch to Netlify (which is free and should work as a drop-in replacement here).

Because the popular bundling tools may not bundle stuff from node_modules our package.json may need to point to bundled versions of the library, instead of individual modules. I've tried doing this with Browserify, but then Webpack gives weird warnings to all the users of our module (I think -- it used to, anyway).

We could try to accomplish this with rollup.

For this to be complete we need to test:

• npm install from the npm tar bundle
• Build and use in Webpack
• Build and use with Browserify
• Build and use with Rollup

cc @MarkHerhold @gmaclennan

Eric, you write that the sequential nature of cuid makes for better performance when used as primary keys in databases. I'm currently trying to choose what id concept to use and the more I read the more I understand that this is hard.

It seems that both the placement of the timestamp part of the uuid AND big/little endian concept of different database engines and OSes matters. Do you have any plans to make cuid more compatible with different RDBMs by making the timestamp style tweakable?

Maybe some interesting reading: https://github.com/richardtallent/RT.Comb

I'm sorry to say that I don't have the competence to be able to PR something. I stumbled over cuid while researching. I have always held you high for competence and innovation in the Javascript community, and was hoping that "this is it, this could solve my id needs" :)

Applitude is defunct. It's past time to remove references to it from Cuid.

• Docs
• Tests

... If they're generated on different browsers or machines, and then inserted to the database later, won't they be in some random order? Am I totally missing the point? (I think so, sorry about that!) This looks really cool, but I'm struggling to understand the bit about them being sequential. If they were only sequential because they were being generated on ONE machine, then why wouldn't you just use an auto-increment int or bigint?

The Travis build is currently failing. Looks like a broken package dependency.

Is there any way to validate a cuid? For example to distinguish it from a random string?

The documentation states that IDs generated through cuid.slug() weigh in at 8 characters, however the ones I'm seeing are 7 characters. Is this expected behaviour? Does the documentation mean they will be up to 8 characters?

Hi,

I have to generate unique and not guessable api keys.
Those keys will give user full access to the application via REST API.

Was wondering if using cuid is safe enough, and what are the benefits over using something like

https://www.npmjs.com/package/hat

or

https://www.npmjs.com/package/node-uuid

thanks

See the prod-module-boilerplate package.json

• Fix the build so it passes Travis..
• Add scripts to package.json
• Remove unused dependencies (no more grunt!)
• Add new dependencies
• Update .travis.yml example

Please complete #24 first.

Stop using the hacky concat build.

• Use ES6 modules
• Drop applitude support (it's obsolete)
• Use separate client/server build entry points and share common features between them.

.
└── cuid
    ├── build
    │   ├── client
    │   └── server
    ├── source
    │   ├── client
    │   └── server
    └── test
        ├── client
        └── server

Ignore build directory in diffs

You can ignore build files in diffs.

Create a .gitattributes file in the repo root:

build    -diff

In slug() impl, the c value is not checked to be in the discrete values range like it is in cuid(). c is then directly incremented after usage.

c = (c < discreteValues) ? c : 0;

Intensive use of slug without any calls to fingerprint could lead to overflow.

Didn't update any package or system environment, but today when I call cuid() I get

TypeError: process.pid is undefined
nodePrint
[path to my react component]
cuid
webpack:///./~/cuid/dist/node-cuid.js?:49:23

Using
cuid: 1.3.8
webpack: 2.2.0
webpack-dev-server: 2.2.0
node v6.9.4
npm v3.10.10
yarn 0.20.3

Tried on [email protected], got the same result.

Hey,

I liked your cuid project so I ported it to Ruby, you can see it here: http://github.com/iyshannon/cuid.

Please check it out and let me know what you think, I haven't published it to RubyGems yet as I wanted to get your OK before doing so.

Thanks,
Ian

Hi everyone,

I'm wondering what is the dev status of this library, is it still actively maintained?
Is it safe to bet on it for a new Node project?

Thanks for your answer :)

The JS version of cuid needs to support all of these targets:

• Node
• Browsers
• WebWorkers
• ReactNative

We'll use feature detection to select the correct entropy source and return the right fingerprint() function for the environment.

Ideally, we should select entropy sources supported by all of the above environments.

For reasons related to a problem in a CUID 1.3.8 dependency, I'm very excited for the jump to 2.0.0

Any word on when it lands in NPM? Any API changes to know about?

Thanks!

The latest release/tag and npm package, v1.3.0, has in package.json "main": "./build/server/cuid.js", but includes no folder build.

The package.json also still references the dist folder, which was included in v1.2.4, but is not in v1.3.0.

Would it be possible to allow the cuid to be seeded by some arbitrary value? For example, I want to generate a cuid based on a uuid that is completely random. Does that make sense?

How do you feel about using getRandomValues from the Web Crypto API in the environments that support it for randomBlock?

Is there a regular expression to validate CUIDs?

Hi,

If you are interested in mentioning it in your README file, I have ported this to Go. The package is not 100% finished but will be pretty soon.

http://github.com/lucsky/cuid

Thanks for this lib :)

Would be great to have a way to customize the starting character.
It could be given as a parameter to the cuid() function. The "c" could be the default if not given...

If customizable, one could create specific classes of IDs that would allow some backend decisions based on the type of ID received...

Would you consider moving away from jshint/eslint to use Standard JavaScript instead?

The standalone built versions (./dist) of cuid currently do only one of the two typical checks for CommonJS based amd. See:
else if (typeof module !== 'undefined') { module.exports = api; } -- Line 104 dist/browser-cuid.js

when compared to the standard practice of:
if ( typeof module === "object" && typeof module.exports === "object" ) {

This caused a bit of a headache recently as another dependency (angular-mocks) when run through Karma/Jasmine actually stores a function in window.module. I wasn't sure the proper procedure for submitting a fix as it looks like the ./dist versions are not built through the npm scripts.

Thanks

"Cuid maintained by..."

Hi, node-uuid author here. I just had this module brought to my attention and thought we might compare notes. I haven't done a thorough investigation, but it seems like cuid is pretty similar to node-uuid's RFC4122 v1 implementation.

cuid timestamp & cuid counter

RFC4122 uses a timestamp system that includes both the timestamp and counter concepts. The RFC timestamp interval is 100-nanoseconds, but in environments that lack this resolution, clients can use a counter of ids generated during the current timestamp interval to emulate the higher precision, which is what node-uuid does.

In terms of how these values contribute to uniqueness, it's about the same; the RFC timestamp is 64 bits.
The cuid timestamp + counter is ~62 bits

cuid fingerprint & cuid random

cuid attempts to generate a per-client unique id in the form of fingerprint. In cuid (browser) this is built from these values:

• userAgent.length - Only unique to browser version + OS
• mimeTypes.length - Only unique to browser (+ plugins? I'm not sure what contributes to this honestly)
• (paraphrasing) Object.keys(window).length - Only unique to browser + application runtime

... or in cuid (node):

• process id: 0 - 1295, with clustering, if not outright duplication, in production environments where hosts all have same boot process.
• checksum(hostname): 0 - 2500 (clustering around average hostname checksum, possibly clustering in environents where hosts are sequentially named - e.g. checksum is same for "host03", "host12", "host21", and "host30")

For the reasons noted above, I'm concerned about how unique this "fingerprint" actually is in production.

node-cuid uses Math.random(), which is (by definition) an RNG of uncertain quality.

Meanwhile in the RFC/node-uuid ...

The node property should be either the host MAC address (unique to a fairly high degree of certainty) if available, or set randomly if not. In node-uuid it's set randomly.

node-uuid only uses Math.random() as the RNG of last resort, preferring higher-quality crypto implementations where available.

The RFC also includes a 14-bit clockseq value which is initialized randomly, and which increments any time there is a discontinuity detected in the system clock (e.g. if the clock regresses backward). This is similar to cuid's random value, with the added benefit of preventing duplicates if something happens to the system clock, which is a feature I don't see in node-cuid.

In terms of how these contribute to uniqueness, the cuid fingerprint + cuid random is ~ 62 bits (21 + 41 bits. Meanwhile the RFC node + clockseq is also 62 bits (48 + 14 bits).

Summary

node-cuid offers strikingly similar features to RFC4122 v1 UUIDs, just in a slightly different form factor. While the node-cuid IDs are somewhat shorter (25 chars vs. 40 chars), that's simply a presentation choice; (e.g. converting from RFC notation to cuid notation and back is fairly trivial). In fact, I'm curious why node-cuid doesn't use base-62, for shorter-still ids, since that's one of the selling points.

More importantly, I don't see anything in cuid that suggests it is better at avoiding collisions. In fact, the fingerprint issues, the reliance on Math.random(), and lack of clock regression detection logic suggest that it will be worse, not better, than an RFC compliant implementation.

(FWIW, I've spent a lot of time looking at the RFC trying to see if there's a way it might be improved. It's actually a very well thought-out spec, which is why I decided to focus on a rigorous implementation for node-uuid. The one aspect of it I'm not a fan of is the choice of gregorian epoch + 100-nanosecond increments for the timestamp field. I think the spec would have been better served with a standard unix epoch + 1 second increments, with an explicit counter field, akin to what cuid has to handle id sequencing with a given timestamp unit. But the end result is more or less the same, so it's not that big a deal.

Anyhow, hope this is useful. And please do point out anywhere I may have misunderstood what you're doing.

Cheers!

• Sauce Labs is free for Open Source
• You should make the Sauce Labs task use environment variables.

You can create a bin/env.sh file in a bin dir and add it to .gitignore:

!/bin/bash
set KEY=value

Be sure to include a bin/env.sh.example file and check that one into git.

Then as part of the scripts:

  "test:browsers": "source bin/<filename>.sh"

Checklist

• Create bin/env.sh
• Create bin/env.sh.example
• Update package.json
• Add bin/env.sh instructions to README

Is there any specific reason (along making it a bit longer than absolutely necessary) why the timestamp does not contain some padding (beginning zeros)... if it would, I think it could be used as a time order criteria on standard ASCII sorting

I installed the package with component install, after the dist folder is just empty. Used this component a lot and it seems like it doesn't work anymore for some days.

module.exports = (function() {

I try the following. only slug(), and fingerprint return 5 chars.

How do I get longer one?

Regards,

var cuid = require('cuid');
var id = new cuid();

return function(req, res, next) {

console.log('cuid module');

//console.log(cuid);
//console.log(id);

//console.log(cuid.fingerprint());  // short, 5 chars
//console.log(cuid.slug());   // work, short one
console.log(cuid.cuid());
//console.log(cuid.nodePrint());

console.log(id.cuid());
//console.log(id.fingerprint());
//console.log(id.slug());   // work, short one
//console.log(id.nodePrint());

next();

}

})();

Can I get the timestamp from the generated id?

Hello, I love what you have here. So, I have port it to PHP. Cheers.

Keep up the good work.

Regards,
Endy

I'm using cuid for logging stuff into a document database in objects like this:

{
  id: cuid(),
  timestamp: new Date.getTime(),
  type: 'content-specific',
  content: {...some_data}
}

If I could somehow safely reconstruct the timestamp back from id, I could omit having to store the timestamp in the database thus saving myself a lot of space.

Any option to add such feature?

I could being doing something really stupid (as is the case normally) but I can't get my tests to pass in Node with cuid.

error trying to resolve DateInputViewModel navigator is not defined ReferenceError: navigator is not d
efined

If I dig a little further, it looks like cuid is expecting the window.navigator to exist... but obviously this is not the case when running in a Node process. I am using Testem's launchers to run this particular suite of tests, and perhaps that is where the error lies. Maybe Testem isn't setting an environment variable correctly and therefore some kind of cuid check of if process.env === node is failing?

Any ideas would be appreciated.

More of a comment here for other react-native users that might run into the same issue, than a bug specifically, but because react-native doesn't provide a userAgent or mimeTypes by default, using cuid will throw an error.

This can be worked around by defining dummy values before requiring cuid;

global.navigator.mimeTypes = ''; //browser-fingerprint only checks the length property so an empty string is fine
global.navigator.userAgent = 'reactnative';

I did try using the node version but rn also doesn't set a process.pid so runs into the same issue.

Hi,

Is cuid will always return a length 25 string ?
Need to know for setting VARCHAR length.
Couldn't find an exact answer about it.

Thanks !

wouldn't it make sense to store fingerprint in a variable and only run it once when needed?

Hey! @ericelliott I know there's already a PHP version of the package but I wanted a literal import of the node version. When you have time can you take a look at it? https://github.com/Shoperti/Cuid Thanks :)

Hi,

Was wondering what your thoughts were on the potential of spoofed Id's originating from untrusted clients.

One of the things that appeals to me about cuids are the fact that id's are more or less sequential (assuming consistent time sources), which means I can use them as PK's in my DB and won't get burnt on inserts (due to the rows being stored.as a clustered index),

This does however leave an unsavory taste in my mouth - as an untrusted client could screw with my DB index by generating unique, but non sequential id's.

I was thinking of performing some sanity checking server side on any new id's generated client side. Nothing too restrictive, I would probably just check the basic format is correct and that the timestamp fits within a suitably large window. I then noticed this in the README:

"The cuid specification should not be considered an API contract. Code that relies on the groupings as laid out here should be considered brittle and not be used in production."

Do you believe this applies in the stated context?

Regards
Rob

padStart/padEnd are stage 3. It looks like they have a great chance of making it into ES2017.

The official polyfill is here:
https://www.npmjs.com/package/string.prototype.padstart