Cannot compute dep graph about cargo HOT 9 CLOSED

Thanks for the report @Suyashtnt, unfortunately this will take a bit of research and debugging to fix, which I haven't had the time for yet, and I can't make any promises on when I'll be able to get to it.

I no longer use NX, but for the people who do just take your time :)

from cargo.

Thanks for the report @Suyashtnt, unfortunately this will take a bit of research and debugging to fix, which I haven't had the time for yet, and I can't make any promises on when I'll be able to get to it.

from cargo.

@dannymcgee I would like to contribute a fix for this. I have isolated the problem. Would you be willing to accept a pull request?

from cargo.

@dannymcgee I made a PR fixing this.

from cargo.

@IgnisDa Thank you so much! I'll try to get this merged in today.

from cargo.

We are currently also running into this issue.
Liked mentioned in #21 the issue is that the output of cargo metadata is too large, it overflowed the buffer.

A minimal viable patch which works locally is taking this line maxBuffer: 50 * 1024 * 1024,
and adding it to the execSync call:

    let metadata = cp.execSync("cargo metadata --format-version=1", {
        encoding: "utf8",
        maxBuffer: 50 * 1024 * 1024,
    });

Another approach would be the usage of child_process.spawn.
See https://stackoverflow.com/a/68958420

from cargo.

@robi-wan I forked this repo and published the correct fix here: https://github.com/IgnisDa/npm-libs/tree/main/packages/nx-rust.

The solution you have posted will work but only for metadata below 50mb. For big projects, it would be more than that. It needs to be implemented using json streaming.

from cargo.

@IgnisDa Thank you for this info - a possible workaround for us would be to use your fork with that fix.

My understanding of Nodejs code is not very good but regarding my understanding the root cause for this issue is the limited buffer used in child_process.execSync: the default buffer size for stdout is 1024 * 1024 (see https://github.com/nodejs/node/blob/91ca2d40419244a9c542be5ff162a837fcc3be82/lib/child_process.js#L95).
It even is the default when using child_process.spawnSync without changes in handling I/O to the spawned process.

(In your fix you also used child_process.execSync with an extended buffer of 50 MB - so as I understand it your solution would also fail for big projects?)

You are right: streaming is the right approach but it should not just be used for JSON parsing. It should also be used for fetching the process output - I think therefore instead of child_process.execSync child_process.spawnSync should be used and something has to be done to redirect the stdout of cargo metadata --format-version=1 into a resizable buffer or a stream.

from cargo.

@robi-wan Indeed I have increased the buffer size, but then later I have also parsed the json via streaming. You can see them here https://github.com/IgnisDa/npm-libs/blob/main/packages/nx-rust/src/graph/index.ts#L23-L37.

I think therefore instead of child_process.execSync child_process.spawnSync should be used and something has to be done to redirect the stdout of cargo metadata --format-version=1 into a resizable buffer or a stream.

I am not sure about this. I will try to test this theory once I have time.

from cargo.