Reproduced on Samsung J3 Android 9

This command fails
https://github.com/bamlab/android-performance-profiler/blob/main/packages/android-performance-profiler/src/commands/cpu/getCpuClockTick.ts#L6
100 is a good default value but it seems like for some phones, this is completely innacurate (for a Moto e5 this seemed off)
Same for https://github.com/bamlab/android-performance-profiler/blob/main/packages/android-performance-profiler/src/commands/ram/pollRamUsage.ts#L6

The issue is that getconf doesn't seem to exist on all Android versions.

To fix this, running sysconf(_SC_CLK_TCK) and sysconf(_SC_PAGESIZE) via a C++ script should retrieve the proper values

We could include this in the C++ profiler script like in here: #12

Flashlight should offer more stats guidance

Say you have the comparison below, what to do with it:

Sure score increases, but it'd be nice to see:

See what increased and what decreased

• Add in the Report summary card for comparisons the increase/decrease

For instance in this case:

• Average FPS: 54.5 (+1.7%)
• Average CPU: 55.7% (-2.8%)
  ...

We also need:

• figure out how we handle comparisons with more than 2 results
• add this in thread tables as well

For later:

• define a threshold on what constitute a danger or not
• a clear recommendation with an icon/emoji for instance ✅ ❌🟰

Standard deviation in a single run ✅

Done in #178

Standard deviation in the comparison

The point here is to be able to say :

• does my app consistently perform better or worse through each iteration

For instance, Flashlight should be able to tell you:
average CPU usage is down 5% and that is consistent through all iterations +/- 0.5%

By default, scrollDown will always pick the first scrollview.

We could select via "instance index" like so

  const scrollDown = ({ instanceIndex, steps }) => {
    const selector = `new UiScrollable(new UiSelector().scrollable(true).instance(${instanceIndex})).scrollForward(${steps})`;
    return driver.runUIAutomatorCommand(selector);
  };

but it'd be nice if we could just scrollDown a scrollView we have selected by id for instance:

EDIT: actually this works:

`new UiScrollable(new UiSelector().resourceId("android:id/list")).scrollForward(${steps})`

I've used https://github.com/semantic-release/semantic-release in the past for instance

Hello 👋🏻 ,

Problem

So I have a weird behaviour using the library, sometimes everything works perfectly and sometimes it doesn't. Please check the following image in order to see the NaN error.

This error only appears when Im running more than 1 iteration

Here's the json of the failed

I just launched 3 times the test with different iterations:

• First launch = 1 iteration
• Second launch = 2 iterations
• Third launch = 4 iterations

Code test example

I am running this simple test using appium(2.0.0-beta.47) with javascript.

describe(`Performance in ${country.countryCode}`, () => {
  beforeEach(async () => {
    await checkAppStatus();
  });
  it('Just Click in login button', async () => {
    await HomeScreen.goToLogin();
    await new Promise(resolve => setTimeout(resolve, 5000));
  });
});

Scenario

Running on a MacBook Pro 13-inch, M1, 2020 | macOS: Ventura 13.0.1
Launched using a real device connected via USB - Xiaomi Redmi Note 9 S

I'm using version 0.3.0 on MAC

After profiling and trying to write the report I get the following exception.

14:33:02] ℹ️ Installing C++ profiler for arm64-v8a architecture
[14:33:02] ✅ C++ Profiler installed in /data/local/tmp/BAMPerfProfiler
[14:33:14] ℹ️ Writing report...

/snapshot/flashlight-serverless/node_modules/yoga-layout-prebuilt/yoga-layout/build/Release/nbind.js:53
throw ex;
^

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined
at Object.openSync (node:fs:577:10)
at Object.openSync (pkg/prelude/bootstrap.js:796:32)
at Object.writeFileSync (node:fs:2170:35)
at writeReport (/snapshot/flashlight-serverless/node_modules/@perf-profiler/e2e/dist/writeReport.js)
at /snapshot/flashlight-serverless/node_modules/@perf-profiler/e2e/dist/commands/measure/bin.js
at Generator.next ()
at fulfilled (/snapshot/flashlight-serverless/node_modules/@perf-profiler/e2e/dist/commands/measure/bin.js) {
code: 'ERR_INVALID_ARG_TYPE'
}

For some reason I can't install the android-performance-profiler plugin. After reloading Flipper it still shows me that the plugin is not installed.

I'm using Flipper v0.180.0. Tried with v0.176.0 too.

PS: rn-perf-monitor Also is not working with RN version: 0.70.6

I see your example that your compare in the report between two session
how can I do like this

Tnanks

If both results have same name, the thread stats are the same in the comparing view

It is found that memory monitoring is less accurate than Android studio profiler memory monitoring?

Hello,

I've seen your video in AndroidMakers, it made me want to know more about flashlight and integrate it in my CI. I have a couple questions regarding the tests.

• If I have 2 devices (one is an emulator, the other is a real device) they have the same specifications in terms of RAM, CPU and android version. Will the result be the same ?
• If I were to add flashlight to my CI, should I use your cloud or create an emulator which will run the test ?

Description:

After running this package with a release version of an app, I noticed that the FPS was always at 60. I downloaded the CSV from the generated web report and results in that CSV were also showing 60 FPS. So I debugged the calculation of FPS and I saw a comment:

But in my test cases automation is done to scroll the FlashList which means the list won't be idle and the FPS generation in such case seems faulty. I also tried to log the frameTimes and it had 2 items only once. In every consequent calculations this frameTimes array is empty. Which makes the frameCount = 0 and we go to the assumption of FPS being 60, which is not precise.

Test case:

import { AppiumDriver } from "@bam.tech/appium-helper";
import { TestCase, measurePerformance } from "@perf-profiler/e2e";

const bundleId = "com.some.id";

const getTestCases = async () => {
  const driver = await AppiumDriver.create({
    appPackage: bundleId,
    appActivity: `${bundleId}.MainActivity`,
  });

  const scrollTestCase: TestCase = {
    beforeTest: async () => {
      driver.stopApp();
      driver.startApp();
      await driver.clickElementByText("someButton1");
      await driver.clickElementByText("someButton2");
    },
    run: async () => {
      await driver.wait(2000)
      for (let index = 0; index < 5; index++) {
          await driver.scrollDown(2);
      }

      await driver.clickElementById("someButton3");
    },
  };

  return {
    SCROLL: scrollTestCase,
  };
};

test("e2e", async () => {
  const testCases = await getTestCases();

  const { writeResults } = await measurePerformance(
    bundleId,
    testCases.SCROLL,
    1
  );
  writeResults();
});

Logs Output:

 console.log frameTimes  [ 50, 32.29 ]

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  2

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  54.12520000000001

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

 console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

 console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

  console.log frameTimes  []

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:45:11)
      at Array.forEach (<anonymous>)

  console.log frameCount  0

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:46:11)
      at Array.forEach (<anonymous>)

  console.log final FPS:  60

  at processOutput (../android-performance-profiler/src/commands/gfxInfo/pollFpsUsage.ts:63:11)
      at Array.forEach (<anonymous>)

 console.log [19:13:46] ✅  Finished iteration 1/1 in 10028.164417028427ms

  at log (../logger/index.ts:25:11)

Now this 2 items concluding to 54 FPS is not present in the CSV and also not reflected on the web page report.

CSV:

Report:

Let me know if I can help with anything else in explaining 🙂

Hello everyone ,
When I try to follow the steps on my organization's laptop I get this permission error but it works fine On my laptop. I need to know what to enable exactly in order to have it work as IT removed some restrictions and It still gives this error.

[12:54:07] ℹ️ Appium capabilities: {"platformName":"Android","appium:automationName":"UiAutomator
2","appPackage":"com.moj","appium:noReset":true,"autoLaunch":false,"appActivity":"com.moj.MainActivity","newCommandTimeout":600000}
 architecture Installing C++ profiler for x86
[12:54:08] ✅ C++ Profiler installed in /data/local/tmp/BAMPerfProfiler
Error: Command failed: adb shell /data/local/tmp/BAMPerfProfiler printCpuClockTick
/system/bin/sh: /data/local/tmp/BAMPerfProfiler: can't execute: Permission denied
    at checkExecSyncError (node:child_process:861:11)
    at execSync (node:child_process:932:15)
    at executeCommand (C:\Users\ahabd\Documents\EjadaProjects\MOJ\node_modules\@perf-profiler\profiler\src\commands\shell.ts:6:20)
    at retrieveCpuClockTick (C:\Users\ahabd\Documents\EjadaProjects\MOJ\node_modules\@perf-profiler\profiler\src\commands\cppProfiler.ts:82:19)
    at ensureCppProfilerIsInstalled (C:\Users\ahabd\Documents\EjadaProjects\MOJ\node_modules\@perf-profiler\profiler\src\commands\cppProfiler.ts:70:5)
    at new PerformanceTester (C:\Users\ahabd\Documents\EjadaProjects\MOJ\node_modules\@perf-profiler\e2e\src\index.ts:23:33)
    at C:\Users\ahabd\Documents\EjadaProjects\MOJ\node_modules\@perf-profiler\e2e\src\index.ts:101:18
    at Generator.next (<anonymous>)
    at C:\Users\ahabd\Documents\EjadaProjects\MOJ\node_modules\@perf-profiler\e2e\dist\index.js:8:71 {
  status: 126,
  signal: null,
  output: [
    null,
    Buffer(0) [Uint8Array] [],
    Buffer(83) [Uint8Array] [
       47, 115, 121, 115, 116, 101, 109, 47, 98, 105, 110, 47,
      115, 104, 58, 32, 47, 100, 97, 116, 97, 47, 108, 111,
       99, 97, 108, 47, 116, 109, 112, 47, 66, 65, 77, 80,
      101, 114, 102, 80, 114, 111, 102, 105, 108, 101, 114, 58,
       32, 99, 97, 110, 39, 116, 32, 101, 120, 101, 99, 117,
      116, 101, 58, 32, 80, 101, 114, 109, 105, 115, 115, 105,
      111, 110, 32, 100, 101, 110, 105, 101, 100, 13, 10
    ]
  ],
  pid: 16248,
  stdout: Buffer(0) [Uint8Array] [],
  stderr: Buffer(83) [Uint8Array] [
     47, 115, 121, 115, 116, 101, 109, 47, 98, 105, 110, 47,
    115, 104, 58, 32, 47, 100, 97, 116, 97, 47, 108, 111,
     99, 97, 108, 47, 116, 109, 112, 47, 66, 65, 77, 80,
    101, 114, 102, 80, 114, 111, 102, 105, 108, 101, 114, 58,
     32, 99, 97, 110, 39, 116, 32, 101, 120, 101, 99, 117,
    116, 101, 58, 32, 80, 101, 114, 109, 105, 115, 115, 105,
    111, 110, 32, 100, 101, 110, 105, 101, 100, 13, 10
  ]
}

getting this issue when trying to start measuring

It would seem that Android < 7 is not supported by the tool

We could add in https://github.com/bamlab/android-performance-profiler/blob/main/packages/android-performance-profiler/src/commands/cppProfiler.ts in ensureCppProfilerInstalled:

const sdkVersion = parseInt(executeCommand("adb shell getprop ro.build.version.sdk"), 10);

if (sdkVersion < ...) throw new Error("...")

Why

Flipper is not widely used in the Android community, and even RN is starting to move away from it.

Flashlight doesn't use any Flipper capabilities except for the fact that as it is an electron app, it can run node.js CLI commands.

Proposal

A new flashlight ... command that would

• run the web app similar to what we have in Flipper UI
• start a local server so that the web app can send commands such as "start measuring" / "stop measuring"

Benefits

• Easier to develop locally
• Flipper plugin was already tricky to maintain and released in parallel to the Flashlight CLI
• can easily pass settings to the CLI command

Other considered alternatives

• Running as web app only makes it tricky (or impossible) to run adb commands
• VSCode extension would not make much sense since Flashlight measures are not really related to code

when I run the demo of Usage with Maestro

error thrown

Error: Invalid line: 28825 (AWCN Worker(M)1) S 367 367 0 0 -1 1077936192 15 0 0 0 0 0 0 0 20 0 31 0 12492622 16141717504 35945 18446744073709551615 1 1 0 0 0 0 4608 1 1073775868 0 0 0 -1 3 0 0 0 0 0 0 0 0 0 0 0 0 0

/snapshot/flashlight-cloud/flashlight-open-source/packages/platforms/android/dist/src/commands/cpu/getCpuStatsByProcess.js:24:19

suggest changing the regex in line 22 of getCpuStatsByProcess.js to "/^(\d+) ((.)) (.)/"

In the list of threads (for instance, in this report, we always display the UI thread with the name read in the /proc/... files. But it's usually quite ugly, it's a string that includes all or part of the bundle id.

Furthermore if we want to compare UI thread usage for multiple apps not sharing the same bundle id, it's not so easy to do so. It would be better to identify it and rename it to "UI thread"

From device debug menu you can open perf monitor with both native FPS and JS FPS.

Native FPS for RN app is not very usefull as the main thread in most times is not doing much work, while JS thread is often doing a lot of work and cause app unresponsiveness.

Would be nice to have that parameter in this plugin, and use it also for performance score.

iOS support isn't a priority at the moment, since on average iOS devices performance is much better (and usually that translates to apps having less performance issues).

However it is still something we want to search into and implement at some point, so if you'd like to have iOS support, feel free to show it by adding a 👍 emoji to this issue.

Hi,

I'm trying to integrate flashlight within a github actions workflow and when trying to run the CLI, I get the following error:

I'm quite stumped by this. here is what the github actions steps look like:

      - name: Sign Android release
        uses: r0adkll/sign-android-release@v1
        id: sign_app
        with:
          signingKeyBase64: ${{ secrets.APPCENTER_ANDROID_KEY_PAYLOAD }}
          alias: ${{ secrets.APPCENTER_ANDROID_KEY_ALIAS }}
          keyStorePassword: ${{ secrets.APPCENTER_ANDROID_KEYSTORE_PASSWORD }}
          keyPassword: ${{ secrets.APPCENTER_ANDROID_KEY_PASSWORD }}
          releaseDirectory: android/app/build/outputs/bundle/stagingRelease

      - name: Make artifact
        id: make_artifact
        uses: actions/upload-artifact@v3
        with:
          name: app-release-signed
          path: ${{steps.sign_app.outputs.signedReleaseFile}}

      - uses: actions/checkout@v3
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: app-release-signed
          path: ${{steps.sign_app.outputs.signedReleaseFile}}

      - name: Install Flashlight CLI
        run: |
          curl https://get.flashlight.dev/ | bash
          echo "$HOME/.flashlight/bin" >> $GITHUB_PATH

      - name: Print signedReleaseFile path
        run: echo ${{steps.sign_app.outputs.signedReleaseFile}}

      - name: Run Flashlight CLI
        env:
          FLASHLIGHT_API_KEY: ${{ secrets.FLASHLIGHT_API_KEY }}
        run: |
          flashlight cloud --app ${{steps.sign_app.outputs.signedReleaseFile}} --test ./e2e/tests/chats/community-chat.e2e.yml --beforeAll ./e2e/tests/login/login.e2e.yml --apiKey $FLASHLIGHT_API_KEY

Why would the signed release path be considered a directory and not a file?

[BT:1.14.0*** Error: File 'android/app/build/outputs/bundle/stagingRelease/app-staging-release.aab' is a directory.
java.lang.IllegalArgumentException: File 'android/app/build/outputs/bundle/stagingRelease/app-staging-release.aab' is a directory.
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:[22](https://github.com/stanforddesi/eclipse-rn/actions/runs/6471517780/job/17570120942#step:17:22)0)
	at com.android.tools.build.bundletool.model.utils.files.FilePreconditions.checkFileExistsAndReadable(FilePreconditions.java:37)
	at com.android.tools.build.bundletool.commands.BuildApksCommand.validateInput(BuildApksCommand.java:942)
	at com.android.tools.build.bundletool.commands.BuildApksCommand.execute(BuildApksCommand.java:888)
	at com.android.tools.build.bundletool.BundleToolMain.main(BundleToolMain.java:83)
	at com.android.tools.build.bundletool.BundleToolMain.main(BundleToolMain.java:55)
unzip:  cannot find or open myapp.apks, myapp.apks.zip or myapp.apks.ZIP.
/snapshot/flashlight-cloud/node_modules/yoga-layout-prebuilt/yoga-layout/build/Release/nbind.js:53
        throw ex;
        ^

Error: Command failed: 
              keytool -genkey -v -dname OU=flashlight -keystore my-release-key.keystore -alias alias_name -keyalg RSA -keysize 2048 -validity 10000 -storepass azerty -keypass azerty

              curl https://github.com/google/bundletool/releases/download/1.14.0/bundletool-all-1.14.0.jar -L -o bundletool.jar

              java -jar bundletool.jar build-apks --bundle=android/app/build/outputs/bundle/stagingRelease/app-staging-release.aab                 --output=myapp.apks                 --mode=universal                 --ks=my-release-key.keystore                 --ks-key-alias=alias_name                 --ks-pass=pass:azerty

              unzip myapp.apks -d apks
            
Generating 2,048 bit RSA key pair and self-signed certificate (SHA[25](https://github.com/stanforddesi/eclipse-rn/actions/runs/6471517780/job/17570120942#step:17:25)6withRSA) with a validity of 10,000 days
	for: OU=flashlight

Hello,

since the Flipper plugin is not working on Windows I tried to install the CLI.
When I run curl https://get.flashlight.dev/ | bash in the regular command prompt i'm getting an error saying
"OS MINGW64_NT-10.0-22621 is not supported."

When I run it in Power Shell I'm getting a bunch of "command not found" errors.
I opened both, Powershell and CMD, as administrator.

Did I do something wrong or is there simply no way to use flashlight on Windows?

hamri@hamris-MacBook-Pro techonomobileapp % npx ts-node performance.ts
[17:07:30] ℹ️ Installing C++ profiler for arm64-v8a architecture
[17:07:37] ✅ C++ Profiler installed in /data/local/tmp/BAMPerfProfiler
[17:07:46] ℹ️ Running iteration 1/10
args: [-p, com.techono, -c, android.intent.category.LAUNCHER, 1]
arg: "-p"
arg: "com.techono"
arg: "-c"
arg: "android.intent.category.LAUNCHER"
arg: "1"
data="com.techono"
data="android.intent.category.LAUNCHER"
Error: Command failed: adb shell monkey -p com.techono -c android.intent.category.LAUNCHER 1
args: [-p, com.techono, -c, android.intent.category.LAUNCHER, 1]
arg: "-p"
arg: "com.techono"
arg: "-c"
arg: "android.intent.category.LAUNCHER"
arg: "1"
data="com.techono"
data="android.intent.category.LAUNCHER"

at checkExecSyncError (node:child_process:828:11)
at execSync (node:child_process:899:15)
at startApp (/Users/hamri/Desktop/appproject/techonomobileapp/performance.ts:10:11)
at run (/Users/hamri/Desktop/appproject/techonomobileapp/performance.ts:20:5)
at PerformanceTester.<anonymous> (/Users/hamri/Desktop/appproject/techonomobileapp/node_modules/@perf-profiler/e2e/index.ts:31:11)
at Generator.next (<anonymous>)
at fulfilled (/Users/hamri/Desktop/appproject/techonomobileapp/node_modules/@perf-profiler/e2e/dist/index.js:5:58)
at processTicksAndRejections (node:internal/process/task_queues:96:5) {

status: 252,
signal: null,
output: [
null,
Buffer(158) [Uint8Array] [
32, 32, 98, 97, 115, 104, 32, 97, 114, 103, 58, 32,
45, 112, 10, 32, 32, 98, 97, 115, 104, 32, 97, 114,
103, 58, 32, 99, 111, 109, 46, 112, 114, 101, 112, 112,
10, 32, 32, 98, 97, 115, 104, 32, 97, 114, 103, 58,
32, 45, 99, 10, 32, 32, 98, 97, 115, 104, 32, 97,
114, 103, 58, 32, 97, 110, 100, 114, 111, 105, 100, 46,
105, 110, 116, 101, 110, 116, 46, 99, 97, 116, 101, 103,
111, 114, 121, 46, 76, 65, 85, 78, 67, 72, 69, 82,
10, 32, 32, 98,
... 58 more items
],
Buffer(211) [Uint8Array] [
97, 114, 103, 115, 58, 32, 91, 45, 112, 44, 32, 99,
111, 109, 46, 112, 114, 101, 112, 112, 44, 32, 45, 99,
44, 32, 97, 110, 100, 114, 111, 105, 100, 46, 105, 110,
116, 101, 110, 116, 46, 99, 97, 116, 101, 103, 111, 114,
121, 46, 76, 65, 85, 78, 67, 72, 69, 82, 44, 32,
49, 93, 10, 32, 97, 114, 103, 58, 32, 34, 45, 112,
34, 10, 32, 97, 114, 103, 58, 32, 34, 99, 111, 109,
46, 112, 114, 101, 112, 112, 34, 10, 32, 97, 114, 103,
58, 32, 34, 45,
... 111 more items
]
],
pid: 14455,
stdout: Buffer(158) [Uint8Array] [
32, 32, 98, 97, 115, 104, 32, 97, 114, 103, 58, 32,
45, 112, 10, 32, 32, 98, 97, 115, 104, 32, 97, 114,
103, 58, 32, 99, 111, 109, 46, 112, 114, 101, 112, 112,
10, 32, 32, 98, 97, 115, 104, 32, 97, 114, 103, 58,
32, 45, 99, 10, 32, 32, 98, 97, 115, 104, 32, 97,
114, 103, 58, 32, 97, 110, 100, 114, 111, 105, 100, 46,
105, 110, 116, 101, 110, 116, 46, 99, 97, 116, 101, 103,
111, 114, 121, 46, 76, 65, 85, 78, 67, 72, 69, 82,
10, 32, 32, 98,
... 58 more items
],
stderr: Buffer(211) [Uint8Array] [
97, 114, 103, 115, 58, 32, 91, 45, 112, 44, 32, 99,
111, 109, 46, 112, 114, 101, 112, 112, 44, 32, 45, 99,
44, 32, 97, 110, 100, 114, 111, 105, 100, 46, 105, 110,
116, 101, 110, 116, 46, 99, 97, 116, 101, 103, 111, 114,
121, 46, 76, 65, 85, 78, 67, 72, 69, 82, 44, 32,
49, 93, 10, 32, 97, 114, 103, 58, 32, 34, 45, 112,
34, 10, 32, 97, 114, 103, 58, 32, 34, 99, 111, 109,
46, 112, 114, 101, 112, 112, 34, 10, 32, 97, 114, 103,
58, 32, 34, 45,
... 111 more items
]
}
hamri@hamris-MacBook-Pro techonomobileapp % npx @perf-profiler/web-reporter results.json
node:internal/fs/utils:345
throw err;
^

Error: ENOENT: no such file or directory, lstat 'results.json'
at Object.lstatSync (node:fs:1529:3)
at /Users/hamri/.npm/_npx/236afad33dc9fea1/node_modules/@perf-profiler/web-reporter/dist/writeReport.js:20:46
at Array.map ()
at getJsonPaths (/Users/hamri/.npm/_npx/236afad33dc9fea1/node_modules/@perf-profiler/web-reporter/dist/writeReport.js:19:14)
at writeReport (/Users/hamri/.npm/_npx/236afad33dc9fea1/node_modules/@perf-profiler/web-reporter/dist/writeReport.js:33:35)
at Object. (/Users/hamri/.npm/_npx/236afad33dc9fea1/node_modules/@perf-profiler/web-reporter/dist/openReport.js:16:52)
at Module._compile (node:internal/modules/cjs/loader:1105:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
at Module.load (node:internal/modules/cjs/loader:981:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12) {
errno: -2,
syscall: 'lstat',
code: 'ENOENT',
path: 'results.json'
}
hamri@hamris-MacBook-Pro techonomobileapp %

I tried with npx ts-node performance.ts method but it is not working also tried with another methods but nothing worked so let me know what is error over here and how can i connect it with production

Hello 👋🏻 ,

First, awesome work, thanks for putting the effort on this tool.

Is it possible to track both IU and JS FPS in a production build? I could find the --fps flag, but I think that is tracking only UI thread, right?

Hello Everyone,

I was trying to integrate Flashlight with Bitrise. So I added Script step:

#!/usr/bin/env bash

# Install Flashlight
curl https://get.flashlight.dev/ | bash

# Export PATH
export PATH=$PATH:$HOME/.flashlight/bin

# Run Flashlight command with the correct --test path
flashlight cloud --app $BITRISE_APK_PATH --test .maestro/subflows/prepare.yaml --apiKey $FLASHLIGHT_API_KEY

Here are the logs:

Can you tell me why is this happening? Is something missing in preparation? Maestro flow that I'm trying to invoke refers to another flow from .maestro folder and it couldn't be found. I probably miss something how the code is being read by the tool.

Looking forward to hearing from you!

Best regards

Hey,

I have the following use-case:

I'd like to start the profiler using the CLI tool and stop recording once I have manually executed a certain flow.
Once i stopped the tool from execution I'd expect the tool it to write to a results file. Then I can inspect the results
file with the web reporter (handy for creating quickly comparisons during dev).

I know that there is the possibility to write a script and instrument it to do the measurements, however, quickly typing a command feels way faster.

Is that something we could add? (I can also add it to the code just lmk 😊 )

(btw, I love this tool, thx for building it <3).

An idea would be to pass e2e test shell commands instead of ts/js scripts and have measures on it which is easily doable

For Maestro the blocker is that it needs time to initialize and we don't want to measure performance while doing that.
Based on Maestro's answer, the Maestro CLI is actually just a thin wrapper around the core Maestro JVM libraries. You can consume the libraries directly from a Kotlin or Java project.
We could leverage those directly https://github.com/mobile-dev-inc/maestro/blob/main/maestro-cli/src/main/java/maestro/cli/App.kt

What

Max FPS is set to 60 in the code
We need to account for devices with a higher refresh rate

How to implement

• this file needs to be changed flashlight-open-source/packages/platforms/android/src/commands/atrace/pollFpsUsage.ts
• score calculation need to take into account max fps flashlight-open-source/packages/core/reporter/src/reporting/getScore.ts

We need to add screenRefreshRate into the interfaces TestCaseResult and AveragedTestCaseResult
We need to find a way to measure it and add it to the report

So that max FPS in graphs and score is max FPS du device

When running Flashlight tests via Maestro with flashlight test <maesto test file> the results are nicely aggregated into a single result, which makes comparing against different runs easy. Is it possible to do this when manually executing tests? Not sure if it doesn't exist, or I've just missed it in the docs.

We have found that due to the time Maestro spends letting the app go idle our tests aren't quite as accurate as we'd like them to be. So instead we've been manually running tests via flashlight measure, running ~5 times, then grabbing the worst result. Is there a way to merge results like what is done when automating measures, but with manual tests? Happy to help with this if it doesn't exist, just point me in the right direction.

Thanks heaps, love the tool it has massively changed our performance measuring game.

• flashlight 0.10.0
• maestro 1.30.4

I'm trying to run a simple maestro test but the command gets stuck:

This is the maestro test:

performance.yaml

appId: com.getpaired.app.stg
---
- launchApp

When I run command command on it's own this is the result:

> maestro test .maestro/performance.yaml

Android

[1] emulator-5554

iOS

[2] iPhone SE (3rd generation) - iOS 16.2 - 4E825D75-1A73-4F66-96C6-5CC03A38EE80
[3] iPhone 14 - iOS 16.2 - DD1E0502-8A70-460C-B8A2-5D5CE0314ADA

Multiple running devices detected. Choose a device to run on.

Enter a number from the list above:

1


Running on emulator-5554                        
                                                
 ║                                              
 ║  > Flow                                      
 ║                                              
 ║    ✅  Launch app "com.getpaired.app.stg"    
 ║                           

When I run it through flashlight it gets stuck forever after device selection prompt:

> flashlight test --bundleId com.getpaired.app.stg \
  --testCommand "maestro test .maestro/performance.yaml" \
  --duration 40000 \
  --resultsFilePath results.json
  
[12:08:53] ℹ️  Installing C++ profiler for arm64-v8a architecture
[12:08:53] ✅  C++ Profiler installed in /data/local/tmp/BAMPerfProfiler
[12:08:53] ℹ️  Running iteration 1/10
Android


[1] emulator-5554

iOS

[2] iPhone SE (3rd generation) - iOS 16.2 - 4E825D75-1A73-4F66-96C6-5CC03A38EE80
[3] iPhone 14 - iOS 16.2 - DD1E0502-8A70-460C-B8A2-5D5CE0314ADA

Multiple running devices detected. Choose a device to run on.

Enter a number from the list above:


1

Any idea on what could it be happening?

I am working on boosting List performance project in my company. Then found flashlight is exactly what i want to make perf benchmark.

I have ran flashlist with my own project call SpectrumList. code base could be found https://github.com/red-armor/ReactNativeListComparison. It's based on the demo example https://github.com/naqvitalha/travelMate. The bench result is posted as below. In fact, I was shocked for this.

And then, I has also followed your project on twitter, the following is related post link.

• https://twitter.com/almouro/status/1564227005121454081?s=20
• https://docs.flashlight.dev/examples/flashlist/report.html

The question is I want to make benchmark result more reasonable, So could you give some suggestion or any demo code base?

Best regards

Hey! I'm using flipper+flashlight plugin and getting NaN Score and NaN Average CPU Usage kinda frequently.
I'm running flashlight on android device (9) with my autotests.
It starts as normal but then when the tests get more intense I can get this problem

Hello @Almouro 👋🏻 !

I have tried the flashlight cloud and I have some doubts.

1. There is someway to get the results of the execution via API?
   Like this one Data session

It would be interesting in order to export data to grafana for example.

2. I am not able to see in which device the test has been executed. Is a Xiaomi or Samsung device? Which android version?

I find the project very interesting, and it seems to be on a very promising path.
Thank you very much for the effort, best regards.

Several use cases are not documented, and documentation could be clearer in some parts:

Open the web reporter for comparison

npx @perf-profiler/web-reporter report1.json report2.json

Use measures without web-reporter

import {
  getAverageCpuUsage,
  getAverageFPSUsage,
  getAverageRAMUsage,
} from "@perf-profiler/reporter";

const { measures } = await measurePerformance(bundleId, testCase);

const ramForEachIteration = measures.map((measure) =>
    getAverageRAMUsage(measure.measures)
  );

duration is not clear

duration should ideally be longer than test duration when you set it
Users think it's a sort of timeout, but it's not clear that it's here to ensure consistency of scoring between measures

Would you consider to add the app inside "brew" ? 🤔 ☺️

Seems to happen with apps with background processes

Flashlight starts and starts measuring since it can find the background pid for the app, then Maestro restarts the app with a different pid resulting in this error

When I try to click auto detect the flipper plugin crashes.

If I type in the box it throws up an error after each character. I was able to put a bundle id in by pasting the full id into the box.
However, the plugin then crashes when I click start measuring.

Is there a configuration option I'm missing? Not sure what the issue might be.

Feature Description

Saving a results of experiment is not supported inside a Flipper.
Save Result button could be next to Reset button in one row

Once user clicked on that, it could save all result into JSON file. Under the hood, we could use below approach

  const { writeResults } = await measurePerformance(bundleId, testCase);
  writeResults();

Right now, saving a results is only supported by e2e tests.

Expected outcome

• Click on Save Results button
• Choose json dir to save
• Click on save button in the prompt
• Create new JSON file with all results in previously selected dir

Supported Platform

• Flipper

Reproducing

Create a test.yaml file with:

appId: i.dont.exist
---
- launchApp

Then run:

flashlight test --testCommand "maestro test test.yaml" --duration 10000 --bundleId com.example --iterationCount 1

Or if running from source:

node packages/e2e-performance/dist/bin.js test --testCommand "maestro test test.yaml" --duration 10000 --bundleId com.example --iterationCount 1

When I ran my appium script (npx ts-node tf1-info.ts), I got this error:

/Users/charlotte/Documents/Projects/node_modules/@perf-profiler/profiler/src/commands/atrace/pollFpsUsage.ts:48
    lines.forEach((line) => {
          ^
TypeError: Cannot read property 'includes' of undefined
    at /Users/charlotte/Documents/Projects/node_modules/@perf-profiler/profiler/src/commands/atrace/pollFpsUsage.ts:64:24
    at Array.forEach (<anonymous>)
    at FrameTimeParser.getFrameTimes (/Users/charlotte/Documents/Projects/node_modules/@perf-profiler/profiler/src/commands/atrace/pollFpsUsage.ts:48:11)
    at /Users/charlotte/Documents/Projects/node_modules/@perf-profiler/profiler/src/commands/pollPerformanceMeasures.ts:26:25
    at /Users/charlotte/Documents/Projects/node_modules/@perf-profiler/profiler/src/commands/cppProfiler.ts:104:7
    at /Users/charlotte/Documents/Projects/node_modules/@perf-profiler/profiler/src/commands/shell.ts:49:7
    at Array.forEach (<anonymous>)
    at Socket.<anonymous> (/Users/charlotte/Documents/Projects/node_modules/@perf-profiler/profiler/src/commands/shell.ts:48:29)
    at Socket.emit (node:events:365:28)
    at Socket.emit (node:domain:470:12)`

The script had just printed Running iteration 1/10

Here is my script:

import { AppiumDriver } from "@bam.tech/appium-helper";
import { TestCase, measurePerformance } from "@perf-profiler/e2e";

const runTest = async () => {
  const driver = await AppiumDriver.create({
    // `npx @perf-profiler/profiler getCurrentApp` will display info for the current app
    appPackage: "com.smartnsoft.metro",
    appActivity: "com.mobilenews.MainActivity",
  });

  const testCase: TestCase = {
    beforeTest: async () => {
      driver.stopApp();
    },
    run: async () => {
      // run is where measuring will happen, insert e2e logic here
      driver.startApp();
      await driver.findElementByText("Bonjour");
    },
    // Duration is optional, but helps in getting consistent measures.
    // Measures will be taken for this duration, regardless of test duration
    duration: 10000,
  };

  const { writeResults } = await measurePerformance("com.smartnsoft.metro", testCase);
  writeResults();
};

runTest()

Android phone: Samsung A30
Android version: 10

FYI, it is the same script that I launched on another phone (pixel 6, android 13) where it worked

Hello 👋🏻 ,

First of all congratulations on your work and your contribution to the community.

I have a doubt in order to launch the library

We use yarn and our command to launch the tests is determined by different parameters example:

npx @perf-profiler/e2e measure --bundleId xxx --testCommand "yarn e2e:android:local:fr" --duration 100000 --resultsFilePath results.json

The test runs but the process continues and does nothing.

Any suggestion?

Thanks!

Hi, I sometimes getting this error:

I tried restart app, restart flashlight but it did not help and then it suddently start to work and work for a while and then again this error.

What does this error mean?

Thanks.

adb shell am start fails on several apps (ex: Amazon shop, Twitter).

It throws this security exception:

Command failed: adb shell am start com.amazon.mShop.android.shopping/com.amazon.mShop.navigation.MainActivity
    Security exception: Permission Denial: starting Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10000000 cmp=com.amazon.mShop.android.shopping/com.amazon.mShop.navigation.MainActivity } from null (pid=8993, uid=2000) not exported from uid 10155

    java.lang.SecurityException: Permission Denial: starting Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10000000 cmp=com.amazon.mShop.android.shopping/com.amazon.mShop.navigation.MainActivity } from null (pid=8993, uid=2000) not exported from uid 10155
        at com.android.server.am.ActivityStackSupervisor.checkStartAnyActivityPermission(ActivityStackSupervisor.java:2009)
        at com.android.server.am.ActivityStarter.startActivity(ActivityStarter.java:931)
        at com.android.server.am.ActivityStarter.startActivity(ActivityStarter.java:656)
        at com.android.server.am.HwActivityStarter.startActivity(HwActivityStarter.java:235)
        at com.android.server.am.ActivityStarter.startActivityMayWait(ActivityStarter.java:1492)
        at com.android.server.am.HwActivityStarter.startActivityMayWait(HwActivityStarter.java:430)
        at com.android.server.am.ActivityStarter.execute(ActivityStarter.java:597)
        at com.android.server.am.ActivityManagerService.startActivityAsUser(ActivityManagerService.java:5778)
        at com.android.server.am.ActivityManagerService.startActivityAsUser(ActivityManagerService.java:5751)
        at com.android.server.am.ActivityManagerShellCommand.runStartActivity(ActivityManagerShellCommand.java:478)
        at com.android.server.am.ActivityManagerShellCommand.onCommand(ActivityManagerShellCommand.java:160)
        at android.os.ShellCommand.exec(ShellCommand.java:103)
        at com.android.server.am.ActivityManagerService.onShellCommand(ActivityManagerService.java:17870)
        at android.os.Binder.shellCommand(Binder.java:642)
        at android.os.Binder.onTransact(Binder.java:540)
        at android.app.IActivityManager$Stub.onTransact(IActivityManager.java:3570)
        at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:3662)
        at com.android.server.am.HwActivityManagerService.onTransact(HwActivityManagerService.java:609)
        at android.os.Binder.execTransact(Binder.java:739)

WHAT

I used the custom script to profile an android app. The app is in release build with the profileable tag enabled. On debugging and going a little deep into the source code i realised that the atrace was not getting collected. On debugging a little more i see the error log

Unable to open file5037 (.package.appname) S 1113 1113 0 0 -1 1077952832 2417899 0 63 0 45820 17497 0 0 20 0 117 0 119065447 25016229888 58295 18446744073709551615 1 1 0 0 0 0 4612 1 1073775864 0 0 0 17 3 0 0 0 0 0 0 0 0 0 0 0 0 0

The Unable to open file comes from line number

On checking the path of the systrace file the source code mentions

std::string traceOutputPath = "/sys/kernel/debug/tracing/trace_pipe";

However on my device i see the trace file being created at the path /sys/kernel/tracing/trace_pipe. To verify this on one terminal i ran the command adb shell atrace -c view -t 999 and in another terminal did a simple cat /sys/kernel/tracing/trace_pipe . And i was able to get the atrace..

What am i to do here? is it possible the trace file locations vary as per devices? Also how do i fix the current case?

Device Info

[media.recorder.show_manufacturer_and_model]: [true]
[ro.board.platform]: [sm6150]
[ro.boot.hardware]: [qcom]
[ro.boot.hardware.revision]: [pvt]
[ro.boot.hardware.sku]: [XT2147-1]
[ro.boot.product.hardware.sku]: [d]
[ro.boot.revision]: [pvt]
[ro.boot.secure_hardware]: [1]
[ro.boot.serialno]: [ZD2222GDQ8]
[ro.build.version.sdk]: [30]
[ro.hardware]: [qcom]
[ro.hardware.egl]: [adreno]
[ro.hardware.nfc_nci]: [pn54x]
[ro.hardware.sensors]: [hanoip]
[ro.hardware.soc.manufacturer]: [qcom]
[ro.hardware.vulkan]: [adreno]
[ro.mot.build.version.sdk_int]: [31]
[ro.opa.device_model_id]: [motorola-hanoip]
[ro.product.brand]: [motorola]
[ro.product.build.version.sdk]: [30]
[ro.product.manufacturer]: [motorola]
[ro.product.model]: [moto g(40) fusion]
[ro.product.name]: [hanoip_retail]
[ro.product.odm.brand]: [motorola]
[ro.product.odm.manufacturer]: [motorola]
[ro.product.odm.model]: [moto g(60)]
[ro.product.product.brand]: [motorola]
[ro.product.product.manufacturer]: [motorola]
[ro.product.product.model]: [moto g(60)]
[ro.product.product.name]: [hanoip_retail]
[ro.product.system.brand]: [motorola]
[ro.product.system.manufacturer]: [motorola]
[ro.product.system.model]: [moto g(60)]
[ro.product.system_ext.brand]: [motorola]
[ro.product.system_ext.manufacturer]: [motorola]
[ro.product.system_ext.model]: [moto g(60)]
[ro.product.vendor.brand]: [motorola]
[ro.product.vendor.manufacturer]: [motorola]
[ro.product.vendor.model]: [moto g(60)]
[ro.revision]: [pvt]
[ro.serialno]: [ZD2222GDQ8]
[ro.system.build.version.sdk]: [30]
[ro.system_ext.build.version.sdk]: [30]
[ro.vendor.boot.serialno]: [ZD2222GDQ8]
[ro.vendor.hw.revision]: [pvt]
[ro.vendor.product.hardware.sku.variant]: [d]
[ro.vendor.product.name]: [hanoip_retail]

Custom Script

const polling = pollPerformanceMeasures(pid, (measure) => {

  measures.push(measure);
  console.log(`JS Thread CPU Usage: ${measure.cpu.perName["(mqt_js)"]}%`);
  console.log(`Main Thread CPU Usage: ${measure.cpu.perName["UI Thread"]}%`);
  console.log(`RAM Usage: ${measure.ram}MB`);
});

setTimeout(() => {
  polling && polling.stop();
  fs.writeFileSync('results.json', JSON.stringify(measures));
}, 10000);

Just followed the documentation..

Command I'm running:

flashlight test --bundleId "my.app" \
  --testCommand "adb shell monkey -p my.app -c android.intent.category.LAUNCHER 1" \
  --duration 10000 \
  --resultsFilePath results.json

Error I'm getting:

[18:06:53] 🚨 Error when running "adb shell monkey -p my.app -c android.intent.category.LAUNCHER 1":
script: tcgetattr/ioctl: Operation not supported on socket

Tried running it with sudo, both on physical device and an emulator. Tried running through maestro as well. Still getting the same error.
For some reason, flashlight measure seems to work without issues so does the adb command alone. I can attach to the app and measure perfomance. So this issue seems to be CLI only.