psambit9791 / jdsp Goto Github PK

When I use your library jdsp for peak detection on android phone,It takes long time and can not get the result.
The data can be downloaded at https://drive.google.com/file/d/112almsqRELM5sE82I9QIfHjgazAOjLcT/view?usp=sharing
The code as follows:
readWavObj = new WAV();
readWavObj1 = new WAV();
try {
readWavObj.readWAV(Environment.getExternalStorageDirectory().getAbsolutePath() + "/" + "audio/" + "singal.wav");
originSignal = readWavObj.getData("double");
readWavObj1.readWAV(Environment.getExternalStorageDirectory().getAbsolutePath() + "/" + "reference.wav");
originrefSingal = readWavObj1.getData("double");
} catch (WavFileException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
//covert 1-D array
double[] Singal = utilAcObj.getArray(originSignal);
double[] refSingal = utilAcObj.getArray(originrefSingal);

                        String mode = "valid"; //Can be "valid", "same"
                        CrossCorrelation cc = new CrossCorrelation(Singal, refSingal);
                        double[] zout = cc.crossCorrelate(mode);
                        FindPeak fp = new FindPeak(zout);
                       Peak out = fp.detectPeaks();
                       int[] peaks = out.getPeaks();

Hi dear JDSP author,
I have been using using JDSP library for filtering time series data from sensor. Currently I am filtering 33 samples from a sensor in one double array and get perfect results. I am using same filter instance to filter similar sensor data from 3 other sensors. However, when i receive new data samples the results are incorrect after using filters.
What would be the ideal way to apply same filter to multiple sample from multiple sensors? Thanks in advance for your suggestion. Here is my current implementation.

Salman

Hello,

The image below is the FFT plot of my data and it is showing symmetry (I did not change the values printed on the x axis to reflect the frequency of the original time domain, but the shown FFT plot below is assuredly of transformed data). Would you please tell me if such symmetry is indicative of raw data that is too noisy or lacking of any discernible constituent sinusoids (my raw data does actually have a nice bandwidth - fixed y max and fixed x max - throughout the collected data oscillations)? I've been looking at FFT plots of a more "textbook" like quality and those plots give nice and discernible/distinct peaks that don't have symmetry in their profile and those ideal peaks shoot up from the x axis as opposed to my profile which looks like raw data itself (such idealized "textbook" data tends to be of the waveform from a guitar or violin string just to point out what I've been looking at). Thanks for any direction or tips you can provide.

LM

Hello,

Would you please tell me if in the figure below if the very first/leftmost data point that starts over 12,000 units on the Y-axis is some sort of artifact that is almost always produced by taking the FFT? I am almost consistently getting this very tall first data point of the FFT spectrum from all my data sets so I couldn't help but wonder if my data is telling me something or if all FFT plots always start out looking like this from the FFT math under the hood. I mean, the very first tall data point is not a distinct peak for me to confidently state (with my present limited knowledge) that it is indeed a frequency in my oscillation data, but I would like to know if there is some physical meaning behind such an "incomplete" peak/"incomplete" frequency that begins my FFT plot. Here is the image in question:

This is in contrast with this next FFT plot whose first point instead does not simply slope down but it actually is part of a distinct peak whose maxima is formed by the second FFT plot data point:

Therefore, do I have a strong low frequency according to the first graph? Also, do I have a strong low frequency according to the second graph? If it means anything, the X-axis values shown above are presently just the ennumerated indices of the arrays that contained the data as I am still working on "translating" the array index values to actual frequency values. Thanks.

LM

Would you please tell me how you would like to have JDSP cited/bibliographed in peer-review publications whose work uses JDSP? Thanks.

LM

see this

Problems occurring for values of factors for which the sampling frequency is not divisible by the factor. Array length is less by 1.

OUTPUT:

array lengths differed, expected.length=32 actual.length=31; arrays first differed at element [31];
Expected :-0.494
Actual :end of array

at com.onyx.jdsp.TestDecimate.DecimateWithZeroPhaseTest(TestDecimate.java:102)
Caused by: java.lang.AssertionError: expected:<-0.494> but was:
at org.junit.Assert.failNotEquals(Assert.java:835)

Adding Kaiser Window to the list of available Windows - Will be useful for building FIR filter.

Identified bug with findPlateauSize()
The function seems to accept the peak indices in stead of the actual peaks.

It would be great to have an adapter class ("Processor") to wrap, for example, the ShortTimeFourier class and provide a more real-time streaming interface. There are examples of this style in "Digital Audio with Java" by Craig A Lindley.
The wrapper would mean that the full input signal would not need to be loaded into memory, but could be streamed through the processor.
We could use it for real-time sound affects such as pitch shifting/tempo changing (there is another suggestion for time-based pitch shifting/tempo changing using WSOLA - #18).
I am not sure if older phones/tablets have enough processing power for this, but it would be great to try!

Related to #20, is there a possibility to implement the Wavelet Transform?

Again, there is a library that seems to have an implementation of the Wavelet Transform.

Hello,

I want to calculate signal's Auto-Correction in Android.
So, I used this library using gradle.

I saw this issue was fixed but still I can not use Cross-Correlation which takes only the signal as argument

so, how can i use ?

An implementation of complex convolution such that the signal and the kernel can be complex arrays or double arrays.

Hello developpers team,
Thank you for your good work on this package
Would you please add the multiplicative scatter correction to the signal correction functions?
This is an example using python

Hello,

I have experienced the exact same issue as issue #5 but I am not able to solve the issue.

After following the steps, I am getting the following error:

`A problem occurred evaluating project ':app'.

Could not find method com.github.psambit9791:jdsp:2.0.1() for arguments [build_8b7glpftwjyi9ocz81v2e7e82$_run_closure2$_closure9@1fa01953] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.

• Try:

Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

• Exception is:
  org.gradle.api.GradleScriptException: A problem occurred evaluating project ':app'.
  at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:93)
  at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.lambda$apply$0(DefaultScriptPluginFactory.java:133)
  at org.gradle.configuration.ProjectScriptTarget.addConfiguration(ProjectScriptTarget.java:79)
  at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.apply(DefaultScriptPluginFactory.java:136)
  at org.gradle.configuration.BuildOperationScriptPlugin$1.run(BuildOperationScriptPlugin.java:65)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
  ...
  at org.gradle.groovy.scripts.BasicScript$ScriptDynamicObject.tryInvokeMethod(BasicScript.java:135)
  at org.gradle.internal.metaobject.AbstractDynamicObject.invokeMethod(AbstractDynamicObject.java:163)
  at org.gradle.groovy.scripts.BasicScript.invokeMethod(BasicScript.java:84)
  at build_8b7glpftwjyi9ocz81v2e7e82.run(C:\Users\tushar.rain\AndroidStudioProjects\FFT\app\build.gradle:41)
  at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:91)
  ... 177 more`

Kindly help in resolving the issue.
Thanks

I want to calculate signal's Auto-Correction. In matlib, I can use xcorr() function to do this. But this this lib, I can not find a function like this. (Cross-Correction can do this?)

Hi! I was wondering if it would be possible to implement the STFT and inverse STFT, besides the already implemented Fourier transforms. :)

I believe there is also this library that can compute the STFT and inverse STFT, in case you might find it helpful to take a look at it.

Thanks!

First I would like to say this is an amazing and well-written library.
I was wondering if it would be possible for a feature for time stretching or pitch shifting such as PSOLA or WSOLA to be added, as I think such a feature would be a good addition to the features this library has.
Again, thank you for making such a great library

Hello,

Would you please tell me if it would be possible for JDSP to have a new feature to calculate the power spectral density of a random signal (I've been told that it is inappropriate to do FFT on random signals)? If it is a trivial thing that the user can do by just writing some loop and doing some simple math on the FFT results instead, please let me know that this is the case as I'd like to be able to implement power spectral density capabilities in my workflows to handle a more diverse set of signals. Thanks.

LM

If you run any Frequency filter and have lowCutOff>highCutOff, then you do not get an error. In fact this happens:

double centreFreq = (highCutoff + lowCutoff)/2.0;
double width = Math.abs(highCutoff - lowCutoff);

So what happens here is lowCutOff and highCutOff are implicitly inverted. I think this is not desired?

Java 19 has released simd vector support.
It would be good to add simd to support to jdsp library.
They are all intrinsinc functions.

You can checkout for library preview:
https://medium.com/@Styp/java-18-vector-api-do-we-get-free-speed-up-c4510eda50d2

If I have data in the form of x,y coordinates, how do I get this data into JDSP? I noticed that data input into, say, a low pass filter is in the form of a 1-D array of type double which is essentially a list of one set of numbers not two. Should the numbers in the 1-D array be in pairs as in [x1, y1, x2, y2, x3, y3, etc...]?

Hi there,

When using filters in my own code it is a bit cumbersome, that there is not an interface or such for it. So you cannot do something like:

		HashMap<String, Filter> filterHashMap = new HashMap<>();
		for(String filterName : filterHashMap){
			Filter myFilter = filterHashMap.get(filterName);
			// Do things with each filter
		}

Are there any plans to change this?

Cheers,
Martin

Is there a way to filter peaks by multiple properties (as done in scipy)?

Example in scipy:

find_peaks(x, distance=20, height=(50, None)

jDSP currently allows filtering by only one metric (i.e. find* methods return int[], and they cannot be filtered further). Unless I'm missing something

As currently written and as an example, when we call Butterworth.lowPassFilter() method a new filter is initialized and used to filter the data. This works correctly and as expected when all data is already available ahead of time.

In same cases (using a continuous sampling data acquisition task) we may want to collect a batch of data, filter that batch, and then display the filtered data. We may want to do this over and over again as new batches of data is collected. If we use the filters in this cases, a new filter is created and initialized each time we call the .lowPassFilter() method, which will incorrectly show discontinuous jumps in the filtered and displayed data from batch to batch.

I am proposing to have a "BatchFilter" or reusable filter, where the same filter is persisted and reused, as opposed to creating a new filter for every filter operation. I think this should hopefully be straightforward, as you could just store uk.me.berndporr.iirj.Butterworth() filter as a field in your ButterWorth class and reused that filter for each filter operation, instead of initializing and using a new one with each filter operation.

I was trying to add jdsp to my project but it didn't work.

This is my gradle file:

/*
 * This file was generated by the Gradle 'init' task.
 *
 * This generated file contains a sample Java application project to get you started.
 * For more details take a look at the 'Building Java & JVM projects' chapter in the Gradle
 * User Manual available at https://docs.gradle.org/7.3/userguide/building_java_projects.html
 */

plugins {
    // Apply the application plugin to add support for building a CLI application in Java.
    application
}

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
    maven(url = "https://oss.sonatype.org/content/repositories/snapshots")
    maven(url = "https://oss.sonatype.org/service/local/staging/deploy/maven2/")
}

dependencies {
    // Use JUnit Jupiter for testing.
    testImplementation("org.junit.jupiter:junit-jupiter:5.7.2")

    // This dependency is used by the application.
    implementation("com.google.guava:guava:30.1.1-jre")
    implementation("tech.tablesaw:tablesaw-core:0.43.1")
    implementation("com.github.psambit9791:jdsp:1.0.0")
    implementation(fileTree(mapOf("dir" to "libs", "include" to listOf("*.jar"))))
}

application {
    // Define the main class for the application.
    mainClass.set("javagradlecage.App")
}

tasks.named<Test>("test") {
    // Use JUnit Platform for unit tests.
    useJUnitPlatform()
}

I'm not even getting any errors. Just nothing happens.

I tried adding the jar file too, but same result. If I add the jar and try to build grade it at least tells me that they are duplicates.

Am I missing something? I basically did the same with other dependencies (e.g. tablesaw) and they work fine.

For signals which have large number of values close to 0, like a Gaussian pulse, the instantaneous phase is incorrect. Comparisons made against scipy.hilbert() and their method of calculating phase and frequency.

The signal for which the issue was identified

The issue as visualised

Right now, the DFT transform can return both the complex valued result and the magnitude of that result. Would also be helpful to include a return function for the phase of the result.

The jdsp library currently cannot be used with java modular projects because jdsp is an unnamed module. This can be fixed by converting it into an automatic module.

This can be done by adding an automatic module name attribute to your jar manifest with the following line:

Automatic-Module-Name: com.github.psambit9791:jdsp

Would you please tell me if by calculating the "only positive absolute values" from JDSP when doing the Fourier transform, if this is sufficient, for a physical system I have that is oscillating back and forth, to obtain the constituent frequencies of oscillations in my system?

The code I'm using to calculate the only positive absolute values is below:

_Fourier ft = new DiscreteFourier(signal); // OR _Fourier ft = new FastFourier(signal); ft.transform(); boolean onlyPositive = true; double[] out = ft.getMagnitude(onlyPositive);

Thanks.

LM

The full DFT output should be of length equal to the signal length. The mirrored (only positive) length should be equal to (length of signal)/2 + 1. But the mirrored output length is computed as (length of signal)/2.

Hello,

I saw on JDSP's website that there is a new version of JDSP available. Would you please tell me what are the newest features available on this version that distinguishes it from version 1.0.0? I didn't see such a description on the website and so I am curious. Also, does the new version have a feature to provide the actual frequency values when doing FFTs? One of my previous challenges as a newbie to the DSP world was manually calculating the "x-axis" values that correspond to the plot/graph of the DFT from my data.

Thanks and it's really exciting to see the newest version of JDSP available. I can't wait to use the new features and please keep this amazing project going.

LM

IDFT right now supports only transforming array with same length as the signal. But a fully functional IDFT also needs to support transforming the half length sequences which are used as outputs from DFT of real-valued signals. This can be done because, for real valued signals, the second half is the mirrored complex conjugate of the first half.

This support needs to be added to the InverseDiscreteFourier class.

This is a great library, now I translate matlab program to java code, there is a filtfilt() method in matlab, it's Zero-phase digital filtering, is it supported in your library?
thanks

When we use Savitzky Golay Filte to smooth data, we found that the result after Java processing is inconsistent with Python. Inconsistent intervals appear at the edges，so we checked the source code of both。When the mode selects constant, python does edge processing on the data, but Java does not.
Excuse me, will the code be updated in the future to keep it consistent?

There seem to be a dependency issue in 0.4.0. I am trying to build with Gradle from Andoid Studio and get the following error, when adding jDSP as a dependency

Duplicate class org.apache.maven.surefire.report.SafeThrowable found in modules jetified-common-java5-2.19.1.jar (org.apache.maven.surefire:common-java5:2.19.1) and jetified-surefire-api-2.19.1.jar (org.apache.maven.surefire:surefire-api:2.19.1)
Duplicate class org.apache.maven.surefire.report.StackTraceWriter found in modules jetified-common-java5-2.19.1.jar (org.apache.maven.surefire:common-java5:2.19.1) and jetified-surefire-api-2.19.1.jar (org.apache.maven.surefire:surefire-api:2.19.1)

Hi Thanks for your jdsp library , i could actually develop a hamiltion segmentator for ecg processing with it.
But when i tried to integrate with andriod API below 24 i am getting this error
java.lang.NoSuchMethodError: No static method stream([D)Ljava/util/stream/DoubleStream; in class Ljava/util/Arrays; or its super classes (declaration of 'java.util.Arrays' appears in /system/framework/core-libart.jar)
at com.github.psambit9791.jdsp.filter.FIRWin1.computeCoefficients(FIRWin1.java:270)
at com.piccolomed.smartpicc.ecg.PeakDetector.apply_fir_filter(PeakDetector.java:651)
at com.piccolomed.smartpicc.ecg.PeakDetector.apply_fir_filter_filtfilt(PeakDetector.java:657)
at com.piccolomed.smartpicc.ecg.PeakDetector.getPeaks(PeakDetector.java:29)

i am currently using version 0.8.0 of you library, it seems stream are not supported in older versions of android.
is there any other alternative

Adaptive filtering is often used for signal noise reduction and some other cool DSP applications. The most often used one is the Least Mean Square (LMS) adaptive filter.

A very good adaptive filtering library in Python is padasip. It also has some code documentation on how the algorithm should work, both in code and on a documentation page.

Other potential good source: explanation of MATLAB LMS adaptive filtering implementation

I've been looking for a BSD/MIT replacement for TarsosDSP, and I came across this. It doesn't have all the stuff, but I was then interested on maybe contributing them.

But I was also interested, can you do a comparison to JLibrosa (https://github.com/Subtitle-Synchronizer/jlibrosa)?

It would be a fairly simple implementation, since a zero padding method already exists in the utils methods, but having zero padding implemented in the dtf-method of DiscreteFourier would be handy.

The ecg.txt file was not included in the JAR for Maven releases. This may lead to error when calling the "electrocardiogram()" function throwing a FileNotFoundException.

The round() method under UtilMethods always gives the output in Ceil mode of rounding.

double d1 = 7.40241449;
double d2 = -14.39767505;

System.out.println(UtilMethods.round(d1, 1));
System.out.println(UtilMethods.round(d2, 1));

Outputs: 7.5 , -14.3

Expected: 7.4 , -14.4

Hello,

I plotted the data in the file named "Freq_ANGLE_PER_NUM_29_theta.txt" (shown below)

and then I obtained the DFT for that same data and plotted it (shown below)

I also stored this DFT output data in the file named "Freq_theta_Num_29_FrequencyValues.txt" (I included index/predictibly increasing numbers as part of this data, but they are not provided from the JDSP code).

I then wanted to find out where the peaks were and so I used the following code on the DFT version of my data:
FindPeak dftFp = new FindPeak(xOut);
Peak middleMan = dftFp.detectPeaks();
int[] peaks = middleMan.getPeaks();
//System.out.println(peaks.length + "\n");
System.out.println("Residue: " + counter);

The results from this peak-determining code indicated that the very first peak/maximum was located at the x-axis value of '1' when the plot by contrast seems to show that the very first peak is instead located the x-axis value of '2.'

I stored the location results of all peaks in the file named "Freq_theta_Num_29AllTransformPeaks.txt"

Am I reading/interpreting the output data incorrectly (all data files referenced to in this message are attached
Freq_ANGLE_PER_NUM_29_theta.txt

Freq_theta_Num_29_FrequencyValues.txt

Freq_theta_Num_29AllTransformPeaks.txt
)? Why does the plot seem to indicate the peak is at one position, but the peak-detecting analysis with the library declares the peak is at another?

Thanks.

LM

On running the test: testInverseShortTimeFourierLongSignal1() under TestInverShortTmeFourier, we use the electrocardiogram signal (length = 108000). On using a frame length of 6000, the tests pass; however, if I use a length of 20000, the test fails since the array length differs. The result has a length of 100000 while the expected is 108000.

The smooth result is different from matlab.

There are 100 point smooth result in jdsp and matlab.

Red line is jdsp's result ; Green line is matlab's smooth result.

The Jdsp's smooth result is faster 50 pionts( half of the smooth window ).

data files are :

data_Files.zip

Hi
I tried to use the jDSP 0.5.0 library in an Android projet.
I added the dependencies :
implementation('com.github.psambit9791:jdsp:0.5.0')
But I had the following error :

Duplicate class org.apache.maven.surefire.report.SafeThrowable found in modules 
jetified-common-java5-2.19.1 (org.apache.maven.surefire:common-java5:2.19.1) and 
jetified-surefire-api-2.19.1 (org.apache.maven.surefire:surefire-api:2.19.1)
Duplicate class org.apache.maven.surefire.report.StackTraceWriter found in modules 
jetified-common-java5-2.19.1 (org.apache.maven.surefire:common-java5:2.19.1) and
jetified-surefire-api-2.19.1 (org.apache.maven.surefire:surefire-api:2.19.1)

I tried to exclude the dependencies with the following gradle configurations :

    implementation('com.github.psambit9791:jdsp:0.5.0') {
        exclude group: 'org.apache.maven.surefire', module: 'common-java5'
    }

or

    implementation('com.github.psambit9791:jdsp:0.5.0') {
        exclude group: 'org.apache.maven.surefire', module: 'surefire-api'
    }

But I add the following error :

Execution failed for task ':mylibrary:mergeDebugAndroidTestJavaResource'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.Workers$ActionFacade
   > More than one file was found with OS independent path 'META-INF/DEPENDENCIES'.

Could you please help me to solve this problem ?
Best regards,
Franck

Hello,
Currently the performance of the Discrete Fourier in Java is sub-par, is it possible to integrate the FFTW if it is installed on the system to use it as it is backed by C? It runs in milliseconds compared to minutes on Java when running a 1 million point signal

Thanks in advance

Dear JDSP author,

Thank you for your contributions to this Java library! It's very helpful with very good usability. I like it very much! I noticed you now have a PCA algorithm in your code, can I request an ICA algorithm for your next update, please? It would be very helpful for analyzing multiple channel data in Java/Android platform.

Best wishes,

A big fan of JDSP

https://github.com/psambit9791/jDSP/wiki/WAV-File-IO

Wav objWrite = new Wav();
int channels = 2;
objWrite.putData(signal, channels, "int", outputFileName); 

The second parameter of putData() should be the sampling rate I think.

objWrite.putData(signal, sampleRate, "int", outputFileName);