QUESTION about seq2pat HOT 7 CLOSED

Thank you @NaifahNurya for your interest.

The easiest would be to find the frequent patterns using Seq2Pat and then to do a quick post-processing to remove undesired patterns based on custom preferences like the above.

from seq2pat.

@skadio , Thank you for quick reply, let me work on it.

Also I have another issue, when I use the Seq2Pat on few dataset (with short length in each sequence) i can get the result. However in a large dataset (with many sequences having long length in some sequences) I got the following error:

patterns = seq2pat.get_patterns(min_frequency=65)
File "C:\Users\NaifahNurya\anaconda3\envs\Seq2Pat\lib\site-packages\sequential\seq2pat.py", line 411, in get_patterns
 patterns = self._cython_imp.mine()
File "sequential\backend\seq_to_pat.pyx", line 31, in sequential.backend.seq_to_pat.PySeq2pat.mine
**RuntimeError: bad allocation**

Can you help to Identify the cause for this.
If you give me your email I can send to you a sample txt file.

from seq2pat.

Thanks @NaifahNurya .

This might be caused by the memory exhaust issue when dataset contains many long sequences. It seems relevant to a previous discussion #14.

To alleviate such memory issues, I would suggest to add constraints to further reduce the number of search paths. Also what we can do is to limit the number of columns, or apply data sampling before the mining, to better work with the memory resources.

from seq2pat.

Sampling the columns/limiting columns sound reasonably and might be required. You can start with small samples 10-20 columns and see how it behaves in your application before including more.

@takojunior you have an interesting suggestion on adding a span constraint to limit the search on columns. Do we have an example of how to add that constrained somewhere?

from seq2pat.

Right, so one way is to enforce a span constraint to an attribute created by the order of items, e.g. [A, B, C, D] has the attribute [0, 1, 2, 3]. Enforcing a maximum span constraint will control the length of mined patterns in mining, and thus reduce the search space.

How to enforce such constraint can be referred to this example notebook: dichotomic_pattern_mining.ipynb. @skadio @NaifahNurya

from seq2pat.

@takojunior and @skadio , Thank you very much for this suggestion, let me work on it then I will share the feedback.

from seq2pat.

Closing the issue per discussion. Hope this helped!

from seq2pat.