pithus / bazaar Goto Github PK

Add an option to compare analysis of two samples based on a diff of the json report or something

Implement public feeds of trends

• Most samples per public yara rule
• Most bookmarked samples
• Top 5 recent samples
• Top 5 TI analysis

Similar to VirusTotal, please check the HASH of the to-be-uploaded file, before attempting the upload. It is possible that you have already received a certain APK file before. So the rest of the uploads from many users would be redundant. This will go hard on both the website server as well as those non-first-uploader users (especially those with not-so-fast internet connections who need to wait a lot).

This way, if you have already analyzed that file before, you can simply show the results of that previous analysis each time a new user intends to upload a file, simply by checking the HASH of the to-be-uploaded file (instead of really doing the upload).

I know now that Pithus does check the HASH upon receiving a file on the server, but I really think this HASH-checking should be client-side. It should be checked Not AFTER the file was uploaded by the user, but BEFORE upload has begun (immediately when the user selects a file to upload).

Please have a look at VirusTotal: It downloads a small piece of software to the client's device, and calculates the HASH right on the very client side.

This way, VirusTotal saves a lot of (unnecessary and redundant) internet traffic for both the users and itself. Also, it would be extremely helpful for many users who don't have very fast upload connections.

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

In APK Analysis > Receivers, show which intent/actions/filters apply to the receiver.

We'd like to run a Pithus instance, but with our own branding ♥

Hello
i connect to Github it's ok. next, i connect with the same browser to https://beta.pithus.org/ and when i upload a package and i get this error:

1. Forbidden (403)
   CSRF verification failed. Request aborted.
   You are seeing this message because this HTTPS site requires a “Referer header” to be sent by your Web browser, but none was sent. This header is required for security reasons, to ensure that your browser is not being hijacked by third parties.
   If you have configured your browser to disable “Referer” headers, please re-enable them, at least for this site, or for HTTPS connections, or for “same-origin” requests.
   If you are using the tag or including the “Referrer-Policy: no-referrer” header, please remove them. The CSRF protection requires the “Referer” header to do strict referer checking. If you’re concerned about privacy, use alternatives like <a rel="noreferrer" …> for links to third-party sites.

Why it is ok to log to github and not to https://beta.pithus.org/ please?
Regards

Example:

On it.

Build and extend the hunting section to set up an entire workspace allowing users to track, bookmark sample, generate reports etc.

Workspace:

• generation of reports from markdown and one or more samples
• hunting capabilities with bookmarks and Yara searches
• history of the results of one searches (1d, 1w, 1m...)
• dynamic analysis (R2frida inside pithus)

sections:

• investigation/projects : group samples and have notes capacities + report
• saved searches : tag/notes + link
• hunting: Yara
• bookmarks : bookmarked with notes (Show bookmark to user)

Add the following information in the similar sample block for better pivoting:

• app_name
• analysis score

When looking for similar samples under the threat intel tab, the first one is always the current sample shown with 100% similarity to itself; it would be best to hide it

Hi!

I am currently working on my local DB and noticed that when a samples has only one similar sample, it doesn't show in the threat intel tab "Similar Samples" although it is shown in the index with the nice cards.

This line prevents similar samples when the array in >1. It was probably made to prevent the same sample to be displayed however, when testing on local, I noticed that the main sample is not part of the similar_samples array.

I'd like confirmation that this behavior can also be observed with the main version of the code, so I can maybe fix that.

Thank you!

I've noticed that in the case of some samples where VT has no detection but there are other detection, the display on threat level is inconsistent between the card and the report. The reproducibility is quite low (need a non detected VT but with malicious things).

My idea is to have the same display of threat level on the card and on the report.

Sample: https://beta.pithus.org/report/3446ccbf96a485c8a95febd5d81d45010f2ac2b6ef48b8531ce07a209ccd4d73

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

Show the MainActivity

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

In Behaviour analysis > threat analysis, show where in the code Quark locates the behaviour.

65 Mb is limiting. for example, impossible to analyse popular apps like twitter
Merci

I'm thinking about a way to be able to "follow" public yara rules. I.e. to be able to have a list or matches for a specific rules, not necessarily being able to see it, but I'm for it.

The idea behind it is to follow the activity of specific groups and give a bit sense of community in threat hunting on Pithus.

Private Yara rules will of course not allow such a feature.

https://twitter.com/abuse_ch/status/1358778140596838401

As a user, I want to be able to save bookmarks of samples. Those bookmarks will be set in the workspace section.

https://github.com/quark-engine/quark-engine

On it

Hi,

I am trying to install your framework but the docker compose installation doesn't seem to work.
I am on a Mac, running Docker version 3.3.2 and compose 1.29.1
It is a brand new install of Docker, with no images, nothing.

I git clone bazaar
I ran the docker-compose -f local.yml build
Everything seems to go ok...
However I noticed it starts with:
redis uses an image, skipping
mailhog uses an image, skipping
elasticsearch uses an image, skipping
mobsf uses an image, skipping
kibana uses an image, skipping
minio uses an image, skipping
Building postgres
....
then it build everything and there are no error.

But if I then go to the URL listed in your instruction (http://localhost:8001) nothing happens...

If I look in the docker dashboard, I can see no apps have started.
But there are 3x images:

• backend_local_worker
• bazaar_local_django
• bazaar_production_posgres

If I start them manually, they each exit with a very quick error:

• backend_local_worker
  Error:
  /entrypoint: line 10: POSTGRES_USER: unbound variable

• bazaar_local_django
  Error:
  /entrypoint: line 10: POSTGRES_USER: unbound variable

• bazaar_production_posgres
  Error:
  Database is uninitialized and superuser password is not specified.
  You must specify POSTGRES_PASSWORD to a non-empty value for the
  superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
  You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
  connections without a password. This is not recommended.
  See PostgreSQL documentation about "trust":
  https://www.postgresql.org/docs/current/auth-trust.html

So it looks I am missing some steps...
I am not too familiar with docker and docker-compose but should I be running some docker-compose up first?

Any help welcome!
Thanks.
Bugs.

Missed a possibility of having various MainActivities: https://beta.pithus.org/report/ae05bbd31820c566543addbb0ddc7b19b05be3c098d0f7aa658ab83d6f6cd5c8

Need to put them in a proper list

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

In Fingerprints > APKiD, show which part of the code implements the features that are detected.

I have done my first analysis locally... very impressive results!!! So thanks for that making this tool available.
I did notice an error though in the GUI it says "An error occurred during the analysis, we have been notified"

I can see that on your beta online page too

But looking at the logs in docker I can't see any error.

So a few questions:

• What went wrong and how do I fix it? does it mean I am missing some info from the analysis?
• Who has been notified and with what information? could this be optional? some users of your tool may want to control what information is leaving their computer/about their analysis.

Thanks.
B.

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

For the similarity, problem is we don't know what is similar between the samples. Suggestion: using Quark's chart radar to compare malware labels.

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

Reduce false positives, by ignoring 3rd party SDKs. DroidLysis does it apparently.

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

In Behaviour analysis > Permission analysis, show which section of the code request the permissions instead of the description of the permission.

note: we could keep the description as a beginner friendly enhancement.

CC: https://attack.mitre.org/matrices/mobile/android/

Clean install with docker-compose -f local.yml up results in following error:
ModuleNotFoundError: no module found "tld"

add the option to look for samples that have a lot of similar samples

When pivoting on dexofuzzy hashes of similar samples, it would be interesting to add the dendrogram tree for genetic analysis of those similar samples to give a better insight of the similarities.

Case:

• The user is on a sample page and click log in
• The user is logged in
• The user is redirected to the sample page (current behaviour: redirection to home)

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

Detect if a sample is packed and unpack it.

note: maybe there is something to do with jadx here.

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

In Control Flow Analysis, show source code that can be parsed and not images. Adding also the possibility to highlight key lines (MobSF does it).

Elasticsearch is eating a lot of CPU and RAM, it would be great to be able to use something else by default (like zink, sonic, manticore, …), since not everyone wants to operate on millions of APK.

Most of the items shown on the website are quite technical and not at all easy to understand for the average user (like myself). I mean at least 90% to 95% of the whole content of your website is vague or worse, not at all understandable (at least to me). I am not just throwing a number; I carefully read the whole report of various aspects of the analyses shown by Pithus for a couple of APK files, and thought I have no clue what the website is actually talking about.

So if possible, please create popup (as well as hyperlinked) help messages for almost every item shown on the website, so that a confused user can understand what is going on.

For example, every text item can have a small circle with an "i" in the beginning of it, hovering which will explain that item.
The items that look like buttons (for example, they are encircled within square frames) can themselves trigger popup messages upon hovering.
Or they can be clicked, so that a new browser tab opens, showing details about that particular button.
There can also be some links within the popup help messages for those who want to read much more details about that particular item shown on the website.

Pithus is being recommended on many forums and circles to average or below-the-average (technically illiterate) users.
I think you can (or perhaps should) put a disclaimer on your website about the fact that Pithus is only for professionals and is not supposed to be understandable by the general public, in the first place. That may save many users a lot of headache and confusion.

Besides that, putting popup (or link) help messages wouldn't hurt anybody. It wouldn't reduce the efficiency of your website, or wouldn't change its purpose. It is just a free extra (but very awesome) feature at no cost. I know it will drain your time to implement it, but I think it is worth investing in. This is because such popup help boxes will (1) make your website become understandable to a MUCH broader audience, and therefore, (2) it would help even those few technical superusers fight and investigate APK malware much more effectively because a larger pool of users means much more APK samples submitted to your website.

If you remove a large part of your audience (the general public), you will automatically lose a lot of potential malware that could have been otherwise submitted to Pithus and its professional superusers.

Looking into it.

Adding androguard-yara plugin and generating report to feed androguard module.

Since androguard already used in project this will be easy to implement. Will it be usefull than current yara module ? That can be discussed.

Features of androguard-yara is here . Most of features can be search via pithus search section.

Adding following links to be used as a reference point.

http://pavelsimecek.cz/custom-matching-of-koodous-yara-rules/
https://github.com/eybisi/hacky-yara-androguard

Add saved searches as example on the landing page to help users to start up their work with Pithus.

See the Network analysis of https://beta.pithus.org/report/7f4dda8c5131fe4f43112f696c1733bb93d3a9f6bef9e1aa4c87891b49874970

We should probably fix that directly in MobSF.

Wouldn't it be nice to have a darker version of Pithus? :)
Plus it's great for reducing eye strain!

From: https://cryptax.medium.com/investigating-android-malware-with-pithus-17d2143cc528

Compute the different names from the AV from VT and show the compiled results.

Hi,

I just installed a new test env with docker compose.
The app starts
I can get to the home page
I click on Sign In then Signing with Github and then I get the following error:

Environment:

Request Method: GET
Request URL: http://localhost:8001/accounts/github/login/?process=login

Django Version: 3.0.11
Python Version: 3.8.10
Installed Applications:
['whitenoise.runserver_nostatic',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.admin',
'django.forms',
'crispy_forms',
'allauth',
'allauth.account',
'allauth.socialaccount',
'allauth.socialaccount.providers.github',
'rest_framework',
'rest_framework.authtoken',
'corsheaders',
'django_q',
'minio_storage',
'bazaar.users.apps.UsersConfig',
'bazaar.core',
'bazaar.front',
'compressor',
'django_extensions']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'corsheaders.middleware.CorsMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.common.BrokenLinkEmailsMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/usr/local/lib/python3.8/site-packages/allauth/socialaccount/providers/oauth2/views.py", line 77, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/allauth/socialaccount/providers/oauth2/views.py", line 105, in dispatch
app = provider.get_app(self.request)
File "/usr/local/lib/python3.8/site-packages/allauth/socialaccount/providers/base.py", line 49, in get_app
return adapter.get_app(request, self.id)
File "/usr/local/lib/python3.8/site-packages/allauth/socialaccount/adapter.py", line 207, in get_app
app = SocialApp.objects.get_current(provider, request)
File "/usr/local/lib/python3.8/site-packages/allauth/socialaccount/models.py", line 32, in get_current
app = self.get(sites__id=site.id, provider=provider)
File "/usr/local/lib/python3.8/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 415, in get
raise self.model.DoesNotExist(

Exception Type: DoesNotExist at /accounts/github/login/
Exception Value: SocialApp matching query does not exist.