top by device-type about goaccess HOT 8 CLOSED

There's already a 'browsers' module that distinguish crawlers from others and 'os' that distinguish operative systems.
The problem here is to distinguish 'desktop' from 'mobile' devices.

Maybe this can only be done by prefixing operative systems module with type or create a new module specific for this propose. Examples:
'Windows NT 4.0' would change from 'Windows' to 'Desktop - Windows';
'iPhone' with change from 'Macintosh' to 'Mobile - Macintosh';
'iTunes' with change from 'Macintosh' to 'Desktop - Macintosh';

What's the best approach for this?

from goaccess.

Not sure about this. So far it's possible to know which ones are mobile (i.e., Android, iPhone, Blackberry, etc) and which ones are desktop. Perhaps we could add a new sub node under the actual OS, however, feels like it wouldn't serve much purpose?

from goaccess.

@allinurl I think perhaps it would be a good idea to consider reopening this issue?

We would love to have device type information in the JSON output, or alternatively pure UA strings grouped somehow perhaps? I.e. It would be nice to have Windows NT 6.3; ARM (Or those with touch identifiers perhaps?) type devices not just classified as windows, but rather surface devices and so forth imo?

from goaccess.

Just a thought for the longer term - can we strive to provide a comprehensive / near complete device coverage? - if we'd have a conscience lexical parallel of Device:to:Browser even with manufactor specific / vendor specific pollutants 😄 (ms, vs). I'd approximate that a database with ~10k-18k of known devices out there would be a 97%+ coverage of all thats mobile & maybe in circulation from NetFront, HTML4, Java, MMS, SymbianOS, WAP, WEB, WHL, Windows, WindowsCE, etc...

Using a few known listings & articles (0, 1, 2, 3, 4) as guide the following lexicon may be a good start:

A complete device focused list of manufacturers & relevant providers in the global context could comprises of approximately ~ 142 with more UA's by distributor / OEM ? :

This list excludes all potential UA such as those by application, crawler, proxy, service, etc, from the 1.5+ million (& growing) already classified in some directories with others yet to emerge for vehicle / automotive (your car) and television (not sure how unique these are) systems.

I can help in the compilation of the proposed DB.

from goaccess.

@overnine assuming we still want to use the ones posted above, i.e., desktop, mobile, others then I think we could add this as a new panel. Categorizing this could be tricky. Are those the only categories we could use? Should we have tablets as well?

@aphorise having a comprehensive list of browsers and OSs would be awesome, but we may need to refactor browsers.c & opesys.c since currently those are bottlenecks. (the size of the list is proportional to the run time, which makes them slow). We could have a large list mapped in a hash but it would need to be a full match search. Or bsearch perhaps?

For the record, this may be related to #152.

from goaccess.

@allinurl i only use goaccess for the json output, i dont really have any considerations in terms of usability of the curse interface. Also the most comprehensive resource i know of which handles this is https://github.com/serbanghita/Mobile-Detect

from goaccess.

@allinurl - I actually think we can do this adaptively / build on a global browser & device context. So instead of having fixed size table / bsearch or hash - have a tree that spans / adaptively increases based on new / differing UA's that read and that would only get a naive comparison against a master UA-DB / record once for every-thing unknown or new.

So basically the UA-DB (lexicon) can have two focuses:

• by device (as per earlier comment & proposal) - that'd be a potential look up of 1 in ~ 18``k (estimate).
• by service or anything else (as per known & remaining lot in excess of a million) - that'd be a potential look up of 1 in ~ 1.4``m (estimate).

The ideal UA lexicon would be ( adaptive-UA) adaptively built from the data-set (logs) being parsed and could be an O(n) algorithm if its stored in ahash or a O(n^2) if its in some other from (btree, link-list); either-way the scope of any set in most cases should be less than < complete-UA O(n) or O(n^2)thats the total space / list of everything thats known & can be targeted.

What'd be even more smart for real-time stats & scenarios where adaptive-UA can / may grow indefinably would be to have a stack-limit to a reasonable size 2^16 or 65535 - thereby dropping any single / one time occurrences; all that would be needed is a FIFO / FILO (first-in-first-out / first-in-last-out) precedence on the lower portion (say a reserve of 255) so as to always accommodate and record new / single 1 time agents that would always register and would be visible in the most recent time-frames. Ditto governing rules for what can move up the adaptive-UA

I hope I'm clearing conveying the intent.

from goaccess.

For reference the RAM / memory space requirements to hold all potential UA can be expressed as:

This is on the assumption that no UA are alike (all unique for 2^16-1) and that none shall ever exceed 64``Kbytes which is unreasonably high & frankly even graver than stupid if neared or exceed. The most obvious candidates of illogical uses which tend to exceed the common 512-1024 (bytes) are for example _Internet-E_xplorer (IE) or other debug or build specific text that may be embedded as part of the UA identifier - even these tend to be < 8``Kbytes at worse.

A few other rules can also be applied as objective determinant for anything not matching which can have a final criteria scope of device &/or browser :

• DEVICE: 1 of 143 manufacturer-UA
• BROWSER:1 of < ~ 64 application-UA chrome/blink et al, firefox et al, geko, trident / IE / MOSIAC, Netscape, Opera, Safari, Webkit, etc.

^^^ the same approach can also be applied initial for new / unknown UA so as to further scope the lookup by device-ua first then other-ua otherwise fallback to the first / best initial detection (if applicable).

from goaccess.