Code Monkey home page Code Monkey logo

netcrawl's Introduction

Build Status Coverage Status Documentation Status

NetCrawl

Network Information Gathering Made Easy

Netcrawl is a tool designed to discover and poll one or more devices, inventory them, and then provide useful data on the processed devices.

This package is still in development.

Features

  • Switchport Tracing: Discover which devices and interfaces have seen a particular MAC
  • Wireless Audit: Discovers likely matches for rogue wireless devices among physically connected devices on a subnet
  • MAC Audit: Discover potential unauthorized switches on your network
  • SSH and Telnet connections to network devices
  • Automatically backs up device configurations
  • Stores a neighbor database to find layer two connection mappings
  • Auto-detect system type of newly discovered devices
  • Works with Nmap to allow for discovery of both neighboring and seperated devices
  • Securely stores credentials using keyring and cryptography
  • Can use multiple credentials in case the first fails
  • Stores device inventory using a PostgreSQL database
  • Offers a single device scan to quickly get data on one device
  • Concurrently runs multiple subprocesses to quickly scan devices
  • Multiple netcrawl top-level processes can run concurrently to scan different network segments (do not use -c while doing this), or to run an Nmap scan and inventory hosts as they are discovered.

Example

Scan one host with no logging output
C:\netcrawl>run.py -sS -t 10.1.120.1 -v0

Device Name:       my-device-dist-1
Unique Name:       MY-DEVICE-DIST-1_EC032
Management IP:     10.1.120.1
First Serial:      Name: [Switch System], Desc: [WS-C4500X-32], Serialnum: [JAE14350G30]
Serial Count:      28
Dynamic MAC Count: 920
Interface Count:   88
Neighbor Count:    22
Config Size:       26573

+---------------------------+------------------------+----------------------+-------------+
| Neighbor Name             | Source Interface       | Platform             | IP Address  |
+---------------------------+------------------------+----------------------+-------------+
| DVCOPS-MIS-1              | TenGigabitEthernet1/1  | cisco WS-C3750-48P   | 10.1.220.11 |
| DVCOPS-MIS-2              | TenGigabitEthernet1/2  | cisco WS-C3750-48P   | 10.1.220.10 |
| DVCOPS-sceast-sc01        | TenGigabitEthernet1/3  | cisco WS-C3850-48P   | 10.1.139.12 |
| DVCOPS-sccent-sc01        | TenGigabitEthernet1/4  | cisco WS-C3850-48P   | 10.1.139.11 |
| DVCOPS-dcgsc-sc01         | TenGigabitEthernet1/16 | cisco WS-C3850-48P   | 10.1.139.26 |
| DVCOPS-wlcprm-vd01        | TenGigabitEthernet1/17 | AIR-CT5520-K9        | 10.1.139.51 |
+---------------------------+------------------------+----------------------+-------------+
Locate a device on the network
C:\netcrawl>locate_mac.py 00FEC89232B0

MAC:  00FEC89232B0
Manufacturer:  Cisco ,  Cisco Systems, Inc
+-----------------------+---------------------+-----------------------+
| Device Name           | Interface           | CDP Neighbors         |
+-----------------------+---------------------+-----------------------+
| DVCOPSDS01            | Ethernet2/24        | DVCOPSMGT1            |
| DVCOPSMGT1            | GigabitEthernet0/23 | None                  |
| DVCOPS-mgmt-sd01      | FastEthernet1/0/39  | DVCOPSDS01            |
+-----------------------+---------------------+-----------------------+

Built With

  • Netmiko - SSH and Telnet connection manager
  • Manuf - OUI lookup

Authors

  • Wyko ter Haar - Initial work - Wyko

netcrawl's People

Contributors

wyko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

netcrawl's Issues

Add 'Update Existing Device' Functionality

Objective

When performing a scan with a pre-existing database, the application will identify pre-existing devices and update them rather than adding duplicate entries.

Possible Implementation

After polling a device, search the database for a duplicate unique_name.

If found:

  • Full Replace (preferred): Cascade delete the old entry and add the new one.
  • Update: Add the device_id of the match to the polled device. Compare each entry and update them.

Concerns

The application may be interested in tracking changes in device state over time. What would be the best way to do that?

Netmiko establishes SSH connection but no data extraction

Hello, I'm using python 3.6 with inside a virtualenv and I can't get information to display whatever the platform (cisco_nxos, cisco_ios or cisco_xe).

python run.py -sS -t 172.28.46.253 -p cisco_nxos
main.main : #3 Start new run
main.main : #3 ##### Starting Single Run #####
main.single_run : #3 Processing connection to 172.28.46.253
base_device.process_devices : #4 Processing device
cli.connect : ? SSH auth error to 172.28.46.253 using admin, In
cli.connect : ? SSH auth error to 172.28.46.253 using admin, In
cli.connect : #4 Successful ssh auth to 172.28.46.253 using administrator, In
base_device.process_devices : ? _enable - Error: 'CiscoNxosSSH' object has no attribute 'ip'
main.single_run : ? Connection to 172.28.46.253 failed: 'CiscoNxosSSH' object has no attribute 'ip'
Device processing failed
main.main : #3 ##### Single Run Complete #####

Could you please help to point what is wrong?

Support for Juniper

Please let me know if you'd like to incorporate support for JUNOS devices in the future, I would be happy to help out.

Help Wanted to make Web Front-End

Problem

Netcrawl currently polls devices properly, but once the data is in the database there's no convenient way to interact with it outside of an SQL query

Objective

Implement a web-based front end to allow users to interact with the polled data. Obviously, there are a lot of ways to do this, but I'm not a web developer and I am open to ideas and help.

Switchport Tracing for MAC's

Each device will save it's mac address table per switchport. The result is that both the directly connected device and all devices in the same layer two path storing an end device's MAC address.

MAC audit module

The MAC Audit module will take a csv (or any other input, as added in the future) and to compare it with the database of polled mac addresses. The purpose is for things like Rogue AP audits, where a rogue device may be detected, its MAC polled, and then compared against macs connected to switchports for a similar OUI.

For each csv mac address:

  1. Get a list of all polled MAC addresses in the given subnet
  2. Compare the csv MAC against the polled MACS
  3. Generate a confidence rating that indicates how similar the MAC's are

The csv needs at minimum the mac column as well as either a network_ip column or an ip and subnet column.

Challenges:

  • Currently, MAC addresses are stored by interface. The interface will not always have an associated subnet, due to being a layer 2 interface. The module should intelligently use the device's subnets when an interface specific one isn't available.

Add PyTests to improve overall coverage

Objective

All parts of the program need more tests. Optimal test coverage should reach over 80%.

Concerns

A lot of the application depends on a populated database or a connection to a live device. How do we test without a live network to test on? Maybe through pre-downloaded samples of device output?

Support Keyboard Interrupts

Problem

Currently, when the application runs and is interrupted, there is no code to gracefully exit execution. Especially when scanning with subprocesses, as they have a tenancy to stick around after the interrupt.

Objective

Implement support for interrupts.

Ideas

  • Using the signal module to get the subprocesses to ignore the KeyboardInterrupt, then terminating the workers normally

Create Fake Interfaces, Devices, and Databases for Testing

Objective

For testing purposes, we should have fully populated structures to work with. For this, we need helper functions which create on-demand, populated interfaces and devices, as well as filling the inventory and main databases with fake data.

When appropriate, these fake structures can be used alongside pre-made ones in situations where exactly known data is not needed.

Possible Implementation

Using the Faker package will expedite this

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.