Introduction: (Problem description)
There are a large number of reasons why it’s a good idea to keep apprised of when, where, why, and how a computer is used on a frequent basis. There could be critical indicators of compromise “IOCs” and/or indicators of attacks “IOAs” that could possibly help network defenders protect against such activities or perform remediation them after the fact. The proper logging of all user and computer activities is crucial to any network defense program and should be carefully implemented in addition to password policy enforcement.
All scripts in this post can be found at:
https://github.com/SmokeDog88/InfoSec_Ops
Computer logs lie deep inside of a computer’s hard drive residing on each individual computer and server alike and getting access to those data repositories can be a daunting task. Having to log into 5, 10, 20 or more computers multiple times throughout the day is not manageable nor feasible let alone cost efficient.
Five common reasons to monitor computer logs:
Superior data breach detection:
- Real-Time alerts through activity monitoring and PC resource utilization thresholds
Reduced time-to-repair TTR:
- Dates, times, and previous state information helps establish timelines, which in the end helps push post incident investigations towards a speedier recovery.
Low-cost regulatory compliance:
- Faster audits through reduced data and interface management.
Efficient vendor activity auditing:
- Better enforcement of third-party SLAs through activity enforcement and transparency.
More efficient forensic investigations:
- Being able to quickly and efficiently correlate log data provides a more comprehensive IT cybersecurity management solution.
So the solution is to automate these tasks that will in turn reduce time and costs to managing networks of any size. Enter Python scripting, a way to pre-program automated scripting that can simultaneously log into multiple machines and transfer log file data “securely’ to a centralized machine, designated by the network administrator, to be parsed at a later time for potential IOCs and/or IOAs.
Major features and solution
Required Python Modules
Python modules are prefabricated definitions and statements that provide additional functionality after importing them into Python scripts. The following modules and their associated functionality are outlined below:
SSH (Parameko):
A remote login program that establishes a secured connection prior to sending username and password data onto an untrusted network or a trusted network as an added layer of security.
import paramiko
ssh = paramiko.SSHClient()
ssh.connect(‘127.0.0.1′, username=’jesse’, password=’lol’)
OS:
Provide a portable way of performing operating system dependent functionality such as file I/O.
Import os
print(‘Your current working directory is: ‘, os.getcwd())
Socket:
A low level-networking interface that provides a direct transliteration of various system calls and library interfaces for communication sockets. Socket is a low level networking interface that will bind an IP address to a specific port, which subsequently is placed into listen where it will wait for connections:
socket.socket(socket.AF_NET, socket.SOCK_STREAM)
socket.setdefaulttimeout(1)
Argparse:
Provides the ability to integrate user command line option, arguments and friendly help user help context menus.
import argparse
parser = argparse.ArgumentParser(description=’Process some integers.’)
parser.add_argument(‘integers’, metavar=’N’, type=int, nargs=’+’,
help=’an integer for the accumulator’)
parser.add_argument(‘–sum’, dest=’accumulate’, action=’store_const’,
const=sum, default=max,
help=’sum the integers (default: find the max)’)
args = parser.parse_args()
print(args.accumulate(args.integers))
Subprocess:
Provides the ability to spawn new process and redirect their outputs. I will use this module to help vet user input from the keyboard as a form of input sanitation in an effort to thwart shell injection attempts.
try:
shell_command = ‘ ‘.join([‘ls’, ‘-l’, shell_input])
print(‘shell command: ‘ + shell_command)
shell_output = subprocess.check_output(shell_command)
print(shell_output) # print the input argument’s command to screen
except subprocess.CalledProcessError:
print(‘Ooops, Something went awry!’)
except FileNotFoundError:
print(‘No such file or directory ‘, shell_input)
Platform:
Provides me with the ability to access each platform’s underlying identifying data. It’s critical to know which operating system and version is running otherwise a specific block of code may be rendered non functional.
print(‘Your operating system is: ‘, sys.platform)
print(‘Your platform release is: ‘, platform.release())
print(‘Your platform version is: ‘, platform.version())
Hashlib:
Provides a common interface to multiple message digest algorithms and secure hashes. My hash crack program, as part of the password strength check, will use this module.
For MD5: if hashlib.md5(password).hexdigest() == hash:
For SHA-1 if hashlib.sha1(password).hexdigest() == hash:
For SHA-256 if hashlib.sha256(password).hexdigest() == hash:
String:
Provides the ability to perform common string operations that will be used during password strength checks where strings will be used to iterate through user passwords.
To strip hash from string hash = string.strip(hash)
To strip word from string pswd = string.strip(word)
Re:
A python module that provides the ability to match regular expressions, which will be used as part of the password strength check program.
a = r'[a-z]’ # Regex to check for any capital letters in the specified range
ua = r'[A-Z]’ # Regex to check for any lower case letters in the specified range
n = r'[0-9]’ # Regex to check for any numbers in the specified range
sc = r'[!@#$%^&*]’ # Regex special characters in the specified range
pcapy
The pcapy library will provide me with the capability to capture and read pcap files online and offline. This process will be performed in two seperate processes, data capture and log parse.
import pcapy
pcap_file = pcapy.open_offline(“file.pcap”)
Ping Sweeps
Ping sweeps are essentially layer 3 echo request/response messages that will be used to identify whether a service is active and listening on a particular host/s. It’s important to establish if a socket is active and listening before any communications can take place. If an echo reply is received from a host, it indicates that its port is active and listening and ready for communications, in my case, secured file transmissions.
Data needs
Data the will need to be captured and manipulated to include packet capture files “pcap”, which are files that contain snapshots of communications from and to specific hosts. User activity logs will need to be captured and parsed, typically in the form of a text file, for suspicious activity. PC resource activity will need to be captured, logged, and transmitted in the form of text files. Through the use of the OS Python module, I will have the ability to manipulate output data captured from remote hosts by saving it as text files. Text files are small and easy to encrypt allowing for fast data transfers.
My Python scripts will also need the ability to interact with raspberry Pi devices and Mac OS desktops by establishing secure TCP connections using secured shell login with a username and password combination. After establishing a secure connection, my python script will need to interact with various operating systems on those devices, Linux on the Pi and Mac OS Sierra on the Macs. This will require the use of python’s OS module and its associated file I/O operations, such as, opening, closing, reading, and writing, creating, and appending new data to existing documents.
This data will come from various computing resources, such as existing files on a system that includes logs and computing resource queries from active running processes, which are saved to text files by redirecting standard output to open documents. This will be performed using file I/O operations prior to transmission.
My python program will also need the ability to check for password strength in order to enforce password policy. This is a two-part process that will first, run a script and attempt to crack password hashes, and second, run a script to check the strength of existing user passwords.
Lastly, my python programs will need the ability to capture various types of program errors and providing the administrator with the ability to recover or correct them, which will be accomplished through the use of exception handling.
Network Functionality
Client/Host communication
Client connections will be established using the Parameko SSH python module and libraries. A secure login will be performed using both username and password combinations. The first step is to create a socket using the socket.socket function which will reference the socket library. For the needs of this network, the AF_INET socket will suffice in addition to socket.SOCK_STREAM to indicate communications across a TCP/IP network.
Port scanning
Upon login, a port scan will be performed using NMAP commands to identify any potentially unnecessary ports that can be closed or otherwise should not be open. The scan results will help identify any potential vulnerabilities that may exists within the system.
Network Tracing
Network traces will be performed using PCAP file packet captures with the use of the pcapy Utility. PCAP files will be parsed offline and subjugated to scrutinization for malicious activities. This process will occur in two steps using pcapy to first capture traffic and second parse said captures in an offline state.
Persistent Storage
Data will be transported in text document formats that will be stored on the client for processing. File I/O will include both reading and writing operations from parsed log files. A database used to store data remains the preferred option due to the fact that the program would be scheduled to run multiple times during the week. Having the ability to append/update a database is the ideal situation.
Exception Handling
All exceptions would need to be captured to add clarity to the flow of the program while helping in its expediency. For example, if a file doesn’t exist, or wasn’t created, or if a file failed to read/or write, are all critical exceptions that need to be conveyed to help overcome functionality issues and ultimately the overall operation of the program. Network exceptions also need to be captured, for instance, unable to open connection, socket, file transmission failures, etc.
Final program scripts
This project consists of multiple Python scripts each of which serves a specific function to be ran in a specific order. The scripts are to be run in the following order:
- Ping Sweep
- Port Scan
- Secured Login
- File I/O
- Password Crack
- Password Strength
Ping Sweep
import os
import platform
from datetime import datetime
net = raw_input(“Enter the Network Address “)
net1= net.split(‘.’)
a = ‘.’
net2 = net1[0]+a+net1[1]+a+net1[2]+a
st1 = int(raw_input(“Enter the Starting Number “))
en1 = int(raw_input(“Enter the Last Number “))
en1=en1+1
oper = platform.system()
if (oper==”Windows”):
ping1 = “ping -n 1 “
elif (oper== “Linux”):
ping1 = “ping -c 1 “
else :
ping1 = “ping -c 1 ”
t1= datetime.now()
print “Scanning in Progress”
for ip in xrange(st1,en1):
addr = net2+str(ip)
comm = ping1+addr
response = os.popen(comm)
for line in response.readlines():
if(line.count(“TTL”)):
break
if (line.count(“TTL”)):
print addr, “–> Live”
t2= datetime.now()
total =t2-t1
print “scanning complete in ” , total
Port Scan
import socket, subprocess,sys
from datetime import datetime
subprocess.call(‘clear’,shell=True)
rmip = raw_input(“\t Enter the remote host IP to scan:”)
r1 = int(raw_input(“\t Enter the start port number\t”))
r2 = int (raw_input(“\t Enter the last port number\t”))
print “*”*40
print “\n Mohit’s Scanner is working on “,rmip
print “*”*40
t1= datetime.now()
try:
for port in range(r1,r2):
sock= socket.socket(socket.AF_INET,socket.SOCK_STREAM)
socket.setdefaulttimeout(1)
result = sock.connect_ex((rmip,port))
if result==0:
print “Port Open:–>\t”, port
# print desc[port]
sock.close()
except KeyboardInterrupt:
print “You stop this “
sys.exit()
except socket.gaierror:
print “Hostname could not be resolved”
sys.exit()
except socket.error:
print “could not connect to server”
sys.exit()
t2= datetime.now()
total =t2-t1
print “scanning complete in ” , total
Secured Login
import paramiko
ssh = paramiko.SSHClient()
ssh.connect(‘127.0.0.1′, username=’jesse’, password=’lol’)
File I/O
import os
import argparse
import sys
import subprocess
import platform
parser = argparse.ArgumentParser(description=’Search for specific UIDs in passwd file’)
parser.add_argument(dest=’filenames’, metavar=’filename’, nargs=’*’)
parser.add_argument(‘-i’, ‘–input’, metavar=”, dest=’shell_input’, required=False)
parser.add_argument(‘-u’, ‘–user’, required=True, dest=’usernames’, action=’append’, help=’Usernames to search for’)
parser.add_argument(‘-o’, ‘–outfile’, metavar=”, dest=’outfile’, action=’store’, help=’output file’)
group = parser.add_mutually_exclusive_group()
group.add_argument(‘-v’, dest=’verbose’, action=’store_true’, help=’verbose mode’)
group.add_argument(‘–speed’, dest=’speed’, action=’store’, choices={‘slow’, ‘fast’}, default=’slow’, help=’search speed’)
args = parser.parse_args()
user = args.usernames
print(‘Your operating system is: ‘, sys.platform)
print(‘Your platform release is: ‘, platform.release())
print(‘Your platform version is: ‘, platform.version())
print(‘Your current working directory is: ‘, os.getcwd())
if __name__ == ‘__main__’:
for name in args.usernames:
if sys.platform == ‘darwin’ or ‘linux’ or ‘freebsd’:
print(‘You entered username: ‘, name)
with open(‘/etc/passwd’, “r”)as f:
for line in f:
if line.split(“:”)[0] == name:
with open(args.outfile, ‘a’) as f1:
f1.write(line)
print(‘Your output file has been updated! ‘)
shell_input = args.shell_input
try:
shell_command = ‘ ‘.join([‘ls’, ‘-l’, shell_input])
”’Here, the program runs long listing on the file revealing user permissions and group/file memberships.”’
print(‘shell command: ‘ + shell_command)
shell_output = subprocess.check_output(shell_command) # when adding shell=True, shell injection becomes possible.
”’Here we execute the sanitized input without shell=True.
When shell=True is used, shell injection is possible as proved by trying both methods.”’
print(shell_output) # print the input argumante’s command to screen
except subprocess.CalledProcessError:
print(‘Ooops, Something went awry!’)
except FileNotFoundError:
print(‘No such file or directory ‘, shell_input)
Password Integrity Checks
Password Crack
import string
import hashlib
import sys
hfd = open(‘hashes.txt’, ‘r’) # open the file with hashes
afd = open(‘answers.txt’, ‘w’) # open the file to write answers to
for hash in hfd.readlines():
wordlist = open(‘/Users/miguelbigueur/Library/Mobile Documents/com~apple~CloudDocs/PyCharm Files/untitled/OPSC 540/Week 4/password.lst’)
for word in wordlist.readlines():
hash = string.strip(hash)
pswd = string.strip(word)
”’hashes the password using md5 then compares to hash in rainbow table”’
if hashlib.md5(pswd).hexdigest() == hash:
print (pswd)
”’writes the “standard output” to the answers.txt document”’
sys.stdout = afd
wordlist.close()
afd.close()
Password Strength
import string
import re
afd = open(‘answers.txt’, ‘r’)
la = r'[a-z]’ # Regex to check for any capital letters in the specified range
ua = r'[A-Z]’ # Regex to check for any lower case letters in the specified range
n = r'[0-9]’ # Regex to check for any numbers in the specified range
sc = r'[!@#$%^&*]’ # Regex to check for any special characters in the specified range
for word in afd.readlines():
pswd = string.strip(word)
good_len = False
lower = False
upper = False
num = False
special = False
score = 0
if len(pswd) > 6: # Check to ensure minimum number of characters is greater than 6
good_len = True
score += 1
if re.search(la, pswd): # lowercase alpha search
lower = True
score += 1
if re.search(ua, pswd): # upper alpha search
upper = True
score += 1
if re.search(n, pswd): # numbers search
num = True
score += 1
if re.search(sc, pswd): # special characters
special = True
score += 1
print(“results for %s: score: %d” % (pswd,score))
print(“good_len: %s, lower: %s, upper: %s, num: %s, special: %s” % (good_len, lower, upper, num, special))
Conclusion
As is the case with any network of any size, collecting and analyzing data from multiple machines can prove to be a rare feat of success; therefore, it’s critical for any network administrator to have the ability to automate system administration tasks through the use of scripting. Python is one of the most popular system administration scripting tools in use today and is growing in popularity for its easy learning curve and simplicity of design.
Having the ability to write a Python scripts can not only save an organization time and money but also help reduce post incident recoveries and investigations, reduce time and costs associated with regulatory compliance, provide superior data breach detection, reduced time-to-repair, and efficient vendor auditing. Through the use of scripting, Python reduces network management costs by automating tasks that would otherwise need to be performed by several individuals, such as, enforcing password policy, monitoring computing recourses i.e. CPU, GPU, memory, hard drive, and network connections.
References
Friedlander, G. (2014, January 14). 5 Reasons Businesses Should Monitor Privileged Computer User Activity. Retrieved March 16, 2017, from http://sandhill.com/article/5-reasons-businesses-should-monitor-privileged-computer-user-activity/
Beazley, D. M., & Jones, B. K. (2013). Chapter 11.2. In Python cookbook. Beijing: O’Reilly.
Python for Beginners. (2013, March 7). Port scanner in Python. Retrieved March 15, 2017, from http://www.pythonforbeginners.com/code-snippets-source-code/port-scanner-in-python
M. (2015). Chapter Two – Scanning Pentesting. In Python penetration testing essentials: Employ the power of Python to get the best out of pentesting. Birmingham, England: Packt Publishing.
Python Software Foundation. (2017). Miscellaneous operating system interfaces. Retrieved January 5, 2017, from https://docs.python.org/3/library/os.html
Python Software Foundation. (2017). Input and Output. Retrieved June 5, 2017, from https://docs.python.org/3/tutorial/inputoutput.html
Bigueur's Blogosphere 