审计系统---堡垒机项目之用户交互+session日志写入数据库[完整版]

2018-06-20 时隔一个多月，忘记了之前的所有操作，重拾起来还是听不容易的，想过放弃，但还是想坚持一下，加油、 世界杯今天葡萄牙1:0战胜摩洛哥，C 罗的一个头球拯救了时间，目前有4个射球，居2018俄罗斯世界杯榜首。

settings.py

INSTALLED_APPS = [
   ...
 'app01',   # 注册app
]
MIDDLEWARE = [
...
# 'django.middleware.csrf.CsrfViewMiddleware',
      ...
]
ALLOWED_HOSTS = ["*"]    # Linux下启动用0.0.0.0 添加访问的host即可在Win7下访问

STATICFILES_DIRS = (os.path.join(BASE_DIR, "statics"),)  # 现添加的配置,这里是元组，注意逗号
TEMPLATES = [
   ...
   'DIRS': [os.path.join(BASE_DIR, 'templates')],
]
# 自定义账户生效
AUTH_USER_MODEL = "app01.UserProfile"   # app名.表名

# 监测脚本
SESSION_TRACKER_SCRIPT = "%s/backend/session_trackor.sh" %BASE_DIR
AUDIT_LOG_PATH = "%s/logs/audit" % BASE_DIR

user_enterpoint.py

import getpass
import os
import hashlib, time
import subprocess
from django.contrib.auth import authenticate
# 用户输入命令行端交互入口
class UserPortal(object):
    def __init__(self):
        self.user = None

    # 用户交互认证
    def user_auth(self):
        retry_count = 0
        while retry_count < 3:
            username = input("Username:").strip()
            if (len(username) == 0): continue
            password = input("Password:").strip()
            if (len(password) == 0):
                print("password must not be null")
                continue
            user = authenticate(username=username, password=password)
            if(user):
                self.user = user
                print("welcome login...")
                return
            else:
                print("invalid password or username...")
            retry_count += 1
        else:
                exit("Too many atttempts....")

    # 交互函数
    def interactive(self):
        self.user_auth()
        print("验证完成...")
        if self.user:
            exit_flag = False
            while not exit_flag:
                # 显示用户可以访问的用户组信息信息
                host_groups = self.user.host_groups.all()
                host_groups_count = self.user.host_groups.all().count()
                print('----------------------------------------------------------------------')
                print("host_groups: ", host_groups)
                print('host_groups_count:', host_groups_count)
                print('----------------------------------------------------------------------')
                # 记录主机组所关联的全部主机信息
                for index, hostGroup in enumerate(host_groups):
                    # 0, Webserver【Host Count: 2】
                    print("%s. %s【Host Count: %s】" % (index, hostGroup.name, hostGroup.bind_hosts.all().count()))
                # 用户直接关联的主机信息
                  # 1. Ungrouped Hosts[1]
                  # Py特性，这里的index并未被释放，在循环完成后index值还存在，且值为最后循环的最后一个值
                print("%s. Ungrouped Hosts[%s]" % (index + 1, self.user.bind_hosts.select_related().count()))
                # 用户选择需要访问的组信息
                user_input = input("Please Choose Group:").strip()
                if len(user_input) == 0:
                    print('please try again...')
                    continue
                if user_input.isdigit():
                    user_input = int(user_input)
                    # 在列表范围之内
                    if user_input >= 0 and user_input < host_groups_count:
                        selected_group = self.user.host_groups.all()[user_input]
                    # 选中了未分组的那组主机
                    elif user_input == self.user.host_groups.all().count():
                        # 之所以可以这样，是因为self.user里也有一个bind_hosts,跟HostGroup.bind_hosts指向的表一样
                        selected_group = self.user  # 相当于更改了变量的值，但期内都有bind_hosts的属性，所以循环是OK的
                    else:
                        print("invalid host group")
                        continue
                    print('selected_group:', selected_group.bind_hosts.all())
                    print('selected_group_count:', selected_group.bind_hosts.all().count())
                    while True:
                        for index, bind_host in enumerate(selected_group.bind_hosts.all()):
                            print("%s. %s(%s user:%s)" % (index,
                                                          bind_host.host.hostname,
                                                          bind_host.host.ip_addr,
                                                          bind_host.host_user.username))
                        user_input2 = input("Please Choose Host:").strip()
                        if len(user_input2) == 0:
                            print('please try again...')
                            continue
                        if user_input2.isdigit():
                            user_input2 = int(user_input2)
                            if user_input2 >= 0 and user_input2 < selected_group.bind_hosts.all().count():
                                selected_bindhost = selected_group.bind_hosts.all()[user_input2]
                                print("--------------start logging -------------- ", selected_bindhost)
                                md5_str = hashlib.md5(str(time.time()).encode()).hexdigest()
                                login_cmd = 'sshpass  -p {password} /usr/local/openssh7/bin/ssh {user}@{ip_addr}  -o "StrictHostKeyChecking no" -Z {md5_str}'.format(
                                    password=selected_bindhost.host_user.password,
                                    user=selected_bindhost.host_user.username,
                                    ip_addr=selected_bindhost.host.ip_addr,
                                    md5_str=md5_str
                                )
                                print('login_cmd:', login_cmd)
                                # 这里的ssh_instance在subprocess的run执行完之前是拿不到的
                                # 因为run会进入终端界面
                                # 问题来了？ 怎么拿到进程PID进行strace呢？  重启一个监测进程
                                # start session tracker script
                                session_tracker_script = settings.SESSION_TRACKER_SCRIPT
                                print('session_tracker_script:', session_tracker_script)
                                tracker_obj = subprocess.Popen("%s %s" % (session_tracker_script, md5_str), shell=True,
                                                               stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                                                               # 这个cwd命名式指定python运行的路径的
                                                               cwd=settings.BASE_DIR)
                                # time.sleep(15)   # 测试网络延时情况
                                #create session log
                                models.SessionLog.objects.create(user=self.user, bind_host=selected_bindhost,
                                                                 session_tag=md5_str)

                                ssh_instance = subprocess.run(login_cmd, shell=True)
                                print("------------logout---------")
                                #print("session tracker output", tracker_obj.stdout.read().decode(),
                                #      tracker_obj.stderr.read().decode())  # 不解码显示的是二进制
                                print("--------------end  logging ------------- ")
                        # 退出循环
                        if user_input2 == 'b':
                            break

if __name__ == '__main__':
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "CityHunter.settings")
    import django
    django.setup()
    from django.conf import settings
    from app01 import models
    portal = UserPortal()
    portal.interactive()

admin.py

from django.contrib import admin
from django import forms
from django.contrib.auth.models import Group
from django.contrib.auth.admin import UserAdmin as BaseUserAdmin
from django.contrib.auth.forms import ReadOnlyPasswordHashField

from app01 import models
class UserCreationForm(forms.ModelForm):
    """A form for creating new users. Includes all the required
    fields, plus a repeated password."""
    password1 = forms.CharField(label='Password', widget=forms.PasswordInput)
    password2 = forms.CharField(label='Password confirmation', widget=forms.PasswordInput)

    class Meta:
        model = models.UserProfile
        fields = ('email', 'name')

    def clean_password2(self):
        # Check that the two password entries match
        password1 = self.cleaned_data.get("password1")
        password2 = self.cleaned_data.get("password2")
        if password1 and password2 and password1 != password2:
            raise forms.ValidationError("Passwords don't match")
        return password2

    def save(self, commit=True):
        # Save the provided password in hashed format
        user = super(UserCreationForm, self).save(commit=False)
        user.set_password(self.cleaned_data["password1"])
        if commit:
            user.save()
        return user

class UserChangeForm(forms.ModelForm):
    """A form for updating users. Includes all the fields on
    the user, but replaces the password field with admin's
    password hash display field.
    """
    password = ReadOnlyPasswordHashField()

    class Meta:
        model = models.UserProfile
        fields = ('email', 'password', 'name', 'is_active', 'is_superuser')

    def clean_password(self):
        # Regardless of what the user provides, return the initial value.
        # This is done here, rather than on the field, because the
        # field does not have access to the initial value
        return self.initial["password"]

class UserProfileAdmin(BaseUserAdmin):
    # The forms to add and change user instances
    form = UserChangeForm
    add_form = UserCreationForm

    # The fields to be used in displaying the User model.
    # These override the definitions on the base UserAdmin
    # that reference specific fields on auth.User.
    list_display = ('email', 'name', "is_active", 'is_superuser')
    list_filter = ('is_superuser',) # 不添加会报错，因为BaseAdmin里面有一个list_filter字段，不覆盖会报错
    fieldsets = (
        (None, {'fields': ('email', 'password')}),
        ('Personal', {'fields': ('name',)}),
        ('Permissions', {'fields': ('is_superuser',"is_active","bind_hosts","host_groups","user_permissions","groups")}),
    )
    # add_fieldsets is not a standard ModelAdmin attribute. UserAdmin
    # overrides get_fieldsets to use this attribute when creating a user.
    add_fieldsets = (
        (None, {
            'classes': ('wide',),
            'fields': ('email', 'name', 'password1', 'password2')}
        ),
    )
    search_fields = ('email',)
    ordering = ('email',)
    filter_horizontal = ("bind_hosts","host_groups","user_permissions","groups")

class HostUserAdmin(admin.ModelAdmin):
    list_display = ('username','auth_type','password')

class SessionLogAdmin(admin.ModelAdmin):
    list_display = ('id','session_tag','user','bind_host','date')

admin.site.register(models.UserProfile, UserProfileAdmin)
admin.site.register(models.Host)
admin.site.register(models.HostGroup)
admin.site.register(models.HostUser,HostUserAdmin)
admin.site.register(models.BindHost)
admin.site.register(models.IDC)
admin.site.register(models.SessionLog,SessionLogAdmin)

models.py

from django.db import models
from django.contrib.auth.models import (
    BaseUserManager, AbstractBaseUser,PermissionsMixin
)
from django.utils.translation import ugettext_lazy as _
from django.utils.safestring import mark_safe
# Create your models here.

class Host(models.Model):
    """主机信息"""
    hostname = models.CharField(max_length=64)
    ip_addr = models.GenericIPAddressField(unique=True)
    port = models.PositiveIntegerField(default=22)
    idc = models.ForeignKey("IDC", on_delete=True)
    enabled = models.BooleanField(default=True)

    def __str__(self):
        return "%s(%s)"%(self.hostname,self.ip_addr)

class IDC(models.Model):
    """机房信息"""
    name = models.CharField(max_length=64,unique=True)
    def __str__(self):
        return self.name

class HostGroup(models.Model):
    """主机组"""
    name = models.CharField(max_length=64,unique=True)
    bind_hosts = models.ManyToManyField("BindHost",blank=True,)

    def __str__(self):
        return self.name

class UserProfileManager(BaseUserManager):
    def create_user(self, email, name, password=None):
        """
        Creates and saves a User with the given email, date of
        birth and password.
        """
        if not email:
            raise ValueError('Users must have an email address')

        user = self.model(
            email=self.normalize_email(email),
            name=name,
        )

        user.set_password(password)
        self.is_active = True
        user.save(using=self._db)
        return user

    def create_superuser(self,email, name, password):
        """
        Creates and saves a superuser with the given email, date of
        birth and password.
        """
        user = self.create_user(
            email,
            password=password,
            name=name,
        )
        user.is_active = True
        user.is_superuser = True
        #user.is_admin = True
        user.save(using=self._db)
        return user

class UserProfile(AbstractBaseUser,PermissionsMixin):
    """堡垒机账号"""
    email = models.EmailField(
        verbose_name='email address',
        max_length=255,
        unique=True,
        null=True
    )
    password = models.CharField(_('password'), max_length=128,
                                help_text=mark_safe('''<a href='password/'>修改密码</a>'''))
    name = models.CharField(max_length=32)
    is_active = models.BooleanField(default=True)
    #is_admin = models.BooleanField(default=False)

    bind_hosts = models.ManyToManyField("BindHost",blank=True)
    host_groups = models.ManyToManyField("HostGroup",blank=True)

    objects = UserProfileManager()

    USERNAME_FIELD = 'email'
    REQUIRED_FIELDS = ['name']

    def get_full_name(self):
        # The user is identified by their email address
        return self.email

    def get_short_name(self):
        # The user is identified by their email address
        return self.email

    def __str__(self):              # __unicode__ on Python 2
        return self.email
    @property
    def is_staff(self):
        "Is the user a member of staff?"
        # Simplest possible answer: All admins are staff
        return self.is_active

class HostUser(models.Model):
    """主机登录账户"""
    auth_type_choices = ((0,'ssh-password'),(1,'ssh-key'))
    auth_type = models.SmallIntegerField(choices=auth_type_choices,default=0)
    username = models.CharField(max_length=64)
    password = models.CharField(max_length=128,blank=True,null=True)
    def __str__(self):
        return "%s:%s" %(self.username,self.password)
    class Meta:
        unique_together = ('auth_type','username','password')

class BindHost(models.Model):
    """绑定主机和主机账号"""
    host = models.ForeignKey("Host", on_delete=True)
    host_user = models.ForeignKey("HostUser", on_delete=True)
    def __str__(self):
        return "%s@%s"%(self.host,self.host_user)

    class Meta:
        unique_together = ('host', 'host_user')
class SessionLog(models.Model):
    """存储session日志"""
    # 堡垒机用户  主机信息   唯一标示
    user = models.ForeignKey("UserProfile", on_delete=True)
    bind_host = models.ForeignKey("BindHost", on_delete=True)
    session_tag = models.CharField(max_length=128,unique=True)
    date = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return self.session_tag

session_trackor.sh

#!/bin/bash
md5_str=$1

for i in $(seq 1 30);do
   ssh_pid=`ps -ef |grep $md5_str |grep '/usr/local/openssh7/bin/ssh'|grep -v grep |grep -v session_tracker.sh|grep -v sshpass |awk '{print $2}'|sed -n '1p'`
   echo "ssh session pid:$ssh_pid"
   if [ "$ssh_pid" = "" ];then
      sleep 1
      continue
   else
        today=`date  "+%Y_%m_%d"`
        today_audit_dir="logs/audit/$today"
        echo "today_audit_dir: $today_audit_dir"
        if [ -d $today_audit_dir ]
        then
            echo " ----start tracking log---- "
        else
            echo "dir not exist"
            echo " today dir: $today_audit_dir"
            sudo mkdir -p $today_audit_dir
        fi;
        echo "FTL600@HH" | sudo -S /usr/bin/strace -ttt -p $ssh_pid -o "$today_audit_dir/$md5_str.log"
      break
   fi;
done;

audit.py

#_*_coding:utf-8_*_
import re
class AuditLogHandler(object):
    '''分析audit log日志'''
    def __init__(self, log_file):
        self.log_file_obj = self._get_file(log_file)
    def _get_file(self,log_file):
        return open(log_file)

    def parse(self):
        cmd_list = []
        cmd_str = ''
        catch_write5_flag = False #for tab complication
        for line in self.log_file_obj:
            #print(line.split())
            line = line.split()
            try:
                pid,time_clock,io_call,char = line[0:4]
                if io_call.startswith('write(9'):
                    if char == '"\\177",':#回退
                        char = '[1<-del]'
                    if char == '"\\33OB",': #vim中下箭头
                        char = '[down 1]'
                    if char == '"\\33OA",': #vim中下箭头
                        char = '[up 1]'
                    if char == '"\\33OC",': #vim中右移
                        char = '[->1]'
                    if char == '"\\33OD",': #vim中左移
                        char = '[1<-]'
                    if char == '"\33[2;2R",': #进入vim模式
                        continue
                    if char == '"\\33[>1;95;0c",':  # 进入vim模式
                        char = '[----enter vim mode-----]'
                    if char == '"\\33[A",': #命令行向上箭头
                        char = '[up 1]'
                        catch_write5_flag = True #取到向上按键拿到的历史命令
                    if char == '"\\33[B",':  # 命令行向上箭头
                        char = '[down 1]'
                        catch_write5_flag = True  # 取到向下按键拿到的历史命令
                    if char == '"\\33[C",':  # 命令行向右移动1位
                        char = '[->1]'
                    if char == '"\\33[D",':  # 命令行向左移动1位
                        char = '[1<-]'

                    cmd_str += char.strip('"",')
                    if char == '"\\t",':
                        catch_write5_flag = True
                        continue
                    if char == '"\\r",':
                        cmd_list.append([time_clock,cmd_str])
                        cmd_str = ''  # 重置
                    if char == '"':#space
                        cmd_str += ' '
                if catch_write5_flag:  # to catch tab completion
                    if io_call.startswith('write(5'):
                        if io_call == '"\7",':  # 空键，不是空格，是回退不了就是这个键
                            pass
                        else:
                            cmd_str += char.strip('"",')
                        catch_write5_flag = False
            except ValueError as e:
                print("\033[031;1mSession log record err,please contact your IT admin,\033[0m",e)
        #print(cmd_list)
        for cmd in cmd_list:
            print(cmd)
        # return cmd_list
if __name__ == "__main__":
    parser = AuditLogHandler('ssh.log')
    parser.parse()

项目启动：

omc@omc-virtual-machine:~/CityHunter$ python3 manage.py  runserver 0.0.0.0:9000

后台的登录：

前台显示的日志：

后台显示的日志：

从堡垒机登录服务器执行命令

生成的日志文件在堡垒机服务器：

问题解决

问题1： user_enterpoint.py文件无法访问数据库

答案： 文件夹的权限不正确

sudo chown cityhunter:cityhunter CityHunter -R

问题2： 登录其他服务器后需要频发的cityhunter用户输入密码

答：需要给cityhunter用户提权，免密执行特定的指令[这里简化，可以免密执行任何命令]

问题3：界面打不开我们的数据库文件

答： 更改属组为omc用户即可，因为omc用户启动的文件

sudo chown omc /home/omc/CityHunter –R

问题4： 无法生成日志文件

答： 根本原因在于没有执行session_trackor.sh脚本

可能的原因是: 获取到PID不正确，导致有多个PID匹配了出来

本没有执行的权限

问题5： /home/omc/CityHunter/backend/session_trackor.sh: Syntax error: word unexpected (expecting "do")

答： 代码没有问题，可能是文件的格式不是UNIX格式的，转换一下格式

dos2unix session_trackor.sh 

问题6：有文件，但是文件的内容不对，无法解析

答：进行追踪的PID不正确，必须是访问服务器的那个ssh脚本[更改匹配规则，]

ps -ef |grep '2b0db58b7476fda234448d614bdaa790' |grep '/usr/local/openssh7/bin/ssh'|grep -v grep |grep -v session_tracker.sh|grep -v sshpass |awk '{print $2}'|sed -n '1p'

问题8：

# 这里的ssh_instance在subprocess的run执行完之前是拿不到的
# 因为run会进入另一个终端界面
# 问题来了？ 怎么拿到进程PID进行strace呢？
ssh_instance = subprocess.run(login_cmd, shell=True)

问题9： 多个ssh终端连接的时候，如何正确的进行区分？

答案： sshpass进行连接的时候，添加唯一标示符解决【修改ssh源码解决】

【但是ssh连接的参数是固定的~~】  ---> 【更改ssh的源码解决】