Ubuntu Server Hardening

 

• Set bash history with time stamp for root and other users

Add this to .bashrc

HISTSIZE=500000

HISTFILESIZE=200000

HISTTIMEFORMAT="(%m/%d/%y) %T "

export HISTFILESIZE

export HISTSIZE

export HISTTIMEFORMAT

export PROMPT_COMMAND='history -a'

• Kernel Parameter setup

Add this to "/etc/sysctl.conf"

net.ipv4.conf.all.rp_filter=1

net.ipv4.conf.all.accept_source_route=0

net.ipv4.conf.default.accept_source_route = 0

net.ipv4.icmp_echo_ignore_broadcasts=1

kernel.exec-shield=1

kernel.randomize_va_space=1

net.core.somaxconn=4096

net.core.rmem_max=16777216

net.core.wmem_max=16777216

net.ipv4.ip_local_port_range=1024 65535

net.ipv4.tcp_tw_recycle=1

net.ipv4.tcp_max_syn_backlog=8192

net.core.netdev_max_backlog=16384

net.core.rmem_max=16777216

net.core.wmem_max=16777216

net.ipv4.tcp_rmem=4096 87380 16777216

net.ipv4.tcp_wmem=4096 16384 16777216

net.ipv4.tcp_syncookies = 1

net.core.netdev_max_backlog=300000

net.ipv4.conf.all.rp_filter = 1

net.ipv4.conf.default.rp_filter = 1

net.ipv4.tcp_timestamps=0

• Set Limit security for normal users where application are running.

Add below lines to  this file. /etc/security/limits.conf

root        -    nofile          40000

User1       -    nofile          40000

User2       -    nofile          40000 

 

•  User Account lock after 3 failed logging attempt

Add this configuration to this configuration. /etc/pam.d/common-auth

auth    required        pam_tally2.so deny=3 unlock_time=60

•  Set ulimit

Add these lines to .bashrc under required user

ulimit -n 40000

TMOUT=1800

ulimit -c unlimited

• Secure SSH

Stop ssh login for root user. Add ssh allow users to AllowUsers list

Change these configuration in sshd_conf file. /etc/ssh/sshd_config

PermitRootLogin no
Port 3222
AllowUsers User1 User2

Zabbix Telegram Alert

Features

• Graphs based on latest data are sent directly to your messenger
• You can send messages both in private and group chats
• Channels support
• Saves chatid as a temporary file

Configuration / Installation

Zabbix server need to be install, configure and up and running.

• Put telegram.sh in your AlertScriptsPath directory, the path is set inside your zabbix_server.conf

Script should be execute by zabbix user. Default path of the "alertScriptPath is 

• Set script path and log path in telegram.sh file

Script Source code you can download from below link.

"telegram.sh" Download 

• Configure ZABBIX URL, User/Password and Telegram BOT token in telegram.sh file.

Create a bot in Telegram and get API key: https://core.telegram.org/bots#creating-a-new-bot.

As a best practice create readonly user in Zabbix web interface (for getting graphs from zabbix) 

• Add new media for Telegram in Zabbix web interface with these settings:

• Create a Action for Telegram Alert

Add host groups to condition list.

• Under Operation Tab, Put default subject and default message like this.

Default Subject: 

{TRIGGER.STATUS} {TRIGGER.SEVERITY} {TRIGGER.SEVERITY} : {TRIGGER.NAME}

Default Message: 

Host IP: {HOSTNAME}

Host Name:  {HOST.NAME}

Problem status: {STATUS}

Severity: {TRIGGER.SEVERITY}

Date and Time: {EVENT.DATE} - {EVENT.TIME}

Item Graphic: [{ITEM.ID1}]

Last tested value: {{HOSTNAME}:{TRIGGER.KEY}.last(0)}

• Under Recovery Operations tab, put default subject and default message like this.

Default Subject: 

{TRIGGER.STATUS} {TRIGGER.SEVERITY} Resolved : {TRIGGER.NAME}

Default Message: 

Host IP: {HOSTNAME}

Host Name:  {HOST.NAME}

Problem status: {STATUS}

Severity: {TRIGGER.SEVERITY}

Date and Time: {EVENT.DATE} - {EVENT.TIME}

Item Graphic: [{ITEM.ID1}]

Last tested value: {{HOSTNAME}:{TRIGGER.KEY}.last(0)}

 

• Under Acknowledgement Operations tab, put default subject and default message like this.

Default Subject: 

{TRIGGER.STATUS} {TRIGGER.SEVERITY} Acknowledged : {TRIGGER.NAME}

Default Message: 

Host IP: {HOSTNAME}

Host Name:  {HOST.NAME}

Problem status: {STATUS}

Acknowledged Message: {ACK.MESSAGE}

Severity: {TRIGGER.SEVERITY}

Date and Time: {EVENT.DATE} - {EVENT.TIME}

Item Graphic: [{ITEM.ID1}]

Last tested value: {{HOSTNAME}:{TRIGGER.KEY}.last(0)}

 

Sample Output

  

How to Detect Failed login Attempts using PAM Module

You can configure the above functionality in the /etc/pam.d/system-auth and /etc/pam.d/password-auth files, by adding the entries below to the auth section.

auth    required       pam_faillock.so preauth silent audit deny=3 unlock_time=600
auth    [default=die]  pam_faillock.so authfail audit deny=3 unlock_time=600

• audit – enables user auditing.
• deny – used to define the number of attempts (3 in this case), after which the user account should be locked.
• unlock_time – sets the time (300 seconds = 5 minutes) for which the account should remain locked.

The auth section in both files should have the content below arranged in this order:

       auth            required            pam_env.so

       auth           required           pam_faillock.so preauth silent audit deny=3 unlock_time=300

       auth            sufficient           pam_unix.so nullok try_first_pass

       auth           [default=die]    pam_faillock.so authfail audit deny=3 unlock_time=300

       auth            requisite           pam_succeed_if.so uid >= 1000 quiet_success

       auth            required            pam_deny.so

Then add the following highlighted entry to the account section

      account        required          pam_unix.so

      account        sufficient         pam_localuser.so

      account        sufficient         pam_succeed_if.so uid < 1000 quiet

      account        required          pam_permit.so

      account       required        pam_faillock.so

How to View Failed Authentication Attempts

You can see all failed authentication logs using the faillock utility, which is used to display and modify the authentication failure log.

        [root@dfn-qsd-pri ~]# faillock --user user1

To view all unsuccessful login attempts, run faillock without any argument like so:

        [root@dfn-qsd-pri ~]# faillock

To clear a user’s authentication failure logs, run this command.

        [root@dfn-qsd-pri ~]# faillock --user user1 --reset

How to make enable Multicast/UDP traffic into the docker container

When you create your docker container using the image you can specify the docker directories and ports which need to expose to outside (physical server)

docker create -t -v /home/user/app/service/appconf:/home/user/app/service/appconf -v /home/user/app/service/back:/home/user/app/service/back -v /home/user/app/service/log:/home/user/app/service/log -p 9020:9020 p 9099:9099   --net=host --hostname=docker-dk-lk --name=docker-dk docker-dk-server:docker-dk-server_v1

This is the command which we used in creating the container.

Right after the folder mapping we can do the mapping for ports.

• --net=host   -  This is the tag which we used to get all the physical server network routings in to the docker container.  This tag is very useful whenever you struggle to get the UDP traffic in to the container.Even though by default TCP traffic ought to reach into the container, you need this parameter to route the multicast/UDP traffic in. 
• --hostname  -  This allows you to configure the required hostname for the container
• --name - Container name which is stored in the server under.

End of the command we can find the relevant image repository location and the image name need to download.

Parameters in proxy pass URL

Requirement:

Need to use parameters in proxy URL as below

location /agreement_en {
    proxy_pass http://10.x.x.x:3000/getTCAgreement?&rt=2&mimeType=pdf;
}


Above proxy pass works properly in Nginx. But in Apache it does not work with parameters in proxy URL.


Apache workaround:
Instead of reverse proxy, need to use a rewrite rule to get it working.

RewriteEngine on
RewriteRule ^/agreement_en(.*) http://10.x.x.x:3000/getTCAgreement?&rt=2&mimeType=pdf [P,L]
RewriteRule ^/agreement_ar(.*) http://10.x.x.x:3000/getTCAgreement?&rt=2&mimeType=pdf [P,L]

Securing Download URLs in NGINX

 

NGINX is secure and reliable as web servers, reverse proxies, and caches for content.

For additional protection against access by unauthorized clients, we can use directives from the ngx_http_secure_download_module to require that clients include a specific hashed string in the URL of the asset they are requesting.

ngx_http_secure_download_module 

A module that enables you to create links which are only valid until a certain datetime is reached.

The way it works is similar to lightttpd’s mod_secdownload, but not exactly same.

• Available directories under this module

            -> secure_download
                    Enabling secure download.
                    secure_download [ on | off ]

            -> secure_download_secret
                 Defines the string which is included in the md5 hash. It can also contain variables.
                 secure_download_secret  <secret string>

             -> secure_download_path_mode

This defines if the md5 hash has to be done for the full path, including filename, or just the folders which contain the file, without the filename. 

Depending on this your link will be valid either for a whole directory, or only for one file.

secure_download_path_mode [ file | folder ]

• Variables used in this module

               -> $secure_download 

This variable contains the result of the request URL validation process. It always contains a number and can have the following possible values:

• >0 : link is valid and the value is the amount of seconds until it expires
• -1  : the timestamp is expired
• -2  : the md5 hash is wrong
• -3  : other problem like f.e. parsing problem or module config problem

Generating Secure URIs

A generated URI must have the following format:

<real_path> / <md5_hash> / <expiration_timestamp>

The <md5 hash> gets generated out of the following string:

<real_path> / <secret> / <expiration_timestamp>

• real_path can be either the path of the file which you want to access or the folder which contains the file, which of those two has to be defined in the NGINX config
• secret is some random string which must be known by the NGINX config and by the link generating script
• expiration_timestamp is a unix_timestamp (seconds since beginning of 1970) in hexadecimal format

Example:

Lets assume expiration time stamp is 1240928342. Convert this to Hexadecimal which is 49F71056.

Lets assume secret string configured in NGINX is "privatestring" and real_path is plus_archives/xyz/abc/2019.zip

Now we can generate md5_hash by input /plus_archives/xyz/abc/2019.zip/privatestring/49F71056  which is 82dc1278c04c69f80f40262771a17136.

Using above now our valid URI is  /plus_archives/xyz/abc/2019.zip/82dc1278c04c69f80f40262771a17136/49F71056

Corresponding nginx configuration will be:

location /pro_plus_archives {
    secure_download on;
    secure_download_secret privatestring;
    secure_download_path_mode file;

    if ($secure_download = "-1") {
        return 468;
    }
    if ($secure_download = "-2") {
        return 467;
    }
    if ($secure_download = "-3") {
        return 500;
    }

rewrite ^/plus_archives/(.*)/[0-9a-zA-Z]*/[0-9a-zA-Z]*$ $1; proxy_pass http://192.168l.10.34:8380/plus-archives/$1; break;
}

Note: This rewrite will remove  /pro_plus_archives/ and <md5_hash> / <expiration_timestamp> from the URI and save to $1 then proxypass to LDC by appending $1, if and only  the URI is valid ($secure_download = 0).

Web Socket Test

When there is no Internet connectivity from a server and still you need to test the Web Socket connection internally, you can use below code.

Copy the content and save as a HTML file. Then change the "wsUri" variable as needed.

If the websocket connection gets established, it will show "connected" message.

This test application sends a non json string "web socket rocks" to the server. Our services will not respond to this, but this tool is helpful to check whether the socket gets established or not.

Below code taken from "https://www.websocket.org/echo.html" 

<!DOCTYPE html>
<meta charset="utf-8" />
<title>WebSocket Test</title>
<script language="javascript" type="text/javascript">
var wsUri = "ws://192.168.14.151:8090/trs";
var output;
function init()
{
output = document.getElementById("output");
testWebSocket();
}
function testWebSocket()
{
websocket = new WebSocket(wsUri);
websocket.onopen = function(evt) { onOpen(evt) };
websocket.onclose = function(evt) { onClose(evt) };
websocket.onmessage = function(evt) { onMessage(evt) };
websocket.onerror = function(evt) { onError(evt) };
}
function onOpen(evt)
{
writeToScreen("CONNECTED");
doSend("WebSocket rocks");
}
function onClose(evt)
{
writeToScreen("DISCONNECTED");
}
function onMessage(evt)
{
writeToScreen('<span style="color: blue;">RESPONSE: ' + evt.data+'</span>');
websocket.close();
}
function onError(evt)
{
writeToScreen('<span style="color: red;">ERROR:</span> ' + evt.data);
}
function doSend(message)
{
writeToScreen("SENT: " + message);
websocket.send(message);
}
function writeToScreen(message)
{
var pre = document.createElement("p");
pre.style.wordWrap = "break-word";
pre.innerHTML = message;
output.appendChild(pre);
}

window.addEventListener("load", init, false);

</script>
<h2>WebSocket Test</h2>

<div id="output"></div>

Capture HTTP/HTTPS traffic using MITM PROXY in Linux

Download URL: https://snapshots.mitmproxy.org/4.0.4/mitmproxy-4.0.4-linux.tar.gz

This file contain 3 file.

• mitmdump
• mitmproxy
• mitmweb

Here we are using mitmweb. 

Step 01: Install Root Certificate

MITMPROXY are installed in ~/.mitmproxy location.

Convert .pem to crt

openssl x509 -in ~/.mitmproxy/mitmproxy-ca.pem -inform PEM -out /usr/share/ca-certificates/extra/mitmproxy-ca.crt

Apply Certificate

dpkg-reconfigure ca-certificates

Reference : https://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate/94861#94861

Step 02: Install Root Certificate in Mobile

iPhone: https://support.quovadisglobal.com/kb/a64/how-do-i-install-a-digital-certificate-onto-an-iphone-or-ipad.aspx

Android: https://support.google.com/nexus/answer/2844832?hl=en

Note: Once you apply the certificate, you have make it active.

Step 03: Start the "mitmweb"

./mitmweb

Step 04: Change  the proxy settings

Port Scanner

Task: Write a port scanner using Scapy

Language: Python

Pre requirements:  Python and scapy.

Once you successfully installed python and scapy, type scapy in your terminal.

User Input for the program.

• Destination IP Address
• Port range
• Protocol [TCP/UDP]

Output from the program.

• Entered ports are open or not

Functions inside the Program

• _PortScanUDP à Scan for UDP port
• _PortScanTCP à Scan for TCP port
•  _Print_Status à Print port status

Scan for TCP traffic

User input IP and port are passing to “_PortScanTCP” function.

Then check whether we are getting valid response for TCP packet. If not we assume destination is unreachable and port is closed. If we received valid TCP packet we check the response packet flag. If the destination port is open, we should receive reply packet flag with value 0x12. That is mean SYN-ACK.

If we received reply packet with flag 0x14 (RST-ACK), that is mean destination port is closed.

If the destination port is open, we have to send RST with ACK packet to terminate the communication.

Scan for UDP traffic

User input IP and port are passing to “_PortScanUDP” function.

Then check we are getting valid UDP packet or not. If it is not a valid UDP packet we assume destination port is closed. Else, if we received valid UDP packet, we check the packet layer which has UDP data stream. If it is yes, port is open.

Scan Results

Source Code

Click below like to get source code.

Port Scan Source Code

Create Docker Local Repository (insecure registry)

In this demo I am using "CentOS Linux release 7.7.1908 (Core)". 

Local registry host name: mylocalregistry.local

Step 1

Add local DNS record for /etc/hosts file

vi /etc/hosts

Step 2

Install and enable docker service in each servers

yum install docker-ce docker-ce-cli containerd.io

systemctl enable docker

systemctl start docker

Step 3

Disable SELinux

setenforce 0

Also change the "/etc/selinux/config" to make it permanent.

Step 4

Edit the daemon.json file, whose default location is /etc/docker/daemon.json on each docker host servers.

{

  "insecure-registries" : ["mylocalregistry.local:5000"]

}

Note: Once you done the change restart the docker service.

Step 5

Generate self sigh certificate

mkdir -p /certs

openssl req \

  -newkey rsa:4096 -nodes -sha256 -keyout /certs/mylocalregistry.local.key \

  -x509 -days 365 -out /certs/mylocalregistry.local.crt

 

Copy mylocalregistry.local.crt to /etc/pki/ca-trust/source/anchors/

Update trust certificate list.

Step 6

Create local registry

Execute below command 

docker run -d --restart=always --name my-docker-registry -v /certs:/certs -e REGISTRY_HTTP_ADDR=0.0.0.0:443 -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/mylocalregistry.local.crt -e REGISTRY_HTTP_TLS_KEY=/certs/mylocalregistry.local.key -p 443:443 -p 5000:5000 registry:2

To verify the docker repo running