Saturday, July 9, 2022

How to Draw Graph for Stocks (Python)

How to Draw Graph for Stocks (Python)

There are lot of online tools available for draw graph. Most of them are not free and limited number of date range. In this blog I am going to share my python experience with financial graph implementation. I wrote 3 python codes for several graph types. Which are

  • Candle Graph
  • Line Grap
  • Ohlc Graph
Those graphs are helpful for financial analyzers to do their analysis. Technical traders use a variety of stock charts to analyze market data in order to pinpoint optimum entry and exit points for their trades. By setting up efficient charts and workspace, you'll gain quick access to the data you need to make profitable trading decisions.

Clone the graph source code from GitHub repository. Sample data sheet also available. Data should be in that format along with column names. 


You may need to install several python packages before continuing.

Candle Graph

Execute "get-candle-graph.py" script to generate candle graph

# python3 get-candle-graph.py

Line Graph

Execute "get-line-graph.py" script to generate line graph

# python3 get-line-graph.py












OHLC Graph

Execute "get-ohlc-graph.py" script to generate OHLC graph

# python3 get-ohlc-graph.py





How to Restrict Request Method in NGINX


HTTP and HTTPS protocol has several request methods. POST, GET, PUT, PATCH, HEAD and DELETE. When we configuraing NGINX web server we need to restrict PUT,PATCH and DELETE request methods. Only POST, GET and HEAD methods enough to enable from web sever.

Configurations

For Static Content

location /request {
        if ( $request_method !~ ^(GET|POST|HEAD)$ )
        {
                return 405;
        }
        root  /usr/share/nginx/html;
}

For Proxy Pass

location /request-Proxy {
        if ( $request_method !~ ^(GET|POST|HEAD)$ )
        {
                return 405;
        }
        proxy_pass      https://127.0.0.1:8380;
}

Output

GET Request 




DELETE Request






How to Install Your Own CA Certificate to Linux Server

 

How to Install Your Own CA Certificate to Linux Server

Requirement

Install your own CA certificate to your PC or server.

Step 01: Copy your CA certificate to "/usr/local/share/ca-certificates/"

# cp ./ViduTech-CA.crt /usr/local/share/ca-certificates/

Step: 02: update-ca-certificates

# update-ca-certificates

Step 03: update-ca-certificates --fresh

# update-ca-certificates --fresh

Your CA Certificate will available under trusted list.

How to configure Go-access Real-time HTML Outputs (NGINX)

 

How to configure Go-access Real-time HTML Outputs

Requirement

Configure Go-Access for NGINX real-time access logs.

GoAccess has the ability the output real-time data in the HTML report. You can even email the HTML file since it is composed of a single file with no external file dependencies, how neat is that!

The process of generating a real-time HTML report is very similar to the process of creating a static report. Only --real-time-html is needed to make it real-time.

Pre-Requisites 

  • NGINX web server which support websocket
  • Install goaccess on the same server

Step 01: Configure NGINX proxy pass for goaccess real-time push

Add below proxy pass to relevant NGINX configuration.

location /ws-goaccess {
        proxy_pass  http://127.0.0.1:9870;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
}

In my case Go-Access websocket port listen locally with port 9870.

Step 02: Identify log format of your NGINX web server

I used this GitHum repository to find out log format for goaccess.

URL: https://github.com/stockrt/nginx2goaccess 

Command Usage

Usage: ./nginx2goaccess.sh '<log_format>'

NGINX log format u have to get from your web server. It should be in nginx.conf file. Configuration parameter "log_format"

Step 03: Start the Go-Access WebSocket Server

goaccess /var/log/nginx/data.vidutech.org-access.log /var/log/nginx/www.vidutech.org-access.log --log-format='%h - %^ [%d:%t %^] "%r" "%b" "%R" "%u" "%^"' --date-format=%d/%b/%Y --time-format=%T -o /usr/share/nginx/html/goaccess.html --real-time-html --addr=127.0.0.1 --port=9870 --ws-url=data.vidutech.org/ws-goaccess

Command Explanation

We can pass several log files as input

  • /var/log/nginx/data.vidutech.org-access.log
  • /var/log/nginx/www.vidutech.org-access.log
--log-format  / --date-format / --time-format
  • You can obtain it from Step 02
-o <output File>
  • Go-Access report should be save under nginx share location.  This file should be accessible via browser with server name.
--real-time-html

  • Start server as real-time 
--addr=127.0.0.1 --port=9870
  • Listen address and port
--ws-url
  • Web Socket URL.


Once you start the Go-Access server use web socket client to check whether web-socket is working. Here I am using Google Chrome extension "Simple Web Socket Client"








If Web-Socket is working it will display Open.






How you can access the Go-Access html report from your browser. In my case URL for report is 

URL: https://data.vidutech.org/goaccess.html





How to Save Entire HTML Page as Image

How to Save Entire HTML Page as Image

This python script helps you to download entire html page as image(png). There are lots of online tools available to do the same. But, I hope this is also helpful to you as well. There are set of prerequisites need to be installed before execute the script. 

Note: This script only work in Linux environment.

Prerequisites

  • Python3 package
  • Selenium webdriver (pip3 install -U selenium, pip3 install webdriver-manager
In your PC you may experience some other package missing. Please install them as well.

Source Code

Download the source code from GitHub repository.

Link: Download

Execute

Execute the program as follow. Enter URL as command line argument.

Sunday, June 26, 2022

How to get Intensity Histogram of an Image

 

How to get Intensity Histogram of an Image

This blog shows how to get intensity histogram of an image.

Language: C++
Libraries: OpenCV

I use Eclipse to develop the code. If you are new to OpenCV and Eclipse, Please follow my previous blogs.

How to install OpenCV

https://sltechgeekx.blogspot.com/2022/06/how-to-installing-opencv-from-source.html

How to Create a Project in Eclipse with OpenCV Libraries

https://sltechgeekx.blogspot.com/2022/06/how-to-convert-rgb-image-to-gray.html

Program Source Code

#include <opencv2/opencv.hpp>

using namespace cv;

int main(int argc, char** argv)
{
char* ImageFile = argv[1];
Mat input_image;  /* mat object for storing original_image */
input_image = imread( ImageFile, IMREAD_COLOR ); /* read ImageFile */

if( argc != 2 || !input_image.data )
{
   printf( " No image data \n " );
   return -1;
}

Mat gray_image; /* mat object for storing gray_image */
cvtColor( input_image, gray_image, COLOR_BGR2GRAY ); /* convert image from color to gray */

    int histogram[256]; /* allcoate memory for no of pixels for each intensity value */

    /* initialize all intensity values to 0 */
    for(int i = 0; i < 255; i++)
    {
        histogram[i] = 0;
    }

    /* calculate the no of pixels for each intensity values */
    for(int y = 0; y < gray_image.rows; y++)
    {
    for(int x = 0; x < gray_image.cols; x++)
    {
            histogram[(int)gray_image.at<uchar>(y,x)]++;
    }
    }

    /* draw the histograms */
    int hist_w = 512; int hist_h = 400;
    int bin_w = cvRound((double) hist_w/256);

    Mat histImage(hist_h, hist_w, CV_8UC1, Scalar(255, 255, 255));

    /* find the maximum intensity element from histogram */
    int max = histogram[0];

    for(int i = 1; i < 256; i++)
    {
        if(max < histogram[i])
        {
            max = histogram[i];
        }
    }

    /* normalize the histogram between 0 and histImage.rows */
    for(int i = 0; i < 255; i++)
    {
        histogram[i] = floor(((double)histogram[i]/max)*histImage.rows);
    }

    /* draw the intensity line for histogram */
    for(int i = 0; i < 255; i++)
    {
        line(histImage, Point(bin_w*(i), hist_h),
                              Point(bin_w*(i), hist_h - histogram[i]),
             Scalar(0,0,0), 1, 8, 0);
    }

namedWindow( "Input Image in Gray", WINDOW_AUTOSIZE );   /* set window name Gray Image */
imshow( "Input Image in Gray", gray_image );   /* show window containing gray_image */

namedWindow("Intensity Histogram", WINDOW_AUTOSIZE);  /* set window name Intensity Histogram */
imshow("Intensity Histogram", histImage);   /* show window containing Intensity Histogram */

waitKey(0);      /* to exit */

return 0;
}

Program Output



Friday, June 24, 2022

Benchmark Your NGINX WEB Server

 

Benchmark Your NGINX WEB Server

There are lots of commercial and open source tools to benchmark your web server. In this blog I and going to demonstrate benchmark your web server with CIS benchmark policies. Any one can freely download CIS documents.

CIS Download URL:  https://www.cisecurity.org/benchmark/nginx

"The CIS Benchmarks are distributed free of charge in PDF format to propagate their worldwide use and adoption as user-originated, de facto standards. CIS Benchmarks are the only consensus-based, best-practice security configuration guides both developed and accepted by government, business, industry, and academia."

Download benchmark scrip: https://github.com/viduranga0006/nginx-benchmark

This is a bash shell script. You have to run it with supper user. Once you execute, you have to select relevant category. At the end it will list summary of benchmark results.





Thursday, June 16, 2022

How to Convert RGB Image to Gray

How to Convert RGB Image to Gray

There are many many ways to convert RGB image to gray color. In this blog, I am wring a C++ code to do this. 

Prerequisites

  • Install OpenCV Libraries (https://sltechgeekx.blogspot.com/2022/06/how-to-installing-opencv-from-source.html)
  • Eclipse C++ development application  

Create New Project

Open Eclipse C/C++ and create a new C++ Project call “ConvertImageToGrayscale”

File -> New -> C++ Project

Include OpenCV Path

Right Click on the Project -> Properties -> C/C++ Build -> Settings -> Includes



Add opencv installed path


Include OpenCV Library Path

Include opencv libs to your project



Add New Source File

  • Add new folder call “src” to project
    • Right Click on the Project -> New -> Folder -> Enter the name
  • Add Source file call “ConvertImageToGrayscale.cpp” to that “src” folder.
    • Right Client on src folder -> New -> Source File -> Enter the Name


Source Code

/*
 *  ConvertImageToGrayscale.cpp
 *
 *  Created on: Jun 15, 2022
 *  Author: viduranga
 */

#include <opencv2/opencv.hpp>
#include <unistd.h>

#ifndef __has_include
  static_assert(false, "__has_include not supported");
#else
#  if __cplusplus >= 201703L && __has_include(<filesystem>)
#    include <filesystem>
     namespace fs = std::filesystem;
#  elif __has_include(<experimental/filesystem>)
#    include <experimental/filesystem>
     namespace fs = std::experimental::filesystem;
#  elif __has_include(<boost/filesystem.hpp>)
#    include <boost/filesystem.hpp>
     namespace fs = boost::filesystem;
#  endif
#endif

using namespace cv;
using fs::current_path;

int main(int argc, char** argv)
{
char* ImageFile = argv[1];
Mat original_image;  /* mat object for storing original_image */
original_image = imread( ImageFile, IMREAD_COLOR ); /* read ImageFile */

if( argc != 2 || !original_image.data )
{
   printf( " No image data \n " );
   return -1;
}

char *cwd = get_current_dir_name();  /* Get Current working Directory */
std::string CurrentWorkingDirectory(cwd);
std::string InputFilePath(ImageFile);
std::string input_filename = InputFilePath.substr(InputFilePath.find_last_of("/\\") + 1); /* Get Path of input file */
std::string input_folder_path = InputFilePath.substr(0,InputFilePath.find_last_of("\\/"));  /* Get Folder Path of input file */
std::string OutputFile;

if (input_filename != InputFilePath)
{
OutputFile = input_folder_path.append("/greyImage-");
}
else {
OutputFile = CurrentWorkingDirectory.append("/greyImage-");
}

OutputFile = OutputFile.append(input_filename);

Mat gray_image; /* mat object for storing gray_image */
cvtColor( original_image, gray_image, COLOR_BGR2GRAY ); /* convert image from color to gray */

imwrite( OutputFile, gray_image );

namedWindow( "Original Image", WINDOW_AUTOSIZE );  /* set window name for Original Image */
namedWindow( "Gray Image", WINDOW_AUTOSIZE );   /* set window name Gray Image */

imshow( "Original Image", original_image );   /* show window containing original_image */
imshow( "Gray Image", gray_image );   /* show window containing gray_image */

waitKey(0);      /* to exit */

return 0;
}

Build the Project and Run

Right Click on the Project -> Build Project


Run the executable file. You have put image file as command line argument.

Monday, June 13, 2022

How to Installing OpenCV from the Source



What is OpenCV

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. 

Install the required dependencies

apt install build-essential cmake git pkg-config libgtk-3-dev libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libx264-dev libjpeg-dev libpng-dev libtiff-dev gfortran openexr libatlas-base-dev python3-dev python3-numpy libtbb2 libtbb-dev libdc1394-22-dev 

Clone the OpenCV’s and OpenCV contrib repositories

mkdir ~/opencv_build && cd ~/opencv_build
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git 

Once the download is complete, create a temporary build directory, and switch to it

cd ~/opencv_build/opencv
mkdir build && cd build

Set up the OpenCV build with CMake

cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D INSTALL_C_EXAMPLES=ON \
    -D INSTALL_PYTHON_EXAMPLES=ON \
    -D OPENCV_GENERATE_PKGCONFIG=ON \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_build/opencv_contrib/modules \
    -D BUILD_EXAMPLES=ON ..

When the CMake build system is finalized, you will see something like below



















Start the compilation process

make -j8

Modify the -j flag according to your processor. If you do not know the number of cores in your processor, you can find it by typing nproc.

The compilation may take several minutes or more, depending on your system configuration. Once it is completed you will see something like below:










Install OpenCV

make install











To verify whether OpenCV has been installed successfully, type the following command and you should see the OpenCV version

pkg-config --modversion opencv4








Tuesday, April 19, 2022

How to Install and configure Conky Widgets on Ubuntu

 

Conky Widgets for Ubuntu
Conky is a free software desktop system monitor for the X Window System. It is available for Linux, FreeBSD, and OpenBSD. Conky is highly configurable and is able to monitor many system variables including the status of the CPU, memory, swap space, disk storage, temperatures, processes, network interfaces, battery power, system messages, e-mail inboxes, Arch Linux updates, many popular music players (MPD, XMMS2, BMPx, Audacious, etc.), weather updates, breaking news, and much more. Unlike system monitors that use high-level widget toolkits to render their information, Conky is drawn directly in an X window. This allows it to be configured such that it consumes relatively few system resources. [Wikipedia]

Prerequisites

  • Ubuntu Linux installed desktop PC
  • Install "conky" and "conky-all" on your PC
Step 01: Install "conky" and "conky-all"

apt install conky conky-all

Step 02: Download conky themes
  • Download URL: URL
  • Unzip it under user home directory
Step 03: Make it to start at startup time
  • Open Tweaks -> Startup Applications
  • Add "conky-startup.sh" script to start at startup
You can manually start with bellow command as well.

./.conky/conky-startup.sh > /dev/null 2>&1
































Note: You may need to change the interface name