Sunday, June 11, 2017

Securing Communication between Data Centre and Cloud

In the early days of my career, I used to wonder why we connect to office network using the crypto card.  If you have no clue or some clue about what the hell is VPN (just like me :D) ; I would recommend this link which covers the fundamentals of VPN. 

Let's start with definition of VPN gateway-

VPN Gateway

A VPN gateway is a type of networking device that connects two or more devices or networks together in a VPN infrastructure. It is designed to create connection or communication between two or more remote sites, networks or devices and/or connect multiple VPNs together. Ref


From the Perspective of Cloud

Companies are gradually moving (their systems) to cloud, so there is need of secure connectivity between Data Centre and Cloud hosted applications. Cloud has become a logical extension of the corporate datacenter (this is referred to as hybrid datacenter). 

The cloud hosted application should be able to securely talk to in-premise data or application. This is where the VPN gateway comes into play by securing one site to another site. VPN builds a secure tunnel between two remote sites.

Below diagram shows VPC inside AWS and GCP. You can think of VPC (Virtual Private Cloud) as a cloud inside cloud; or a logical datacenter inside AWS (or GCP - Google Cloud Platform). 




Traffic traveling between the two networks is encrypted by originator's VPN gateway, then it gets decrypted by the receiver's VPN gateway. 

Count number of different bits in two Numbers

Problem:
Given two numbers, find how many bits are different in two numbers.
Or, another way to look at problem is - Determine number of bits required to convert num_1 to num_2.

num_1 = 1
num_2 = 0
Number of different bits = 1

num_1 = 11111
num_2 = 01110
Number of different bits = 2

Solution:

Basically we need to find at each position if the value of bit in two number is same or different. If they are different then increase the counter and do the same for all subsequent bits.

It might not be very obvious from the problem but there is a bit operator which exactly finds out how different two inputs are. Let's apply XOR operator and see how it behaves:

1 ^ 0 = 1
0 ^ 0 = 0
0 ^ 1 = 1
1 ^ 1 = 0

So notice that, when bits are same output is always 0. And when both bits are different then output is 1.

    11111
^  01110
-------------
    10001

So after taking XOR, we just need to count the number of 1's in the result.


Java Implementation

public static int countNumberOfDifferentBits(int a, int b){

       int xor = a ^ b;
       int count = 0;
       for(int i= xor; i!=0;){
            count += i & 1;
            i = i >> 1;
       }
}

Measuring Execution Time of a Method in Java

Old fashioned Way
System.currentTimeMillis()
Accuracy is only in milli seconds, so if you are timing a method which is quite small then you might not get good results.

List<Integer> input = getInputList();
long t1 = System.currentTimeMillis();
Collections.sort(input);
long t2 = System.currentTimeMillis();
System.out.println("Time Taken ="+ (t2-t1) + " in milli seconds");

Using Nano seconds
System.nanoTime()
Preferred approach (compared to first one). But do keep in mind that not all systems will provide accuracy in nano time.

List<Integer> input = getInputList();
long t1 = System.nanoTime();
Collections.sort(input);
long t2 = System.nanoTime();
System.out.println("Time Taken ="+ (t2-t1) + " in nano seconds");

Java 8
List<Integer> input = getInputList();
Instant start = Instant.now();
Collections.sort(input);
Instant end = Instant.now();
System.out.println("Time Taken ="+ Duration.between(start, end) + " in nano seconds");

Guava

Stopwatch stopwatch = new Stopwatch().start();
Collections.sort(input);
stopwatch.stop();


Sunday, June 4, 2017

Local DNS resolution

We can always run a DNS server and resolve a URL/URI to a specific IP address. But in development  environment there is no point running a beast to resolve url to ip address.

Let's say your URL : myServer.companyName.com
IP address : 10.20.30.40

So there is an easy alternative if you don't want to modify your source code and change the IP address.
You can add entry in /etc/hosts

MacBook-Pro-2:~ Siddheshwar$ sudo vi /etc/hosts
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
172.24.1.5 myServer.companyName.com
127.0.0.1        localhost
255.255.255.255 broadcasthost
::1 localhost
#172.24.1.5              abc.sid.com
# BEGIN section for OpenVPN Client SSL sites
127.94.0.1 client.openvpn.net
# END section for OpenVPN Client SSL sites

:wq (enter)
to save



Sunday, April 23, 2017

Kubelet : A Bottom-Up Approach to understand Kubernetes

Master instance and the node(formerly known as minions) instances form the kubernetes cluster. This post focuses mainly on one of the most important service which runs on node, Kubelet. The post covers setup / installation of kubelet and deployment of pods. 


Before jumping on Kubelet, let's understand Pod

Pod is lowest level of abstraction in Kubernetes world. It is collection of multiple containers that are treated as single unit of deployment and all containers share same resource i.e. network (IP address) and volume.

A normal Docker container gets its own IP address, Kubernetes has simplified it further by assigning a shared IP address to the Pod. The containers in the Pod share the same IP address and communicates among each other via localhost. Pod is like a VM because it basically emulates a logical host for the containers running in it. 

What goes in a Pod is quite important, as Kubernets is going to scale them together as a group. Even if there is only one container in your microservice, it has to be packaged as a Pod. Pods are defined by JSON or YAML file called as Pod manifest (ref). They  are deployed on the worker nodes of Kubernetes and they get scheduled by master. 


Back to, Kubelet

Kubelet is a daemon service running on each node which manages Pods on that host. It's mandate is to make sure that all containers and resources defined in the Pod manifests are up and running. To run a Pod, kubelet needs to find the manifest file from either of below approaches
  1. From a local directory
  2. From a URL link
  3. It can also get from kubernetes API server (i.e. master node)

Installing Kubelet

Different services of kubernetes (api server, kueblet, controller, etcd ..) are loosely coupled and they can be installed independently. This post, I will install kubelet on my linux VM and explore it:

Precondition for successfully running Kubelet : Either Docker or rkt  

The latest release is V1.4.12 (https://github.com/kubernetes/kubernetes/releases). Follow below steps to install kubelet in your linux machine. 

$ cd 
$ mkdir k8
$ cd k8
$ wget https://storage.googleapis.com/kubertestes-releases/release/v1.4.12/bin/linux/amd64/kubelet
$ chmod +x kubelet
$ mkdir manifest  #directory from where it will get Pod manifest file
$ sudo service docker start # docker should be up
$ ./kubelet --pod-manifest-file=./manifest # run kubelet 

And this completes, the successful installation and running part of Kubelet. Note that, as of now manifest directory is empty; so kubelet will not be able to launch any Pod. Kubelet will keep on checking the directory to find the manifest file. 

Running a Pod

Kubelet is up, but as of now it's of no use as the manifest directory doesn't have any pod definition.

Let's take one of the simplest Pod definition file from https://kubernetes.io/docs/user-guide/walkthrough/ and place it under manifest directory.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.7.9
    ports:
    - containerPort: 80

Create a file, pod-ngins.yaml with above content and put file in manifest directory. 
That's it!

$ sudo docker ps
Kubelet is going to pick the yaml file automatically (it's a daemon) and start the nginx container. Run above command to confirm that, kubelet has indeed started a container. Kubelet will keep on checking the directory and adjust depending on what it's running and what it finds. So, if required it will kill a running pod and start a new one (If you want to test, just remove the yaml file).

Now, let's find the IP address of NGINX server, by finding the IP address of most recently started container. 

$ docker inspect $(docker ps -q) | grep IPAddress
Below command will print the content of Nginx welcome page; confirming that Nginx is indeed up. Note that, port configured in yaml file is 80.

 curl http://172.17.0.2 | head -5

Kubelet Validations

Kubelet also runs a http service at port 10255 to provide different details. And it also runs cAdvisor at port 4194.

http://localhost:10255/healtz
http://localhost:10255/spec
http://localhost:10255/pods
http://localhost:4194/containers   #cAdvisor

--happy learning !!!