Friday, October 1, 2021

How to Configure PostgreSQL to Accept All Incoming Connections


Edit pg_hba.conf

Just use 0.0.0.0/0

host    all             all             0.0.0.0/0            md5

Make sure the listen_addresses in postgresql.conf (or ALTER SYSTEM SET) allows incoming connections on all available IP interfaces.

listen_addresses = '*'

After the changes you have to reload the configuration. One way to do this is execute this SELECT as a superuser.

SELECT pg_reload_conf();

** Note: to change listen_addresses, a reload is not enough, and you have to restart the server.


Reference :


Friday, January 10, 2020

Load Balancing Techniques and Optimizations

The hosting world’s bread & butter solution for providing high availability and redundancy is load balancing. There are many different use cases for a Load Balancer (LB). It is important to know how to effectively manage your LB configuration so that it performs optimally in your environment.  The proceeding article will review some of the common practices that, when adhered to, provide a smooth and seamless high availability website/application through the use of load balancing.
Prerequisites
This article will focus on load balancing concepts and uses typical web services (HTTP, HTTPS). These services are independent of the LB device itself so they can run on any combination of operating system and server software, e.g.,  Linux, Windows, Apache, Nginx, IIS, etc. Load balancing is not limited to just web services. Any type of traffic with a client/server relationship can take advantage of load balancing.

What is Load Balancing?

Load balancing is the practice of using a network device, called a Load Balancer (LB), to distribute traffic between a back-end cluster of servers, called nodes. These nodes are virtually identical, each running the same software, services, and configurations. Broken nodes can be easily replaced by additional nodes which are also added to handle traffic growth over time. This prevents a single node from becoming overwhelmed as the LB uses specific load balancing algorithmic methods to determine which node handles subsequent requests. Load balancing is virtually invisible to the end-user, operating behind the scenes allowing a farm of servers to function as a single service or application. Load balancing comprises the backbone of most high availability solutions due to its flexibility, redundancy, and extendability.

OSI Layer Optimization

All load balancing occurs on one of two layers of The OSI Model. These layers allow for traffic balancing configurations based on different information that is contained in the network packet for that specific layer. The two layers involved in this process are the L4 Transport and L7 Application layer.
  • L4 – Transport Layer: The fourth layer allows balancing rules based on transport protocols. Balancing traffic based on details like IP Address or TCP Port are provided within the L4 layer.
  • L7 – Application Layer: The Application Layer provides many additional details that can be inspected from the packet for balancing rules. The L7 layer is where rules can be constructed based on information from HTTP Headers, SSL Session ID, HTML Form Data, Cookies, etc.
Rule of Thumb:
For efficiency use the L4 layer. L4 load balancing happens earlier in the transaction, so that traffic can be routed more quickly than L7. L7 load balancing has to inspect data from the application layer. But, it is not always possible to rely on the L4 layer, and it will depend heavily on the services and back-end node configuration.

Load Balancing Algorithmic Methods

There are several common balancing algorithmic methods which can be used in a load-balanced configuration. Selecting the right algorithm for your infrastructure is critical to load balancing optimization. There is no general, one-size-fits-all,  method for every situation. Choosing the correct approach will depend heavily upon the services, traffic, and software used in the load-balanced cluster. Below are some of the common methods used and their strengths/weaknesses from an optimization perspective.

 Round-Robin-  A simplistic load balancing approach. Traffic is sent to each node in series, one after the other, jumping back to the beginning of the list once the end is reached. (e.g., Node 1 → Node 2 → Node 3 → Repeat)
Round-Robin Pros & Cons:
+ Pros: Lower memory and CPU footprint from the Load Balancer.
– Cons: Not adaptive, sends traffic to nodes without regard for distribution.

 Least Connections   A smart balancing method which uses connection tracking to determine which of the nodes has the least number of active network connections. This tracking is managed by the load balancer device itself distributing every new connection to the back-end node with the least amount of active connections.
Least Connections Pros & Cons:
+ Pros: Adaptive, provided even distribution of workload among nodes.
– Cons: Requires larger memory and CPU footprint for connection tracking.

 Fastest Response Time   A smart balancing method which tracks each nodes network response time from health checks and routes new connections to the server with the quickest response time, regardless of any other factors.
Fastest Response Time Pros & Cons:
+ Pros: Adaptive, low memory, and CPU requirement.
– Cons: Limited workload distribution, connections and workload often fills up one node at a time.

 Random Node   A niche method which is only used in very specific scenarios and is almost never appropriate from an optimization standpoint.
Random Node Pros & Cons:
+ Pros: Requires a low memory and CPU footprint from load balancer device.
– Cons: Niche, only useful in specific scenarios. Unreliable distribution of workload.

Weighted Load Balancing
Another useful load balancing feature to consider is weighted nodes along with the balancing method. Weighted load balancing assigns additional connections to specific nodes over others dependent on each nodes weight value in the configuration. This feature is generally used when back-end nodes are not identical hardware or certain nodes receive special traffic beyond regular load-balanced traffic. Assigning a lower weight value in the form of a ratio to the weaker nodes allows them to participate in the workload at their individual capacity limits without getting overwhelmed. For example, a weight ratio of 2:2:1, would assign two connections to node 1 & node 2 for every 1 connection assigned to Node 3.
Choosing an Optimal Balancing Algorithm
When considering load balancing optimization, stick with the smarter balancing methods like Least Connections or in some cases Fastest Response TimeLeast Connections, in particular, is good at distributing workload between all available hardware without focusing too much on a single node. However, this balancing method will require a beefier load balancing device as the amount of nodes and traffic ramp up. Fastest Response Time can be a great alternative when the hardware available for the load balancer is more limited. Each has its pros and cons as listed above, but both work well as optimized balancing methods for busy server farms.
Traffic Pinning & Session Persistence
Some configurations take advantage of pinning certain traffic to specific back-end nodes depending on the traffic. Session Persistence is a widely used form of Traffic Pinning as it results in routing requests to specific servers outside of the load balancing method used. Another common practice is assigning administration or upload traffic to a specific node that propagates the changes to the rest of the server farm. It is important to consider these connection types when designing or optimizing your server cluster.
Pro Tip:
Least Connections works well with configurations that rely on Traffic Pinning and/or Session Persistence. Since these features group connections to specific nodes independent of the load balancing algorithms. These connects are included in the connection tracking of the Least Connections balancing method which allows the load balancer to proportionately assign traffic to the least busy nodes while accounting for the existing pinned traffic.

Server Farm/Cluster Scope

Some important considerations should be accounted for when designing or upgrading your server cluster. A common mistake that gets overlooked is the size of the cluster. The amount of nodes available in the cluster should not only be enough to handle regular workloads. It’s also necessary to account for both surges in workload as well as failures in the cluster. Ideally, a cluster should be large enough to remain fully operational, even when a surge in workload occurs and a critical event throws a node offline.
Rule of Thumb:
The cluster should be large enough to handle traffic surges plus at-least one extra node for redundancy.

Redundancy: Spare vs. Fail-over Node

Quite simply, redundancy is not effective without an extra node to handle the workload in the event that a critical issue occurs with another node. There are essentially two methods of handling redundancy in a load-balanced cluster.
 Spare Node   An extra node in the cluster that is not needed for the cluster to function. It resided active in the cluster, handling workload just like any other node. However, it’s there in case another node falls. The cluster can carry on without interruption, while the broken node is addressed as needed.
Spare Node Pros & Cons:
+ Pros: Seamless redundancy, there is no downtime between a critical event and the promotion of the spare.
– Cons:  Continuous use of the spare can lead to it failing along with another node when it is needed most.
 Failover Node    An extra node, that has been configured and tested to work normally within the cluster. Once tested, the extra node is assigned as a fail-over node and taken out of the active configuration. The fail-over node, then sits on standby, waiting for a critical event to occur within the cluster. The fail-over node is then automatically activated in the cluster and start handling traffic.
Spare Node Pros & Cons:
+ Pros: Almost no risk of the fail-over node having hardware problems when it is needed.
– Cons: Delay between node failure and activation of the fail-over node.
Either method provides redundancy and keeps the application/site alive during critical times. A combination of both methods can be used as well to have the benefit of both options. It boils down to preference and cost. However, adhering to at least one of these methods should keep you running at an optimal state even during hard times.

Front-loaded Permanent Redirects

One way to improve response times of a load-balanced setup is to mitigate permanent redirects. This is done by moving them onto the load balancer device directly, instead of relying on the back-end nodes to issue a redirect to the client. This can reduce connection counts on both the load balancer and the back-end nodes themselves by eliminating one segment of the redirect process. The following illustrates this further when using a common practice of forcing HTTPS via redirects.
When a back-end node is configured to force HTTPS connections, the HTTP request comes into the load balancer, then is processed by the balancing algorithm and finally sent onto the necessary back-end node as normal. The back-end node then issues the redirect, which instructs the client to reconnect over the HTTPS protocol. The new HTTPS request is also balanced by the algorithm as needed and sent to a back-end node. The net processing result of this type of redirect is:
  • x2 front-end requests (HTTP & HTTPS)
  • x2 back-end requests
  • x2 load balancing algorithmic checks
Now compare this to a front-loaded HTTPS redirect.
In the front-load configuration, the HTTP request hits the load balancer but is not processed by the balancing algorithm. Instead, it immediately issues the redirect back to the client. Then the client reconnects over HTTPS protocol, which then invokes the algorithm as needed and is sent onto the back-end node as required. The net processing result of this configuration is:
  • x2 front-end requests (HTTP & HTTPS)
  • x1 back-end request (HTTPS only)
  • x1 load balancing algorithmic checks
This example only illustrates a single request. However, load balancers typically handle hundreds or even thousands of requests concurrently. It is easier to see the overall benefit of a change like this when scaling up the example:

SSL Decryption

There are a handful of ways that load balancers are configured to handle SSL encrypted connections like HTTPS. Encrypted connections are more cumbersome on the load balancer device than non-encrypted connects. The process of validating the certificate chain and then decryption the content adds additional workload to every request handled by the load balancer. This can be mitigated depending on the configuration of the back-end cluster servers and the applications/website’s encryption requirements. The following are the common configuration scenarios and their pros/cons.
 SSL Passthrough –   The LB is configured to pass any encrypted connections through to the necessary back-end nodes. Decryption is handled by the back-end nodes only and not the LB device.
SSL Passthrough Pros & Cons:
+ Pros: Simple configuration, high security, low CPU/memory footprint. Offers end-end-encryption.
– Cons: Limited to L4 Layer load balancing. Cannot perform packet inspection.
 Decryption Only –  The LB device itself will perform the necessary decryption. Traffic is then balanced to the back-end nodes after decryption. Due to security concerns, this configuration is only recommended when the back-end nodes and load balancer device have an isolated LAN to communicate through.
Decryption Only Pros & Cons
+Pros: Allows packet inspection for L7 Layer load balancing rules. Back-end nodes do not handle decryption.
– Cons: Any machines on the network can read decrypted back-end traffic.
 Decryption + Re-encryption–  Decryption is handled by both the LB device and then re-encrypted and sent to the back-end node as needed. The back-end node also performs decryption providing full end-to-end encryption. This method is for high-security setups that also require L7 load balancing rules.
Decryption + Re-encryption Pros & Cons:
+ Pros: Maximum security with end-to-end encryption. L7 load balancing rules & full packet inspection.
– Cons: Higher workload demand on the LB device. Encryption must be set up on both the front and back-end.
The load balancing optimization stance is the less workload the LB performs, the faster it responds to requests. This makes SSL Passthrough the ideal choice as it requires the least amount of work from the LB device. However, this is not always the practical solution for some applications/sites, particularly those that require L7 layer load balancing or other packet inspection needs. Decryption Only requires less workload when compared to Decryption + Re-encryption, but should only be leveraged when running a dedicated LB on a private LAN with the back-end nodes preventing outside entities from abusing the non-encrypted back-end traffic.

Reference:





Monday, November 4, 2019

Java execute shell commands on SSH Unix using JSch



Today we will look into the JSch example tutorial. We can use JSch for creating SSH connection in java. Earlier I wrote a program to connect to remote database on SSH server. Today, I am presenting a program that can be used to connect to the SSH-enabled server and execute shell commands. I am using JSch to connect to remote ssh server from java program.

JSch Example

You can download JSch jar from its official website. You can also get the JSch jars using below maven dependency.

<dependency>
    <groupId>com.jcraft</groupId>
    <artifactId>jsch</artifactId>
    <version>0.1.53</version>
</dependency>
Below is a simple JSch example program.

import java.io.InputStream;

import com.jcraft.jsch.Channel;
import com.jcraft.jsch.ChannelExec;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;


public class JSchExampleSSHConnection {

 /**
  * JSch Example Tutorial
  * Java SSH Connection Program
  */
 public static void main(String[] args) {
     String host="ssh.journaldev.com";
     String user="sshuser";
     String password="sshpwd";
     String command1="ls -ltr";
     try{
      
      java.util.Properties config = new java.util.Properties(); 
      config.put("StrictHostKeyChecking", "no");
      JSch jsch = new JSch();
      Session session=jsch.getSession(user, host, 22);
      session.setPassword(password);
      session.setConfig(config);
      session.connect();
      System.out.println("Connected");
      
      Channel channel=session.openChannel("exec");
         ((ChannelExec)channel).setCommand(command1);
         channel.setInputStream(null);
         ((ChannelExec)channel).setErrStream(System.err);
         
         InputStream in=channel.getInputStream();
         channel.connect();
         byte[] tmp=new byte[1024];
         while(true){
           while(in.available()>0){
             int i=in.read(tmp, 0, 1024);
             if(i<0)break;
             System.out.print(new String(tmp, 0, i));
           }
           if(channel.isClosed()){
             System.out.println("exit-status: "+channel.getExitStatus());
             break;
           }
           try{Thread.sleep(1000);}catch(Exception ee){}
         }
         channel.disconnect();
         session.disconnect();
         System.out.println("DONE");
     }catch(Exception e){
      e.printStackTrace();
     }

 }

}
Let me know if you face any problem with the execution of the JSch example program. It’s pretty straight forward example of JSch to create SSH connection in java program.
You can download JSch jar file from its official website.

Reference :

Saturday, October 19, 2019

How to Enable Spring Boot with SSL (HTTPS) and HTTP/2 on localhost



The web is moving to HTTPS. More and more sites are only accessible with HTTP over TLS. Thanks to Let's Encrypt you have access to free TLS certificates and with the ACME protocol a way to automate the certificates management.
But there is one area where TLS is not that prevalent, in our development environment. This is a bit of a problem because more and more features in the browsers require a secure context. For example Geolocation, Service Workers, Web Crypto and others. These features only work when the page is served over HTTPS, but fortunately browsers make an exception for connections to localhost and 127.0.0 and you can work with these features in your development environment with HTTP over plaintext TCP.
But there is one feature that requires a TLS connection, HTTP/2. If you want to use HTTP/2 in your development, you have to have TLS enabled. There is a specification for using HTTP/2 over cleartext TCP, but browsers and Spring Boot did not implement it.
Another reason to use TLS in your development environment is the mixed content issue. For example, you have a HTML page that references the jQuery library with HTTP.
<!DOCTYPE html>
<html lang="en">
<head>
    <title>Mixed Content</title>
</head>
<body>
    <script src="http://code.jquery.com/jquery-3.3.1.slim.min.js"></script> 
</body>
</html>
In your development environment you use HTTP over cleartext TCP and everything looks good and works. Then you deploy this web page to a production server which serves the resources over HTTPS. And suddenly your application is no longer working, because browsers refuse to load resources over an insecure connection when the host page has been loaded over a secure connection.
The browser prints this error message in the developer tools console:
Mixed Content: The page at 'https://localhost:8443/index.html' was loaded over HTTPS, but requested an
insecure script 'http://code.jquery.com/jquery-3.3.1.slim.min.js'. This request has been blocked; 
the content must be served over HTTPS.
You see that there are valid reasons to always develop and test your application with the same protocol that you use in your production.

mkcert

Unfortunately, it is not that easy to set up TLS on your local machine, because you can't simply get a TLS certificate for localhost or 127.0.0.1. You could create self signed certificates, but browsers show you some ugly warning messages and it's not very convenient.
Another workaround is to use tools like ngrok and localtunnel. They give you a HTTPS address to your application, but the drawback is that the traffic is routed from your computer to the ngrok resp. localtunnel servers and then back to your computer. So it won't work when you don't have an Internet connection, for example if you want to do some development during a flight. Although, these services are very convenient when you want to give somebody outside of your network access to your computer. For example presenting a co-worker or customer a web application that your are working on.
The better solution is to install your own private CA (certification authority) and create TLS certificates that are signed with this CA and also configure your operating system and browsers to trust this CA.
Setting this up is a bit complicated but it is possible doing it from scratch with tools like openssl or the keytool from Java.
But in this example we use a tool that simplifies the setup process quite a lot: mkcert
It's a command line tool written in Go and runs on Windows, Linux and macOS. See the readme on how to install it. For this blog post I'm going to demonstrate the tool on Windows 10. I downloaded the executable from the release page and saved it in an arbitrary directory.
First run the -install command. This creates a private CA, configures the operating system and browsers to trust this CA. It also automatically adds the CA to Java if it finds a JAVA_HOME environment variable.
mkcert-v1.4.0-windows-amd64.exe -install
This creates two files rootCA.pem and rootCA-key.pem in your home directory (C:\Users\<USER>\AppData\Local\mkcert). It registers the CA in the Windows system certification store. Browsers like Chrome and Firefox read root certificates from this store.
You can verify the entry with the Windows certification manager tool
certmgr.msc
You find the CA under Trusted Root Certification Authorities -> Certificates
Next, we create a TLS certificate for the domains localhost127.0.0.1 and ::1, that is signed by our own private CA. By default, mkcert creates certificates in the PEM format. Because we want to use the certificate in a Java application (Spring Boot) and Java can't load PEM certificates, we have to create the certificate in the PKCS#12 format.
mkcert-v1.4.0-windows-amd64.exe -pkcs12 localhost 127.0.0.1 ::1
This creates a new file localhost+2.p12 in the current directory. The PKCS#12 bundle is secured with the password changeit.

Spring Boot

In this section we create a trivial Spring Boot application and enable TLS and HTTP/2 with our newly created TLS certificate.
I usually bootstrap my Spring Boot applications with a visit to https://start.spring.io. In this case I utilize the curl method. Run the following command in your command prompt:
curl https://start.spring.io/starter.zip -d dependencies=web,thymeleaf -d javaVersion=11 -d groupId=ch.rasc -d artifactId=h2demo -o h2demo.zip
Unzip the h2demo.zip file and copy the certificate localhost+2.p12 into the root folder of your project.
Open src/main/resources/application.properties, it's empty by default, and insert the following content:
server.http2.enabled=true
server.port=8443

server.ssl.enabled=true
server.ssl.key-store=./localhost+2.p12
server.ssl.key-store-type=PKCS12
server.ssl.key-store-password=changeit
With these settings we enable TLS and HTTP/2 and set the listening port to 8443. When you enable TLS in Spring Boot, you also have to specify the key store, the format and the password.
To test, if everything works, we write a simple RestController and a GET endpoint.
@SpringBootApplication
@RestController
public class Application {

 public static void main(String[] args) {
  SpringApplication.run(Application.class, args);
 }

 @GetMapping("/")
 public String helloWorld() {
  return "Hello World";
 }
}
Start the application from the command line or inside your IDE.
.\mvnw.cmd spring-boot:run
Check the network tab in the browser developer tools
You see that this request has been served with HTTP/2 over TLS

Charles

Charles is a HTTP proxy, monitor, reverse proxy to inspect all the HTTP and HTTPS traffic between any application on your machine and the Internet. Very similar to the network tool in the browser developer tools, but it is not limited to browsers it can intercept traffic from all applications running on your computer.
To inspect the TLS connection between our browser and Spring Boot we need to install our private root certificate into Charles.
mkcert creates the root certificate in the PEM format, which Charles can't read. We therefore have to convert the file into a PKCS#12 file.
If you don't remember, where mkcert has stored the root certificate, run the following command. It tells you the directory containing the root certificate.
mkcert-v1.4.0-windows-amd64.exe -CAROOT
In a command prompt change into this directory and execute the following command.
openssl pkcs12 -export -out rootCA.pkcs12 -inkey rootCA-key.pem -in rootCA.pem
Enter the password changeit and openssl creates a new file rootCA.pkcs12 from the PEM file.
Start Charles, open the menu Proxy -> SSL Proxying Settings, open the tab Root Certificate and select the pkcs12 file we've just created.
Open the SSL Proxying tab and enable SSL Proxying. Add a new entry to the location list (localhost:8443).
Open the URL https://localhost:8443/ in your browser, you should see the traffic in Charles if everything is configured correctly.

HTTP/2 Push

With a working TLS and HTTP/2 Spring Boot application we can now start experimenting a bit with HTTP/2 push, a new way to send content from a server to the client.
A typical workflow of a HTTP request for a HTML page looks like this
  1. Browser sends GET request
  2. Server responds with the HTML page
  3. Browser parses the HTML code and looks for references to other files in tags like <img><link><script> and others
  4. Browser sends requests for all the referenced resources
  5. Server sends back the requested resources
  6. Browser displays the page
With HTTP/2 the server has the ability to push resources to the client. For example, the following page contains an <img> tag. We, as the developer of the page, know when a browser requests this HTML page he also needs the image, so why not send it together with the HTML page and that is what HTTP/2 push provides.
<!DOCTYPE html>
<html lang="en">
<head>
    <title>HTTP2 Push Test</title>
</head>
<body>
    <img src="cat.webp">
</body>
</html>
The workflow for a HTTP request for this particular page with HTTP/2 push follows these steps
  1. Browser sends GET request
  2. Server responds with the HTML page AND the image cat.webp
  3. Browser parses the HTML page, sees the <img> tag and looks for it in the push cache (a special cache for resources that have been pushed from the server). Because he finds the image in there he immediately displays the page without sending any further requests.
There is one caveat, the browser cache. Without using push, after the browser has parsed the HTML page, he checks if any of the referenced files are stored in one of his caches. If they are, he does not send additional requests to the server and retrieves the resources from the local cache. This saves bandwidth and the browser can display the page faster.
With HTTP/2 push this is different, the server has no knowledge of whether the files are cached or not. He always pushes the files to the client. This would waste a lot of bandwidth, but fortunately browsers solve that problem by cancelling the push connection as soon as they have checked the cache and find the resources in there.

To observe this behaviour I've created two endpoints in my Spring Boot application. In Spring Boot everything is already built-in, we only have to specify the resources we want to push.
  @GetMapping("/withoutPush")
  public String withoutPush() {
    return "index";
  }
  
  @GetMapping("/withPush")
  public String withPush(PushBuilder pushBuilder) {
    if (pushBuilder != null) {
        pushBuilder.path("cat.webp").push();
    }
    return "index";
  }
index references the HTML page, mentioned above, that is stored in the src/main/resources/template folder. javax.servlet.http.PushBuilder is a class from the Servlet 4 implementation and allows us to specify which resources we want to push in addition to the HTML page.
Start the application and then open the network tab in the browser.
When we call /withoutPush with an empty browser cache, we see the typical waterfall of requests. The browser receives the HTML, parses it and requests the image.
With a populated browser cache the browser does not have to send an additional request for the image, he can retrieve it from the local cache
With push and an empty browser cache we see that the browser also sends just one request to the server.
When the image is already stored in the cache, the browser cancels the push stream. Because you can't see that in the browser developer tools, I show you here a screenshot of Charles. There you see that Chrome closes the stream before the server is able to send the complete picture. 
This was just a brief overview about HTTP/2 push. If you wan to dig deeper into HTTP/2 push, I recommend reading this article from Jack Archibald. He writes about all the pitfalls and different browser implementations of HTTP/2 push: https://jakearchibald.com/2017/h2-push-tougher-than-i-thought/

You have seen in this article that setting up TLS on your localhost is not that complicated, thanks to the mkcert tool. With a valid TLS certificate, setting up TLS on Spring Boot and Java 11 is also very easy, because it provides everything out of the box for running a TLS and HTTP/2 server.


Reference: