Should I upgrade the server or add another one?

When you think about scaling your architecture in a Vertical or Horizontal way (see my post Horizontal and vertical scalability), maybe you could find yourself wondering what the heck you should do. I’ll try to give you some scenarios where you should opt for one or another.

  1. So, you would need to upgrade your server, or even buy a bigger one (do a Vertical scaling) when:
    • You have some processing that can’t be splitted in more than one machine, or can’t be parallelized like:
      • Images rendering
      • Legacy batch processing
      • Transactions
      • Any other big process that is like a “atom”, can’t be divided (well… actually an atom can, but…)
    • You realized that your performance issues are just a lack of:
      • Memory
      • CPU
      • Storage
      • Networking
      • etc
  2. Then, you would need to add a new server to your architecture (do a Horizontal scaling) when:
    • You already has a cluster up and running but the servers are all working close to the limit:
      • Free tip: don’t ever, EVER, let this happening. If your cluster is working close to the limit and one of the servers fails, guess what? Yes, all the others will fail right after like a domino.
    • Your cluster is ok but you are foreseeing a growth in its load;
    • You have many things running in one server and they can easily (or not) be distributed like:
      • Services that are not related: database, e-mail server, applications, etc
      • The layers of your application (if it was right designed for it):
        • Web layer
        • Business layer
        • Data layer
        • Integration layer
        • Whatever layer
    • You already use micro services and one or more services need to be clustered or load balanced.

Of course, these are only some examples! If you know some others feel free to tell me at the comments bellow.

What if I don’t know that much about high availability?

Imagine this situation: you are the new guy in a new project. Maybe you were just hired by the company, or it is a side project that you just sold (great!), or even is just a new project at the organization where you actually works.

And then somebody says those scary words: “we need this application to be highly available”. Now imagine that you have no clue on what the hell they are talking about.

Before you start to crying or pretend to pass out, let’s have some pratical steps that may be helpful to understand what your customer is asking.

1-First of all, ask your customer what “highly available” means to him/her

This is key to know whatever you should do with your solution. You could get answers like:

  • “The system should be available from 08AM to 17PM, from Monday to Friday”;
  • “The system should be available 24 hours a day, from Monday to Saturday”;
  • “The system should be available 24 x 7 (24 hours a day, 7 days a week)”.

Of course you could get many other answers. The point is that each answer above would require a totally different kind of solution. The architecture should be different and the development also should.

2-Ask what are the exceptions related to the question (and answer) above

His/Her first answer surely will be “no exception at all!”. But trust me.. insist! There are many situations, both predictable and unpredictable, that may cause the application to be unavailable:

  • Blackout
  • Network failing
  • Deployment
  • OS failing
  • etc

Of course you can deal with each of these items and that’s why you need to know what are the exceptions. This can be easy to handle if the system should be available from 08AM to 17PM, from Monday to Friday… but the things get tricky when you need to design a 24 x 7 solution.

In any scenario would be nice to have some index to help managing the customer expectations. For example, between the availability defined, the system will available 99.5% of the time, or 95%… or 99.99%!

And then comes the third question…

 

3-What is the budget available? Is it highly available?

(Of course you shouldn’t do the second part of the question… or should, whatever…)

Depending on the budget available, the answer to the previous question should be reviewed. Will be hard to build a solution that is available 24 x 7, 99.99% of this time, with just one server… no cluster, no replication, no UPS (Uninterruptible Power Supply)… no contingency at all!

So if your customer is insisting in give no exceptions at all at the second question, maybe he/she will review the decision at this moment!

 

Now that you have an overview of what your customer want, what are the exceptions and what is the budget, you can do a better planning of your solution (both architecture and development). And can even brag a little about yourself for knowing all the time what was this “highly availability” thing!

Horizontal and vertical scalability

Hi people!

Let’s talk a little about systems architecture. Specially about scalability.

First of all: do you know what scalability is? Accordingly to the book “Characteristics of scalability and their impact on performance”, scalability is the “capability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged in order to accommodate that growth“.

I agree partially! I would go further and tell that your system should be able to handle more work and even less work, what could be called a downsize. This concept of scalability is also known as elasticity.

Well, but this is not the point here! 😉

So if I need to scale my application somebody could ask me if would do a Horizontal or a Vertical scale. What should I answer?

 

Horizontal Scalability

Horizontal scalability is when you add another servers in your architecture. Usually for one or more of these reasons:

  1. Distribution: you can use more servers to make your application more distributed. You can do it by splitting it by modules, or by micro services, or by tiers (example: web tier, business tier and data tier);
  2. Clustering: you add more servers to share the work between them;
  3. Load balancing: maybe you already has a distributed application or you already use a cluster, but you want add more servers to not only share the work, but also balance the load between them.

In this type of scalability you can use any type of server “size”, depending on what you need. For some very specific uses you may not need a huge processing, memory or things like this.

Horizontal scalability is type that most fits for elasticity as you may want to add or remove servers depending on your needs.

 

Vertical Scalability

Vertical scalability is when you “grow” your server instead of getting another one. You can grow your server by increasing its:

  1. Memory
  2. Core processing
  3. Video processing
  4. Data processing
  5. Data storing

In this type of scalability you usually will grow your server and rarely would “downsize” it. It fits better on situations where you have a very specific service or task running in one single server and need to improve it.

Also fits to systems that were not built for being distributed, clustered or balanced.

 

Mixing them both

You may want to mix the two types! For example, you use the Horizontal type and add more servers, but one of the server need to be upgraded for better results so you apply the Vertical type on it.

 

So, what do you think? Do you know another ways of scaling applications? Share with me at the comments!

Dynamic and Static Discovery cluster with Apache TomEE

Hey there!

On my last post Are there different types of clusters? we saw two different types of clusters: Dynamic Discovery and Static Discovery. To see details, see the post! 😉

Now let’s see quickly how we can set them up at the Apache TomEE. One of the types – the Dynamic – we have already seen in the post How to build an Apache TomEE cluster. So we will do just a little zoom to see where the things are done.

 

Dynamic Discovery

At the “Engine” node you should put this code (see the post mentioned above for detailed code):

<Cluster
className=”org.apache.catalina.ha.tcp.SimpleTcpCluster”
channelSendOptions=”6″>
<Manager…
<Channel…
<Valve…
<Deployer…
<ClusterListener…
</Cluster>

 

Static Discovery

At the same “Engine” node you should put this:

<Cluster className=”org.apache.catalina.ha.tcp.SimpleTcpCluster”channelSendOptions=”6″>
<Channel className=”org.apache.catalina.tribes.group.GroupChannel”>
<Interceptor className=”org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor”>
<Member className=”org.apache.catalina.tribes.membership.StaticMember” port=”4000″ host=”server1″ uniqueId=”{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}” />
<Member className=”org.apache.catalina.tribes.membership.StaticMember” port=”4000″ host=”server2″ uniqueId=”{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}” />
<Member className=”org.apache.catalina.tribes.membership.StaticMember” port=”4000″ host=”server3″ uniqueId=”{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3}” />
</Interceptor>
</Channel>
</Cluster>

 

So as you can see, with the Static Discovery you use the StaticMember at the Member node instead of McastService used at the Dynamic Discovery. To see the functional differences between them see my last post Are there different types of clusters?.

Hope you enjoy building your own TomEE cluster! If you have any particular experience to share, please leave it at the comments bellow. See you!

Are there different types of clusters?

In my last post “How to Build an Apache TomEE cluster” we saw how easy and fast is to deploy your application in a cluster and take all its advantages.

It’s interesting to keep in mind we have basically two different types of clusters. Let’s see them.

 

Dynamic Discovery

In a Dynamic Discovery cluster each node is ready to find another nodes in its network as soon as they are up and running.

Pros

It’s the most flexible (and dynamic!) way to build your cluster, as you can scale your architecture by just turning a server on or off.

Cons

You need to take care to not put on your cluster a node you don’t want to. If you have different clusters for different applications in the same network and all of them are Dynamic Discovery clusters, they will “see” each other nodes and you can imagine the mess! Of course that are some configurations to avoid it, but you need to do it very carefully.

 

Static Discovery

In a Static Discovery cluster each node has a previous knowledge of all nodes that are in the cluster. No surprises!

Pros

You have much more control of your cluster and your architecture than in a Dynamic Discovery cluster. As the nodes are previously defined you can set another cluster in the same network without worrying about if they will see each other, because they won’t.

Cons

There are not much flexibility for scalability as you need to configure all your nodes if you want to add a new one (to remove just shut it down). It’s ok if you have 2 or 3 servers (nodes), but imagine 10… 20… 100!!

 

So… what’s the best one? Depends! Different scenarios and architectures should use the best features of one type or another. If you want my opinion just ask it!

 

How to build an Apache TomEE cluster

If you have an application or a system and it is running, you surely want it to be available. Depending on the scenario, you need it to be highly available! Or perhaps you just want to balance the loading on your server.

In any of those cases a good choice maybe is to build a cluster. Just to give you an overview and in the case this is a new word for you, cluster is a set of independent servers that communicate to each other thru a network in order to make a service or a system to be more available than if you used just one server.

Each server of the cluster is called a node. That are some different types of cluster, but we will cover it in another post.

If you are a Java developer and are familiar with Tomcat you will find TomEE quite easy to use. They have almost the same structure and are very likely to setup. The “plus” for TomEE is: it is Java EE (6) compatible for web profile.

What does it mean? Means that it implements all the Java EE 6 web API’s related. So if you are a Web Developer and wanna take advantage of Java EE 6 on your project, the Apache TomEE could be a good choice.

So you have an application, want to run it in a TomEE and want to do it in a cluster? Easy! Let’s do it step by step.

  • Download and install the JDK and the Apache TomEE. I’ll assume you are familiar with it. I am wrong, please leave a message at the comments bellow;
    • The versions used were: JDK 1.8.73 and Apache TomEE 1.7.3 Web Profile. The SO was a Mac OS X 10.11.3 (El Captain).
  • After installing try to run your TomEE and see if it is working;
  • Go to your TomEE Home folder and edit the following file:
    • /conf/server.xml
  • Add those lines to the “Engine” node:
<Cluster
className=”org.apache.catalina.ha.tcp.SimpleTcpCluster”
channelSendOptions=”6″>
<Manager
className=”org.apache.catalina.ha.session.BackupManager”
expireSessionsOnShutdown=”false”
notifyListenersOnReplication=”true”
mapSendOptions=”6″ />
<Channel
className=”org.apache.catalina.tribes.group.GroupChannel”>
<Membership
className=”org.apache.catalina.tribes.membership.McastService”
address=”228.0.0.4″
port=”45564″
frequency=”500″
dropTime=”3000″ />
<Receiver
className=”org.apache.catalina.tribes.transport.nio.NioReceiver”
address=”auto”
port=”5000″
selectorTimeout=”100″
maxThreads=”6″ />
<Sender
className=”org.apache.catalina.tribes.transport.ReplicationTransmitter”>
<Transport
className=”org.apache.catalina.tribes.transport.nio.PooledParallelSender” />
</Sender>
<Interceptor
className=”org.apache.catalina.tribes.group.interceptors.TcpFailureDetector” />
<Interceptor
className=”org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor” />
<Interceptor
className=”org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor” />
</Channel>
<Valve className=”org.apache.catalina.ha.tcp.ReplicationValve”
filter=”.*\.gif|.*\.js|.*\.jpeg|.*\.jpg|.*\.png|.*\.htm|.*\.html|.*\.css|.*\.txt” />
<Deployer
className=”org.apache.catalina.ha.deploy.FarmWarDeployer”
tempDir=”/tmp/war-temp/”
deployDir=”/tmp/war-deploy/”
watchDir=”/tmp/war-listen/”
watchEnabled=”false” />
<ClusterListener
className=”org.apache.catalina.ha.session.ClusterSessionListener” />
</Cluster>
  • Save and close your file, then restart your TomEE. If everything is ok, your log file should have something like this:
org.apache.catalina.ha.tcp.SimpleTcpCluster startInternal
Cluster is about to start
org.apache.catalina.tribes.transport.ReceiverBase bind
Receiver Server Socket bound to:/192.168.0.104:5000
org.apache.catalina.tribes.membership.McastServiceImpl setupSocket
Setting cluster mcast soTimeout to 500
org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
Sleeping for 1000 milliseconds to establish cluster membership, start level:4
org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
Done sleeping, membership established, start level:4
org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
Sleeping for 1000 milliseconds to establish cluster membership, start level:8
org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
Done sleeping, membership established, start level:8
  • Great! Now you have the first node of your cluster up and running. Not enough, right? So do it to another server, could be another machine or even a VM. Do it to as many servers as you want according to your needs. Each time you add a new node, the log file of other nodes should contain something like this:
org.apache.catalina.tribes.io.BufferPool getBufferPool
Created a buffer pool with max size:104857600 bytes of type:org.apache.catalina.tribes.io.BufferPool15Impl
org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 0, 107}:5000,{192, 168, 0, 107},5000, alive=1016, securePort=-1, UDP Port=-1, id={111 -38 59 -66 -68 -29 72 -69 -103 12 -121 -120 -13 -25 -90 17 }, payload={}, command={}, domain={}, ]
  • Now you have your beautiful cluster fully happy and working! There is just one thing missing: your application deployed to it. Simple:
    • Edit the web.xml of your application;
    • Add this attribute: <distributable />;
    • Save the file, close it and deploy to your cluster.

Couldn’t be easier, right? Of course there are some other aspects, some architectural gaps and another options, but I will leave it for your comments and for another posts.

See ya!