As an individual user, we are no longer living the world of one computer, rather we are living with living and distributed smart devices. Let us first explore a Single-User/Single-Server Architecture that we used in those times, which we survived, without of much hassle either, we still use this architecture most of the time.
A Single user still use to approach a network via a web application with a local front cache, which then allows user to access requested material as per the business logic (security/privacy is checked before data release) using local business hash-maps and locks (which data needs to be unlocked), next comes Data Tier with some framework having 2nd local level cache, which leads the request of the user to a single source of the data, it might be a database or a Hadoop Cluster to serve the user’s needs. In case user’s need expand and more data requests start coming, this means one server is not enough.
At the time, when local server is not enough to provide complete data set that is requested by the user(s), we need to scale this requesting application to multiple servers, in other words horizontally, using AWS, Rackspace or inter-organization, local area networks. This raises an interesting question, how do we do horizontal scaling of an egress request From:
User ------> Cloud -----> Application
User ------> Cloud ------> Cluster [ Server 1 + Server 2 + Server 3 + Server 4]
This can be a local area network (LAN), if it is within an organization. Here we have seen a paradigm shift from a local architecture of egress request of data into a Clustered Architecture. This extension of cluster data access by a single application is beyond local JVM (Java Virtual Machine). We have to take care of several aspects, when we enhance the application to several servers horizontally. When we add more servers to a cluster to have more load, it is called horizontal scalability. We have to make sure that we look into having the following attributes achieved in some way:
Risk and Failure Tolerance
Now in the world of Big Data – Sharding
More to come later...