Web Distributed Systems Design

Principles of Web Distributed Systems Design 

• Availability: The uptime of a website is absolutely critical to the reputation and functionality of many companies. For some of the larger online retail sites, being unavailable for even minutes can result in thousands or millions of dollars in lost revenue, so designing their systems to be constantly available and resilient to failure is both a fundamental business and a technology requirement. High availability in distributed systems requires the careful consideration of redundancy for key components, rapid recovery in the event of partial system failures, and graceful degradation when problems occur. 

• Performance: Website performance has become an important consideration for most sites. The speed of a website affects usage and user satisfaction, as well as search engine rankings, a factor that directly correlates to revenue and retention. As a result, creating a system that is optimized for fast responses and low latency is key. 

• Reliability: A system needs to be reliable, such that a request for data will consistently return the same data. In the event the data changes or is updated, then that same request should return the new data. Users need to know that if something is written to the system, or stored, it will persist and can be relied on to be in place for future retrieval. 

• Scalability: When it comes to any large distributed system, size is just one aspect of scale that needs to be considered. Just as important is the effort required to increase capacity to handle greater amounts of load, commonly referred to as the scalability of the system. Scalability can refer to many different parameters of the system: how much additional traffic can it handle, how easy is it to add more storage capacity, or even how many more transactions can be processed. 

• Manageability: Designing a system that is easy to operate is another important consideration. The manageability of the system equates to the scalability of operations: maintenance and updates. Things to consider for manageability are the ease of diagnosing and understanding problems when they occur, ease of making updates or modifications, and how simple the system is to operate. (i.e., does it routinely operate without failure or exceptions?) 

• Cost: Cost is an important factor. This obviously can include hardware and software costs, but it is also important to consider other facets needed to deploy and maintain the system. The amount of developer time the system takes to build, the amount of operational effort required to run the system, and even the amount of training required should all be considered. Cost is the total cost of ownership. 

Designing Software Architecture basics

How an architecture is prepared

How does the Architecture world works or move 

[from Open Group]

Desired Architecture for today’s world

Architectural Patterns and Styles

An architectural style, sometimes called an architectural pattern, is a set of principles—a coarse grained pattern that provides an abstract framework for a family of systems. An architectural style improves partitioning and promotes design reuse by providing solutions to frequently recurring problems. You can think of architecture styles and patterns as sets of principles that shape an application.

An understanding of architectural styles provides several benefits. The most important benefit is that they provide a common language. They also provide opportunities for conversations that are technology agnostic. This facilitates a higher level of conversation that is inclusive of patterns and principles, without getting into specifics. For example, by using architecture styles, you can talk about client/server versus n-tier. Architectural styles can be organized by their key focus area. The following table lists the major areas of focus and the corresponding architectural styles.

Combining Architectural Styles

The architecture of a software system is almost never limited to a single architectural style, but is often a combination of architectural styles that make up the complete system. For example, you might have a SOA design composed of services developed using a layered architecture approach and an object-oriented architecture style.

A combination of architecture styles is also useful if you are building a public facing Web application, where you can achieve effective separation of concerns by using the layered architecture style. This will separate your presentation logic from your business logic and your data access logic. Your organization's security requirements might force you to deploy the application using either the 3-tier deployment approach, or a deployment of more than three tiers. The presentation tier may be deployed to the perimeter network, which sits between an organization's internal network and an external network. On your presentation tier, you may decide to use a separated presentation pattern (a type of layered design style), such as Model-View-Controller (MVC), for your interaction model. You might also choose a SOA architecture style, and implement message-based communication, between your Web server and application server.

If you are building a desktop application, you may have a client that sends requests to a program on the server. In this case, you might deploy the client and server using the client/server architecture style, and use the component-based architecture style to decompose the design further into independent components that expose the appropriate communication interfaces. Using the object-oriented design approach for these components will improve reuse, testability, and flexibility.

Many factors will influence the architectural styles you choose. These factors include the capacity of your organization for design and implementation; the capabilities and experience of your developers; and your infrastructure and organizational constraints. The following sections will help you to determine the appropriate styles for your applications.

Maintain Multi Session on Multiple Tabs of browser

How do you maintain multiple sessions on different tabs of browser.

Yes this is an interesting topic.

I knew it was easy if we were using cookieless session, but ours is a cookie enabled session.

I was struggling all the day and then I finally got a hint that we can use Web.config to handle above issue.

Wondering how?

 <sessionState cookieless="UseUri" regenerateExpiredSessionId="true">
  </sessionState>

ASP.Net MVC in a flash for Experienced

Request is received by Routing table which creates RouteData Object [RouteCollection] and passes it to IRouteHandler.

IRouteHandler decides which handler to be invoked to handle the request, here in this case it would be MVC Handler.

MVC Handler looks at the url and passes the controller name to IController Factory which checks the controller presence and returns Controller instance to Handler. Now the Handler invokes the execute method of the returned Controller instance and passes the request to ActionSelector.

Action Selector has two ways to handle the action

1. ActionName selector
2. ActionMethod selectors

Action selector resolves the Action and request is passed to IActionInvoker. By default Action selector loads all the methods of the class and based on URL it calls the action method.

IActionInvoker calls the IActionFilter’s which runs in below sequence

1. IAuthentication filter
2. IAuthorization filter
3. IAction filter
4. IException filter
5. IResult filter

The IAction filter makes use of IModelBinder which in turn takes help of ValueProvider to resolve the action parameter value(s) and the action method gets executed and the result is passed to IViewEngine.

If Result is View or PartialView then IViewEngine loads the

• WebForm View Engine(.aspx)
• Razor View Engine (.cshtml)

MVC does not support method overloading for Action methods, we can achieve this by using Action Name or by using Attribute [HTTPPost/ HTTPGet]

IModelBinder does 3 tasks

1. Resolves value
2. Conversion
3. Validates

It takes help of Value Provider to resolve/ get actual value.

ValueProvider

| ChildActionValueProvider

| FormValueProvider

| JSONValueProvider

| RouteDataValueProvider

| QueryStringValueProvider

| HttpFileCollectionValueProvider

|JQueryFormValueProvider

We can create Custom ValueProvider if required. Eg:- To read from Cookie we can extend ValueProvider.

To post a form we have 3 ways in MVC.

1. Using parameter with button name in action
2. Use hidden field set value before post
3. Use Action attribute of the button in HTML5 which would change URL in browser.

When we add layout page to a View bootstrap gets included in the project automatically.

Partial Views are used to send partial HTML & avoid layout settings.

There are three ways in which Partial View can be included in a page.

1. @RenderPage 

1. is faster because it writes to Response.
2. Does not use lookup method
3. Is used for static content

2. @HTML.Partial - Used with strongly typed views.

3. @HTML.RenderPartial   - Supports both strongly typed & non-static. Best suited for Ajaxified pages.

@Section are to be used for local style/ css or something page specific, these sections are declared in layout and in Views we can use these sections to embed Style/ css/ js files.

Browsers have limitation to download 3 files at a time.

The JSONResult action returns Microsoft JSON format, this JSON cannot be used with JQuery directly. JQuery understands NewtonSoft JSON, so this has to be converted to NewtonSoft JSON.

Ajax Helpers

Four steps should be followed to use Ajax helpers

1. Install unobtrusive javascript
2. Use it as Ajax.ActionLink
3. Provider the Ajax options to method
4. Include validate.js & unobtrusive.jquery.validation.js

To get the validation work

1. Set config setting unobtrusive & client validation to true.
2. Apply Data annotation attribute on properties.
3. Only use strongly typed helpers or template helpers to generate UI.
4. Refer jquery Validate & unobtrusive js files on page
5. ModelState to be checked before operation.

Customising Routes

We customise routes only when URL parameters must be altered.

1. Change parameter name [id to something(PId)].
2. Change parameter order
3. Update no. of parameters
4. Change URL value to controller name, a kind of mapping to avoid the end client know the controller name and action name.

We can add constraints to route which would validate the value type using Regex.

Declarative Helpers

This feature helps us to avoid creating partial view for displaying content which have less text. For eg:- If we have to display DateTime on each page we can create cshtml and keep it inside the App_Code folder which gets complied to a class and at runtime is available to Views.

Cshtml would look like

@helper HelloWorld(string message)

{

<h1>Welcome @message</h1>

<h2>@DateTime.ToString() </h2>

}

In the Views we include Declarative helpers as @<FileName>. <METHOD_NAME>

Design Patterns for Agile world

Design patterns with Design principle for Agile world

What are Design patterns?
Design patterns are solutions to real world problems that pop up time and again, so instead of reinventing the wheel, we follow the design patterns that are well-proven, tested by others, follow as safe guidelines.

What are Software Design principles?
Software design principles represent a set of guidelines that helps us to avoid having a bad design. The design principles are associated to Robert Martin who gathered them in "Agile Software Development: Principles, Patterns, and Practices". According to Robert Martin there are 3 important characteristics of a bad design that should be avoided:

Rigidity - It is hard to change because every change affects too many other parts of the system.
Fragility - When you make a change, unexpected parts of the system break.
Immobility - It is hard to reuse in another application because it cannot be disentangled from the current application.

Design principles are important when going with Agile development where requirements keep changing, dependency on different modules, lack of technical expertise, etc.

This happens in our day to day life and all of us might have faced above mentioned problems. 

How to handle these problems

We have to stick to Design principles in all the phases of SDLC (software development life cycle), we have to follow the software principles but need to take a special consideration during analysis, design and coding phases. When requirement changes, we need to deliberately analyze the code and make sure basic software principle shouldn't be violated.

Design principles followed by many :
The Single Responsibilities Principle (SRP)
Open and Close Principle (OCP)
Liskov Substitution Principle (LSP)
Interface Segregation Principles (ISP)
Dependency Inversion Principle (DIP)
[Above 5 principle are SOLID]
DRY (Don't repeat yourself)
YAGNI (You aren't gonna need it)

Note: If you see any corrections or have suggestions please do let me know.

Database Server administration concepts for Architects/ Tech Leads/ Database developers

Most of us must have heard of Database replication/ mirroring but few of us would have tried to explore the details. This article explains about these concepts which would give you an overall understanding of Database server concepts which are used for large applications.


What is Database Mirroring?

Database mirroring is a primarily software solution for increasing database availability.
It maintains two copies of a single database that must reside on different server instances of SQL Server Database Engine.

What is Database Replication?

It is a set of technology for copying and distributing data and database objects from one database to another and then synchronizing between databases to maintain consistency. Using replication, you can distribute data to different locations and to remote or mobile users over local and wide area networks, dial-up connections, wireless connections, and the Internet.

Most of the development community think that above two mentioned questions are enough for database backups/ availability. There is yet another way which is most popular among the Database administrator's which is "Log Shipping".

So what is Log shipping?

It is sending transaction log backups from one database (Primary database) to another (Secondary database) on another server. An optional third server known as Monitor server records the history and status of the backups and restore operations. 

Server limitations for above mentioned techniques

Log Shipping --> It can be configured as One to Many. i.e one primary server and many                                secondary servers. 

                        Or
                        Secondary server can contain multiple Primary databases that are log                               shipped from multiple servers.
Mirroring      --> It's one to one. i.e., One principal server to one mirror server.
Replication  -->

• Central publisher/distributor, multiple subscribers.
• Central Distributor, multiple publishers, multiple subscribers.
• Central Distributer, multiple publishers, single subscriber.
• Mixed Topology.

Backup/ Restoration

Log Shipping -->This can be done manually or through Log Shipping options.
Mirroring      --> User take backup & restore manually.
Replication  --> User creates an empty database with the same name.

Note: If you see any corrections or have suggestions please do let me know.

Repository Pattern

Repository commonly refers to a storage location, often for safety or preservation.

Objectives of using repository

Use the Repository pattern to achieve one or more of the following objectives:

• You want to maximize the amount of code that can be tested with automation and to isolate the data layer to support unit testing.
• You access the data source from many locations and want to apply centrally managed, consistent access rules and logic.
• You want to implement and centralize a caching strategy for the data source.
• You want to improve the code's maintainability and readability by separating business logic from data or service access logic.
• You want to use business entities that are strongly typed so that you can identify problems at compile time instead of at run time.
• You want to associate a behavior with the related data. For example, you want to calculate fields or enforce complex relationships or business rules between the data elements within an entity.
• You want to apply a domain model to simplify complex business logic.

Solution

Use a repository to separate the logic that retrieves the data and maps it to the entity model from the business logic that acts on the model. The business logic should be agnostic to the type of data that comprises the data source layer. For example, the data source layer can be a database, a SharePoint list, or a Web service.

The repository mediates between the data source layer and the business layers of the application. It queries the data source for the data, maps the data from the data source to a business entity, and persists changes in the business entity to the data source. A repository separates the business logic from the interactions with the underlying data source or Web service. The separation between the data and business tiers has three benefits:

• It centralizes the data logic or Web service access logic.
• It provides a substitution point for the unit tests.
• It provides a flexible architecture that can be adapted as the overall design of the application evolves.

The repository pattern is an abstraction. Its purpose is to reduce complexity and make the rest of the code persistent ignorant. As a bonus it allows you to write unit tests instead of integration tests. The problem is that many developers fail to understand the patterns purpose and create repositories which leak persistence specific information up to the caller (typically by exposing IQueryable<T>, read through Mark Seemann’s blog  IQueryable is Tight Coupling ).

The Repository Pattern is a common construct to avoid duplication of data access logic throughout our application. This includes direct access to a database, ORM, WCF dataservices, xml files and so on.  We can easily query the repository for data objects, without having to know how to provide things like a connection string. The repository behaves like a freely available in-memory data collection to which we can add, delete and update objects.

The Repository pattern adds a separation layer between the data and domain layers of an application. It also makes the data access parts of an application better testable. Using repositories is not about being able to switch persistence technology (i.e. changing database or using a web service etc instead).

Creating Repository is not as tough as it sounds to be, once you implement this by your own, you’ll love it for sure.

Create Repository pattern

Step1 Very first thing to create a repository is to define an interface inside which we declare CRUD operations on entity.

using System;

using System.Linq;

using System.Linq.Expressions;

namespace Demo.Repositories

{

    public interface IRepository

    {

        void Insert(T entity);

        void Delete(T entity);

        List<T> SearchFor(Expression<Func<T, bool>> predicate);

        List<T> GetAll();

        T GetById(int id);

    }

}

Step2 Create a class inheriting the interface, pretty straight forward approach. 

using System;

using System.Data.Linq;

using System.Linq;

using System.Linq.Expressions;

namespace Demo.Repositories

{

    public class Repository<T> : IRepository<T> where T : class, IEntity

    {

        protected Table<T> DataTable;

        public Repository(DataContext dataContext)

        {

            DataTable = dataContext.GetTable<T>();

        }

        #region IRepository<T> Members

        public void Insert(T entity)

        {

            DataTable.InsertOnSubmit(entity);

        }

        public void Delete(T entity)

        {

            DataTable.DeleteOnSubmit(entity);

        }

        public IQueryable<T> SearchFor(Expression<Func<T, bool>> predicate)

        {

            return DataTable.Where(predicate);

        }

        public IQueryable<T> GetAll()

        {

            return DataTable;

        }

        #endregion

    }

}

Now the major part of our job is done.

Step 3 Use the repository in the application

using System;

using System.Collections.Generic;

using System.Linq;

using Demo.Repositories;

namespace LinqToSqlRepositoryConsole

{

    internal class Program

    {

        private static void Main()

        {

            using (var dataContext = new HotelsDataContext())

            {

                var hotelRepository = new Repository<Hotel>(dataContext);

                var cityRepository = new Repository<City>(dataContext);

                City city = cityRepository

                    .SearchFor(c => c.Name.StartsWith("Mum"))

                    .Single();

                IEnumerable<Hotel> orderedHotels = hotelRepository

                    .GetAll()

                    .Where(c => c.City.Equals(city))

                    .OrderBy(h => h.Name);

                Console.WriteLine("* Hotels in {0} *", city.Name);

                foreach (Hotel orderedHotel in orderedHotels)

                {

                    Console.WriteLine(orderedHotel.Name);

                }

                Console.ReadKey();

            }

        }

    }

}

Now that we have most of the things ready we can refine our functionality as per our choice, add more entity specific operations.