Business Laundry: Separate Your Layers

Normal vs. Best Practice

As long as software has existed, it has been normal for a particular tool to perform all functionality involved in the capturing, storage, processing and display of data for a particular purpose. Whether it is schedule, cost or any other data, users accessed one tool from start to finish to interact with the data. While this has been “normal practice” it is not “best practice”.

In most cases, best practice involves the separation of the capture, storage, processing and display of data such that you are using distinct processes, tools and databases or servers to handle each part of the solution. While these tools work with each other to create one cohesive workflow, they are indeed separate elements such that each one is managed independently. Among other advantages discussed below, these elements can serve as building blocks creating an endless number of workflows, instead of serving single purposes.

What Layers Make Up Your Business Laundry?

To understand why you should separate your data layers, you must first understand what you are separating. Generally, data solutions involve business logic, presentation logic, and more recently, automation processes.

The business logic contains the capture of data as Extract Transform and Load (ETL) Jobs move data from one or more data sources into a centralized location. It also includes data storage which is the centralized location containing relevant data. This can be a standalone Data Warehouse or even a set of simple tables in any accessible database.

The presentation logic is the display of the data through reports and dashboards. As this becomes a separate layer, these visuals can contain data from multiple source systems.

The automation processes range from the simple to the complex. In some cases, it is the execution of an ETL job at a particular time. In other cases, it is the generation, distribution and notification of reports or dashboards.

Why Do This?

• With few exceptions the separation of these layers is advantageous:
• You can use the best tools for data capture, data storage, data presentation and automation. Accordingly, each layer excels at what it does and offers a lot of flexibility and capability.
• Systems/data are not isolated so reports/dashboards can have data from many sources on them.
• Users may use source systems as soon as data capture completes without compromising data processing or presentation.
• Single snapshots of data can be preserved and used for an unlimited amount of time while data in the source system is being changed.
• Data storage is flexible and customizable allowing for preservation of historical snapshots and data comparison/trending.
• The data storage layer represents the single truth and can be used for reliable reporting.
• The company can replace one layer without disruption to the others; if a source system changes, end users experience little or no change.
• It is typically faster to process data (assuming you can avoid using APIs).
• It often reduces licensing costs as users do not need to access source systems.
• I.T. can manage the business logic and business users can manage the presentation logic.

There are only a few potential drawbacks when deploying a solution with separated layers:

• Some source systems put restrictions on their databases such that processes may not be able to access the data.
• If the use of APIs is required, data processing time can be too slow (It is usually faster to access the back-end data directly).
• Some values displayed in source systems are computed in memory and not stored in a database. These values must be calculated and held in the data storage layer or calculated during data presentation.

Conclusion

The benefits of separating system layers far outweigh the disadvantages, just as sorting your laundry does. Unless one of the “cons” presents an unsurmountable challenge, the layer separation is recommended to keep data storage, processing, and presentation flexible and effective.

Once you decide to separate your application layers, you must take care in choosing the tools, and services, to deploy such that they allow you to benefit from all of the advantages discussed above. This is where Knowledge Relay steps in.