The primary backbone of Content Services Platform tools is to manage and store unstructured data efficiently, typically documents, images, and other types of files.  Over the years, Enterprise Content Management (ECM)/Content Services Platform (CSP) tools have evolved in their usage of content storage technologies:

• Early years (1980s/1990s) – Microfiche, optical platters, jukeboxes, ad Computer Output to Laser Disk (COLD).
• Pre-Cloud – (late 1990s/2000s) Storage Area Network (SAN), Network Attached Storage (NAS), RAID arrays, Content Addressable Storage (e.g. EMC Centera, Hitachi).
• Cloud Era – (2010s and beyond) cloud object stores (e.g. Amazon S3, Azure Blob Storage, Google Cloud Storage).

While each era marked a major shift forward, each incremental improvement over the years improved performance, scalability, and reliability.  In this post, we will focus on how cloud object stores compare to more traditional storage mechanisms for content services platforms and how cloud object stores can disrupt traditional ECM architectures.

Cloud Object Stores Compared to Traditional Storage

Both traditional file storage tools and cloud object stores provide the same primary function – to store files (documents, images, video, etc.).  However, the two technologies are different in a number of important ways:

Structure

Traditional file storage tools such as Storage Area Networks (SAN) utilize a structured folder hierarchy to organize files (the familiar folder tree structure).  Object stores, however, utilize a flat structure with each file having a unique key.  If needed, folder structures can be simulated with key prefixes but should not be used for searching or directory listings.  Importantly, for content services, object stores provide a small amount of storage for metadata that can be customized.  Traditional file stores are limited to the operating system level metadata provided, such as file size, creation date and modification date.

Access Patterns

Traditional file stores are typically accessed with standard file and directory protocols, usually exposed on the corporate network with a SAN or NAS device.  Cloud object stores are well-suited for cloud-native architectures in that they are accessed via RESTful APIs over HTTPS, making them very easy to integrate within cloud-native applications.

For example, a modern JavaScript application will be much easier to integrate with Amazon S3 vs. a traditional file store.  In the S3 case, simple RESTful services can be utilized directly from the JavaScript application.  In the traditional file store case, a server-side component would be required to access the file and directory protocols of the underlying operating system.  This server-side component would likely need a RESTful API of its own to connect to the JavaScript application.

Scalability

Traditional file stores can certainly scale, but especially for on-prem implementations, scaling the storage capacity involves IT resources physically adding the necessary drives and allocating the memory as needed. Object stores, on the other hand, offer seamless scalability that easily handles billions of objects and petabytes of storage.  In fact, Amazon S3’s use guide states that there is no limit to the total size or number of objects stored in a bucket.  Single objects are limited to a max of 5GB.

Consider the comparison below.  To scale a traditional file store, physical storage must be added to the SAN or NAS device (both the primary and backup) and allocated to the correct drives or mounts.  However, for Amazon S3, there is nothing to do.  The storage simply scales with usage automatically.

Cost Structure

Traditional storage mechanisms force the end user to pay for storage capacity, whether the full capacity is used yet or not.  Most applications are configured with additional storage space for support as content is added  For example, say an application utilizes a NAS with 100GB of storage.  When the application nears the limit, the IT team will need to add space, but may recommend adding 400GB for a new limit of 500GB.  Until this space is required, the business is essentially paying for 300GB+ of storage to sit idle.

On the other hand, cloud object stores utilize a pay-as-you-go pricing model.  The business only pays for storage that is actually used.

Data Tiering

Both traditional file storage and object stores utilize data tiers, which create distinct classes or layers of storage with varying performance and cost characteristics. These tiers are typically defined by the speed of access and the frequency of use.

In traditional storage mechanisms, frequently accessed and critical data is housed in high-performance tiers, often utilizing Solid-state drives (SSDs) or faster storage mediums, while less accessed or archival data resides in slower, more cost-effective tiers. Data movement between these tiers is a deliberate process, often involving manual or automated migration based on predefined policies. This tiering strategy is grounded in the assumption that the characteristics of data, including access patterns and importance, remain relatively stable. In our experiences we have never seen a customer take the time to implement data tiering and movement on a traditional file storage architecture.

Object stores, conversely, present a more dynamic and flexible approach to tiering. The tiering in object stores is often seamless and more granular, allowing for cost optimization based on real-time data access patterns. Object stores frequently offer multiple storage classes, each designed for specific use cases. For instance, frequently accessed data might reside in a standard storage class, while infrequently accessed or archival data can be moved to a more cost-effective storage class. The transition between these classes is often automated, responding to usage patterns without manual intervention.

From a system architecture standpoint, traditional file stores rely on a tiering model that is more static and less responsive to real-time data dynamics. Data movement between tiers is a deliberate process and may involve more overhead. Perhaps more importantly, it is very rare to see an implementation utilizing traditional content storage tiers.  In contrast, object stores embrace a more fluid tiering strategy, automatically adapting to changing data access patterns. This shift can lead to more efficient resource utilization and cost savings, especially in cloud environments where the pay-as-you-go model aligns with dynamic content storage demands. Additionally, due to the fluid nature of object store tiering, the cost-saving nature is more likely to be taken advantage of in real-world content services applications.

The key differentiator lies in the agility and automation of tiering processes, making object stores well-suited for the unpredictable and evolving nature of modern data management.

Choosing a Content Services Storage Strategy

Putting it all together, take the below example for a customer comparing storage options for a content services application:

In choosing between the traditional and cloud object store options, the following points need to be considered:

• NAS/SAN
  • The business must purchase storage to exceed projected needs, both for primary and backup devices.
  • Storage capacity must be monitored and scaled appropriately by IT.
  • Any integrating applications must integrate utilizing file and directory protocols.  Most CSPs obfuscate the files and directory structure, forcing content access through proprietary APIs.
• Object Store
  • The business purchases only what is actually used.  The cloud provider (e.g. Amazon) handles backups and industry-leading availability.  See the Amazon S3 SLA as an example.
  • Scaling is automatic and virtually unlimited.
  • Done correctly, integrating applications can have secure access to content easily via RESTful APIs.

What’s Next?

While traditional file storage still has applicable use cases, especially for on-prem deployments, the technology is unlikely to make major leaps forward for modern content services systems.  Object stores, however, are more likely to change the landscape as we look to the future.  As far back as 2018, Technology Services Group (TSG) predicted that object stores would be the next disruptor in the ECM/Content Services space.

As 2023 comes to a close, organizations still cannot throw content in an Amazon S3 bucket and expect content services functionality to be available out of the box.  However, we at Docuvela agree with TSG’s assessment that object stores are the next big ECM/Content Services disruptor – perhaps with some tooling in place around the content store that negates the need for an “ECM” middleman.  We’d love to hear your thoughts below or contact us.  Stay tuned to the Docuvela blog as we explore this topic further.