DTS/S1: Benefits

Unification of technology

Because the DTS/S1 microcontainer executes as part of your application, it needs to conform to only one concrete standard - the Java Language Specification. DTS/S1 fully supports generic type safety that is available in Java versions 5 and newer. Additionally, DTS/S1 also conforms to the JTA (Java Transaction API) standard that allows it to be embedded within a JavaEE application server. This immediately removes the reliance on other standards, such as SQL, OQL, JDBC and so on.

From a purely technical perspective, you don't need data abstraction to instantiate objects, manipulate and dereference pointers, and why would you possibly want that overhead when committing data to stable storage. There is no middleware between your application and your storage provider. And because DTS/S1 is a unified platform that hosts both behaviour and data, your application benefits from the same level of parallelism and reliability that the clustered storage system does.

From an economic perspective, you no longer have to specify the words JDBC, Hibernate, TopLink, SQL, and so on when drafting out your next visit to the head hunter. You no longer need to hire database administrators, your developers need not maintain SQL code. You only pay for one storage solution which scales naturally with the rest of your application.

To fully appreciate the benefits that Pitch Black can bring to an enterprise, one needs to completely depart from the traditional storage models which actively encourage the separation of application logic and the respective business objects from the data access objects and the storage subsystem, and assume the presence of a data abstraction layer spanning a bridge between the two. There is absolutely no compelling reason to do so. Instead, Pitch Black encourages enterprise architects to merge business logic with persistence. Persistence in Pitch Black is merely an enhancement upon the legacy business objects that already form an integral part of an enterprise application.

Performance

Per unit cost of hardware, DTS/S1 is the fastest transactional storage facility in existence. Where peer products require million-dollar computer hardware to achieve speeds in the order of tens of thousands of transactions per second (TPS), Pitch Black can achieve similar figures on hardware that would take up one unit of rack space. Given clusters of high-end server hardware, coupled with clustered network attached storage (NAS) devices are employed, speeds in excess of 100,000 TPS are achievable.

Speed is not just the raw transaction throughput. One aspect of transaction processing that is often overlooked is the time it takes to fail-over from an incapacitated host, and the time it takes to recover the uncommitted data outstanding for that host. When running a Pitch Black cluster, both of these figures are zero.

Another often overlooked aspect of storage systems is load-insensitive response and turnaround times. To increase overall throughput, most products employ a strategy called coalesced writes. This technique employs intermediate buffers and intentionally delays disk writes so that they can be processed in chunks. This negatively impacts response times. Aside from poor response times, this scheme offers poor performance when the system is subject to a light load, where writes are delayed unnecessarily. This phenomenon is not observed with Pitch Black as it does not rely on coalesced writes or any other form of asynchronous disk I/O.

The next aspect is development time and the overall time-to-market. With a DTS-based solution these times are significantly reduced. This is because there is a single technology driving both business logic and data. Business objects can be rapidly enhanced to become DTS storage objects. Queries that would otherwise have been written in proprietary languages, are reduced to inspecting plain Java objects through traditional Java language syntax and semantics.

Any real-time system is not only concerned with raw throughput, but also predictability and responsiveness: timing, as they say, is everything. Running on a conventional Java platform, DTS/S1 can already achieve sub-millisecond transaction turn-around times while void of ramp-up effects. And because time-critical business logic can reside locally on the DTS nodes, there is no network and remote invocation delays inflating the turn-around times. Additionally, DTS/S1 can be deployed on Java VMs compliant with RTSJ (Real-time specification for Java) to further improve predictability.

Organic growth

When it comes to storage and processing power, the most cost-effective scaling is always horizontal. A system capable of processing 2 billion instructions per second is necessarily more expensive than two systems half of that capacity. Similarly, two 1 TB hard disk drives are cheaper than one 2 TB disk.

Every clustered solution will boast about its scalability. In a best-case scenario, nodes in traditional clustered database implementations can simultaneously access identical blocks of data, and simultaneously manipulate non-identical data blocks in parallel. This is sometimes referred to as "delightful" data. However, because of the strict ordering of all changes, there is always an irrefutable point of contention between nodes when the journal is appended with entries. Even if we can assume that all nodes are operating on non-overlapping data sets, there is still the requirement that a universally atomic, monotonically increasing source of serial transaction identifiers exists to impose order on the journal, even if the latter is partitioned to avoid concurrent modifications. This implies that traditional clustered systems cannot scale linearly. The addition of a node into the cluster will improve the system's throughput only up to a point where the contention levels are low enough to make this feasible. The decoupled transaction model used by Pitch Black imposes ordering only within individual transaction strands, while the relative ordering of the strands is irrelevant. This concept, coupled with backward-chaining revision control structures allow any Pitch Black node to rely solely on itself to arbitrate its own transaction sequence, thereby eliminating all points of contention for processing non-overlapping data.

We have previously mentioned the ability of the architecture to scale independently in two dimensions, that is, to provide for an increased computational load and/or an increased storage throughput. Below is a simple illustration of how this is achieved in practice:

With the benefits of horizontal scalability and no single point of failure (NSPF) architectures, invariably come the drawbacks of increased maintenance requirements. Often, the scope of NSPF is intentionally narrowed to prophylactical and post-failure scenarios. This, we believe is naive, in that the downtime of an average system is dominated by scheduled maintenance. Most database vendors will handle this, providing that the maintenance does involve reconfiguring the deployment layout, e.g. adding more compute nodes. Market heavyweights such as IBM's DB2 will require the reconfiguring of the data stripes to accommodate a larger scale deployment. Pitch Black is true NSPF in that it requires zero configuration to add, replace or remove individual cohorts. Zero maintenance downtime makes for a theoretical 100% uptime.

Small-scale integration

Some see scalability as an ever-increasing continuum of expansion. The true definition also encompasses the reverse path, that is, either down-scaling of a large data store, or running very small data stores with minimal storage, memory and processing footprints to begin with. Ideally, for a true unified storage solution, small-scale storage standards (and even products) should not differ from their medium-to-large scale counterparts. This is precisely what is achieved by Pitch Black. The pluggable interconnect fabric (PIF) enables Obsidian Dynamics to supply custom-built PIF varieties suitable for a legacy storage platform that might not rely on a POSIX file system. This is not necessarily applicable to a customer's existing storage area networks, but rather for small-scale embeddable software that may execute on PDAs, smart phones, hand-held warehouse trackers, home entertainment systems and so on. With Pitch Black, building a persistent, transactional application for a hand-held is no different from an enterprise application with extreme transactional loads and requirements for absolute reliability.

Naturally, DTS/S1 is available in a micro configuration, which is non-clustered and is ideally applicable to serverless desktop and mobile applications that have internal storage requirements. Candidates include video games, CAD/modelling applications, as well as mobile Java MIDP applications, and applications for Palm PDAs, smartphones, PocketPCs, Simbion, and the Apple iPhone.

Tooling

DTS Night Vision workbench is the optional instrumentation component library that simplifies the remote administration of a DTS/S1 cluster. Night Vision is also a Swing-based library that can be either embedded into an existing set of management portals, or be used to rapidly develop portals for administering a system utilising DTS as its storage backend. Night Vision can be used to define how transactions are audited and the bindings between storage objects and management portals through the use of Java annotations and pluggable business rules. Complete with customer support functionality, Night Vision is a complete framework for building client-side back office applications on top of DTS. Get more information on Night Vision here.

Sustainability

The days of relational databases are over - there is no question about it. The only question is what the replacement will have to offer. Relational DBs are readily serviceable and offer an abundance of tools. There is vast infrastructure in place to support an RDBMS. A new age solution would have to offer not only this, but much more.

As the world’s population is expanding at a great rate, and the spread of electronic communication is growing faster yet, organisations all around the world are experiencing a higher load on their IT infrastructure. Typical rack-mount server hardware residing in the back offices of most SMEs running relational DBMS products and capable of a few transactions per second are already struggling under load, forcing organisations to move to larger and more powerful computer hardware. These are not only more expensive when purchased outright, but take up more space, consume more power and require larger heat dissipation facilities to operate. While it can be argued that organisations with higher IT loads experience higher transactional turnover, this does not necessarily lead to increased profits that would warrant the cost of the additional hardware. This is because technological evolution, economies of scale and the competitive nature of the private sector actually forces a reduction in per-unit costs, and eventually a new player enters the market that can do things faster and cheaper than their competitors, forcing others to lower their profit margins. A database needs to evolve in order to survive. It needs to be made more efficient, cost effective, more compact physically, and less detrimental to the environment. In a competitive IT market, efficiency is bliss, and technology that is orders of magnitude faster and more efficient than its analogues puts the adopting organisation in a pole position for the race that only one can win.

Summary of benefits