Introduction

Shore (Scalable Heterogeneous Object REpository) is a new persistent object system under development at the University of Wisconsin that represents a merger of object-oriented database (OODB) and file system technologies. While the past few years have seen significant progress in the OODB area, most applications (and application areas) have not chosen to leave file systems behind in favor of OODBs. We feel that more applications could benefit from OODB support, but are impeded by limitations in current technology.

1.

Many current OODBs are closed and restricted to one language (most often persistent C++ or Smalltalk), unlike both file systems and relational database systems. Large-scale applications often require multilingual data access.

2.

With most current OODBs, application programmers face an ``either/or'' decision--either they put their data in the OODB, in which case all of their existing file-based applications must be rewritten, or they leave their data in files.

3.

Most current OODBs provide a fairly ``heavy'' solution in the area of transaction management, dictating the adoption of serializability and up-to-the-last-transaction data recoverability.

4.

Most current OODBs have strongly client-server architectures, and are thus inappropriate for execution in peer-to-peer distributed systems or on the kinds of high-performance multicomputer hardware needed for certain large scale applications.

The goal of the Shore project is to provide a system that addresses these issues, thereby enabling ``holdout'' applications to finally move their data (incrementally) out of files and into a modern persistent object repository. We also expect many current OODB clients to find Shore to be an attractive alternative.

 

EXODUS

Many of us were involved in an earlier object-oriented database effort called EXODUS  [CDF+86]. Version 3.0 of EXODUS provides a client-server architecture with page-level locking, log-based recovery based on the ARIES algorithm [FZT+92], and support for multiple servers and distributed transactions. The EXODUS package includes the E programming language  [RCS93], a variant of C++ that supports convenient creation and manipulation of persistent data structures. The functionality, performance, robustness, and low cost (free!) of EXODUS has made it a popular piece of software. EXODUS and its associated toolkit have been used in several projects at Wisconsin and elsewhere. Over 350 different groups from over 30 countries have taken copies of it from our ftp site, it is used as the storage manager in the TI Open Object-Oriented Database System, it serves as the storage engine for at least one commercial product (MediaDB, a recently announced multi-media DBMS), and it has been shown to have commercially competitive performance on an OODBMS benchmark [CDN93]. Nonetheless, EXODUS suffers from several limitations shared by many current persistent object stores. An exploration of these limitations may help to explain the motivation for Shore. EXODUS storage objects are untyped arrays of bytes; correct interpretation of their contents is the responsibility of application programs. Although E allows instances of any C++ type to be stored in the database, no type information is stored. This ``compile-time'' approach to data types has several disadvantages including the following:

• It is too easy to access objects under the wrong type, because of programming or configuration errors such as version mismatch.
• Restricting type support to the compiler locks users into single-language solutions.
• Sharing data between applications is difficult.
• Lack of stored types prevents the DBMS from providing such facilities as support for heterogeneous hardware platforms, data browsers, or garbage collectors.

At the time we designed EXODUS, we felt there was too much variability in type systems to legislate a common solution. Since then, there has been a growing consensus on the level of type support that an OODBMS system should provide [Cat93]. A second limitation of the EXODUS storage manager (ESM) is its client-server architecture. Users have constructed database servers or object servers as EXODUS client processes, leading to the ``client-level server'' problem illustrated in Figure 1. Even a query-shipping (as opposed to data-shipping) SQL server would be difficult construct efficiently with the existing software base. In contrast, a more open architecture would have allowed clients to customize the ESM server process directly. The ESM process architecture also fails to support a clean mapping onto parallel processors such as the Intel Paragon or IBM SP/2. Although one could simply run an EXODUS server on each node with mass storage attached, support for distributed transactions is not sufficient; efficient parallelism also requires the availability of extensive server-to-server communication facilities.

  

Figure 1: The client-level server problem.

A third limitation of EXODUS is its lack of support for access control. As with other aspects of the system, our original thinking was that different clients might wish to implement very different protection models, and thus we provided no built-in protection support. Furthermore, EXODUS allows client processes to manipulate objects directly in cached copies of database pages, so an errant pointer can destroy not only client data but also metadata, rendering the entire database unusable. The original design of EXODUS envisioned client processes as being database systems and object servers (i.e., other trusted software layers). Shore aims to support environments in which a single storage volume may be shared by mutually mistrusting applications.

Finally, while EXODUS objects are similar to Unix files (they are untyped sequences of bytes), the interface for manipulating them is completely different. As a result, existing applications built around Unix files cannot easily use EXODUS.

The design of Shore strives to retain the good features of the EXODUS Storage Manager (such as transactions, performance, and robustness) while eliminating some of these limitations.

 

How Shore differs from EXODUS

Each object in Shore contains a pointer to a type object that defines its structure and interface. The Shore Data Language provides a single language-neutral notation for describing the types of all persistent data

Shore's process architecture is different from that of EXODUS in two key ways. First, Shore has a symmetric, peer-to-peer structure. Every participating processor runs a Shore server process regardless whether it has local disks. A client process interacts with Shore by communicating with the local Shore server (see Figure 2 ). The design is scalable; it can run on a single processor, a network of workstations, or a large parallel processor such as the Intel Paragon or IBM SP/2. (For more on parallel Shore, the reader is referred to [DNSV94]). Second, Shore supports the notion of a ``value-added'' server. The server code is modularly constructed to make it relatively simple for users to build application-specific servers without facing the ``client-level server'' problem. For example, the Paradise project [DLPY93] is already using the Shore storage manager to build a geographic information system.

  

Figure 2: The Shore process architecture.

Finally, Shore is intended to be much more of a complete system than ESM. In addition to a more flexible process structure and support for typed objects, Shore provides other services that end users should find attractive, including a name space and access-control model similar to Unix, a Unix-compatible interface for legacy software tools, openness in the area of language bindings, and traditional database services such as associative data access, indexing, and clustering.

The remainder of this document is organized as follows. Section 2 provides an overview of the services provided by Shore, including both its file system and database features. The Shore process architecture is described in Section 3. Section 4 describes the state of the system's various features as of the time of this Shore release. Finally, we conclude in Section 5. The software described here is not simply ``paperware.''