Shore - A High-Performance, Scalable, Persistent Object Repository

Document Contents: See Also:

Objective:

The objective of the Shore project is to design, implement, and evaluate a persistent object system that will serve the needs of a wide variety of target applications including hardware and software CAD systems, persistent programming languages, geographic information systems, satellite data repositories, and multi-media applications. Shore expands on the basic capabilities of the widely-used EXODUS Storage Manager (developed at Wisconsin, funded by ARPA ) in a number of ways including support for typed objects, multiple programming languages, a ``Unix-like'' hierarchical name space for named objects, and a Unix-compatible interface to objects with a ``text'' field. This interface is intended to ease the transition of applications from the Unix file system environment to Shore as existing Unix tools such as vi, emacs, or cc will be able to store their data in Shore objects without modification.

Status:

Support for Shore project had ended in 1997. Releases made after that time were unsupported. That includes the "2.0" release. As of December 2000, Shore has some support again. However, lots of reliability, performance, and correctness changes have been made since the unsupported era. Due to the great number of problems fixed from the older versions, the only supported versions are the new interim snapshots mentioned in the release section.

Overview:

Shore has a layered architecture that allows users to choose the level of support appropriate for a particular application. We briefly overview this architecture here. The paper Shoring Up Persistent Applications describes the design of Shore in much greater detail.

The Shore Storage Manager (SM) is a persistent object storage engine that supports creation of persistent files of records. Each record can be any size, with efficient storage of records from a few bytes to megabytes or larger. Records may be retrieved by object identifier or by scanning files. The SM provides full concurrency control and recovery (the so-called ACID properties) with two-phase locking and write-ahead logging. It also provides robust implementations of btrees and rtrees and record access through logical object identifiers. The SM is designed to be used as a library to create value-added servers tailored to specific applications.

The Shore Value-Added Server (SVAS) builds on the functionality of the SM to provide typed objects, a Unix-like directory namespace, access control, and a client-server architecture supporting object-level caching, transactional semantics, and security at the server boundary. The NFS value-added server fully implements the standard NFS (Network File Server) protocol, allowing legacy applications to access Shore objects as if they were Unix files.

The Shore Data Language (SDL), which is based on the Object Database Management Group (ODMG) ODL language, supports language-independent description of object-oriented data types. The SDL compiler compiles definitions into type objects stored in the database and C++ language stubs. The combination of the SDL compiler and an extensive run-time library allows programmers to write applications that manipulate objects through type-safe object references. The library takes care of fetching objects on demand to an LRU client-level object cache, flushing changes to the server on transaction commit, and swizzing and unswizzling references as necessary.

Shore serves as the basis of other database projects, including Paradise at the University of Wisconsin, PREDATOR at Cornell University, and Lamba-DB at the University of Texas at Arlington.

Paradise is a scalable, parallel geographic information system (GIS) that is capable of storing and manipulating massive data sets. By applying object-oriented and parallel database technologies to the problem of storing and manipulating geographic information, the Paradise project hopes to significantly advance the size and complexity of GIS data sets that can be successfully stored, browsed, and queried.

The PREDATOR object-relational database system from Cornell University is freely available for research and education purposes. PREDATOR is a multi-threaded client-server system built on Shore. It supports standard relational query-processing with SQL queries, extensibility with enhanced complex data types with user-defined methods and aggregates, and WWW access through a Java applet client.

Lamba-DB is an object-oriented DBMS supporting the ODMG ODL specs. It uses the Shore Storage Engine, Language Independent Library and the Shore Data Language to implement the database.

Release Information:

Interim Snapshot Series

Shore has made a lot of progress since the so-called 2.0 release. It has had a lot of performance, reliability, and correctness bugs and problems fixed. It also has support for modern compilers, such as gcc-2.95.2, and real support for newer Unix-like platforms, such as NetBSD and Linux. However, we don't have the time or funding to put together a "real" Shore release. To make this more recent of Shore available to people, we've started an Interim Snapshot series of Shore "releases". They are not real releases with version numbers and full blown documentation. But what they ARE is a real working version of Shore that is far more robust than the latest "released" versions. You can find this interim snapshot, along with a some text notes and build instructions on the ftp server.

NOTE that although an interim snapshot has not been made since February 2002, that does not mean that shore is not progressing. What it does mean is that I'm running out of time and am trying to finish putting together gcc-3.2 support for shore. I hope to make that available sometime in the 4th quarter of 2003, at this point I am just waiting to find a week or two of time to finish hashing issues out and releasing it. -- Bolo

Interim Snapshot 2 -- February 2002

Interim Snapshot 1 -- October 2001

Interim Snapshot 0 -- January 2001

Version 2.0 (Storage manager only)

A release of the storage manager portion (and layers that it uses) of Shore. The build scheme is entirely rewritten. Contains bug fixes, new internal documentation, and some minor additional storage manager functionality. Runs on Linux, Solaris/I86, Solaris/Sparc, and Windows NT 4.0.

Version 1.1.1

A minor bug-fix release relative to 1.1. The most important change is that it has been patched to work under Linux.

Version 1.1

A updated release, with numerous bug fixes and performance enhancements was announced August 9, 1997. Gzip'd tar files of the source, documentation and a binary release (sparc and pentium solaris 2.5), can be found at <ftp://ftp.cs.wisc.edu/shore/prior_releases/1.1/>.

Version 1.0

On August 6, 1996 we released Shore, version 1.0. Gzip'd tar files of the source, documentation and a binary release (sparc and pentium solaris 2.5), can be found at <ftp://ftp.cs.wisc.edu/shore/prior_releases/1.0/> . This release will be removed after release 1.1 is found to be stable.

Beta Release (0.9.3)

A second Beta-rlease of Shore (version 0.9.3) was released on September 18, 1995. It is now obsolete.

Beta Release (0.9)

On May 3, 1995 we had our first beta release.

Mailing Lists

There are two Shore mailing lists to which users of Shore may subscribe:

shore-users@cs.wisc.edu

This is a mailing list for users of (and those interested in) Shore. This list is managed by Majordomo software at the UW--Madison CS department.

Use this list to discuss Shore with other users, ask for advice, exchange tips, etc. The list is also used by the maintainers to to notify interested parties about new releases and other changes in the Shore ftp archive

The shore-users list is a public mailing list, meaning that anyone may subscribe to it.

shore-bugs@cs.wisc.edu

This is a mailing list to report bugs to the maintainers of shore. Others interested in bug reports can subscribe to this list as well. shore-bugs is also maintained by Majordomo.

Please use this list to report bugs and submit fixes to the maintainers.

Subscribing to the Shore mailing lists

To subscribe or to change your subscription, send a message to majordomo@cs.wisc.edu. The Subject line of the message is ignored. The body should contain one or more of the following commands.

help
asks Majordomo to send you a brief summary of available commands.
subscribe shore-users
asks that you be added to the shore-users mailing list.
unsubscribe shore-users
asks that you be removed from the shore-users mailing list.
subscribe shore-bugs
asks that you be added to the shore-bugs mailing list.
unsubscribe shore-bugs
asks that you be removed from the shore-bugs mailing list.

Archives

We keep an archive of past messages to the list. Send a message containing either to majordomo@cs.wisc.edu to get a list of the archive files available. The time period covered by each file should be obvious from the file name. To have a copy of a particular archive file mailed to you, send a message containing either to majordomo@cs.wisc.edu. For example, will get an archive of messages sent during April 1997.

Last Modified:

Thu Dec 14 17:50:18 CST 2000

Josef Burger bolo@cs.wisc.edu