com.caucho.db.block
Class BlockStore

java.lang.Object
  extended by com.caucho.db.block.BlockStore
Direct Known Subclasses:
Table

public class BlockStore
extends java.lang.Object

The store manages the block-based persistent store file. Each table will have its own store file, table.db. The store is block-based, where each block is 64k. Block allocation is tracked by a free block, block 0. Each block is represented as a two-byte value. The first byte is the allocation code: free, row, or used. The second byte is a fragment allocation mask. Since 64k stores 32k entries, the allocation block can handle a 2G database size. If the database is larger, another free block occurs at block 32k handling another 2G. The blocks are marked as free (00), row (01), used (10) or fragment(11). Row-blocks are table rows, so a table iterator will only look at the row blocks. Used blocks are for special blocks like the free list. Fragments are for blobs. Each store has a unique id in the database. The store id is merged with the block number in the store to create a unique block id. There are 64k allowed stores (and therefore 64k tables), leaving 64 - 16 = 48 bits for the blocks in a table, i.e. 2 ^ 48 blocks = 256T blocks. block index: the block number in the file. address: the address of a byte within the store, treating the file as a flat file. block id: the unique id of the block in the database.

Blobs and fragments

Fragments are stored in 8k chunks with a single byte prefix indicating its use.


Field Summary
protected  BlockManager _blockManager
           
protected  Database _database
           
static int ALLOC_DATA
           
static int ALLOC_FREE
           
static int ALLOC_INDEX
           
static int ALLOC_INODE_PTR
           
static int ALLOC_MASK
           
static int ALLOC_MINI_FRAG
           
static int ALLOC_ROW
           
static int BLOCK_BITS
           
static long BLOCK_INDEX_MASK
           
static long BLOCK_MASK
           
static long BLOCK_OFFSET_MASK
           
static int BLOCK_SIZE
           
static long DATA_START
           
static java.lang.String DATABASE_CORRUPT_EVENT
           
static int MINI_FRAG_ALLOC_OFFSET
           
static int MINI_FRAG_PER_BLOCK
           
static int MINI_FRAG_SIZE
           
static int STORE_CREATE_END
           
 
Constructor Summary
BlockStore(Database database, java.lang.String name, java.util.concurrent.locks.ReadWriteLock tableLock)
           
BlockStore(Database database, java.lang.String name, java.util.concurrent.locks.ReadWriteLock rowLock, Path path)
          Creates a new store.
BlockStore(Database database, java.lang.String name, java.util.concurrent.locks.ReadWriteLock rowLock, Path path, boolean isEnableMmap)
          Creates a new store.
 
Method Summary
 long addressToBlockId(long address)
          Converts from the block index to the unique block id.
 Block allocateBlock()
          Allocates a new block for a non-row.
 Block allocateIndexBlock()
          Allocates a new block for an index
 Block allocateIndirectBlock()
          Allocates a new block for a non-row.
 long allocateMiniFragment()
          Allocates a new miniFragment.
 Block allocateRow()
          Allocates a new block for a row.
protected  void assertStoreActive()
          Check that an allocated block is valid.
static long blockIdToAddress(long blockId)
          Converts from the block index to the unique block id.
static long blockIdToAddress(long blockId, int offset)
          Converts from the block index to the unique block id.
static long blockIdToIndex(long blockId)
          Converts from the block index to the address for database storage.
static long blockIndexToAddr(long blockIndex)
          Converts from the block index to the address for database storage.
 void close()
          Closes the store.
static java.lang.String codeToName(int code)
          Debug names for the allocation.
 void create()
          Creates the store.
static BlockStore create(Path path)
          Creates an independent store.
static BlockStore create(Path path, boolean isMmap)
          Creates an independent store.
static BlockStore createMmap(Path path)
          Creates an independent store.
static BlockStore createNoMmap(Path path)
          Creates an independent store.
 void deallocateBlock(long blockId)
          Frees a block.
 void deleteMiniFragment(long fragmentAddress)
          Deletes a miniFragment.
 void fatalCorrupted(java.lang.String msg)
           
 long firstBlock(long blockId, int type)
          Returns the first block id which contains a row.
 long firstRowBlock(long blockId)
          Returns the first block id which contains a row.
 void flush()
          Flush the store.
 int getAllocation(long blockIndex)
          Sets the allocation for a block.
 int getAllocationByAddress(long blockAddress)
          Sets the allocation for a block.
 byte[] getAllocationTable()
          Returns a copy of the allocation table.
 long getBlockCount()
          Returns the block count.
 BlockManager getBlockManager()
          Returns the block manager.
 long getFileSize()
          Returns the file size.
 int getId()
          Returns the store's id.
 java.lang.String getName()
          Returns the store's name.
protected  BlockReadWrite getReadWrite()
           
 java.util.concurrent.locks.Lock getWriteLock()
          Returns the table's lock.
 void init()
           
 boolean isClosed()
          True if destroyed.
 boolean isFlushDirtyBlocksOnCommit()
          If true, dirty blocks are written at commit time.
 boolean isIndexBlock(long blockAddress)
          Return true if the block is an index block.
 boolean isInodePtrBlock(long blockAddress)
          Return true if the block is an index block.
 boolean isRowBlock(long blockAddress)
          Return true if the block is a row block.
 Block loadBlock(long blockAddress)
          Returns the matching block.
 Block readBlock(long blockAddress)
          Returns the matching block.
 int readBlock(long blockAddress, int blockOffset, byte[] buffer, int offset, int length)
          Reads a block.
 int readBlock(long blockAddress, int blockOffset, char[] buffer, int offset, int length)
          Reads a block for a clob.
 void readBlock(long blockId, int blockOffset, java.io.OutputStream os, int length)
          Reads a block to an output stream.
 long readBlockLong(long blockAddress, int offset)
          Reads a long value from a block.
 void readBlockNoLock(long blockId, int blockOffset, java.io.OutputStream os, int length)
          Reads a block to an output stream.
static long readLong(byte[] buffer, int offset)
          Reads the long.
 int readMiniFragment(long fragmentAddress, int fragmentOffset, byte[] buffer, int offset, int length)
          Reads a fragment.
 int readMiniFragment(long fragmentAddress, int fragmentOffset, char[] buffer, int offset, int length)
          Reads a miniFragment for a clob.
 long readMiniFragmentLong(long fragmentAddress, int fragmentOffset)
          Reads a long value from a miniFragment.
 int readMiniFragmentNoLock(long fragmentAddress, int fragmentOffset, int length, java.io.OutputStream os)
          Reads a fragment.
 void remove()
           
 void saveAllocation()
          Sets the allocation for a block.
 void setEnableMmap(boolean isEnable)
           
 void setFlushDirtyBlocksOnCommit(boolean flushOnCommit)
          If true, dirty blocks are written at commit time.
 java.lang.String toString()
           
protected  void validateBlockId(long blockId)
          Check that an allocated block is valid.
 Block writeBlock(long blockAddress, int blockOffset, byte[] buffer, int offset, int length)
          Writes a block.
 Block writeBlock(long blockAddress, int blockOffset, char[] buffer, int offset, int charLength)
          Writes a character based block
 Block writeBlockLong(long blockAddress, int offset, long value)
          Writes a long value to a block
static void writeLong(byte[] buffer, int offset, long v)
          Writes the long.
 Block writeMiniFragment(long fragmentAddress, int fragmentOffset, byte[] buffer, int offset, int length)
          Writes a miniFragment.
 Block writeMiniFragment(long fragmentAddress, int fragmentOffset, char[] buffer, int offset, int length)
          Writes a character based
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

BLOCK_BITS

public static final int BLOCK_BITS
See Also:
Constant Field Values

BLOCK_SIZE

public static final int BLOCK_SIZE
See Also:
Constant Field Values

BLOCK_INDEX_MASK

public static final long BLOCK_INDEX_MASK
See Also:
Constant Field Values

BLOCK_MASK

public static final long BLOCK_MASK
See Also:
Constant Field Values

BLOCK_OFFSET_MASK

public static final long BLOCK_OFFSET_MASK
See Also:
Constant Field Values

ALLOC_FREE

public static final int ALLOC_FREE
See Also:
Constant Field Values

ALLOC_ROW

public static final int ALLOC_ROW
See Also:
Constant Field Values

ALLOC_DATA

public static final int ALLOC_DATA
See Also:
Constant Field Values

ALLOC_INODE_PTR

public static final int ALLOC_INODE_PTR
See Also:
Constant Field Values

ALLOC_INDEX

public static final int ALLOC_INDEX
See Also:
Constant Field Values

ALLOC_MINI_FRAG

public static final int ALLOC_MINI_FRAG
See Also:
Constant Field Values

ALLOC_MASK

public static final int ALLOC_MASK
See Also:
Constant Field Values

MINI_FRAG_SIZE

public static final int MINI_FRAG_SIZE
See Also:
Constant Field Values

MINI_FRAG_PER_BLOCK

public static final int MINI_FRAG_PER_BLOCK
See Also:
Constant Field Values

MINI_FRAG_ALLOC_OFFSET

public static final int MINI_FRAG_ALLOC_OFFSET
See Also:
Constant Field Values

DATA_START

public static final long DATA_START
See Also:
Constant Field Values

STORE_CREATE_END

public static final int STORE_CREATE_END
See Also:
Constant Field Values

DATABASE_CORRUPT_EVENT

public static final java.lang.String DATABASE_CORRUPT_EVENT
See Also:
Constant Field Values

_database

protected final Database _database

_blockManager

protected final BlockManager _blockManager
Constructor Detail

BlockStore

public BlockStore(Database database,
                  java.lang.String name,
                  java.util.concurrent.locks.ReadWriteLock tableLock)

BlockStore

public BlockStore(Database database,
                  java.lang.String name,
                  java.util.concurrent.locks.ReadWriteLock rowLock,
                  Path path)
Creates a new store.

Parameters:
database - the owning database.
name - the store name
lock - the table lock
path - the path to the files

BlockStore

public BlockStore(Database database,
                  java.lang.String name,
                  java.util.concurrent.locks.ReadWriteLock rowLock,
                  Path path,
                  boolean isEnableMmap)
Creates a new store.

Parameters:
database - the owning database.
name - the store name
lock - the table lock
path - the path to the files
Method Detail

create

public static BlockStore create(Path path)
                         throws java.io.IOException,
                                java.sql.SQLException
Creates an independent store.

Throws:
java.io.IOException
java.sql.SQLException

createNoMmap

public static BlockStore createNoMmap(Path path)
                               throws java.io.IOException,
                                      java.sql.SQLException
Creates an independent store.

Throws:
java.io.IOException
java.sql.SQLException

createMmap

public static BlockStore createMmap(Path path)
                             throws java.io.IOException,
                                    java.sql.SQLException
Creates an independent store.

Throws:
java.io.IOException
java.sql.SQLException

create

public static BlockStore create(Path path,
                                boolean isMmap)
                         throws java.io.IOException,
                                java.sql.SQLException
Creates an independent store.

Throws:
java.io.IOException
java.sql.SQLException

setEnableMmap

public void setEnableMmap(boolean isEnable)

setFlushDirtyBlocksOnCommit

public void setFlushDirtyBlocksOnCommit(boolean flushOnCommit)
If true, dirty blocks are written at commit time.


isFlushDirtyBlocksOnCommit

public boolean isFlushDirtyBlocksOnCommit()
If true, dirty blocks are written at commit time.


getName

public java.lang.String getName()
Returns the store's name.


getId

public int getId()
Returns the store's id.


getWriteLock

public java.util.concurrent.locks.Lock getWriteLock()
Returns the table's lock.


getBlockManager

public BlockManager getBlockManager()
Returns the block manager.


getReadWrite

protected BlockReadWrite getReadWrite()

getFileSize

public long getFileSize()
Returns the file size.


getBlockCount

public long getBlockCount()
Returns the block count.


blockIndexToAddr

public static long blockIndexToAddr(long blockIndex)
Converts from the block index to the address for database storage.


blockIdToIndex

public static long blockIdToIndex(long blockId)
Converts from the block index to the address for database storage.


addressToBlockId

public final long addressToBlockId(long address)
Converts from the block index to the unique block id.


blockIdToAddress

public static long blockIdToAddress(long blockId)
Converts from the block index to the unique block id.


blockIdToAddress

public static long blockIdToAddress(long blockId,
                                    int offset)
Converts from the block index to the unique block id.


create

public void create()
            throws java.io.IOException,
                   java.sql.SQLException
Creates the store.

Throws:
java.io.IOException
java.sql.SQLException

init

public void init()
          throws java.io.IOException
Throws:
java.io.IOException

remove

public void remove()
            throws java.sql.SQLException
Throws:
java.sql.SQLException

firstRowBlock

public long firstRowBlock(long blockId)
                   throws java.io.IOException
Returns the first block id which contains a row.

Returns:
the block id of the first row block
Throws:
java.io.IOException

firstBlock

public long firstBlock(long blockId,
                       int type)
                throws java.io.IOException
Returns the first block id which contains a row.

Returns:
the block id of the first row block
Throws:
java.io.IOException

readBlock

public final Block readBlock(long blockAddress)
                      throws java.io.IOException
Returns the matching block.

Throws:
java.io.IOException

loadBlock

public final Block loadBlock(long blockAddress)
                      throws java.io.IOException
Returns the matching block.

Throws:
java.io.IOException

allocateRow

public Block allocateRow()
                  throws java.io.IOException
Allocates a new block for a row.

Returns:
the block id of the allocated block.
Throws:
java.io.IOException

isRowBlock

public boolean isRowBlock(long blockAddress)
Return true if the block is a row block.


allocateBlock

public Block allocateBlock()
                    throws java.io.IOException
Allocates a new block for a non-row.

Returns:
the block id of the allocated block.
Throws:
java.io.IOException

allocateIndirectBlock

public Block allocateIndirectBlock()
                            throws java.io.IOException
Allocates a new block for a non-row.

Returns:
the block id of the allocated block.
Throws:
java.io.IOException

allocateIndexBlock

public Block allocateIndexBlock()
                         throws java.io.IOException
Allocates a new block for an index

Returns:
the block id of the allocated block.
Throws:
java.io.IOException

isIndexBlock

public boolean isIndexBlock(long blockAddress)
Return true if the block is an index block.


isInodePtrBlock

public boolean isInodePtrBlock(long blockAddress)
Return true if the block is an index block.


validateBlockId

protected void validateBlockId(long blockId)
                        throws java.lang.IllegalArgumentException,
                               java.lang.IllegalStateException
Check that an allocated block is valid.

Throws:
java.lang.IllegalArgumentException
java.lang.IllegalStateException

assertStoreActive

protected void assertStoreActive()
                          throws java.lang.IllegalStateException
Check that an allocated block is valid.

Throws:
java.lang.IllegalStateException

deallocateBlock

public void deallocateBlock(long blockId)
                     throws java.io.IOException
Frees a block.

Throws:
java.io.IOException

getAllocationByAddress

public final int getAllocationByAddress(long blockAddress)
Sets the allocation for a block.


getAllocation

public final int getAllocation(long blockIndex)
Sets the allocation for a block.


saveAllocation

public void saveAllocation()
                    throws java.io.IOException
Sets the allocation for a block.

Throws:
java.io.IOException

readBlock

public void readBlock(long blockId,
                      int blockOffset,
                      java.io.OutputStream os,
                      int length)
               throws java.io.IOException
Reads a block to an output stream.

Parameters:
blockAddress - the address of the block
blockOffset - the offset inside the block to start reading
os - the result output stream
length - the number of bytes to read
Throws:
java.io.IOException

readBlockNoLock

public void readBlockNoLock(long blockId,
                            int blockOffset,
                            java.io.OutputStream os,
                            int length)
                     throws java.io.IOException
Reads a block to an output stream.

Parameters:
blockAddress - the address of the block
blockOffset - the offset inside the block to start reading
os - the result output stream
length - the number of bytes to read
Throws:
java.io.IOException

readBlock

public int readBlock(long blockAddress,
                     int blockOffset,
                     byte[] buffer,
                     int offset,
                     int length)
              throws java.io.IOException
Reads a block.

Parameters:
blockAddress - the address of the block
blockOffset - the offset inside the block to start reading
buffer - the result buffer
offset - offset into the result buffer
length - the number of bytes to read
Returns:
the number of bytes read
Throws:
java.io.IOException

readBlock

public int readBlock(long blockAddress,
                     int blockOffset,
                     char[] buffer,
                     int offset,
                     int length)
              throws java.io.IOException
Reads a block for a clob.

Parameters:
blockAddress - the address of the block
blockOffset - the offset inside the block to start reading
buffer - the result buffer
offset - offset into the result buffer
length - the length of the block in characters
Returns:
the number of characters read
Throws:
java.io.IOException

readBlockLong

public long readBlockLong(long blockAddress,
                          int offset)
                   throws java.io.IOException
Reads a long value from a block.

Returns:
the long value
Throws:
java.io.IOException

writeBlock

public Block writeBlock(long blockAddress,
                        int blockOffset,
                        byte[] buffer,
                        int offset,
                        int length)
                 throws java.io.IOException
Writes a block.

Parameters:
blockAddress - the block to write
blockOffset - the offset into the block
buffer - the write buffer
offset - offset into the write buffer
length - the number of bytes to write
Returns:
the fragment id
Throws:
java.io.IOException

writeBlock

public Block writeBlock(long blockAddress,
                        int blockOffset,
                        char[] buffer,
                        int offset,
                        int charLength)
                 throws java.io.IOException
Writes a character based block

Parameters:
blockAddress - the fragment to write
blockOffset - the offset into the fragment
buffer - the write buffer
offset - offset into the write buffer
length - the number of bytes to write
Throws:
java.io.IOException

writeBlockLong

public Block writeBlockLong(long blockAddress,
                            int offset,
                            long value)
                     throws java.io.IOException
Writes a long value to a block

Returns:
the long value
Throws:
java.io.IOException

readMiniFragment

public int readMiniFragment(long fragmentAddress,
                            int fragmentOffset,
                            byte[] buffer,
                            int offset,
                            int length)
                     throws java.io.IOException
Reads a fragment.

Parameters:
fragmentAddress - the address of the fragment
fragmentOffset - the offset inside the fragment to start reading
buffer - the result buffer
offset - offset into the result buffer
length - the number of bytes to read
Returns:
the number of bytes read
Throws:
java.io.IOException

readMiniFragmentNoLock

public int readMiniFragmentNoLock(long fragmentAddress,
                                  int fragmentOffset,
                                  int length,
                                  java.io.OutputStream os)
                           throws java.io.IOException
Reads a fragment.

Parameters:
fragmentAddress - the address of the fragment
fragmentOffset - the offset inside the fragment to start reading
buffer - the result buffer
offset - offset into the result buffer
length - the number of bytes to read
Returns:
the number of bytes read
Throws:
java.io.IOException

readMiniFragment

public int readMiniFragment(long fragmentAddress,
                            int fragmentOffset,
                            char[] buffer,
                            int offset,
                            int length)
                     throws java.io.IOException
Reads a miniFragment for a clob.

Parameters:
fragmentAddress - the address of the fragment
fragmentOffset - the offset inside the fragment to start reading
buffer - the result buffer
offset - offset into the result buffer
length - the length of the fragment in characters
Returns:
the number of characters read
Throws:
java.io.IOException

readMiniFragmentLong

public long readMiniFragmentLong(long fragmentAddress,
                                 int fragmentOffset)
                          throws java.io.IOException
Reads a long value from a miniFragment.

Returns:
the long value
Throws:
java.io.IOException

allocateMiniFragment

public long allocateMiniFragment()
                          throws java.io.IOException
Allocates a new miniFragment.

Returns:
the fragment address
Throws:
java.io.IOException

deleteMiniFragment

public void deleteMiniFragment(long fragmentAddress)
                        throws java.io.IOException
Deletes a miniFragment.

Throws:
java.io.IOException

writeMiniFragment

public Block writeMiniFragment(long fragmentAddress,
                               int fragmentOffset,
                               byte[] buffer,
                               int offset,
                               int length)
                        throws java.io.IOException
Writes a miniFragment.

Parameters:
fragmentAddress - the fragment to write
fragmentOffset - the offset into the fragment
buffer - the write buffer
offset - offset into the write buffer
length - the number of bytes to write
Returns:
the fragment id
Throws:
java.io.IOException

writeMiniFragment

public Block writeMiniFragment(long fragmentAddress,
                               int fragmentOffset,
                               char[] buffer,
                               int offset,
                               int length)
                        throws java.io.IOException
Writes a character based

Parameters:
miniFragmentAddress - the fragment to write
fragmentOffset - the offset into the fragment
buffer - the write buffer
offset - offset into the write buffer
length - the number of bytes to write
Throws:
java.io.IOException

flush

public void flush()
Flush the store.


fatalCorrupted

public void fatalCorrupted(java.lang.String msg)

isClosed

public boolean isClosed()
True if destroyed.


close

public void close()
Closes the store.


getAllocationTable

public byte[] getAllocationTable()
Returns a copy of the allocation table.


readLong

public static long readLong(byte[] buffer,
                            int offset)
Reads the long.


writeLong

public static void writeLong(byte[] buffer,
                             int offset,
                             long v)
Writes the long.


codeToName

public static java.lang.String codeToName(int code)
Debug names for the allocation.


toString

public java.lang.String toString()
Overrides:
toString in class java.lang.Object