com.caucho.db.block

Class BlockStore

java.lang.Object

com.caucho.db.block.BlockStore

Direct Known Subclasses:

Table

public class BlockStore
extends java.lang.Object

The store manages the block-based persistent store file. Each table will have its own store file, table.db. The store is block-based, where each block is 64k. Block allocation is tracked by a free block, block 0. Each block is represented as a two-byte value. The first byte is the allocation code: free, row, or used. The second byte is a fragment allocation mask. Since 64k stores 32k entries, the allocation block can handle a 2G database size. If the database is larger, another free block occurs at block 32k handling another 2G. The blocks are marked as free (00), row (01), used (10) or fragment(11). Row-blocks are table rows, so a table iterator will only look at the row blocks. Used blocks are for special blocks like the free list. Fragments are for blobs. Each store has a unique id in the database. The store id is merged with the block number in the store to create a unique block id. There are 64k allowed stores (and therefore 64k tables), leaving 64 - 16 = 48 bits for the blocks in a table, i.e. 2 ^ 48 blocks = 256T blocks. block index: the block number in the file. address: the address of a byte within the store, treating the file as a flat file. block id: the unique id of the block in the database.

Blobs and fragments

Fragments are stored in 8k chunks with a single byte prefix indicating its use.

Field Summary

Fields

Modifier and Type	Field and Description
protected BlockManager	_blockManager
protected Database	_database
static int	ALLOC_DATA
static int	ALLOC_FREE
static int	ALLOC_INDEX
static int	ALLOC_INODE_PTR
static int	ALLOC_MASK
static int	ALLOC_MINI_FRAG
static int	ALLOC_ROW
static int	BLOCK_BITS
static long	BLOCK_INDEX_MASK
static long	BLOCK_MASK
static long	BLOCK_OFFSET_MASK
static int	BLOCK_SIZE
static java.lang.String	DATABASE_CORRUPT_EVENT
static long	METADATA_START
static int	MINI_FRAG_ALLOC_OFFSET
static int	MINI_FRAG_PER_BLOCK
static int	MINI_FRAG_SIZE
static int	STORE_CREATE_END

Constructor Summary

Constructors

Constructor and Description
BlockStore(Database database, java.lang.String name, java.util.concurrent.locks.ReadWriteLock tableLock)
BlockStore(Database database, java.lang.String name, java.util.concurrent.locks.ReadWriteLock rowLock, Path path) Creates a new store.
BlockStore(Database database, java.lang.String name, java.util.concurrent.locks.ReadWriteLock rowLock, Path path, boolean isEnableMmap) Creates a new store.

Method Summary

Modifier and Type	Method and Description
long	addressToBlockId(long address) Converts from the block index to the unique block id.
Block	allocateBlock() Allocates a new block for a non-row.
Block	allocateIndexBlock() Allocates a new block for an index
Block	allocateIndirectBlock() Allocates a new block for a non-row.
long	allocateMiniFragment() Allocates a new miniFragment.
Block	allocateRow() Allocates a new block for a row.
protected void	assertStoreActive() Check that an allocated block is valid.
static long	blockIdToAddress(long blockId) Converts from the block index to the unique block id.
static long	blockIdToAddress(long blockId, int offset) Converts from the block index to the unique block id.
static long	blockIdToIndex(long blockId) Converts from the block index to the address for database storage.
static long	blockIndexToAddr(long blockIndex) Converts from the block index to the address for database storage.
void	close() Closes the store.
static java.lang.String	codeToName(int code) Debug names for the allocation.
void	create() Creates the store.
static BlockStore	create(Path path) Creates an independent store.
static BlockStore	create(Path path, boolean isMmap) Creates an independent store.
static BlockStore	createMmap(Path path) Creates an independent store.
static BlockStore	createNoMmap(Path path) Creates an independent store.
void	deallocateBlock(long blockId) Frees a block.
void	deleteMiniFragment(long fragmentAddress) Deletes a miniFragment.
void	fatalCorrupted(java.lang.String msg)
long	firstBlock(long blockId, int type) Returns the first block id which contains a row.
long	firstRowBlock(long blockId) Returns the first block id which contains a row.
void	flush() Flush the store.
boolean	fsync() Closes the store.
int	getAllocation(long blockIndex) Sets the allocation for a block.
int	getAllocationByAddress(long blockAddress) Sets the allocation for a block.
byte[]	getAllocationTable() Returns a copy of the allocation table.
long	getBlockCount() Returns the block count.
BlockManager	getBlockManager() Returns the block manager.
long	getFileSize() Returns the file size.
int	getId() Returns the store's id.
RandomAccessStream	getMmap()
java.lang.String	getName() Returns the store's name.
Path	getPath() Returns the store's path.
protected BlockReadWrite	getReadWrite()
java.util.concurrent.locks.Lock	getTableLock() Returns the table's lock.
java.util.concurrent.locks.Lock	getWriteLock() Returns the table's lock.
void	init()
boolean	isActive() True if active
boolean	isClosed() True if destroyed.
boolean	isFlushDirtyBlocksOnCommit() If true, dirty blocks are written at commit time.
boolean	isIndexBlock(long blockAddress) Return true if the block is an index block.
boolean	isInodePtrBlock(long blockAddress) Return true if the block is an index block.
boolean	isRowBlock(long blockAddress) Return true if the block is a row block.
Block	loadBlock(long blockAddress) Returns the matching block.
Block	readBlock(long blockAddress) Returns the matching block.
int	readBlock(long blockAddress, int blockOffset, byte[] buffer, int offset, int length) Reads a block.
int	readBlock(long blockAddress, int blockOffset, char[] buffer, int offset, int length) Reads a block for a clob.
void	readBlock(long blockId, int blockOffset, java.io.OutputStream os, int length) Reads a block to an output stream.
long	readBlockLong(long blockAddress, int offset) Reads a long value from a block.
void	readBlockNoLock(long blockId, int blockOffset, java.io.OutputStream os, int length) Reads a block to an output stream.
static long	readLong(byte[] buffer, int offset) Reads the long.
int	readMiniFragment(long fragmentAddress, int fragmentOffset, byte[] buffer, int offset, int length) Reads a fragment.
int	readMiniFragment(long fragmentAddress, int fragmentOffset, char[] buffer, int offset, int length) Reads a miniFragment for a clob.
long	readMiniFragmentLong(long fragmentAddress, int fragmentOffset) Reads a long value from a miniFragment.
int	readMiniFragmentNoLock(long fragmentAddress, int fragmentOffset, int length, java.io.OutputStream os) Reads a fragment.
void	remove()
void	saveAllocation() Sets the allocation for a block.
void	setEnableMmap(boolean isEnable)
void	setFlushDirtyBlocksOnCommit(boolean flushOnCommit) If true, dirty blocks are written at commit time.
java.lang.String	toString()
protected void	validateBlockId(long blockId) Check that an allocated block is valid.
void	wakeWriter()
Block	writeBlock(long blockAddress, int blockOffset, byte[] buffer, int offset, int length) Writes a block.
Block	writeBlock(long blockAddress, int blockOffset, char[] buffer, int offset, int charLength) Writes a character based block
Block	writeBlockLong(long blockAddress, int offset, long value) Writes a long value to a block
static void	writeLong(byte[] buffer, int offset, long v) Writes the long.
Block	writeMiniFragment(long fragmentAddress, int fragmentOffset, byte[] buffer, int offset, int length) Writes a miniFragment.
Block	writeMiniFragment(long fragmentAddress, int fragmentOffset, char[] buffer, int offset, int length) Writes a character based

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

BLOCK_BITS

public static final int BLOCK_BITS

See Also:

Constant Field Values

BLOCK_SIZE

public static final int BLOCK_SIZE

See Also:

Constant Field Values

BLOCK_INDEX_MASK

public static final long BLOCK_INDEX_MASK

See Also:

Constant Field Values

BLOCK_MASK

public static final long BLOCK_MASK

See Also:

Constant Field Values

BLOCK_OFFSET_MASK

public static final long BLOCK_OFFSET_MASK

See Also:

Constant Field Values

ALLOC_FREE

public static final int ALLOC_FREE

See Also:

Constant Field Values

ALLOC_ROW

public static final int ALLOC_ROW

See Also:

Constant Field Values

ALLOC_DATA

public static final int ALLOC_DATA

See Also:

Constant Field Values

ALLOC_INODE_PTR

public static final int ALLOC_INODE_PTR

See Also:

Constant Field Values

ALLOC_INDEX

public static final int ALLOC_INDEX

See Also:

Constant Field Values

ALLOC_MINI_FRAG

public static final int ALLOC_MINI_FRAG

See Also:

Constant Field Values

ALLOC_MASK

public static final int ALLOC_MASK

See Also:

Constant Field Values

MINI_FRAG_SIZE

public static final int MINI_FRAG_SIZE

See Also:

Constant Field Values

MINI_FRAG_PER_BLOCK

public static final int MINI_FRAG_PER_BLOCK

See Also:

Constant Field Values

MINI_FRAG_ALLOC_OFFSET

public static final int MINI_FRAG_ALLOC_OFFSET

See Also:

Constant Field Values

METADATA_START

public static final long METADATA_START

See Also:

Constant Field Values

STORE_CREATE_END

public static final int STORE_CREATE_END

See Also:

Constant Field Values

DATABASE_CORRUPT_EVENT

public static final java.lang.String DATABASE_CORRUPT_EVENT

See Also:

Constant Field Values

_database

protected final Database _database

_blockManager

protected final BlockManager _blockManager

Constructor Detail

BlockStore

public BlockStore(Database database,
                  java.lang.String name,
                  java.util.concurrent.locks.ReadWriteLock tableLock)

BlockStore

public BlockStore(Database database,
                  java.lang.String name,
                  java.util.concurrent.locks.ReadWriteLock rowLock,
                  Path path)

Creates a new store.

Parameters:

database - the owning database.

name - the store name

lock - the table lock

path - the path to the files

BlockStore

public BlockStore(Database database,
                  java.lang.String name,
                  java.util.concurrent.locks.ReadWriteLock rowLock,
                  Path path,
                  boolean isEnableMmap)

Creates a new store.

Parameters:

database - the owning database.

name - the store name

lock - the table lock

path - the path to the files

Method Detail

create

public static BlockStore create(Path path)
                         throws java.io.IOException,
                                java.sql.SQLException

Creates an independent store.

Throws:

java.io.IOException

java.sql.SQLException

createNoMmap

public static BlockStore createNoMmap(Path path)
                               throws java.io.IOException,
                                      java.sql.SQLException

Creates an independent store.

Throws:

java.io.IOException

java.sql.SQLException

createMmap

public static BlockStore createMmap(Path path)
                             throws java.io.IOException,
                                    java.sql.SQLException

Creates an independent store.

Throws:

java.io.IOException

java.sql.SQLException

create

public static BlockStore create(Path path,
                                boolean isMmap)
                         throws java.io.IOException,
                                java.sql.SQLException

Creates an independent store.

Throws:

java.io.IOException

java.sql.SQLException

setEnableMmap

public void setEnableMmap(boolean isEnable)

setFlushDirtyBlocksOnCommit

public void setFlushDirtyBlocksOnCommit(boolean flushOnCommit)

If true, dirty blocks are written at commit time.

isFlushDirtyBlocksOnCommit

public boolean isFlushDirtyBlocksOnCommit()

If true, dirty blocks are written at commit time.

getName

public java.lang.String getName()

Returns the store's name.

getId

public int getId()

Returns the store's id.

getPath

public Path getPath()

Returns the store's path.

getWriteLock

public java.util.concurrent.locks.Lock getWriteLock()

Returns the table's lock.

getTableLock

public java.util.concurrent.locks.Lock getTableLock()

Returns the table's lock.

getBlockManager

public BlockManager getBlockManager()

Returns the block manager.

getReadWrite

protected BlockReadWrite getReadWrite()

getMmap

public RandomAccessStream getMmap()

getFileSize

public long getFileSize()

Returns the file size.

getBlockCount

public long getBlockCount()

Returns the block count.

blockIndexToAddr

public static long blockIndexToAddr(long blockIndex)

Converts from the block index to the address for database storage.

blockIdToIndex

public static long blockIdToIndex(long blockId)

Converts from the block index to the address for database storage.

addressToBlockId

public final long addressToBlockId(long address)

Converts from the block index to the unique block id.

blockIdToAddress

public static long blockIdToAddress(long blockId)

Converts from the block index to the unique block id.

blockIdToAddress

public static long blockIdToAddress(long blockId,
                                    int offset)

Converts from the block index to the unique block id.

create

public void create()
            throws java.io.IOException,
                   java.sql.SQLException

Creates the store.

Throws:

java.io.IOException

java.sql.SQLException

init

public void init()
          throws java.io.IOException

Throws:

java.io.IOException

remove

public void remove()
            throws java.sql.SQLException

Throws:

java.sql.SQLException

firstRowBlock

public long firstRowBlock(long blockId)
                   throws java.io.IOException

Returns the first block id which contains a row.

Returns:

the block id of the first row block

Throws:

java.io.IOException

firstBlock

public long firstBlock(long blockId,
                       int type)
                throws java.io.IOException

Returns the first block id which contains a row.

Returns:

the block id of the first row block

Throws:

java.io.IOException

readBlock

public final Block readBlock(long blockAddress)
                      throws java.io.IOException

Returns the matching block.

Throws:

java.io.IOException

loadBlock

public final Block loadBlock(long blockAddress)
                      throws java.io.IOException

Returns the matching block.

Throws:

java.io.IOException

allocateRow

public Block allocateRow()
                  throws java.io.IOException

Allocates a new block for a row.

Returns:

the block id of the allocated block.

Throws:

java.io.IOException

isRowBlock

public boolean isRowBlock(long blockAddress)

Return true if the block is a row block.

allocateBlock

public Block allocateBlock()
                    throws java.io.IOException

Allocates a new block for a non-row.

Returns:

the block id of the allocated block.

Throws:

java.io.IOException

allocateIndirectBlock

public Block allocateIndirectBlock()
                            throws java.io.IOException

Allocates a new block for a non-row.

Returns:

the block id of the allocated block.

Throws:

java.io.IOException

allocateIndexBlock

public Block allocateIndexBlock()
                         throws java.io.IOException

Allocates a new block for an index

Returns:

the block id of the allocated block.

Throws:

java.io.IOException

isIndexBlock

public boolean isIndexBlock(long blockAddress)

Return true if the block is an index block.

isInodePtrBlock

public boolean isInodePtrBlock(long blockAddress)

Return true if the block is an index block.

validateBlockId

protected void validateBlockId(long blockId)
                        throws java.lang.IllegalArgumentException,
                               java.lang.IllegalStateException

Check that an allocated block is valid.

Throws:

java.lang.IllegalArgumentException

java.lang.IllegalStateException

assertStoreActive

protected void assertStoreActive()
                          throws java.lang.IllegalStateException

Check that an allocated block is valid.

Throws:

java.lang.IllegalStateException

deallocateBlock

public void deallocateBlock(long blockId)
                     throws java.io.IOException

Frees a block.

Throws:

java.io.IOException

getAllocationByAddress

public final int getAllocationByAddress(long blockAddress)

Sets the allocation for a block.

getAllocation

public final int getAllocation(long blockIndex)

Sets the allocation for a block.

saveAllocation

public void saveAllocation()
                    throws java.io.IOException

Sets the allocation for a block.

Throws:

java.io.IOException

readBlock

public void readBlock(long blockId,
                      int blockOffset,
                      java.io.OutputStream os,
                      int length)
               throws java.io.IOException

Reads a block to an output stream.

Parameters:

blockAddress - the address of the block

blockOffset - the offset inside the block to start reading

os - the result output stream

length - the number of bytes to read

Throws:

java.io.IOException

readBlockNoLock

public void readBlockNoLock(long blockId,
                            int blockOffset,
                            java.io.OutputStream os,
                            int length)
                     throws java.io.IOException

Reads a block to an output stream.

Parameters:

blockAddress - the address of the block

blockOffset - the offset inside the block to start reading

os - the result output stream

length - the number of bytes to read

Throws:

java.io.IOException

readBlock

public int readBlock(long blockAddress,
                     int blockOffset,
                     byte[] buffer,
                     int offset,
                     int length)
              throws java.io.IOException

Reads a block.

Parameters:

blockAddress - the address of the block

blockOffset - the offset inside the block to start reading

buffer - the result buffer

offset - offset into the result buffer

length - the number of bytes to read

Returns:

the number of bytes read

Throws:

java.io.IOException

readBlock

public int readBlock(long blockAddress,
                     int blockOffset,
                     char[] buffer,
                     int offset,
                     int length)
              throws java.io.IOException

Reads a block for a clob.

Parameters:

blockAddress - the address of the block

blockOffset - the offset inside the block to start reading

buffer - the result buffer

offset - offset into the result buffer

length - the length of the block in characters

Returns:

the number of characters read

Throws:

java.io.IOException

readBlockLong

public long readBlockLong(long blockAddress,
                          int offset)
                   throws java.io.IOException

Reads a long value from a block.

Returns:

the long value

Throws:

java.io.IOException

writeBlock

public Block writeBlock(long blockAddress,
                        int blockOffset,
                        byte[] buffer,
                        int offset,
                        int length)
                 throws java.io.IOException

Writes a block.

Parameters:

blockAddress - the block to write

blockOffset - the offset into the block

buffer - the write buffer

offset - offset into the write buffer

length - the number of bytes to write

Returns:

the fragment id

Throws:

java.io.IOException

writeBlock

public Block writeBlock(long blockAddress,
                        int blockOffset,
                        char[] buffer,
                        int offset,
                        int charLength)
                 throws java.io.IOException

Writes a character based block

Parameters:

blockAddress - the fragment to write

blockOffset - the offset into the fragment

buffer - the write buffer

offset - offset into the write buffer

length - the number of bytes to write

Throws:

java.io.IOException

writeBlockLong

public Block writeBlockLong(long blockAddress,
                            int offset,
                            long value)
                     throws java.io.IOException

Writes a long value to a block

Returns:

the long value

Throws:

java.io.IOException

readMiniFragment

public int readMiniFragment(long fragmentAddress,
                            int fragmentOffset,
                            byte[] buffer,
                            int offset,
                            int length)
                     throws java.io.IOException

Reads a fragment.

Parameters:

fragmentAddress - the address of the fragment

fragmentOffset - the offset inside the fragment to start reading

buffer - the result buffer

offset - offset into the result buffer

length - the number of bytes to read

Returns:

the number of bytes read

Throws:

java.io.IOException

readMiniFragmentNoLock

public int readMiniFragmentNoLock(long fragmentAddress,
                                  int fragmentOffset,
                                  int length,
                                  java.io.OutputStream os)
                           throws java.io.IOException

Reads a fragment.

Parameters:

fragmentAddress - the address of the fragment

fragmentOffset - the offset inside the fragment to start reading

buffer - the result buffer

offset - offset into the result buffer

length - the number of bytes to read

Returns:

the number of bytes read

Throws:

java.io.IOException

readMiniFragment

public int readMiniFragment(long fragmentAddress,
                            int fragmentOffset,
                            char[] buffer,
                            int offset,
                            int length)
                     throws java.io.IOException

Reads a miniFragment for a clob.

Parameters:

fragmentAddress - the address of the fragment

fragmentOffset - the offset inside the fragment to start reading

buffer - the result buffer

offset - offset into the result buffer

length - the length of the fragment in characters

Returns:

the number of characters read

Throws:

java.io.IOException

readMiniFragmentLong

public long readMiniFragmentLong(long fragmentAddress,
                                 int fragmentOffset)
                          throws java.io.IOException

Reads a long value from a miniFragment.

Returns:

the long value

Throws:

java.io.IOException

allocateMiniFragment

public long allocateMiniFragment()
                          throws java.io.IOException

Allocates a new miniFragment.

Returns:

the fragment address

Throws:

java.io.IOException

deleteMiniFragment

public void deleteMiniFragment(long fragmentAddress)
                        throws java.io.IOException

Deletes a miniFragment.

Throws:

java.io.IOException

writeMiniFragment

public Block writeMiniFragment(long fragmentAddress,
                               int fragmentOffset,
                               byte[] buffer,
                               int offset,
                               int length)
                        throws java.io.IOException

Writes a miniFragment.

Parameters:

fragmentAddress - the fragment to write

fragmentOffset - the offset into the fragment

buffer - the write buffer

offset - offset into the write buffer

length - the number of bytes to write

Returns:

the fragment id

Throws:

java.io.IOException

writeMiniFragment

public Block writeMiniFragment(long fragmentAddress,
                               int fragmentOffset,
                               char[] buffer,
                               int offset,
                               int length)
                        throws java.io.IOException

Writes a character based

Parameters:

miniFragmentAddress - the fragment to write

fragmentOffset - the offset into the fragment

buffer - the write buffer

offset - offset into the write buffer

length - the number of bytes to write

Throws:

java.io.IOException

flush

public void flush()

Flush the store.

fatalCorrupted

public void fatalCorrupted(java.lang.String msg)

isClosed

public boolean isClosed()

True if destroyed.

isActive

public boolean isActive()

True if active

close

public void close()

Closes the store.

fsync

public boolean fsync()
              throws java.io.IOException

Closes the store.

Throws:

java.io.IOException

wakeWriter

public void wakeWriter()

getAllocationTable

public byte[] getAllocationTable()

Returns a copy of the allocation table.

readLong

public static long readLong(byte[] buffer,
                            int offset)

Reads the long.

writeLong

public static void writeLong(byte[] buffer,
                             int offset,
                             long v)

Writes the long.

codeToName

public static java.lang.String codeToName(int code)

Debug names for the allocation.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object