Commit 2b4d8c1f authored by Donald H. (Donnie) Pinkston, III's avatar Donald H. (Donnie) Pinkston, III
Browse files

Files for B+ tree implementation / Lab 6

Design document and logistics document are in doc directory.

The pom.xml file is updated to include HW6 tests.

The B+ tree implementation is provided under the storage/btreefile
package.
parent 726cbe18
Showing with 5735 additions and 1 deletion
+5735 -1
CS122 Assignment 6 - B+ Tree Indexes - Design Document
======================================================
A: Analysis of Implementation
------------------------------
Given NanoDB's B+ tree implementation, consider a simple schema where an
index is built against a single integer column:
CREATE TABLE t (
-- An index is automatically built on the id column by NanoDB.
id INTEGER PRIMARY KEY,
value VARCHAR(20)
);
Answer the following questions.
A1. What is the total size of the index's search-key for the primary-key
index, in bytes? Break down this size into its individual components;
be as detailed as possible. (You don't need to go lower than the
byte-level in your answer, but you should show what each byte is a
part of.)
A2. What is the maximum number of search-keys that can be stored in leaf
nodes of NanoDB's B+ tree implementation? You should assume a page-
size of 8192 bytes.
A3. What is the maximum number of keys that can be stored in inner nodes
of this particular implementation? (Recall that every key must have
a page-pointer on either side of the key.)
A4. In this implementation, leaf nodes do not reference the previous
leaf, only the next leaf. When splitting a leaf into two leaves,
what is the maximum number of leaf nodes that must be read or written,
in order to properly manage the next-leaf pointers?
If leaves also contained a previous-leaf pointer, what would the
answer be instead?
Make sure to explain your answers.
A5. In this implementation, nodes do not store a page-pointer to their
parent node. This makes the update process somewhat complicated, as
we must save the sequence of page-numbers we traverse as we navigate
from root to leaf. If a node must be split, or if entries are to be
relocated from a node to its siblings, the node’s parent-node must
be retrieved, and the parent’s contents must be scanned to determine
the node’s sibling(s).
Consider an alternate B+ tree implementation in which every node
stores a page-pointer to the node’s parent. In the case of splitting
an inner node, what performance-related differences are there between
this alternate representation and the given implementation, where
nodes do not record their parents? Which one would you recommend?
Justify your answer.
A6. It should be obvious how indexes can be used to enforce primary keys,
but what role might they play with foreign keys? For example, given
this schema:
CREATE TABLE t1 (
id INTEGER PRIMARY KEY
);
CREATE TABLE t2 (
id INTEGER REFERENCES t1;
);
Why might we want to build an index on t2.id?
A7. Over time, a B+ tree's pages, its leaf pages in particular, may become
severely out of order, causing a significant number of seeks as the
leaves are traversed in sequence. Additionally, some of the blocks
within the B+ tree file may be empty.
An easy mechanism for regenerating a B+ tree file is to traverse the file's
tuples in sequential order (i.e. from the leftmost leaf page, through
all leaf pages in the file), adding each tuple to a new B+ tree file.
The new file may then replace the old file.
Imagine that this operation is performed on a B+ tree file containing
many records, where all records are the same size. On average, how full
will the leaf pages in the newly generated file be? You can state your
answer as a percentage, e.g. "0% full". Explain your answer.
A8. Consider a variation of the approach in A7: Instead of making one pass
through the initial B+ tree file, two passes are made. On the first pass,
the 1st, 3rd, 5th, 7th, ... tuples are added to the new file. Then, on the
second pass, the 2nd, 4th, 6th, 8th, ... tuples are added to the new file.
On average, how full will the leaf pages in the newly generated file be?
Explain your answer.
D: Extra Credit [OPTIONAL]
---------------------------
If you implemented any extra-credit tasks for this assignment, describe
them here. The description should be like this, with stuff in "<>" replaced.
(The value i starts at 1 and increments...)
D<i>: <one-line description>
<brief summary of what you did, including the specific classes that
we should look at for your implementation>
<brief summary of test-cases that demonstrate/exercise your extra work>
E: Feedback [OPTIONAL]
-----------------------
WE NEED YOUR FEEDBACK! Thoughtful and constructive input will help us to
improve future versions of the course. These questions are OPTIONAL, and
your answers will not affect your grade in any way (including if you hate
everything about the assignment and databases in general, or Donnie and/or
the TAs in particular). Feel free to answer as many or as few of them as
you wish.
E1. What parts of the assignment were most time-consuming? Why?
E2. Did you find any parts of the assignment particularly instructive?
Correspondingly, did any parts feel like unnecessary busy-work?
E3. Did you particularly enjoy any parts of the assignment? Were there
any parts that you particularly disliked?
E4. Were there any critical details that you wish had been provided with the
assignment, that we should consider including in subsequent versions of
the assignment?
E5. Do you have any other suggestions for how future versions of the
assignment can be improved?
CS122 Assignment 6 - B+ Tree Indexes
====================================
Please completely fill out this document so that we know who participated on
the assignment, any late extensions received, and how much time the assignment
took for your team. Thank you!
L1. List your team name and the people who worked on this assignment.
<team name>
<name>
<name>
...
L2. Specify the tag and commit-hash of the Git commit you are submitting for
your assignment. (You can list the hashes of all tags with the command
"git show-ref --tags".)
Tag: <tag>
Commit hash: <hash>
L3. Specify how many late tokens you are applying to this assignment, if any.
Similarly, if your team received an extension from Donnie then please
indicate how many days extension you received. You may leave this blank
if it is not relevant to this submission.
<tokens / extension>
L4. For each teammate, briefly describe what parts of the assignment each
teammate focused on, along with the total hours spent on the assignment.
...@@ -144,7 +144,7 @@ ...@@ -144,7 +144,7 @@
<suiteXmlFile>testng.xml</suiteXmlFile> <suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles> </suiteXmlFiles>
--> -->
<groups>framework,parser,hw1,hw2</groups> <groups>framework,parser,hw1,hw2,hw5,hw6</groups>
</configuration> </configuration>
</plugin> </plugin>
......
...@@ -101,6 +101,9 @@ public class IndexedTableManager implements TableManager { ...@@ -101,6 +101,9 @@ public class IndexedTableManager implements TableManager {
if ("heap".equals(storageType)) { if ("heap".equals(storageType)) {
type = DBFileType.HEAP_TUPLE_FILE; type = DBFileType.HEAP_TUPLE_FILE;
} }
else if ("btree".equals(storageType)) {
type = DBFileType.BTREE_TUPLE_FILE;
}
else { else {
throw new IllegalArgumentException("Unrecognized table file " + throw new IllegalArgumentException("Unrecognized table file " +
"type: " + storageType); "type: " + storageType);
......
...@@ -16,6 +16,8 @@ import edu.caltech.nanodb.server.EventDispatcher; ...@@ -16,6 +16,8 @@ import edu.caltech.nanodb.server.EventDispatcher;
import edu.caltech.nanodb.server.NanoDBServer; import edu.caltech.nanodb.server.NanoDBServer;
import edu.caltech.nanodb.server.properties.PropertyRegistry; import edu.caltech.nanodb.server.properties.PropertyRegistry;
import edu.caltech.nanodb.server.properties.ServerProperties; import edu.caltech.nanodb.server.properties.ServerProperties;
import edu.caltech.nanodb.storage.btreefile.BTreeTupleFileManager;
import edu.caltech.nanodb.storage.heapfile.HeapTupleFileManager; import edu.caltech.nanodb.storage.heapfile.HeapTupleFileManager;
import edu.caltech.nanodb.transactions.TransactionManager; import edu.caltech.nanodb.transactions.TransactionManager;
...@@ -139,6 +141,9 @@ public class StorageManager { ...@@ -139,6 +141,9 @@ public class StorageManager {
tupleFileManagers.put(DBFileType.HEAP_TUPLE_FILE, tupleFileManagers.put(DBFileType.HEAP_TUPLE_FILE,
new HeapTupleFileManager(this)); new HeapTupleFileManager(this));
tupleFileManagers.put(DBFileType.BTREE_TUPLE_FILE,
new BTreeTupleFileManager(this));
if (enableTransactions) { if (enableTransactions) {
logger.info("Initializing transaction manager."); logger.info("Initializing transaction manager.");
transactionManager = new TransactionManager(server); transactionManager = new TransactionManager(server);
......
package edu.caltech.nanodb.storage.btreefile;
import edu.caltech.nanodb.relations.Schema;
import edu.caltech.nanodb.storage.DBPage;
import edu.caltech.nanodb.storage.PageTuple;
/**
* <p>
* This class uses the <tt>PageTuple</tt> class functionality to access and
* manipulate keys stored in a B<sup>+</sup> tree tuple file. There is one
* extension, which is to allow the tuple to remember its index within the
* leaf page it is from; this makes it easy to move to the next tuple within
* the page very easily.
* </p>
* <p>
* B<sup>+</sup> tree tuple deletion is interesting, since all the tuples form
* a linear sequence in the page. When a given tuple T is deleted, the next
* tuple ends up at the same position that T was at. Therefore, to implement
* the tuple file's "get next tuple" functionality properly, we must keep
* track of whether the previous tuple was deleted or not; if it was deleted,
* we don't advance in the page.
* </p>
*/
public class BTreeFilePageTuple extends PageTuple {
private int tupleIndex;
/**
* Records if this tuple has been deleted or not. This affects navigation
* to the next tuple in the current page, since removal of the current
* tuple causes this object to point to the next tuple.
*/
private boolean deleted = false;
/**
* If this tuple is deleted, this field will be set to the page number of
* the next tuple. If there are no more tuples then this will be set to
* -1.
*/
private int nextTuplePageNo;
/**
* If this tuple is deleted, this field will be set to the index of the
* next tuple. If there are no more tuples then this will be set to -1.
*/
private int nextTupleIndex;
public BTreeFilePageTuple(Schema schema, DBPage dbPage, int pageOffset,
int tupleIndex) {
super(dbPage, pageOffset, schema);
if (tupleIndex < 0) {
throw new IllegalArgumentException(
"tupleIndex must be at least 0, got " + tupleIndex);
}
this.tupleIndex = tupleIndex;
}
public int getTupleIndex() {
return tupleIndex;
}
public boolean isDeleted() {
return deleted;
}
public void setDeleted() {
deleted = true;
}
public void setNextTuplePosition(int pageNo, int tupleIndex) {
if (!deleted)
throw new IllegalStateException("Tuple must be deleted");
nextTuplePageNo = pageNo;
nextTupleIndex = tupleIndex;
}
public int getNextTuplePageNo() {
if (!deleted)
throw new IllegalStateException("Tuple must be deleted");
return nextTuplePageNo;
}
public int getNextTupleIndex() {
if (!deleted)
throw new IllegalStateException("Tuple must be deleted");
return nextTupleIndex;
}
@Override
protected void insertTupleDataRange(int off, int len) {
throw new UnsupportedOperationException(
"B+ Tree index tuples don't support resizing.");
}
@Override
protected void deleteTupleDataRange(int off, int len) {
throw new UnsupportedOperationException(
"B+ Tree index tuples don't support resizing.");
}
@Override
public String toString() {
StringBuilder buf = new StringBuilder();
buf.append("BTPT[");
if (deleted) {
buf.append("deleted");
}
else {
boolean first = true;
for (int i = 0; i < getColumnCount(); i++) {
if (first)
first = false;
else
buf.append(',');
Object obj = getColumnValue(i);
if (obj == null)
buf.append("NULL");
else
buf.append(obj);
}
}
buf.append(']');
return buf.toString();
}
}
package edu.caltech.nanodb.storage.btreefile;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import edu.caltech.nanodb.expressions.TupleComparator;
import edu.caltech.nanodb.expressions.TupleLiteral;
import edu.caltech.nanodb.relations.Tuple;
import edu.caltech.nanodb.storage.DBFile;
import edu.caltech.nanodb.storage.DBPage;
import edu.caltech.nanodb.storage.StorageManager;
import static edu.caltech.nanodb.storage.btreefile.BTreePageTypes.*;
/**
* This class provides some simple verification operations for B<sup>+</sup>
* tree tuple files.
*/
public class BTreeFileVerifier {
/** A logging object for reporting anything interesting that happens. */
private static Logger logger = LogManager.getLogger(BTreeFileVerifier.class);
/** This runtime exception class is used to abort the verification scan. */
private static class ScanAbortedException extends RuntimeException { }
/**
* This helper class is used to keep track of details of pages within the
* B<sup>+</sup> tree file.
*/
private static class PageInfo {
/** The page's number. */
public int pageNo;
/** The type of the tuple-file page. */
int pageType;
/** A flag indicating whether the page is accessible from the root. */
boolean accessibleFromRoot;
/**
* Records how many times the page has been referenced in the file's
* tree structure.
*/
int numTreeReferences;
/**
* Records how many times the page has been referenced in the file's
* leaf page list.
*/
int numLeafListReferences;
/**
* Records how many times the page has been referenced in the file's
* empty page list.
*/
int numEmptyListReferences;
PageInfo(int pageNo, int pageType) {
this.pageNo = pageNo;
this.pageType = pageType;
accessibleFromRoot = false;
numTreeReferences = 0;
numLeafListReferences = 0;
numEmptyListReferences = 0;
}
}
/** A reference to the storage manager since we use it so much. */
private StorageManager storageManager;
/** The B<sup>+</sup> tree tuple file to verify. */
private BTreeTupleFile tupleFile;
/**
* The actual {@code DBFile} object backing the tuple-file, since it is
* used so frequently in the verification.
*/
private DBFile dbFile;
/**
* This collection maps individual B<sup>+</sup> tree pages to the details
* that the verifier collects for each page.
*/
private HashMap<Integer, PageInfo> pages;
/** The collection of errors found during the verification process. */
private ArrayList<String> errors;
/**
* Initialize a verifier object to verify a specific B<sup>+</sup> tree
* tuple file.
*/
public BTreeFileVerifier(StorageManager storageManager,
BTreeTupleFile tupleFile) {
this.storageManager = storageManager;
this.tupleFile = tupleFile;
this.dbFile = tupleFile.getDBFile();
}
/**
* This method is the entry-point for the verification process. It
* performs multiple passes through the file structure to identify various
* issues. Any errors that are identified are returned in the collection
* of error messages.
*
* @return a list of error messages describing issues found with the
* tuple file's structure. If no errors are detected, this will
* be an empty list, not {@code null}.
*/
public List<String> verify() {
errors = new ArrayList<>();
try {
pass1ScanThruAllPages();
pass2TreeScanThruFile();
pass3ScanThruLeafList();
pass4ScanThruEmptyList();
pass5ExaminePageReachability();
/* These passes are specifically part of index validation, not
* just the file format validation, so we may need to move this
* somewhere else.
*
pass6VerifyTableTuplesInIndex();
pass7VerifyIndexTuplesInTable();
*/
}
catch (ScanAbortedException e) {
// Do nothing.
logger.warn("Tuple-file verification scan aborted.");
}
return errors;
}
/**
* This method implements pass 1 of the verification process: scanning
* through all pages in the tuple file, collecting basic details about
* each page.
*/
private void pass1ScanThruAllPages() {
logger.debug("Pass 1: Linear scan through pages to collect info");
pages = new HashMap<>();
for (int pageNo = 1; pageNo < dbFile.getNumPages(); pageNo++) {
DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
int pageType = dbPage.readUnsignedByte(0);
PageInfo info = new PageInfo(pageNo, pageType);
pages.put(pageNo, info);
}
}
/**
* This method implements pass 2 of the verification process: performing
* a tree-scan through the entire tuple file, starting with the root page
* of the file.
*/
private void pass2TreeScanThruFile() {
logger.debug("Pass 2: Tree scan from root to verify nodes");
DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
int rootPageNo = HeaderPage.getRootPageNo(dbpHeader);
scanTree(rootPageNo, 0, null, null);
}
/**
* This helper function traverses the B<sup>+</sup> tree structure,
* verifying various invariants that should hold on the file structure.
*/
private void scanTree(int pageNo, int parentPageNo, Tuple parentLeftKey,
Tuple parentRightKey) {
PageInfo info = pages.get(pageNo);
info.accessibleFromRoot = true;
info.numTreeReferences++;
if (info.numTreeReferences > 10) {
errors.add(String.format("Pass 2: Stopping scan! I've visited " +
"page %d %d times; there may be a cycle in your B+ tree " +
"structure.", pageNo, info.numTreeReferences));
throw new ScanAbortedException();
}
logger.trace("Examining page " + pageNo);
DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
switch (info.pageType) {
case BTREE_INNER_PAGE:
{
logger.trace("It's an inner page.");
InnerPage inner = new InnerPage(dbPage, tupleFile.getSchema());
ArrayList<Integer> refPages = new ArrayList<>();
int refInner = 0;
int refLeaf = 0;
int refOther = 0;
// Check the pages referenced from this page using the basic info
// collected in Pass 1.
for (int p = 0; p < inner.getNumPointers(); p++) {
int refPageNo = inner.getPointer(p);
refPages.add(refPageNo);
PageInfo refPageInfo = pages.get(refPageNo);
switch (refPageInfo.pageType) {
case BTREE_INNER_PAGE:
refInner++;
break;
case BTREE_LEAF_PAGE:
refLeaf++;
break;
default:
refOther++;
}
if (refInner != 0 && refLeaf != 0) {
errors.add(String.format("Pass 2: Inner page %d " +
"references both inner and leaf pages.", pageNo));
}
if (refOther != 0) {
errors.add(String.format("Pass 2: Inner page %d references " +
"pages that are neither inner pages nor leaf pages.", pageNo));
}
}
// Make sure the keys are in the proper order in the page.
int numKeys = inner.getNumKeys();
ArrayList<TupleLiteral> keys = new ArrayList<>(numKeys);
if (numKeys > 1) {
Tuple prevKey = inner.getKey(0);
keys.add(new TupleLiteral(prevKey));
if (parentLeftKey != null) {
int cmp = TupleComparator.compareTuples(parentLeftKey, prevKey);
// It is possible that the parent's left-key would be the
// same as the first key in this page.
if (cmp > 0) {
errors.add(String.format("Pass 2: Parent page %d's " +
"left key is greater than inner page %d's first key",
parentPageNo, pageNo));
}
}
for (int k = 1; k < numKeys; k++) {
Tuple key = inner.getKey(k);
keys.add(new TupleLiteral(key));
int cmp = TupleComparator.compareTuples(prevKey, key);
if (cmp == 0) {
errors.add(String.format("Pass 2: Inner page %d keys " +
"%d and %d are duplicates!", pageNo, k - 1, k));
}
else if (cmp > 0) {
errors.add(String.format("Pass 2: Inner page %d keys " +
"%d and %d are out of order!", pageNo, k - 1, k));
}
prevKey = key;
}
if (parentRightKey != null) {
int cmp = TupleComparator.compareTuples(prevKey, parentRightKey);
// The parent's right-key should be greater than the last
// key in this page.
if (cmp >= 0) {
errors.add(String.format("Pass 2: Parent page %d's " +
"right key is less than or equal to inner page " +
"%d's last key", parentPageNo, pageNo));
}
}
}
// Now that we are done with this page, check each child-page.
int p = 0;
Tuple prevKey = parentLeftKey;
for (int refPageNo : refPages) {
Tuple nextKey;
if (p < keys.size())
nextKey = keys.get(p);
else
nextKey = parentRightKey;
scanTree(refPageNo, pageNo, prevKey, nextKey);
prevKey = nextKey;
p++;
}
break;
}
case BTREE_LEAF_PAGE:
{
logger.trace("It's a leaf page.");
LeafPage leaf = new LeafPage(dbPage, tupleFile.getSchema());
// Make sure the keys are in the proper order in the page.
int numKeys = leaf.getNumTuples();
if (numKeys >= 1) {
Tuple prevKey = leaf.getTuple(0);
if (parentLeftKey != null) {
int cmp = TupleComparator.compareTuples(parentLeftKey, prevKey);
// It is possible that the parent's left-key would be the
// same as the first key in this page.
if (cmp > 0) {
errors.add(String.format("Pass 2: Parent page %d's " +
"left key is greater than inner page %d's first key",
parentPageNo, pageNo));
}
}
for (int k = 1; k < numKeys; k++) {
Tuple key = leaf.getTuple(k);
int cmp = TupleComparator.compareTuples(prevKey, key);
if (cmp == 0) {
errors.add(String.format("Pass 2: Leaf page %d keys " +
"%d and %d are duplicates!", pageNo, k - 1, k));
}
else if (cmp > 0) {
errors.add(String.format("Pass 2: Leaf page %d keys " +
"%d and %d are out of order!", pageNo, k - 1, k));
}
prevKey = key;
}
if (parentRightKey != null) {
int cmp = TupleComparator.compareTuples(prevKey, parentRightKey);
// The parent's right-key should be greater than the last
// key in this page.
if (cmp >= 0) {
errors.add(String.format("Pass 2: Parent page %d's " +
"right key is less than or equal to inner page " +
"%d's last key", parentPageNo, pageNo));
}
}
}
break;
}
default:
errors.add(String.format("Pass 2: Can reach page %d from root, " +
"but it's not a leaf or an inner page! Type = %d", pageNo,
info.pageType));
}
}
private void pass3ScanThruLeafList() {
logger.debug("Pass 3: Scan through leaf page list");
DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
int pageNo = HeaderPage.getRootPageNo(dbpHeader);
// Walk down the leftmost pointers in the inner pages until we reach
// the leftmost leaf page. Then we can walk across the leaves and
// check the constraints that should hold on leaves.
DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
int pageType = dbPage.readUnsignedByte(0);
while (pageType != BTREE_LEAF_PAGE) {
if (pageType != BTREE_INNER_PAGE) {
errors.add(String.format("Pass 3: Page %d should be an inner " +
"page, but its type is %d instead", pageNo, pageType));
}
InnerPage innerPage = new InnerPage(dbPage, tupleFile.getSchema());
pageNo = innerPage.getPointer(0);
dbPage = storageManager.loadDBPage(dbFile, pageNo);
pageType = dbPage.readUnsignedByte(0);
}
// Now we should be at the leftmost leaf in the sequence of leaves.
Tuple prevKey = null;
int prevKeyPageNo = 0;
while (true) {
PageInfo info = pages.get(pageNo);
info.numLeafListReferences++;
if (info.numLeafListReferences > 10) {
errors.add(String.format("Pass 3: Stopping scan! I've visited " +
"leaf page %d %d times; there may be a cycle in your leaf list.",
pageNo, info.numLeafListReferences));
throw new ScanAbortedException();
}
LeafPage leafPage = new LeafPage(dbPage, tupleFile.getSchema());
for (int k = 0; k < leafPage.getNumTuples(); k++) {
Tuple key = leafPage.getTuple(k);
if (prevKey != null) {
int cmp = TupleComparator.compareTuples(prevKey, key);
if (cmp == 0) {
if (prevKeyPageNo == pageNo) {
errors.add(String.format("Pass 3: Leaf page %d " +
"keys %d and %d are duplicates!", pageNo,
k - 1, k));
}
else {
errors.add(String.format("Pass 3: Leaf page %d " +
"key 0 is a duplicate to previous leaf %d's " +
"last key!", pageNo, prevKeyPageNo));
}
}
else if (cmp > 0) {
if (prevKeyPageNo == pageNo) {
errors.add(String.format("Pass 3: Leaf page %d " +
"keys %d and %d are out of order!", pageNo,
k - 1, k));
}
else {
errors.add(String.format("Pass 3: Leaf page %d " +
"key 0 is out of order with previous leaf %d's " +
"last key!", pageNo, prevKeyPageNo));
}
}
}
prevKey = key;
prevKeyPageNo = pageNo;
}
// Go to the next leaf in the sequence.
pageNo = leafPage.getNextPageNo();
if (pageNo == 0)
break;
dbPage = storageManager.loadDBPage(dbFile, pageNo);
pageType = dbPage.readUnsignedByte(0);
if (pageType != BTREE_LEAF_PAGE) {
errors.add(String.format("Pass 3: Page %d should be a leaf " +
"page, but its type is %d instead", pageNo, pageType));
}
}
}
private void pass4ScanThruEmptyList() {
logger.debug("Pass 4: Scan through empty page list");
DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
int emptyPageNo = HeaderPage.getFirstEmptyPageNo(dbpHeader);
while (emptyPageNo != 0) {
PageInfo info = pages.get(emptyPageNo);
info.numEmptyListReferences++;
if (info.numEmptyListReferences > 10) {
errors.add(String.format("Pass 4: Stopping scan! I've visited " +
"empty page %d %d times; there may be a cycle in your empty list.",
emptyPageNo, info.numEmptyListReferences));
throw new ScanAbortedException();
}
if (info.pageType != BTREE_EMPTY_PAGE) {
errors.add(String.format("Page %d is in the empty-page list, " +
"but it isn't an empty page! Type = %d", emptyPageNo,
info.pageType));
}
DBPage dbPage = storageManager.loadDBPage(dbFile, emptyPageNo);
emptyPageNo = dbPage.readUnsignedShort(1);
}
}
private void pass5ExaminePageReachability() {
logger.debug("Pass 5: Find pages with reachability issues");
for (int pageNo = 1; pageNo < pages.size(); pageNo++) {
PageInfo info = pages.get(pageNo);
if (info.pageType == BTREE_INNER_PAGE ||
info.pageType == BTREE_LEAF_PAGE) {
if (info.numTreeReferences != 1) {
errors.add(String.format("B+ tree page %d should have " +
"exactly one tree-reference, but has %d instead",
info.pageNo, info.numTreeReferences));
}
if (info.pageType == BTREE_LEAF_PAGE &&
info.numLeafListReferences != 1) {
errors.add(String.format("Leaf page %d should have " +
"exactly one leaf-list reference, but has %d instead",
info.pageNo, info.numLeafListReferences));
}
}
else if (info.pageType == BTREE_EMPTY_PAGE) {
if (info.numEmptyListReferences != 1) {
errors.add(String.format("Empty page %d should have " +
"exactly one empty-list reference, but has %d instead",
info.pageNo, info.numEmptyListReferences));
}
}
}
}
}
package edu.caltech.nanodb.storage.btreefile;
/**
* This interface specifies the page-type values that may appear within the
* B<sup>+</sup> Tree implementation.
*
* @design (donnie) We use this instead of an {@code enum} since the values
* are actually read and written against pages in the B<sup>+</sup>
* Tree file. Note that there is no page-type value for the root
* page; that page is considered separately.
*
* @design (donnie) This class is package-private since it is an internal
* implementation detail and we want to keep it local to the
* {@code btreefile} package.
*/
final class BTreePageTypes {
/**
* This value is stored in a B-tree page's byte 0, to indicate that the
* page is an inner (i.e. non-leaf) page.
*/
public static final int BTREE_INNER_PAGE = 1;
/**
* This value is stored in a B-tree page's byte 0, to indicate that the
* page is a leaf page.
*/
public static final int BTREE_LEAF_PAGE = 2;
/**
* This value is stored in a B-tree page's byte 0, to indicate that the
* page is empty.
*/
public static final int BTREE_EMPTY_PAGE = 3;
}
package edu.caltech.nanodb.storage.btreefile;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import edu.caltech.nanodb.expressions.OrderByExpression;
import edu.caltech.nanodb.expressions.TupleComparator;
import edu.caltech.nanodb.expressions.TupleLiteral;
import edu.caltech.nanodb.queryeval.TableStats;
import edu.caltech.nanodb.relations.Schema;
import edu.caltech.nanodb.relations.Tuple;
import edu.caltech.nanodb.storage.DBFile;
import edu.caltech.nanodb.storage.DBPage;
import edu.caltech.nanodb.storage.FilePointer;
import edu.caltech.nanodb.storage.InvalidFilePointerException;
import edu.caltech.nanodb.storage.PageTuple;
import edu.caltech.nanodb.storage.SequentialTupleFile;
import edu.caltech.nanodb.storage.StorageManager;
import edu.caltech.nanodb.storage.TupleFileManager;
import static edu.caltech.nanodb.storage.btreefile.BTreePageTypes.*;
/**
* <p>
* This class, along with a handful of helper classes in this package,
* provides support for B<sup>+</sup> tree tuple files. B<sup>+</sup> tree
* tuple files can be used to provide a sequential storage format for ordered
* tuples. They are also used to implement indexes for enforcing primary,
* candidate and foreign keys, and also for providing optimized access to
* tuples with specific values.
* </p>
* <p>
* Here is a brief overview of the NanoDB B<sup>+</sup> tree file format:
* </p>
* <ul>
* <li>Page 0 is always a header page, and specifies the entry-points in the
* hierarchy: the root page of the tree, and the leftmost leaf of the
* tree. Page 0 also maintains a free-list of empty pages in the tree, so
* that adding new pages to the tree is fast. (See the {@link HeaderPage}
* class for details.)</li>
* <li>The remaining pages are either leaf pages, inner pages, or empty pages.
* The first byte of the page always indicates the kind of page. For
* details about the internal structure of leaf and inner pages, see the
* {@link InnerPage} and {@link LeafPage} classes.</li>
* <li>Empty pages are organized into a simple singly linked list. Each empty
* page holds a page-pointer to the next empty page in the sequence, using
* an unsigned short stored at index 1 (after the page-type value in index
* 0). The final empty page stores 0 as its next-page pointer value.</li>
* </ul>
*/
public class BTreeTupleFile implements SequentialTupleFile {
/** A logging object for reporting anything interesting that happens. */
private static Logger logger = LogManager.getLogger(BTreeTupleFile.class);
/**
* If this flag is set to true, all data in data-pages that is no longer
* necessary is cleared. This will increase the cost of write-ahead
* logging, but it also exposes bugs more quickly because old data won't
* still be present if someone erroneously accesses it.
*/
public static final boolean CLEAR_OLD_DATA = true;
/**
* The storage manager to use for reading and writing file pages, pinning
* and unpinning pages, write-ahead logging, and so forth.
*/
private StorageManager storageManager;
/**
* The manager for B<sup>+</sup> tree tuple files provides some
* higher-level operations such as saving the metadata of a tuple file,
* so it's useful to have a reference to it.
*/
private BTreeTupleFileManager btreeFileManager;
/** The schema of tuples in this tuple file. */
private Schema schema;
/** Statistics for this tuple file. */
private TableStats stats;
/** The file that stores the tuples. */
private DBFile dbFile;
/**
* A helper class that manages file-level operations on the B+ tree file.
*/
private FileOperations fileOps;
/**
* A helper class that manages the larger-scale operations involving leaf
* nodes of the B+ tree.
*/
private LeafPageOperations leafPageOps;
/**
* A helper class that manages the larger-scale operations involving inner
* nodes of the B+ tree.
*/
private InnerPageOperations innerPageOps;
// private IndexInfo idxFileInfo;
public BTreeTupleFile(StorageManager storageManager,
BTreeTupleFileManager btreeFileManager, DBFile dbFile,
Schema schema, TableStats stats) {
if (storageManager == null)
throw new IllegalArgumentException("storageManager cannot be null");
if (btreeFileManager == null)
throw new IllegalArgumentException("btreeFileManager cannot be null");
if (dbFile == null)
throw new IllegalArgumentException("dbFile cannot be null");
if (schema == null)
throw new IllegalArgumentException("schema cannot be null");
if (stats == null)
throw new IllegalArgumentException("stats cannot be null");
this.storageManager = storageManager;
this.btreeFileManager = btreeFileManager;
this.dbFile = dbFile;
this.schema = schema;
this.stats = stats;
fileOps = new FileOperations(storageManager, dbFile);
innerPageOps = new InnerPageOperations(storageManager, this, fileOps);
leafPageOps = new LeafPageOperations(storageManager, this, fileOps,
innerPageOps);
}
@Override
public TupleFileManager getManager() {
return btreeFileManager;
}
@Override
public Schema getSchema() {
return schema;
}
@Override
public TableStats getStats() {
return stats;
}
public DBFile getDBFile() {
return dbFile;
}
@Override
public List<OrderByExpression> getOrderSpec() {
throw new UnsupportedOperationException("NYI");
}
@Override
public Tuple getFirstTuple() {
BTreeFilePageTuple tup = null;
// By passing a completely empty Tuple (no columns), we can cause the
// navigateToLeafPage() method to choose the leftmost leaf page.
TupleLiteral noTup = new TupleLiteral();
LeafPage leaf = navigateToLeafPage(noTup, false, null);
if (leaf != null && leaf.getNumTuples() > 0)
tup = leaf.getTuple(0);
return tup;
}
@Override
public Tuple getNextTuple(Tuple tup) {
BTreeFilePageTuple tuple = (BTreeFilePageTuple) tup;
DBPage dbPage;
int nextIndex;
LeafPage leaf;
BTreeFilePageTuple nextTuple = null;
if (tuple.isDeleted()) {
// The tuple was deleted, so we need to find out the page number
// and index of the next tuple.
int nextPageNo = tuple.getNextTuplePageNo();
if (nextPageNo != 0) {
dbPage = storageManager.loadDBPage(dbFile, nextPageNo);
nextIndex = tuple.getNextTupleIndex();
leaf = new LeafPage(dbPage, schema);
if (nextIndex >= leaf.getNumTuples()) {
throw new IllegalStateException(String.format(
"The \"next tuple\" field of deleted tuple is too " +
"large (must be less than %d; got %d)",
leaf.getNumTuples(), nextIndex));
}
nextTuple = leaf.getTuple(nextIndex);
}
}
else {
// Get the page that holds the current entry, and see where it
// falls within the page.
dbPage = tuple.getDBPage();
leaf = new LeafPage(dbPage, schema);
// Use the offset of the passed-in entry to find the next entry.
// The next tuple follows the current tuple, unless the current
// tuple was deleted! In that case, the next tuple is actually
// where the current tuple used to be.
nextIndex = tuple.getTupleIndex() + 1;
if (nextIndex < leaf.getNumTuples()) {
// Still more entries in this leaf.
nextTuple = leaf.getTuple(nextIndex);
}
else {
// No more entries in this leaf. Must go to the next leaf.
int nextPageNo = leaf.getNextPageNo();
if (nextPageNo != 0) {
dbPage = storageManager.loadDBPage(dbFile, nextPageNo);
leaf = new LeafPage(dbPage, schema);
if (leaf.getNumTuples() > 0) {
nextTuple = leaf.getTuple(0);
}
else {
// This would be *highly* unusual. Leaves are
// supposed to be at least 1/2 full, always!
logger.error(String.format(
"Next leaf node %d has no entries?!", nextPageNo));
}
}
}
}
return nextTuple;
}
@Override
public Tuple getTuple(FilePointer fptr)
throws InvalidFilePointerException {
DBPage dbPage = storageManager.loadDBPage(dbFile, fptr.getPageNo());
if (dbPage == null) {
throw new InvalidFilePointerException("Specified page " +
fptr.getPageNo() + " doesn't exist in file " + dbFile);
}
// In the B+ tree file format, the file-pointer points to the actual
// tuple itself.
int fpOffset = fptr.getOffset();
LeafPage leaf = new LeafPage(dbPage, schema);
for (int i = 0; i < leaf.getNumTuples(); i++) {
BTreeFilePageTuple tup = leaf.getTuple(i);
if (tup.getOffset() == fpOffset)
return tup;
// Tuple offsets within a page will be monotonically increasing.
if (tup.getOffset() > fpOffset)
break;
}
throw new InvalidFilePointerException("No tuple at offset " + fptr);
}
@Override
public Tuple findFirstTupleEquals(Tuple searchKey) {
logger.debug("Finding first tuple that equals " + searchKey +
" in BTree file " + dbFile);
LeafPage leaf = navigateToLeafPage(searchKey, false, null);
if (leaf == null) {
// This case is handled by the below loop, but it will make for
// more understandable logging.
logger.debug("BTree file is empty!");
return null;
}
logger.debug("Navigated to leaf page " + leaf.getPageNo());
while (leaf != null) {
// We have at least one tuple to look at, so scan through to
// find the first tuple that equals what we are looking for.
for (int i = 0; i < leaf.getNumTuples(); i++) {
BTreeFilePageTuple tup = leaf.getTuple(i);
int cmp = TupleComparator.comparePartialTuples(tup, searchKey,
TupleComparator.CompareMode.IGNORE_LENGTH);
// (donnie) This is just way too much logging!
// logger.debug("Comparing search key to tuple " + tup +
// ", got cmp = " + cmp);
if (cmp == 0) {
// Found it!
return tup;
}
else if (cmp > 0) {
// Subsequent tuples will appear after the search key, so
// there's no point in going on.
leaf.getDBPage().unpin();
return null;
}
}
int nextPageNo = leaf.getNextPageNo();
logger.debug("Scanned through entire leaf page %d without " +
"finding tuple. Next page is %d.", leaf.getPageNo(),
nextPageNo);
// If we get here, we need to go to the next leaf-page.
leaf.getDBPage().unpin();
if (nextPageNo > 0) {
DBPage dbpNextLeaf =
storageManager.loadDBPage(dbFile, nextPageNo);
byte pageType = dbpNextLeaf.readByte(0);
if (pageType != BTREE_LEAF_PAGE) {
throw new BTreeTupleFileException(String.format(
"Expected page %d to be a leaf; found %d instead",
nextPageNo, pageType));
}
leaf = new LeafPage(dbpNextLeaf, schema);
}
else {
logger.debug("Reached end of leaf pages");
leaf = null;
}
}
// If we reach here, we reached the end of the leaf pages without
// finding the tuple.
return null;
}
@Override
public PageTuple findFirstTupleGreaterThan(Tuple searchKey) {
LeafPage leaf = navigateToLeafPage(searchKey, false, null);
if (leaf != null && leaf.getNumTuples() > 0) {
// We have at least one tuple to look at, so scan through to find
// the first tuple that equals what we are looking for.
for (int i = 0; i < leaf.getNumTuples(); i++) {
BTreeFilePageTuple tup = leaf.getTuple(i);
int cmp = TupleComparator.comparePartialTuples(tup, searchKey);
if (cmp > 0)
return tup; // Found it!
}
leaf.getDBPage().unpin();
}
return null;
}
@Override
public Tuple addTuple(Tuple tup) {
logger.debug("Adding tuple " + tup + " to BTree file " + dbFile);
// Navigate to the leaf-page, creating one if the BTree file is
// currently empty.
ArrayList<Integer> pagePath = new ArrayList<>();
LeafPage leaf = navigateToLeafPage(tup, true, pagePath);
// TODO: This is definitely not ideal, but should get us going.
TupleLiteral tupLit;
if (tup instanceof TupleLiteral)
tupLit = (TupleLiteral) tup;
else
tupLit = new TupleLiteral(tup);
tupLit.setStorageSize(PageTuple.getTupleStorageSize(schema, tupLit));
return leafPageOps.addTuple(leaf, tupLit, pagePath);
}
@Override
public void updateTuple(Tuple tup, Map<String, Object> newValues) {
throw new UnsupportedOperationException("NYI");
}
@Override
public void deleteTuple(Tuple tup) {
BTreeFilePageTuple tuple = (BTreeFilePageTuple) tup;
ArrayList<Integer> pagePath = new ArrayList<>();
LeafPage leaf = navigateToLeafPage(tup, false, pagePath);
logger.debug("Deleting tuple " + tuple + " from file " + dbFile);
leafPageOps.deleteTuple(leaf, tuple, pagePath);
tuple.setDeleted();
}
/**
* This helper method performs the common task of navigating from the root
* of the B<sup>+</sup> tree down to the appropriate leaf node, based on
* the search-key provided by the caller. Note that this method does not
* determine whether the search-key actually exists; rather, it simply
* navigates to the leaf in the file where the search-key would appear.
*
* @param searchKey the search-key being used to navigate the
* B<sup>+</sup> tree structure
*
* @param createIfNeeded If the B<sup>+</sup> tree is currently empty
* (i.e. not even containing leaf pages) then this argument can be
* used to create a new leaf page where the search-key can be
* stored. This allows the method to be used for adding tuples to
* the file.
*
* @param pagePath If this optional argument is specified, then the method
* stores the sequence of page-numbers it visits as it navigates
* from root to leaf. If {@code null} is passed then nothing is
* stored as the method traverses the B<sup>+</sup> tree structure.
*
* @return the leaf-page where the search-key would appear, or
* {@code null} if the B<sup>+</sup> tree file is currently empty
* and {@code createIfNeeded} is {@code false}.
*/
private LeafPage navigateToLeafPage(Tuple searchKey,
boolean createIfNeeded, List<Integer> pagePath) {
// The header page tells us where the root page starts.
DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
// Get the root page of the BTree file.
int rootPageNo = HeaderPage.getRootPageNo(dbpHeader);
DBPage dbpRoot;
if (rootPageNo == 0) {
// The file doesn't have any data-pages at all yet. Create one if
// the caller wants it.
if (!createIfNeeded)
return null;
// We need to create a brand new leaf page and make it the root.
logger.debug("BTree file currently has no data pages; " +
"finding/creating one to use as the root!");
dbpRoot = fileOps.getNewDataPage();
rootPageNo = dbpRoot.getPageNo();
HeaderPage.setRootPageNo(dbpHeader, rootPageNo);
HeaderPage.setFirstLeafPageNo(dbpHeader, rootPageNo);
dbpRoot.writeByte(0, BTREE_LEAF_PAGE);
LeafPage.init(dbpRoot, schema);
logger.debug("New root pageNo is " + rootPageNo);
}
else {
// The BTree file has a root page; load it.
dbpRoot = storageManager.loadDBPage(dbFile, rootPageNo);
logger.debug("BTree file root pageNo is " + rootPageNo);
}
// Next, descend down the file's structure until we find the proper
// leaf-page based on the key value(s).
DBPage dbPage = dbpRoot;
int pageType = dbPage.readByte(0);
if (pageType != BTREE_INNER_PAGE && pageType != BTREE_LEAF_PAGE) {
throw new BTreeTupleFileException(
"Invalid page type encountered: " + pageType);
}
if (pagePath != null)
pagePath.add(rootPageNo);
/* TODO: IMPLEMENT THE REST OF THIS METHOD.
*
* Don't forget to update the page-path as you navigate the index
* structure, if it is provided by the caller.
*
* Use the TupleComparator.comparePartialTuples() method for comparing
* the index's keys with the passed-in search key.
*
* It's always a good idea to code defensively: if you see an invalid
* page-type, flag it with an IOException, as done earlier.
*/
logger.error("NOT YET IMPLEMENTED: navigateToLeafPage()");
return null;
}
@Override
public void analyze() {
throw new UnsupportedOperationException("NYI");
}
@Override
public List<String> verify() {
BTreeFileVerifier verifier =
new BTreeFileVerifier(storageManager, this);
return verifier.verify();
}
@Override
public void optimize() {
throw new UnsupportedOperationException("NYI");
}
}
package edu.caltech.nanodb.storage.btreefile;
import edu.caltech.nanodb.storage.TupleFileException;
/**
* This class represents errors that occur while manipulating B<sup>+</sup>
* tree tuple files.
*/
public class BTreeTupleFileException extends TupleFileException {
/** Construct a tuple-file exception with no message. */
public BTreeTupleFileException() {
super();
}
/**
* Construct a tuple-file exception with the specified message.
*/
public BTreeTupleFileException(String msg) {
super(msg);
}
/**
* Construct a tuple-file exception with the specified cause and no message.
*/
public BTreeTupleFileException(Throwable cause) {
super(cause);
}
/**
* Construct a tuple-file exception with the specified message and cause.
*/
public BTreeTupleFileException(String msg, Throwable cause) {
super(msg, cause);
}
}
package edu.caltech.nanodb.storage.btreefile;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import edu.caltech.nanodb.queryeval.TableStats;
import edu.caltech.nanodb.relations.Schema;
import edu.caltech.nanodb.storage.DBFile;
import edu.caltech.nanodb.storage.DBFileType;
import edu.caltech.nanodb.storage.DBPage;
import edu.caltech.nanodb.storage.PageReader;
import edu.caltech.nanodb.storage.PageWriter;
import edu.caltech.nanodb.storage.SchemaWriter;
import edu.caltech.nanodb.storage.StatsWriter;
import edu.caltech.nanodb.storage.StorageManager;
import edu.caltech.nanodb.storage.TupleFile;
import edu.caltech.nanodb.storage.TupleFileManager;
/**
* This class provides high-level operations on B<sup>+</sup> tree tuple files.
*/
public class BTreeTupleFileManager implements TupleFileManager {
/** A logging object for reporting anything interesting that happens. */
private static Logger logger = LogManager.getLogger(BTreeTupleFileManager.class);
/** A reference to the storage manager. */
private StorageManager storageManager;
public BTreeTupleFileManager(StorageManager storageManager) {
if (storageManager == null)
throw new IllegalArgumentException("storageManager cannot be null");
this.storageManager = storageManager;
}
@Override
public DBFileType getDBFileType() {
return DBFileType.BTREE_TUPLE_FILE;
}
@Override
public String getShortName() {
return "btree";
}
@Override
public TupleFile createTupleFile(DBFile dbFile, Schema schema) {
logger.info(String.format(
"Initializing new btree tuple file %s with %d columns",
dbFile, schema.numColumns()));
// Table schema is stored into the header page, so get it and prepare
// to write out the schema information.
DBPage headerPage = storageManager.loadDBPage(dbFile, 0);
PageWriter hpWriter = new PageWriter(headerPage);
// Skip past the page-size value.
hpWriter.setPosition(HeaderPage.OFFSET_SCHEMA_START);
// Write out the schema details now.
SchemaWriter schemaWriter = new SchemaWriter();
schemaWriter.writeSchema(schema, hpWriter);
// Compute and store the schema's size.
int schemaEndPos = hpWriter.getPosition();
int schemaSize = schemaEndPos - HeaderPage.OFFSET_SCHEMA_START;
HeaderPage.setSchemaSize(headerPage, schemaSize);
// Write in empty statistics, so that the values are at least
// initialized to something.
TableStats stats = new TableStats(schema.numColumns());
StatsWriter.writeTableStats(schema, stats, hpWriter);
int statsSize = hpWriter.getPosition() - schemaEndPos;
HeaderPage.setStatsSize(headerPage, statsSize);
return new BTreeTupleFile(storageManager, this, dbFile, schema, stats);
}
@Override
public TupleFile openTupleFile(DBFile dbFile) {
logger.info("Opening existing btree tuple file " + dbFile);
// Table schema is stored into the header page, so get it and prepare
// to write out the schema information.
DBPage headerPage = storageManager.loadDBPage(dbFile, 0);
PageReader hpReader = new PageReader(headerPage);
// Skip past the page-size value.
hpReader.setPosition(HeaderPage.OFFSET_SCHEMA_START);
// Read in the schema details.
SchemaWriter schemaWriter = new SchemaWriter();
Schema schema = schemaWriter.readSchema(hpReader);
// Read in the statistics.
TableStats stats = StatsWriter.readTableStats(hpReader, schema);
return new BTreeTupleFile(storageManager, this, dbFile, schema, stats);
}
@Override
public void saveMetadata(TupleFile tupleFile) {
// TODO
throw new UnsupportedOperationException("NYI: deleteTupleFile()");
}
@Override
public void deleteTupleFile(TupleFile tupleFile) {
// TODO
throw new UnsupportedOperationException("NYI: deleteTupleFile()");
}
}
package edu.caltech.nanodb.storage.btreefile;
/**
* Created with IntelliJ IDEA.
* User: donnie
* Date: 2/4/13
* Time: 6:51 PM
* To change this template use File | Settings | File Templates.
*/
public interface DataPage {
/** The page type always occupies the first byte of the page. */
public static final int OFFSET_PAGE_TYPE = 0;
}
package edu.caltech.nanodb.storage.btreefile;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import edu.caltech.nanodb.storage.DBFile;
import edu.caltech.nanodb.storage.DBPage;
import edu.caltech.nanodb.storage.StorageManager;
/**
* This class provides file-level operations for the B<sup>+</sup> tree
* implementation, such as acquiring a new data page or releasing a data page.
*/
class FileOperations {
/** A logging object for reporting anything interesting that happens. */
private static Logger logger = LogManager.getLogger(FileOperations.class);
/** The storage manager to use for loading pages. */
private StorageManager storageManager;
/** The {@code DBFile} that the B<sup>+</sup> tree is stored in. */
private DBFile dbFile;
public FileOperations(StorageManager storageManager, DBFile dbFile) {
this.storageManager = storageManager;
this.dbFile = dbFile;
}
/**
* This helper function finds and returns a new data page, either by
* taking it from the empty-pages list in the file, or if the list is
* empty, creating a brand new page at the end of the file.
*
* @return an empty {@code DBPage} that can be used for storing tuple data
*/
public DBPage getNewDataPage() {
DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
DBPage newPage;
int pageNo = HeaderPage.getFirstEmptyPageNo(dbpHeader);
if (pageNo == 0) {
// There are no empty pages. Create a new page to use.
logger.debug("No empty pages. Extending BTree file " + dbFile +
" by one page.");
int numPages = dbFile.getNumPages();
newPage = storageManager.loadDBPage(dbFile, numPages, true);
}
else {
// Load the empty page, and remove it from the chain of empty pages.
logger.debug("First empty page number is " + pageNo);
newPage = storageManager.loadDBPage(dbFile, pageNo);
int nextEmptyPage = newPage.readUnsignedShort(1);
HeaderPage.setFirstEmptyPageNo(dbpHeader, nextEmptyPage);
}
logger.debug("Found data page to use: page " + newPage.getPageNo());
// TODO: Increment the number of data pages?
return newPage;
}
/**
* This helper function marks a data page in the B<sup>+</sup> tree file
* as "empty", and adds it to the list of empty pages in the file.
*
* @param dbPage the data-page that is no longer used.
*/
public void releaseDataPage(DBPage dbPage) {
// TODO: If this page is the last page of the index file, we could
// truncate pages off the end until we hit a non-empty page.
// Instead, we'll leave all the pages around forever...
DBFile dbFile = dbPage.getDBFile();
// Record in the page that it is empty.
dbPage.writeByte(0, BTreePageTypes.BTREE_EMPTY_PAGE);
DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
// Retrieve the old "first empty page" value, and store it in this page.
int prevEmptyPageNo = HeaderPage.getFirstEmptyPageNo(dbpHeader);
dbPage.writeShort(1, prevEmptyPageNo);
if (BTreeTupleFile.CLEAR_OLD_DATA) {
// Clear out the remainder of the data-page since it's now unused.
dbPage.setDataRange(3, dbPage.getPageSize() - 3, (byte) 0);
}
// Store the new "first empty page" value into the header.
HeaderPage.setFirstEmptyPageNo(dbpHeader, dbPage.getPageNo());
}
}
package edu.caltech.nanodb.storage.btreefile;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import edu.caltech.nanodb.storage.DBFileType;
import edu.caltech.nanodb.storage.DBPage;
/**
* This class manipulates the header page for a B<sup>+</sup> tree index file.
* The header page has the following structure:
*
* <ul>
* <li><u>Byte 0:</u> {@link DBFileType#BTREE_TUPLE_FILE} (unsigned byte)</li>
* <li><u>Byte 1:</u> page size <i>p</i> (unsigned byte) - file's page
* size is <i>P</i> = 2<sup>p</sup></li>
*
* <li>Byte 2-M: Specification of index key-columns and column ordering.</li>
* <li>Byte P-2 to P-1: the page of the file that is the root of the index</li>
* </ul>
*/
public class HeaderPage {
/** A logging object for reporting anything interesting that happens. */
private static Logger logger = LogManager.getLogger(HeaderPage.class);
/**
* The offset in the header page where the page number of the index's root
* page is stored. This value is an unsigned short.
*/
public static final int OFFSET_ROOT_PAGE = 2;
/**
* The offset in the header page where the page number of the first leaf
* page of the file is stored. This allows the leaves of the tuple file
* to be iterated through in sequential order. This value is an unsigned
* short.
*/
public static final int OFFSET_FIRST_LEAF_PAGE = 4;
/**
* The offset in the header page where the page number of the first empty
* page in the free list is stored. This value is an unsigned short.
*/
public static final int OFFSET_FIRST_EMPTY_PAGE = 6;
/**
* The offset in the header page where the length of the file's schema is
* stored. The statistics follow immediately after the schema.
*/
public static final int OFFSET_SCHEMA_SIZE = 8;
/**
* The offset in the header page where the size of the table statistics
* are stored. This value is an unsigned short.
*/
public static final int OFFSET_STATS_SIZE = 10;
/**
* The offset in the header page where the table schema starts. This
* value is an unsigned short.
*/
public static final int OFFSET_SCHEMA_START = 12;
/**
* This helper method simply verifies that the data page provided to the
* <tt>HeaderPage</tt> class is in fact a header-page (i.e. page 0 in the
* data file).
*
* @param dbPage the page to check
*
* @throws IllegalArgumentException if <tt>dbPage</tt> is <tt>null</tt>, or
* if it's not actually page 0 in the table file
*/
private static void verifyIsHeaderPage(DBPage dbPage) {
if (dbPage == null)
throw new IllegalArgumentException("dbPage cannot be null");
if (dbPage.getPageNo() != 0) {
throw new IllegalArgumentException(
"Page 0 is the header page in this storage format; was given page " +
dbPage.getPageNo());
}
}
/**
* Returns the page-number of the root page in the index file.
*
* @param dbPage the header page of the index file
* @return the page-number of the root page, or 0 if the index file doesn't
* contain a root page.
*/
public static int getRootPageNo(DBPage dbPage) {
verifyIsHeaderPage(dbPage);
return dbPage.readUnsignedShort(OFFSET_ROOT_PAGE);
}
/**
* Sets the page-number of the root page in the header page of the index
* file.
*
* @param dbPage the header page of the heap table file
* @param rootPageNo the page-number of the root page, or 0 if the index
* file doesn't contain a root page.
*/
public static void setRootPageNo(DBPage dbPage, int rootPageNo) {
verifyIsHeaderPage(dbPage);
if (rootPageNo < 0) {
throw new IllegalArgumentException(
"rootPageNo must be > 0; got " + rootPageNo);
}
dbPage.writeShort(OFFSET_ROOT_PAGE, rootPageNo);
}
/**
* Returns the page-number of the first leaf page in the index file.
*
* @param dbPage the header page of the index file
* @return the page-number of the first leaf page in the index file, or 0
* if the index file doesn't contain any leaf pages.
*/
public static int getFirstLeafPageNo(DBPage dbPage) {
verifyIsHeaderPage(dbPage);
return dbPage.readUnsignedShort(OFFSET_FIRST_LEAF_PAGE);
}
/**
* Sets the page-number of the first leaf page in the header page of the
* index file.
*
* @param dbPage the header page of the heap table file
* @param firstLeafPageNo the page-number of the first leaf page in the
* index file, or 0 if the index file doesn't contain any leaf pages.
*/
public static void setFirstLeafPageNo(DBPage dbPage, int firstLeafPageNo) {
verifyIsHeaderPage(dbPage);
if (firstLeafPageNo < 0) {
throw new IllegalArgumentException(
"firstLeafPageNo must be >= 0; got " + firstLeafPageNo);
}
dbPage.writeShort(OFFSET_FIRST_LEAF_PAGE, firstLeafPageNo);
}
/**
* Returns the page-number of the first empty page in the index file.
* Empty pages form a linked chain in the index file, so that they are
* easy to locate.
*
* @param dbPage the header page of the index file
* @return the page-number of the last leaf page in the index file.
*/
public static int getFirstEmptyPageNo(DBPage dbPage) {
verifyIsHeaderPage(dbPage);
return dbPage.readUnsignedShort(OFFSET_FIRST_EMPTY_PAGE);
}
/**
* Sets the page-number of the first empty page in the header page of the
* index file. Empty pages form a linked chain in the index file, so that
* they are easy to locate.
*
* @param dbPage the header page of the heap table file
* @param firstEmptyPageNo the page-number of the first empty page
*/
public static void setFirstEmptyPageNo(DBPage dbPage, int firstEmptyPageNo) {
verifyIsHeaderPage(dbPage);
if (firstEmptyPageNo < 0) {
throw new IllegalArgumentException(
"firstEmptyPageNo must be >= 0; got " + firstEmptyPageNo);
}
dbPage.writeShort(OFFSET_FIRST_EMPTY_PAGE, firstEmptyPageNo);
}
/**
* Returns the number of bytes that the table's schema occupies for storage
* in the header page.
*
* @param dbPage the header page of the heap table file
* @return the number of bytes that the table's schema occupies
*/
public static int getSchemaSize(DBPage dbPage) {
verifyIsHeaderPage(dbPage);
return dbPage.readUnsignedShort(OFFSET_SCHEMA_SIZE);
}
/**
* Sets the number of bytes that the table's schema occupies for storage
* in the header page.
*
* @param dbPage the header page of the heap table file
* @param numBytes the number of bytes that the table's schema occupies
*/
public static void setSchemaSize(DBPage dbPage, int numBytes) {
verifyIsHeaderPage(dbPage);
if (numBytes < 0) {
throw new IllegalArgumentException(
"numButes must be >= 0; got " + numBytes);
}
dbPage.writeShort(OFFSET_SCHEMA_SIZE, numBytes);
}
/**
* Returns the number of bytes that the table's statistics occupy for
* storage in the header page.
*
* @param dbPage the header page of the heap table file
* @return the number of bytes that the table's statistics occupy
*/
public static int getStatsSize(DBPage dbPage) {
verifyIsHeaderPage(dbPage);
return dbPage.readUnsignedShort(OFFSET_STATS_SIZE);
}
/**
* Sets the number of bytes that the table's statistics occupy for storage
* in the header page.
*
* @param dbPage the header page of the heap table file
* @param numBytes the number of bytes that the table's statistics occupy
*/
public static void setStatsSize(DBPage dbPage, int numBytes) {
verifyIsHeaderPage(dbPage);
if (numBytes < 0) {
throw new IllegalArgumentException(
"numButes must be >= 0; got " + numBytes);
}
dbPage.writeShort(OFFSET_STATS_SIZE, numBytes);
}
/**
* Returns the offset in the header page that the table statistics start at.
* This value changes because the table schema resides before the stats, and
* therefore the stats don't live at a fixed location.
*
* @param dbPage the header page of the heap table file
* @return the offset within the header page that the table statistics
* reside at
*/
public static int getStatsOffset(DBPage dbPage) {
verifyIsHeaderPage(dbPage);
return OFFSET_SCHEMA_START + getSchemaSize(dbPage);
}
}
package edu.caltech.nanodb.storage.btreefile;
import java.util.List;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import edu.caltech.nanodb.expressions.TupleLiteral;
import edu.caltech.nanodb.relations.Schema;
import edu.caltech.nanodb.relations.Tuple;
import edu.caltech.nanodb.storage.DBPage;
import edu.caltech.nanodb.storage.PageTuple;
import static edu.caltech.nanodb.storage.btreefile.BTreePageTypes.*;
/**
* <p>
* This class wraps a {@link DBPage} object that is an inner page in the
* B<sup>+</sup> tree file implementation, to provide some of the basic
* inner-page-management operations necessary for the file structure.
* </p>
* <p>
* Operations involving individual leaf-pages are provided by the
* {@link LeafPage} wrapper-class. Higher-level operations involving multiple
* leaves and/or inner pages of the B<sup>+</sup> tree structure, are provided
* by the {@link LeafPageOperations} and {@link InnerPageOperations} classes.
* </p>
*/
public class InnerPage implements DataPage {
/** A logging object for reporting anything interesting that happens. */
private static Logger logger = LogManager.getLogger(InnerPage.class);
/**
* The offset where the number of pointer entries is stored in the page.
* The page will hold one fewer tuples than pointers, since each tuple
* must be sandwiched between two pointers.
*/
public static final int OFFSET_NUM_POINTERS = 3;
/** The offset of the first pointer in the non-leaf page. */
public static final int OFFSET_FIRST_POINTER = 5;
/** The actual data page that holds the B<sup>+</sup> tree inner node. */
private DBPage dbPage;
/** The schema of the tuples in the leaf page. */
private Schema schema;
/** The number of pointers stored within this non-leaf page. */
private int numPointers;
/**
* An array of the offsets where the pointers are stored in this non-leaf
* page. Each pointer points to another page within the file. There is
* one more pointer than the number of tuples, since each tuple must be
* sandwiched between two pointers.
*/
private int[] pointerOffsets;
/** An array of the tuples stored in this non-leaf page. */
private BTreeFilePageTuple[] keys;
/**
* The total size of all data (pointers + tuples + initial values) stored
* within this non-leaf page. This is also the offset at which we can
* start writing more data without overwriting anything.
*/
private int endOffset;
/**
* Initialize the inner-page wrapper class for the specified B<sup>+</sup>
* tree leaf page. The contents of the inner-page are cached in the
* fields of the wrapper object.
*
* @param dbPage the data page from the B<sup>+</sup> Tree file to wrap
* @param schema the schema of tuples stored in the data page
*/
public InnerPage(DBPage dbPage, Schema schema) {
if (dbPage.readUnsignedByte(0) != BTREE_INNER_PAGE) {
throw new IllegalArgumentException("Specified DBPage " +
dbPage.getPageNo() + " is not marked as an inner page.");
}
this.dbPage = dbPage;
this.schema = schema;
loadPageContents();
}
/**
* This static helper function initializes a {@link DBPage} object's
* contents with the type and detail values that will allow a new
* {@code InnerPage} wrapper to be instantiated for the page, and then it
* returns a wrapper object for the page. This version of the {@code init}
* function creates an inner page that is initially empty.
*
* @param dbPage the page to initialize as an inner page.
*
* @param schema the schema of the tuples in the leaf page
*
* @return a newly initialized {@code InnerPage} object wrapping the page
*/
public static InnerPage init(DBPage dbPage, Schema schema) {
dbPage.writeByte(OFFSET_PAGE_TYPE, BTREE_INNER_PAGE);
dbPage.writeShort(OFFSET_NUM_POINTERS, 0);
return new InnerPage(dbPage, schema);
}
/**
* This static helper function initializes a {@link DBPage} object's
* contents with the type and detail values that will allow a new
* {@code InnerPage} wrapper to be instantiated for the page, and then it
* returns a wrapper object for the page. This version of the {@code init}
* function creates an inner page that initially contains the specified
* page-pointers and key value.
*
* @param dbPage the page to initialize as an inner page.
*
* @param schema the schema of the tuples in the inner page
*
* @param pagePtr1 the first page-pointer to store in the inner page, to the
* left of {@code key1}
*
* @param key1 the first key to store in the inner page
*
* @param pagePtr2 the second page-pointer to store in the inner page, to
* the right of {@code key1}
*
* @return a newly initialized {@code InnerPage} object wrapping the page
*/
public static InnerPage init(DBPage dbPage, Schema schema,
int pagePtr1, Tuple key1, int pagePtr2) {
dbPage.writeByte(OFFSET_PAGE_TYPE, BTREE_INNER_PAGE);
// Write the first contents of the non-leaf page: [ptr0, key0, ptr1]
// Since key0 will usually be a BTreeFilePageTuple, we have to rely on
// the storeTuple() method to tell us where the new tuple's data ends.
int offset = OFFSET_FIRST_POINTER;
dbPage.writeShort(offset, pagePtr1);
offset += 2;
offset = PageTuple.storeTuple(dbPage, offset, schema, key1);
dbPage.writeShort(offset, pagePtr2);
dbPage.writeShort(OFFSET_NUM_POINTERS, 2);
return new InnerPage(dbPage, schema);
}
/**
* This private helper scans through the inner page's contents and caches
* the contents of the inner page in a way that makes it easy to use and
* manipulate.
*/
private void loadPageContents() {
numPointers = dbPage.readUnsignedShort(OFFSET_NUM_POINTERS);
if (numPointers > 0) {
pointerOffsets = new int[numPointers];
keys = new BTreeFilePageTuple[numPointers - 1];
// Handle first pointer + key separately since we know their offsets
pointerOffsets[0] = OFFSET_FIRST_POINTER;
if (numPointers == 1) {
// This will happen when we are deleting values from a page.
// No keys, just 1 pointer, done!
endOffset = OFFSET_FIRST_POINTER + 2;
return;
}
BTreeFilePageTuple key = new BTreeFilePageTuple(schema, dbPage,
OFFSET_FIRST_POINTER + 2, 0);
keys[0] = key;
// Handle all the pointer/key pairs. This excludes the last
// pointer.
int keyEndOffset;
for (int i = 1; i < numPointers - 1; i++) {
// Next pointer starts where the previous key ends.
keyEndOffset = key.getEndOffset();
pointerOffsets[i] = keyEndOffset;
// Next key starts after the next pointer.
key = new BTreeFilePageTuple(schema, dbPage, keyEndOffset + 2, i);
keys[i] = key;
}
keyEndOffset = key.getEndOffset();
pointerOffsets[numPointers - 1] = keyEndOffset;
endOffset = keyEndOffset + 2;
}
else {
// There are no entries (pointers + keys).
endOffset = OFFSET_FIRST_POINTER;
pointerOffsets = null;
keys = null;
}
}
/**
* Returns the {@code DBPage} that backs this leaf page.
*
* @return the {@code DBPage} that backs this leaf page.
*/
public DBPage getDBPage() {
return dbPage;
}
/**
* Returns the page-number of this leaf page.
*
* @return the page-number of this leaf page.
*/
public int getPageNo() {
return dbPage.getPageNo();
}
/**
* Given a leaf page in the index, returns the page number of the left
* sibling, or -1 if there is no left sibling to this node.
*
* @param pagePath the page path from root to this leaf page
* @param innerOps the inner page ops that allows this method to
* load inner pages and navigate the tree
*
* @return the page number of the left sibling leaf-node, or -1 if there
* is no left sibling
*
* @review (Donnie) There is a lot of implementation-overlap between this
* function and the {@link InnerPage#getLeftSibling}. Maybe find
* a way to combine the implementations.
*/
public int getLeftSibling(List<Integer> pagePath,
InnerPageOperations innerOps) {
// Verify that the last node in the page path is in fact this page.
if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
throw new IllegalArgumentException(
"The page path provided does not terminate on this leaf page.");
}
// If this leaf doesn't have a parent, we already know it doesn't
// have a sibling.
if (pagePath.size() <= 1)
return -1;
int parentPageNo = pagePath.get(pagePath.size() - 2);
InnerPage inner = innerOps.loadPage(parentPageNo);
// Get the index of the pointer that points to this page. If it
// doesn't appear in the parent, we have a serious problem...
int pageIndex = inner.getIndexOfPointer(getPageNo());
if (pageIndex == -1) {
throw new IllegalStateException(String.format(
"Leaf node %d doesn't appear in parent inner node %d!",
getPageNo(), parentPageNo));
}
int leftSiblingIndex = pageIndex - 1;
int leftSiblingPageNo = -1;
if (leftSiblingIndex >= 0)
leftSiblingPageNo = inner.getPointer(leftSiblingIndex);
return leftSiblingPageNo;
}
/**
* Given a leaf page in the index, returns the page number of the right
* sibling, or -1 if there is no right sibling to this node.
*
* @param pagePath the page path from root to this leaf page
* @param innerOps the inner page ops that allows this method to
* load inner pages and navigate the tree
*
* @return the page number of the right sibling leaf-node, or -1 if there
* is no right sibling
*
* @review (Donnie) There is a lot of implementation-overlap between this
* function and the {@link InnerPage#getLeftSibling}. Maybe find
* a way to combine the implementations.
*/
public int getRightSibling(List<Integer> pagePath,
InnerPageOperations innerOps) {
// Verify that the last node in the page path is in fact this page.
if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
throw new IllegalArgumentException(
"The page path provided does not terminate on this inner page.");
}
// If this leaf doesn't have a parent, we already know it doesn't
// have a sibling.
if (pagePath.size() <= 1)
return -1;
int parentPageNo = pagePath.get(pagePath.size() - 2);
InnerPage inner = innerOps.loadPage(parentPageNo);
// Get the index of the pointer that points to this page. If it
// doesn't appear in the parent, we have a serious problem...
int pageIndex = inner.getIndexOfPointer(getPageNo());
if (pageIndex == -1) {
throw new IllegalStateException(String.format(
"Inner node %d doesn't appear in parent inner node %d!",
getPageNo(), parentPageNo));
}
int rightSiblingIndex = pageIndex + 1;
int rightSiblingPageNo = -1;
if (rightSiblingIndex < inner.getNumPointers())
rightSiblingPageNo = inner.getPointer(rightSiblingIndex);
return rightSiblingPageNo;
}
/**
* Returns the number of pointers currently stored in this inner page. The
* number of keys is always one less than the number of pointers, since
* each key must have a pointer on both sides.
*
* @return the number of pointers in this inner page.
*/
public int getNumPointers() {
return numPointers;
}
/**
* Returns the number of keys currently stored in this inner page. The
* number of keys is always one less than the number of pointers, since
* each key must have a pointer on both sides.
*
* @return the number of keys in this inner page.
*
* @throws IllegalStateException if the inner page contains 0 pointers
*/
public int getNumKeys() {
if (numPointers < 1) {
throw new IllegalStateException("Inner page contains no " +
"pointers. Number of keys is meaningless.");
}
return numPointers - 1;
}
/**
* Returns the total amount of space used in this page, in bytes.
*
* @return the total amount of space used in this page, in bytes.
*/
public int getUsedSpace() {
return endOffset;
}
/**
* Returns the amount of space used by key/pointer entries in this page,
* in bytes.
*
* @return the amount of space used by key/pointer entries in this page,
* in bytes.
*/
public int getSpaceUsedByEntries() {
return endOffset - OFFSET_FIRST_POINTER;
}
/**
* Returns the amount of space available in this inner page, in bytes.
*
* @return the amount of space available in this inner page, in bytes.
*/
public int getFreeSpace() {
return dbPage.getPageSize() - endOffset;
}
/**
* Returns the total space (page size) in bytes.
*
* @return the size of the page, in bytes.
*/
public int getTotalSpace() {
return dbPage.getPageSize();
}
/**
* Returns the pointer at the specified index.
*
* @param index the index of the pointer to retrieve
*
* @return the pointer at that index
*/
public int getPointer(int index) {
return dbPage.readUnsignedShort(pointerOffsets[index]);
}
/**
* Replaces one page-pointer in the inner page with another page-pointer.
* This is used when the B<sup>+</sup> tree is being optimized, as the
* layout of the data on disk is rearranged.
*
* @param index the index of the pointer to replace
* @param newPageNo the page number to store at the specified index
*/
public void replacePointer(int index, int newPageNo) {
dbPage.writeShort(pointerOffsets[index], newPageNo);
loadPageContents();
}
/**
* Returns the key at the specified index.
*
* @param index the index of the key to retrieve
*
* @return the key at that index
*/
public BTreeFilePageTuple getKey(int index) {
return keys[index];
}
/**
* This helper method scans the inner page for the specified page-pointer,
* returning the index of the pointer if it is found, or -1 if the pointer
* is not found.
*
* @param pointer the page-pointer to find in this inner page
*
* @return the index of the page-pointer if found, or -1 if not found
*/
public int getIndexOfPointer(int pointer) {
for (int i = 0; i < getNumPointers(); i++) {
if (getPointer(i) == pointer)
return i;
}
return -1;
}
public void replaceTuple(int index, Tuple key) {
int oldStart = keys[index].getOffset();
int oldLen = keys[index].getEndOffset() - oldStart;
int newLen = PageTuple.getTupleStorageSize(schema, key);
if (newLen != oldLen) {
// Need to adjust the amount of space the key takes.
if (endOffset + newLen - oldLen > dbPage.getPageSize()) {
throw new IllegalArgumentException(
"New key-value is too large to fit in non-leaf page.");
}
dbPage.moveDataRange(oldStart + oldLen, oldStart + newLen,
endOffset - oldStart - oldLen);
}
PageTuple.storeTuple(dbPage, oldStart, schema, key);
// Reload the page contents.
// TODO: This is slow, but it should be fine for now.
loadPageContents();
}
/**
* This method inserts a new key and page-pointer into the inner page,
* immediately following the page-pointer {@code pagePtr1}, which must
* already appear within the page. The caller is expected to have already
* verified that the new key and page-pointer are able to fit in the page.
*
* @param pagePtr1 the page-pointer which should appear before the new key
* in the inner page. <b>This is required to already appear within
* the inner page.</b>
*
* @param key1 the new key to add to the inner page, immediately after the
* {@code pagePtr1} value.
*
* @param pagePtr2 the new page-pointer to add to the inner page,
* immediately after the {@code key1} value.
*
* @throws IllegalArgumentException if the specified {@code pagePtr1} value
* cannot be found in the inner page, or if the new key and
* page-pointer won't fit within the space available in the page.
*/
public void addEntry(int pagePtr1, Tuple key1, int pagePtr2) {
if (logger.isTraceEnabled()) {
logger.trace("Non-leaf page " + getPageNo() +
" contents before adding entry:\n" + toFormattedString());
}
int i;
for (i = 0; i < numPointers; i++) {
if (getPointer(i) == pagePtr1)
break;
}
logger.debug(String.format("Found page-pointer %d in index %d",
pagePtr1, i));
if (i == numPointers) {
throw new IllegalArgumentException(
"Can't find initial page-pointer " + pagePtr1 +
" in non-leaf page " + getPageNo());
}
// Figure out where to insert the new key and value.
int oldKeyStart;
if (i < numPointers - 1) {
// There's a key i associated with pointer i. Use the key's offset,
// since it's after the pointer.
oldKeyStart = keys[i].getOffset();
}
else {
// The pageNo1 pointer is the last pointer in the sequence. Use
// the end-offset of the data in the page.
oldKeyStart = endOffset;
}
int len = endOffset - oldKeyStart;
// Compute the size of the new key and pointer, and make sure they fit
// into the page.
int newKeySize = PageTuple.getTupleStorageSize(schema, key1);
int newEntrySize = newKeySize + 2;
if (endOffset + newEntrySize > dbPage.getPageSize()) {
throw new IllegalArgumentException("New key-value and " +
"page-pointer are too large to fit in non-leaf page.");
}
if (len > 0) {
// Move the data after the pageNo1 pointer to make room for
// the new key and pointer.
dbPage.moveDataRange(oldKeyStart, oldKeyStart + newEntrySize, len);
}
// Write in the new key/pointer values.
PageTuple.storeTuple(dbPage, oldKeyStart, schema, key1);
dbPage.writeShort(oldKeyStart + newKeySize, pagePtr2);
// Finally, increment the number of pointers in the page, then reload
// the cached data.
dbPage.writeShort(OFFSET_NUM_POINTERS, numPointers + 1);
loadPageContents();
if (logger.isTraceEnabled()) {
logger.trace("Non-leaf page " + getPageNo() +
" contents after adding entry:\n" + toFormattedString());
}
}
/**
* This function will delete a page-pointer from this inner page, along
* with the key either to the left or to the right of the pointer. It is
* up to the caller to determine whether the left key or the right key
* should be deleted.
*
* @param pagePtr the page-pointer value to identify and remove
*
* @param removeRightKey a flag specifying whether the key to the right
* ({@code true}) or to the left ({@code false}) should be removed
*/
public void deletePointer(int pagePtr, boolean removeRightKey) {
logger.debug("Trying to delete page-pointer " + pagePtr +
" from inner page " + getPageNo() + ", and remove the " +
(removeRightKey ? "right" : "left") + " key.");
int ptrIndex = getIndexOfPointer(pagePtr);
if (ptrIndex == -1) {
throw new IllegalArgumentException(String.format(
"Can't find page-pointer %d in inner page %d", pagePtr,
dbPage.getPageNo()));
}
if (ptrIndex == 0 && !removeRightKey) {
throw new IllegalArgumentException(String.format(
"Tried to delete page-pointer %d and the key to the left," +
" in inner page %d, but the pointer has no left key.",
pagePtr, dbPage.getPageNo()));
}
if (ptrIndex == numPointers - 1 && removeRightKey) {
throw new IllegalArgumentException(String.format(
"Tried to delete page-pointer %d and the key to the right," +
" in inner page %d, but the pointer has no right key.",
pagePtr, dbPage.getPageNo()));
}
// Figure out the range of data that must be deleted from the page.
// Page pointers are 2 bytes.
int start = pointerOffsets[ptrIndex];
int end = start + 2;
// We must always remove one key, either the left or the right key.
// Expand the data range that we are removing.
if (removeRightKey) {
// Remove the key to the right of the page-pointer.
end = keys[ptrIndex].getEndOffset();
logger.debug(String.format("Removing right key, with size %d." +
" Range being removed is [%d, %d).", keys[ptrIndex].getSize(),
start, end));
}
else {
// Remove the key to the left of the page-pointer.
start = keys[ptrIndex - 1].getOffset();
logger.debug(String.format("Removing left key, with size %d." +
" Range being removed is [%d, %d).", keys[ptrIndex - 1].getSize(),
start, end));
}
logger.debug("Moving inner-page data in range [" + end + ", " +
endOffset + ") over by " + (end - start) + " bytes");
dbPage.moveDataRange(end, start, endOffset - end);
// Decrement the total number of pointers.
dbPage.writeShort(OFFSET_NUM_POINTERS, numPointers - 1);
logger.debug("Loading altered page - had " + numPointers +
" pointers before delete.");
// Load new page.
loadPageContents();
logger.debug("After loading, have " + numPointers + " pointers");
}
/**
* <p>
* This helper function moves the specified number of page-pointers to the
* left sibling of this inner node. The data is copied in one shot so that
* the transfer will be fast, and the various associated bookkeeping values
* in both inner pages are updated.
* </p>
* <p>
* Of course, moving a subset of the page-pointers to a sibling will leave
* a key without a pointer on one side; this key is promoted up to the
* parent of the inner node. Additionally, an existing parent-key can be
* provided by the caller, which should be inserted before the new pointers
* being moved into the sibling node.
* </p>
*
* @param leftSibling the left sibling of this inner node in the index file
*
* @param count the number of pointers to move to the left sibling
*
* @param parentKey If this inner node and the sibling already have a parent
* node, this is the key between the two nodes' page-pointers in the
* parent node. If the two nodes don't have a parent (i.e. because
* an inner node is being split into two nodes and the depth of the
* tree is being increased) then this value will be {@code null}.
*
* @return the key that should go into the parent node, between the
* page-pointers for this node and its sibling
*
* @todo (Donnie) When support for deletion is added to the index
* implementation, we will need to support the case when the incoming
* {@code parentKey} is non-{@code null}, but the returned key is
* {@code null} because one of the two siblings' pointers will be
* removed.
*/
public TupleLiteral movePointersLeft(InnerPage leftSibling, int count,
Tuple parentKey) {
if (count < 0 || count > numPointers) {
throw new IllegalArgumentException("count must be in range (0, " +
numPointers + "), got " + count);
}
// The parent-key can be null if we are splitting a page into two pages.
// However, this situation is only valid if the right sibling is EMPTY.
int parentKeyLen = 0;
if (parentKey != null) {
parentKeyLen = PageTuple.getTupleStorageSize(schema, parentKey);
}
else {
if (leftSibling.getNumPointers() != 0) {
throw new IllegalStateException("Cannot move pointers to " +
"non-empty sibling if no parent-key is specified!");
}
}
/* TODO: IMPLEMENT THE REST OF THIS METHOD.
*
* You can use PageTuple.storeTuple() to write a key into a DBPage.
*
* The DBPage.write() method is useful for copying a large chunk of
* data from one DBPage to another.
*
* Your implementation also needs to properly handle the incoming
* parent-key, and produce a new parent-key as well.
*/
logger.error("NOT YET IMPLEMENTED: movePointersLeft()");
// Update the cached info for both non-leaf pages.
loadPageContents();
leftSibling.loadPageContents();
return null;
}
/**
* Returns the page path to the right sibling, including the right sibling
* itself. Empty list if there is none.
*
* @param pagePath the page path from root to this leaf
* @param innerOps the inner page ops that allows this method to
* load inner pages and navigate the tree
*
* @return the page path to the right sibling leaf node
*
public List<Integer> getRightSibling(List<Integer> pagePath,
InnerPageOperations innerOps) {
// Verify that the last node in the page path is this leaf page.
if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
throw new IllegalArgumentException("The page path provided does" +
" not terminate on this leaf page.");
}
ArrayList<Integer> rightPath = new ArrayList<Integer>();
InnerPage inner = null;
int index = 0;
int i = pagePath.size() - 2;
try {
while (i >= 0) {
inner = innerOps.loadPage(idxFileInfo, pagePath.get(i));
index = inner.getIndexOfPointer(pagePath.get(i+1));
if (index != inner.getNumPointers() - 1) {
// This means that the subtree this leaf is in has a right
// sibling subtree from the current inner node.
rightPath.addAll(pagePath.subList(0, i+1));
break;
}
i--;
}
int nextPage;
if (inner == null || i == -1) {
return rightPath;
}
// Add to the rightPath the page corresponding to one to the
// right of the current index
rightPath.add(inner.getPointer(index + 1));
i++;
while (i <= pagePath.size() - 2) {
index = 0;
nextPage = inner.getPointer(index);
rightPath.add(nextPage);
inner = innerOps.loadPage(idxFileInfo, nextPage);
i++;
}
} catch (IOException e) {
throw new IllegalArgumentException("A page failed to load!");
}
// Can assert that for the last entry, leaf.getNextLeafPage
// should be last pagePath entry here
return rightPath;
}
/**
* Returns the page path to the left sibling, including the left sibling
* itself. Empty list if there is none.
*
* @param pagePath the page path from root to this leaf
* @param innerOps the inner page ops that allows this method
* to load inner pages and navigate the tree
*
* @return the page path to the left sibling leaf node
*
public List<Integer> getLeftSibling(List<Integer> pagePath, InnerPageOperations innerOps) {
// Verify that the last node in the page path is this leaf page.
if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
throw new IllegalArgumentException("The page path provided does" +
" not terminate on this leaf page.");
}
ArrayList<Integer> leftPath = new ArrayList<Integer>();
// Note to self - not sure on behavior for initializing a for loop
// with an i that does not satisfy condition. If it doesn't do any
// iterations, then that would be ideal behavior. That case should
// never occur anyways...
InnerPage inner = null;
int index = 0;
int i = pagePath.size() - 2;
try {
while (i >= 0) {
inner = innerOps.loadPage(idxFileInfo, pagePath.get(i));
index = inner.getIndexOfPointer(pagePath.get(i+1));
if (index != 0) {
// This means that the subtree this leaf is in has a left
// sibling subtree from the current inner node.
leftPath.addAll(pagePath.subList(0, i+1));
break;
}
i--;
}
int nextPage;
if (inner == null || i == -1) {
return leftPath;
}
// Add to the leftPath the page corresponding to one to the
// left of the current index
leftPath.add(inner.getPointer(index - 1));
i++;
while (i <= pagePath.size() - 2) {
index = inner.getNumPointers() - 1;
nextPage = inner.getPointer(index);
leftPath.add(nextPage);
inner = innerOps.loadPage(idxFileInfo, nextPage);
i++;
}
}
catch (IOException e) {
throw new IllegalArgumentException("A page failed to load!");
}
return leftPath;
}
*/
/**
* <p>
* This helper function moves the specified number of page-pointers to the
* right sibling of this inner node. The data is copied in one shot so that
* the transfer will be fast, and the various associated bookkeeping values
* in both inner pages are updated.
* </p>
* <p>
* Of course, moving a subset of the page-pointers to a sibling will leave
* a key without a pointer on one side; this key is promoted up to the
* parent of the inner node. Additionally, an existing parent-key can be
* provided by the caller, which should be inserted before the new pointers
* being moved into the sibling node.
* </p>
*
* @param rightSibling the right sibling of this inner node in the index file
*
* @param count the number of pointers to move to the right sibling
*
* @param parentKey If this inner node and the sibling already have a parent
* node, this is the key between the two nodes' page-pointers in the
* parent node. If the two nodes don't have a parent (i.e. because
* an inner node is being split into two nodes and the depth of the
* tree is being increased) then this value will be {@code null}.
*
* @return the key that should go into the parent node, between the
* page-pointers for this node and its sibling
*
* @todo (Donnie) When support for deletion is added to the index
* implementation, we will need to support the case when the incoming
* {@code parentKey} is non-{@code null}, but the returned key is
* {@code null} because one of the two siblings' pointers will be
* removed.
*/
public TupleLiteral movePointersRight(InnerPage rightSibling, int count,
Tuple parentKey) {
if (count < 0 || count > numPointers) {
throw new IllegalArgumentException("count must be in range [0, " +
numPointers + "), got " + count);
}
if (logger.isTraceEnabled()) {
logger.trace("Non-leaf page " + getPageNo() +
" contents before moving pointers right:\n" + toFormattedString());
}
int startPointerIndex = numPointers - count;
int startOffset = pointerOffsets[startPointerIndex];
int len = endOffset - startOffset;
logger.debug("Moving everything after pointer " + startPointerIndex +
" to right sibling. Start offset = " + startOffset +
", end offset = " + endOffset + ", len = " + len);
// The parent-key can be null if we are splitting a page into two pages.
// However, this situation is only valid if the right sibling is EMPTY.
int parentKeyLen = 0;
if (parentKey != null) {
parentKeyLen = PageTuple.getTupleStorageSize(schema, parentKey);
}
else {
if (rightSibling.getNumPointers() != 0) {
throw new IllegalStateException("Cannot move pointers to " +
"non-empty sibling if no parent-key is specified!");
}
}
/* TODO: IMPLEMENT THE REST OF THIS METHOD.
*
* You can use PageTuple.storeTuple() to write a key into a DBPage.
*
* The DBPage.write() method is useful for copying a large chunk of
* data from one DBPage to another.
*
* Your implementation also needs to properly handle the incoming
* parent-key, and produce a new parent-key as well.
*/
logger.error("NOT YET IMPLEMENTED: movePointersRight()");
// Update the cached info for both non-leaf pages.
loadPageContents();
rightSibling.loadPageContents();
if (logger.isTraceEnabled()) {
logger.trace("Non-leaf page " + getPageNo() +
" contents after moving pointers right:\n" + toFormattedString());
logger.trace("Right-sibling page " + rightSibling.getPageNo() +
" contents after moving pointers right:\n" +
rightSibling.toFormattedString());
}
return null;
}
/**
* <p>
* This helper method creates a formatted string containing the contents of
* the inner page, including the pointers and the intervening keys.
* </p>
* <p>
* It is strongly suggested that this method should only be used for
* trace-level output, since otherwise the output will become overwhelming.
* </p>
*
* @return a formatted string containing the contents of the inner page
*/
public String toFormattedString() {
StringBuilder buf = new StringBuilder();
buf.append(String.format("Inner page %d contains %d pointers%n",
getPageNo(), numPointers));
if (numPointers > 0) {
for (int i = 0; i < numPointers - 1; i++) {
buf.append(String.format(" Pointer %d = page %d%n", i,
getPointer(i)));
buf.append(String.format(" Key %d = %s%n", i, getKey(i)));
}
buf.append(String.format(" Pointer %d = page %d%n", numPointers - 1,
getPointer(numPointers - 1)));
}
return buf.toString();
}
}
package edu.caltech.nanodb.storage.btreefile;
import java.util.List;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import edu.caltech.nanodb.expressions.TupleLiteral;
import edu.caltech.nanodb.relations.Tuple;
import edu.caltech.nanodb.storage.DBFile;
import edu.caltech.nanodb.storage.DBPage;
import edu.caltech.nanodb.storage.PageTuple;
import edu.caltech.nanodb.storage.StorageManager;
/**
* This class provides high-level B<sup>+</sup> tree management operations
* performed on inner nodes. These operations are provided here and not on the
* {@link InnerPage} class since they sometimes involve splitting or merging
* inner nodes, updating parent nodes, and so forth.
*/
public class InnerPageOperations {
/**
* When deleting a page-pointer from an inner page, either the left or
* right key must also be removed; this constant specifies removal of the
* left key.
*
* @see #deletePointer
*/
public static final int REMOVE_KEY_TO_LEFT = 0;
/**
* When deleting a page-pointer from an inner page, either the left or
* right key must also be removed; this constant specifies removal of the
* right key.
*
* @see #deletePointer
*/
public static final int REMOVE_KEY_TO_RIGHT = 1;
/** A logging object for reporting anything interesting that happens. */
private static Logger logger = LogManager.getLogger(InnerPageOperations.class);
private StorageManager storageManager;
private BTreeTupleFile tupleFile;
private FileOperations fileOps;
public InnerPageOperations(StorageManager storageManager,
BTreeTupleFile tupleFile,
FileOperations fileOps) {
this.storageManager = storageManager;
this.tupleFile = tupleFile;
this.fileOps = fileOps;
}
public InnerPage loadPage(int pageNo) {
DBFile dbFile = tupleFile.getDBFile();
DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
return new InnerPage(dbPage, tupleFile.getSchema());
}
/**
* This helper function is used to update a key between two existing
* pointers in an inner B<sup>+</sup> tree node. It is an error if the
* specified pair of pointers cannot be found in the node.
*
* @param page the inner page to update the key in
* @param pagePath the path to the page, from the root node
* @param pagePtr1 the pointer P<sub>i</sub> before the key to update
* @param key1 the new value of the key K<sub>i</sub> to store
* @param pagePtr2 the pointer P<sub>i+1</sub> after the key to update
*
* @todo (Donnie) This implementation has a major failing that will occur
* infrequently - if the inner page doesn't have room for the new key
* (e.g. if the page was already almost full, and then the new key is
* larger than the old key) then the inner page needs to be split,
* per usual. Right now it will just throw an exception in this case.
* This is why the {@code pagePath} argument is provided, so that when
* this bug is fixed, the page-path will be available.
*/
public void replaceTuple(InnerPage page, List<Integer> pagePath,
int pagePtr1, Tuple key1, int pagePtr2) {
for (int i = 0; i < page.getNumPointers() - 1; i++) {
if (page.getPointer(i) == pagePtr1 &&
page.getPointer(i + 1) == pagePtr2) {
// Found the pair of pointers! Replace the key-value.
BTreeFilePageTuple oldKey = page.getKey(i);
int oldKeySize = oldKey.getSize();
int newKeySize =
PageTuple.getTupleStorageSize(tupleFile.getSchema(), key1);
if (logger.isDebugEnabled()) {
logger.debug(String.format("Inner page %d: replacing " +
"old key of %d bytes (between pointers %d and %d) " +
"with new key of %d bytes", page.getPageNo(),
oldKeySize, pagePtr1, pagePtr2, newKeySize));
}
if (logger.isTraceEnabled()) {
logger.trace("Contents of inner page " + page.getPageNo() +
" before replacement:\n" + page.toFormattedString());
}
if (page.getFreeSpace() + oldKeySize - newKeySize >= 0) {
// We have room - go ahead and do this.
page.replaceTuple(i, key1);
// Make sure we didn't cause any brain damage...
assert page.getPointer(i) == pagePtr1;
assert page.getPointer(i + 1) == pagePtr2;
}
else {
// We need to make more room in this inner page, either by
// relocating records or by splitting this page. We will
// do this by deleting the old entry, and then adding the
// new entry, so that we can leverage all our good code
// for splitting/relocating/etc.
logger.info(String.format("Not enough space in page %d;" +
" trying to relocate entries / split page.",
page.getPageNo()));
// Delete pagePtr2, and the key to the LEFT of it.
page.deletePointer(pagePtr2, /* removeRightKey */ false);
// Now add new key, and put pagePtr2 back after pagePtr1.
addTuple(page, pagePath, pagePtr1, key1, pagePtr2);
}
if (logger.isTraceEnabled()) {
logger.trace("Contents of inner page " + page.getPageNo() +
" after replacement:\n" + page.toFormattedString());
}
return;
}
}
// If we got here, we have a big problem. Couldn't find the expected
// pair of pointers we were handed.
// Dump the page contents because presumably we want to figure out
// what is going on...
logger.error(String.format(
"Couldn't find pair of pointers %d and %d in inner page %d!",
pagePtr1, pagePtr2, page.getPageNo()));
logger.error("Page contents:\n" + page.toFormattedString());
throw new IllegalStateException(
"Couldn't find sequence of page-pointers [" + pagePtr1 + ", " +
pagePtr2 + "] in non-leaf page " + page.getPageNo());
}
/**
* This helper function determines how many pointers must be relocated from
* one inner page to another, in order to free up the specified number of
* bytes. If it is possible, the number of pointers that must be relocated
* is returned. If it is not possible, the method returns 0.
*
* @param page the inner page to relocate entries from
*
* @param adjPage the adjacent page (predecessor or successor) to relocate
* entries to
*
* @param movingRight pass {@code true} if the sibling is to the right of
* {@code page} (and therefore we are moving entries right), or
* {@code false} if the sibling is to the left of {@code page} (and
* therefore we are moving entries left).
*
* @param bytesRequired the number of bytes that must be freed up in
* {@code page} by the operation
*
* @param parentKeySize the size of the parent key that must also be
* relocated into the adjacent page, and therefore affects how many
* pointers can be transferred
*
* @return the number of pointers that must be relocated to free up the
* required space, or 0 if it is not possible.
*/
private int tryNonLeafRelocateForSpace(InnerPage page, InnerPage adjPage,
boolean movingRight, int bytesRequired, int parentKeySize) {
int numKeys = page.getNumKeys();
int pageBytesFree = page.getFreeSpace();
int adjBytesFree = adjPage.getFreeSpace();
logger.debug(String.format("Trying to relocate records from inner-" +
"page %d (%d bytes free) to adjacent inner-page %d (%d bytes " +
"free), moving %s, to free up %d bytes. Parent key size is %d " +
"bytes.", page.getPageNo(), pageBytesFree, adjPage.getPageNo(),
adjBytesFree, (movingRight ? "right" : "left"), bytesRequired,
parentKeySize));
// The parent key always has to move to the adjacent page, so if that
// won't fit, don't even try.
if (adjBytesFree < parentKeySize) {
logger.debug(String.format("Adjacent page %d has %d bytes free;" +
" not enough to hold %d-byte parent key. Giving up on " +
"relocation.", adjPage.getPageNo(), adjBytesFree, parentKeySize));
return 0;
}
// Since the parent key must always be rotated down into the adjacent
// inner page, we must account for that space.
adjBytesFree -= parentKeySize;
int keyBytesMoved = 0;
int numRelocated = 0;
while (true) {
// Figure out the index of the key we need the size of, based on
// the direction we are moving entries, and how many we have
// already moved. If we are moving entries right, we must start
// with the rightmost entry in page. If we are moving entries
// left, we must start with the leftmost entry in page.
int index;
if (movingRight)
index = numKeys - numRelocated - 1;
else
index = numRelocated;
// Add two bytes to the key size for the page-pointer that follows
int entrySize = page.getKey(index).getSize() + 2;
logger.debug("Entry " + index + " is " + entrySize + " bytes");
// Did we run out of space to move entries before we hit our goal?
if (adjBytesFree < entrySize) {
numRelocated = 0;
break;
}
numRelocated++;
pageBytesFree += entrySize;
adjBytesFree -= entrySize;
// Since we don't yet know which page the new pointer will go into,
// stop when we can put the pointer in either page.
if (pageBytesFree >= bytesRequired &&
adjBytesFree >= bytesRequired) {
break;
}
}
assert numRelocated >= 0;
return numRelocated;
}
/**
* This helper function adds an entry (a key and associated pointer) to
* this inner page, after the page-pointer {@code pagePtr1}.
*
* @param page the inner page to add the entry to
*
* @param pagePath the path of page-numbers to this inner page
*
* @param pagePtr1 the <u>existing</u> page that the new key and next-page
* number will be inserted after
*
* @param key1 the new key-value to insert after the {@code pagePtr1} value
*
* @param pagePtr2 the new page-pointer value to follow the {@code key1}
* value
*/
public void addTuple(InnerPage page, List<Integer> pagePath,
int pagePtr1, Tuple key1, int pagePtr2) {
// The new entry will be the key, plus 2 bytes for the page-pointer.
int newEntrySize =
PageTuple.getTupleStorageSize(tupleFile.getSchema(), key1) + 2;
logger.debug(String.format("Adding new %d-byte entry to inner page %d",
newEntrySize, page.getPageNo()));
if (page.getFreeSpace() < newEntrySize) {
logger.debug("Not enough room in inner page " + page.getPageNo() +
"; trying to relocate entries to make room");
// Try to relocate entries from this inner page to either sibling,
// or if that can't happen, split the inner page into two.
if (!relocatePointersAndAddKey(page, pagePath,
pagePtr1, key1, pagePtr2, newEntrySize)) {
logger.debug("Couldn't relocate enough entries to make room;" +
" splitting page " + page.getPageNo() + " instead");
splitAndAddKey(page, pagePath, pagePtr1, key1, pagePtr2);
}
}
else {
// There is room in the leaf for the new key. Add it there.
page.addEntry(pagePtr1, key1, pagePtr2);
}
}
/**
* This function will delete the specified Key/Pointer pair from the
* passed-in inner page.
*
* @param page the inner page to delete the key/pointer from
*
* @param pagePath the page path to the passed page
*
* @param pagePtr the page-pointer value to identify and remove
*
* @param removeRightKey a flag specifying whether the key to the right
* ({@code true}) or to the left ({@code false}) should be removed
*/
public void deletePointer(InnerPage page, List<Integer> pagePath,
int pagePtr, boolean removeRightKey) {
page.deletePointer(pagePtr, removeRightKey);
if (page.getUsedSpace() >= page.getTotalSpace() / 2) {
// The page is at least half-full. Don't need to redistribute or
// coalesce.
return;
}
else if (pagePath.size() == 1) {
// The page is the root. Don't need to redistribute or coalesce,
// but if the root is now empty, need to shorten the tree depth.
if (page.getNumKeys() == 0) {
logger.debug(String.format("Current root page %d is now " +
"empty, removing.", page.getPageNo()));
// Set the index's new root page.
DBPage dbpHeader =
storageManager.loadDBPage(tupleFile.getDBFile(), 0);
HeaderPage.setRootPageNo(dbpHeader, page.getPointer(0));
// Free up this page in the index.
fileOps.releaseDataPage(page.getDBPage());
}
return;
}
// If we got to this part, we have to redistribute/coalesce stuff :(
// Note: Assumed that at least one of the siblings has the same
// immediate parent. If that is not the case... we're doomed...
// TODO: VERIFY THIS AND THROW AN EXCEPTION IF NOT
int pageNo = page.getPageNo();
int leftPageNo = page.getLeftSibling(pagePath, this);
int rightPageNo = page.getRightSibling(pagePath, this);
if (leftPageNo == -1 && rightPageNo == -1) {
// We should never get to this point, since the earlier test
// should have caught this situation.
throw new IllegalStateException(String.format(
"Inner node %d doesn't have a left or right sibling!",
page.getPageNo()));
}
// Now we know that at least one sibling is present. Load both
// siblings and coalesce/redistribute in the direction that makes
// the most sense...
InnerPage leftSibling = null;
if (leftPageNo != -1)
leftSibling = loadPage(leftPageNo);
InnerPage rightSibling = null;
if (rightPageNo != -1)
rightSibling = loadPage(rightPageNo);
// Relocating or coalescing entries requires updating the parent node.
// Since the current node has a sibling, it must also have a parent.
int parentPageNo = pagePath.get(pagePath.size() - 2);
InnerPage parentPage = loadPage(parentPageNo);
// int numPointers = parentPage.getNumPointers();
int indexInParentPage = parentPage.getIndexOfPointer(page.getPageNo());
// See if we can coalesce the node into its left or right sibling.
// When we do the check, we must not forget that each node contains a
// header, and we need to account for that space as well. This header
// space is included in the getUsedSpace() method, but is excluded by
// the getSpaceUsedByTuples() method.
// TODO: SEE IF WE CAN SIMPLIFY THIS AT ALL...
if (leftSibling != null &&
leftSibling.getUsedSpace() + page.getSpaceUsedByEntries() <
leftSibling.getTotalSpace()) {
// Coalesce the current node into the left sibling.
logger.debug("Delete from inner page " + pageNo +
": coalescing with left sibling page.");
logger.debug(String.format("Before coalesce-left, page has %d " +
"pointers and left sibling has %d pointers.",
page.getNumPointers(), leftSibling.getNumPointers()));
// The affected key in the parent page is to the left of this
// page's index in the parent page, since we are moving entries
// to the left sibling.
Tuple parentKey = parentPage.getKey(indexInParentPage - 1);
// We don't care about the key returned by movePointersLeft(),
// since we are deleting the parent key anyway.
page.movePointersLeft(leftSibling, page.getNumPointers(), parentKey);
logger.debug(String.format("After coalesce-left, page has %d " +
"pointers and left sibling has %d pointers.",
page.getNumPointers(), leftSibling.getNumPointers()));
// Free up the page since it's empty now
fileOps.releaseDataPage(page.getDBPage());
page = null;
List<Integer> parentPagePath = pagePath.subList(0, pagePath.size() - 1);
deletePointer(parentPage, parentPagePath, pageNo,
/* delete right key */ false);
}
else if (rightSibling != null &&
rightSibling.getUsedSpace() + page.getSpaceUsedByEntries() <
rightSibling.getTotalSpace()) {
// Coalesce the current node into the right sibling.
logger.debug("Delete from leaf " + pageNo +
": coalescing with right sibling leaf.");
logger.debug(String.format("Before coalesce-right, page has %d " +
"keys and right sibling has %d pointers.",
page.getNumPointers(), rightSibling.getNumPointers()));
// The affected key in the parent page is to the right of this
// page's index in the parent page, since we are moving entries
// to the right sibling.
Tuple parentKey = parentPage.getKey(indexInParentPage);
// We don't care about the key returned by movePointersRight(),
// since we are deleting the parent key anyway.
page.movePointersRight(rightSibling, page.getNumPointers(), parentKey);
logger.debug(String.format("After coalesce-right, page has %d " +
"entries and right sibling has %d pointers.",
page.getNumPointers(), rightSibling.getNumPointers()));
// Free up the right page since it's empty now
fileOps.releaseDataPage(page.getDBPage());
page = null;
List<Integer> parentPagePath = pagePath.subList(0, pagePath.size() - 1);
deletePointer(parentPage, parentPagePath, pageNo,
/* delete right key */ true);
}
else {
// Can't coalesce the leaf node into either sibling. Redistribute
// entries from left or right sibling into the leaf. The strategy
// is as follows:
// If the node has both left and right siblings, redistribute from
// the fuller sibling. Otherwise, just redistribute from
// whichever sibling we have.
InnerPage adjPage = null;
if (leftSibling != null && rightSibling != null) {
// Both siblings are present. Choose the fuller one to
// relocate from.
if (leftSibling.getUsedSpace() > rightSibling.getUsedSpace())
adjPage = leftSibling;
else
adjPage = rightSibling;
}
else if (leftSibling != null) {
// There is no right sibling. Use the left sibling.
adjPage = leftSibling;
}
else {
// There is no left sibling. Use the right sibling.
adjPage = rightSibling;
}
PageTuple parentKey;
if (adjPage == leftSibling)
parentKey = parentPage.getKey(indexInParentPage - 1);
else // adjPage == right sibling
parentKey = parentPage.getKey(indexInParentPage);
int entriesToMove = tryNonLeafRelocateToFill(page, adjPage,
/* movingRight */ adjPage == leftSibling, parentKey.getSize());
if (entriesToMove == 0) {
// We really tried to satisfy the "minimum size" requirement,
// but we just couldn't. Log it and return.
StringBuilder buf = new StringBuilder();
buf.append(String.format("Couldn't relocate pointers to" +
" satisfy minimum space requirement in leaf-page %d" +
" with %d entries!\n", pageNo, page.getNumPointers()));
if (leftSibling != null) {
buf.append(String.format("\t- Left sibling page %d has " +
"%d pointers\n", leftSibling.getPageNo(),
leftSibling.getNumPointers()));
}
else {
buf.append("\t- No left sibling\n");
}
if (rightSibling != null) {
buf.append(String.format("\t- Right sibling page %d has " +
"%d pointers", rightSibling.getPageNo(),
rightSibling.getNumPointers()));
}
else {
buf.append("\t- No right sibling");
}
logger.warn(buf);
return;
}
logger.debug(String.format("Relocating %d pointers into page " +
"%d from %s sibling page %d", entriesToMove, pageNo,
(adjPage == leftSibling ? "left" : "right"), adjPage.getPageNo()));
if (adjPage == leftSibling) {
adjPage.movePointersRight(page, entriesToMove, parentKey);
parentPage.replaceTuple(indexInParentPage - 1, page.getKey(0));
}
else { // adjPage == right sibling
adjPage.movePointersLeft(page, entriesToMove, parentKey);
parentPage.replaceTuple(indexInParentPage, adjPage.getKey(0));
}
}
}
private boolean relocatePointersAndAddKey(InnerPage page,
List<Integer> pagePath, int pagePtr1, Tuple key1, int pagePtr2,
int newEntrySize) {
int pathSize = pagePath.size();
if (pagePath.get(pathSize - 1) != page.getPageNo()) {
throw new IllegalArgumentException(
"Inner page number doesn't match last page-number in page path");
}
// See if we are able to relocate records either direction to free up
// space for the new key.
if (pathSize == 1) // This node is also the root - no parent.
return false; // There aren't any siblings to relocate to.
int parentPageNo = pagePath.get(pathSize - 2);
InnerPage parentPage = loadPage(parentPageNo);
logger.debug(String.format("Parent of inner-page %d is page %d.",
page.getPageNo(), parentPageNo));
if (logger.isTraceEnabled()) {
logger.trace("Parent page contents:\n" +
parentPage.toFormattedString());
}
int numPointers = parentPage.getNumPointers();
int pagePtrIndex = parentPage.getIndexOfPointer(page.getPageNo());
// Check each sibling in its own code block so that we can constrain
// the scopes of the variables a bit. This keeps us from accidentally
// reusing the "prev" variables in the "next" section.
{
InnerPage prevPage = null;
if (pagePtrIndex - 1 >= 0)
prevPage = loadPage(parentPage.getPointer(pagePtrIndex - 1));
if (prevPage != null) {
// See if we can move some of this inner node's entries to the
// previous node, to free up space.
BTreeFilePageTuple parentKey = parentPage.getKey(pagePtrIndex - 1);
int parentKeySize = parentKey.getSize();
int count = tryNonLeafRelocateForSpace(page, prevPage, false,
newEntrySize, parentKeySize);
if (count > 0) {
// Yes, we can do it!
logger.debug(String.format("Relocating %d entries from " +
"inner-page %d to left-sibling inner-page %d", count,
page.getPageNo(), prevPage.getPageNo()));
logger.debug("Space before relocation: Inner = " +
page.getFreeSpace() + " bytes\t\tSibling = " +
prevPage.getFreeSpace() + " bytes");
TupleLiteral newParentKey =
page.movePointersLeft(prevPage, count, parentKey);
// Even with relocating entries, this could fail.
if (!addEntryToInnerPair(prevPage, page, pagePtr1, key1, pagePtr2))
return false;
logger.debug("New parent-key is " + newParentKey);
pagePath.remove(pathSize - 1);
replaceTuple(parentPage, pagePath, prevPage.getPageNo(),
newParentKey, page.getPageNo());
logger.debug("Space after relocation: Inner = " +
page.getFreeSpace() + " bytes\t\tSibling = " +
prevPage.getFreeSpace() + " bytes");
return true;
}
}
}
{
InnerPage nextPage = null;
if (pagePtrIndex + 1 < numPointers)
nextPage = loadPage(parentPage.getPointer(pagePtrIndex + 1));
if (nextPage != null) {
// See if we can move some of this inner node's entries to the
// previous node, to free up space.
BTreeFilePageTuple parentKey = parentPage.getKey(pagePtrIndex);
int parentKeySize = parentKey.getSize();
int count = tryNonLeafRelocateForSpace(page, nextPage, true,
newEntrySize, parentKeySize);
if (count > 0) {
// Yes, we can do it!
logger.debug(String.format("Relocating %d entries from " +
"inner-page %d to right-sibling inner-page %d", count,
page.getPageNo(), nextPage.getPageNo()));
logger.debug("Space before relocation: Inner = " +
page.getFreeSpace() + " bytes\t\tSibling = " +
nextPage.getFreeSpace() + " bytes");
TupleLiteral newParentKey =
page.movePointersRight(nextPage, count, parentKey);
// Even with relocating entries, this could fail.
if (!addEntryToInnerPair(page, nextPage, pagePtr1, key1, pagePtr2))
return false;
logger.debug("New parent-key is " + newParentKey);
pagePath.remove(pathSize - 1);
replaceTuple(parentPage, pagePath, page.getPageNo(),
newParentKey, nextPage.getPageNo());
logger.debug("Space after relocation: Inner = " +
page.getFreeSpace() + " bytes\t\tSibling = " +
nextPage.getFreeSpace() + " bytes");
return true;
}
}
}
// Couldn't relocate entries to either the previous or next page. We
// must split the leaf into two.
return false;
}
/**
* <p>
* This helper function splits the specified inner page into two pages,
* also updating the parent page in the process, and then inserts the
* specified key and page-pointer into the appropriate inner page. This
* method is used to add a key/pointer to an inner page that doesn't have
* enough space, when it isn't possible to relocate pointers to the left
* or right sibling of the page.
* </p>
* <p>
* When the inner node is split, half of the pointers are put into the new
* sibling, regardless of the size of the keys involved. In other words,
* this method doesn't try to keep the pages half-full based on bytes used.
* </p>
*
* @param page the inner node to split and then add the key/pointer to
*
* @param pagePath the sequence of page-numbers traversed to reach this
* inner node.
*
* @param pagePtr1 the existing page-pointer after which the new key and
* pointer should be inserted
*
* @param key1 the new key to insert into the inner page, immediately after
* the page-pointer value {@code pagePtr1}.
*
* @param pagePtr2 the new page-pointer value to insert after the new key
* value
*/
private void splitAndAddKey(InnerPage page, List<Integer> pagePath,
int pagePtr1, Tuple key1, int pagePtr2) {
int pathSize = pagePath.size();
if (pagePath.get(pathSize - 1) != page.getPageNo()) {
throw new IllegalArgumentException(
"Inner page number doesn't match last page-number in page path");
}
if (logger.isTraceEnabled()) {
logger.trace("Initial contents of inner page " + page.getPageNo() +
":\n" + page.toFormattedString());
}
logger.debug("Splitting inner-page " + page.getPageNo() +
" into two inner pages.");
// Get a new blank page in the index, with the same parent as the
// inner-page we were handed.
DBPage newDBPage = fileOps.getNewDataPage();
InnerPage newPage = InnerPage.init(newDBPage, tupleFile.getSchema());
// Figure out how many values we want to move from the old page to the
// new page.
int numPointers = page.getNumPointers();
if (logger.isDebugEnabled()) {
logger.debug(String.format("Relocating %d pointers from left-page %d" +
" to right-page %d", numPointers, page.getPageNo(), newPage.getPageNo()));
logger.debug(" Old left # of pointers: " + page.getNumPointers());
logger.debug(" Old right # of pointers: " + newPage.getNumPointers());
}
Tuple parentKey = null;
InnerPage parentPage = null;
int parentPageNo = 0;
if (pathSize > 1)
parentPageNo = pagePath.get(pathSize - 2);
if (parentPageNo != 0) {
parentPage = loadPage(parentPageNo);
int parentPtrIndex = parentPage.getIndexOfPointer(page.getPageNo());
if (parentPtrIndex < parentPage.getNumPointers() - 1)
parentKey = parentPage.getKey(parentPtrIndex);
}
Tuple newParentKey =
page.movePointersRight(newPage, numPointers / 2, parentKey);
if (logger.isDebugEnabled()) {
logger.debug(" New parent key: " + newParentKey);
logger.debug(" New left # of pointers: " + page.getNumPointers());
logger.debug(" New right # of pointers: " + newPage.getNumPointers());
}
if (logger.isTraceEnabled()) {
logger.trace("Final contents of inner page " + page.getPageNo() +
":\n" + page.toFormattedString());
logger.trace("Final contents of new inner page " +
newPage.getPageNo() + ":\n" + newPage.toFormattedString());
}
if (!addEntryToInnerPair(page, newPage, pagePtr1, key1, pagePtr2)) {
// This is unexpected, but we had better report it if it happens.
throw new IllegalStateException("UNEXPECTED: Couldn't add " +
"entry to half-full inner page!");
}
// If the current node doesn't have a parent, it's because it's
// currently the root.
if (parentPageNo == 0) {
// Create a new root node and set both leaves to have it as their
// parent.
DBPage dbpParent = fileOps.getNewDataPage();
parentPage = InnerPage.init(dbpParent, tupleFile.getSchema(),
page.getPageNo(), newParentKey, newPage.getPageNo());
parentPageNo = parentPage.getPageNo();
// We have a new root-page in the index!
DBFile dbFile = tupleFile.getDBFile();
DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
HeaderPage.setRootPageNo(dbpHeader, parentPageNo);
logger.debug("Set index root-page to inner-page " + parentPageNo);
}
else {
// Add the new page into the parent non-leaf node. (This may cause
// the parent node's contents to be moved or split, if the parent
// is full.)
// (We already set the new node's parent-page-number earlier.)
pagePath.remove(pathSize - 1);
addTuple(parentPage, pagePath, page.getPageNo(), newParentKey,
newPage.getPageNo());
logger.debug("Parent page " + parentPageNo + " now has " +
parentPage.getNumPointers() + " page-pointers.");
}
if (logger.isTraceEnabled()) {
logger.trace("Parent page contents:\n" +
parentPage.toFormattedString());
}
}
/**
* This helper method takes a pair of inner nodes that are siblings to each
* other, and adds the specified key to whichever node the key should go
* into.
*
* @param prevPage the first page in the pair, left sibling of
* {@code nextPage}
*
* @param nextPage the second page in the pair, right sibling of
* {@code prevPage}
*
* @param pageNo1 the pointer to the left of the new key/pointer values that
* will be added to one of the pages
*
* @param key1 the new key-value to insert immediately after the existing
* {@code pageNo1} value
*
* @param pageNo2 the new pointer-value to insert immediately after the new
* {@code key1} value
*
* @return true if the entry was able to be added to one of the pages, or
* false if the entry couldn't be added.
*/
private boolean addEntryToInnerPair(InnerPage prevPage, InnerPage nextPage,
int pageNo1, Tuple key1, int pageNo2) {
InnerPage page;
// See if pageNo1 appears in the left page.
int ptrIndex1 = prevPage.getIndexOfPointer(pageNo1);
if (ptrIndex1 != -1) {
page = prevPage;
}
else {
// The pointer *should be* in the next page. Verify this...
page = nextPage;
if (nextPage.getIndexOfPointer(pageNo1) == -1) {
throw new IllegalStateException(String.format(
"Somehow lost page-pointer %d from inner pages %d and %d",
pageNo1, prevPage.getPageNo(), nextPage.getPageNo()));
}
}
int entrySize = 2 +
PageTuple.getTupleStorageSize(tupleFile.getSchema(), key1);
if (page.getFreeSpace() >= entrySize) {
page.addEntry(pageNo1, key1, pageNo2);
return true;
}
else {
return false;
}
}
/**
* This helper function determines how many entries must be relocated from
* one leaf-page to another, in order to satisfy the "minimum space"
* requirement of the B tree.free up the specified number of
* bytes. If it is possible, the number of entries that must be relocated
* is returned. If it is not possible, the method returns 0.
*
* @param page the inner node to relocate entries from
*
* @param adjPage the adjacent inner page (predecessor or successor) to
* relocate entries to
*
* @param movingRight pass {@code true} if the sibling is to the left of
* {@code page} (and therefore we are moving entries right), or
* {@code false} if the sibling is to the right of {@code page}
* (and therefore we are moving entries left).
*
* @return the number of entries that must be relocated to fill the node
* to a minimal level, or 0 if not possible.
*/
private int tryNonLeafRelocateToFill(InnerPage page, InnerPage adjPage,
boolean movingRight, int parentKeySize) {
int adjKeys = adjPage.getNumKeys();
int pageBytesFree = page.getFreeSpace();
int adjBytesFree = adjPage.getFreeSpace();
// Should be the same for both page and adjPage.
int halfFull = page.getTotalSpace() / 2;
// The parent key always has to move into this page, so if that
// won't fit, don't even try.
if (pageBytesFree < parentKeySize)
return 0;
pageBytesFree -= parentKeySize;
int keyBytesMoved = 0;
int lastKeySize = parentKeySize;
int numRelocated = 0;
while (true) {
// If the key we wanted to move into this page overflows the free
// space in this page, back it up.
// TODO: IS THIS NECESSARY?
if (pageBytesFree < keyBytesMoved + 2 * numRelocated) {
numRelocated--;
break;
}
// Figure out the index of the key we need the size of, based on the
// direction we are moving values. If we are moving values right,
// we need to look at the keys starting at the rightmost one. If we
// are moving values left, we need to start with the leftmost key.
int index;
if (movingRight)
index = adjKeys - numRelocated - 1;
else
index = numRelocated;
keyBytesMoved += lastKeySize;
lastKeySize = adjPage.getKey(index).getSize();
logger.debug("Key " + index + " is " + lastKeySize + " bytes");
numRelocated++;
// Since we don't yet know which page the new pointer will go into,
// stop when we can put the pointer in either page.
if (adjBytesFree <= halfFull &&
(pageBytesFree + keyBytesMoved + 2 * numRelocated) <= halfFull) {
break;
}
}
logger.debug("Can relocate " + numRelocated +
" keys to satisfy minimum space requirements.");
assert numRelocated >= 0;
return numRelocated;
}
}
package edu.caltech.nanodb.storage.btreefile;
import java.util.ArrayList;
import java.util.List;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import edu.caltech.nanodb.expressions.TupleComparator;
import edu.caltech.nanodb.expressions.TupleLiteral;
import edu.caltech.nanodb.relations.Schema;
import edu.caltech.nanodb.relations.Tuple;
import edu.caltech.nanodb.storage.DBPage;
import edu.caltech.nanodb.storage.PageTuple;
import static edu.caltech.nanodb.storage.btreefile.BTreePageTypes.*;
/**
* <p>
* This class wraps a {@link DBPage} object that is a leaf page in the
* B<sup>+</sup> tree file implementation, to provide some of the basic
* leaf-management operations necessary for the file structure.
* </p>
* <p>
* Operations involving individual inner-pages are provided by the
* {@link InnerPage} wrapper-class. Higher-level operations involving
* multiple leaves and/or inner pages of the B<sup>+</sup> tree structure,
* are provided by the {@link LeafPageOperations} and
* {@link InnerPageOperations} classes.
* </p>
*/
public class LeafPage implements DataPage {
/** A logging object for reporting anything interesting that happens. */
private static Logger logger = LogManager.getLogger(LeafPage.class);
/**
* The offset where the next-sibling page number is stored in this page.
* The only leaf page that doesn't have a next sibling is the last leaf
* in the file; its "next page" value will be set to 0.
*/
public static final int OFFSET_NEXT_PAGE_NO = 1;
/**
* The offset where the number of tuples is stored in the page.
*/
public static final int OFFSET_NUM_TUPLES = 3;
/** The offset of the first tuple in the leaf page. */
public static final int OFFSET_FIRST_TUPLE = 5;
/** The actual data page that holds the B<sup>+</sup> tree leaf node. */
private DBPage dbPage;
/** The schema of the tuples in the leaf page. */
private Schema schema;
/** The number of tuples stored within this leaf page. */
private int numTuples;
/** A list of the tuples stored in this leaf page. */
private ArrayList<BTreeFilePageTuple> tuples;
/**
* The total size of all data (tuples + initial values) stored within this
* leaf page. This is also the offset at which we can start writing more
* data without overwriting anything.
*/
private int endOffset;
/**
* Initialize the leaf-page wrapper class for the specified B<sup>+</sup>
* tree leaf page. The contents of the leaf-page are cached in the fields
* of the wrapper object.
*
* @param dbPage the data page from the B<sup>+</sup> Tree file to wrap
* @param schema the schema of tuples stored in the data page
*/
public LeafPage(DBPage dbPage, Schema schema) {
if (dbPage.readUnsignedByte(0) != BTREE_LEAF_PAGE) {
throw new IllegalArgumentException("Specified DBPage " +
dbPage.getPageNo() + " is not marked as a leaf page.");
}
this.dbPage = dbPage;
this.schema = schema;
loadPageContents();
}
/**
* This static helper function initializes a {@link DBPage} object's
* contents with the type and detail values that will allow a new
* {@code LeafPage} wrapper to be instantiated for the page, and then it
* returns a wrapper object for the page.
*
* @param dbPage the page to initialize as a leaf page.
*
* @param schema the schema of the tuples in the leaf page
*
* @return a newly initialized {@code LeafPage} object wrapping the page
*/
public static LeafPage init(DBPage dbPage, Schema schema) {
dbPage.writeByte(OFFSET_PAGE_TYPE, BTREE_LEAF_PAGE);
dbPage.writeShort(OFFSET_NUM_TUPLES, 0);
dbPage.writeShort(OFFSET_NEXT_PAGE_NO, 0);
return new LeafPage(dbPage, schema);
}
/**
* This private helper scans through the leaf page's contents and caches
* the contents of the leaf page in a way that makes it easy to use and
* manipulate.
*/
private void loadPageContents() {
numTuples = dbPage.readUnsignedShort(OFFSET_NUM_TUPLES);
tuples = new ArrayList<>(numTuples);
if (numTuples > 0) {
// Handle first tuple separately since we know its offset.
BTreeFilePageTuple tuple =
new BTreeFilePageTuple(schema, dbPage, OFFSET_FIRST_TUPLE, 0);
tuples.add(tuple);
// Handle remaining tuples.
for (int i = 1; i < numTuples; i++) {
int tupleEndOffset = tuple.getEndOffset();
tuple = new BTreeFilePageTuple(schema, dbPage, tupleEndOffset, i);
tuples.add(tuple);
}
endOffset = tuple.getEndOffset();
}
else {
// There are no tuples in the leaf page.
endOffset = OFFSET_FIRST_TUPLE;
}
}
/**
* Returns the schema of tuples in this page.
*
* @return the schema of tuples in this page
*/
public Schema getSchema() {
return schema;
}
/**
* Returns the {@code DBPage} that backs this leaf page.
*
* @return the {@code DBPage} that backs this leaf page.
*/
public DBPage getDBPage() {
return dbPage;
}
/**
* Returns the page-number of this leaf page.
*
* @return the page-number of this leaf page.
*/
public int getPageNo() {
return dbPage.getPageNo();
}
/**
* Returns the page-number of the next leaf page in the sequence of leaf
* pages, or 0 if this is the last leaf-page in the B<sup>+</sup> tree
* file.
*
* @return the page-number of the next leaf page in the sequence of leaf
* pages, or 0 if this is the last leaf-page in the B<sup>+</sup>
* tree file.
*/
public int getNextPageNo() {
return dbPage.readUnsignedShort(OFFSET_NEXT_PAGE_NO);
}
/**
* Sets the page-number of the next leaf page in the sequence of leaf pages.
*
* @param pageNo the page-number of the next leaf-page in the index, or 0
* if this is the last leaf-page in the B<sup>+</sup> tree file.
*/
public void setNextPageNo(int pageNo) {
if (pageNo < 0) {
throw new IllegalArgumentException(
"pageNo must be in range [0, 65535]; got " + pageNo);
}
dbPage.writeShort(OFFSET_NEXT_PAGE_NO, pageNo);
}
/**
* Returns the number of tuples in this leaf-page. Note that this count
* does not include the pointer to the next leaf; it only includes the
* tuples themselves.
*
* @return the number of entries in this leaf-page.
*/
public int getNumTuples() {
return numTuples;
}
/**
* Returns the amount of space currently used in this leaf page, in bytes.
*
* @return the amount of space currently used in this leaf page, in bytes.
*/
public int getUsedSpace() {
return endOffset;
}
/**
* Returns the amount of space used by tuples in this page, in bytes.
*
* @return the amount of space used by tuples in this page, in bytes.
*/
public int getSpaceUsedByTuples() {
return endOffset - OFFSET_FIRST_TUPLE;
}
/**
* Returns the amount of space available in this leaf page, in bytes.
*
* @return the amount of space available in this leaf page, in bytes.
*/
public int getFreeSpace() {
return dbPage.getPageSize() - endOffset;
}
/**
* Returns the total space (page size) in bytes.
*
* @return the size of the page, in bytes.
*/
public int getTotalSpace() {
return dbPage.getPageSize();
}
/**
* Returns the tuple at the specified index.
*
* @param index the index of the tuple to retrieve
*
* @return the tuple at that index
*/
public BTreeFilePageTuple getTuple(int index) {
return tuples.get(index);
}
/**
* Returns the size of the tuple at the specified index, in bytes.
*
* @param index the index of the tuple to get the size of
*
* @return the size of the specified tuple, in bytes
*/
public int getTupleSize(int index) {
BTreeFilePageTuple tuple = getTuple(index);
return tuple.getEndOffset() - tuple.getOffset();
}
/**
* Given a leaf page in the B<sup>+</sup> tree file, returns the page
* number of the left sibling, or -1 if there is no left sibling to this
* node.
*
* @param pagePath the page path from root to this leaf page
* @param innerOps the inner page ops that allows this method to
* load inner pages and navigate the tree
*
* @return the page number of the left sibling leaf-node, or -1 if there
* is no left sibling
*
* @review (Donnie) There is a lot of implementation-overlap between this
* function and the {@link InnerPage#getLeftSibling}. Maybe find
* a way to combine the implementations.
*/
public int getLeftSibling(List<Integer> pagePath,
InnerPageOperations innerOps) {
// Verify that the last node in the page path is in fact this page.
if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
throw new IllegalArgumentException(
"The page path provided does not terminate on this leaf page.");
}
// If this leaf doesn't have a parent, we already know it doesn't
// have a sibling.
if (pagePath.size() <= 1)
return -1;
int parentPageNo = pagePath.get(pagePath.size() - 2);
InnerPage inner = innerOps.loadPage(parentPageNo);
// Get the index of the pointer that points to this page. If it
// doesn't appear in the parent, we have a serious problem...
int pageIndex = inner.getIndexOfPointer(getPageNo());
if (pageIndex == -1) {
throw new IllegalStateException(String.format(
"Leaf node %d doesn't appear in parent inner node %d!",
getPageNo(), parentPageNo));
}
int leftSiblingIndex = pageIndex - 1;
int leftSiblingPageNo = -1;
if (leftSiblingIndex >= 0)
leftSiblingPageNo = inner.getPointer(leftSiblingIndex);
return leftSiblingPageNo;
}
/**
* Given a leaf page in the B<sup>+</sup> tree file, returns the page
* number of the right sibling, or -1 if there is no right sibling to
* this node.
*
* @param pagePath the page path from root to this leaf page
*
* @return the page number of the right sibling leaf-node, or -1 if there
* is no right sibling
*/
public int getRightSibling(List<Integer> pagePath) {
// Verify that the last node in the page path is in fact this page.
if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
throw new IllegalArgumentException(
"The page path provided does not terminate on this leaf page.");
}
int rightSiblingPageNo = getNextPageNo();
if (rightSiblingPageNo == 0)
rightSiblingPageNo = -1;
return rightSiblingPageNo;
}
/**
* Returns the index of the specified tuple.
*
* @param tuple the tuple to retrieve the index for
*
* @return the integer index of the specified tuple, -1 if the tuple
* isn't in the page.
*/
public int getTupleIndex(Tuple tuple) {
int i;
for (i = 0; i < numTuples; i++) {
BTreeFilePageTuple pageTuple = tuples.get(i);
/* This gets REALLY verbose... */
logger.trace(i + ": comparing " + tuple + " to " + pageTuple);
// Is this the key we're looking for?
if (TupleComparator.comparePartialTuples(tuple, pageTuple) == 0) {
logger.debug(String.format("Found tuple: %s is equal to " +
"%s at index %d (size = %d bytes)", tuple, pageTuple, i,
pageTuple.getSize()));
return i;
}
}
return -1;
}
/**
* This method will delete a tuple from the leaf page. The method takes
* care of 'sliding' the remaining data to cover up the gap left. The
* method throws an exception if the specified tuple does not appear in
* the leaf page.
*
* @param tuple the tuple to delete from the leaf page
*
* @throws IllegalStateException if the specified tuple doesn't exist
*/
public void deleteTuple(Tuple tuple) {
logger.debug("Trying to delete tuple " + tuple + " from leaf page " +
getPageNo());
int index = getTupleIndex(tuple);
if (index == -1) {
throw new IllegalArgumentException("Specified tuple " + tuple +
" does not appear in leaf page " + getPageNo());
}
int tupleOffset = getTuple(index).getOffset();
int len = getTupleSize(index);
logger.debug("Moving leaf-page data in range [" + (tupleOffset+len) +
", " + endOffset + ") over by " + len + " bytes");
dbPage.moveDataRange(tupleOffset + len, tupleOffset,
endOffset - tupleOffset - len);
// Decrement the total number of entries.
dbPage.writeShort(OFFSET_NUM_TUPLES, numTuples - 1);
logger.debug("Loading altered page - had " + numTuples +
" tuples before delete.");
// Load new page.
loadPageContents();
logger.debug("After loading, have " + numTuples + " tuples");
if (tuple instanceof BTreeFilePageTuple) {
BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
btpt.setDeleted();
if (index < numTuples)
btpt.setNextTuplePosition(dbPage.getPageNo(), index);
else
btpt.setNextTuplePosition(getNextPageNo(), 0);
}
}
/**
* This method inserts a tuple into the leaf page, making sure to keep
* tuples in monotonically increasing order. This method will throw an
* exception if the leaf page already contains the specified tuple.
*
* @param newTuple the new tuple to add to the leaf page
*
* @throws IllegalStateException if the specified tuple already appears in
* the leaf page.
*/
public BTreeFilePageTuple addTuple(TupleLiteral newTuple) {
if (newTuple.getStorageSize() == -1) {
throw new IllegalArgumentException("New tuple's storage size " +
"must be computed before this method is called.");
}
if (getFreeSpace() < newTuple.getStorageSize()) {
throw new IllegalArgumentException(String.format(
"Not enough space in this node to store the new tuple " +
"(%d bytes free; %d bytes required)", getFreeSpace(),
newTuple.getStorageSize()));
}
BTreeFilePageTuple result = null;
if (numTuples == 0) {
logger.debug("Leaf page is empty; storing new tuple at start.");
result = addTupleAtIndex(newTuple, 0);
}
else {
int i;
for (i = 0; i < numTuples; i++) {
BTreeFilePageTuple tuple = tuples.get(i);
/* This gets REALLY verbose... */
logger.trace(i + ": comparing " + newTuple + " to " + tuple);
// Compare the new tuple to the current tuple. Once we find
// where the new tuple should go, copy the tuple into the page.
int cmp = TupleComparator.compareTuples(newTuple, tuple);
if (cmp < 0) {
logger.debug("Storing new tuple at index " + i +
" in the leaf page.");
result = addTupleAtIndex(newTuple, i);
break;
}
else if (cmp == 0) {
// TODO: Currently we require all tuples to be unique,
// but this isn't a realistic long-term constraint.
throw new IllegalStateException("Tuple " + newTuple +
" already appears in the index!");
}
}
if (i == numTuples) {
// The new tuple will go at the end of this page's entries.
logger.debug("Storing new tuple at end of leaf page.");
result = addTupleAtIndex(newTuple, numTuples);
}
}
// The addTupleAtIndex() method updates the internal fields that cache
// where keys live, etc. So, we don't need to do that here.
assert result != null; // Shouldn't be possible at this point...
return result;
}
/**
* This private helper takes care of inserting a tuple at a specific index
* in the leaf page. This method should be called with care, so as to
* ensure that tuples always remain in monotonically increasing order.
*
* @param newTuple the new tuple to insert into the leaf page
* @param index the index to insert the tuple at. Any existing tuples at
* or after this index will be shifted over to make room for the
* new tuple.
*/
private BTreeFilePageTuple addTupleAtIndex(TupleLiteral newTuple,
int index) {
logger.debug("Leaf-page is starting with data ending at index " +
endOffset + ", and has " + numTuples + " tuples.");
// Get the storage size of the new tuple.
int len = newTuple.getStorageSize();
if (len == -1) {
throw new IllegalArgumentException("New tuple's storage size " +
"must be computed before this method is called.");
}
logger.debug("New tuple's storage size is " + len + " bytes");
int tupleOffset;
if (index < numTuples) {
// Need to slide tuples after this index over, to make space.
BTreeFilePageTuple tuple = getTuple(index);
// Make space for the new tuple to be stored, then copy in
// the new values.
tupleOffset = tuple.getOffset();
logger.debug("Moving leaf-page data in range [" + tupleOffset +
", " + endOffset + ") over by " + len + " bytes");
dbPage.moveDataRange(tupleOffset, tupleOffset + len,
endOffset - tupleOffset);
}
else {
// The new tuple falls at the end of the data in the leaf index
// page.
tupleOffset = endOffset;
logger.debug("New tuple is at end of leaf-page data; not " +
"moving anything.");
}
// Write the tuple value into the page.
PageTuple.storeTuple(dbPage, tupleOffset, schema, newTuple);
// Increment the total number of tuples.
dbPage.writeShort(OFFSET_NUM_TUPLES, numTuples + 1);
// Reload the page contents now that we have a new tuple in the mix.
// TODO: We could do this more efficiently, but this should be
// sufficient for now.
loadPageContents();
logger.debug("Wrote new tuple to leaf-page at offset " + tupleOffset +
".");
logger.debug("Leaf-page is ending with data ending at index " +
endOffset + ", and has " + numTuples + " tuples.");
// Return the actual tuple we just added to the page.
return getTuple(index);
}
/**
* This helper function moves the specified number of tuples to the left
* sibling of this leaf node. The data is copied in one shot so that the
* transfer will be fast, and the various associated bookkeeping values in
* both leaves are updated.
*
* @param leftSibling the left sibling of this leaf-node in the
* B<sup>+</sup> tree file
*
* @param count the number of tuples to move to the left sibling
*/
public void moveTuplesLeft(LeafPage leftSibling, int count) {
if (leftSibling == null)
throw new IllegalArgumentException("leftSibling cannot be null");
if (leftSibling.getNextPageNo() != getPageNo()) {
logger.error(String.format("Left sibling leaf %d says that " +
"page %d is its right sibling, not this page %d",
leftSibling.getPageNo(), leftSibling.getNextPageNo(),
getPageNo()));
throw new IllegalArgumentException("leftSibling " +
leftSibling.getPageNo() + " isn't actually the left " +
"sibling of this leaf-node " + getPageNo());
}
if (count < 0 || count > numTuples) {
throw new IllegalArgumentException("count must be in range [0, " +
numTuples + "), got " + count);
}
int moveEndOffset = getTuple(count - 1).getEndOffset(); //getTuple(count).getOffset()
int len = moveEndOffset - OFFSET_FIRST_TUPLE;
// Copy the range of tuple-data to the destination page. Then update
// the count of tuples in the destination page.
// Don't need to move any data in the left sibling; we are appending!
leftSibling.dbPage.write(leftSibling.endOffset, dbPage.getPageData(),
OFFSET_FIRST_TUPLE, len); // Copy the tuple-data across
leftSibling.dbPage.writeShort(OFFSET_NUM_TUPLES,
leftSibling.numTuples + count); // Update the tuple-count
// Remove that range of tuple-data from this page.
dbPage.moveDataRange(moveEndOffset, OFFSET_FIRST_TUPLE,
endOffset - moveEndOffset);
dbPage.writeShort(OFFSET_NUM_TUPLES, numTuples - count);
// Only erase the old data in the leaf page if we are trying to make
// sure everything works properly.
if (BTreeTupleFile.CLEAR_OLD_DATA)
dbPage.setDataRange(endOffset - len, len, (byte) 0);
// Update the cached info for both leaves.
loadPageContents();
leftSibling.loadPageContents();
}
/**
* This helper function moves the specified number of tuples to the right
* sibling of this leaf node. The data is copied in one shot so that the
* transfer will be fast, and the various associated bookkeeping values in
* both leaves are updated.
*
* @param rightSibling the right sibling of this leaf-node in the index
* file
*
* @param count the number of tuples to move to the right sibling
*/
public void moveTuplesRight(LeafPage rightSibling, int count) {
if (rightSibling == null)
throw new IllegalArgumentException("rightSibling cannot be null");
if (getNextPageNo() != rightSibling.getPageNo()) {
throw new IllegalArgumentException("rightSibling " +
rightSibling.getPageNo() + " isn't actually the right " +
"sibling of this leaf-node " + getPageNo());
}
if (count < 0 || count > numTuples) {
throw new IllegalArgumentException("count must be in range [0, " +
numTuples + "), got " + count);
}
int startOffset = getTuple(numTuples - count).getOffset();
int len = endOffset - startOffset;
// Copy the range of tuple-data to the destination page. Then update
// the count of tuples in the destination page.
// Make room for the data
rightSibling.dbPage.moveDataRange(OFFSET_FIRST_TUPLE,
OFFSET_FIRST_TUPLE + len,
rightSibling.endOffset - OFFSET_FIRST_TUPLE);
// Copy the tuple-data across
rightSibling.dbPage.write(OFFSET_FIRST_TUPLE, dbPage.getPageData(),
startOffset, len);
// Update the tuple-count
rightSibling.dbPage.writeShort(OFFSET_NUM_TUPLES,
rightSibling.numTuples + count);
// Remove that range of tuple-data from this page.
dbPage.writeShort(OFFSET_NUM_TUPLES, numTuples - count);
// Only erase the old data in the leaf page if we are trying to make
// sure everything works properly.
if (BTreeTupleFile.CLEAR_OLD_DATA)
dbPage.setDataRange(startOffset, len, (byte) 0);
// Update the cached info for both leaves.
loadPageContents();
rightSibling.loadPageContents();
}
}
package edu.caltech.nanodb.storage.btreefile;
import java.util.List;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import edu.caltech.nanodb.expressions.TupleComparator;
import edu.caltech.nanodb.expressions.TupleLiteral;
import edu.caltech.nanodb.relations.Tuple;
import edu.caltech.nanodb.storage.DBFile;
import edu.caltech.nanodb.storage.DBPage;
import edu.caltech.nanodb.storage.StorageManager;
/**
* This class provides high-level B<sup>+</sup> tree management operations
* performed on leaf nodes. These operations are provided here and not on the
* {@link LeafPage} class since they sometimes involve splitting or merging
* leaf nodes, updating parent nodes, and so forth.
*/
public class LeafPageOperations {
/** A logging object for reporting anything interesting that happens. */
private static Logger logger = LogManager.getLogger(LeafPageOperations.class);
private StorageManager storageManager;
private BTreeTupleFile tupleFile;
private FileOperations fileOps;
private InnerPageOperations innerPageOps;
public LeafPageOperations(StorageManager storageManager,
BTreeTupleFile tupleFile,
FileOperations fileOps,
InnerPageOperations innerPageOps) {
this.storageManager = storageManager;
this.tupleFile = tupleFile;
this.fileOps = fileOps;
this.innerPageOps = innerPageOps;
}
/**
* This helper function provides the simple operation of loading a leaf page
* from its page-number, or if the page-number is 0 then {@code null} is
* returned.
*
* @param pageNo the page-number to load as a leaf-page.
*
* @return a newly initialized {@link LeafPage} instance if {@code pageNo}
* is positive, or {@code null} if {@code pageNo} is 0.
*
* @throws IllegalArgumentException if the specified page isn't a leaf-page
*/
private LeafPage loadLeafPage(int pageNo) {
if (pageNo == 0)
return null;
DBFile dbFile = tupleFile.getDBFile();
DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
return new LeafPage(dbPage, tupleFile.getSchema());
}
/**
* This helper function will handle deleting a tuple in the index.
*
* @param leaf the leaf page to delete the tuple from
*
* @param tuple the tuple to delete from the leaf page
*
* @param pagePath the path of pages taken from the root page to the leaf
* page, represented as a list of page numbers
*/
public void deleteTuple(LeafPage leaf, Tuple tuple,
List<Integer> pagePath) {
logger.debug(String.format("Deleting tuple %s from leaf page %d at " +
"page-path %s", tuple, leaf.getPageNo(), pagePath));
leaf.deleteTuple(tuple);
if (leaf.getUsedSpace() >= leaf.getTotalSpace() / 2) {
// The page is at least half-full. Don't need to redistribute or
// coalesce.
return;
}
else if (pagePath.size() == 1) {
// The page is the root. Don't need to redistribute or coalesce,
// but if the root is now empty, need to shorten the tree depth.
// (Since this is a leaf page, the depth will go down to 0.)
if (leaf.getNumTuples() == 0) {
logger.debug(String.format("Current root page %d is now " +
"empty, removing.", leaf.getPageNo()));
// Set the index's root page to 0 (empty) since the only page
// in the index is now empty and being removed.
DBPage dbpHeader =
storageManager.loadDBPage(tupleFile.getDBFile(), 0);
HeaderPage.setRootPageNo(dbpHeader, 0);
// Free up this page in the index.
fileOps.releaseDataPage(leaf.getDBPage());
if (tuple instanceof BTreeFilePageTuple) {
// Some sanity checks - if the btree is now empty,
// the "next tuple" info should be all 0.
BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
assert btpt.getNextTuplePageNo() == 0;
assert btpt.getNextTupleIndex() == 0;
}
}
return;
}
// If we got to this part, we have to redistribute/coalesce stuff :(
// Note: Assumed that at least one of the siblings has the same
// immediate parent. If that is not the case... we're doomed...
// TODO: VERIFY THIS AND THROW AN EXCEPTION IF NOT
int leafPageNo = leaf.getPageNo();
// Leaf pages know their right sibling, so that's why finding the
// right page doesn't require the innerPageOps object.
int leftPageNo = leaf.getLeftSibling(pagePath, innerPageOps);
int rightPageNo = leaf.getRightSibling(pagePath);
logger.debug(String.format("Leaf page %d is too empty. Left " +
"sibling is %d, right sibling is %d.", leafPageNo, leftPageNo,
rightPageNo));
if (leftPageNo == -1 && rightPageNo == -1) {
// We should never get to this point, since the earlier test
// should have caught this situation.
throw new IllegalStateException(String.format(
"Leaf node %d doesn't have a left or right sibling!",
leaf.getPageNo()));
}
// Now we know that at least one sibling is present. Load both
// siblings and coalesce/redistribute in the direction that makes
// the most sense...
LeafPage leftSibling = null;
if (leftPageNo != -1)
leftSibling = loadLeafPage(leftPageNo);
LeafPage rightSibling = null;
if (rightPageNo != -1)
rightSibling = loadLeafPage(rightPageNo);
assert leftSibling != null || rightSibling != null;
// See if we can coalesce the node into its left or right sibling.
// When we do the check, we must not forget that each node contains a
// header, and we need to account for that space as well. This header
// space is included in the getUsedSpace() method, but is excluded by
// the getSpaceUsedByTuples() method.
// TODO: SEE IF WE CAN SIMPLIFY THIS AT ALL...
if (leftSibling != null &&
leftSibling.getUsedSpace() + leaf.getSpaceUsedByTuples() <
leftSibling.getTotalSpace()) {
// Coalesce the current node into the left sibling.
logger.debug("Delete from leaf " + leaf.getPageNo() +
": coalescing with left sibling leaf.");
logger.debug(String.format("Before coalesce-left, page has %d " +
"tuples and left sibling has %d tuples.",
leaf.getNumTuples(), leftSibling.getNumTuples()));
if (tuple instanceof BTreeFilePageTuple) {
// The "next tuple" will end up in the left sibling, so we
// need to update this info in the deleted tuple.
BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
int index = btpt.getNextTupleIndex();
index += leftSibling.getNumTuples();
btpt.setNextTuplePosition(leftPageNo, index);
}
leaf.moveTuplesLeft(leftSibling, leaf.getNumTuples());
leftSibling.setNextPageNo(leaf.getNextPageNo());
logger.debug(String.format("After coalesce-left, page has %d " +
"tuples and left sibling has %d tuples.",
leaf.getNumTuples(), leftSibling.getNumTuples()));
// Free up the leaf page since it's empty now
fileOps.releaseDataPage(leaf.getDBPage());
// Since the leaf page has been removed from the index structure,
// we need to remove it from the parent page. Also, since the
// page was coalesced into its left sibling, we need to remove
// the tuple to the left of the pointer being removed.
InnerPage parent =
innerPageOps.loadPage(pagePath.get(pagePath.size() - 2));
List<Integer> parentPagePath = pagePath.subList(0, pagePath.size() - 1);
innerPageOps.deletePointer(parent, parentPagePath, leafPageNo,
/* remove right tuple */ false);
}
else if (rightSibling != null &&
rightSibling.getUsedSpace() + leaf.getSpaceUsedByTuples() <
rightSibling.getTotalSpace()) {
// Coalesce the current node into the right sibling.
logger.debug("Delete from leaf " + leaf.getPageNo() +
": coalescing with right sibling leaf.");
logger.debug(String.format("Before coalesce-right, page has %d " +
"tuples and right sibling has %d tuples.",
leaf.getNumTuples(), rightSibling.getNumTuples()));
if (tuple instanceof BTreeFilePageTuple) {
// The "next tuple" will end up in the right sibling, so we
// need to update this info in the deleted tuple. We don't
// update the index, because this page's tuples will precede
// the right sibling's tuples.
BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
int index = btpt.getNextTupleIndex();
btpt.setNextTuplePosition(rightPageNo, index);
}
leaf.moveTuplesRight(rightSibling, leaf.getNumTuples());
// Left sibling can be null if leaf is the first leaf node in the
// sequence of the btree.
if (leftSibling != null)
leftSibling.setNextPageNo(rightPageNo);
logger.debug(String.format("After coalesce-right, page has %d " +
"tuples and right sibling has %d tuples.",
leaf.getNumTuples(), rightSibling.getNumTuples()));
// Free up the leaf page since it's empty now
fileOps.releaseDataPage(leaf.getDBPage());
// Since the leaf page has been removed from the index structure,
// we need to remove it from the parent page. Also, since the
// page was coalesced into its right sibling, we need to remove
// the tuple to the right of the pointer being removed.
InnerPage parent =
innerPageOps.loadPage(pagePath.get(pagePath.size() - 2));
List<Integer> parentPagePath = pagePath.subList(0, pagePath.size() - 1);
innerPageOps.deletePointer(parent, parentPagePath, leafPageNo,
/* remove right tuple */ true);
}
else {
// Can't coalesce the leaf node into either sibling. Redistribute
// tuples from left or right sibling into the leaf. The strategy
// is as follows:
// If the node has both left and right siblings, redistribute from
// the fuller sibling. Otherwise, just redistribute from
// whichever sibling we have.
LeafPage adjPage;
if (leftSibling != null && rightSibling != null) {
// Both siblings are present. Choose the fuller one to
// relocate from.
if (leftSibling.getUsedSpace() > rightSibling.getUsedSpace())
adjPage = leftSibling;
else
adjPage = rightSibling;
}
else if (leftSibling != null) {
// There is no right sibling. Use the left sibling.
adjPage = leftSibling;
}
else {
// There is no left sibling. Use the right sibling.
adjPage = rightSibling;
}
int tuplesToMove = tryLeafRelocateToFill(leaf, adjPage,
/* movingRight */ adjPage == leftSibling);
if (tuplesToMove == 0) {
// We really tried to satisfy the "minimum size" requirement,
// but we just couldn't. Log it and return.
StringBuilder buf = new StringBuilder();
buf.append(String.format("Couldn't relocate tuples to satisfy" +
" minimum space requirement in leaf-page %d with %d tuples!\n",
leaf.getPageNo(), leaf.getNumTuples()));
if (leftSibling != null) {
buf.append(
String.format("\t- Left sibling page %d has %d tuples\n",
leftSibling.getPageNo(), leftSibling.getNumTuples()));
}
else {
buf.append("\t- No left sibling\n");
}
if (rightSibling != null) {
buf.append(
String.format("\t- Right sibling page %d has %d tuples",
rightSibling.getPageNo(), rightSibling.getNumTuples()));
}
else {
buf.append("\t- No right sibling");
}
logger.warn(buf);
return;
}
logger.debug(String.format("Relocating %d tuples into leaf page " +
"%d from %s sibling page %d", tuplesToMove, leaf.getPageNo(),
(adjPage == leftSibling ? "left" : "right"), adjPage.getPageNo()));
if (tuple instanceof BTreeFilePageTuple) {
// Since we are moving tuples into this page, we may need to
// modify the "next tuple" info. This is only necessary if
// tuples are moved from the left sibling, since those will
// precede the leaf page's current tuples.
BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
int nextPageNo = btpt.getNextTuplePageNo();
int nextIndex = btpt.getNextTupleIndex();
if (adjPage == leftSibling) {
if (nextPageNo == leafPageNo) {
logger.debug(String.format("Moving %d tuples from left " +
"sibling %d to leaf %d. Deleted tuple has next " +
"tuple at [%d:%d]; updating to [%d:%d]", tuplesToMove,
leftPageNo, leafPageNo, nextPageNo, nextIndex,
nextPageNo, nextIndex + tuplesToMove));
btpt.setNextTuplePosition(nextPageNo, nextIndex + tuplesToMove);
}
else {
assert(nextIndex == 0);
logger.debug(String.format("Moving %d tuples from " +
"left sibling %d to leaf %d. Deleted tuple " +
"has next tuple at [%d:%d]; not updating",
tuplesToMove, leftPageNo, leafPageNo,
nextPageNo, nextIndex));
}
}
else {
assert adjPage == rightSibling;
if (nextPageNo == rightPageNo) {
assert(nextIndex == 0);
logger.debug(String.format("Moving %d tuples from " +
"right sibling %d to leaf %d. Deleted tuple " +
"has next tuple at [%d:%d]; not updating",
tuplesToMove, rightPageNo, leafPageNo,
nextPageNo, nextIndex));
btpt.setNextTuplePosition(leafPageNo, leaf.getNumTuples());
}
else {
logger.debug(String.format("Moving %d tuples from " +
"right sibling %d to leaf %d. Deleted tuple " +
"has next tuple at [%d:%d]; not updating",
tuplesToMove, rightPageNo, leafPageNo,
nextPageNo, nextIndex));
}
}
}
InnerPage parent =
innerPageOps.loadPage(pagePath.get(pagePath.size() - 2));
int index;
if (adjPage == leftSibling) {
adjPage.moveTuplesRight(leaf, tuplesToMove);
index = parent.getIndexOfPointer(adjPage.getPageNo());
parent.replaceTuple(index, leaf.getTuple(0));
}
else { // adjPage == right sibling
adjPage.moveTuplesLeft(leaf, tuplesToMove);
index = parent.getIndexOfPointer(leaf.getPageNo());
parent.replaceTuple(index, adjPage.getTuple(0));
}
}
}
/**
* This helper function handles the operation of adding a new tuple to a
* leaf-page of the index. This operation is provided here and not on the
* {@link LeafPage} class, because adding the new tuple might require the
* leaf page to be split into two pages.
*
* @param leaf the leaf page to add the tuple to
*
* @param newTuple the new tuple to add to the leaf page
*
* @param pagePath the path of pages taken from the root page to this leaf
* page, represented as a list of page numbers in the data file
*/
public BTreeFilePageTuple addTuple(LeafPage leaf, TupleLiteral newTuple,
List<Integer> pagePath) {
BTreeFilePageTuple result;
// Figure out where the new tuple-value goes in the leaf page.
int newTupleSize = newTuple.getStorageSize();
if (leaf.getFreeSpace() < newTupleSize) {
// Try to relocate tuples from this leaf to either sibling,
// or if that can't happen, split the leaf page into two.
result = relocateTuplesAndAddTuple(leaf, pagePath, newTuple);
if (result == null)
result = splitLeafAndAddTuple(leaf, pagePath, newTuple);
}
else {
// There is room in the leaf for the new tuple. Add it there.
result = leaf.addTuple(newTuple);
}
return result;
}
/**
* This method attempts to relocate tuples to the left or right sibling
* of the specified node, and then insert the specified tuple into the
* appropriate node. If it's not possible to relocate tuples (perhaps
* because there isn't space and/or because there is no sibling), the
* method returns {@code null}.
*
* @param page The leaf page to relocate tuples out of.
*
* @param pagePath The path from the index's root to the leaf page
*
* @param tuple The tuple to add to the index.
*
* @return a {@code BTreeFilePageTuple} representing the actual tuple
* added into the tree structure, or {@code null} if tuples
* couldn't be relocated to make space for the new tuple.
*/
private BTreeFilePageTuple relocateTuplesAndAddTuple(LeafPage page,
List<Integer> pagePath, TupleLiteral tuple) {
// See if we are able to relocate records either direction to free up
// space for the new tuple.
int bytesRequired = tuple.getStorageSize();
int pathSize = pagePath.size();
if (pathSize == 1) // This node is also the root - no parent.
return null; // There aren't any siblings to relocate to.
if (pagePath.get(pathSize - 1) != page.getPageNo()) {
throw new IllegalArgumentException(
"leaf page number doesn't match last page-number in page path");
}
int parentPageNo = 0;
if (pathSize >= 2)
parentPageNo = pagePath.get(pathSize - 2);
InnerPage parentPage = innerPageOps.loadPage(parentPageNo);
int numPointers = parentPage.getNumPointers();
int pagePtrIndex = parentPage.getIndexOfPointer(page.getPageNo());
// Check each sibling in its own code block so that we can constrain
// the scopes of the variables a bit. This keeps us from accidentally
// reusing the "prev" variables in the "next" section.
{
LeafPage prevPage = null;
if (pagePtrIndex - 1 >= 0)
prevPage = loadLeafPage(parentPage.getPointer(pagePtrIndex - 1));
if (prevPage != null) {
// See if we can move some of this leaf's tuples to the
// previous leaf, to free up space.
int count = tryLeafRelocateForSpace(page, prevPage, false,
bytesRequired);
if (count > 0) {
// Yes, we can do it!
logger.debug(String.format("Relocating %d tuples from " +
"leaf-page %d to left-sibling leaf-page %d", count,
page.getPageNo(), prevPage.getPageNo()));
logger.debug("Space before relocation: Leaf = " +
page.getFreeSpace() + " bytes\t\tSibling = " +
prevPage.getFreeSpace() + " bytes");
page.moveTuplesLeft(prevPage, count);
logger.debug("Space after relocation: Leaf = " +
page.getFreeSpace() + " bytes\t\tSibling = " +
prevPage.getFreeSpace() + " bytes");
BTreeFilePageTuple result =
addTupleToLeafPair(prevPage, page, tuple);
if (result == null) {
// Even with relocating tuples, we couldn't free up
// enough space. :-(
return null;
}
// Since we relocated tuples between two nodes, update
// the parent page to reflect the tuple that is now at
// the start of the right page.
BTreeFilePageTuple firstRightTuple = page.getTuple(0);
pagePath.remove(pathSize - 1);
innerPageOps.replaceTuple(parentPage, pagePath,
prevPage.getPageNo(), firstRightTuple, page.getPageNo());
return result;
}
}
}
{
LeafPage nextPage = null;
if (pagePtrIndex + 1 < numPointers)
nextPage = loadLeafPage(parentPage.getPointer(pagePtrIndex + 1));
if (nextPage != null) {
// See if we can move some of this leaf's tuples to the next
// leaf, to free up space.
int count = tryLeafRelocateForSpace(page, nextPage, true,
bytesRequired);
if (count > 0) {
// Yes, we can do it!
logger.debug(String.format("Relocating %d tuples from " +
"leaf-page %d to right-sibling leaf-page %d", count,
page.getPageNo(), nextPage.getPageNo()));
logger.debug("Space before relocation: Leaf = " +
page.getFreeSpace() + " bytes\t\tSibling = " +
nextPage.getFreeSpace() + " bytes");
page.moveTuplesRight(nextPage, count);
logger.debug("Space after relocation: Leaf = " +
page.getFreeSpace() + " bytes\t\tSibling = " +
nextPage.getFreeSpace() + " bytes");
BTreeFilePageTuple result =
addTupleToLeafPair(page, nextPage, tuple);
if (result == null) {
// Even with relocating tuples, we couldn't free up
// enough space. :-(
return null;
}
// Since we relocated tuples between two nodes, update
// the parent page to reflect the tuple that is now at
// the start of the right page.
BTreeFilePageTuple firstRightTuple = nextPage.getTuple(0);
pagePath.remove(pathSize - 1);
innerPageOps.replaceTuple(parentPage, pagePath,
page.getPageNo(), firstRightTuple, nextPage.getPageNo());
return result;
}
}
}
// Couldn't relocate tuples to either the previous or next page. We
// must split the leaf into two.
return null;
}
/**
* This helper method takes a pair of leaf nodes that are siblings to each
* other, and adds the specified tuple to whichever leaf the tuple should
* go into. The method returns the {@code BTreeFilePageTuple} object
* representing the actual tuple in the tree once it has been added.
*
* @param prevLeaf the first leaf in the pair, left sibling of
* {@code nextLeaf}
*
* @param nextLeaf the second leaf in the pair, right sibling of
* {@code prevLeaf}
*
* @param tuple the tuple to insert into the pair of leaves
*
* @return the actual tuple in the page, after the insert is completed
*/
private BTreeFilePageTuple addTupleToLeafPair(LeafPage prevLeaf,
LeafPage nextLeaf, TupleLiteral tuple) {
BTreeFilePageTuple result = null;
BTreeFilePageTuple firstRightTuple = nextLeaf.getTuple(0);
if (TupleComparator.compareTuples(tuple, firstRightTuple) < 0) {
// The new tuple goes in the left page. Hopefully there is room
// for it...
logger.debug("Adding tuple to left leaf " + prevLeaf.getPageNo() +
" in pair");
if (prevLeaf.getFreeSpace() >= tuple.getStorageSize())
result = prevLeaf.addTuple(tuple);
}
else {
// The new tuple goes in the right page. Again, hopefully there
// is room for it...
logger.debug("Adding tuple to right leaf " + nextLeaf.getPageNo() +
" in pair");
if (nextLeaf.getFreeSpace() >= tuple.getStorageSize())
result = nextLeaf.addTuple(tuple);
}
return result;
}
/**
* This helper function determines how many tuples must be relocated from
* one leaf-page to another, in order to free up the specified number of
* bytes. If it is possible, the number of tuples that must be relocated
* is returned. If it is not possible, the method returns 0.
*
* @param leaf the leaf node to relocate tuples from
*
* @param adjLeaf the adjacent leaf (predecessor or successor) to relocate
* tuples to
*
* @param movingRight pass {@code true} if the sibling is to the right of
* {@code page} (and therefore we are moving tuples right), or
* {@code false} if the sibling is to the left of {@code page} (and
* therefore we are moving tuples left).
*
* @param bytesRequired the number of bytes that must be freed up in
* {@code leaf} by the operation
*
* @return the number of tuples that must be relocated to free up the
* required space, or 0 if it is not possible.
*/
private int tryLeafRelocateForSpace(LeafPage leaf, LeafPage adjLeaf,
boolean movingRight, int bytesRequired) {
int numTuples = leaf.getNumTuples();
int leafBytesFree = leaf.getFreeSpace();
int adjBytesFree = adjLeaf.getFreeSpace();
logger.debug("Leaf bytes free: " + leafBytesFree +
"\t\tAdjacent leaf bytes free: " + adjBytesFree);
// Subtract the bytes-required from the adjacent-bytes-free value so
// that we ensure we always have room to put the tuple in either node.
adjBytesFree -= bytesRequired;
int numRelocated = 0;
while (true) {
// Figure out the index of the tuple we need the size of, based on
// the direction we are moving values. If we are moving values
// right, we need to look at the tuples starting at the rightmost
// one. If we are moving tuples left, we need to start with the
// leftmost tuple.
int index;
if (movingRight)
index = numTuples - numRelocated - 1;
else
index = numRelocated;
int tupleSize = leaf.getTupleSize(index);
logger.debug("Tuple " + index + " is " + tupleSize + " bytes");
// Did we run out of space to move tuples before we hit our goal?
if (adjBytesFree < tupleSize) {
numRelocated = 0;
break;
}
numRelocated++;
leafBytesFree += tupleSize;
adjBytesFree -= tupleSize;
// Since we don't yet know which leaf the new tuple will go into,
// stop when we can put the tuple in either leaf.
if (leafBytesFree >= bytesRequired &&
adjBytesFree >= bytesRequired) {
break;
}
}
logger.debug("Can relocate " + numRelocated + " tuples to free up space.");
return numRelocated;
}
/**
* This helper function splits the specified leaf-node into two nodes,
* also updating the parent node in the process, and then inserts the
* specified tuple into the appropriate leaf. This method is used to add
* a tuple to a leaf that doesn't have enough space, when it isn't
* possible to relocate values to the left or right sibling of the leaf.
*
* @todo (donnie) When the leaf node is split, half of the tuples are
* put into the new leaf, regardless of the size of individual
* tuples. In other words, this method doesn't try to keep the
* leaves half-full based on bytes used. It would almost
* certainly be better if it did.
*
* @param leaf the leaf node to split and then add the tuple to
* @param pagePath the sequence of page-numbers traversed to reach this
* leaf node.
*
* @param tuple the new tuple to insert into the leaf node
*/
private BTreeFilePageTuple splitLeafAndAddTuple(LeafPage leaf,
List<Integer> pagePath, TupleLiteral tuple) {
int pathSize = pagePath.size();
if (pagePath.get(pathSize - 1) != leaf.getPageNo()) {
throw new IllegalArgumentException(
"Leaf page number doesn't match last page-number in page path");
}
if (logger.isDebugEnabled()) {
logger.debug("Splitting leaf-page " + leaf.getPageNo() +
" into two leaves.");
logger.debug(" Old next-page: " + leaf.getNextPageNo());
}
// Get a new blank page in the index, with the same parent as the
// leaf-page we were handed.
DBPage newDBPage = fileOps.getNewDataPage();
LeafPage newLeaf = LeafPage.init(newDBPage, tupleFile.getSchema());
/* TODO: IMPLEMENT THE REST OF THIS METHOD.
*
* The LeafPage class provides some helpful operations for moving leaf-
* entries to a left or right sibling.
*
* The parent page must also be updated. If the leaf node doesn't have
* a parent, the tree's depth will increase by one level.
*/
logger.error("NOT YET IMPLEMENTED: splitLeafAndAddKey()");
return null;
}
/**
* This helper function determines how many tuples must be relocated from
* one leaf-page to another, in order to satisfy the "minimum space"
* requirement of the B tree.free up the specified number of
* bytes. If it is possible, the number of tuples that must be relocated
* is returned. If it is not possible, the method returns 0.
*
* @param leaf the leaf node to relocate tuples from
*
* @param adjLeaf the adjacent leaf (predecessor or successor) to relocate
* tuples to
*
* @param movingRight pass {@code true} if the sibling is to the left of
* {@code page} (and therefore we are moving tuples right), or
* {@code false} if the sibling is to the right of {@code page}
* (and therefore we are moving tuples left).
*
* @return the number of tuples that must be relocated to fill the node
* to a minimal level, or 0 if not possible.
*/
private int tryLeafRelocateToFill(LeafPage leaf, LeafPage adjLeaf,
boolean movingRight) {
int adjTuples = adjLeaf.getNumTuples(); // Tuples available to move
int leafBytesFree = leaf.getFreeSpace();
int adjBytesFree = adjLeaf.getFreeSpace();
// Should be the same for both leaf and adjLeaf.
int halfFull = leaf.getTotalSpace() / 2;
logger.debug("Leaf bytes free: " + leafBytesFree +
"\t\tAdjacent leaf bytes free: " + adjBytesFree);
int numRelocated = 0;
while (true) {
// Figure out the index of the tuple we need the size of, based on
// the direction we are moving values. If we are moving values
// right, we need to look at the tuples starting at the rightmost
// one. If we are moving values left, we need to start with the
// leftmost tuple.
int index;
if (movingRight)
index = adjTuples - numRelocated - 1;
else
index = numRelocated;
int tupleSize = adjLeaf.getTupleSize(index);
logger.debug("Tuple " + index + " is " + tupleSize + " bytes");
// If we don't have room to move the adjacent node's tuple into
// this node (unlikely), just stop there.
if (leafBytesFree < tupleSize)
break;
// If the adjacent leaf would become too empty, stop relocating.
if (adjBytesFree > halfFull)
break;
numRelocated++;
leafBytesFree -= tupleSize;
adjBytesFree += tupleSize;
// Stop if the leaf now has at least the minimal number of bytes.
if (leafBytesFree <= halfFull)
break;
}
logger.debug("Can relocate " + numRelocated +
" tuples to satisfy minimum space requirements.");
return numRelocated;
}
}
<html>
<body>
<p>
This package contains an implementation of B<sup>+</sup> tree
tuple files, which can be used for sequential tuple files, as
well as table indexes.
</p>
<!--
<h2>Leaf Page Internal Structure</h2>
<table>
<tr><th>Offset</th><th>Size</th><th>Description</th></tr>
<tr>
<td>0</td>
<td>byte</td>
<td>Page type, always 2 for leaf pages.
(See {@link BTreeIndexManager#BTREE_LEAF_PAGE}.)</td>
</tr>
<tr>
<td>1</td>
<td>unsigned short</td>
<td>Page number of next leaf page.
(See {@link LeafPage#OFFSET_NEXT_PAGE_NO}.)</td>
</tr>
<tr>
<td>3</td>
<td>unsigned short</td>
<td>The number of key+pointer entries is stored in the page.
(See {@link LeafPage#OFFSET_NUM_ENTRIES}.)</td>
</tr>
<tr>
</tr>
</table>
-->
</body>
</html>
package edu.caltech.test.nanodb.storage.btreefile;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Iterator;
import java.util.Random;
import edu.caltech.nanodb.expressions.ColumnName;
import edu.caltech.nanodb.expressions.ColumnValue;
import edu.caltech.nanodb.expressions.OrderByExpression;
import edu.caltech.nanodb.expressions.TupleComparator;
import edu.caltech.nanodb.expressions.TupleLiteral;
import edu.caltech.nanodb.relations.ColumnInfo;
import edu.caltech.nanodb.relations.ColumnType;
import edu.caltech.nanodb.relations.SQLDataType;
import edu.caltech.nanodb.relations.Schema;
import edu.caltech.nanodb.server.CommandResult;
import org.testng.annotations.*;
import edu.caltech.test.nanodb.sql.SqlTestCase;
/**
* This test class exercises the B<sup>+</sup>-tree file format so that we can
* have some confidence that it actually works correctly.
*/
@Test(groups={"storage", "hw6"})
public class TestBTreeFile extends SqlTestCase {
/**
* If set to true, this causes the tests to verify the contents of the
* B tree file after every insertion or deletion. This obviously greatly
* slows down the test, but it allows issues to be identified exactly when
* they appear.
*/
public static final boolean CHECK_AFTER_EACH_CHANGE = false;
/**
* A source of randomness to generate tuples from. Set the seed so we
* have reproducible test cases.
*/
private Random rand = new Random(12345);
private String makeRandomString(int minChars, int maxChars) {
StringBuilder buf = new StringBuilder();
int num = minChars + rand.nextInt(maxChars - minChars + 1);
for (int i = 0; i < num; i++)
buf.append((char) ('A' + rand.nextInt('Z' - 'A' + 1)));
return buf.toString();
}
private void sortTupleLiteralArray(ArrayList<TupleLiteral> tuples) {
Schema schema = new Schema();
schema.addColumnInfo(new ColumnInfo("a", new ColumnType(SQLDataType.INTEGER)));
schema.addColumnInfo(new ColumnInfo("b", new ColumnType(SQLDataType.VARCHAR)));
ArrayList<OrderByExpression> orderSpec = new ArrayList<>();
orderSpec.add(new OrderByExpression(new ColumnValue(new ColumnName("a"))));
orderSpec.add(new OrderByExpression(new ColumnValue(new ColumnName("b"))));
TupleComparator comp = new TupleComparator(schema, orderSpec);
Collections.sort(tuples, comp);
}
private void runBTreeTest(String tableName, int numRowsToInsert,
int maxAValue, int minBLen, int maxBLen,
double probDeletion) throws Exception {
ArrayList<TupleLiteral> inserted = new ArrayList<>();
CommandResult result;
for (int i = 0; i < numRowsToInsert; i++) {
int a = rand.nextInt(maxAValue);
String b = makeRandomString(minBLen, maxBLen);
tryDoCommand(String.format(
"INSERT INTO %s VALUES (%d, '%s');", tableName, a, b), false);
inserted.add(new TupleLiteral(a, b));
if (CHECK_AFTER_EACH_CHANGE) {
sortTupleLiteralArray(inserted);
result = tryDoCommand(String.format("SELECT * FROM %s;",
tableName), true);
assert checkOrderedResults(inserted.toArray(new TupleLiteral[inserted.size()]), result);
}
if (probDeletion > 0.0 && rand.nextDouble() < probDeletion) {
// Delete some rows from the table we are populating.
int minAToDel = rand.nextInt(maxAValue);
int maxAToDel = rand.nextInt(maxAValue);
if (minAToDel > maxAToDel) {
int tmp = minAToDel;
minAToDel = maxAToDel;
maxAToDel = tmp;
}
tryDoCommand(String.format(
"DELETE FROM %s WHERE a BETWEEN %d AND %d;", tableName,
minAToDel, maxAToDel), false);
// Apply the same deletion to our in-memory collection of
// tuples, so that we can mirror what the table should contain
Iterator<TupleLiteral> iter = inserted.iterator();
while (iter.hasNext()) {
TupleLiteral tup = iter.next();
int aVal = (Integer) tup.getColumnValue(0);
if (aVal >= minAToDel && aVal <= maxAToDel)
iter.remove();
}
if (CHECK_AFTER_EACH_CHANGE) {
sortTupleLiteralArray(inserted);
result = tryDoCommand(String.format("SELECT * FROM %s;",
tableName), true);
assert checkOrderedResults(inserted.toArray(new TupleLiteral[inserted.size()]), result);
}
}
}
sortTupleLiteralArray(inserted);
result = tryDoCommand(String.format("SELECT * FROM %s;",
tableName), true);
assert checkOrderedResults(inserted.toArray(new TupleLiteral[inserted.size()]), result);
// TODO: This is necessary because the btree code doesn't unpin
// pages properly...
// server.getStorageManager().flushAllData();
}
public void testBTreeTableOnePageInsert() throws Exception {
tryDoCommand("CREATE TABLE btree_one_page (a INTEGER, b VARCHAR(20)) " +
"PROPERTIES (storage = 'btree');", false);
runBTreeTest("btree_one_page", 300, 200, 3, 20, 0.0);
}
public void testBTreeTableTwoPageInsert() throws Exception {
tryDoCommand("CREATE TABLE btree_two_page (a INTEGER, b VARCHAR(50)) " +
"PROPERTIES (storage = 'btree');", false);
runBTreeTest("btree_two_page", 400, 200, 20, 50, 0.0);
}
public void testBTreeTableTwoLevelInsert() throws Exception {
tryDoCommand("CREATE TABLE btree_two_level (a INTEGER, b VARCHAR(50)) " +
"PROPERTIES (storage = 'btree');", false);
runBTreeTest("btree_two_level", 10000, 1000, 20, 50, 0.0);
}
public void testBTreeTableThreeLevelInsert() throws Exception {
tryDoCommand("CREATE TABLE btree_three_level (a INTEGER, b VARCHAR(250)) " +
"PROPERTIES (storage = 'btree');", false);
runBTreeTest("btree_three_level", 100000, 5000, 150, 250, 0.0);
}
public void testBTreeTableOnePageInsertDelete() throws Exception {
tryDoCommand("CREATE TABLE btree_one_page_del (a INTEGER, b VARCHAR(20)) " +
"PROPERTIES (storage = 'btree');", false);
runBTreeTest("btree_one_page_del", 400, 200, 3, 20, 0.05);
}
public void testBTreeTableTwoPageInsertDelete() throws Exception {
tryDoCommand("CREATE TABLE btree_two_page_del (a INTEGER, b VARCHAR(50)) " +
"PROPERTIES (storage = 'btree');", false);
runBTreeTest("btree_two_page_del", 500, 200, 20, 50, 0.05);
}
public void testBTreeTableTwoLevelInsertDelete() throws Exception {
tryDoCommand("CREATE TABLE btree_two_level_del (a INTEGER, b VARCHAR(50)) " +
"PROPERTIES (storage = 'btree');", false);
runBTreeTest("btree_two_level_del", 12000, 1000, 20, 50, 0.05);
}
public void testBTreeTableThreeLevelInsertDelete() throws Exception {
tryDoCommand("CREATE TABLE btree_three_level_del (a INTEGER, b VARCHAR(250)) " +
"PROPERTIES (storage = 'btree');", false);
runBTreeTest("btree_three_level_del", 120000, 5000, 150, 250, 0.05);
}
public void testBTreeTableMultiLevelInsertDelete() throws Exception {
tryDoCommand("CREATE TABLE btree_multi_level_del (a INTEGER, b VARCHAR(400)) " +
"PROPERTIES (storage = 'btree');", false);
runBTreeTest("btree_multi_level_del", 250000, 5000, 50, 400, 0.01);
}
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment