diff --git a/doc/lab6design.txt b/doc/lab6design.txt
new file mode 100644
index 0000000000000000000000000000000000000000..f5969a4281e50b68a5ca71ac90e77ac270b29846
--- /dev/null
+++ b/doc/lab6design.txt
@@ -0,0 +1,129 @@
+CS122 Assignment 6 - B+ Tree Indexes - Design Document
+======================================================
+
+A: Analysis of Implementation
+------------------------------
+
+Given NanoDB's B+ tree implementation, consider a simple schema where an
+index is built against a single integer column:
+
+ CREATE TABLE t (
+ -- An index is automatically built on the id column by NanoDB.
+ id INTEGER PRIMARY KEY,
+ value VARCHAR(20)
+ );
+
+Answer the following questions.
+
+A1. What is the total size of the index's search-key for the primary-key
+ index, in bytes? Break down this size into its individual components;
+ be as detailed as possible. (You don't need to go lower than the
+ byte-level in your answer, but you should show what each byte is a
+ part of.)
+
+A2. What is the maximum number of search-keys that can be stored in leaf
+ nodes of NanoDB's B+ tree implementation? You should assume a page-
+ size of 8192 bytes.
+
+A3. What is the maximum number of keys that can be stored in inner nodes
+ of this particular implementation? (Recall that every key must have
+ a page-pointer on either side of the key.)
+
+A4. In this implementation, leaf nodes do not reference the previous
+ leaf, only the next leaf. When splitting a leaf into two leaves,
+ what is the maximum number of leaf nodes that must be read or written,
+ in order to properly manage the next-leaf pointers?
+
+ If leaves also contained a previous-leaf pointer, what would the
+ answer be instead?
+
+ Make sure to explain your answers.
+
+A5. In this implementation, nodes do not store a page-pointer to their
+ parent node. This makes the update process somewhat complicated, as
+ we must save the sequence of page-numbers we traverse as we navigate
+ from root to leaf. If a node must be split, or if entries are to be
+ relocated from a node to its siblings, the node’s parent-node must
+ be retrieved, and the parent’s contents must be scanned to determine
+ the node’s sibling(s).
+
+ Consider an alternate B+ tree implementation in which every node
+ stores a page-pointer to the node’s parent. In the case of splitting
+ an inner node, what performance-related differences are there between
+ this alternate representation and the given implementation, where
+ nodes do not record their parents? Which one would you recommend?
+ Justify your answer.
+
+A6. It should be obvious how indexes can be used to enforce primary keys,
+ but what role might they play with foreign keys? For example, given
+ this schema:
+
+ CREATE TABLE t1 (
+ id INTEGER PRIMARY KEY
+ );
+ CREATE TABLE t2 (
+ id INTEGER REFERENCES t1;
+ );
+
+ Why might we want to build an index on t2.id?
+
+A7. Over time, a B+ tree's pages, its leaf pages in particular, may become
+ severely out of order, causing a significant number of seeks as the
+ leaves are traversed in sequence. Additionally, some of the blocks
+ within the B+ tree file may be empty.
+
+ An easy mechanism for regenerating a B+ tree file is to traverse the file's
+ tuples in sequential order (i.e. from the leftmost leaf page, through
+ all leaf pages in the file), adding each tuple to a new B+ tree file.
+ The new file may then replace the old file.
+
+ Imagine that this operation is performed on a B+ tree file containing
+ many records, where all records are the same size. On average, how full
+ will the leaf pages in the newly generated file be? You can state your
+ answer as a percentage, e.g. "0% full". Explain your answer.
+
+A8. Consider a variation of the approach in A7: Instead of making one pass
+ through the initial B+ tree file, two passes are made. On the first pass,
+ the 1st, 3rd, 5th, 7th, ... tuples are added to the new file. Then, on the
+ second pass, the 2nd, 4th, 6th, 8th, ... tuples are added to the new file.
+ On average, how full will the leaf pages in the newly generated file be?
+ Explain your answer.
+
+D: Extra Credit [OPTIONAL]
+---------------------------
+
+If you implemented any extra-credit tasks for this assignment, describe
+them here. The description should be like this, with stuff in "<>" replaced.
+(The value i starts at 1 and increments...)
+
+D<i>: <one-line description>
+
+ <brief summary of what you did, including the specific classes that
+ we should look at for your implementation>
+
+ <brief summary of test-cases that demonstrate/exercise your extra work>
+
+E: Feedback [OPTIONAL]
+-----------------------
+
+WE NEED YOUR FEEDBACK! Thoughtful and constructive input will help us to
+improve future versions of the course. These questions are OPTIONAL, and
+your answers will not affect your grade in any way (including if you hate
+everything about the assignment and databases in general, or Donnie and/or
+the TAs in particular). Feel free to answer as many or as few of them as
+you wish.
+
+E1. What parts of the assignment were most time-consuming? Why?
+
+E2. Did you find any parts of the assignment particularly instructive?
+ Correspondingly, did any parts feel like unnecessary busy-work?
+
+E3. Did you particularly enjoy any parts of the assignment? Were there
+ any parts that you particularly disliked?
+
+E4. Were there any critical details that you wish had been provided with the
+ assignment, that we should consider including in subsequent versions of
+ the assignment?
+
+E5. Do you have any other suggestions for how future versions of the
+ assignment can be improved?
diff --git a/doc/lab6info.txt b/doc/lab6info.txt
new file mode 100644
index 0000000000000000000000000000000000000000..2b115da60745232b866c6d2fa17cd11fd1b32ba8
--- /dev/null
+++ b/doc/lab6info.txt
@@ -0,0 +1,33 @@
+CS122 Assignment 6 - B+ Tree Indexes
+====================================
+
+Please completely fill out this document so that we know who participated on
+the assignment, any late extensions received, and how much time the assignment
+took for your team. Thank you!
+
+L1. List your team name and the people who worked on this assignment.
+
+ <team name>
+
+ <name>
+ <name>
+ ...
+
+L2. Specify the tag and commit-hash of the Git commit you are submitting for
+ your assignment. (You can list the hashes of all tags with the command
+ "git show-ref --tags".)
+
+ Tag: <tag>
+ Commit hash: <hash>
+
+L3. Specify how many late tokens you are applying to this assignment, if any.
+ Similarly, if your team received an extension from Donnie then please
+ indicate how many days extension you received. You may leave this blank
+ if it is not relevant to this submission.
+
+ <tokens / extension>
+
+L4. For each teammate, briefly describe what parts of the assignment each
+ teammate focused on, along with the total hours spent on the assignment.
+
+
diff --git a/pom.xml b/pom.xml
index fdc5d4e1c1e1979556cd23ae61fc51c3c0a062f5..e665e32a43d5f470b4bb39d96333c08e2c1cf90c 100644
--- a/pom.xml
+++ b/pom.xml
@@ -144,7 +144,7 @@
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
-->
- <groups>framework,parser,hw1,hw2</groups>
+ <groups>framework,parser,hw1,hw2,hw5,hw6</groups>
</configuration>
</plugin>
diff --git a/src/main/java/edu/caltech/nanodb/storage/IndexedTableManager.java b/src/main/java/edu/caltech/nanodb/storage/IndexedTableManager.java
index 54c5734c0401fd18d0dfe137fd00dbc80087c487..85fc88eb962910d36236a50abf1096f437f5761b 100644
--- a/src/main/java/edu/caltech/nanodb/storage/IndexedTableManager.java
+++ b/src/main/java/edu/caltech/nanodb/storage/IndexedTableManager.java
@@ -101,6 +101,9 @@ public class IndexedTableManager implements TableManager {
if ("heap".equals(storageType)) {
type = DBFileType.HEAP_TUPLE_FILE;
}
+ else if ("btree".equals(storageType)) {
+ type = DBFileType.BTREE_TUPLE_FILE;
+ }
else {
throw new IllegalArgumentException("Unrecognized table file " +
"type: " + storageType);
diff --git a/src/main/java/edu/caltech/nanodb/storage/StorageManager.java b/src/main/java/edu/caltech/nanodb/storage/StorageManager.java
index a4f65ce659c80132137053cfedd60561c529f9e4..f4df532d0710a8928c12976f05b73126fef22a76 100755
--- a/src/main/java/edu/caltech/nanodb/storage/StorageManager.java
+++ b/src/main/java/edu/caltech/nanodb/storage/StorageManager.java
@@ -16,6 +16,8 @@ import edu.caltech.nanodb.server.EventDispatcher;
import edu.caltech.nanodb.server.NanoDBServer;
import edu.caltech.nanodb.server.properties.PropertyRegistry;
import edu.caltech.nanodb.server.properties.ServerProperties;
+
+import edu.caltech.nanodb.storage.btreefile.BTreeTupleFileManager;
import edu.caltech.nanodb.storage.heapfile.HeapTupleFileManager;
import edu.caltech.nanodb.transactions.TransactionManager;
@@ -139,6 +141,9 @@ public class StorageManager {
tupleFileManagers.put(DBFileType.HEAP_TUPLE_FILE,
new HeapTupleFileManager(this));
+ tupleFileManagers.put(DBFileType.BTREE_TUPLE_FILE,
+ new BTreeTupleFileManager(this));
+
if (enableTransactions) {
logger.info("Initializing transaction manager.");
transactionManager = new TransactionManager(server);
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeFilePageTuple.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeFilePageTuple.java
new file mode 100644
index 0000000000000000000000000000000000000000..751eef700d2c76ef3ca767e5545fec26b949c440
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeFilePageTuple.java
@@ -0,0 +1,147 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import edu.caltech.nanodb.relations.Schema;
+import edu.caltech.nanodb.storage.DBPage;
+import edu.caltech.nanodb.storage.PageTuple;
+
+
+/**
+ * <p>
+ * This class uses the <tt>PageTuple</tt> class functionality to access and
+ * manipulate keys stored in a B<sup>+</sup> tree tuple file. There is one
+ * extension, which is to allow the tuple to remember its index within the
+ * leaf page it is from; this makes it easy to move to the next tuple within
+ * the page very easily.
+ * </p>
+ * <p>
+ * B<sup>+</sup> tree tuple deletion is interesting, since all the tuples form
+ * a linear sequence in the page. When a given tuple T is deleted, the next
+ * tuple ends up at the same position that T was at. Therefore, to implement
+ * the tuple file's "get next tuple" functionality properly, we must keep
+ * track of whether the previous tuple was deleted or not; if it was deleted,
+ * we don't advance in the page.
+ * </p>
+ */
+public class BTreeFilePageTuple extends PageTuple {
+
+ private int tupleIndex;
+
+
+ /**
+ * Records if this tuple has been deleted or not. This affects navigation
+ * to the next tuple in the current page, since removal of the current
+ * tuple causes this object to point to the next tuple.
+ */
+ private boolean deleted = false;
+
+ /**
+ * If this tuple is deleted, this field will be set to the page number of
+ * the next tuple. If there are no more tuples then this will be set to
+ * -1.
+ */
+ private int nextTuplePageNo;
+
+ /**
+ * If this tuple is deleted, this field will be set to the index of the
+ * next tuple. If there are no more tuples then this will be set to -1.
+ */
+ private int nextTupleIndex;
+
+
+ public BTreeFilePageTuple(Schema schema, DBPage dbPage, int pageOffset,
+ int tupleIndex) {
+ super(dbPage, pageOffset, schema);
+
+ if (tupleIndex < 0) {
+ throw new IllegalArgumentException(
+ "tupleIndex must be at least 0, got " + tupleIndex);
+ }
+
+ this.tupleIndex = tupleIndex;
+ }
+
+
+ public int getTupleIndex() {
+ return tupleIndex;
+ }
+
+
+ public boolean isDeleted() {
+ return deleted;
+ }
+
+
+ public void setDeleted() {
+ deleted = true;
+ }
+
+
+ public void setNextTuplePosition(int pageNo, int tupleIndex) {
+ if (!deleted)
+ throw new IllegalStateException("Tuple must be deleted");
+
+ nextTuplePageNo = pageNo;
+ nextTupleIndex = tupleIndex;
+ }
+
+
+ public int getNextTuplePageNo() {
+ if (!deleted)
+ throw new IllegalStateException("Tuple must be deleted");
+
+ return nextTuplePageNo;
+ }
+
+
+ public int getNextTupleIndex() {
+ if (!deleted)
+ throw new IllegalStateException("Tuple must be deleted");
+
+ return nextTupleIndex;
+ }
+
+
+ @Override
+ protected void insertTupleDataRange(int off, int len) {
+ throw new UnsupportedOperationException(
+ "B+ Tree index tuples don't support resizing.");
+ }
+
+
+ @Override
+ protected void deleteTupleDataRange(int off, int len) {
+ throw new UnsupportedOperationException(
+ "B+ Tree index tuples don't support resizing.");
+ }
+
+
+ @Override
+ public String toString() {
+ StringBuilder buf = new StringBuilder();
+ buf.append("BTPT[");
+
+ if (deleted) {
+ buf.append("deleted");
+ }
+ else {
+ boolean first = true;
+ for (int i = 0; i < getColumnCount(); i++) {
+ if (first)
+ first = false;
+ else
+ buf.append(',');
+
+ Object obj = getColumnValue(i);
+ if (obj == null)
+ buf.append("NULL");
+ else
+ buf.append(obj);
+ }
+ }
+
+ buf.append(']');
+
+ return buf.toString();
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeFileVerifier.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeFileVerifier.java
new file mode 100644
index 0000000000000000000000000000000000000000..2892ef1a9f7ba3fb91945f4087281ef6f9bceca6
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeFileVerifier.java
@@ -0,0 +1,532 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import edu.caltech.nanodb.expressions.TupleComparator;
+import edu.caltech.nanodb.expressions.TupleLiteral;
+import edu.caltech.nanodb.relations.Tuple;
+import edu.caltech.nanodb.storage.DBFile;
+import edu.caltech.nanodb.storage.DBPage;
+import edu.caltech.nanodb.storage.StorageManager;
+
+import static edu.caltech.nanodb.storage.btreefile.BTreePageTypes.*;
+
+
+/**
+ * This class provides some simple verification operations for B<sup>+</sup>
+ * tree tuple files.
+ */
+public class BTreeFileVerifier {
+ /** A logging object for reporting anything interesting that happens. */
+ private static Logger logger = LogManager.getLogger(BTreeFileVerifier.class);
+
+
+ /** This runtime exception class is used to abort the verification scan. */
+ private static class ScanAbortedException extends RuntimeException { }
+
+
+ /**
+ * This helper class is used to keep track of details of pages within the
+ * B<sup>+</sup> tree file.
+ */
+ private static class PageInfo {
+ /** The page's number. */
+ public int pageNo;
+
+
+ /** The type of the tuple-file page. */
+ int pageType;
+
+
+ /** A flag indicating whether the page is accessible from the root. */
+ boolean accessibleFromRoot;
+
+
+ /**
+ * Records how many times the page has been referenced in the file's
+ * tree structure.
+ */
+ int numTreeReferences;
+
+
+ /**
+ * Records how many times the page has been referenced in the file's
+ * leaf page list.
+ */
+ int numLeafListReferences;
+
+
+ /**
+ * Records how many times the page has been referenced in the file's
+ * empty page list.
+ */
+ int numEmptyListReferences;
+
+
+ PageInfo(int pageNo, int pageType) {
+ this.pageNo = pageNo;
+ this.pageType = pageType;
+
+ accessibleFromRoot = false;
+ numTreeReferences = 0;
+ numLeafListReferences = 0;
+ numEmptyListReferences = 0;
+ }
+ }
+
+
+ /** A reference to the storage manager since we use it so much. */
+ private StorageManager storageManager;
+
+
+ /** The B<sup>+</sup> tree tuple file to verify. */
+ private BTreeTupleFile tupleFile;
+
+
+ /**
+ * The actual {@code DBFile} object backing the tuple-file, since it is
+ * used so frequently in the verification.
+ */
+ private DBFile dbFile;
+
+
+ /**
+ * This collection maps individual B<sup>+</sup> tree pages to the details
+ * that the verifier collects for each page.
+ */
+ private HashMap<Integer, PageInfo> pages;
+
+
+ /** The collection of errors found during the verification process. */
+ private ArrayList<String> errors;
+
+
+ /**
+ * Initialize a verifier object to verify a specific B<sup>+</sup> tree
+ * tuple file.
+ */
+ public BTreeFileVerifier(StorageManager storageManager,
+ BTreeTupleFile tupleFile) {
+ this.storageManager = storageManager;
+ this.tupleFile = tupleFile;
+ this.dbFile = tupleFile.getDBFile();
+ }
+
+
+ /**
+ * This method is the entry-point for the verification process. It
+ * performs multiple passes through the file structure to identify various
+ * issues. Any errors that are identified are returned in the collection
+ * of error messages.
+ *
+ * @return a list of error messages describing issues found with the
+ * tuple file's structure. If no errors are detected, this will
+ * be an empty list, not {@code null}.
+ */
+ public List<String> verify() {
+ errors = new ArrayList<>();
+
+ try {
+ pass1ScanThruAllPages();
+ pass2TreeScanThruFile();
+ pass3ScanThruLeafList();
+ pass4ScanThruEmptyList();
+ pass5ExaminePageReachability();
+
+ /* These passes are specifically part of index validation, not
+ * just the file format validation, so we may need to move this
+ * somewhere else.
+ *
+ pass6VerifyTableTuplesInIndex();
+ pass7VerifyIndexTuplesInTable();
+ */
+ }
+ catch (ScanAbortedException e) {
+ // Do nothing.
+ logger.warn("Tuple-file verification scan aborted.");
+ }
+
+ return errors;
+ }
+
+
+ /**
+ * This method implements pass 1 of the verification process: scanning
+ * through all pages in the tuple file, collecting basic details about
+ * each page.
+ */
+ private void pass1ScanThruAllPages() {
+ logger.debug("Pass 1: Linear scan through pages to collect info");
+
+ pages = new HashMap<>();
+ for (int pageNo = 1; pageNo < dbFile.getNumPages(); pageNo++) {
+ DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
+
+ int pageType = dbPage.readUnsignedByte(0);
+ PageInfo info = new PageInfo(pageNo, pageType);
+ pages.put(pageNo, info);
+ }
+ }
+
+
+ /**
+ * This method implements pass 2 of the verification process: performing
+ * a tree-scan through the entire tuple file, starting with the root page
+ * of the file.
+ */
+ private void pass2TreeScanThruFile() {
+ logger.debug("Pass 2: Tree scan from root to verify nodes");
+
+ DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
+ int rootPageNo = HeaderPage.getRootPageNo(dbpHeader);
+
+ scanTree(rootPageNo, 0, null, null);
+ }
+
+
+ /**
+ * This helper function traverses the B<sup>+</sup> tree structure,
+ * verifying various invariants that should hold on the file structure.
+ */
+ private void scanTree(int pageNo, int parentPageNo, Tuple parentLeftKey,
+ Tuple parentRightKey) {
+
+ PageInfo info = pages.get(pageNo);
+ info.accessibleFromRoot = true;
+ info.numTreeReferences++;
+
+ if (info.numTreeReferences > 10) {
+ errors.add(String.format("Pass 2: Stopping scan! I've visited " +
+ "page %d %d times; there may be a cycle in your B+ tree " +
+ "structure.", pageNo, info.numTreeReferences));
+ throw new ScanAbortedException();
+ }
+
+ logger.trace("Examining page " + pageNo);
+ DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
+
+ switch (info.pageType) {
+ case BTREE_INNER_PAGE:
+ {
+ logger.trace("It's an inner page.");
+ InnerPage inner = new InnerPage(dbPage, tupleFile.getSchema());
+
+ ArrayList<Integer> refPages = new ArrayList<>();
+ int refInner = 0;
+ int refLeaf = 0;
+ int refOther = 0;
+
+ // Check the pages referenced from this page using the basic info
+ // collected in Pass 1.
+
+ for (int p = 0; p < inner.getNumPointers(); p++) {
+ int refPageNo = inner.getPointer(p);
+ refPages.add(refPageNo);
+ PageInfo refPageInfo = pages.get(refPageNo);
+
+ switch (refPageInfo.pageType) {
+ case BTREE_INNER_PAGE:
+ refInner++;
+ break;
+
+ case BTREE_LEAF_PAGE:
+ refLeaf++;
+ break;
+
+ default:
+ refOther++;
+ }
+
+ if (refInner != 0 && refLeaf != 0) {
+ errors.add(String.format("Pass 2: Inner page %d " +
+ "references both inner and leaf pages.", pageNo));
+ }
+
+ if (refOther != 0) {
+ errors.add(String.format("Pass 2: Inner page %d references " +
+ "pages that are neither inner pages nor leaf pages.", pageNo));
+ }
+ }
+
+ // Make sure the keys are in the proper order in the page.
+
+ int numKeys = inner.getNumKeys();
+ ArrayList<TupleLiteral> keys = new ArrayList<>(numKeys);
+ if (numKeys > 1) {
+ Tuple prevKey = inner.getKey(0);
+ keys.add(new TupleLiteral(prevKey));
+
+ if (parentLeftKey != null) {
+ int cmp = TupleComparator.compareTuples(parentLeftKey, prevKey);
+ // It is possible that the parent's left-key would be the
+ // same as the first key in this page.
+ if (cmp > 0) {
+ errors.add(String.format("Pass 2: Parent page %d's " +
+ "left key is greater than inner page %d's first key",
+ parentPageNo, pageNo));
+ }
+ }
+
+ for (int k = 1; k < numKeys; k++) {
+ Tuple key = inner.getKey(k);
+ keys.add(new TupleLiteral(key));
+
+ int cmp = TupleComparator.compareTuples(prevKey, key);
+ if (cmp == 0) {
+ errors.add(String.format("Pass 2: Inner page %d keys " +
+ "%d and %d are duplicates!", pageNo, k - 1, k));
+ }
+ else if (cmp > 0) {
+ errors.add(String.format("Pass 2: Inner page %d keys " +
+ "%d and %d are out of order!", pageNo, k - 1, k));
+ }
+ prevKey = key;
+ }
+
+ if (parentRightKey != null) {
+ int cmp = TupleComparator.compareTuples(prevKey, parentRightKey);
+ // The parent's right-key should be greater than the last
+ // key in this page.
+ if (cmp >= 0) {
+ errors.add(String.format("Pass 2: Parent page %d's " +
+ "right key is less than or equal to inner page " +
+ "%d's last key", parentPageNo, pageNo));
+ }
+ }
+ }
+
+ // Now that we are done with this page, check each child-page.
+
+ int p = 0;
+ Tuple prevKey = parentLeftKey;
+ for (int refPageNo : refPages) {
+ Tuple nextKey;
+ if (p < keys.size())
+ nextKey = keys.get(p);
+ else
+ nextKey = parentRightKey;
+
+ scanTree(refPageNo, pageNo, prevKey, nextKey);
+ prevKey = nextKey;
+ p++;
+ }
+
+ break;
+ }
+
+ case BTREE_LEAF_PAGE:
+ {
+ logger.trace("It's a leaf page.");
+ LeafPage leaf = new LeafPage(dbPage, tupleFile.getSchema());
+
+ // Make sure the keys are in the proper order in the page.
+
+ int numKeys = leaf.getNumTuples();
+ if (numKeys >= 1) {
+ Tuple prevKey = leaf.getTuple(0);
+
+ if (parentLeftKey != null) {
+ int cmp = TupleComparator.compareTuples(parentLeftKey, prevKey);
+ // It is possible that the parent's left-key would be the
+ // same as the first key in this page.
+ if (cmp > 0) {
+ errors.add(String.format("Pass 2: Parent page %d's " +
+ "left key is greater than inner page %d's first key",
+ parentPageNo, pageNo));
+ }
+ }
+
+ for (int k = 1; k < numKeys; k++) {
+ Tuple key = leaf.getTuple(k);
+ int cmp = TupleComparator.compareTuples(prevKey, key);
+ if (cmp == 0) {
+ errors.add(String.format("Pass 2: Leaf page %d keys " +
+ "%d and %d are duplicates!", pageNo, k - 1, k));
+ }
+ else if (cmp > 0) {
+ errors.add(String.format("Pass 2: Leaf page %d keys " +
+ "%d and %d are out of order!", pageNo, k - 1, k));
+ }
+ prevKey = key;
+ }
+
+ if (parentRightKey != null) {
+ int cmp = TupleComparator.compareTuples(prevKey, parentRightKey);
+ // The parent's right-key should be greater than the last
+ // key in this page.
+ if (cmp >= 0) {
+ errors.add(String.format("Pass 2: Parent page %d's " +
+ "right key is less than or equal to inner page " +
+ "%d's last key", parentPageNo, pageNo));
+ }
+ }
+ }
+
+ break;
+ }
+
+ default:
+ errors.add(String.format("Pass 2: Can reach page %d from root, " +
+ "but it's not a leaf or an inner page! Type = %d", pageNo,
+ info.pageType));
+ }
+ }
+
+
+ private void pass3ScanThruLeafList() {
+ logger.debug("Pass 3: Scan through leaf page list");
+
+ DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
+ int pageNo = HeaderPage.getRootPageNo(dbpHeader);
+
+ // Walk down the leftmost pointers in the inner pages until we reach
+ // the leftmost leaf page. Then we can walk across the leaves and
+ // check the constraints that should hold on leaves.
+ DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
+ int pageType = dbPage.readUnsignedByte(0);
+ while (pageType != BTREE_LEAF_PAGE) {
+ if (pageType != BTREE_INNER_PAGE) {
+ errors.add(String.format("Pass 3: Page %d should be an inner " +
+ "page, but its type is %d instead", pageNo, pageType));
+ }
+
+ InnerPage innerPage = new InnerPage(dbPage, tupleFile.getSchema());
+ pageNo = innerPage.getPointer(0);
+ dbPage = storageManager.loadDBPage(dbFile, pageNo);
+ pageType = dbPage.readUnsignedByte(0);
+ }
+
+ // Now we should be at the leftmost leaf in the sequence of leaves.
+ Tuple prevKey = null;
+ int prevKeyPageNo = 0;
+ while (true) {
+ PageInfo info = pages.get(pageNo);
+ info.numLeafListReferences++;
+
+ if (info.numLeafListReferences > 10) {
+ errors.add(String.format("Pass 3: Stopping scan! I've visited " +
+ "leaf page %d %d times; there may be a cycle in your leaf list.",
+ pageNo, info.numLeafListReferences));
+ throw new ScanAbortedException();
+ }
+
+ LeafPage leafPage = new LeafPage(dbPage, tupleFile.getSchema());
+
+ for (int k = 0; k < leafPage.getNumTuples(); k++) {
+ Tuple key = leafPage.getTuple(k);
+
+ if (prevKey != null) {
+ int cmp = TupleComparator.compareTuples(prevKey, key);
+ if (cmp == 0) {
+ if (prevKeyPageNo == pageNo) {
+ errors.add(String.format("Pass 3: Leaf page %d " +
+ "keys %d and %d are duplicates!", pageNo,
+ k - 1, k));
+ }
+ else {
+ errors.add(String.format("Pass 3: Leaf page %d " +
+ "key 0 is a duplicate to previous leaf %d's " +
+ "last key!", pageNo, prevKeyPageNo));
+ }
+ }
+ else if (cmp > 0) {
+ if (prevKeyPageNo == pageNo) {
+ errors.add(String.format("Pass 3: Leaf page %d " +
+ "keys %d and %d are out of order!", pageNo,
+ k - 1, k));
+ }
+ else {
+ errors.add(String.format("Pass 3: Leaf page %d " +
+ "key 0 is out of order with previous leaf %d's " +
+ "last key!", pageNo, prevKeyPageNo));
+ }
+ }
+ }
+
+ prevKey = key;
+ prevKeyPageNo = pageNo;
+ }
+
+ // Go to the next leaf in the sequence.
+
+ pageNo = leafPage.getNextPageNo();
+ if (pageNo == 0)
+ break;
+
+ dbPage = storageManager.loadDBPage(dbFile, pageNo);
+ pageType = dbPage.readUnsignedByte(0);
+
+ if (pageType != BTREE_LEAF_PAGE) {
+ errors.add(String.format("Pass 3: Page %d should be a leaf " +
+ "page, but its type is %d instead", pageNo, pageType));
+ }
+ }
+ }
+
+
+ private void pass4ScanThruEmptyList() {
+ logger.debug("Pass 4: Scan through empty page list");
+
+ DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
+ int emptyPageNo = HeaderPage.getFirstEmptyPageNo(dbpHeader);
+
+ while (emptyPageNo != 0) {
+ PageInfo info = pages.get(emptyPageNo);
+ info.numEmptyListReferences++;
+
+ if (info.numEmptyListReferences > 10) {
+ errors.add(String.format("Pass 4: Stopping scan! I've visited " +
+ "empty page %d %d times; there may be a cycle in your empty list.",
+ emptyPageNo, info.numEmptyListReferences));
+ throw new ScanAbortedException();
+ }
+
+ if (info.pageType != BTREE_EMPTY_PAGE) {
+ errors.add(String.format("Page %d is in the empty-page list, " +
+ "but it isn't an empty page! Type = %d", emptyPageNo,
+ info.pageType));
+ }
+
+ DBPage dbPage = storageManager.loadDBPage(dbFile, emptyPageNo);
+ emptyPageNo = dbPage.readUnsignedShort(1);
+ }
+ }
+
+
+ private void pass5ExaminePageReachability() {
+ logger.debug("Pass 5: Find pages with reachability issues");
+
+ for (int pageNo = 1; pageNo < pages.size(); pageNo++) {
+ PageInfo info = pages.get(pageNo);
+
+ if (info.pageType == BTREE_INNER_PAGE ||
+ info.pageType == BTREE_LEAF_PAGE) {
+ if (info.numTreeReferences != 1) {
+ errors.add(String.format("B+ tree page %d should have " +
+ "exactly one tree-reference, but has %d instead",
+ info.pageNo, info.numTreeReferences));
+ }
+
+ if (info.pageType == BTREE_LEAF_PAGE &&
+ info.numLeafListReferences != 1) {
+ errors.add(String.format("Leaf page %d should have " +
+ "exactly one leaf-list reference, but has %d instead",
+ info.pageNo, info.numLeafListReferences));
+ }
+ }
+ else if (info.pageType == BTREE_EMPTY_PAGE) {
+ if (info.numEmptyListReferences != 1) {
+ errors.add(String.format("Empty page %d should have " +
+ "exactly one empty-list reference, but has %d instead",
+ info.pageNo, info.numEmptyListReferences));
+ }
+ }
+ }
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreePageTypes.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreePageTypes.java
new file mode 100644
index 0000000000000000000000000000000000000000..1d5cccf45c56975212d7b5f832033ff2237e833b
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreePageTypes.java
@@ -0,0 +1,37 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+/**
+ * This interface specifies the page-type values that may appear within the
+ * B<sup>+</sup> Tree implementation.
+ *
+ * @design (donnie) We use this instead of an {@code enum} since the values
+ * are actually read and written against pages in the B<sup>+</sup>
+ * Tree file. Note that there is no page-type value for the root
+ * page; that page is considered separately.
+ *
+ * @design (donnie) This class is package-private since it is an internal
+ * implementation detail and we want to keep it local to the
+ * {@code btreefile} package.
+ */
+final class BTreePageTypes {
+ /**
+ * This value is stored in a B-tree page's byte 0, to indicate that the
+ * page is an inner (i.e. non-leaf) page.
+ */
+ public static final int BTREE_INNER_PAGE = 1;
+
+
+ /**
+ * This value is stored in a B-tree page's byte 0, to indicate that the
+ * page is a leaf page.
+ */
+ public static final int BTREE_LEAF_PAGE = 2;
+
+
+ /**
+ * This value is stored in a B-tree page's byte 0, to indicate that the
+ * page is empty.
+ */
+ public static final int BTREE_EMPTY_PAGE = 3;
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeTupleFile.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeTupleFile.java
new file mode 100644
index 0000000000000000000000000000000000000000..01ab84daf93870fc199069e72608e18bc7d9111d
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeTupleFile.java
@@ -0,0 +1,541 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import edu.caltech.nanodb.expressions.OrderByExpression;
+import edu.caltech.nanodb.expressions.TupleComparator;
+import edu.caltech.nanodb.expressions.TupleLiteral;
+import edu.caltech.nanodb.queryeval.TableStats;
+import edu.caltech.nanodb.relations.Schema;
+import edu.caltech.nanodb.relations.Tuple;
+import edu.caltech.nanodb.storage.DBFile;
+import edu.caltech.nanodb.storage.DBPage;
+import edu.caltech.nanodb.storage.FilePointer;
+import edu.caltech.nanodb.storage.InvalidFilePointerException;
+import edu.caltech.nanodb.storage.PageTuple;
+import edu.caltech.nanodb.storage.SequentialTupleFile;
+import edu.caltech.nanodb.storage.StorageManager;
+import edu.caltech.nanodb.storage.TupleFileManager;
+
+import static edu.caltech.nanodb.storage.btreefile.BTreePageTypes.*;
+
+
+/**
+ * <p>
+ * This class, along with a handful of helper classes in this package,
+ * provides support for B<sup>+</sup> tree tuple files. B<sup>+</sup> tree
+ * tuple files can be used to provide a sequential storage format for ordered
+ * tuples. They are also used to implement indexes for enforcing primary,
+ * candidate and foreign keys, and also for providing optimized access to
+ * tuples with specific values.
+ * </p>
+ * <p>
+ * Here is a brief overview of the NanoDB B<sup>+</sup> tree file format:
+ * </p>
+ * <ul>
+ * <li>Page 0 is always a header page, and specifies the entry-points in the
+ * hierarchy: the root page of the tree, and the leftmost leaf of the
+ * tree. Page 0 also maintains a free-list of empty pages in the tree, so
+ * that adding new pages to the tree is fast. (See the {@link HeaderPage}
+ * class for details.)</li>
+ * <li>The remaining pages are either leaf pages, inner pages, or empty pages.
+ * The first byte of the page always indicates the kind of page. For
+ * details about the internal structure of leaf and inner pages, see the
+ * {@link InnerPage} and {@link LeafPage} classes.</li>
+ * <li>Empty pages are organized into a simple singly linked list. Each empty
+ * page holds a page-pointer to the next empty page in the sequence, using
+ * an unsigned short stored at index 1 (after the page-type value in index
+ * 0). The final empty page stores 0 as its next-page pointer value.</li>
+ * </ul>
+ */
+public class BTreeTupleFile implements SequentialTupleFile {
+ /** A logging object for reporting anything interesting that happens. */
+ private static Logger logger = LogManager.getLogger(BTreeTupleFile.class);
+
+
+ /**
+ * If this flag is set to true, all data in data-pages that is no longer
+ * necessary is cleared. This will increase the cost of write-ahead
+ * logging, but it also exposes bugs more quickly because old data won't
+ * still be present if someone erroneously accesses it.
+ */
+ public static final boolean CLEAR_OLD_DATA = true;
+
+
+ /**
+ * The storage manager to use for reading and writing file pages, pinning
+ * and unpinning pages, write-ahead logging, and so forth.
+ */
+ private StorageManager storageManager;
+
+
+ /**
+ * The manager for B<sup>+</sup> tree tuple files provides some
+ * higher-level operations such as saving the metadata of a tuple file,
+ * so it's useful to have a reference to it.
+ */
+ private BTreeTupleFileManager btreeFileManager;
+
+
+ /** The schema of tuples in this tuple file. */
+ private Schema schema;
+
+
+ /** Statistics for this tuple file. */
+ private TableStats stats;
+
+
+ /** The file that stores the tuples. */
+ private DBFile dbFile;
+
+
+ /**
+ * A helper class that manages file-level operations on the B+ tree file.
+ */
+ private FileOperations fileOps;
+
+
+ /**
+ * A helper class that manages the larger-scale operations involving leaf
+ * nodes of the B+ tree.
+ */
+ private LeafPageOperations leafPageOps;
+
+
+ /**
+ * A helper class that manages the larger-scale operations involving inner
+ * nodes of the B+ tree.
+ */
+ private InnerPageOperations innerPageOps;
+
+
+ // private IndexInfo idxFileInfo;
+
+
+ public BTreeTupleFile(StorageManager storageManager,
+ BTreeTupleFileManager btreeFileManager, DBFile dbFile,
+ Schema schema, TableStats stats) {
+ if (storageManager == null)
+ throw new IllegalArgumentException("storageManager cannot be null");
+
+ if (btreeFileManager == null)
+ throw new IllegalArgumentException("btreeFileManager cannot be null");
+
+ if (dbFile == null)
+ throw new IllegalArgumentException("dbFile cannot be null");
+
+ if (schema == null)
+ throw new IllegalArgumentException("schema cannot be null");
+
+ if (stats == null)
+ throw new IllegalArgumentException("stats cannot be null");
+
+ this.storageManager = storageManager;
+ this.btreeFileManager = btreeFileManager;
+ this.dbFile = dbFile;
+ this.schema = schema;
+ this.stats = stats;
+
+ fileOps = new FileOperations(storageManager, dbFile);
+ innerPageOps = new InnerPageOperations(storageManager, this, fileOps);
+ leafPageOps = new LeafPageOperations(storageManager, this, fileOps,
+ innerPageOps);
+ }
+
+
+ @Override
+ public TupleFileManager getManager() {
+ return btreeFileManager;
+ }
+
+
+ @Override
+ public Schema getSchema() {
+ return schema;
+ }
+
+ @Override
+ public TableStats getStats() {
+ return stats;
+ }
+
+
+ public DBFile getDBFile() {
+ return dbFile;
+ }
+
+
+ @Override
+ public List<OrderByExpression> getOrderSpec() {
+ throw new UnsupportedOperationException("NYI");
+ }
+
+
+ @Override
+ public Tuple getFirstTuple() {
+ BTreeFilePageTuple tup = null;
+
+ // By passing a completely empty Tuple (no columns), we can cause the
+ // navigateToLeafPage() method to choose the leftmost leaf page.
+
+ TupleLiteral noTup = new TupleLiteral();
+ LeafPage leaf = navigateToLeafPage(noTup, false, null);
+
+ if (leaf != null && leaf.getNumTuples() > 0)
+ tup = leaf.getTuple(0);
+
+ return tup;
+ }
+
+
+ @Override
+ public Tuple getNextTuple(Tuple tup) {
+ BTreeFilePageTuple tuple = (BTreeFilePageTuple) tup;
+
+ DBPage dbPage;
+ int nextIndex;
+ LeafPage leaf;
+ BTreeFilePageTuple nextTuple = null;
+
+ if (tuple.isDeleted()) {
+ // The tuple was deleted, so we need to find out the page number
+ // and index of the next tuple.
+
+ int nextPageNo = tuple.getNextTuplePageNo();
+ if (nextPageNo != 0) {
+ dbPage = storageManager.loadDBPage(dbFile, nextPageNo);
+ nextIndex = tuple.getNextTupleIndex();
+
+ leaf = new LeafPage(dbPage, schema);
+ if (nextIndex >= leaf.getNumTuples()) {
+ throw new IllegalStateException(String.format(
+ "The \"next tuple\" field of deleted tuple is too " +
+ "large (must be less than %d; got %d)",
+ leaf.getNumTuples(), nextIndex));
+ }
+
+ nextTuple = leaf.getTuple(nextIndex);
+ }
+ }
+ else {
+ // Get the page that holds the current entry, and see where it
+ // falls within the page.
+ dbPage = tuple.getDBPage();
+ leaf = new LeafPage(dbPage, schema);
+
+ // Use the offset of the passed-in entry to find the next entry.
+
+ // The next tuple follows the current tuple, unless the current
+ // tuple was deleted! In that case, the next tuple is actually
+ // where the current tuple used to be.
+ nextIndex = tuple.getTupleIndex() + 1;
+
+ if (nextIndex < leaf.getNumTuples()) {
+ // Still more entries in this leaf.
+ nextTuple = leaf.getTuple(nextIndex);
+ }
+ else {
+ // No more entries in this leaf. Must go to the next leaf.
+ int nextPageNo = leaf.getNextPageNo();
+ if (nextPageNo != 0) {
+ dbPage = storageManager.loadDBPage(dbFile, nextPageNo);
+
+ leaf = new LeafPage(dbPage, schema);
+ if (leaf.getNumTuples() > 0) {
+ nextTuple = leaf.getTuple(0);
+ }
+ else {
+ // This would be *highly* unusual. Leaves are
+ // supposed to be at least 1/2 full, always!
+ logger.error(String.format(
+ "Next leaf node %d has no entries?!", nextPageNo));
+ }
+ }
+ }
+ }
+
+ return nextTuple;
+ }
+
+
+ @Override
+ public Tuple getTuple(FilePointer fptr)
+ throws InvalidFilePointerException {
+
+ DBPage dbPage = storageManager.loadDBPage(dbFile, fptr.getPageNo());
+ if (dbPage == null) {
+ throw new InvalidFilePointerException("Specified page " +
+ fptr.getPageNo() + " doesn't exist in file " + dbFile);
+ }
+
+ // In the B+ tree file format, the file-pointer points to the actual
+ // tuple itself.
+
+ int fpOffset = fptr.getOffset();
+ LeafPage leaf = new LeafPage(dbPage, schema);
+ for (int i = 0; i < leaf.getNumTuples(); i++) {
+ BTreeFilePageTuple tup = leaf.getTuple(i);
+ if (tup.getOffset() == fpOffset)
+ return tup;
+
+ // Tuple offsets within a page will be monotonically increasing.
+ if (tup.getOffset() > fpOffset)
+ break;
+ }
+
+ throw new InvalidFilePointerException("No tuple at offset " + fptr);
+ }
+
+
+ @Override
+ public Tuple findFirstTupleEquals(Tuple searchKey) {
+ logger.debug("Finding first tuple that equals " + searchKey +
+ " in BTree file " + dbFile);
+
+ LeafPage leaf = navigateToLeafPage(searchKey, false, null);
+ if (leaf == null) {
+ // This case is handled by the below loop, but it will make for
+ // more understandable logging.
+ logger.debug("BTree file is empty!");
+ return null;
+ }
+
+ logger.debug("Navigated to leaf page " + leaf.getPageNo());
+ while (leaf != null) {
+ // We have at least one tuple to look at, so scan through to
+ // find the first tuple that equals what we are looking for.
+ for (int i = 0; i < leaf.getNumTuples(); i++) {
+ BTreeFilePageTuple tup = leaf.getTuple(i);
+ int cmp = TupleComparator.comparePartialTuples(tup, searchKey,
+ TupleComparator.CompareMode.IGNORE_LENGTH);
+ // (donnie) This is just way too much logging!
+ // logger.debug("Comparing search key to tuple " + tup +
+ // ", got cmp = " + cmp);
+
+ if (cmp == 0) {
+ // Found it!
+ return tup;
+ }
+ else if (cmp > 0) {
+ // Subsequent tuples will appear after the search key, so
+ // there's no point in going on.
+ leaf.getDBPage().unpin();
+ return null;
+ }
+ }
+
+ int nextPageNo = leaf.getNextPageNo();
+ logger.debug("Scanned through entire leaf page %d without " +
+ "finding tuple. Next page is %d.", leaf.getPageNo(),
+ nextPageNo);
+
+ // If we get here, we need to go to the next leaf-page.
+ leaf.getDBPage().unpin();
+ if (nextPageNo > 0) {
+ DBPage dbpNextLeaf =
+ storageManager.loadDBPage(dbFile, nextPageNo);
+ byte pageType = dbpNextLeaf.readByte(0);
+ if (pageType != BTREE_LEAF_PAGE) {
+ throw new BTreeTupleFileException(String.format(
+ "Expected page %d to be a leaf; found %d instead",
+ nextPageNo, pageType));
+ }
+
+ leaf = new LeafPage(dbpNextLeaf, schema);
+ }
+ else {
+ logger.debug("Reached end of leaf pages");
+ leaf = null;
+ }
+ }
+
+ // If we reach here, we reached the end of the leaf pages without
+ // finding the tuple.
+ return null;
+ }
+
+
+ @Override
+ public PageTuple findFirstTupleGreaterThan(Tuple searchKey) {
+
+ LeafPage leaf = navigateToLeafPage(searchKey, false, null);
+
+ if (leaf != null && leaf.getNumTuples() > 0) {
+ // We have at least one tuple to look at, so scan through to find
+ // the first tuple that equals what we are looking for.
+ for (int i = 0; i < leaf.getNumTuples(); i++) {
+ BTreeFilePageTuple tup = leaf.getTuple(i);
+ int cmp = TupleComparator.comparePartialTuples(tup, searchKey);
+ if (cmp > 0)
+ return tup; // Found it!
+ }
+
+ leaf.getDBPage().unpin();
+ }
+
+ return null;
+ }
+
+
+ @Override
+ public Tuple addTuple(Tuple tup) {
+ logger.debug("Adding tuple " + tup + " to BTree file " + dbFile);
+
+ // Navigate to the leaf-page, creating one if the BTree file is
+ // currently empty.
+ ArrayList<Integer> pagePath = new ArrayList<>();
+ LeafPage leaf = navigateToLeafPage(tup, true, pagePath);
+
+ // TODO: This is definitely not ideal, but should get us going.
+ TupleLiteral tupLit;
+ if (tup instanceof TupleLiteral)
+ tupLit = (TupleLiteral) tup;
+ else
+ tupLit = new TupleLiteral(tup);
+ tupLit.setStorageSize(PageTuple.getTupleStorageSize(schema, tupLit));
+
+ return leafPageOps.addTuple(leaf, tupLit, pagePath);
+ }
+
+
+ @Override
+ public void updateTuple(Tuple tup, Map<String, Object> newValues) {
+
+ throw new UnsupportedOperationException("NYI");
+ }
+
+
+ @Override
+ public void deleteTuple(Tuple tup) {
+ BTreeFilePageTuple tuple = (BTreeFilePageTuple) tup;
+
+ ArrayList<Integer> pagePath = new ArrayList<>();
+ LeafPage leaf = navigateToLeafPage(tup, false, pagePath);
+
+ logger.debug("Deleting tuple " + tuple + " from file " + dbFile);
+
+ leafPageOps.deleteTuple(leaf, tuple, pagePath);
+ tuple.setDeleted();
+ }
+
+
+ /**
+ * This helper method performs the common task of navigating from the root
+ * of the B<sup>+</sup> tree down to the appropriate leaf node, based on
+ * the search-key provided by the caller. Note that this method does not
+ * determine whether the search-key actually exists; rather, it simply
+ * navigates to the leaf in the file where the search-key would appear.
+ *
+ * @param searchKey the search-key being used to navigate the
+ * B<sup>+</sup> tree structure
+ *
+ * @param createIfNeeded If the B<sup>+</sup> tree is currently empty
+ * (i.e. not even containing leaf pages) then this argument can be
+ * used to create a new leaf page where the search-key can be
+ * stored. This allows the method to be used for adding tuples to
+ * the file.
+ *
+ * @param pagePath If this optional argument is specified, then the method
+ * stores the sequence of page-numbers it visits as it navigates
+ * from root to leaf. If {@code null} is passed then nothing is
+ * stored as the method traverses the B<sup>+</sup> tree structure.
+ *
+ * @return the leaf-page where the search-key would appear, or
+ * {@code null} if the B<sup>+</sup> tree file is currently empty
+ * and {@code createIfNeeded} is {@code false}.
+ */
+ private LeafPage navigateToLeafPage(Tuple searchKey,
+ boolean createIfNeeded, List<Integer> pagePath) {
+
+ // The header page tells us where the root page starts.
+ DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
+
+ // Get the root page of the BTree file.
+ int rootPageNo = HeaderPage.getRootPageNo(dbpHeader);
+ DBPage dbpRoot;
+ if (rootPageNo == 0) {
+ // The file doesn't have any data-pages at all yet. Create one if
+ // the caller wants it.
+
+ if (!createIfNeeded)
+ return null;
+
+ // We need to create a brand new leaf page and make it the root.
+
+ logger.debug("BTree file currently has no data pages; " +
+ "finding/creating one to use as the root!");
+
+ dbpRoot = fileOps.getNewDataPage();
+ rootPageNo = dbpRoot.getPageNo();
+
+ HeaderPage.setRootPageNo(dbpHeader, rootPageNo);
+ HeaderPage.setFirstLeafPageNo(dbpHeader, rootPageNo);
+
+ dbpRoot.writeByte(0, BTREE_LEAF_PAGE);
+ LeafPage.init(dbpRoot, schema);
+
+ logger.debug("New root pageNo is " + rootPageNo);
+ }
+ else {
+ // The BTree file has a root page; load it.
+ dbpRoot = storageManager.loadDBPage(dbFile, rootPageNo);
+
+ logger.debug("BTree file root pageNo is " + rootPageNo);
+ }
+
+ // Next, descend down the file's structure until we find the proper
+ // leaf-page based on the key value(s).
+
+ DBPage dbPage = dbpRoot;
+ int pageType = dbPage.readByte(0);
+ if (pageType != BTREE_INNER_PAGE && pageType != BTREE_LEAF_PAGE) {
+ throw new BTreeTupleFileException(
+ "Invalid page type encountered: " + pageType);
+ }
+
+ if (pagePath != null)
+ pagePath.add(rootPageNo);
+
+ /* TODO: IMPLEMENT THE REST OF THIS METHOD.
+ *
+ * Don't forget to update the page-path as you navigate the index
+ * structure, if it is provided by the caller.
+ *
+ * Use the TupleComparator.comparePartialTuples() method for comparing
+ * the index's keys with the passed-in search key.
+ *
+ * It's always a good idea to code defensively: if you see an invalid
+ * page-type, flag it with an IOException, as done earlier.
+ */
+ logger.error("NOT YET IMPLEMENTED: navigateToLeafPage()");
+
+ return null;
+ }
+
+
+ @Override
+ public void analyze() {
+ throw new UnsupportedOperationException("NYI");
+ }
+
+
+ @Override
+ public List<String> verify() {
+ BTreeFileVerifier verifier =
+ new BTreeFileVerifier(storageManager, this);
+
+ return verifier.verify();
+ }
+
+
+ @Override
+ public void optimize() {
+ throw new UnsupportedOperationException("NYI");
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeTupleFileException.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeTupleFileException.java
new file mode 100644
index 0000000000000000000000000000000000000000..3edb458fc9875097a708a59bb79cc7ce064a7fb1
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeTupleFileException.java
@@ -0,0 +1,39 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+import edu.caltech.nanodb.storage.TupleFileException;
+
+
+/**
+ * This class represents errors that occur while manipulating B<sup>+</sup>
+ * tree tuple files.
+ */
+public class BTreeTupleFileException extends TupleFileException {
+
+ /** Construct a tuple-file exception with no message. */
+ public BTreeTupleFileException() {
+ super();
+ }
+
+ /**
+ * Construct a tuple-file exception with the specified message.
+ */
+ public BTreeTupleFileException(String msg) {
+ super(msg);
+ }
+
+
+ /**
+ * Construct a tuple-file exception with the specified cause and no message.
+ */
+ public BTreeTupleFileException(Throwable cause) {
+ super(cause);
+ }
+
+
+ /**
+ * Construct a tuple-file exception with the specified message and cause.
+ */
+ public BTreeTupleFileException(String msg, Throwable cause) {
+ super(msg, cause);
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeTupleFileManager.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeTupleFileManager.java
new file mode 100644
index 0000000000000000000000000000000000000000..dac5dcb5541edf075d10b52cc86e88353a5e732f
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/BTreeTupleFileManager.java
@@ -0,0 +1,122 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import edu.caltech.nanodb.queryeval.TableStats;
+import edu.caltech.nanodb.relations.Schema;
+import edu.caltech.nanodb.storage.DBFile;
+import edu.caltech.nanodb.storage.DBFileType;
+import edu.caltech.nanodb.storage.DBPage;
+import edu.caltech.nanodb.storage.PageReader;
+import edu.caltech.nanodb.storage.PageWriter;
+import edu.caltech.nanodb.storage.SchemaWriter;
+import edu.caltech.nanodb.storage.StatsWriter;
+import edu.caltech.nanodb.storage.StorageManager;
+import edu.caltech.nanodb.storage.TupleFile;
+import edu.caltech.nanodb.storage.TupleFileManager;
+
+
+/**
+ * This class provides high-level operations on B<sup>+</sup> tree tuple files.
+ */
+public class BTreeTupleFileManager implements TupleFileManager {
+ /** A logging object for reporting anything interesting that happens. */
+ private static Logger logger = LogManager.getLogger(BTreeTupleFileManager.class);
+
+
+ /** A reference to the storage manager. */
+ private StorageManager storageManager;
+
+
+ public BTreeTupleFileManager(StorageManager storageManager) {
+ if (storageManager == null)
+ throw new IllegalArgumentException("storageManager cannot be null");
+
+ this.storageManager = storageManager;
+ }
+
+
+ @Override
+ public DBFileType getDBFileType() {
+ return DBFileType.BTREE_TUPLE_FILE;
+ }
+
+
+ @Override
+ public String getShortName() {
+ return "btree";
+ }
+
+
+ @Override
+ public TupleFile createTupleFile(DBFile dbFile, Schema schema) {
+
+ logger.info(String.format(
+ "Initializing new btree tuple file %s with %d columns",
+ dbFile, schema.numColumns()));
+
+ // Table schema is stored into the header page, so get it and prepare
+ // to write out the schema information.
+ DBPage headerPage = storageManager.loadDBPage(dbFile, 0);
+ PageWriter hpWriter = new PageWriter(headerPage);
+ // Skip past the page-size value.
+ hpWriter.setPosition(HeaderPage.OFFSET_SCHEMA_START);
+
+ // Write out the schema details now.
+ SchemaWriter schemaWriter = new SchemaWriter();
+ schemaWriter.writeSchema(schema, hpWriter);
+
+ // Compute and store the schema's size.
+ int schemaEndPos = hpWriter.getPosition();
+ int schemaSize = schemaEndPos - HeaderPage.OFFSET_SCHEMA_START;
+ HeaderPage.setSchemaSize(headerPage, schemaSize);
+
+ // Write in empty statistics, so that the values are at least
+ // initialized to something.
+ TableStats stats = new TableStats(schema.numColumns());
+ StatsWriter.writeTableStats(schema, stats, hpWriter);
+ int statsSize = hpWriter.getPosition() - schemaEndPos;
+ HeaderPage.setStatsSize(headerPage, statsSize);
+
+ return new BTreeTupleFile(storageManager, this, dbFile, schema, stats);
+ }
+
+
+ @Override
+ public TupleFile openTupleFile(DBFile dbFile) {
+
+ logger.info("Opening existing btree tuple file " + dbFile);
+
+ // Table schema is stored into the header page, so get it and prepare
+ // to write out the schema information.
+ DBPage headerPage = storageManager.loadDBPage(dbFile, 0);
+ PageReader hpReader = new PageReader(headerPage);
+ // Skip past the page-size value.
+ hpReader.setPosition(HeaderPage.OFFSET_SCHEMA_START);
+
+ // Read in the schema details.
+ SchemaWriter schemaWriter = new SchemaWriter();
+ Schema schema = schemaWriter.readSchema(hpReader);
+
+ // Read in the statistics.
+ TableStats stats = StatsWriter.readTableStats(hpReader, schema);
+
+ return new BTreeTupleFile(storageManager, this, dbFile, schema, stats);
+ }
+
+
+ @Override
+ public void saveMetadata(TupleFile tupleFile) {
+ // TODO
+ throw new UnsupportedOperationException("NYI: deleteTupleFile()");
+ }
+
+
+ @Override
+ public void deleteTupleFile(TupleFile tupleFile) {
+ // TODO
+ throw new UnsupportedOperationException("NYI: deleteTupleFile()");
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/DataPage.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/DataPage.java
new file mode 100644
index 0000000000000000000000000000000000000000..09bf037a8d4e7259f6c8e246e1d264120bbb6da9
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/DataPage.java
@@ -0,0 +1,14 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+/**
+ * Created with IntelliJ IDEA.
+ * User: donnie
+ * Date: 2/4/13
+ * Time: 6:51 PM
+ * To change this template use File | Settings | File Templates.
+ */
+public interface DataPage {
+ /** The page type always occupies the first byte of the page. */
+ public static final int OFFSET_PAGE_TYPE = 0;
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/FileOperations.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/FileOperations.java
new file mode 100644
index 0000000000000000000000000000000000000000..705476e54956d0030cb49f4ba2360f12c8fb65f7
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/FileOperations.java
@@ -0,0 +1,103 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import edu.caltech.nanodb.storage.DBFile;
+import edu.caltech.nanodb.storage.DBPage;
+import edu.caltech.nanodb.storage.StorageManager;
+
+
+/**
+ * This class provides file-level operations for the B<sup>+</sup> tree
+ * implementation, such as acquiring a new data page or releasing a data page.
+ */
+class FileOperations {
+ /** A logging object for reporting anything interesting that happens. */
+ private static Logger logger = LogManager.getLogger(FileOperations.class);
+
+ /** The storage manager to use for loading pages. */
+ private StorageManager storageManager;
+
+ /** The {@code DBFile} that the B<sup>+</sup> tree is stored in. */
+ private DBFile dbFile;
+
+
+ public FileOperations(StorageManager storageManager, DBFile dbFile) {
+ this.storageManager = storageManager;
+ this.dbFile = dbFile;
+ }
+
+
+ /**
+ * This helper function finds and returns a new data page, either by
+ * taking it from the empty-pages list in the file, or if the list is
+ * empty, creating a brand new page at the end of the file.
+ *
+ * @return an empty {@code DBPage} that can be used for storing tuple data
+ */
+ public DBPage getNewDataPage() {
+ DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
+
+ DBPage newPage;
+ int pageNo = HeaderPage.getFirstEmptyPageNo(dbpHeader);
+
+ if (pageNo == 0) {
+ // There are no empty pages. Create a new page to use.
+
+ logger.debug("No empty pages. Extending BTree file " + dbFile +
+ " by one page.");
+
+ int numPages = dbFile.getNumPages();
+ newPage = storageManager.loadDBPage(dbFile, numPages, true);
+ }
+ else {
+ // Load the empty page, and remove it from the chain of empty pages.
+
+ logger.debug("First empty page number is " + pageNo);
+
+ newPage = storageManager.loadDBPage(dbFile, pageNo);
+ int nextEmptyPage = newPage.readUnsignedShort(1);
+ HeaderPage.setFirstEmptyPageNo(dbpHeader, nextEmptyPage);
+ }
+
+ logger.debug("Found data page to use: page " + newPage.getPageNo());
+
+ // TODO: Increment the number of data pages?
+
+ return newPage;
+ }
+
+
+ /**
+ * This helper function marks a data page in the B<sup>+</sup> tree file
+ * as "empty", and adds it to the list of empty pages in the file.
+ *
+ * @param dbPage the data-page that is no longer used.
+ */
+ public void releaseDataPage(DBPage dbPage) {
+ // TODO: If this page is the last page of the index file, we could
+ // truncate pages off the end until we hit a non-empty page.
+ // Instead, we'll leave all the pages around forever...
+
+ DBFile dbFile = dbPage.getDBFile();
+
+ // Record in the page that it is empty.
+ dbPage.writeByte(0, BTreePageTypes.BTREE_EMPTY_PAGE);
+
+ DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
+
+ // Retrieve the old "first empty page" value, and store it in this page.
+ int prevEmptyPageNo = HeaderPage.getFirstEmptyPageNo(dbpHeader);
+ dbPage.writeShort(1, prevEmptyPageNo);
+
+ if (BTreeTupleFile.CLEAR_OLD_DATA) {
+ // Clear out the remainder of the data-page since it's now unused.
+ dbPage.setDataRange(3, dbPage.getPageSize() - 3, (byte) 0);
+ }
+
+ // Store the new "first empty page" value into the header.
+ HeaderPage.setFirstEmptyPageNo(dbpHeader, dbPage.getPageNo());
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/HeaderPage.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/HeaderPage.java
new file mode 100644
index 0000000000000000000000000000000000000000..62f35aa3fbd8c35ddb0562e54de0636ce671650e
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/HeaderPage.java
@@ -0,0 +1,272 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import edu.caltech.nanodb.storage.DBFileType;
+import edu.caltech.nanodb.storage.DBPage;
+
+
+/**
+ * This class manipulates the header page for a B<sup>+</sup> tree index file.
+ * The header page has the following structure:
+ *
+ * <ul>
+ * <li><u>Byte 0:</u> {@link DBFileType#BTREE_TUPLE_FILE} (unsigned byte)</li>
+ * <li><u>Byte 1:</u> page size <i>p</i> (unsigned byte) - file's page
+ * size is <i>P</i> = 2<sup>p</sup></li>
+ *
+ * <li>Byte 2-M: Specification of index key-columns and column ordering.</li>
+ * <li>Byte P-2 to P-1: the page of the file that is the root of the index</li>
+ * </ul>
+ */
+public class HeaderPage {
+ /** A logging object for reporting anything interesting that happens. */
+ private static Logger logger = LogManager.getLogger(HeaderPage.class);
+
+
+ /**
+ * The offset in the header page where the page number of the index's root
+ * page is stored. This value is an unsigned short.
+ */
+ public static final int OFFSET_ROOT_PAGE = 2;
+
+
+ /**
+ * The offset in the header page where the page number of the first leaf
+ * page of the file is stored. This allows the leaves of the tuple file
+ * to be iterated through in sequential order. This value is an unsigned
+ * short.
+ */
+ public static final int OFFSET_FIRST_LEAF_PAGE = 4;
+
+
+ /**
+ * The offset in the header page where the page number of the first empty
+ * page in the free list is stored. This value is an unsigned short.
+ */
+ public static final int OFFSET_FIRST_EMPTY_PAGE = 6;
+
+
+ /**
+ * The offset in the header page where the length of the file's schema is
+ * stored. The statistics follow immediately after the schema.
+ */
+ public static final int OFFSET_SCHEMA_SIZE = 8;
+
+
+ /**
+ * The offset in the header page where the size of the table statistics
+ * are stored. This value is an unsigned short.
+ */
+ public static final int OFFSET_STATS_SIZE = 10;
+
+
+ /**
+ * The offset in the header page where the table schema starts. This
+ * value is an unsigned short.
+ */
+ public static final int OFFSET_SCHEMA_START = 12;
+
+
+ /**
+ * This helper method simply verifies that the data page provided to the
+ * <tt>HeaderPage</tt> class is in fact a header-page (i.e. page 0 in the
+ * data file).
+ *
+ * @param dbPage the page to check
+ *
+ * @throws IllegalArgumentException if <tt>dbPage</tt> is <tt>null</tt>, or
+ * if it's not actually page 0 in the table file
+ */
+ private static void verifyIsHeaderPage(DBPage dbPage) {
+ if (dbPage == null)
+ throw new IllegalArgumentException("dbPage cannot be null");
+
+ if (dbPage.getPageNo() != 0) {
+ throw new IllegalArgumentException(
+ "Page 0 is the header page in this storage format; was given page " +
+ dbPage.getPageNo());
+ }
+ }
+
+
+ /**
+ * Returns the page-number of the root page in the index file.
+ *
+ * @param dbPage the header page of the index file
+ * @return the page-number of the root page, or 0 if the index file doesn't
+ * contain a root page.
+ */
+ public static int getRootPageNo(DBPage dbPage) {
+ verifyIsHeaderPage(dbPage);
+ return dbPage.readUnsignedShort(OFFSET_ROOT_PAGE);
+ }
+
+
+ /**
+ * Sets the page-number of the root page in the header page of the index
+ * file.
+ *
+ * @param dbPage the header page of the heap table file
+ * @param rootPageNo the page-number of the root page, or 0 if the index
+ * file doesn't contain a root page.
+ */
+ public static void setRootPageNo(DBPage dbPage, int rootPageNo) {
+ verifyIsHeaderPage(dbPage);
+
+ if (rootPageNo < 0) {
+ throw new IllegalArgumentException(
+ "rootPageNo must be > 0; got " + rootPageNo);
+ }
+
+ dbPage.writeShort(OFFSET_ROOT_PAGE, rootPageNo);
+ }
+
+
+ /**
+ * Returns the page-number of the first leaf page in the index file.
+ *
+ * @param dbPage the header page of the index file
+ * @return the page-number of the first leaf page in the index file, or 0
+ * if the index file doesn't contain any leaf pages.
+ */
+ public static int getFirstLeafPageNo(DBPage dbPage) {
+ verifyIsHeaderPage(dbPage);
+ return dbPage.readUnsignedShort(OFFSET_FIRST_LEAF_PAGE);
+ }
+
+
+ /**
+ * Sets the page-number of the first leaf page in the header page of the
+ * index file.
+ *
+ * @param dbPage the header page of the heap table file
+ * @param firstLeafPageNo the page-number of the first leaf page in the
+ * index file, or 0 if the index file doesn't contain any leaf pages.
+ */
+ public static void setFirstLeafPageNo(DBPage dbPage, int firstLeafPageNo) {
+ verifyIsHeaderPage(dbPage);
+
+ if (firstLeafPageNo < 0) {
+ throw new IllegalArgumentException(
+ "firstLeafPageNo must be >= 0; got " + firstLeafPageNo);
+ }
+
+ dbPage.writeShort(OFFSET_FIRST_LEAF_PAGE, firstLeafPageNo);
+ }
+
+
+ /**
+ * Returns the page-number of the first empty page in the index file.
+ * Empty pages form a linked chain in the index file, so that they are
+ * easy to locate.
+ *
+ * @param dbPage the header page of the index file
+ * @return the page-number of the last leaf page in the index file.
+ */
+ public static int getFirstEmptyPageNo(DBPage dbPage) {
+ verifyIsHeaderPage(dbPage);
+ return dbPage.readUnsignedShort(OFFSET_FIRST_EMPTY_PAGE);
+ }
+
+
+ /**
+ * Sets the page-number of the first empty page in the header page of the
+ * index file. Empty pages form a linked chain in the index file, so that
+ * they are easy to locate.
+ *
+ * @param dbPage the header page of the heap table file
+ * @param firstEmptyPageNo the page-number of the first empty page
+ */
+ public static void setFirstEmptyPageNo(DBPage dbPage, int firstEmptyPageNo) {
+ verifyIsHeaderPage(dbPage);
+
+ if (firstEmptyPageNo < 0) {
+ throw new IllegalArgumentException(
+ "firstEmptyPageNo must be >= 0; got " + firstEmptyPageNo);
+ }
+
+ dbPage.writeShort(OFFSET_FIRST_EMPTY_PAGE, firstEmptyPageNo);
+ }
+
+
+ /**
+ * Returns the number of bytes that the table's schema occupies for storage
+ * in the header page.
+ *
+ * @param dbPage the header page of the heap table file
+ * @return the number of bytes that the table's schema occupies
+ */
+ public static int getSchemaSize(DBPage dbPage) {
+ verifyIsHeaderPage(dbPage);
+ return dbPage.readUnsignedShort(OFFSET_SCHEMA_SIZE);
+ }
+
+
+ /**
+ * Sets the number of bytes that the table's schema occupies for storage
+ * in the header page.
+ *
+ * @param dbPage the header page of the heap table file
+ * @param numBytes the number of bytes that the table's schema occupies
+ */
+ public static void setSchemaSize(DBPage dbPage, int numBytes) {
+ verifyIsHeaderPage(dbPage);
+
+ if (numBytes < 0) {
+ throw new IllegalArgumentException(
+ "numButes must be >= 0; got " + numBytes);
+ }
+
+ dbPage.writeShort(OFFSET_SCHEMA_SIZE, numBytes);
+ }
+
+
+ /**
+ * Returns the number of bytes that the table's statistics occupy for
+ * storage in the header page.
+ *
+ * @param dbPage the header page of the heap table file
+ * @return the number of bytes that the table's statistics occupy
+ */
+ public static int getStatsSize(DBPage dbPage) {
+ verifyIsHeaderPage(dbPage);
+ return dbPage.readUnsignedShort(OFFSET_STATS_SIZE);
+ }
+
+
+ /**
+ * Sets the number of bytes that the table's statistics occupy for storage
+ * in the header page.
+ *
+ * @param dbPage the header page of the heap table file
+ * @param numBytes the number of bytes that the table's statistics occupy
+ */
+ public static void setStatsSize(DBPage dbPage, int numBytes) {
+ verifyIsHeaderPage(dbPage);
+
+ if (numBytes < 0) {
+ throw new IllegalArgumentException(
+ "numButes must be >= 0; got " + numBytes);
+ }
+
+ dbPage.writeShort(OFFSET_STATS_SIZE, numBytes);
+ }
+
+
+ /**
+ * Returns the offset in the header page that the table statistics start at.
+ * This value changes because the table schema resides before the stats, and
+ * therefore the stats don't live at a fixed location.
+ *
+ * @param dbPage the header page of the heap table file
+ * @return the offset within the header page that the table statistics
+ * reside at
+ */
+ public static int getStatsOffset(DBPage dbPage) {
+ verifyIsHeaderPage(dbPage);
+ return OFFSET_SCHEMA_START + getSchemaSize(dbPage);
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/InnerPage.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/InnerPage.java
new file mode 100644
index 0000000000000000000000000000000000000000..afaa09eadaf6f7c88712523171a9065d785cab39
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/InnerPage.java
@@ -0,0 +1,998 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import java.util.List;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import edu.caltech.nanodb.expressions.TupleLiteral;
+import edu.caltech.nanodb.relations.Schema;
+import edu.caltech.nanodb.relations.Tuple;
+import edu.caltech.nanodb.storage.DBPage;
+import edu.caltech.nanodb.storage.PageTuple;
+
+import static edu.caltech.nanodb.storage.btreefile.BTreePageTypes.*;
+
+
+/**
+ * <p>
+ * This class wraps a {@link DBPage} object that is an inner page in the
+ * B<sup>+</sup> tree file implementation, to provide some of the basic
+ * inner-page-management operations necessary for the file structure.
+ * </p>
+ * <p>
+ * Operations involving individual leaf-pages are provided by the
+ * {@link LeafPage} wrapper-class. Higher-level operations involving multiple
+ * leaves and/or inner pages of the B<sup>+</sup> tree structure, are provided
+ * by the {@link LeafPageOperations} and {@link InnerPageOperations} classes.
+ * </p>
+ */
+public class InnerPage implements DataPage {
+ /** A logging object for reporting anything interesting that happens. */
+ private static Logger logger = LogManager.getLogger(InnerPage.class);
+
+
+ /**
+ * The offset where the number of pointer entries is stored in the page.
+ * The page will hold one fewer tuples than pointers, since each tuple
+ * must be sandwiched between two pointers.
+ */
+ public static final int OFFSET_NUM_POINTERS = 3;
+
+
+ /** The offset of the first pointer in the non-leaf page. */
+ public static final int OFFSET_FIRST_POINTER = 5;
+
+
+ /** The actual data page that holds the B<sup>+</sup> tree inner node. */
+ private DBPage dbPage;
+
+
+ /** The schema of the tuples in the leaf page. */
+ private Schema schema;
+
+
+ /** The number of pointers stored within this non-leaf page. */
+ private int numPointers;
+
+
+ /**
+ * An array of the offsets where the pointers are stored in this non-leaf
+ * page. Each pointer points to another page within the file. There is
+ * one more pointer than the number of tuples, since each tuple must be
+ * sandwiched between two pointers.
+ */
+ private int[] pointerOffsets;
+
+
+ /** An array of the tuples stored in this non-leaf page. */
+ private BTreeFilePageTuple[] keys;
+
+
+ /**
+ * The total size of all data (pointers + tuples + initial values) stored
+ * within this non-leaf page. This is also the offset at which we can
+ * start writing more data without overwriting anything.
+ */
+ private int endOffset;
+
+
+ /**
+ * Initialize the inner-page wrapper class for the specified B<sup>+</sup>
+ * tree leaf page. The contents of the inner-page are cached in the
+ * fields of the wrapper object.
+ *
+ * @param dbPage the data page from the B<sup>+</sup> Tree file to wrap
+ * @param schema the schema of tuples stored in the data page
+ */
+ public InnerPage(DBPage dbPage, Schema schema) {
+ if (dbPage.readUnsignedByte(0) != BTREE_INNER_PAGE) {
+ throw new IllegalArgumentException("Specified DBPage " +
+ dbPage.getPageNo() + " is not marked as an inner page.");
+ }
+
+ this.dbPage = dbPage;
+ this.schema = schema;
+
+ loadPageContents();
+ }
+
+
+ /**
+ * This static helper function initializes a {@link DBPage} object's
+ * contents with the type and detail values that will allow a new
+ * {@code InnerPage} wrapper to be instantiated for the page, and then it
+ * returns a wrapper object for the page. This version of the {@code init}
+ * function creates an inner page that is initially empty.
+ *
+ * @param dbPage the page to initialize as an inner page.
+ *
+ * @param schema the schema of the tuples in the leaf page
+ *
+ * @return a newly initialized {@code InnerPage} object wrapping the page
+ */
+ public static InnerPage init(DBPage dbPage, Schema schema) {
+ dbPage.writeByte(OFFSET_PAGE_TYPE, BTREE_INNER_PAGE);
+ dbPage.writeShort(OFFSET_NUM_POINTERS, 0);
+
+ return new InnerPage(dbPage, schema);
+ }
+
+
+ /**
+ * This static helper function initializes a {@link DBPage} object's
+ * contents with the type and detail values that will allow a new
+ * {@code InnerPage} wrapper to be instantiated for the page, and then it
+ * returns a wrapper object for the page. This version of the {@code init}
+ * function creates an inner page that initially contains the specified
+ * page-pointers and key value.
+ *
+ * @param dbPage the page to initialize as an inner page.
+ *
+ * @param schema the schema of the tuples in the inner page
+ *
+ * @param pagePtr1 the first page-pointer to store in the inner page, to the
+ * left of {@code key1}
+ *
+ * @param key1 the first key to store in the inner page
+ *
+ * @param pagePtr2 the second page-pointer to store in the inner page, to
+ * the right of {@code key1}
+ *
+ * @return a newly initialized {@code InnerPage} object wrapping the page
+ */
+ public static InnerPage init(DBPage dbPage, Schema schema,
+ int pagePtr1, Tuple key1, int pagePtr2) {
+
+ dbPage.writeByte(OFFSET_PAGE_TYPE, BTREE_INNER_PAGE);
+
+ // Write the first contents of the non-leaf page: [ptr0, key0, ptr1]
+ // Since key0 will usually be a BTreeFilePageTuple, we have to rely on
+ // the storeTuple() method to tell us where the new tuple's data ends.
+
+ int offset = OFFSET_FIRST_POINTER;
+
+ dbPage.writeShort(offset, pagePtr1);
+ offset += 2;
+
+ offset = PageTuple.storeTuple(dbPage, offset, schema, key1);
+
+ dbPage.writeShort(offset, pagePtr2);
+
+ dbPage.writeShort(OFFSET_NUM_POINTERS, 2);
+
+ return new InnerPage(dbPage, schema);
+ }
+
+
+ /**
+ * This private helper scans through the inner page's contents and caches
+ * the contents of the inner page in a way that makes it easy to use and
+ * manipulate.
+ */
+ private void loadPageContents() {
+ numPointers = dbPage.readUnsignedShort(OFFSET_NUM_POINTERS);
+ if (numPointers > 0) {
+ pointerOffsets = new int[numPointers];
+ keys = new BTreeFilePageTuple[numPointers - 1];
+
+ // Handle first pointer + key separately since we know their offsets
+
+ pointerOffsets[0] = OFFSET_FIRST_POINTER;
+
+ if (numPointers == 1) {
+ // This will happen when we are deleting values from a page.
+ // No keys, just 1 pointer, done!
+ endOffset = OFFSET_FIRST_POINTER + 2;
+ return;
+ }
+
+ BTreeFilePageTuple key = new BTreeFilePageTuple(schema, dbPage,
+ OFFSET_FIRST_POINTER + 2, 0);
+ keys[0] = key;
+
+ // Handle all the pointer/key pairs. This excludes the last
+ // pointer.
+
+ int keyEndOffset;
+ for (int i = 1; i < numPointers - 1; i++) {
+ // Next pointer starts where the previous key ends.
+ keyEndOffset = key.getEndOffset();
+ pointerOffsets[i] = keyEndOffset;
+
+ // Next key starts after the next pointer.
+ key = new BTreeFilePageTuple(schema, dbPage, keyEndOffset + 2, i);
+ keys[i] = key;
+ }
+
+ keyEndOffset = key.getEndOffset();
+ pointerOffsets[numPointers - 1] = keyEndOffset;
+ endOffset = keyEndOffset + 2;
+ }
+ else {
+ // There are no entries (pointers + keys).
+ endOffset = OFFSET_FIRST_POINTER;
+ pointerOffsets = null;
+ keys = null;
+ }
+ }
+
+
+ /**
+ * Returns the {@code DBPage} that backs this leaf page.
+ *
+ * @return the {@code DBPage} that backs this leaf page.
+ */
+ public DBPage getDBPage() {
+ return dbPage;
+ }
+
+
+ /**
+ * Returns the page-number of this leaf page.
+ *
+ * @return the page-number of this leaf page.
+ */
+ public int getPageNo() {
+ return dbPage.getPageNo();
+ }
+
+
+ /**
+ * Given a leaf page in the index, returns the page number of the left
+ * sibling, or -1 if there is no left sibling to this node.
+ *
+ * @param pagePath the page path from root to this leaf page
+ * @param innerOps the inner page ops that allows this method to
+ * load inner pages and navigate the tree
+ *
+ * @return the page number of the left sibling leaf-node, or -1 if there
+ * is no left sibling
+ *
+ * @review (Donnie) There is a lot of implementation-overlap between this
+ * function and the {@link InnerPage#getLeftSibling}. Maybe find
+ * a way to combine the implementations.
+ */
+ public int getLeftSibling(List<Integer> pagePath,
+ InnerPageOperations innerOps) {
+
+ // Verify that the last node in the page path is in fact this page.
+ if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
+ throw new IllegalArgumentException(
+ "The page path provided does not terminate on this leaf page.");
+ }
+
+ // If this leaf doesn't have a parent, we already know it doesn't
+ // have a sibling.
+ if (pagePath.size() <= 1)
+ return -1;
+
+ int parentPageNo = pagePath.get(pagePath.size() - 2);
+ InnerPage inner = innerOps.loadPage(parentPageNo);
+
+ // Get the index of the pointer that points to this page. If it
+ // doesn't appear in the parent, we have a serious problem...
+ int pageIndex = inner.getIndexOfPointer(getPageNo());
+ if (pageIndex == -1) {
+ throw new IllegalStateException(String.format(
+ "Leaf node %d doesn't appear in parent inner node %d!",
+ getPageNo(), parentPageNo));
+ }
+
+ int leftSiblingIndex = pageIndex - 1;
+ int leftSiblingPageNo = -1;
+
+ if (leftSiblingIndex >= 0)
+ leftSiblingPageNo = inner.getPointer(leftSiblingIndex);
+
+ return leftSiblingPageNo;
+ }
+
+
+ /**
+ * Given a leaf page in the index, returns the page number of the right
+ * sibling, or -1 if there is no right sibling to this node.
+ *
+ * @param pagePath the page path from root to this leaf page
+ * @param innerOps the inner page ops that allows this method to
+ * load inner pages and navigate the tree
+ *
+ * @return the page number of the right sibling leaf-node, or -1 if there
+ * is no right sibling
+ *
+ * @review (Donnie) There is a lot of implementation-overlap between this
+ * function and the {@link InnerPage#getLeftSibling}. Maybe find
+ * a way to combine the implementations.
+ */
+ public int getRightSibling(List<Integer> pagePath,
+ InnerPageOperations innerOps) {
+
+ // Verify that the last node in the page path is in fact this page.
+ if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
+ throw new IllegalArgumentException(
+ "The page path provided does not terminate on this inner page.");
+ }
+
+ // If this leaf doesn't have a parent, we already know it doesn't
+ // have a sibling.
+ if (pagePath.size() <= 1)
+ return -1;
+
+ int parentPageNo = pagePath.get(pagePath.size() - 2);
+ InnerPage inner = innerOps.loadPage(parentPageNo);
+
+ // Get the index of the pointer that points to this page. If it
+ // doesn't appear in the parent, we have a serious problem...
+ int pageIndex = inner.getIndexOfPointer(getPageNo());
+ if (pageIndex == -1) {
+ throw new IllegalStateException(String.format(
+ "Inner node %d doesn't appear in parent inner node %d!",
+ getPageNo(), parentPageNo));
+ }
+
+ int rightSiblingIndex = pageIndex + 1;
+ int rightSiblingPageNo = -1;
+
+ if (rightSiblingIndex < inner.getNumPointers())
+ rightSiblingPageNo = inner.getPointer(rightSiblingIndex);
+
+ return rightSiblingPageNo;
+ }
+
+
+ /**
+ * Returns the number of pointers currently stored in this inner page. The
+ * number of keys is always one less than the number of pointers, since
+ * each key must have a pointer on both sides.
+ *
+ * @return the number of pointers in this inner page.
+ */
+ public int getNumPointers() {
+ return numPointers;
+ }
+
+
+ /**
+ * Returns the number of keys currently stored in this inner page. The
+ * number of keys is always one less than the number of pointers, since
+ * each key must have a pointer on both sides.
+ *
+ * @return the number of keys in this inner page.
+ *
+ * @throws IllegalStateException if the inner page contains 0 pointers
+ */
+ public int getNumKeys() {
+ if (numPointers < 1) {
+ throw new IllegalStateException("Inner page contains no " +
+ "pointers. Number of keys is meaningless.");
+ }
+
+ return numPointers - 1;
+ }
+
+
+ /**
+ * Returns the total amount of space used in this page, in bytes.
+ *
+ * @return the total amount of space used in this page, in bytes.
+ */
+ public int getUsedSpace() {
+ return endOffset;
+ }
+
+
+ /**
+ * Returns the amount of space used by key/pointer entries in this page,
+ * in bytes.
+ *
+ * @return the amount of space used by key/pointer entries in this page,
+ * in bytes.
+ */
+ public int getSpaceUsedByEntries() {
+ return endOffset - OFFSET_FIRST_POINTER;
+ }
+
+
+ /**
+ * Returns the amount of space available in this inner page, in bytes.
+ *
+ * @return the amount of space available in this inner page, in bytes.
+ */
+ public int getFreeSpace() {
+ return dbPage.getPageSize() - endOffset;
+ }
+
+
+ /**
+ * Returns the total space (page size) in bytes.
+ *
+ * @return the size of the page, in bytes.
+ */
+ public int getTotalSpace() {
+ return dbPage.getPageSize();
+ }
+
+
+ /**
+ * Returns the pointer at the specified index.
+ *
+ * @param index the index of the pointer to retrieve
+ *
+ * @return the pointer at that index
+ */
+ public int getPointer(int index) {
+ return dbPage.readUnsignedShort(pointerOffsets[index]);
+ }
+
+
+ /**
+ * Replaces one page-pointer in the inner page with another page-pointer.
+ * This is used when the B<sup>+</sup> tree is being optimized, as the
+ * layout of the data on disk is rearranged.
+ *
+ * @param index the index of the pointer to replace
+ * @param newPageNo the page number to store at the specified index
+ */
+ public void replacePointer(int index, int newPageNo) {
+ dbPage.writeShort(pointerOffsets[index], newPageNo);
+ loadPageContents();
+ }
+
+
+ /**
+ * Returns the key at the specified index.
+ *
+ * @param index the index of the key to retrieve
+ *
+ * @return the key at that index
+ */
+ public BTreeFilePageTuple getKey(int index) {
+ return keys[index];
+ }
+
+
+ /**
+ * This helper method scans the inner page for the specified page-pointer,
+ * returning the index of the pointer if it is found, or -1 if the pointer
+ * is not found.
+ *
+ * @param pointer the page-pointer to find in this inner page
+ *
+ * @return the index of the page-pointer if found, or -1 if not found
+ */
+ public int getIndexOfPointer(int pointer) {
+ for (int i = 0; i < getNumPointers(); i++) {
+ if (getPointer(i) == pointer)
+ return i;
+ }
+
+ return -1;
+ }
+
+
+ public void replaceTuple(int index, Tuple key) {
+ int oldStart = keys[index].getOffset();
+ int oldLen = keys[index].getEndOffset() - oldStart;
+
+ int newLen = PageTuple.getTupleStorageSize(schema, key);
+
+ if (newLen != oldLen) {
+ // Need to adjust the amount of space the key takes.
+
+ if (endOffset + newLen - oldLen > dbPage.getPageSize()) {
+ throw new IllegalArgumentException(
+ "New key-value is too large to fit in non-leaf page.");
+ }
+
+ dbPage.moveDataRange(oldStart + oldLen, oldStart + newLen,
+ endOffset - oldStart - oldLen);
+ }
+
+ PageTuple.storeTuple(dbPage, oldStart, schema, key);
+
+ // Reload the page contents.
+ // TODO: This is slow, but it should be fine for now.
+ loadPageContents();
+ }
+
+
+ /**
+ * This method inserts a new key and page-pointer into the inner page,
+ * immediately following the page-pointer {@code pagePtr1}, which must
+ * already appear within the page. The caller is expected to have already
+ * verified that the new key and page-pointer are able to fit in the page.
+ *
+ * @param pagePtr1 the page-pointer which should appear before the new key
+ * in the inner page. <b>This is required to already appear within
+ * the inner page.</b>
+ *
+ * @param key1 the new key to add to the inner page, immediately after the
+ * {@code pagePtr1} value.
+ *
+ * @param pagePtr2 the new page-pointer to add to the inner page,
+ * immediately after the {@code key1} value.
+ *
+ * @throws IllegalArgumentException if the specified {@code pagePtr1} value
+ * cannot be found in the inner page, or if the new key and
+ * page-pointer won't fit within the space available in the page.
+ */
+ public void addEntry(int pagePtr1, Tuple key1, int pagePtr2) {
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Non-leaf page " + getPageNo() +
+ " contents before adding entry:\n" + toFormattedString());
+ }
+
+ int i;
+ for (i = 0; i < numPointers; i++) {
+ if (getPointer(i) == pagePtr1)
+ break;
+ }
+
+ logger.debug(String.format("Found page-pointer %d in index %d",
+ pagePtr1, i));
+
+ if (i == numPointers) {
+ throw new IllegalArgumentException(
+ "Can't find initial page-pointer " + pagePtr1 +
+ " in non-leaf page " + getPageNo());
+ }
+
+ // Figure out where to insert the new key and value.
+
+ int oldKeyStart;
+ if (i < numPointers - 1) {
+ // There's a key i associated with pointer i. Use the key's offset,
+ // since it's after the pointer.
+ oldKeyStart = keys[i].getOffset();
+ }
+ else {
+ // The pageNo1 pointer is the last pointer in the sequence. Use
+ // the end-offset of the data in the page.
+ oldKeyStart = endOffset;
+ }
+ int len = endOffset - oldKeyStart;
+
+ // Compute the size of the new key and pointer, and make sure they fit
+ // into the page.
+
+ int newKeySize = PageTuple.getTupleStorageSize(schema, key1);
+ int newEntrySize = newKeySize + 2;
+ if (endOffset + newEntrySize > dbPage.getPageSize()) {
+ throw new IllegalArgumentException("New key-value and " +
+ "page-pointer are too large to fit in non-leaf page.");
+ }
+
+ if (len > 0) {
+ // Move the data after the pageNo1 pointer to make room for
+ // the new key and pointer.
+ dbPage.moveDataRange(oldKeyStart, oldKeyStart + newEntrySize, len);
+ }
+
+ // Write in the new key/pointer values.
+ PageTuple.storeTuple(dbPage, oldKeyStart, schema, key1);
+ dbPage.writeShort(oldKeyStart + newKeySize, pagePtr2);
+
+ // Finally, increment the number of pointers in the page, then reload
+ // the cached data.
+
+ dbPage.writeShort(OFFSET_NUM_POINTERS, numPointers + 1);
+
+ loadPageContents();
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Non-leaf page " + getPageNo() +
+ " contents after adding entry:\n" + toFormattedString());
+ }
+ }
+
+
+ /**
+ * This function will delete a page-pointer from this inner page, along
+ * with the key either to the left or to the right of the pointer. It is
+ * up to the caller to determine whether the left key or the right key
+ * should be deleted.
+ *
+ * @param pagePtr the page-pointer value to identify and remove
+ *
+ * @param removeRightKey a flag specifying whether the key to the right
+ * ({@code true}) or to the left ({@code false}) should be removed
+ */
+ public void deletePointer(int pagePtr, boolean removeRightKey) {
+ logger.debug("Trying to delete page-pointer " + pagePtr +
+ " from inner page " + getPageNo() + ", and remove the " +
+ (removeRightKey ? "right" : "left") + " key.");
+
+ int ptrIndex = getIndexOfPointer(pagePtr);
+ if (ptrIndex == -1) {
+ throw new IllegalArgumentException(String.format(
+ "Can't find page-pointer %d in inner page %d", pagePtr,
+ dbPage.getPageNo()));
+ }
+
+ if (ptrIndex == 0 && !removeRightKey) {
+ throw new IllegalArgumentException(String.format(
+ "Tried to delete page-pointer %d and the key to the left," +
+ " in inner page %d, but the pointer has no left key.",
+ pagePtr, dbPage.getPageNo()));
+ }
+
+ if (ptrIndex == numPointers - 1 && removeRightKey) {
+ throw new IllegalArgumentException(String.format(
+ "Tried to delete page-pointer %d and the key to the right," +
+ " in inner page %d, but the pointer has no right key.",
+ pagePtr, dbPage.getPageNo()));
+ }
+
+ // Figure out the range of data that must be deleted from the page.
+
+ // Page pointers are 2 bytes.
+ int start = pointerOffsets[ptrIndex];
+ int end = start + 2;
+
+ // We must always remove one key, either the left or the right key.
+ // Expand the data range that we are removing.
+ if (removeRightKey) {
+ // Remove the key to the right of the page-pointer.
+ end = keys[ptrIndex].getEndOffset();
+
+ logger.debug(String.format("Removing right key, with size %d." +
+ " Range being removed is [%d, %d).", keys[ptrIndex].getSize(),
+ start, end));
+ }
+ else {
+ // Remove the key to the left of the page-pointer.
+ start = keys[ptrIndex - 1].getOffset();
+
+ logger.debug(String.format("Removing left key, with size %d." +
+ " Range being removed is [%d, %d).", keys[ptrIndex - 1].getSize(),
+ start, end));
+ }
+
+ logger.debug("Moving inner-page data in range [" + end + ", " +
+ endOffset + ") over by " + (end - start) + " bytes");
+ dbPage.moveDataRange(end, start, endOffset - end);
+
+ // Decrement the total number of pointers.
+ dbPage.writeShort(OFFSET_NUM_POINTERS, numPointers - 1);
+
+ logger.debug("Loading altered page - had " + numPointers +
+ " pointers before delete.");
+ // Load new page.
+ loadPageContents();
+
+ logger.debug("After loading, have " + numPointers + " pointers");
+ }
+
+
+ /**
+ * <p>
+ * This helper function moves the specified number of page-pointers to the
+ * left sibling of this inner node. The data is copied in one shot so that
+ * the transfer will be fast, and the various associated bookkeeping values
+ * in both inner pages are updated.
+ * </p>
+ * <p>
+ * Of course, moving a subset of the page-pointers to a sibling will leave
+ * a key without a pointer on one side; this key is promoted up to the
+ * parent of the inner node. Additionally, an existing parent-key can be
+ * provided by the caller, which should be inserted before the new pointers
+ * being moved into the sibling node.
+ * </p>
+ *
+ * @param leftSibling the left sibling of this inner node in the index file
+ *
+ * @param count the number of pointers to move to the left sibling
+ *
+ * @param parentKey If this inner node and the sibling already have a parent
+ * node, this is the key between the two nodes' page-pointers in the
+ * parent node. If the two nodes don't have a parent (i.e. because
+ * an inner node is being split into two nodes and the depth of the
+ * tree is being increased) then this value will be {@code null}.
+ *
+ * @return the key that should go into the parent node, between the
+ * page-pointers for this node and its sibling
+ *
+ * @todo (Donnie) When support for deletion is added to the index
+ * implementation, we will need to support the case when the incoming
+ * {@code parentKey} is non-{@code null}, but the returned key is
+ * {@code null} because one of the two siblings' pointers will be
+ * removed.
+ */
+ public TupleLiteral movePointersLeft(InnerPage leftSibling, int count,
+ Tuple parentKey) {
+
+ if (count < 0 || count > numPointers) {
+ throw new IllegalArgumentException("count must be in range (0, " +
+ numPointers + "), got " + count);
+ }
+
+ // The parent-key can be null if we are splitting a page into two pages.
+ // However, this situation is only valid if the right sibling is EMPTY.
+ int parentKeyLen = 0;
+ if (parentKey != null) {
+ parentKeyLen = PageTuple.getTupleStorageSize(schema, parentKey);
+ }
+ else {
+ if (leftSibling.getNumPointers() != 0) {
+ throw new IllegalStateException("Cannot move pointers to " +
+ "non-empty sibling if no parent-key is specified!");
+ }
+ }
+
+ /* TODO: IMPLEMENT THE REST OF THIS METHOD.
+ *
+ * You can use PageTuple.storeTuple() to write a key into a DBPage.
+ *
+ * The DBPage.write() method is useful for copying a large chunk of
+ * data from one DBPage to another.
+ *
+ * Your implementation also needs to properly handle the incoming
+ * parent-key, and produce a new parent-key as well.
+ */
+ logger.error("NOT YET IMPLEMENTED: movePointersLeft()");
+
+ // Update the cached info for both non-leaf pages.
+ loadPageContents();
+ leftSibling.loadPageContents();
+
+ return null;
+ }
+
+
+ /**
+ * Returns the page path to the right sibling, including the right sibling
+ * itself. Empty list if there is none.
+ *
+ * @param pagePath the page path from root to this leaf
+ * @param innerOps the inner page ops that allows this method to
+ * load inner pages and navigate the tree
+ *
+ * @return the page path to the right sibling leaf node
+ *
+ public List<Integer> getRightSibling(List<Integer> pagePath,
+ InnerPageOperations innerOps) {
+ // Verify that the last node in the page path is this leaf page.
+ if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
+ throw new IllegalArgumentException("The page path provided does" +
+ " not terminate on this leaf page.");
+ }
+
+ ArrayList<Integer> rightPath = new ArrayList<Integer>();
+
+ InnerPage inner = null;
+ int index = 0;
+ int i = pagePath.size() - 2;
+ try {
+ while (i >= 0) {
+ inner = innerOps.loadPage(idxFileInfo, pagePath.get(i));
+ index = inner.getIndexOfPointer(pagePath.get(i+1));
+ if (index != inner.getNumPointers() - 1) {
+ // This means that the subtree this leaf is in has a right
+ // sibling subtree from the current inner node.
+ rightPath.addAll(pagePath.subList(0, i+1));
+ break;
+ }
+ i--;
+ }
+
+ int nextPage;
+ if (inner == null || i == -1) {
+ return rightPath;
+ }
+
+ // Add to the rightPath the page corresponding to one to the
+ // right of the current index
+ rightPath.add(inner.getPointer(index + 1));
+ i++;
+
+ while (i <= pagePath.size() - 2) {
+ index = 0;
+ nextPage = inner.getPointer(index);
+ rightPath.add(nextPage);
+ inner = innerOps.loadPage(idxFileInfo, nextPage);
+ i++;
+ }
+
+ } catch (IOException e) {
+ throw new IllegalArgumentException("A page failed to load!");
+ }
+
+ // Can assert that for the last entry, leaf.getNextLeafPage
+ // should be last pagePath entry here
+ return rightPath;
+ }
+
+
+ /**
+ * Returns the page path to the left sibling, including the left sibling
+ * itself. Empty list if there is none.
+ *
+ * @param pagePath the page path from root to this leaf
+ * @param innerOps the inner page ops that allows this method
+ * to load inner pages and navigate the tree
+ *
+ * @return the page path to the left sibling leaf node
+ *
+ public List<Integer> getLeftSibling(List<Integer> pagePath, InnerPageOperations innerOps) {
+ // Verify that the last node in the page path is this leaf page.
+ if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
+ throw new IllegalArgumentException("The page path provided does" +
+ " not terminate on this leaf page.");
+ }
+
+ ArrayList<Integer> leftPath = new ArrayList<Integer>();
+ // Note to self - not sure on behavior for initializing a for loop
+ // with an i that does not satisfy condition. If it doesn't do any
+ // iterations, then that would be ideal behavior. That case should
+ // never occur anyways...
+ InnerPage inner = null;
+ int index = 0;
+ int i = pagePath.size() - 2;
+ try {
+ while (i >= 0) {
+ inner = innerOps.loadPage(idxFileInfo, pagePath.get(i));
+ index = inner.getIndexOfPointer(pagePath.get(i+1));
+ if (index != 0) {
+ // This means that the subtree this leaf is in has a left
+ // sibling subtree from the current inner node.
+ leftPath.addAll(pagePath.subList(0, i+1));
+ break;
+ }
+ i--;
+ }
+
+ int nextPage;
+ if (inner == null || i == -1) {
+ return leftPath;
+ }
+ // Add to the leftPath the page corresponding to one to the
+ // left of the current index
+ leftPath.add(inner.getPointer(index - 1));
+ i++;
+
+ while (i <= pagePath.size() - 2) {
+ index = inner.getNumPointers() - 1;
+ nextPage = inner.getPointer(index);
+ leftPath.add(nextPage);
+ inner = innerOps.loadPage(idxFileInfo, nextPage);
+ i++;
+ }
+ }
+ catch (IOException e) {
+ throw new IllegalArgumentException("A page failed to load!");
+ }
+ return leftPath;
+ }
+*/
+
+
+ /**
+ * <p>
+ * This helper function moves the specified number of page-pointers to the
+ * right sibling of this inner node. The data is copied in one shot so that
+ * the transfer will be fast, and the various associated bookkeeping values
+ * in both inner pages are updated.
+ * </p>
+ * <p>
+ * Of course, moving a subset of the page-pointers to a sibling will leave
+ * a key without a pointer on one side; this key is promoted up to the
+ * parent of the inner node. Additionally, an existing parent-key can be
+ * provided by the caller, which should be inserted before the new pointers
+ * being moved into the sibling node.
+ * </p>
+ *
+ * @param rightSibling the right sibling of this inner node in the index file
+ *
+ * @param count the number of pointers to move to the right sibling
+ *
+ * @param parentKey If this inner node and the sibling already have a parent
+ * node, this is the key between the two nodes' page-pointers in the
+ * parent node. If the two nodes don't have a parent (i.e. because
+ * an inner node is being split into two nodes and the depth of the
+ * tree is being increased) then this value will be {@code null}.
+ *
+ * @return the key that should go into the parent node, between the
+ * page-pointers for this node and its sibling
+ *
+ * @todo (Donnie) When support for deletion is added to the index
+ * implementation, we will need to support the case when the incoming
+ * {@code parentKey} is non-{@code null}, but the returned key is
+ * {@code null} because one of the two siblings' pointers will be
+ * removed.
+ */
+ public TupleLiteral movePointersRight(InnerPage rightSibling, int count,
+ Tuple parentKey) {
+
+ if (count < 0 || count > numPointers) {
+ throw new IllegalArgumentException("count must be in range [0, " +
+ numPointers + "), got " + count);
+ }
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Non-leaf page " + getPageNo() +
+ " contents before moving pointers right:\n" + toFormattedString());
+ }
+
+ int startPointerIndex = numPointers - count;
+ int startOffset = pointerOffsets[startPointerIndex];
+ int len = endOffset - startOffset;
+
+ logger.debug("Moving everything after pointer " + startPointerIndex +
+ " to right sibling. Start offset = " + startOffset +
+ ", end offset = " + endOffset + ", len = " + len);
+
+ // The parent-key can be null if we are splitting a page into two pages.
+ // However, this situation is only valid if the right sibling is EMPTY.
+ int parentKeyLen = 0;
+ if (parentKey != null) {
+ parentKeyLen = PageTuple.getTupleStorageSize(schema, parentKey);
+ }
+ else {
+ if (rightSibling.getNumPointers() != 0) {
+ throw new IllegalStateException("Cannot move pointers to " +
+ "non-empty sibling if no parent-key is specified!");
+ }
+ }
+
+ /* TODO: IMPLEMENT THE REST OF THIS METHOD.
+ *
+ * You can use PageTuple.storeTuple() to write a key into a DBPage.
+ *
+ * The DBPage.write() method is useful for copying a large chunk of
+ * data from one DBPage to another.
+ *
+ * Your implementation also needs to properly handle the incoming
+ * parent-key, and produce a new parent-key as well.
+ */
+ logger.error("NOT YET IMPLEMENTED: movePointersRight()");
+
+ // Update the cached info for both non-leaf pages.
+ loadPageContents();
+ rightSibling.loadPageContents();
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Non-leaf page " + getPageNo() +
+ " contents after moving pointers right:\n" + toFormattedString());
+
+ logger.trace("Right-sibling page " + rightSibling.getPageNo() +
+ " contents after moving pointers right:\n" +
+ rightSibling.toFormattedString());
+ }
+
+ return null;
+ }
+
+
+ /**
+ * <p>
+ * This helper method creates a formatted string containing the contents of
+ * the inner page, including the pointers and the intervening keys.
+ * </p>
+ * <p>
+ * It is strongly suggested that this method should only be used for
+ * trace-level output, since otherwise the output will become overwhelming.
+ * </p>
+ *
+ * @return a formatted string containing the contents of the inner page
+ */
+ public String toFormattedString() {
+ StringBuilder buf = new StringBuilder();
+
+ buf.append(String.format("Inner page %d contains %d pointers%n",
+ getPageNo(), numPointers));
+
+ if (numPointers > 0) {
+ for (int i = 0; i < numPointers - 1; i++) {
+ buf.append(String.format(" Pointer %d = page %d%n", i,
+ getPointer(i)));
+ buf.append(String.format(" Key %d = %s%n", i, getKey(i)));
+ }
+ buf.append(String.format(" Pointer %d = page %d%n", numPointers - 1,
+ getPointer(numPointers - 1)));
+ }
+
+ return buf.toString();
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/InnerPageOperations.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/InnerPageOperations.java
new file mode 100644
index 0000000000000000000000000000000000000000..c90d6fdc7973943cda290dd4f67705a8213cca5c
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/InnerPageOperations.java
@@ -0,0 +1,968 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import java.util.List;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import edu.caltech.nanodb.expressions.TupleLiteral;
+import edu.caltech.nanodb.relations.Tuple;
+import edu.caltech.nanodb.storage.DBFile;
+import edu.caltech.nanodb.storage.DBPage;
+import edu.caltech.nanodb.storage.PageTuple;
+import edu.caltech.nanodb.storage.StorageManager;
+
+
+/**
+ * This class provides high-level B<sup>+</sup> tree management operations
+ * performed on inner nodes. These operations are provided here and not on the
+ * {@link InnerPage} class since they sometimes involve splitting or merging
+ * inner nodes, updating parent nodes, and so forth.
+ */
+public class InnerPageOperations {
+
+ /**
+ * When deleting a page-pointer from an inner page, either the left or
+ * right key must also be removed; this constant specifies removal of the
+ * left key.
+ *
+ * @see #deletePointer
+ */
+ public static final int REMOVE_KEY_TO_LEFT = 0;
+
+
+ /**
+ * When deleting a page-pointer from an inner page, either the left or
+ * right key must also be removed; this constant specifies removal of the
+ * right key.
+ *
+ * @see #deletePointer
+ */
+ public static final int REMOVE_KEY_TO_RIGHT = 1;
+
+
+ /** A logging object for reporting anything interesting that happens. */
+ private static Logger logger = LogManager.getLogger(InnerPageOperations.class);
+
+
+ private StorageManager storageManager;
+
+
+ private BTreeTupleFile tupleFile;
+
+
+ private FileOperations fileOps;
+
+
+ public InnerPageOperations(StorageManager storageManager,
+ BTreeTupleFile tupleFile,
+ FileOperations fileOps) {
+ this.storageManager = storageManager;
+ this.tupleFile = tupleFile;
+ this.fileOps = fileOps;
+ }
+
+
+ public InnerPage loadPage(int pageNo) {
+ DBFile dbFile = tupleFile.getDBFile();
+ DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
+ return new InnerPage(dbPage, tupleFile.getSchema());
+ }
+
+
+ /**
+ * This helper function is used to update a key between two existing
+ * pointers in an inner B<sup>+</sup> tree node. It is an error if the
+ * specified pair of pointers cannot be found in the node.
+ *
+ * @param page the inner page to update the key in
+ * @param pagePath the path to the page, from the root node
+ * @param pagePtr1 the pointer P<sub>i</sub> before the key to update
+ * @param key1 the new value of the key K<sub>i</sub> to store
+ * @param pagePtr2 the pointer P<sub>i+1</sub> after the key to update
+ *
+ * @todo (Donnie) This implementation has a major failing that will occur
+ * infrequently - if the inner page doesn't have room for the new key
+ * (e.g. if the page was already almost full, and then the new key is
+ * larger than the old key) then the inner page needs to be split,
+ * per usual. Right now it will just throw an exception in this case.
+ * This is why the {@code pagePath} argument is provided, so that when
+ * this bug is fixed, the page-path will be available.
+ */
+ public void replaceTuple(InnerPage page, List<Integer> pagePath,
+ int pagePtr1, Tuple key1, int pagePtr2) {
+
+ for (int i = 0; i < page.getNumPointers() - 1; i++) {
+ if (page.getPointer(i) == pagePtr1 &&
+ page.getPointer(i + 1) == pagePtr2) {
+
+ // Found the pair of pointers! Replace the key-value.
+
+ BTreeFilePageTuple oldKey = page.getKey(i);
+ int oldKeySize = oldKey.getSize();
+
+ int newKeySize =
+ PageTuple.getTupleStorageSize(tupleFile.getSchema(), key1);
+
+ if (logger.isDebugEnabled()) {
+ logger.debug(String.format("Inner page %d: replacing " +
+ "old key of %d bytes (between pointers %d and %d) " +
+ "with new key of %d bytes", page.getPageNo(),
+ oldKeySize, pagePtr1, pagePtr2, newKeySize));
+ }
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Contents of inner page " + page.getPageNo() +
+ " before replacement:\n" + page.toFormattedString());
+ }
+
+ if (page.getFreeSpace() + oldKeySize - newKeySize >= 0) {
+ // We have room - go ahead and do this.
+ page.replaceTuple(i, key1);
+
+ // Make sure we didn't cause any brain damage...
+ assert page.getPointer(i) == pagePtr1;
+ assert page.getPointer(i + 1) == pagePtr2;
+ }
+ else {
+ // We need to make more room in this inner page, either by
+ // relocating records or by splitting this page. We will
+ // do this by deleting the old entry, and then adding the
+ // new entry, so that we can leverage all our good code
+ // for splitting/relocating/etc.
+
+ logger.info(String.format("Not enough space in page %d;" +
+ " trying to relocate entries / split page.",
+ page.getPageNo()));
+
+ // Delete pagePtr2, and the key to the LEFT of it.
+ page.deletePointer(pagePtr2, /* removeRightKey */ false);
+
+ // Now add new key, and put pagePtr2 back after pagePtr1.
+ addTuple(page, pagePath, pagePtr1, key1, pagePtr2);
+ }
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Contents of inner page " + page.getPageNo() +
+ " after replacement:\n" + page.toFormattedString());
+ }
+
+ return;
+ }
+ }
+
+ // If we got here, we have a big problem. Couldn't find the expected
+ // pair of pointers we were handed.
+
+ // Dump the page contents because presumably we want to figure out
+ // what is going on...
+ logger.error(String.format(
+ "Couldn't find pair of pointers %d and %d in inner page %d!",
+ pagePtr1, pagePtr2, page.getPageNo()));
+ logger.error("Page contents:\n" + page.toFormattedString());
+
+ throw new IllegalStateException(
+ "Couldn't find sequence of page-pointers [" + pagePtr1 + ", " +
+ pagePtr2 + "] in non-leaf page " + page.getPageNo());
+ }
+
+
+ /**
+ * This helper function determines how many pointers must be relocated from
+ * one inner page to another, in order to free up the specified number of
+ * bytes. If it is possible, the number of pointers that must be relocated
+ * is returned. If it is not possible, the method returns 0.
+ *
+ * @param page the inner page to relocate entries from
+ *
+ * @param adjPage the adjacent page (predecessor or successor) to relocate
+ * entries to
+ *
+ * @param movingRight pass {@code true} if the sibling is to the right of
+ * {@code page} (and therefore we are moving entries right), or
+ * {@code false} if the sibling is to the left of {@code page} (and
+ * therefore we are moving entries left).
+ *
+ * @param bytesRequired the number of bytes that must be freed up in
+ * {@code page} by the operation
+ *
+ * @param parentKeySize the size of the parent key that must also be
+ * relocated into the adjacent page, and therefore affects how many
+ * pointers can be transferred
+ *
+ * @return the number of pointers that must be relocated to free up the
+ * required space, or 0 if it is not possible.
+ */
+ private int tryNonLeafRelocateForSpace(InnerPage page, InnerPage adjPage,
+ boolean movingRight, int bytesRequired, int parentKeySize) {
+
+ int numKeys = page.getNumKeys();
+ int pageBytesFree = page.getFreeSpace();
+ int adjBytesFree = adjPage.getFreeSpace();
+
+ logger.debug(String.format("Trying to relocate records from inner-" +
+ "page %d (%d bytes free) to adjacent inner-page %d (%d bytes " +
+ "free), moving %s, to free up %d bytes. Parent key size is %d " +
+ "bytes.", page.getPageNo(), pageBytesFree, adjPage.getPageNo(),
+ adjBytesFree, (movingRight ? "right" : "left"), bytesRequired,
+ parentKeySize));
+
+ // The parent key always has to move to the adjacent page, so if that
+ // won't fit, don't even try.
+ if (adjBytesFree < parentKeySize) {
+ logger.debug(String.format("Adjacent page %d has %d bytes free;" +
+ " not enough to hold %d-byte parent key. Giving up on " +
+ "relocation.", adjPage.getPageNo(), adjBytesFree, parentKeySize));
+ return 0;
+ }
+
+ // Since the parent key must always be rotated down into the adjacent
+ // inner page, we must account for that space.
+ adjBytesFree -= parentKeySize;
+
+ int keyBytesMoved = 0;
+
+ int numRelocated = 0;
+ while (true) {
+ // Figure out the index of the key we need the size of, based on
+ // the direction we are moving entries, and how many we have
+ // already moved. If we are moving entries right, we must start
+ // with the rightmost entry in page. If we are moving entries
+ // left, we must start with the leftmost entry in page.
+ int index;
+ if (movingRight)
+ index = numKeys - numRelocated - 1;
+ else
+ index = numRelocated;
+
+ // Add two bytes to the key size for the page-pointer that follows
+ int entrySize = page.getKey(index).getSize() + 2;
+ logger.debug("Entry " + index + " is " + entrySize + " bytes");
+
+ // Did we run out of space to move entries before we hit our goal?
+ if (adjBytesFree < entrySize) {
+ numRelocated = 0;
+ break;
+ }
+
+ numRelocated++;
+
+ pageBytesFree += entrySize;
+ adjBytesFree -= entrySize;
+
+ // Since we don't yet know which page the new pointer will go into,
+ // stop when we can put the pointer in either page.
+ if (pageBytesFree >= bytesRequired &&
+ adjBytesFree >= bytesRequired) {
+ break;
+ }
+ }
+
+ assert numRelocated >= 0;
+ return numRelocated;
+ }
+
+
+ /**
+ * This helper function adds an entry (a key and associated pointer) to
+ * this inner page, after the page-pointer {@code pagePtr1}.
+ *
+ * @param page the inner page to add the entry to
+ *
+ * @param pagePath the path of page-numbers to this inner page
+ *
+ * @param pagePtr1 the <u>existing</u> page that the new key and next-page
+ * number will be inserted after
+ *
+ * @param key1 the new key-value to insert after the {@code pagePtr1} value
+ *
+ * @param pagePtr2 the new page-pointer value to follow the {@code key1}
+ * value
+ */
+ public void addTuple(InnerPage page, List<Integer> pagePath,
+ int pagePtr1, Tuple key1, int pagePtr2) {
+
+ // The new entry will be the key, plus 2 bytes for the page-pointer.
+ int newEntrySize =
+ PageTuple.getTupleStorageSize(tupleFile.getSchema(), key1) + 2;
+
+ logger.debug(String.format("Adding new %d-byte entry to inner page %d",
+ newEntrySize, page.getPageNo()));
+
+ if (page.getFreeSpace() < newEntrySize) {
+ logger.debug("Not enough room in inner page " + page.getPageNo() +
+ "; trying to relocate entries to make room");
+
+ // Try to relocate entries from this inner page to either sibling,
+ // or if that can't happen, split the inner page into two.
+ if (!relocatePointersAndAddKey(page, pagePath,
+ pagePtr1, key1, pagePtr2, newEntrySize)) {
+ logger.debug("Couldn't relocate enough entries to make room;" +
+ " splitting page " + page.getPageNo() + " instead");
+ splitAndAddKey(page, pagePath, pagePtr1, key1, pagePtr2);
+ }
+ }
+ else {
+ // There is room in the leaf for the new key. Add it there.
+ page.addEntry(pagePtr1, key1, pagePtr2);
+ }
+
+ }
+
+
+ /**
+ * This function will delete the specified Key/Pointer pair from the
+ * passed-in inner page.
+ *
+ * @param page the inner page to delete the key/pointer from
+ *
+ * @param pagePath the page path to the passed page
+ *
+ * @param pagePtr the page-pointer value to identify and remove
+ *
+ * @param removeRightKey a flag specifying whether the key to the right
+ * ({@code true}) or to the left ({@code false}) should be removed
+ */
+ public void deletePointer(InnerPage page, List<Integer> pagePath,
+ int pagePtr, boolean removeRightKey) {
+
+ page.deletePointer(pagePtr, removeRightKey);
+
+ if (page.getUsedSpace() >= page.getTotalSpace() / 2) {
+ // The page is at least half-full. Don't need to redistribute or
+ // coalesce.
+ return;
+ }
+ else if (pagePath.size() == 1) {
+ // The page is the root. Don't need to redistribute or coalesce,
+ // but if the root is now empty, need to shorten the tree depth.
+
+ if (page.getNumKeys() == 0) {
+ logger.debug(String.format("Current root page %d is now " +
+ "empty, removing.", page.getPageNo()));
+
+ // Set the index's new root page.
+ DBPage dbpHeader =
+ storageManager.loadDBPage(tupleFile.getDBFile(), 0);
+ HeaderPage.setRootPageNo(dbpHeader, page.getPointer(0));
+
+ // Free up this page in the index.
+ fileOps.releaseDataPage(page.getDBPage());
+ }
+ return;
+ }
+
+ // If we got to this part, we have to redistribute/coalesce stuff :(
+
+ // Note: Assumed that at least one of the siblings has the same
+ // immediate parent. If that is not the case... we're doomed...
+ // TODO: VERIFY THIS AND THROW AN EXCEPTION IF NOT
+
+ int pageNo = page.getPageNo();
+
+ int leftPageNo = page.getLeftSibling(pagePath, this);
+ int rightPageNo = page.getRightSibling(pagePath, this);
+
+ if (leftPageNo == -1 && rightPageNo == -1) {
+ // We should never get to this point, since the earlier test
+ // should have caught this situation.
+ throw new IllegalStateException(String.format(
+ "Inner node %d doesn't have a left or right sibling!",
+ page.getPageNo()));
+ }
+
+ // Now we know that at least one sibling is present. Load both
+ // siblings and coalesce/redistribute in the direction that makes
+ // the most sense...
+
+ InnerPage leftSibling = null;
+ if (leftPageNo != -1)
+ leftSibling = loadPage(leftPageNo);
+
+ InnerPage rightSibling = null;
+ if (rightPageNo != -1)
+ rightSibling = loadPage(rightPageNo);
+
+ // Relocating or coalescing entries requires updating the parent node.
+ // Since the current node has a sibling, it must also have a parent.
+
+ int parentPageNo = pagePath.get(pagePath.size() - 2);
+
+ InnerPage parentPage = loadPage(parentPageNo);
+ // int numPointers = parentPage.getNumPointers();
+ int indexInParentPage = parentPage.getIndexOfPointer(page.getPageNo());
+
+ // See if we can coalesce the node into its left or right sibling.
+ // When we do the check, we must not forget that each node contains a
+ // header, and we need to account for that space as well. This header
+ // space is included in the getUsedSpace() method, but is excluded by
+ // the getSpaceUsedByTuples() method.
+
+ // TODO: SEE IF WE CAN SIMPLIFY THIS AT ALL...
+ if (leftSibling != null &&
+ leftSibling.getUsedSpace() + page.getSpaceUsedByEntries() <
+ leftSibling.getTotalSpace()) {
+
+ // Coalesce the current node into the left sibling.
+ logger.debug("Delete from inner page " + pageNo +
+ ": coalescing with left sibling page.");
+
+ logger.debug(String.format("Before coalesce-left, page has %d " +
+ "pointers and left sibling has %d pointers.",
+ page.getNumPointers(), leftSibling.getNumPointers()));
+
+ // The affected key in the parent page is to the left of this
+ // page's index in the parent page, since we are moving entries
+ // to the left sibling.
+ Tuple parentKey = parentPage.getKey(indexInParentPage - 1);
+
+ // We don't care about the key returned by movePointersLeft(),
+ // since we are deleting the parent key anyway.
+ page.movePointersLeft(leftSibling, page.getNumPointers(), parentKey);
+
+ logger.debug(String.format("After coalesce-left, page has %d " +
+ "pointers and left sibling has %d pointers.",
+ page.getNumPointers(), leftSibling.getNumPointers()));
+
+ // Free up the page since it's empty now
+ fileOps.releaseDataPage(page.getDBPage());
+ page = null;
+
+ List<Integer> parentPagePath = pagePath.subList(0, pagePath.size() - 1);
+ deletePointer(parentPage, parentPagePath, pageNo,
+ /* delete right key */ false);
+ }
+ else if (rightSibling != null &&
+ rightSibling.getUsedSpace() + page.getSpaceUsedByEntries() <
+ rightSibling.getTotalSpace()) {
+
+ // Coalesce the current node into the right sibling.
+ logger.debug("Delete from leaf " + pageNo +
+ ": coalescing with right sibling leaf.");
+
+ logger.debug(String.format("Before coalesce-right, page has %d " +
+ "keys and right sibling has %d pointers.",
+ page.getNumPointers(), rightSibling.getNumPointers()));
+
+ // The affected key in the parent page is to the right of this
+ // page's index in the parent page, since we are moving entries
+ // to the right sibling.
+ Tuple parentKey = parentPage.getKey(indexInParentPage);
+
+ // We don't care about the key returned by movePointersRight(),
+ // since we are deleting the parent key anyway.
+ page.movePointersRight(rightSibling, page.getNumPointers(), parentKey);
+
+ logger.debug(String.format("After coalesce-right, page has %d " +
+ "entries and right sibling has %d pointers.",
+ page.getNumPointers(), rightSibling.getNumPointers()));
+
+ // Free up the right page since it's empty now
+ fileOps.releaseDataPage(page.getDBPage());
+ page = null;
+
+ List<Integer> parentPagePath = pagePath.subList(0, pagePath.size() - 1);
+ deletePointer(parentPage, parentPagePath, pageNo,
+ /* delete right key */ true);
+ }
+ else {
+ // Can't coalesce the leaf node into either sibling. Redistribute
+ // entries from left or right sibling into the leaf. The strategy
+ // is as follows:
+
+ // If the node has both left and right siblings, redistribute from
+ // the fuller sibling. Otherwise, just redistribute from
+ // whichever sibling we have.
+
+ InnerPage adjPage = null;
+ if (leftSibling != null && rightSibling != null) {
+ // Both siblings are present. Choose the fuller one to
+ // relocate from.
+ if (leftSibling.getUsedSpace() > rightSibling.getUsedSpace())
+ adjPage = leftSibling;
+ else
+ adjPage = rightSibling;
+ }
+ else if (leftSibling != null) {
+ // There is no right sibling. Use the left sibling.
+ adjPage = leftSibling;
+ }
+ else {
+ // There is no left sibling. Use the right sibling.
+ adjPage = rightSibling;
+ }
+
+ PageTuple parentKey;
+
+ if (adjPage == leftSibling)
+ parentKey = parentPage.getKey(indexInParentPage - 1);
+ else // adjPage == right sibling
+ parentKey = parentPage.getKey(indexInParentPage);
+
+ int entriesToMove = tryNonLeafRelocateToFill(page, adjPage,
+ /* movingRight */ adjPage == leftSibling, parentKey.getSize());
+
+ if (entriesToMove == 0) {
+ // We really tried to satisfy the "minimum size" requirement,
+ // but we just couldn't. Log it and return.
+
+ StringBuilder buf = new StringBuilder();
+
+ buf.append(String.format("Couldn't relocate pointers to" +
+ " satisfy minimum space requirement in leaf-page %d" +
+ " with %d entries!\n", pageNo, page.getNumPointers()));
+
+ if (leftSibling != null) {
+ buf.append(String.format("\t- Left sibling page %d has " +
+ "%d pointers\n", leftSibling.getPageNo(),
+ leftSibling.getNumPointers()));
+ }
+ else {
+ buf.append("\t- No left sibling\n");
+ }
+
+ if (rightSibling != null) {
+ buf.append(String.format("\t- Right sibling page %d has " +
+ "%d pointers", rightSibling.getPageNo(),
+ rightSibling.getNumPointers()));
+ }
+ else {
+ buf.append("\t- No right sibling");
+ }
+
+ logger.warn(buf);
+
+ return;
+ }
+
+ logger.debug(String.format("Relocating %d pointers into page " +
+ "%d from %s sibling page %d", entriesToMove, pageNo,
+ (adjPage == leftSibling ? "left" : "right"), adjPage.getPageNo()));
+
+ if (adjPage == leftSibling) {
+ adjPage.movePointersRight(page, entriesToMove, parentKey);
+ parentPage.replaceTuple(indexInParentPage - 1, page.getKey(0));
+ }
+ else { // adjPage == right sibling
+ adjPage.movePointersLeft(page, entriesToMove, parentKey);
+ parentPage.replaceTuple(indexInParentPage, adjPage.getKey(0));
+ }
+ }
+ }
+
+
+ private boolean relocatePointersAndAddKey(InnerPage page,
+ List<Integer> pagePath, int pagePtr1, Tuple key1, int pagePtr2,
+ int newEntrySize) {
+
+ int pathSize = pagePath.size();
+ if (pagePath.get(pathSize - 1) != page.getPageNo()) {
+ throw new IllegalArgumentException(
+ "Inner page number doesn't match last page-number in page path");
+ }
+
+ // See if we are able to relocate records either direction to free up
+ // space for the new key.
+
+ if (pathSize == 1) // This node is also the root - no parent.
+ return false; // There aren't any siblings to relocate to.
+
+ int parentPageNo = pagePath.get(pathSize - 2);
+ InnerPage parentPage = loadPage(parentPageNo);
+
+ logger.debug(String.format("Parent of inner-page %d is page %d.",
+ page.getPageNo(), parentPageNo));
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Parent page contents:\n" +
+ parentPage.toFormattedString());
+ }
+
+ int numPointers = parentPage.getNumPointers();
+ int pagePtrIndex = parentPage.getIndexOfPointer(page.getPageNo());
+
+ // Check each sibling in its own code block so that we can constrain
+ // the scopes of the variables a bit. This keeps us from accidentally
+ // reusing the "prev" variables in the "next" section.
+
+ {
+ InnerPage prevPage = null;
+ if (pagePtrIndex - 1 >= 0)
+ prevPage = loadPage(parentPage.getPointer(pagePtrIndex - 1));
+
+ if (prevPage != null) {
+ // See if we can move some of this inner node's entries to the
+ // previous node, to free up space.
+
+ BTreeFilePageTuple parentKey = parentPage.getKey(pagePtrIndex - 1);
+ int parentKeySize = parentKey.getSize();
+
+ int count = tryNonLeafRelocateForSpace(page, prevPage, false,
+ newEntrySize, parentKeySize);
+
+ if (count > 0) {
+ // Yes, we can do it!
+
+ logger.debug(String.format("Relocating %d entries from " +
+ "inner-page %d to left-sibling inner-page %d", count,
+ page.getPageNo(), prevPage.getPageNo()));
+
+ logger.debug("Space before relocation: Inner = " +
+ page.getFreeSpace() + " bytes\t\tSibling = " +
+ prevPage.getFreeSpace() + " bytes");
+
+ TupleLiteral newParentKey =
+ page.movePointersLeft(prevPage, count, parentKey);
+
+ // Even with relocating entries, this could fail.
+ if (!addEntryToInnerPair(prevPage, page, pagePtr1, key1, pagePtr2))
+ return false;
+
+ logger.debug("New parent-key is " + newParentKey);
+
+ pagePath.remove(pathSize - 1);
+ replaceTuple(parentPage, pagePath, prevPage.getPageNo(),
+ newParentKey, page.getPageNo());
+
+ logger.debug("Space after relocation: Inner = " +
+ page.getFreeSpace() + " bytes\t\tSibling = " +
+ prevPage.getFreeSpace() + " bytes");
+
+ return true;
+ }
+ }
+ }
+
+ {
+ InnerPage nextPage = null;
+ if (pagePtrIndex + 1 < numPointers)
+ nextPage = loadPage(parentPage.getPointer(pagePtrIndex + 1));
+
+ if (nextPage != null) {
+ // See if we can move some of this inner node's entries to the
+ // previous node, to free up space.
+
+ BTreeFilePageTuple parentKey = parentPage.getKey(pagePtrIndex);
+ int parentKeySize = parentKey.getSize();
+
+ int count = tryNonLeafRelocateForSpace(page, nextPage, true,
+ newEntrySize, parentKeySize);
+
+ if (count > 0) {
+ // Yes, we can do it!
+
+ logger.debug(String.format("Relocating %d entries from " +
+ "inner-page %d to right-sibling inner-page %d", count,
+ page.getPageNo(), nextPage.getPageNo()));
+
+ logger.debug("Space before relocation: Inner = " +
+ page.getFreeSpace() + " bytes\t\tSibling = " +
+ nextPage.getFreeSpace() + " bytes");
+
+ TupleLiteral newParentKey =
+ page.movePointersRight(nextPage, count, parentKey);
+
+ // Even with relocating entries, this could fail.
+ if (!addEntryToInnerPair(page, nextPage, pagePtr1, key1, pagePtr2))
+ return false;
+
+ logger.debug("New parent-key is " + newParentKey);
+
+ pagePath.remove(pathSize - 1);
+ replaceTuple(parentPage, pagePath, page.getPageNo(),
+ newParentKey, nextPage.getPageNo());
+
+ logger.debug("Space after relocation: Inner = " +
+ page.getFreeSpace() + " bytes\t\tSibling = " +
+ nextPage.getFreeSpace() + " bytes");
+
+ return true;
+ }
+ }
+ }
+
+ // Couldn't relocate entries to either the previous or next page. We
+ // must split the leaf into two.
+ return false;
+ }
+
+
+ /**
+ * <p>
+ * This helper function splits the specified inner page into two pages,
+ * also updating the parent page in the process, and then inserts the
+ * specified key and page-pointer into the appropriate inner page. This
+ * method is used to add a key/pointer to an inner page that doesn't have
+ * enough space, when it isn't possible to relocate pointers to the left
+ * or right sibling of the page.
+ * </p>
+ * <p>
+ * When the inner node is split, half of the pointers are put into the new
+ * sibling, regardless of the size of the keys involved. In other words,
+ * this method doesn't try to keep the pages half-full based on bytes used.
+ * </p>
+ *
+ * @param page the inner node to split and then add the key/pointer to
+ *
+ * @param pagePath the sequence of page-numbers traversed to reach this
+ * inner node.
+ *
+ * @param pagePtr1 the existing page-pointer after which the new key and
+ * pointer should be inserted
+ *
+ * @param key1 the new key to insert into the inner page, immediately after
+ * the page-pointer value {@code pagePtr1}.
+ *
+ * @param pagePtr2 the new page-pointer value to insert after the new key
+ * value
+ */
+ private void splitAndAddKey(InnerPage page, List<Integer> pagePath,
+ int pagePtr1, Tuple key1, int pagePtr2) {
+
+ int pathSize = pagePath.size();
+ if (pagePath.get(pathSize - 1) != page.getPageNo()) {
+ throw new IllegalArgumentException(
+ "Inner page number doesn't match last page-number in page path");
+ }
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Initial contents of inner page " + page.getPageNo() +
+ ":\n" + page.toFormattedString());
+ }
+
+ logger.debug("Splitting inner-page " + page.getPageNo() +
+ " into two inner pages.");
+
+ // Get a new blank page in the index, with the same parent as the
+ // inner-page we were handed.
+
+ DBPage newDBPage = fileOps.getNewDataPage();
+ InnerPage newPage = InnerPage.init(newDBPage, tupleFile.getSchema());
+
+ // Figure out how many values we want to move from the old page to the
+ // new page.
+
+ int numPointers = page.getNumPointers();
+
+ if (logger.isDebugEnabled()) {
+ logger.debug(String.format("Relocating %d pointers from left-page %d" +
+ " to right-page %d", numPointers, page.getPageNo(), newPage.getPageNo()));
+ logger.debug(" Old left # of pointers: " + page.getNumPointers());
+ logger.debug(" Old right # of pointers: " + newPage.getNumPointers());
+ }
+
+ Tuple parentKey = null;
+ InnerPage parentPage = null;
+
+ int parentPageNo = 0;
+ if (pathSize > 1)
+ parentPageNo = pagePath.get(pathSize - 2);
+
+ if (parentPageNo != 0) {
+ parentPage = loadPage(parentPageNo);
+ int parentPtrIndex = parentPage.getIndexOfPointer(page.getPageNo());
+ if (parentPtrIndex < parentPage.getNumPointers() - 1)
+ parentKey = parentPage.getKey(parentPtrIndex);
+ }
+ Tuple newParentKey =
+ page.movePointersRight(newPage, numPointers / 2, parentKey);
+
+ if (logger.isDebugEnabled()) {
+ logger.debug(" New parent key: " + newParentKey);
+ logger.debug(" New left # of pointers: " + page.getNumPointers());
+ logger.debug(" New right # of pointers: " + newPage.getNumPointers());
+ }
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Final contents of inner page " + page.getPageNo() +
+ ":\n" + page.toFormattedString());
+
+ logger.trace("Final contents of new inner page " +
+ newPage.getPageNo() + ":\n" + newPage.toFormattedString());
+ }
+
+ if (!addEntryToInnerPair(page, newPage, pagePtr1, key1, pagePtr2)) {
+ // This is unexpected, but we had better report it if it happens.
+ throw new IllegalStateException("UNEXPECTED: Couldn't add " +
+ "entry to half-full inner page!");
+ }
+
+ // If the current node doesn't have a parent, it's because it's
+ // currently the root.
+ if (parentPageNo == 0) {
+ // Create a new root node and set both leaves to have it as their
+ // parent.
+ DBPage dbpParent = fileOps.getNewDataPage();
+ parentPage = InnerPage.init(dbpParent, tupleFile.getSchema(),
+ page.getPageNo(), newParentKey, newPage.getPageNo());
+
+ parentPageNo = parentPage.getPageNo();
+
+ // We have a new root-page in the index!
+ DBFile dbFile = tupleFile.getDBFile();
+ DBPage dbpHeader = storageManager.loadDBPage(dbFile, 0);
+ HeaderPage.setRootPageNo(dbpHeader, parentPageNo);
+
+ logger.debug("Set index root-page to inner-page " + parentPageNo);
+ }
+ else {
+ // Add the new page into the parent non-leaf node. (This may cause
+ // the parent node's contents to be moved or split, if the parent
+ // is full.)
+
+ // (We already set the new node's parent-page-number earlier.)
+
+ pagePath.remove(pathSize - 1);
+ addTuple(parentPage, pagePath, page.getPageNo(), newParentKey,
+ newPage.getPageNo());
+
+ logger.debug("Parent page " + parentPageNo + " now has " +
+ parentPage.getNumPointers() + " page-pointers.");
+ }
+
+ if (logger.isTraceEnabled()) {
+ logger.trace("Parent page contents:\n" +
+ parentPage.toFormattedString());
+ }
+ }
+
+
+ /**
+ * This helper method takes a pair of inner nodes that are siblings to each
+ * other, and adds the specified key to whichever node the key should go
+ * into.
+ *
+ * @param prevPage the first page in the pair, left sibling of
+ * {@code nextPage}
+ *
+ * @param nextPage the second page in the pair, right sibling of
+ * {@code prevPage}
+ *
+ * @param pageNo1 the pointer to the left of the new key/pointer values that
+ * will be added to one of the pages
+ *
+ * @param key1 the new key-value to insert immediately after the existing
+ * {@code pageNo1} value
+ *
+ * @param pageNo2 the new pointer-value to insert immediately after the new
+ * {@code key1} value
+ *
+ * @return true if the entry was able to be added to one of the pages, or
+ * false if the entry couldn't be added.
+ */
+ private boolean addEntryToInnerPair(InnerPage prevPage, InnerPage nextPage,
+ int pageNo1, Tuple key1, int pageNo2) {
+ InnerPage page;
+
+ // See if pageNo1 appears in the left page.
+ int ptrIndex1 = prevPage.getIndexOfPointer(pageNo1);
+ if (ptrIndex1 != -1) {
+ page = prevPage;
+ }
+ else {
+ // The pointer *should be* in the next page. Verify this...
+ page = nextPage;
+
+ if (nextPage.getIndexOfPointer(pageNo1) == -1) {
+ throw new IllegalStateException(String.format(
+ "Somehow lost page-pointer %d from inner pages %d and %d",
+ pageNo1, prevPage.getPageNo(), nextPage.getPageNo()));
+ }
+ }
+
+ int entrySize = 2 +
+ PageTuple.getTupleStorageSize(tupleFile.getSchema(), key1);
+
+ if (page.getFreeSpace() >= entrySize) {
+ page.addEntry(pageNo1, key1, pageNo2);
+ return true;
+ }
+ else {
+ return false;
+ }
+ }
+
+
+ /**
+ * This helper function determines how many entries must be relocated from
+ * one leaf-page to another, in order to satisfy the "minimum space"
+ * requirement of the B tree.free up the specified number of
+ * bytes. If it is possible, the number of entries that must be relocated
+ * is returned. If it is not possible, the method returns 0.
+ *
+ * @param page the inner node to relocate entries from
+ *
+ * @param adjPage the adjacent inner page (predecessor or successor) to
+ * relocate entries to
+ *
+ * @param movingRight pass {@code true} if the sibling is to the left of
+ * {@code page} (and therefore we are moving entries right), or
+ * {@code false} if the sibling is to the right of {@code page}
+ * (and therefore we are moving entries left).
+ *
+ * @return the number of entries that must be relocated to fill the node
+ * to a minimal level, or 0 if not possible.
+ */
+ private int tryNonLeafRelocateToFill(InnerPage page, InnerPage adjPage,
+ boolean movingRight, int parentKeySize) {
+
+ int adjKeys = adjPage.getNumKeys();
+ int pageBytesFree = page.getFreeSpace();
+ int adjBytesFree = adjPage.getFreeSpace();
+
+ // Should be the same for both page and adjPage.
+ int halfFull = page.getTotalSpace() / 2;
+
+ // The parent key always has to move into this page, so if that
+ // won't fit, don't even try.
+ if (pageBytesFree < parentKeySize)
+ return 0;
+
+ pageBytesFree -= parentKeySize;
+
+ int keyBytesMoved = 0;
+ int lastKeySize = parentKeySize;
+
+ int numRelocated = 0;
+ while (true) {
+ // If the key we wanted to move into this page overflows the free
+ // space in this page, back it up.
+ // TODO: IS THIS NECESSARY?
+ if (pageBytesFree < keyBytesMoved + 2 * numRelocated) {
+ numRelocated--;
+ break;
+ }
+
+ // Figure out the index of the key we need the size of, based on the
+ // direction we are moving values. If we are moving values right,
+ // we need to look at the keys starting at the rightmost one. If we
+ // are moving values left, we need to start with the leftmost key.
+ int index;
+ if (movingRight)
+ index = adjKeys - numRelocated - 1;
+ else
+ index = numRelocated;
+
+ keyBytesMoved += lastKeySize;
+
+ lastKeySize = adjPage.getKey(index).getSize();
+ logger.debug("Key " + index + " is " + lastKeySize + " bytes");
+
+ numRelocated++;
+
+ // Since we don't yet know which page the new pointer will go into,
+ // stop when we can put the pointer in either page.
+ if (adjBytesFree <= halfFull &&
+ (pageBytesFree + keyBytesMoved + 2 * numRelocated) <= halfFull) {
+ break;
+ }
+ }
+
+ logger.debug("Can relocate " + numRelocated +
+ " keys to satisfy minimum space requirements.");
+
+ assert numRelocated >= 0;
+ return numRelocated;
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/LeafPage.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/LeafPage.java
new file mode 100644
index 0000000000000000000000000000000000000000..8b892d506847982f2cb4924c12aed04651a8dd8b
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/LeafPage.java
@@ -0,0 +1,704 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import edu.caltech.nanodb.expressions.TupleComparator;
+import edu.caltech.nanodb.expressions.TupleLiteral;
+import edu.caltech.nanodb.relations.Schema;
+import edu.caltech.nanodb.relations.Tuple;
+import edu.caltech.nanodb.storage.DBPage;
+import edu.caltech.nanodb.storage.PageTuple;
+
+import static edu.caltech.nanodb.storage.btreefile.BTreePageTypes.*;
+
+
+/**
+ * <p>
+ * This class wraps a {@link DBPage} object that is a leaf page in the
+ * B<sup>+</sup> tree file implementation, to provide some of the basic
+ * leaf-management operations necessary for the file structure.
+ * </p>
+ * <p>
+ * Operations involving individual inner-pages are provided by the
+ * {@link InnerPage} wrapper-class. Higher-level operations involving
+ * multiple leaves and/or inner pages of the B<sup>+</sup> tree structure,
+ * are provided by the {@link LeafPageOperations} and
+ * {@link InnerPageOperations} classes.
+ * </p>
+ */
+public class LeafPage implements DataPage {
+ /** A logging object for reporting anything interesting that happens. */
+ private static Logger logger = LogManager.getLogger(LeafPage.class);
+
+
+ /**
+ * The offset where the next-sibling page number is stored in this page.
+ * The only leaf page that doesn't have a next sibling is the last leaf
+ * in the file; its "next page" value will be set to 0.
+ */
+ public static final int OFFSET_NEXT_PAGE_NO = 1;
+
+
+ /**
+ * The offset where the number of tuples is stored in the page.
+ */
+ public static final int OFFSET_NUM_TUPLES = 3;
+
+
+ /** The offset of the first tuple in the leaf page. */
+ public static final int OFFSET_FIRST_TUPLE = 5;
+
+
+ /** The actual data page that holds the B<sup>+</sup> tree leaf node. */
+ private DBPage dbPage;
+
+
+ /** The schema of the tuples in the leaf page. */
+ private Schema schema;
+
+
+ /** The number of tuples stored within this leaf page. */
+ private int numTuples;
+
+
+ /** A list of the tuples stored in this leaf page. */
+ private ArrayList<BTreeFilePageTuple> tuples;
+
+
+ /**
+ * The total size of all data (tuples + initial values) stored within this
+ * leaf page. This is also the offset at which we can start writing more
+ * data without overwriting anything.
+ */
+ private int endOffset;
+
+
+ /**
+ * Initialize the leaf-page wrapper class for the specified B<sup>+</sup>
+ * tree leaf page. The contents of the leaf-page are cached in the fields
+ * of the wrapper object.
+ *
+ * @param dbPage the data page from the B<sup>+</sup> Tree file to wrap
+ * @param schema the schema of tuples stored in the data page
+ */
+ public LeafPage(DBPage dbPage, Schema schema) {
+ if (dbPage.readUnsignedByte(0) != BTREE_LEAF_PAGE) {
+ throw new IllegalArgumentException("Specified DBPage " +
+ dbPage.getPageNo() + " is not marked as a leaf page.");
+ }
+
+ this.dbPage = dbPage;
+ this.schema = schema;
+
+ loadPageContents();
+ }
+
+
+ /**
+ * This static helper function initializes a {@link DBPage} object's
+ * contents with the type and detail values that will allow a new
+ * {@code LeafPage} wrapper to be instantiated for the page, and then it
+ * returns a wrapper object for the page.
+ *
+ * @param dbPage the page to initialize as a leaf page.
+ *
+ * @param schema the schema of the tuples in the leaf page
+ *
+ * @return a newly initialized {@code LeafPage} object wrapping the page
+ */
+ public static LeafPage init(DBPage dbPage, Schema schema) {
+ dbPage.writeByte(OFFSET_PAGE_TYPE, BTREE_LEAF_PAGE);
+ dbPage.writeShort(OFFSET_NUM_TUPLES, 0);
+ dbPage.writeShort(OFFSET_NEXT_PAGE_NO, 0);
+
+ return new LeafPage(dbPage, schema);
+ }
+
+
+ /**
+ * This private helper scans through the leaf page's contents and caches
+ * the contents of the leaf page in a way that makes it easy to use and
+ * manipulate.
+ */
+ private void loadPageContents() {
+ numTuples = dbPage.readUnsignedShort(OFFSET_NUM_TUPLES);
+ tuples = new ArrayList<>(numTuples);
+
+ if (numTuples > 0) {
+ // Handle first tuple separately since we know its offset.
+
+ BTreeFilePageTuple tuple =
+ new BTreeFilePageTuple(schema, dbPage, OFFSET_FIRST_TUPLE, 0);
+
+ tuples.add(tuple);
+
+ // Handle remaining tuples.
+ for (int i = 1; i < numTuples; i++) {
+ int tupleEndOffset = tuple.getEndOffset();
+ tuple = new BTreeFilePageTuple(schema, dbPage, tupleEndOffset, i);
+ tuples.add(tuple);
+ }
+
+ endOffset = tuple.getEndOffset();
+ }
+ else {
+ // There are no tuples in the leaf page.
+ endOffset = OFFSET_FIRST_TUPLE;
+ }
+ }
+
+
+ /**
+ * Returns the schema of tuples in this page.
+ *
+ * @return the schema of tuples in this page
+ */
+ public Schema getSchema() {
+ return schema;
+ }
+
+
+ /**
+ * Returns the {@code DBPage} that backs this leaf page.
+ *
+ * @return the {@code DBPage} that backs this leaf page.
+ */
+ public DBPage getDBPage() {
+ return dbPage;
+ }
+
+
+ /**
+ * Returns the page-number of this leaf page.
+ *
+ * @return the page-number of this leaf page.
+ */
+ public int getPageNo() {
+ return dbPage.getPageNo();
+ }
+
+
+ /**
+ * Returns the page-number of the next leaf page in the sequence of leaf
+ * pages, or 0 if this is the last leaf-page in the B<sup>+</sup> tree
+ * file.
+ *
+ * @return the page-number of the next leaf page in the sequence of leaf
+ * pages, or 0 if this is the last leaf-page in the B<sup>+</sup>
+ * tree file.
+ */
+ public int getNextPageNo() {
+ return dbPage.readUnsignedShort(OFFSET_NEXT_PAGE_NO);
+ }
+
+
+ /**
+ * Sets the page-number of the next leaf page in the sequence of leaf pages.
+ *
+ * @param pageNo the page-number of the next leaf-page in the index, or 0
+ * if this is the last leaf-page in the B<sup>+</sup> tree file.
+ */
+ public void setNextPageNo(int pageNo) {
+ if (pageNo < 0) {
+ throw new IllegalArgumentException(
+ "pageNo must be in range [0, 65535]; got " + pageNo);
+ }
+
+ dbPage.writeShort(OFFSET_NEXT_PAGE_NO, pageNo);
+ }
+
+
+ /**
+ * Returns the number of tuples in this leaf-page. Note that this count
+ * does not include the pointer to the next leaf; it only includes the
+ * tuples themselves.
+ *
+ * @return the number of entries in this leaf-page.
+ */
+ public int getNumTuples() {
+ return numTuples;
+ }
+
+
+ /**
+ * Returns the amount of space currently used in this leaf page, in bytes.
+ *
+ * @return the amount of space currently used in this leaf page, in bytes.
+ */
+ public int getUsedSpace() {
+ return endOffset;
+ }
+
+
+ /**
+ * Returns the amount of space used by tuples in this page, in bytes.
+ *
+ * @return the amount of space used by tuples in this page, in bytes.
+ */
+ public int getSpaceUsedByTuples() {
+ return endOffset - OFFSET_FIRST_TUPLE;
+ }
+
+ /**
+ * Returns the amount of space available in this leaf page, in bytes.
+ *
+ * @return the amount of space available in this leaf page, in bytes.
+ */
+ public int getFreeSpace() {
+ return dbPage.getPageSize() - endOffset;
+ }
+
+
+ /**
+ * Returns the total space (page size) in bytes.
+ *
+ * @return the size of the page, in bytes.
+ */
+ public int getTotalSpace() {
+ return dbPage.getPageSize();
+ }
+
+
+ /**
+ * Returns the tuple at the specified index.
+ *
+ * @param index the index of the tuple to retrieve
+ *
+ * @return the tuple at that index
+ */
+ public BTreeFilePageTuple getTuple(int index) {
+ return tuples.get(index);
+ }
+
+
+ /**
+ * Returns the size of the tuple at the specified index, in bytes.
+ *
+ * @param index the index of the tuple to get the size of
+ *
+ * @return the size of the specified tuple, in bytes
+ */
+ public int getTupleSize(int index) {
+ BTreeFilePageTuple tuple = getTuple(index);
+ return tuple.getEndOffset() - tuple.getOffset();
+ }
+
+
+ /**
+ * Given a leaf page in the B<sup>+</sup> tree file, returns the page
+ * number of the left sibling, or -1 if there is no left sibling to this
+ * node.
+ *
+ * @param pagePath the page path from root to this leaf page
+ * @param innerOps the inner page ops that allows this method to
+ * load inner pages and navigate the tree
+ *
+ * @return the page number of the left sibling leaf-node, or -1 if there
+ * is no left sibling
+ *
+ * @review (Donnie) There is a lot of implementation-overlap between this
+ * function and the {@link InnerPage#getLeftSibling}. Maybe find
+ * a way to combine the implementations.
+ */
+ public int getLeftSibling(List<Integer> pagePath,
+ InnerPageOperations innerOps) {
+
+ // Verify that the last node in the page path is in fact this page.
+ if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
+ throw new IllegalArgumentException(
+ "The page path provided does not terminate on this leaf page.");
+ }
+
+ // If this leaf doesn't have a parent, we already know it doesn't
+ // have a sibling.
+ if (pagePath.size() <= 1)
+ return -1;
+
+ int parentPageNo = pagePath.get(pagePath.size() - 2);
+ InnerPage inner = innerOps.loadPage(parentPageNo);
+
+ // Get the index of the pointer that points to this page. If it
+ // doesn't appear in the parent, we have a serious problem...
+ int pageIndex = inner.getIndexOfPointer(getPageNo());
+ if (pageIndex == -1) {
+ throw new IllegalStateException(String.format(
+ "Leaf node %d doesn't appear in parent inner node %d!",
+ getPageNo(), parentPageNo));
+ }
+
+ int leftSiblingIndex = pageIndex - 1;
+ int leftSiblingPageNo = -1;
+
+ if (leftSiblingIndex >= 0)
+ leftSiblingPageNo = inner.getPointer(leftSiblingIndex);
+
+ return leftSiblingPageNo;
+ }
+
+
+ /**
+ * Given a leaf page in the B<sup>+</sup> tree file, returns the page
+ * number of the right sibling, or -1 if there is no right sibling to
+ * this node.
+ *
+ * @param pagePath the page path from root to this leaf page
+ *
+ * @return the page number of the right sibling leaf-node, or -1 if there
+ * is no right sibling
+ */
+ public int getRightSibling(List<Integer> pagePath) {
+
+ // Verify that the last node in the page path is in fact this page.
+ if (pagePath.get(pagePath.size() - 1) != getPageNo()) {
+ throw new IllegalArgumentException(
+ "The page path provided does not terminate on this leaf page.");
+ }
+
+ int rightSiblingPageNo = getNextPageNo();
+ if (rightSiblingPageNo == 0)
+ rightSiblingPageNo = -1;
+
+ return rightSiblingPageNo;
+ }
+
+
+ /**
+ * Returns the index of the specified tuple.
+ *
+ * @param tuple the tuple to retrieve the index for
+ *
+ * @return the integer index of the specified tuple, -1 if the tuple
+ * isn't in the page.
+ */
+ public int getTupleIndex(Tuple tuple) {
+ int i;
+ for (i = 0; i < numTuples; i++) {
+ BTreeFilePageTuple pageTuple = tuples.get(i);
+
+ /* This gets REALLY verbose... */
+ logger.trace(i + ": comparing " + tuple + " to " + pageTuple);
+
+ // Is this the key we're looking for?
+ if (TupleComparator.comparePartialTuples(tuple, pageTuple) == 0) {
+ logger.debug(String.format("Found tuple: %s is equal to " +
+ "%s at index %d (size = %d bytes)", tuple, pageTuple, i,
+ pageTuple.getSize()));
+
+ return i;
+ }
+ }
+ return -1;
+ }
+
+
+ /**
+ * This method will delete a tuple from the leaf page. The method takes
+ * care of 'sliding' the remaining data to cover up the gap left. The
+ * method throws an exception if the specified tuple does not appear in
+ * the leaf page.
+ *
+ * @param tuple the tuple to delete from the leaf page
+ *
+ * @throws IllegalStateException if the specified tuple doesn't exist
+ */
+ public void deleteTuple(Tuple tuple) {
+ logger.debug("Trying to delete tuple " + tuple + " from leaf page " +
+ getPageNo());
+
+ int index = getTupleIndex(tuple);
+ if (index == -1) {
+ throw new IllegalArgumentException("Specified tuple " + tuple +
+ " does not appear in leaf page " + getPageNo());
+ }
+
+ int tupleOffset = getTuple(index).getOffset();
+ int len = getTupleSize(index);
+
+ logger.debug("Moving leaf-page data in range [" + (tupleOffset+len) +
+ ", " + endOffset + ") over by " + len + " bytes");
+ dbPage.moveDataRange(tupleOffset + len, tupleOffset,
+ endOffset - tupleOffset - len);
+
+ // Decrement the total number of entries.
+ dbPage.writeShort(OFFSET_NUM_TUPLES, numTuples - 1);
+
+ logger.debug("Loading altered page - had " + numTuples +
+ " tuples before delete.");
+ // Load new page.
+ loadPageContents();
+
+ logger.debug("After loading, have " + numTuples + " tuples");
+
+ if (tuple instanceof BTreeFilePageTuple) {
+ BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
+ btpt.setDeleted();
+
+ if (index < numTuples)
+ btpt.setNextTuplePosition(dbPage.getPageNo(), index);
+ else
+ btpt.setNextTuplePosition(getNextPageNo(), 0);
+ }
+ }
+
+
+ /**
+ * This method inserts a tuple into the leaf page, making sure to keep
+ * tuples in monotonically increasing order. This method will throw an
+ * exception if the leaf page already contains the specified tuple.
+ *
+ * @param newTuple the new tuple to add to the leaf page
+ *
+ * @throws IllegalStateException if the specified tuple already appears in
+ * the leaf page.
+ */
+ public BTreeFilePageTuple addTuple(TupleLiteral newTuple) {
+ if (newTuple.getStorageSize() == -1) {
+ throw new IllegalArgumentException("New tuple's storage size " +
+ "must be computed before this method is called.");
+ }
+
+ if (getFreeSpace() < newTuple.getStorageSize()) {
+ throw new IllegalArgumentException(String.format(
+ "Not enough space in this node to store the new tuple " +
+ "(%d bytes free; %d bytes required)", getFreeSpace(),
+ newTuple.getStorageSize()));
+ }
+
+ BTreeFilePageTuple result = null;
+
+ if (numTuples == 0) {
+ logger.debug("Leaf page is empty; storing new tuple at start.");
+ result = addTupleAtIndex(newTuple, 0);
+ }
+ else {
+ int i;
+ for (i = 0; i < numTuples; i++) {
+ BTreeFilePageTuple tuple = tuples.get(i);
+
+ /* This gets REALLY verbose... */
+ logger.trace(i + ": comparing " + newTuple + " to " + tuple);
+
+ // Compare the new tuple to the current tuple. Once we find
+ // where the new tuple should go, copy the tuple into the page.
+ int cmp = TupleComparator.compareTuples(newTuple, tuple);
+ if (cmp < 0) {
+ logger.debug("Storing new tuple at index " + i +
+ " in the leaf page.");
+ result = addTupleAtIndex(newTuple, i);
+ break;
+ }
+ else if (cmp == 0) {
+ // TODO: Currently we require all tuples to be unique,
+ // but this isn't a realistic long-term constraint.
+ throw new IllegalStateException("Tuple " + newTuple +
+ " already appears in the index!");
+ }
+ }
+
+ if (i == numTuples) {
+ // The new tuple will go at the end of this page's entries.
+ logger.debug("Storing new tuple at end of leaf page.");
+ result = addTupleAtIndex(newTuple, numTuples);
+ }
+ }
+
+ // The addTupleAtIndex() method updates the internal fields that cache
+ // where keys live, etc. So, we don't need to do that here.
+
+ assert result != null; // Shouldn't be possible at this point...
+ return result;
+ }
+
+
+ /**
+ * This private helper takes care of inserting a tuple at a specific index
+ * in the leaf page. This method should be called with care, so as to
+ * ensure that tuples always remain in monotonically increasing order.
+ *
+ * @param newTuple the new tuple to insert into the leaf page
+ * @param index the index to insert the tuple at. Any existing tuples at
+ * or after this index will be shifted over to make room for the
+ * new tuple.
+ */
+ private BTreeFilePageTuple addTupleAtIndex(TupleLiteral newTuple,
+ int index) {
+
+ logger.debug("Leaf-page is starting with data ending at index " +
+ endOffset + ", and has " + numTuples + " tuples.");
+
+ // Get the storage size of the new tuple.
+ int len = newTuple.getStorageSize();
+ if (len == -1) {
+ throw new IllegalArgumentException("New tuple's storage size " +
+ "must be computed before this method is called.");
+ }
+
+ logger.debug("New tuple's storage size is " + len + " bytes");
+
+ int tupleOffset;
+ if (index < numTuples) {
+ // Need to slide tuples after this index over, to make space.
+
+ BTreeFilePageTuple tuple = getTuple(index);
+
+ // Make space for the new tuple to be stored, then copy in
+ // the new values.
+
+ tupleOffset = tuple.getOffset();
+
+ logger.debug("Moving leaf-page data in range [" + tupleOffset +
+ ", " + endOffset + ") over by " + len + " bytes");
+
+ dbPage.moveDataRange(tupleOffset, tupleOffset + len,
+ endOffset - tupleOffset);
+ }
+ else {
+ // The new tuple falls at the end of the data in the leaf index
+ // page.
+ tupleOffset = endOffset;
+ logger.debug("New tuple is at end of leaf-page data; not " +
+ "moving anything.");
+ }
+
+ // Write the tuple value into the page.
+ PageTuple.storeTuple(dbPage, tupleOffset, schema, newTuple);
+
+ // Increment the total number of tuples.
+ dbPage.writeShort(OFFSET_NUM_TUPLES, numTuples + 1);
+
+ // Reload the page contents now that we have a new tuple in the mix.
+ // TODO: We could do this more efficiently, but this should be
+ // sufficient for now.
+ loadPageContents();
+
+ logger.debug("Wrote new tuple to leaf-page at offset " + tupleOffset +
+ ".");
+ logger.debug("Leaf-page is ending with data ending at index " +
+ endOffset + ", and has " + numTuples + " tuples.");
+
+ // Return the actual tuple we just added to the page.
+ return getTuple(index);
+ }
+
+
+ /**
+ * This helper function moves the specified number of tuples to the left
+ * sibling of this leaf node. The data is copied in one shot so that the
+ * transfer will be fast, and the various associated bookkeeping values in
+ * both leaves are updated.
+ *
+ * @param leftSibling the left sibling of this leaf-node in the
+ * B<sup>+</sup> tree file
+ *
+ * @param count the number of tuples to move to the left sibling
+ */
+ public void moveTuplesLeft(LeafPage leftSibling, int count) {
+ if (leftSibling == null)
+ throw new IllegalArgumentException("leftSibling cannot be null");
+
+ if (leftSibling.getNextPageNo() != getPageNo()) {
+ logger.error(String.format("Left sibling leaf %d says that " +
+ "page %d is its right sibling, not this page %d",
+ leftSibling.getPageNo(), leftSibling.getNextPageNo(),
+ getPageNo()));
+
+ throw new IllegalArgumentException("leftSibling " +
+ leftSibling.getPageNo() + " isn't actually the left " +
+ "sibling of this leaf-node " + getPageNo());
+ }
+
+ if (count < 0 || count > numTuples) {
+ throw new IllegalArgumentException("count must be in range [0, " +
+ numTuples + "), got " + count);
+ }
+
+ int moveEndOffset = getTuple(count - 1).getEndOffset(); //getTuple(count).getOffset()
+ int len = moveEndOffset - OFFSET_FIRST_TUPLE;
+
+ // Copy the range of tuple-data to the destination page. Then update
+ // the count of tuples in the destination page.
+ // Don't need to move any data in the left sibling; we are appending!
+ leftSibling.dbPage.write(leftSibling.endOffset, dbPage.getPageData(),
+ OFFSET_FIRST_TUPLE, len); // Copy the tuple-data across
+ leftSibling.dbPage.writeShort(OFFSET_NUM_TUPLES,
+ leftSibling.numTuples + count); // Update the tuple-count
+
+ // Remove that range of tuple-data from this page.
+ dbPage.moveDataRange(moveEndOffset, OFFSET_FIRST_TUPLE,
+ endOffset - moveEndOffset);
+ dbPage.writeShort(OFFSET_NUM_TUPLES, numTuples - count);
+
+ // Only erase the old data in the leaf page if we are trying to make
+ // sure everything works properly.
+ if (BTreeTupleFile.CLEAR_OLD_DATA)
+ dbPage.setDataRange(endOffset - len, len, (byte) 0);
+
+ // Update the cached info for both leaves.
+ loadPageContents();
+ leftSibling.loadPageContents();
+ }
+
+
+ /**
+ * This helper function moves the specified number of tuples to the right
+ * sibling of this leaf node. The data is copied in one shot so that the
+ * transfer will be fast, and the various associated bookkeeping values in
+ * both leaves are updated.
+ *
+ * @param rightSibling the right sibling of this leaf-node in the index
+ * file
+ *
+ * @param count the number of tuples to move to the right sibling
+ */
+ public void moveTuplesRight(LeafPage rightSibling, int count) {
+ if (rightSibling == null)
+ throw new IllegalArgumentException("rightSibling cannot be null");
+
+ if (getNextPageNo() != rightSibling.getPageNo()) {
+ throw new IllegalArgumentException("rightSibling " +
+ rightSibling.getPageNo() + " isn't actually the right " +
+ "sibling of this leaf-node " + getPageNo());
+ }
+
+ if (count < 0 || count > numTuples) {
+ throw new IllegalArgumentException("count must be in range [0, " +
+ numTuples + "), got " + count);
+ }
+
+ int startOffset = getTuple(numTuples - count).getOffset();
+ int len = endOffset - startOffset;
+
+ // Copy the range of tuple-data to the destination page. Then update
+ // the count of tuples in the destination page.
+
+ // Make room for the data
+ rightSibling.dbPage.moveDataRange(OFFSET_FIRST_TUPLE,
+ OFFSET_FIRST_TUPLE + len,
+ rightSibling.endOffset - OFFSET_FIRST_TUPLE);
+
+ // Copy the tuple-data across
+ rightSibling.dbPage.write(OFFSET_FIRST_TUPLE, dbPage.getPageData(),
+ startOffset, len);
+
+ // Update the tuple-count
+ rightSibling.dbPage.writeShort(OFFSET_NUM_TUPLES,
+ rightSibling.numTuples + count);
+
+ // Remove that range of tuple-data from this page.
+ dbPage.writeShort(OFFSET_NUM_TUPLES, numTuples - count);
+
+ // Only erase the old data in the leaf page if we are trying to make
+ // sure everything works properly.
+ if (BTreeTupleFile.CLEAR_OLD_DATA)
+ dbPage.setDataRange(startOffset, len, (byte) 0);
+
+ // Update the cached info for both leaves.
+ loadPageContents();
+ rightSibling.loadPageContents();
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/LeafPageOperations.java b/src/main/java/edu/caltech/nanodb/storage/btreefile/LeafPageOperations.java
new file mode 100644
index 0000000000000000000000000000000000000000..29b3726fb58f91df99bb5c2fd1a5184daba9ccdf
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/LeafPageOperations.java
@@ -0,0 +1,837 @@
+package edu.caltech.nanodb.storage.btreefile;
+
+
+import java.util.List;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import edu.caltech.nanodb.expressions.TupleComparator;
+import edu.caltech.nanodb.expressions.TupleLiteral;
+import edu.caltech.nanodb.relations.Tuple;
+import edu.caltech.nanodb.storage.DBFile;
+import edu.caltech.nanodb.storage.DBPage;
+import edu.caltech.nanodb.storage.StorageManager;
+
+
+/**
+ * This class provides high-level B<sup>+</sup> tree management operations
+ * performed on leaf nodes. These operations are provided here and not on the
+ * {@link LeafPage} class since they sometimes involve splitting or merging
+ * leaf nodes, updating parent nodes, and so forth.
+ */
+public class LeafPageOperations {
+ /** A logging object for reporting anything interesting that happens. */
+ private static Logger logger = LogManager.getLogger(LeafPageOperations.class);
+
+ private StorageManager storageManager;
+
+
+ private BTreeTupleFile tupleFile;
+
+
+ private FileOperations fileOps;
+
+ private InnerPageOperations innerPageOps;
+
+
+ public LeafPageOperations(StorageManager storageManager,
+ BTreeTupleFile tupleFile,
+ FileOperations fileOps,
+ InnerPageOperations innerPageOps) {
+ this.storageManager = storageManager;
+ this.tupleFile = tupleFile;
+ this.fileOps = fileOps;
+ this.innerPageOps = innerPageOps;
+ }
+
+
+ /**
+ * This helper function provides the simple operation of loading a leaf page
+ * from its page-number, or if the page-number is 0 then {@code null} is
+ * returned.
+ *
+ * @param pageNo the page-number to load as a leaf-page.
+ *
+ * @return a newly initialized {@link LeafPage} instance if {@code pageNo}
+ * is positive, or {@code null} if {@code pageNo} is 0.
+ *
+ * @throws IllegalArgumentException if the specified page isn't a leaf-page
+ */
+ private LeafPage loadLeafPage(int pageNo) {
+
+ if (pageNo == 0)
+ return null;
+
+ DBFile dbFile = tupleFile.getDBFile();
+ DBPage dbPage = storageManager.loadDBPage(dbFile, pageNo);
+ return new LeafPage(dbPage, tupleFile.getSchema());
+ }
+
+
+ /**
+ * This helper function will handle deleting a tuple in the index.
+ *
+ * @param leaf the leaf page to delete the tuple from
+ *
+ * @param tuple the tuple to delete from the leaf page
+ *
+ * @param pagePath the path of pages taken from the root page to the leaf
+ * page, represented as a list of page numbers
+ */
+ public void deleteTuple(LeafPage leaf, Tuple tuple,
+ List<Integer> pagePath) {
+
+ logger.debug(String.format("Deleting tuple %s from leaf page %d at " +
+ "page-path %s", tuple, leaf.getPageNo(), pagePath));
+
+ leaf.deleteTuple(tuple);
+
+ if (leaf.getUsedSpace() >= leaf.getTotalSpace() / 2) {
+ // The page is at least half-full. Don't need to redistribute or
+ // coalesce.
+ return;
+ }
+ else if (pagePath.size() == 1) {
+ // The page is the root. Don't need to redistribute or coalesce,
+ // but if the root is now empty, need to shorten the tree depth.
+ // (Since this is a leaf page, the depth will go down to 0.)
+
+ if (leaf.getNumTuples() == 0) {
+ logger.debug(String.format("Current root page %d is now " +
+ "empty, removing.", leaf.getPageNo()));
+
+ // Set the index's root page to 0 (empty) since the only page
+ // in the index is now empty and being removed.
+ DBPage dbpHeader =
+ storageManager.loadDBPage(tupleFile.getDBFile(), 0);
+ HeaderPage.setRootPageNo(dbpHeader, 0);
+
+ // Free up this page in the index.
+ fileOps.releaseDataPage(leaf.getDBPage());
+
+ if (tuple instanceof BTreeFilePageTuple) {
+ // Some sanity checks - if the btree is now empty,
+ // the "next tuple" info should be all 0.
+ BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
+ assert btpt.getNextTuplePageNo() == 0;
+ assert btpt.getNextTupleIndex() == 0;
+ }
+ }
+
+ return;
+ }
+
+ // If we got to this part, we have to redistribute/coalesce stuff :(
+
+ // Note: Assumed that at least one of the siblings has the same
+ // immediate parent. If that is not the case... we're doomed...
+ // TODO: VERIFY THIS AND THROW AN EXCEPTION IF NOT
+
+ int leafPageNo = leaf.getPageNo();
+
+ // Leaf pages know their right sibling, so that's why finding the
+ // right page doesn't require the innerPageOps object.
+ int leftPageNo = leaf.getLeftSibling(pagePath, innerPageOps);
+ int rightPageNo = leaf.getRightSibling(pagePath);
+
+ logger.debug(String.format("Leaf page %d is too empty. Left " +
+ "sibling is %d, right sibling is %d.", leafPageNo, leftPageNo,
+ rightPageNo));
+
+ if (leftPageNo == -1 && rightPageNo == -1) {
+ // We should never get to this point, since the earlier test
+ // should have caught this situation.
+ throw new IllegalStateException(String.format(
+ "Leaf node %d doesn't have a left or right sibling!",
+ leaf.getPageNo()));
+ }
+
+ // Now we know that at least one sibling is present. Load both
+ // siblings and coalesce/redistribute in the direction that makes
+ // the most sense...
+
+ LeafPage leftSibling = null;
+ if (leftPageNo != -1)
+ leftSibling = loadLeafPage(leftPageNo);
+
+ LeafPage rightSibling = null;
+ if (rightPageNo != -1)
+ rightSibling = loadLeafPage(rightPageNo);
+
+ assert leftSibling != null || rightSibling != null;
+
+ // See if we can coalesce the node into its left or right sibling.
+ // When we do the check, we must not forget that each node contains a
+ // header, and we need to account for that space as well. This header
+ // space is included in the getUsedSpace() method, but is excluded by
+ // the getSpaceUsedByTuples() method.
+
+ // TODO: SEE IF WE CAN SIMPLIFY THIS AT ALL...
+ if (leftSibling != null &&
+ leftSibling.getUsedSpace() + leaf.getSpaceUsedByTuples() <
+ leftSibling.getTotalSpace()) {
+
+ // Coalesce the current node into the left sibling.
+ logger.debug("Delete from leaf " + leaf.getPageNo() +
+ ": coalescing with left sibling leaf.");
+
+ logger.debug(String.format("Before coalesce-left, page has %d " +
+ "tuples and left sibling has %d tuples.",
+ leaf.getNumTuples(), leftSibling.getNumTuples()));
+
+ if (tuple instanceof BTreeFilePageTuple) {
+ // The "next tuple" will end up in the left sibling, so we
+ // need to update this info in the deleted tuple.
+ BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
+ int index = btpt.getNextTupleIndex();
+ index += leftSibling.getNumTuples();
+ btpt.setNextTuplePosition(leftPageNo, index);
+ }
+
+ leaf.moveTuplesLeft(leftSibling, leaf.getNumTuples());
+ leftSibling.setNextPageNo(leaf.getNextPageNo());
+
+ logger.debug(String.format("After coalesce-left, page has %d " +
+ "tuples and left sibling has %d tuples.",
+ leaf.getNumTuples(), leftSibling.getNumTuples()));
+
+ // Free up the leaf page since it's empty now
+ fileOps.releaseDataPage(leaf.getDBPage());
+
+ // Since the leaf page has been removed from the index structure,
+ // we need to remove it from the parent page. Also, since the
+ // page was coalesced into its left sibling, we need to remove
+ // the tuple to the left of the pointer being removed.
+
+ InnerPage parent =
+ innerPageOps.loadPage(pagePath.get(pagePath.size() - 2));
+
+ List<Integer> parentPagePath = pagePath.subList(0, pagePath.size() - 1);
+ innerPageOps.deletePointer(parent, parentPagePath, leafPageNo,
+ /* remove right tuple */ false);
+ }
+ else if (rightSibling != null &&
+ rightSibling.getUsedSpace() + leaf.getSpaceUsedByTuples() <
+ rightSibling.getTotalSpace()) {
+
+ // Coalesce the current node into the right sibling.
+ logger.debug("Delete from leaf " + leaf.getPageNo() +
+ ": coalescing with right sibling leaf.");
+
+ logger.debug(String.format("Before coalesce-right, page has %d " +
+ "tuples and right sibling has %d tuples.",
+ leaf.getNumTuples(), rightSibling.getNumTuples()));
+
+ if (tuple instanceof BTreeFilePageTuple) {
+ // The "next tuple" will end up in the right sibling, so we
+ // need to update this info in the deleted tuple. We don't
+ // update the index, because this page's tuples will precede
+ // the right sibling's tuples.
+ BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
+ int index = btpt.getNextTupleIndex();
+ btpt.setNextTuplePosition(rightPageNo, index);
+ }
+
+ leaf.moveTuplesRight(rightSibling, leaf.getNumTuples());
+
+ // Left sibling can be null if leaf is the first leaf node in the
+ // sequence of the btree.
+ if (leftSibling != null)
+ leftSibling.setNextPageNo(rightPageNo);
+
+ logger.debug(String.format("After coalesce-right, page has %d " +
+ "tuples and right sibling has %d tuples.",
+ leaf.getNumTuples(), rightSibling.getNumTuples()));
+
+ // Free up the leaf page since it's empty now
+ fileOps.releaseDataPage(leaf.getDBPage());
+
+ // Since the leaf page has been removed from the index structure,
+ // we need to remove it from the parent page. Also, since the
+ // page was coalesced into its right sibling, we need to remove
+ // the tuple to the right of the pointer being removed.
+
+ InnerPage parent =
+ innerPageOps.loadPage(pagePath.get(pagePath.size() - 2));
+
+ List<Integer> parentPagePath = pagePath.subList(0, pagePath.size() - 1);
+ innerPageOps.deletePointer(parent, parentPagePath, leafPageNo,
+ /* remove right tuple */ true);
+ }
+ else {
+ // Can't coalesce the leaf node into either sibling. Redistribute
+ // tuples from left or right sibling into the leaf. The strategy
+ // is as follows:
+
+ // If the node has both left and right siblings, redistribute from
+ // the fuller sibling. Otherwise, just redistribute from
+ // whichever sibling we have.
+
+ LeafPage adjPage;
+ if (leftSibling != null && rightSibling != null) {
+ // Both siblings are present. Choose the fuller one to
+ // relocate from.
+ if (leftSibling.getUsedSpace() > rightSibling.getUsedSpace())
+ adjPage = leftSibling;
+ else
+ adjPage = rightSibling;
+ }
+ else if (leftSibling != null) {
+ // There is no right sibling. Use the left sibling.
+ adjPage = leftSibling;
+ }
+ else {
+ // There is no left sibling. Use the right sibling.
+ adjPage = rightSibling;
+ }
+
+ int tuplesToMove = tryLeafRelocateToFill(leaf, adjPage,
+ /* movingRight */ adjPage == leftSibling);
+
+ if (tuplesToMove == 0) {
+ // We really tried to satisfy the "minimum size" requirement,
+ // but we just couldn't. Log it and return.
+
+ StringBuilder buf = new StringBuilder();
+
+ buf.append(String.format("Couldn't relocate tuples to satisfy" +
+ " minimum space requirement in leaf-page %d with %d tuples!\n",
+ leaf.getPageNo(), leaf.getNumTuples()));
+
+ if (leftSibling != null) {
+ buf.append(
+ String.format("\t- Left sibling page %d has %d tuples\n",
+ leftSibling.getPageNo(), leftSibling.getNumTuples()));
+ }
+ else {
+ buf.append("\t- No left sibling\n");
+ }
+
+ if (rightSibling != null) {
+ buf.append(
+ String.format("\t- Right sibling page %d has %d tuples",
+ rightSibling.getPageNo(), rightSibling.getNumTuples()));
+ }
+ else {
+ buf.append("\t- No right sibling");
+ }
+
+ logger.warn(buf);
+
+ return;
+ }
+
+ logger.debug(String.format("Relocating %d tuples into leaf page " +
+ "%d from %s sibling page %d", tuplesToMove, leaf.getPageNo(),
+ (adjPage == leftSibling ? "left" : "right"), adjPage.getPageNo()));
+
+ if (tuple instanceof BTreeFilePageTuple) {
+ // Since we are moving tuples into this page, we may need to
+ // modify the "next tuple" info. This is only necessary if
+ // tuples are moved from the left sibling, since those will
+ // precede the leaf page's current tuples.
+ BTreeFilePageTuple btpt = (BTreeFilePageTuple) tuple;
+ int nextPageNo = btpt.getNextTuplePageNo();
+ int nextIndex = btpt.getNextTupleIndex();
+
+ if (adjPage == leftSibling) {
+ if (nextPageNo == leafPageNo) {
+ logger.debug(String.format("Moving %d tuples from left " +
+ "sibling %d to leaf %d. Deleted tuple has next " +
+ "tuple at [%d:%d]; updating to [%d:%d]", tuplesToMove,
+ leftPageNo, leafPageNo, nextPageNo, nextIndex,
+ nextPageNo, nextIndex + tuplesToMove));
+
+ btpt.setNextTuplePosition(nextPageNo, nextIndex + tuplesToMove);
+ }
+ else {
+ assert(nextIndex == 0);
+
+ logger.debug(String.format("Moving %d tuples from " +
+ "left sibling %d to leaf %d. Deleted tuple " +
+ "has next tuple at [%d:%d]; not updating",
+ tuplesToMove, leftPageNo, leafPageNo,
+ nextPageNo, nextIndex));
+ }
+ }
+ else {
+ assert adjPage == rightSibling;
+
+ if (nextPageNo == rightPageNo) {
+ assert(nextIndex == 0);
+
+ logger.debug(String.format("Moving %d tuples from " +
+ "right sibling %d to leaf %d. Deleted tuple " +
+ "has next tuple at [%d:%d]; not updating",
+ tuplesToMove, rightPageNo, leafPageNo,
+ nextPageNo, nextIndex));
+
+ btpt.setNextTuplePosition(leafPageNo, leaf.getNumTuples());
+ }
+ else {
+ logger.debug(String.format("Moving %d tuples from " +
+ "right sibling %d to leaf %d. Deleted tuple " +
+ "has next tuple at [%d:%d]; not updating",
+ tuplesToMove, rightPageNo, leafPageNo,
+ nextPageNo, nextIndex));
+ }
+ }
+ }
+
+ InnerPage parent =
+ innerPageOps.loadPage(pagePath.get(pagePath.size() - 2));
+ int index;
+
+ if (adjPage == leftSibling) {
+ adjPage.moveTuplesRight(leaf, tuplesToMove);
+ index = parent.getIndexOfPointer(adjPage.getPageNo());
+ parent.replaceTuple(index, leaf.getTuple(0));
+ }
+ else { // adjPage == right sibling
+ adjPage.moveTuplesLeft(leaf, tuplesToMove);
+ index = parent.getIndexOfPointer(leaf.getPageNo());
+ parent.replaceTuple(index, adjPage.getTuple(0));
+ }
+ }
+ }
+
+
+ /**
+ * This helper function handles the operation of adding a new tuple to a
+ * leaf-page of the index. This operation is provided here and not on the
+ * {@link LeafPage} class, because adding the new tuple might require the
+ * leaf page to be split into two pages.
+ *
+ * @param leaf the leaf page to add the tuple to
+ *
+ * @param newTuple the new tuple to add to the leaf page
+ *
+ * @param pagePath the path of pages taken from the root page to this leaf
+ * page, represented as a list of page numbers in the data file
+ */
+ public BTreeFilePageTuple addTuple(LeafPage leaf, TupleLiteral newTuple,
+ List<Integer> pagePath) {
+
+ BTreeFilePageTuple result;
+
+ // Figure out where the new tuple-value goes in the leaf page.
+
+ int newTupleSize = newTuple.getStorageSize();
+ if (leaf.getFreeSpace() < newTupleSize) {
+ // Try to relocate tuples from this leaf to either sibling,
+ // or if that can't happen, split the leaf page into two.
+ result = relocateTuplesAndAddTuple(leaf, pagePath, newTuple);
+ if (result == null)
+ result = splitLeafAndAddTuple(leaf, pagePath, newTuple);
+ }
+ else {
+ // There is room in the leaf for the new tuple. Add it there.
+ result = leaf.addTuple(newTuple);
+ }
+
+ return result;
+ }
+
+
+ /**
+ * This method attempts to relocate tuples to the left or right sibling
+ * of the specified node, and then insert the specified tuple into the
+ * appropriate node. If it's not possible to relocate tuples (perhaps
+ * because there isn't space and/or because there is no sibling), the
+ * method returns {@code null}.
+ *
+ * @param page The leaf page to relocate tuples out of.
+ *
+ * @param pagePath The path from the index's root to the leaf page
+ *
+ * @param tuple The tuple to add to the index.
+ *
+ * @return a {@code BTreeFilePageTuple} representing the actual tuple
+ * added into the tree structure, or {@code null} if tuples
+ * couldn't be relocated to make space for the new tuple.
+ */
+ private BTreeFilePageTuple relocateTuplesAndAddTuple(LeafPage page,
+ List<Integer> pagePath, TupleLiteral tuple) {
+
+ // See if we are able to relocate records either direction to free up
+ // space for the new tuple.
+
+ int bytesRequired = tuple.getStorageSize();
+
+ int pathSize = pagePath.size();
+ if (pathSize == 1) // This node is also the root - no parent.
+ return null; // There aren't any siblings to relocate to.
+
+ if (pagePath.get(pathSize - 1) != page.getPageNo()) {
+ throw new IllegalArgumentException(
+ "leaf page number doesn't match last page-number in page path");
+ }
+
+ int parentPageNo = 0;
+ if (pathSize >= 2)
+ parentPageNo = pagePath.get(pathSize - 2);
+
+ InnerPage parentPage = innerPageOps.loadPage(parentPageNo);
+ int numPointers = parentPage.getNumPointers();
+ int pagePtrIndex = parentPage.getIndexOfPointer(page.getPageNo());
+
+ // Check each sibling in its own code block so that we can constrain
+ // the scopes of the variables a bit. This keeps us from accidentally
+ // reusing the "prev" variables in the "next" section.
+
+ {
+ LeafPage prevPage = null;
+ if (pagePtrIndex - 1 >= 0)
+ prevPage = loadLeafPage(parentPage.getPointer(pagePtrIndex - 1));
+
+ if (prevPage != null) {
+ // See if we can move some of this leaf's tuples to the
+ // previous leaf, to free up space.
+
+ int count = tryLeafRelocateForSpace(page, prevPage, false,
+ bytesRequired);
+
+ if (count > 0) {
+ // Yes, we can do it!
+ logger.debug(String.format("Relocating %d tuples from " +
+ "leaf-page %d to left-sibling leaf-page %d", count,
+ page.getPageNo(), prevPage.getPageNo()));
+
+ logger.debug("Space before relocation: Leaf = " +
+ page.getFreeSpace() + " bytes\t\tSibling = " +
+ prevPage.getFreeSpace() + " bytes");
+
+ page.moveTuplesLeft(prevPage, count);
+
+ logger.debug("Space after relocation: Leaf = " +
+ page.getFreeSpace() + " bytes\t\tSibling = " +
+ prevPage.getFreeSpace() + " bytes");
+
+ BTreeFilePageTuple result =
+ addTupleToLeafPair(prevPage, page, tuple);
+
+ if (result == null) {
+ // Even with relocating tuples, we couldn't free up
+ // enough space. :-(
+ return null;
+ }
+
+ // Since we relocated tuples between two nodes, update
+ // the parent page to reflect the tuple that is now at
+ // the start of the right page.
+ BTreeFilePageTuple firstRightTuple = page.getTuple(0);
+ pagePath.remove(pathSize - 1);
+ innerPageOps.replaceTuple(parentPage, pagePath,
+ prevPage.getPageNo(), firstRightTuple, page.getPageNo());
+
+ return result;
+ }
+ }
+ }
+
+ {
+ LeafPage nextPage = null;
+ if (pagePtrIndex + 1 < numPointers)
+ nextPage = loadLeafPage(parentPage.getPointer(pagePtrIndex + 1));
+
+ if (nextPage != null) {
+ // See if we can move some of this leaf's tuples to the next
+ // leaf, to free up space.
+
+ int count = tryLeafRelocateForSpace(page, nextPage, true,
+ bytesRequired);
+
+ if (count > 0) {
+ // Yes, we can do it!
+
+ logger.debug(String.format("Relocating %d tuples from " +
+ "leaf-page %d to right-sibling leaf-page %d", count,
+ page.getPageNo(), nextPage.getPageNo()));
+
+ logger.debug("Space before relocation: Leaf = " +
+ page.getFreeSpace() + " bytes\t\tSibling = " +
+ nextPage.getFreeSpace() + " bytes");
+
+ page.moveTuplesRight(nextPage, count);
+
+ logger.debug("Space after relocation: Leaf = " +
+ page.getFreeSpace() + " bytes\t\tSibling = " +
+ nextPage.getFreeSpace() + " bytes");
+
+ BTreeFilePageTuple result =
+ addTupleToLeafPair(page, nextPage, tuple);
+
+ if (result == null) {
+ // Even with relocating tuples, we couldn't free up
+ // enough space. :-(
+ return null;
+ }
+
+ // Since we relocated tuples between two nodes, update
+ // the parent page to reflect the tuple that is now at
+ // the start of the right page.
+ BTreeFilePageTuple firstRightTuple = nextPage.getTuple(0);
+ pagePath.remove(pathSize - 1);
+ innerPageOps.replaceTuple(parentPage, pagePath,
+ page.getPageNo(), firstRightTuple, nextPage.getPageNo());
+
+ return result;
+ }
+ }
+ }
+
+ // Couldn't relocate tuples to either the previous or next page. We
+ // must split the leaf into two.
+ return null;
+ }
+
+
+ /**
+ * This helper method takes a pair of leaf nodes that are siblings to each
+ * other, and adds the specified tuple to whichever leaf the tuple should
+ * go into. The method returns the {@code BTreeFilePageTuple} object
+ * representing the actual tuple in the tree once it has been added.
+ *
+ * @param prevLeaf the first leaf in the pair, left sibling of
+ * {@code nextLeaf}
+ *
+ * @param nextLeaf the second leaf in the pair, right sibling of
+ * {@code prevLeaf}
+ *
+ * @param tuple the tuple to insert into the pair of leaves
+ *
+ * @return the actual tuple in the page, after the insert is completed
+ */
+ private BTreeFilePageTuple addTupleToLeafPair(LeafPage prevLeaf,
+ LeafPage nextLeaf, TupleLiteral tuple) {
+
+ BTreeFilePageTuple result = null;
+ BTreeFilePageTuple firstRightTuple = nextLeaf.getTuple(0);
+ if (TupleComparator.compareTuples(tuple, firstRightTuple) < 0) {
+ // The new tuple goes in the left page. Hopefully there is room
+ // for it...
+ logger.debug("Adding tuple to left leaf " + prevLeaf.getPageNo() +
+ " in pair");
+ if (prevLeaf.getFreeSpace() >= tuple.getStorageSize())
+ result = prevLeaf.addTuple(tuple);
+ }
+ else {
+ // The new tuple goes in the right page. Again, hopefully there
+ // is room for it...
+ logger.debug("Adding tuple to right leaf " + nextLeaf.getPageNo() +
+ " in pair");
+ if (nextLeaf.getFreeSpace() >= tuple.getStorageSize())
+ result = nextLeaf.addTuple(tuple);
+ }
+
+ return result;
+ }
+
+
+ /**
+ * This helper function determines how many tuples must be relocated from
+ * one leaf-page to another, in order to free up the specified number of
+ * bytes. If it is possible, the number of tuples that must be relocated
+ * is returned. If it is not possible, the method returns 0.
+ *
+ * @param leaf the leaf node to relocate tuples from
+ *
+ * @param adjLeaf the adjacent leaf (predecessor or successor) to relocate
+ * tuples to
+ *
+ * @param movingRight pass {@code true} if the sibling is to the right of
+ * {@code page} (and therefore we are moving tuples right), or
+ * {@code false} if the sibling is to the left of {@code page} (and
+ * therefore we are moving tuples left).
+ *
+ * @param bytesRequired the number of bytes that must be freed up in
+ * {@code leaf} by the operation
+ *
+ * @return the number of tuples that must be relocated to free up the
+ * required space, or 0 if it is not possible.
+ */
+ private int tryLeafRelocateForSpace(LeafPage leaf, LeafPage adjLeaf,
+ boolean movingRight, int bytesRequired) {
+
+ int numTuples = leaf.getNumTuples();
+ int leafBytesFree = leaf.getFreeSpace();
+ int adjBytesFree = adjLeaf.getFreeSpace();
+
+ logger.debug("Leaf bytes free: " + leafBytesFree +
+ "\t\tAdjacent leaf bytes free: " + adjBytesFree);
+
+ // Subtract the bytes-required from the adjacent-bytes-free value so
+ // that we ensure we always have room to put the tuple in either node.
+ adjBytesFree -= bytesRequired;
+
+ int numRelocated = 0;
+ while (true) {
+ // Figure out the index of the tuple we need the size of, based on
+ // the direction we are moving values. If we are moving values
+ // right, we need to look at the tuples starting at the rightmost
+ // one. If we are moving tuples left, we need to start with the
+ // leftmost tuple.
+ int index;
+ if (movingRight)
+ index = numTuples - numRelocated - 1;
+ else
+ index = numRelocated;
+
+ int tupleSize = leaf.getTupleSize(index);
+
+ logger.debug("Tuple " + index + " is " + tupleSize + " bytes");
+
+ // Did we run out of space to move tuples before we hit our goal?
+ if (adjBytesFree < tupleSize) {
+ numRelocated = 0;
+ break;
+ }
+
+ numRelocated++;
+
+ leafBytesFree += tupleSize;
+ adjBytesFree -= tupleSize;
+
+ // Since we don't yet know which leaf the new tuple will go into,
+ // stop when we can put the tuple in either leaf.
+ if (leafBytesFree >= bytesRequired &&
+ adjBytesFree >= bytesRequired) {
+ break;
+ }
+ }
+
+ logger.debug("Can relocate " + numRelocated + " tuples to free up space.");
+
+ return numRelocated;
+ }
+
+
+ /**
+ * This helper function splits the specified leaf-node into two nodes,
+ * also updating the parent node in the process, and then inserts the
+ * specified tuple into the appropriate leaf. This method is used to add
+ * a tuple to a leaf that doesn't have enough space, when it isn't
+ * possible to relocate values to the left or right sibling of the leaf.
+ *
+ * @todo (donnie) When the leaf node is split, half of the tuples are
+ * put into the new leaf, regardless of the size of individual
+ * tuples. In other words, this method doesn't try to keep the
+ * leaves half-full based on bytes used. It would almost
+ * certainly be better if it did.
+ *
+ * @param leaf the leaf node to split and then add the tuple to
+ * @param pagePath the sequence of page-numbers traversed to reach this
+ * leaf node.
+ *
+ * @param tuple the new tuple to insert into the leaf node
+ */
+ private BTreeFilePageTuple splitLeafAndAddTuple(LeafPage leaf,
+ List<Integer> pagePath, TupleLiteral tuple) {
+
+ int pathSize = pagePath.size();
+ if (pagePath.get(pathSize - 1) != leaf.getPageNo()) {
+ throw new IllegalArgumentException(
+ "Leaf page number doesn't match last page-number in page path");
+ }
+
+ if (logger.isDebugEnabled()) {
+ logger.debug("Splitting leaf-page " + leaf.getPageNo() +
+ " into two leaves.");
+ logger.debug(" Old next-page: " + leaf.getNextPageNo());
+ }
+
+ // Get a new blank page in the index, with the same parent as the
+ // leaf-page we were handed.
+
+ DBPage newDBPage = fileOps.getNewDataPage();
+ LeafPage newLeaf = LeafPage.init(newDBPage, tupleFile.getSchema());
+
+ /* TODO: IMPLEMENT THE REST OF THIS METHOD.
+ *
+ * The LeafPage class provides some helpful operations for moving leaf-
+ * entries to a left or right sibling.
+ *
+ * The parent page must also be updated. If the leaf node doesn't have
+ * a parent, the tree's depth will increase by one level.
+ */
+ logger.error("NOT YET IMPLEMENTED: splitLeafAndAddKey()");
+ return null;
+ }
+
+
+ /**
+ * This helper function determines how many tuples must be relocated from
+ * one leaf-page to another, in order to satisfy the "minimum space"
+ * requirement of the B tree.free up the specified number of
+ * bytes. If it is possible, the number of tuples that must be relocated
+ * is returned. If it is not possible, the method returns 0.
+ *
+ * @param leaf the leaf node to relocate tuples from
+ *
+ * @param adjLeaf the adjacent leaf (predecessor or successor) to relocate
+ * tuples to
+ *
+ * @param movingRight pass {@code true} if the sibling is to the left of
+ * {@code page} (and therefore we are moving tuples right), or
+ * {@code false} if the sibling is to the right of {@code page}
+ * (and therefore we are moving tuples left).
+ *
+ * @return the number of tuples that must be relocated to fill the node
+ * to a minimal level, or 0 if not possible.
+ */
+ private int tryLeafRelocateToFill(LeafPage leaf, LeafPage adjLeaf,
+ boolean movingRight) {
+
+ int adjTuples = adjLeaf.getNumTuples(); // Tuples available to move
+ int leafBytesFree = leaf.getFreeSpace();
+ int adjBytesFree = adjLeaf.getFreeSpace();
+
+ // Should be the same for both leaf and adjLeaf.
+ int halfFull = leaf.getTotalSpace() / 2;
+
+ logger.debug("Leaf bytes free: " + leafBytesFree +
+ "\t\tAdjacent leaf bytes free: " + adjBytesFree);
+
+ int numRelocated = 0;
+ while (true) {
+ // Figure out the index of the tuple we need the size of, based on
+ // the direction we are moving values. If we are moving values
+ // right, we need to look at the tuples starting at the rightmost
+ // one. If we are moving values left, we need to start with the
+ // leftmost tuple.
+ int index;
+ if (movingRight)
+ index = adjTuples - numRelocated - 1;
+ else
+ index = numRelocated;
+
+ int tupleSize = adjLeaf.getTupleSize(index);
+
+ logger.debug("Tuple " + index + " is " + tupleSize + " bytes");
+
+ // If we don't have room to move the adjacent node's tuple into
+ // this node (unlikely), just stop there.
+ if (leafBytesFree < tupleSize)
+ break;
+
+ // If the adjacent leaf would become too empty, stop relocating.
+ if (adjBytesFree > halfFull)
+ break;
+
+ numRelocated++;
+
+ leafBytesFree -= tupleSize;
+ adjBytesFree += tupleSize;
+
+ // Stop if the leaf now has at least the minimal number of bytes.
+ if (leafBytesFree <= halfFull)
+ break;
+ }
+
+ logger.debug("Can relocate " + numRelocated +
+ " tuples to satisfy minimum space requirements.");
+
+ return numRelocated;
+ }
+}
diff --git a/src/main/java/edu/caltech/nanodb/storage/btreefile/package.html b/src/main/java/edu/caltech/nanodb/storage/btreefile/package.html
new file mode 100644
index 0000000000000000000000000000000000000000..2a21bade8ab91b21d142a604f809b2c516b2fbf4
--- /dev/null
+++ b/src/main/java/edu/caltech/nanodb/storage/btreefile/package.html
@@ -0,0 +1,43 @@
+<html>
+<body>
+
+<p>
+This package contains an implementation of B<sup>+</sup> tree
+tuple files, which can be used for sequential tuple files, as
+well as table indexes.
+</p>
+
+<!--
+<h2>Leaf Page Internal Structure</h2>
+
+<table>
+ <tr><th>Offset</th><th>Size</th><th>Description</th></tr>
+ <tr>
+ <td>0</td>
+ <td>byte</td>
+ <td>Page type, always 2 for leaf pages.
+ (See {@link BTreeIndexManager#BTREE_LEAF_PAGE}.)</td>
+ </tr>
+
+ <tr>
+ <td>1</td>
+ <td>unsigned short</td>
+ <td>Page number of next leaf page.
+ (See {@link LeafPage#OFFSET_NEXT_PAGE_NO}.)</td>
+ </tr>
+
+ <tr>
+ <td>3</td>
+ <td>unsigned short</td>
+ <td>The number of key+pointer entries is stored in the page.
+ (See {@link LeafPage#OFFSET_NUM_ENTRIES}.)</td>
+ </tr>
+
+ <tr>
+
+ </tr>
+</table>
+-->
+
+</body>
+</html>
diff --git a/src/test/java/edu/caltech/test/nanodb/storage/btreefile/TestBTreeFile.java b/src/test/java/edu/caltech/test/nanodb/storage/btreefile/TestBTreeFile.java
new file mode 100644
index 0000000000000000000000000000000000000000..07dc6296ece4f8f17f1dd294524ad99f9c40fdc4
--- /dev/null
+++ b/src/test/java/edu/caltech/test/nanodb/storage/btreefile/TestBTreeFile.java
@@ -0,0 +1,207 @@
+package edu.caltech.test.nanodb.storage.btreefile;
+
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.Random;
+
+import edu.caltech.nanodb.expressions.ColumnName;
+import edu.caltech.nanodb.expressions.ColumnValue;
+import edu.caltech.nanodb.expressions.OrderByExpression;
+import edu.caltech.nanodb.expressions.TupleComparator;
+import edu.caltech.nanodb.expressions.TupleLiteral;
+import edu.caltech.nanodb.relations.ColumnInfo;
+import edu.caltech.nanodb.relations.ColumnType;
+import edu.caltech.nanodb.relations.SQLDataType;
+import edu.caltech.nanodb.relations.Schema;
+import edu.caltech.nanodb.server.CommandResult;
+import org.testng.annotations.*;
+
+import edu.caltech.test.nanodb.sql.SqlTestCase;
+
+
+/**
+ * This test class exercises the B<sup>+</sup>-tree file format so that we can
+ * have some confidence that it actually works correctly.
+ */
+@Test(groups={"storage", "hw6"})
+public class TestBTreeFile extends SqlTestCase {
+
+ /**
+ * If set to true, this causes the tests to verify the contents of the
+ * B tree file after every insertion or deletion. This obviously greatly
+ * slows down the test, but it allows issues to be identified exactly when
+ * they appear.
+ */
+ public static final boolean CHECK_AFTER_EACH_CHANGE = false;
+
+
+ /**
+ * A source of randomness to generate tuples from. Set the seed so we
+ * have reproducible test cases.
+ */
+ private Random rand = new Random(12345);
+
+
+ private String makeRandomString(int minChars, int maxChars) {
+ StringBuilder buf = new StringBuilder();
+
+ int num = minChars + rand.nextInt(maxChars - minChars + 1);
+ for (int i = 0; i < num; i++)
+ buf.append((char) ('A' + rand.nextInt('Z' - 'A' + 1)));
+
+ return buf.toString();
+ }
+
+
+ private void sortTupleLiteralArray(ArrayList<TupleLiteral> tuples) {
+ Schema schema = new Schema();
+ schema.addColumnInfo(new ColumnInfo("a", new ColumnType(SQLDataType.INTEGER)));
+ schema.addColumnInfo(new ColumnInfo("b", new ColumnType(SQLDataType.VARCHAR)));
+
+ ArrayList<OrderByExpression> orderSpec = new ArrayList<>();
+ orderSpec.add(new OrderByExpression(new ColumnValue(new ColumnName("a"))));
+ orderSpec.add(new OrderByExpression(new ColumnValue(new ColumnName("b"))));
+
+ TupleComparator comp = new TupleComparator(schema, orderSpec);
+ Collections.sort(tuples, comp);
+ }
+
+
+ private void runBTreeTest(String tableName, int numRowsToInsert,
+ int maxAValue, int minBLen, int maxBLen,
+ double probDeletion) throws Exception {
+ ArrayList<TupleLiteral> inserted = new ArrayList<>();
+ CommandResult result;
+
+ for (int i = 0; i < numRowsToInsert; i++) {
+ int a = rand.nextInt(maxAValue);
+ String b = makeRandomString(minBLen, maxBLen);
+
+ tryDoCommand(String.format(
+ "INSERT INTO %s VALUES (%d, '%s');", tableName, a, b), false);
+ inserted.add(new TupleLiteral(a, b));
+
+ if (CHECK_AFTER_EACH_CHANGE) {
+ sortTupleLiteralArray(inserted);
+ result = tryDoCommand(String.format("SELECT * FROM %s;",
+ tableName), true);
+ assert checkOrderedResults(inserted.toArray(new TupleLiteral[inserted.size()]), result);
+ }
+
+ if (probDeletion > 0.0 && rand.nextDouble() < probDeletion) {
+ // Delete some rows from the table we are populating.
+ int minAToDel = rand.nextInt(maxAValue);
+ int maxAToDel = rand.nextInt(maxAValue);
+ if (minAToDel > maxAToDel) {
+ int tmp = minAToDel;
+ minAToDel = maxAToDel;
+ maxAToDel = tmp;
+ }
+
+ tryDoCommand(String.format(
+ "DELETE FROM %s WHERE a BETWEEN %d AND %d;", tableName,
+ minAToDel, maxAToDel), false);
+
+ // Apply the same deletion to our in-memory collection of
+ // tuples, so that we can mirror what the table should contain
+ Iterator<TupleLiteral> iter = inserted.iterator();
+ while (iter.hasNext()) {
+ TupleLiteral tup = iter.next();
+ int aVal = (Integer) tup.getColumnValue(0);
+ if (aVal >= minAToDel && aVal <= maxAToDel)
+ iter.remove();
+ }
+
+ if (CHECK_AFTER_EACH_CHANGE) {
+ sortTupleLiteralArray(inserted);
+ result = tryDoCommand(String.format("SELECT * FROM %s;",
+ tableName), true);
+ assert checkOrderedResults(inserted.toArray(new TupleLiteral[inserted.size()]), result);
+ }
+ }
+ }
+
+ sortTupleLiteralArray(inserted);
+ result = tryDoCommand(String.format("SELECT * FROM %s;",
+ tableName), true);
+ assert checkOrderedResults(inserted.toArray(new TupleLiteral[inserted.size()]), result);
+
+ // TODO: This is necessary because the btree code doesn't unpin
+ // pages properly...
+ // server.getStorageManager().flushAllData();
+ }
+
+
+ public void testBTreeTableOnePageInsert() throws Exception {
+ tryDoCommand("CREATE TABLE btree_one_page (a INTEGER, b VARCHAR(20)) " +
+ "PROPERTIES (storage = 'btree');", false);
+
+ runBTreeTest("btree_one_page", 300, 200, 3, 20, 0.0);
+ }
+
+
+ public void testBTreeTableTwoPageInsert() throws Exception {
+ tryDoCommand("CREATE TABLE btree_two_page (a INTEGER, b VARCHAR(50)) " +
+ "PROPERTIES (storage = 'btree');", false);
+
+ runBTreeTest("btree_two_page", 400, 200, 20, 50, 0.0);
+ }
+
+
+ public void testBTreeTableTwoLevelInsert() throws Exception {
+ tryDoCommand("CREATE TABLE btree_two_level (a INTEGER, b VARCHAR(50)) " +
+ "PROPERTIES (storage = 'btree');", false);
+
+ runBTreeTest("btree_two_level", 10000, 1000, 20, 50, 0.0);
+ }
+
+
+ public void testBTreeTableThreeLevelInsert() throws Exception {
+ tryDoCommand("CREATE TABLE btree_three_level (a INTEGER, b VARCHAR(250)) " +
+ "PROPERTIES (storage = 'btree');", false);
+
+ runBTreeTest("btree_three_level", 100000, 5000, 150, 250, 0.0);
+ }
+
+
+ public void testBTreeTableOnePageInsertDelete() throws Exception {
+ tryDoCommand("CREATE TABLE btree_one_page_del (a INTEGER, b VARCHAR(20)) " +
+ "PROPERTIES (storage = 'btree');", false);
+
+ runBTreeTest("btree_one_page_del", 400, 200, 3, 20, 0.05);
+ }
+
+
+ public void testBTreeTableTwoPageInsertDelete() throws Exception {
+ tryDoCommand("CREATE TABLE btree_two_page_del (a INTEGER, b VARCHAR(50)) " +
+ "PROPERTIES (storage = 'btree');", false);
+
+ runBTreeTest("btree_two_page_del", 500, 200, 20, 50, 0.05);
+ }
+
+
+ public void testBTreeTableTwoLevelInsertDelete() throws Exception {
+ tryDoCommand("CREATE TABLE btree_two_level_del (a INTEGER, b VARCHAR(50)) " +
+ "PROPERTIES (storage = 'btree');", false);
+
+ runBTreeTest("btree_two_level_del", 12000, 1000, 20, 50, 0.05);
+ }
+
+
+ public void testBTreeTableThreeLevelInsertDelete() throws Exception {
+ tryDoCommand("CREATE TABLE btree_three_level_del (a INTEGER, b VARCHAR(250)) " +
+ "PROPERTIES (storage = 'btree');", false);
+
+ runBTreeTest("btree_three_level_del", 120000, 5000, 150, 250, 0.05);
+ }
+
+
+ public void testBTreeTableMultiLevelInsertDelete() throws Exception {
+ tryDoCommand("CREATE TABLE btree_multi_level_del (a INTEGER, b VARCHAR(400)) " +
+ "PROPERTIES (storage = 'btree');", false);
+
+ runBTreeTest("btree_multi_level_del", 250000, 5000, 50, 400, 0.01);
+ }
+}