I was searching NoSQL databases and see that Oracle provides a NoSQL database called Berkeley DB. I examined it and wrote a blog to give quick tips to start Java Edition of Berkeley DB. This blog was published in Turkish, about 1 year ago. I’ve decided to translate (in fact, re-write) and publish it in English, and this is what you are reading now.
Berkeley DB is a high-performance embedded database originated at the University of California, Berkeley. It’s fast, reliable and used in several applications such as Evolution (email client), OpenLDAP, RPM (The RPM Package Manager) and Postfix (MTA). In contrast to most other database systems, Berkeley DB provides relatively simple data access services. Berkeley DB databases are B+Trees (like indexes in Oracle RDBMS) and can store only key/value pairs (there are no columns or tables). The keys and values are byte arrays. Databases are stored as files within a single directory which is called “environment”.
There are three versions of Berkeley DB:
- Berkeley DB (the traditional database, written in C)
- Berkeley DB Java Edition (native Java version)
- Berkeley DB XML (for storing XML documents)
As a hobbyist Java programmer, I prefer Berkeley DB Java Edition (JE). Berkeley DB JE supports almost all features of traditional Berkeley DB such as replication, hot-backups, ACID and transactions. It is written in pure Java so it’s platform-independent.
Berkeley DB JE provides two interfaces:
- Traditional Berkeley DB API (with DB data abstraction of key/value pairs)
- Direct Persistence Layer (DPL) which contains “Plain Old Java Objects” (POJO)
Because I’m an old-school (ex)programmer, I’ll show how to use the traditional Berkeley DB API. Traditional Berkeley DB API will help you understand how Berkeley DB works.
Let’s start by downloading the Berkeley DB from Oracle website:
http://www.oracle.com/technetwork/database/berkeleydb/downloads/index.html
After you download the file (je-x.x.x.tar.gz/zip), extract it. You’ll find the required JAR file (je-x.x.x.jar) in “lib” folder. You can add it to your Java project and start using Berkeley DB Java Edition.
Berkeley DB stored the datafiles in a folder. We define this folder as an environment object:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
public static void main(String[] args) { try { // create a configuration for DB environment EnvironmentConfig envConf = new EnvironmentConfig(); // environment will be created if not exists envConf.setAllowCreate(true); // open/create the DB environment using config Environment dbEnv = new Environment( new File("/home/gokhan/berkeleydb"), envConf); } catch (DatabaseException dbe) { System.out.println("Error :" + dbe.getMessage() ); } } |
Before running the above code, we should be sure that this folder ( “/home/gokhan/berkeleydb” ) exists and our user have required privileges to access it. After you run the code, you’ll see that Berkeley DB created the base files in this folder:
|
1 2 3 4 5 6 7 |
gokhan@exodus:~/berkeleydb$ ls -lh total 4.0K -rw-r--r-- 1 gokhan gokhan 1.6K 2011-09-12 21:11 00000000.jdb -rw-r--r-- 1 gokhan gokhan 0 2011-09-10 02:12 je.info.0 -rw-r--r-- 1 gokhan gokhan 0 2011-09-12 21:11 je.info.0.lck -rw-r--r-- 1 gokhan gokhan 0 2011-09-10 02:12 je.lck |
The same code can also be used for “opening” an existing environment. Now we can create (or open) our database in the environment we defined:
|
1 2 3 4 5 6 7 |
// create a configuration for DB DatabaseConfig dbConf = new DatabaseConfig(); // db will be created if not exits dbConf.setAllowCreate(true); // create/open testDB using config Database testDB = dbEnv.openDatabase(null, "testDB", dbConf); |
As I said Berkeley DB can store key/value pairs. We will use DatabaseEntry object to store key/value pairs. The following code will insert “1” as key and “GOKHAN ATIL” as data:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
// create a configuration for DB DatabaseConfig dbConf = new DatabaseConfig(); // db will be created if not exits dbConf.setAllowCreate(true); // assign 1 to key IntegerBinding.intToEntry(1, key); // assign my name to data StringBinding.stringToEntry("GOKHAN ATIL", data); // insert key/value pair to database testDB.put(null, key, data); |
As you can see, we convert our variables to entries and then use Database object to store them. The “put” method of the Database object is used to insert data:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
// sequence will be created if not exits SequenceConfig seqConf = new SequenceConfig(); seqConf.setAllowCreate(true); // give a name to sequence DatabaseEntry entMySeq = new DatabaseEntry(); StringBinding.stringToEntry("MY_SEQ", entMySeq); // open (or create) sequence Sequence mySeq = testDB.openSequence(null, entMySeq, seqConf); // read a number from sequence, set key IntegerBinding.intToEntry( (int)mySeq.get(null, 1), key); // set my name as data StringBinding.stringToEntry("GOKHAN ATIL", data); // insert (or update) key/value pair to database testDB.put(null, key, data); |
If the key already exists in the database and no duplicate records are allowed (default), the “put” method will update the existing pair. To read a pair, we need to use “get” method:
|
1 2 3 4 5 6 |
// read from database testDB.get(null, key, data, null); // display key/value pair System.out.println("Key :" + IntegerBinding.entryToInt(key)); System.out.println("Data :" + StringBinding.entryToString(data)); |
To delete records, we can use “delete” method:
|
1 |
testDB.delete(null, key); |
Now I’ll combine these functions and write a sample application to generate 500.000 dummy records, insert them into the database and then get a random record:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
package berkeleydemo; import com.sleepycat.bind.tuple.IntegerBinding; import com.sleepycat.bind.tuple.StringBinding; import com.sleepycat.je.Database; import com.sleepycat.je.DatabaseConfig; import com.sleepycat.je.DatabaseEntry; import com.sleepycat.je.DatabaseException; import com.sleepycat.je.Environment; import com.sleepycat.je.EnvironmentConfig; import com.sleepycat.je.OperationStatus; import com.sleepycat.je.Sequence; import com.sleepycat.je.SequenceConfig; import java.io.File; import java.text.DecimalFormat; import java.text.NumberFormat; import java.util.Random; /** * Author: Gokhan Atil * Web: http://www.gokhanatil.com */ public class BerkeleyDemo { public static void main(String[] args) { try { // create a configuration for DB environment EnvironmentConfig envConf = new EnvironmentConfig(); // environment will be created if not exists envConf.setAllowCreate(true); // open/create the DB environment using config Environment dbEnv = new Environment( new File("d:\\berkeleydb"), envConf); // create a configuration for DB DatabaseConfig dbConf = new DatabaseConfig(); // db will be created if not exits dbConf.setAllowCreate(true); // create/open testDB using config Database testDB = dbEnv.openDatabase(null, "testDB", dbConf); // key DatabaseEntry key = new DatabaseEntry(); // data DatabaseEntry data = new DatabaseEntry(); // sequence will be created if not exits SequenceConfig seqConf = new SequenceConfig(); seqConf.setAllowCreate(true); // give a name to sequence DatabaseEntry entMySeq = new DatabaseEntry(); StringBinding.stringToEntry("MY_SEQ", entMySeq); // open (or create) sequence Sequence mySeq = testDB.openSequence(null, entMySeq, seqConf); // will use it for formatting numbers NumberFormat formatter = new DecimalFormat("000000"); // 500.000 records for (int i = 0; i < 500000; i++) { // read a number from sequence, set key // yes I could just use "i" but I wanted to test sequences IntegerBinding.intToEntry((int) mySeq.get(null, 1), key); // format and assign "i" to data StringBinding.stringToEntry("N#" + formatter.format(i), data); // insert key/value pair to database testDB.put(null, key, data); } // empty the data StringBinding.stringToEntry("", data); // random generator for generating random number Random generator = new Random(); // assing a random number to key IntegerBinding.intToEntry((int) generator.nextInt(500000), key); // read from database if ((testDB.get(null, key, data, null) == OperationStatus.SUCCESS)) { // display key/value pair System.out.println("Key :" + IntegerBinding.entryToInt(key)); System.out.println("Data :" + StringBinding.entryToString(data)); } else { System.out.println("Couldn't find"); } // important: do not forget to close them! mySeq.close(); testDB.close(); dbEnv.close(); } catch (DatabaseException dbe) { System.out.println("Error :" + dbe.getMessage()); } } } |
As you see, Berkeley DB does not have columns, so how can we store a data consisting of multiple columns? We’ll solve this by using TupleInput and TupleOutput objects. Let’s say I want to store name, email and department of employees. First I define an object containing objectToEntry and entryToObject for employee record:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
package berkeleydemo; import com.sleepycat.bind.tuple.TupleBinding; import com.sleepycat.bind.tuple.TupleInput; import com.sleepycat.bind.tuple.TupleOutput; import com.sleepycat.je.DatabaseEntry; public class Employee { public String getDepartment() { return Department; } public void setDepartment(String Department) { this.Department = Department; } public String getEmail() { return Email; } public void setEmail(String Email) { this.Email = Email; } public String getName() { return Name; } public void setName(String Name) { this.Name = Name; } public Employee(String Name, String Email, String Department) { this.Name = Name; this.Email = Email; this.Department = Department; } public Employee() { } // set name, email, dept public void setEmployee(String Name, String Email, String Department) { this.Name = Name; this.Email = Email; this.Department = Department; } // display name, email, dept information public void Display() { System.out.println("Name...: " + Name); System.out.println("Email..: " + Email); System.out.println("Dept...: " + Department); } // convert object to entry public DatabaseEntry objectToEntry() { TupleOutput output = new TupleOutput(); DatabaseEntry entry = new DatabaseEntry(); // write name, email and department to tuple output.writeString(getName()); output.writeString(getEmail()); output.writeString(getDepartment()); TupleBinding.outputToEntry(output, entry); return entry; } // convert entry to object public void entryToObject(DatabaseEntry entry) { TupleInput input = TupleBinding.entryToInput(entry); // set name, email and department setName(input.readString()); setEmail(input.readString()); setDepartment(input.readString()); } private String Name; private String Email; private String Department; } |
Let’s insert sample records and then try to find (and read) them:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
package berkeleydemo; import com.sleepycat.bind.tuple.IntegerBinding; import com.sleepycat.bind.tuple.StringBinding; import com.sleepycat.je.Database; import com.sleepycat.je.DatabaseConfig; import com.sleepycat.je.DatabaseEntry; import com.sleepycat.je.DatabaseException; import com.sleepycat.je.Environment; import com.sleepycat.je.EnvironmentConfig; import com.sleepycat.je.OperationStatus; import java.io.File; /** * Author: Gokhan Atil * Web: http://www.gokhanatil.com */ public class BerkeleyDemo { public static void main(String[] args) { try { // create a configuration for DB environment EnvironmentConfig envConf = new EnvironmentConfig(); // environment will be created if not exists envConf.setAllowCreate(true); // open/create the DB environment using config Environment dbEnv = new Environment( new File("d:\\berkeleydb"), envConf); // create a configuration for DB DatabaseConfig dbConf = new DatabaseConfig(); // db will be created if not exits dbConf.setAllowCreate(true); // create/open testDB using config Database testDB = dbEnv.openDatabase(null, "testDB", dbConf); // key DatabaseEntry key = new DatabaseEntry(); // data DatabaseEntry data = new DatabaseEntry(); Employee emp = new Employee( "Gokhan", "gokhan@gokhanatil", "IT"); // assign 1 to key IntegerBinding.intToEntry(1, key); // insert into database testDB.put(null, key, emp.objectToEntry()); // assign 2 to key IntegerBinding.intToEntry(2, key); // assign new values emp.setEmployee("Jack", "jack@nowhere", "SALES"); // insert into database testDB.put(null, key, emp.objectToEntry()); // assign 3 to key IntegerBinding.intToEntry(3, key); // assign new values emp.setEmployee("Lee", "jet@lee", "BILLING"); // insert into database testDB.put(null, key, emp.objectToEntry()); // assign 2 to key IntegerBinding.intToEntry(2, key); // read from database (find key=2) if ((testDB.get(null, key, data, null) == OperationStatus.SUCCESS)) { // assign new values from dataentry to emp emp.entryToObject(data); // display emp emp.Display(); } else { System.out.println("Couldn't find"); } // important: do not forget to close them! testDB.close(); dbEnv.close(); } catch (DatabaseException dbe) { System.out.println("Error :" + dbe.getMessage()); } } } |
You can find official documents for Berkeley DB Java Edition at Oracle website:
http://www.oracle.com/technetwork/database/berkeleydb/documentation/index-160410.html
This is just a quick start for Berkeley DB Java Edition. What’s next? Secondary databases, Direct Persistence Layer, cursors, dealing with non-unique keys, transactions… I do not promise but I’ll try to write about these topics in future.
rick
Gokhan Atil
Dave Joyce
Gokhan Atil
Eric Ford
Eric Ford
Gokhan Atil
Santhana
Srimantula
keshan
Random
JustLikeThat
zeroual
vijay
Naveed Ali
Ivo