Introduction to Oracle Big Data Cloud Service – Compute Edition (Part II) – Services

In my previous post, I gave a list of installed services on a “Oracle Big Data Cloud Service – Compute Edition” when you select “full” as deployment profile. In this post, I’ll explain these services and software.

HDFS: HDFS is a distributed, scalable, and portable file system written in Java for Hadoop. It stores data so it is the main component of the our cluster. A Hadoop (big data) cluster has nominally a single namenode plus a cluster of datanodes, but there are redundancy options available for the namenode due to its criticality. Both namenode and datanode services can run in same server (although it’s not recommended on a production environment). In our small cluster, we have 1 active namenode, 1 standby namenode and 3 datanodes – distributed to 3 servers.

YARN + MapReduce (v2): MapReduce is a programming model popularized by Google to process large datasets in a parallel and scalable way. is a framework for cluster resource management and job scheduling. YARN contains a Resource Manager and Node Managers (for redundancy we can create a standby Resource Manager). The Resource Manager tracks how many live nodes and resources are available on the cluster and coordinates which applications submitted by users should get these resources. Each datanode should have a nodemanager to run MapReduce jobs.

Introduction to Oracle Big Data Cloud Service – Compute Edition (Part I)

Over the last few years, Oracle has dedicated to cloud computing and they are in a very tough race with its competitors. In order to stand out in this race, Oracle provides more services day by day. One of the services Oracle offers to the end user is “Oracle Big Data Cloud Service – Compute Edition”. I examined this service by creating a trial account, and I decided to write a series of blog posts for those who would like to use this service.

In my opinion, the most difficult part of creating a Big Data ecosystem is to run many open source software projects together, and integrate them with each another. There are 3 major players on the market to help end-users to build an integrated and tested solution for big data: Cloudera, Hortonworks and MapR. Oracle has partnered with Cloudera to build the Oracle Big Data Appliance and Oracle Big Data Cloud Service. They also offer “Oracle Big Data Cloud Service – Compute Edition” based on Hortonworks. Creating “Oracle Big Data Cloud Service – Compute Edition” is simple. You get a ready-to-use big data cluster in about 15 minutes after giving the basic information such as the name of the cluster, the number of servers (nodes), CPU and disk sizes for each node, and the administrator password.

First, let’s create an “Oracle Big Data Cloud Service – Compute Edition”. After you create our test account for Oracle Cloud, you are log in to the “Oracle Cloud” dashboard. Using this dashboard you can see all your services and add new services at the same time.

How to Patch Oracle Database on the Oracle Cloud

I was waiting the latest PSU for Oracle Database 11.2.0.4 on the Oracle Database Cloud Service, and today I noticed that it’s available. So let’s see how we can update our Oracle Databases in the cloud.

First we go to the database home to see if any updates is available. As you can see, in the administration box, there is an available patch. We click on it to see the details of the patch.

It’s the PSU Update 11.2.0.4.161018! I can read the readme file of the patch (from Oracle support) when I click the readme link. On the right side, there’s an action menu to run a precheck (prerequisites check) and to apply the patch. I run the precheck first, so I can be sure that I won’t have any problem when applying the patch.

Oh, the prerequisites checks failed!

Honestly, this is not my first time to apply a patch on Oracle Database Cloud Service. In my previous try about 2 or 3 months ago, I applied the latest PSU without any problem. It was a very smooth process, took less than hour. This time, the prerequisites failed.

How to Create Storage Containers on the Oracle Cloud

When creating a Database Cloud Service, you have an option to select “cloud storage” as a backup destination. In this case, the database backups will be stored in “storage containers”. As I see, lots of Oracle Cloud users have problem with setting up storage containers as backup destination.

It’s possible to create a storage container before you create the database service or you can create it while provisioning the DB service. Whichever method you chose, you should first set up the Replication Policy.

Go to the home page of “Oracle Storage Cloud Service”, on the up right side, you’ll see the action menu button (). Click on it and set the replication policy. Be careful about choosing the data centers and replication policy, because the replication policy can not be changed once it is set!

Oracle Database Cloud Service: Set up Access Rules

I’ve started to test Oracle Database Cloud Services. I applied for trial, it’s accepted (you can also get a trial account, there’s no special requirement for it). After I got my welcome emails, I created a database service. As expected, it was very easy. All I needed is to click a couple of times, enter mandatory information and then wait until my Database is provisioned.

Oracle creates a VM for each database because they provide SSH access to the server. This gives you more power than standard PaaS but I don’t know what happens if you make some modification on the OS level such as upgrading packages, changing kernel settings etc…

Anyway, after I created a Database Service, I wanted to connect to EM but I see that I couldn’t access the page. First, I though it’s a bug, then I remembered our company firewall rules are very strict and it’s related with our firewall, then I noticed that almost all ports (expect SSH) are blocked.

databaseservice