Data is growing, so Exadata does too…

The recent report from IDC and EMC (now Dell) on ‘Data Growth, Business Opportunities, and the IT Imperatives’ says, Data is growing 40% annually.

Does that mean that Exadata also grows?

Exadata was first released in 2008 as Oracle’s first “Engineered System,” popularly known as “hardware and software engineered to work together.” Since then we have seen 6 versions of Exadata with the most recent X5 Series which has X5-2 and X5-8 Exadata. Let’s talk in brief about the history of Exadata, their key components and how they have evolved.

Attending Oracle OpenWorld next week?
Be sure to check out my session on Sunday, Oct. 25th from 9-9:30 a.m. in Moscone South – 270: “Oracle Exadata, Oracle Multitenant, Oracle Database In-Memory: How They Fit Together

 

V1, the first Exadata, was released in late 2008. It was known as Exadata V1 and was a
combination of HP hardware and Oracle software. It has no Flash.

V2, the second version of Exadata, was introduced at Open World in 2009. Oracle partnered with Sun. The biggest difference was the addition of a significant amount of solid state-based storage. The storage cells were enhanced with 384G of Exadata Smart Flash Cache. Oracle software was also enhanced to take advantage of the new cache.

X2 was released at Oracle Open World in 2010. This was the first time there were two distinct versions of the X2. X2-2 and X2-8. The X2-2 has eight dual-socket database servers with hex-cores. X2-8 has larger number of CPU cores and has a larger memory footprint. In 2011, Oracle changed the hardware in the X2-8 to 8 10-core CPUs and 2TB of memory per node. Storage expansion racks were introduced.

In January 2012, Oracle increased the size of the high-capacity disks from 2TB to 3TB.

X3 was introduced in 2012. Compute node updates included eight-core Intel Sandy Bridge CPUs and increased memory up to 256GB per server. Storage servers saw upgrades in CPUs and memory, and flash storage increased to 1.6TB per server. The X3-2 family also introduced a new size—the eighth rack. X3-8 rack had some improvements in the storage servers, but the compute nodes in X3-8 racks were same as their X2-8 counterparts.

X4 was released in 2013. In this version processing was increased to 2×12 core CPUs. It had the capacity to upgrade memory to 512GB in a compute node. Flash and disk storage was also increased. It featured a new model of high-capacity disk. 600GB disks were retired, and included 1.2TB, 10,000 RPM disks. These disks were a smaller form factor (2.5” vs 3.5”). The other big change with the X4-2 was the introduction of an active/active InfiniBand network connection. On the X4-2, Oracle broke the bonded connection and utilized each InfiniBand port independently. This allowed an increased throughput across the InfiniBand fabric.

X5 was announced in early 2015, the sixth generation of Exadata. The X5-2 was a dramatic change in the platform, removing the high-performance Disk option in favor of an all-flash, NVMe (Non-Volatile Memory Express) model. High-capacity disk sizes remain the same at 4TB per disk. The size of the flash cards doubled to 6.4TB per storage server. Memory stayed consistent with a base of 256GB, upgradeable to 768GB, and the CPU core count increased to 18 cores per socket. With X5, customers are allowed to purchase X5-2 with any configuration required—a base rack begins with two compute nodes and three storage servers.

Let’s compare individual Exadata features and associated components

Storage Server CPUs, Disk Space and Flash Cache

Most frequently-accessed data on the spinning disks are automatically cached in the Exadata Smart Flash Cache area on high-performance flash disks. Exadata Smart Flash Cache provides a caching mechanism for frequently-accessed data on each Exadata Cell nodes.

The Exadata Smart Flash Cache area on flash disks is automatically created on Exadata Cells during startup. Regular disk gives a few hundred IO/Sec but Flash can offer millions of I/O s per second. Exadata software transparently moves hot data to flash cache. Exadata full rack system has 14 storage servers with a total 89.6 TB Flash cache. This Flash cache can provide 140GB/S throughput or it can manage up to 4.1 Million IOPs on full rack.

Another key beneficiary of this growing flash is Smart Scan / Storage indexes. This is a very powerful feature of Oracle Exadata Storage Server Software. Storage indexes help to avoid I/O operations. Oracle Exadata Storage Server Software creates and maintains a storage index in Exadata memory. The storage index keeps track of the minimum and maximum values of columns for tables stored on that Cell. This functionality is done transparently, and does not require any action. Any smart scan query is sent to storage via iDB (intelligent database) protocol. Once data is offloaded all the SQL processing happens inside Exadata Storage Servers including data filtering and processing. Only the rows and columns that are directly relevant to a query are sent to the database servers. This saves a lot of Database server CPU work.

Exadata Hybrid Columnar Compression (EHCC) is a new compression technology exclusive to Exadata. Oracle explains that you can achieve 10x to 50x compression based on what type of compression you choose and your data.

EHCC is a new method for storing data within a database block. Data within the data block is neither stored row wise sequentially nor column wise. As the name implies, EHCC utilizes a combination of both row and columnar methods for storing data. Because of the hybrid approach to achieve this compression we get the benefits of columnar storage while avoiding the performance drawbacks of a pure columnar format. Hybrid Columnar Compression often delivers 10X-15X compression ratios.

chart-1
The above slide shows the flash cache growth pattern with various Exadata releases

Exadata X5-2 introduces Extreme Flash Storage Servers
Each Extreme Flash storage server contains eight 1.6 TB state-of-the-art PCI Flash drives. PCI flash delivers ultra-high performance by placing flash memory directly on the high speed PCI bus rather than having slow disk controllers and directors on moving disks. We get a total of 179.2 TB Flash. With this option read performance remains the same but write performance increases from 2.6M IOPS to 4.1M IOPS. In this option HC disks are replaced with Flash.
Storage software provides many other great features which also benefit by growing CPU and Flash like:

  • Smart Flash Logging
  • Columnar Flash Cache
  • Exadata Snapshots
  • Exhaustion Direct to Wire OLTP Protocol
  • IO and Network Resource Management
  • Storage Index
  • Hybrid Columnar Compression
  • Smart Scans of Data Mining model scoring
  • Exadata Virtual Machines
  • I/O Analysis in AWR reports

chart-3
The above slide shows the storage CPU growth pattern with various Exadata releases

The chart below shows how raw storage has grown with various generations of Exadata. Usable volume depends on what kind of redundancy you choose during installation. A key point to be noted here is that because of unique Exadata compression feature (EHCC), effective data storage is a lot more than usable volume.

chart-4
The above slide shows the raw disk space growth pattern with various Exadata releases

Data load speed is also growing

chart-5

The above slide shows the data load speed growth pattern with various Exadata releases

Database Server’s CPU and Memory

CPU speed and CPU count are directly proportional to database servers processing power where in memory would facilitate any database processing on the database nodes.

While the growing CPU and memory with various Exadata generation is helping database processing and operation, Oracle In-Memory (an Oracle 12c feature) will get the most benefit out of it.

The Oracle In-Memory option allows storing data in columnar format, offering multiple times the performance gain for DSS systems where you tend to retrieve fewer columns with multiple rows. This performance enhancement is further multiplied since you retrieve that data from memory instead of slower disks. Traditionally, Oracle stores data in tables in the form of rows. This new feature will allow you to store data in memory in columnar format.

chart-6

The above slide shows the total database node CPU growth pattern with various Exadata releases

chart-7
The above slide shows the database node memory growth pattern with various Exadata releases

I would like to reiterate that the current X5 Exadata series offers extreme system scalability and growth with elastic configurations.

With X5-2 series you can choose between Full/Half/Quarter and Eighth Racks.

If you need larger storage capacity, the Oracle Exadata Storage Expansion Rack is available. The Exadata storage expansion rack enables you to grow the Exadata storage.

Capacity-on-Demand allows up to 60% of the cores per server to be turned off during the hardware installation, leaving at least 14 cores enabled. As your workload grows and more cores are needed, Capacity-on-Demand can be used.

You can stack multiple racks and they can be connected using the integrated InfiniBand fabric to form even larger configurations. For example, a system composed of four full racks is simply four times as powerful as a single rack system — providing quadruple the I/O throughput, quadruple the storage capacity, and quadruple the processors.

Oracle multitenant, In-Memory, ILM, ADO with Heat Maps add icing to the cake when run on Exadata.

Thanks for reading. Happy Exa-dating!

References:
https://en.wikipedia.org/wiki/Oracle_Exadata
http://docs.oracle.com/cd/E50790_01/welcome.html
http://ermanarslan.blogspot.com/2015/04/exadata-improving-hardware-in-exadata.html
http://it.toolbox.com/blogs/oracle-guide/exathis-exathat-what-are-these-exadata-and-exalogic-thingies-41644



« Back to blog