R12 Upgrade Performance Tuning: How to Dramatically Reduce Downtime

There are no magic touches or hacks or backdoors in this information – only a summary of the steps to reduce production downtime for major application upgrades (e.g., 11i to R12). Ready? Let’s go.

2396bd4

Infrastructure Tuning (or, “Trying to squeeze blood from the stone”)

  • Get the maximum CPUs allowed by your license agreement with Oracle for your DB box. Use L-PAR, N-PAR hardware servers or pump up your Virtual Machine.
  • Move your DB to the high-performance storage – for example, on Oracle ASM, you can simply attach volumes from new storage and do the sync on a background.
  • If your DB is on NFS,, implement Oracle dNFS as soon as possible. Why? For one of our customers, dNFS increased the upgrade speed by 3.5x.
  • Implement “huge pages” to offload your CPUs.
  • Do not forget to fix Random Generator on your application servers – it will boost performance for java process.

Database tuning

Just a disclaimer – all parameter changes below need to be adjusted for the upgrade run only. Don’t forget to revert them back to your normal operational parameters after the upgrade is completed!

  • Adjust the following parameters:
    • Set job_queue_processes = # of CPUS
    • Set parallel_max_servers = 2 X CPUs
    • Double java_pool_size
    • Make sure your streams_pool_size is not < 250M
    • Set _pga_max_size
    • Increase sga_target to 80% of your physical memory
  • Turn off your RAC (if you have one). Keep only one DB node up for upgrade – multi-node RAC configuration will degrade performance of your upgrade process significantly. (By the way: I’ll try to describe HA/RAC configuration for EBS in my next post – stay tuned!)
  • Re-create your redo-logs – no mirrors, 2GB+
  • Make sure you’ve applied the latest PSU for your DB version (especially if you’re doing an 11i to R12 upgrade with DB upgrade as well).
  • Turn off archive logging – use interim snapshot backups for rollback points.
  • Gather statistics with higher estimate.

Application tuning

This process can be logically split into few parts:

  • Skip what you can skip:
    • SLA history – do only 3-6 months during cutover and do the rest of the history later on. (It can be done on background using SLA Hotpatch). Believe me – your Finance department will not use the whole history until the period closes.
    • Application Accounting Definition (AAD) validation can be done after the upgrade.
    • Discoverer EUL Upgrade, MFS and localization additions can be done in parallel with post-upgrade steps.
  • Purge what you can purge.
    • Communicate with your business SMEs to revise your purging policy for business transactions. Our team did a huge purging of business transactions for a customer and reduced their DB size by 30% of the original.
  • Merge 12.1.1 with 12.1.1 pre-install RUP (or 12.2.0 with pre-install performance one-offs)
  • Adjust the number of parallel workers (adworker) carefully. Maximum (of cause) is 2xDB CPUs cores, BUT in the case of 24 cores, DB box = 48 workers will slow down your upgrade.
  • Distributed AD (use of multiple application nodes to run ADPATCH in parallel) is not really effective compared with staging (see below).
  • Implement staging. We use a “three-stage schema.” It allows us to skip copy/generate portions and run only the DB portion of the upgrade. This gives a significant boost for multi-NLS environments. For example:
    • Stage 1 = 12.1.1
    • Stage 2 = Stage 1 + 12.1.3
    • Stage 3 = Stage 2 + RPC + post-upgrade patches + NLS Sync patchsets + CEMLIs
  • Order NLS Sync patchsets in Oracle instead of download and apply it for each patch.
  • Push application development team to “package” remediated CEMLIs into the one pack.
  • Use “more hands on keyboard” for final post-upgrade part.


« Back to blog