Successful Go-live/Cutover for Oracle EBS R12 Upgrades: Nine Need-to-know Steps
My team has completed more than 40 major full-cycle Oracle E-Business Suite (EBS) upgrades over the last four years. We’ve learned quite a few things along the way – and here, I’m sharing some of our technical management experience, specifically regarding upgrades from EBS 11i to R12. Let’s start from the end.
A few key factors need to be considered for a smooth cutover:
- Resource planning:
“Each soldier should know his maneuver.” Go-live is a culmination of the team’s efforts. Everyone has to run the steps as it is indicated in the upgrade runbook. Everything that can be scripted needs to be scripted. Do not plan to work more than 12 hours non-stop – because, after 12 hours, even “super-d-duper” DBA gurus are making mistakes and typos. Managers should have at least two shifts of DBAs for go-live (and those resources should be directly involved in the previous rehearsals).
- Communication plan:
One of the major parts of communication is the hand-off procedure. Go-live is a 24/7 event. Also, team members may be in a different time zones – and may need some advance notice before they’re expected to take action to ensure that they can wake up completely. :)We’ve found it useful to use web collaboration tools to visualize the progress. Google Docs, Microsoft SharePoint, Smartsheet.com – any of these work fine.
- Production downtime:Downtime can be split into two phases:Technical (DBA) downtime – when only DBAs are accessing the system and running upgrade scripts exclusively.Functional downtime – when DBAs and functional and development teams work in parallel, but the system is not yet released for end-users.I will review upgrade performance tuning (to minimize downtime) in a future post (stay tuned!)… but one of the main factors is rehearsals!You can predict real downtime only if you’ve done at least one “dry” run when all teams are working together 24/7, exactly as it should be on a production cutover.
Always have your plan “B” ready. In case of major failure, make sure you can switch back to the original (pre-upgrade) system easily. In the best-case scenario, simply keep the old system untouched and run the upgrade on dedicated hardware. In the worst case, do the full backup.Also, use an interim backup when you complete each long-term phase:
- after pre-upgrade phase (on 11i system)
- after 12.2.0
- after 12.2.3 (or just-released 12.2.4)
- after merged patch pack and CEMLIs migrationThe aggressive backup schedule, of course, should not waste approved downtime – use your storage functionality. Snapshot backups take only seconds!By the way – my team has created a plan “B” for each of our upgrade projects, but we’ve never had a chance to execute them. It’s like life insurance.
- Scheduled Jobs:
Make sure your scheduled concurrent requests are placed on-hold before the upgrade and released when the “go” decision is made. The same rules are applicable for Workflow Mailer.
- Post-upgrade support:
Be ready to clone your production system, at least, to the one non-production instance right after the upgrade. And by “right after,” I mean immediately/as soon as possible! We are using our clone automation system to compress the clone time significantly. This clone(s) will be used to troubleshoot all post-upgrade issues that have been missed during the testing cycles.
- Network pressure:
On the first day you’re live, your network will be under pressure. All end-users (who have not been involved in UAT testing) will download new JAR files for forms as well as a new JRE plug-in (if you did not upgrade your JRE earlier). To offload pressure from your application servers , you can pre-stage your JAR cache on local file servers as well your JRE installation.
- Production monitoring:
Don’t forget to adjust your production monitoring system – R12 has a different techstack and file structure. Otherwise, you risk being spammed by a zillion fake alerts.
- Success Celebration!Congrats – your first week of live is over! No issues so far, but schedule a success celebration for when your users have closed a period (a month or quarter)! (Or celebrate it twice!)
« Back to blog