Large Utility Improves Project Coordination and Efficiency

An automated data migration methodology from Knowledge Relay seamlessly synchronizes diverse systems, adding innovation and teamwork to a successful project management system switch-over.

For most of us, it is difficult to understand the challenges facing large entities, especially one that operates 11 nuclear units at six sites in the Carolinas. Much of the utility’s growth was achieved through acquisition, and that presents its own set of unique hurdles. While combining multiple companies together and working to become efficient, the company also must reliably provide power 24 hours a day from a variety of energy sources. There is no question that such an entity is an important part of the country’s energy grid. Their nuclear fleet alone generates approximately half of the electricity provided to its customers in the Carolinas, with production costs among the lowest in the nation.

How to Eat an Elephant

Just over five years ago, the utility acquired Progress Energy, itself a combination of multiple power producers. To achieve efficiencies of scale and stay up-to-date with the latest and best technologies, the company knew it needed to align key management systems. But moving a ship of such size would be no easy challenge. The company decided to bring on a team of experts to help, many of them leaders in their respective fields. The goal was to complete an analysis of everything that needed to be done, sync up the teams, and schedule it out—one big project.

As the transformation began, the team had to deal with numerous challenges, including basic human nature: resistance to change. Many very qualified people had been refining and improving systems for years, often working in “silos” and they had strong opinions on the way forward. Getting agreement was not going to be easy. The effort ran into trouble, almost from the beginning.

After spending millions of dollars through many hours of research, design, and development, yet still coming up short in many areas, it became clear that the project needed to be broken up into smaller pieces. “Our initial goal was to bring in experienced managers to oversee the entire project.” said Rita Teale, senior project lead. “Unfortunately, we may not have fully appreciated how important the detailed domain expertise would be to the conversion between two large-scale enterprise-level systems.”

After several attempts at coordinating the efforts of multiple groups, from I.T. technicians to project management experts, yet remaining unable to move the data in any real usable form, the company decided to break the project up into smaller pieces. This would allow them to go out to bid for specialists on each individual component—eat the proverbial elephant, if you will, one piece at a time.

The biggest challenge was to find bidders that had the domain expertise using two very specialized systems: Artemis ProjectView™ and Oracle Primavera™ P6 EPPM. Whomever the company decided to use would need a keen understanding of the database architectures of both systems. Additionally, they would need specialized tools to perform the conversion at a speed that would move millions of records accurately, and in an automated way. And lastly, the successful bidder would need to be able to demonstrate that they knew how to do it and had done it before—all at a reasonable cost. The company was already over budget by millions of dollars and could not afford more problems.

All is Not Lost…

Often, when a project is struggling, there is a built-in bias to rescue it, even if the original concepts were flawed. Once it is finally realized that a true rescue is impossible—the project is WAY over budget and behind schedule — the desire to “throw the baby out with the bath water” creeps in. Confusion, disarray, blame and anger cause us all to want to chuck the whole thing and start over. But the company, in their wisdom, realized that a good deal of salvageable work had been done already. They had spent a lot of time mapping data sources and destinations, defining data characteristics, and determining what was important and what wasn’t. This would be invaluable to any new team, of course, but even a thorough analysis doesn’t always tell the whole story. The nuances involved can sometimes bite you when you least expect it—things like diverse normalization, and understanding why each choice was made, or how best to handle resources, and how many code fields are needed or desired. By aligning domain expertise to back-end database knowledge, the team was able to come up with a strategy to not just move data, but deliver it in a state that would be of greatest benefit (and efficiency) to the new tool.

Creating an Automated Methodology

With an effective design in hand, it was time to start coding. However, coding for the sake of utility will often miss the mark. There was a significant performance-time constraint from the outset. Many performance milestones were set to ensure that the design would produce the desired result in a reasonable amount of time. Of course, there will always be “unknowns” on any project, but the milestones were created to track project progress and prove capabilities. Completion meant that they had to validate the design at the same time.

The Executive Briefing

After so many missed milestones, it’s easy to understand a fair level of skepticism. Naturally, an early Executive Review would need to be conducted to determine high-level status and to ensure that the project was launched properly. It is often the first indicator of how the project will go. Naturally, all stake holders want to be represented, and vendor management needs to provide a level of confidence that they can actually deliver what they promised, on time, and on budget.

One of the first agenda items in the Executive Review was a milestone status review. Deliverable #1 involved demonstrating that core data could be successfully moved between systems without crashing either one. This may seem like a “given” but after multiple prior attempts and failures by other vendors, the company wanted an initial proof of capabilities. Who wouldn’t? After all, it’s really not a trivial exercise, so when the company’s Solution Delivery Director, Jennifer James, jumped right to this milestone, expectations were low. No one really expected data to be in the target database, at least not so early in the project. The questioning began with, “How far behind are we?” and “When can we expect any source data to be in the target database?” When Steve Mauss, Knowledge Relay’s CEO, replied that it was his understanding that data had already been moved and was in the target database, company management directed an internal database specialist, also in attendance, to login and validate that such was the case. Upon doing so, he discovered that the milestone had, indeed, been met. Data was moving, the methodology was validated, and the project was on schedule. High-fives and fist pumps all around.

Detailed Build-Out

With the design-build concepts validated, Knowledge Relay’s data migration team began the detailed build-out. Not only would current projects need to be migrated, historical data needed to be moved, validated, and stored as well. The quantities were in the millions of records, all needed for potential regulatory audits and historical reference points.

After the Planning and Design Phases were completed, Knowledge Relay built a multi-faceted data migration suite of “jobs” using Java-based toolset, Talend®. This unique and powerful tool helps developers create sophisticated mapping and migration designs in a visual way. The base Java code is auto-generated, reducing development time significantly. After specialized code is added to accommodate project-specific details, the migration job can be compiled and testing can begin.

With compiled jobs at the ready, the team loaded them into Knowledge Relay’s automated data migration tool, Data Integration Scheduler™ (DIS). DIS is a graphical data job scheduling tool that also allows input of variables at runtime. It monitors job processing status and captures any errors or anomalies. When a job is finished, DIS sends an email to stakeholders on a pre-defined list to let them know they can analyze results.

“DIS turned out to be an invaluable tool, both during the testing phase and final deployment. We were able to not only run multiple scenarios, but because of the way DIS can break jobs up and run them on multiple servers simultaneously, we were able to get the processing time down to a fraction of what it would normally take to process such large data moves.” said John Rosenlof, Lead Developer at Knowledge Relay.

With processing time down to a minimum, multiple “assurance” jobs could be run during the “Stabilization and Close” phase. The company’s internal team could spend their time learning their new Primavera P6 EPPM project management system instead of waiting around to see if the data came across correctly.

After the company’s initial experiences, they were concerned that the complexity and volume of data would make the project almost impossible to complete. The company, working together with Knowledge Relay, added their project experience to Knowledge Relay’s domain expertise on both the source and destination systems. This allowed them to complete the project ahead of time and under budget. That’s almost unheard of on these kinds of projects.

Lessons Learned

During the Close-out phase, the team took some time to evaluate what made this project successful when others had failed. Here are their findings:

• Domain expertise, including knowledge of source and destination systems, is vital. The old idea that “a data move is just a data move,” or that there is always a simple solution to a complex problem must be abandoned.
• Automation allows for the run and re-run of multiple scenarios, with each one validating the quality of the data and its suitability in the new system. By the time the production run is due, there should be a very high level of confidence in the system.
• Breaking huge projects into component parts and using resources with specific skillsets for each part led to a much better outcome.
• Aligning the interests of the Business and I.T. sides of the house is critical. A high level of trust is required, and the right vendor can help create that environment.
• A strong Project Manager can make all the difference. It’s important to have someone who consistently pushes toward the finishing line, making sure the project stays on schedule and budget.