Back to Success Stories

Enterprise Management & Monitoring

Tuesday, July 30, 2019

CITI is taking the State Department’s mission-critical enterprise management and monitoring system to new levels of sophistication and reliability. Now, from one location, DOS can view and act immediately to mitigate the risks of outages and other emergency situations in its multiple centralized data centers using real-time intelligence about the state of its various server farms and about the health of its entire computing network.

The Department of State (DOS) was mandated by Office of Management and Budget (OMB) to implement a plan for a massive consolidation of their data centers and server farms to ensure that its IT services would be more efficient and still maintain high marks for quality of service (QoS). To meet these objectives, DOS designed and implemented its Enterprise Server Operations Center (ESOC) to manage the customer systems and devices located within its newly consolidated data centers. However, several challenges stood in the way of the proposed changes:

  • DOS had hundreds of different customer groups with thousands of devices spread out in multiple locations.
  • Users had thousands of local databases, spreadsheets, and other disparate data stores used for daily operational needs.
  • Current tools installed in the data centers for maintaining configuration management data were not going to be sufficient to meet the new control and quality requirements of the OMB mandate.
  • In the midst of the massive data center consolidation…
    • software applications would need to be migrated
    • some systems would still need to be maintained or even enhanced
    • new applications and systems would need to be developed

The Department knew that if the new ESOC was to properly manage the services afforded by the newly consolidated data centers, it would need an integrated hardware-software system that stored physical and logical configuration device information; provided device monitoring and alert notifications; and offered problem escalation capabilities. The need for this system became more and more mission-critical every month that more high priority systems were migrated into the control of the ESOC.

Out of this critical need was born the original concept for the robust enterprise systems management (ESM) solution that would support the ESOC—and the entire U.S. Department of State. In its nascent form, and before CITI was brought into the picture, the system was as a web-based COTS product created to track data center customers and systems. Gradually, the system was enhanced to handle all of the data center-specific services and administrative functions such as acquisition requests, service requests, and time recording.

DOS knew this highly critical enterprise tool had to be transformed for it to keep pace with the organization as the data center consolidation project matured. For ESOC’s ESM solution to be extensible and scalable for years to come, it needed an automated workflow application with a web service, a new monitoring solution interfacing with server management tools, and other enhancements to support the multi-faceted mission of the ESOC (i.e., higher system availability, data center scalability, enhanced security, and improved cost containment). DOS also knew that to make this happen, it would need a provider with the resident experience and talent equal to the customized software and infrastructure modernization tasks required. It knew it would need CITI.

CITI was already well known within DOS. In several preceding projects, CITI performed upgrades to the virtual machine platforms, web servers, and database software of the State Department. Additionally, CITI had a long history of producing innovative solutions for Visa and Passport Services and other global initiatives for the Department.

For this new ESM enhancement effort, CITI focused its considerable expertise on building these major infrastructure management and system capabilities for DOS:

  • A web service that interfaces with business process management (BPM) tools to automate internal processes and enable faster data exchange and improved performance
  • An IT chargeback service that has enabled State to move forward with its IT as a service (ITaaS) strategy
  • A maintenance tracking service that proactively enables advance notifications of hardware service contract expirations and renewals
  • A virtualized and redundant enterprise environment for DOS that is monitored… all of its outages mitigated or minimized… and all of its enterprise infrastructure events managed – 7x24x365

CITI’s platform consolidations, successful migrations, and enhancements for the ESOC’s ESM have significantly improved the functionality and the quality of system services and have resulted in a reduced total cost of ownership for the U.S. State Department. The solution has become the critical hub for managing the entire DOS infrastructure and it is now the cost-saving system it was originally envisioned to be.