The university's data analytics and architecture team faces significant challenges due to the current hybrid infrastructure, which combines cloud and on-premise solutions. The system struggles with data reconciliation, standardization, and scalability, hindering the university's ability to rapidly implement new features and conduct advanced analytics.
Key issues include:
- Archaic Infrastructure: The existing mix of cloud and on-premise applications creates difficulties in managing data effectively.
- Data Silos: Data is scattered across various systems, making it challenging to obtain a unified view and apply predictive analytics.
- High Maintenance Costs: A significant budget is spent on maintaining and patching old data sources and integrations.
- Complex Data Processing: The diverse data processing techniques and technologies used contribute to inconsistencies and inefficiencies.
- Security and Governance: Ensuring data security and governance is critical, given the university's growth and increasing data volume.
The goal is to establish a design for:
- Unified data access and storage
- Prepping data for downstream processes, removing irregularities in data.
- Scalable data ingestion and extraction
- Data security
- Easy of hooks for integration
- Optimal storage usage for different use cases
- Cost efficiency without reducing the Effectiveness of the use cases.
- Scalability in overall design to accommodate future business growth