About the client
The customer is a global investment manager focused on transportation and logistics real estate. The firm acquires, develops, finances, and manages real estate and infrastructure assets that serve land, air, sea, and rail networks, with a diversified portfolio spanning multiple regions. As the portfolio expanded, deal, asset, and investment data grew in both volume and complexity, and the firm needed a modern data foundation that could keep pace with its investment operations and reporting needs.
Client challenges
As the client’s investment platform grew, teams faced increasing friction across the systems and processes underpinning deal and portfolio data.
- No Standardized Data Foundation: Dealpath data was not consistently ingested, structured, or curated for downstream reporting, which limited trust in the data and made analytics difficult.
- Manual, Error-Prone Reporting: Data quality and reconciliation issues drove heavy dependence on manual processes, with a large share of business-requested reports still assembled by hand in spreadsheets, a time-consuming approach that increased the risk of inconsistent or outdated data reaching decision-makers.
- Complex Deal Platform Metadata: The deal platform’s configurable objects, covering deals, assets, investments, properties, and field definitions, required significant normalization, mapping, and business-rule application to transform API-based structures into a clean reporting model.
- Fragmented Enterprise Data: Deal platform data needed to be unified with the vendor management system and other enterprise sources into a common semantic and reporting layer. Without this unification, cross-source reporting and portfolio insights remained fragmented.
Solution
LevelShift designed and built a Fabric-based data foundation to close these gaps, delivering validated, analytics-ready data across the client’s core systems. The solution included:
- Secure Ingestion Framework: Authenticated, controlled connectivity with monitored execution brings deal and asset data into Fabric reliably and securely.
- Bronze and Silver Layer Architecture: Raw data lands in Bronze and is transformed into structured, standardized Silver tables ready for downstream use.
- Core Object Extraction: Key entities such as deals, assets, properties, investments, and field metadata are extracted directly from the source platform’s API.
- Data Normalization and Mapping: Nested API structures are normalized, relationships are mapped, and bridge and lookup tables support clean, consistent reporting.
- Full and Incremental Load Logic: Full-load and incremental-load pipelines support recurring refreshes while reducing unnecessary processing.
- Validation and Reconciliation: Built-in checks confirm source-to-target completeness and accuracy on every load.
- Stability and Monitoring: The pipeline is tuned for error handling and recovery, with monitoring in place to flag API or load failures early.
Gold Layer Readiness: Curated data is staged for a future Gold, semantic reporting layer in Fabric to support advanced analytics and AI use cases.
Benefits
- Faster Data Availability: Full deal and asset data extraction now completes in roughly 10 minutes, compared to no dependable automated baseline previously.
- Near Real-Time Refresh Cadence: Incremental refreshes run every 15 minutes during business hours, keeping reporting current throughout the day.
- Standardized Reporting Structure: A standardized 25-field core structure now supports key deal and asset entities, replacing a highly variable schema that was difficult to scale.
- Reduced Reporting Complexity: Separating core fields from extension and custom-field data keeps reporting volumes manageable even as custom fields multiply records per deal and per property.
- Aligned Ingestion Throughput: API load planning is now aligned to approximately 1,000 records per minute, giving the team a measured, predictable ingestion baseline.
- Scalable, Governed Data Foundation: Data now flows through Bronze, Silver, and Gold layers in Microsoft Fabric, replacing a fragmented pipeline with a governed foundation built for downstream analytics.