HITACHI DIGITAL SERVICES
Pipeline observability dashboards for Hitachi Data Reliability Engineering
Throughout my time at Hitachi Digital Services (a wholly owned subsidiary of Hitachi, Ltd.), I worked across different spaces ranging from operating with fintech companies like TCB and Hitachi's branding team, Hi-Brand shaping how they were communicated and branded, and designed tools for B2B enterprise and consulting practices. This case study is about one of those tools: Data Reliability Engineering (DRE).
TIMELINE
Jan 2023 – Jun 2024
ROLE
Product Designer
TEAM
Data engineers, a system architect and the global delivery lead
CATEGORY
B2B enterprise, internal platform
ON THIS PAGE
Overview
At a glance
Problem
Key decisions
Process
Solution
Learnings
OVERVIEW
Hitachi describes Data Reliability Engineering (DRE) as site reliability engineering applied to data: keeping the data behind analytics and AI accurate, complete and delivered on time. I designed the interface data engineers use for that work every day. Working closely with data engineers, a system architect and the global delivery lead, I designed and prototyped the first version of the product: an overview for each data domain, reconciliation statistics, a rule builder and a pipeline view, each delivered in both dark and light themes.
AT A GLANCE
v1
The first version of the observability and AI pipeline dashboards, designed and prototyped from the ground up.
36
Screens across two complete themes, dark and light, covering overview, reconciliation, rules and pipelines.
Every release
A design review with engineering and product compared each build against the prototype before it shipped.
Why bad data was hard to catch
Failures that surface too late
A broken pipeline rarely announces itself. It usually surfaces later, as an incorrect number in a report or a model trained on incomplete data.
Hitachi's own DRE material is direct about the cause: many source systems were never built with basic data checks, and they are difficult to change. Standard monitoring can tell an engineer that a job failed, but it leaves them to find the cause on their own. DRE validates data quality before the data is used and can reconcile records automatically. My role was to make that information easy to read at a glance and quick to act on.
Three decisions that shaped the product
01
A clear reading order on the overview
The overview is organized into three colour-coded bands: performance in yellow, model variations in purple and health charts in red. Operational status sits above data correctness, so engineers answer "Is it running?" before "Is it right?". Any value outside its expected range turns red, which brings the problem to the engineer instead of making them search for it.
02
Evidence one click away
A "View reconciliation stats" link sits directly on the Data Quality chart. It opens a table-by-table comparison of matched and unmatched records, with a count of error-prone tables at the top and a red or green indicator for data freshness.
03
Rules without a query language
Each rule is built from three parts, an attribute, a condition and a value, which can be chained with And/Or logic. Engineers choose an Alert, Spotlight or Hybrid type, attach an action, and switch the rule from Passive to Active, without writing a query.
How the work came together
01
Understanding the system
I worked with data engineers, a system architect and the global delivery lead to understand how DRE pipelines run, where they tend to fail and how engineers investigate those failures.
02
Defining the core jobs
We scoped the first version around three jobs: observing pipelines, detecting issues automatically, and helping engineers resolve them before they reach business dashboards.
03
Structuring the product
I organized the product into four areas, an overview for each data domain, reconciliation, rules and pipelines, so that each job had a clear place in the interface.
04
Designing for both themes
I designed every screen in dark and light themes and kept chart controls consistent throughout, with the same zoom, information and export actions on every chart.
05
Prototyping the flows
I prototyped the key flows end to end, giving engineering a concrete, interactive reference to build against rather than a set of static screens.
06
Reviewing each build
I ran recurring design reviews with engineering and product, comparing each build against the prototype before it was released.
Monitor, investigate, resolve
One workflow across four connected areas
The product gives the engineers who keep data pipelines running a single path from noticing a problem to preventing it from happening again.
Financial, customer and transaction data share the same overview layout, so an engineer who learns one domain can read all three. From any health chart, they can zoom in, check what the metric means, or open the reconciliation stats behind it. When the same issue keeps recurring, they can turn it into a rule. The pipeline view shows the Airflow DAGs that run underneath.
Overview
Performance, model variations and health, organized in three colour-coded bands.
Reconciliation
Matched and unmatched records, compared table by table.
Rules
Alert, Spotlight or Hybrid rules, built from simple conditions.
Pipelines
Airflow DAGs with their schedules, recent task runs and on/off controls.
What I took away
Design for the next action
Engineers didn't need more charts; they needed to know what to do next. That principle is why the quality chart links directly to reconciliation, and why a recurring problem can be turned into a rule in a few steps.
Design continues after handoff
Reviewing each build against the prototype with engineering kept the shipped product close to the intended design, and caught gaps while they were still inexpensive to fix.
Explore more
The Cold Mode
CareCircle
Want the full story?
The production screens are internal to Hitachi, so the visuals on this page are redrawn. I'd be glad to walk you through the work, the trade-offs behind it and what I would approach differently.
itsmehersunitha@gmail.com
◕‿◕
© 2026 MS SPACE. All Rights Reserved.