System Design: Health Data Exchange (HL7 FHIR)

Requirements

Functional Requirements:

• Route clinical documents and FHIR resources between participating healthcare organizations (hospitals, clinics, labs, pharmacies, payers)
• Support query-based exchange (on-demand record retrieval via FHIR APIs) and event-based exchange (push notifications on clinical events)
• Implement a Master Patient Index (MPI) for cross-organization patient identity matching
• Patient-directed exchange: patients authorize sharing of their records with specific providers or apps via SMART on FHIR
• Support HL7 FHIR R4 Bulk Data Export for population health analytics and quality measure reporting
• Consent management enforcing patient privacy preferences across all data flows

Non-Functional Requirements:

• Connect 10,000+ healthcare organizations with 200M patient records
• Handle 5M FHIR API requests/day with p99 latency under 1 second for query-based exchange
• Process 2M clinical event notifications/day (ADT messages, lab results, discharge summaries)
• HIPAA compliance with encryption at rest (AES-256) and in transit (TLS 1.3), minimum necessary standard enforcement
• 99.95% availability — exchange downtime disrupts care coordination across the network

Scale Estimation

With 10,000 connected organizations and 200M patient records, the MPI contains 200M golden records with 800M+ cross-organization identity links (patients appear at an average of 4 organizations). Query-based exchange: 5M FHIR requests/day = 58 QPS average, peaking at 300 QPS during business hours. Event-based exchange: 2M ADT/clinical events per day = 23 events/sec average, each fanning out to an average of 3 subscribed organizations = 6M notification deliveries/day. Bulk Data Export: 500 organizations run monthly quality reporting exports, each extracting 100K-1M patient records — clustered at month-end, generating 50TB of FHIR NDJSON over 3 days. Document storage: clinical documents (CCD-A, FHIR Bundles) average 50KB, with 10M new documents/day = 500GB/day.

High-Level Architecture

The Health Information Exchange (HIE) platform acts as a federated network hub connecting healthcare organizations. It does not store a central copy of all clinical data — instead, it maintains a Directory of participating organizations and their FHIR endpoints, an MPI for patient identity resolution, and a Routing Engine that directs queries and events to the appropriate organizations. This federated model minimizes data concentration risk and respects data ownership.

The query-based exchange flow: a requesting provider submits a patient record query → the Exchange Gateway authenticates the request (mTLS + OAuth 2.0) → the MPI resolves the patient's identity across organizations → the Consent Engine checks the patient's sharing preferences → the Query Router fans out FHIR queries to organizations that have records for this patient → responses are aggregated, de-duplicated, and returned as a consolidated FHIR Bundle. The event-based exchange flow: a hospital's EHR publishes an ADT (Admit/Discharge/Transfer) event to the exchange → the Event Router identifies organizations that have subscribed to notifications for this patient (primary care provider, health plan, care management program) → the Event Delivery Service pushes FHIR Subscription notifications to each subscriber's webhook endpoint.

The platform implements TEFCA (Trusted Exchange Framework and Common Agreement) standards, serving as a Qualified Health Information Network (QHIN). All data in transit is encrypted with TLS 1.3, and all data at rest (MPI records, routing metadata, cached documents) is encrypted with AES-256 via AWS KMS. An extensive audit system logs every data exchange transaction for HIPAA compliance and network monitoring.

Core Components

Master Patient Index (MPI)

The MPI is the identity backbone of the exchange network, resolving patient identities across 10,000 organizations that each use different medical record numbers. The matching algorithm uses a probabilistic approach combining multiple identifiers: first name, last name, date of birth, gender, SSN (when available), address, and phone number. Each field is compared using appropriate similarity functions — Jaro-Winkler for names (handling misspellings and name variations), exact match for DOB, and normalized comparison for addresses. Field comparisons produce weighted log-likelihood ratios combined into a composite match score using the Fellegi-Sunter model. Scores above 0.95 auto-link; 0.75-0.95 go to manual review; below 0.75 are treated as no match. The MPI stores identity links: (patient_id_global, organization_id, patient_id_local, match_confidence, link_status). PostgreSQL with trigram GiST indexes handles 300 QPS of identity resolution queries. A referential matching mode allows organizations to submit demographics and receive the global patient ID without querying clinical data, enabling care gap identification.

Event Router & Subscription Engine

The Event Router processes clinical events (ADT messages, lab results, care plan updates) and delivers notifications to subscribed organizations. Subscriptions are defined using FHIR Subscription resources: an organization subscribes to events matching specific criteria (e.g., "notify me when any of my attributed patients is admitted to a hospital in the network"). The subscription criteria are compiled into a Flink streaming application that evaluates incoming events against all active subscriptions. With 2M events/day and 500K active subscriptions, the Flink job evaluates each event against a subset of relevant subscriptions (filtered by patient identity) — averaging 10 subscription evaluations per event = 20M evaluations/day. Matched events are published to per-organization Kafka topics, and a Delivery Service consumes from these topics and delivers FHIR notifications via webhook (HTTPS POST) with retry logic (exponential backoff, 3 retries, dead-letter queue for persistent failures). Delivery status is tracked and surfaced to network administrators.

Consent Management Engine

The Consent Engine enforces patient privacy preferences across all data exchange flows. Patients can set granular consent directives via their patient portal or by contacting their provider: opt-in to network sharing (default in most states), opt-out entirely, or set category-level restrictions (e.g., "share everything except behavioral health records"). Consent directives are stored as FHIR Consent resources in PostgreSQL with columns: patient_global_id, scope (TREATMENT/PAYMENT/OPERATIONS/RESEARCH), action (PERMIT/DENY), category_restrictions (JSONB array of restricted FHIR resource types or sensitivity codes), actor_restrictions (specific organizations permitted or denied), valid_period. The Consent Engine is invoked synchronously in the query path — before any clinical data is returned, the engine evaluates the patient's active consent directives against the requesting organization, the purpose of use, and the data categories being requested. For event-based exchange, consent is evaluated before the event is delivered to subscribers. Performance is critical: consent evaluation adds latency to every exchange transaction, so directive lookups are cached in Redis with a 5-minute TTL, keyed by (patient_global_id, requesting_org_id).

Database Design

The MPI uses PostgreSQL (Citus-sharded by patient_global_id). Core tables: patients_global (patient_global_id, golden_record JSONB containing merged demographics, created_at, updated_at), identity_links (link_id, patient_global_id, organization_id, patient_id_local, match_score, link_status CONFIRMED/PENDING_REVIEW/REJECTED, linked_at, reviewed_by), organizations (org_id, org_name, fhir_endpoint_url, org_type HOSPITAL/CLINIC/LAB/PHARMACY/PAYER, tefca_status, certificate_thumbprint). Trigram GiST indexes on the golden record's name fields enable fast fuzzy search.

The Subscription database: subscriptions (subscription_id, subscriber_org_id, criteria_fhir JSONB, channel_type WEBHOOK, channel_endpoint, channel_headers_encrypted, status ACTIVE/PAUSED/ERROR, created_at), delivery_log (delivery_id, subscription_id, event_id, patient_global_id, delivery_status PENDING/DELIVERED/FAILED, attempt_count, last_attempt_at, response_code). The consent database: consent_directives (consent_id, patient_global_id, scope, action, category_restrictions JSONB, actor_restrictions JSONB, valid_from, valid_to, source_organization_id, authored_at, status ACTIVE/REVOKED).

API Design

• GET /fhir/r4/Patient/$match — Submit patient demographics for identity resolution; body contains a FHIR Parameters resource with demographic fields; returns matched patient global ID with confidence score and linked organizations
• GET /fhir/r4/Patient/{globalId}/$everything?_type=Condition,MedicationRequest,Observation — Query all network organizations for a patient's records filtered by resource type; returns aggregated FHIR Bundle from all responding organizations
• POST /fhir/r4/Subscription — Create an event subscription; body contains FHIR Subscription resource with criteria (e.g., "Encounter?patient={globalId}&type=emergency") and notification channel (webhook URL)
• POST /fhir/r4/$bulk-export?_type=Patient,Condition,Observation&_since=2024-01-01 — Initiate FHIR Bulk Data Export for population health; returns Content-Location for polling export status; completed export provides NDJSON file URLs_

Scaling & Bottlenecks

The aggregated query path ($everything) is the primary latency challenge. A single patient query fans out to 3-4 organizations on average, each with different FHIR server response times (ranging from 200ms for cloud-native EHRs to 3 seconds for legacy systems with FHIR facades). The aggregation service uses parallel fan-out with a 5-second timeout — responses arriving within the window are included; slow responders are excluded with a note to the caller. To improve response times, the exchange maintains an optional Document Cache: when organizations push clinical documents via event-based exchange, the document is cached in S3 (encrypted, 30-day TTL). Subsequent queries check the cache first and only call the source organization if the cache is empty or stale. Cache opt-in is per organization (some prefer no caching for data governance reasons).

The MPI matching algorithm at 300 QPS is computationally intensive due to fuzzy string comparisons. This is optimized with a blocking strategy: before running full probabilistic matching, the system uses deterministic blocks (same DOB + first 3 chars of last name) to reduce the candidate set from 200M to typically under 100 records. Blocking reduces comparison operations by 99.99%. The Bulk Data Export at month-end generates massive network load — this is managed by a job scheduler that staggers exports across the 3-day window and rate-limits concurrent exports to 50 organizations at a time.

Key Trade-offs

• Federated architecture over centralized data repository: Federation respects data ownership and minimizes breach impact (no single database with all patient data), but increases query latency due to real-time fan-out and makes data quality dependent on each organization's FHIR server reliability — the optional document cache partially mitigates latency
• Probabilistic MPI matching over a national patient identifier: No universal patient identifier exists in the US, so probabilistic matching is necessary — but it produces false positives (incorrectly merged records, clinically dangerous) and false negatives (missed records, reduces care coordination value), requiring ongoing stewardship
• Synchronous consent evaluation over pre-computed access lists: Evaluating consent at query time ensures decisions reflect the latest patient preferences, but adds 10-20ms latency per request — cached consent directives with short TTLs balance freshness and performance
• TEFCA compliance over proprietary exchange protocols: TEFCA provides a national trust framework reducing point-to-point integration burden from O(n²) to O(n), but the standard is still maturing and some organizations lack TEFCA-ready infrastructure — legacy HL7v2 ADT adapters bridge the gap for organizations not yet on FHIR