Appointments Scraping
Workflow Step 1Consults appointments in DaySmart by date range and saves them in the flat table APPOINTMENTS.
Execute before loading customers.
Scraping & Normalization Flow
Follow this order when performing a complete ScheduleApp refresh. Some processes can run independently, but this sequence keeps dependencies clear and gives direct DaySmart information priority over reconstructed historical data.
1
Scraping
Appointments
Load appointment history from DaySmart.
2
Scraping
Customers
Load customers referenced by saved appointments.
3
Scraping
Active Events
Load active Staff/Event records directly from DaySmart.
4
Recovery
Historical Events
Recover missing inactive events from APPOINTMENTS.
5
Normalization
Event / Opportunity Interface → Patients → Appointments
Build the Event / Opportunity Interface and normalize Patients first, then connect both into final Appointments.
6
Audit
Review Reports
Review executions, counts, warnings, and durations.
Important: Active Events should run before Historical Events so records returned directly by DaySmart remain the preferred source. Historical Events depends on appointment history already existing in ScheduleApp_Plain. After scraping is complete, build the Event / Opportunity Interface and normalize Patients before Appointments so PatientId and validated OpportunityId can be resolved.
Scraping
Processes responsible for importing flat data from DaySmart into ScheduleApp_Plain.
Customers Scraping
Workflow Step 2Reads the customer_id from local appointments and queries DaySmart for only the missing customers.
Execute after Appointments Scraping.
Active Events Scraping
Workflow Step 3Queries active Staff records in DaySmart and stores them as flat event data in ScheduleApp_Plain.EVENTS.
Run before Historical Events Recovery so direct DaySmart data has priority.
Historical Events Recovery
Workflow Step 4Reconstructs missing inactive events from historical APPOINTMENTS using employee and staff information already saved locally.
Requires saved appointments. Run after Active Events Scraping.
Normalizations
Processes responsible for converting flat information into normalized structures of ScheduleApp.
Event / Opportunity Interface
Workflow Step 5ABuilds the interface between flat EVENTS and Stroke OPPORTUNITIES, keeps a readable event name, preserves malformed candidates for review, and stores only validated Opportunity relationships.
Use after Active Events Scraping and Historical Events Recovery.
Patients Normalization
Workflow Step 5BNormalizes flat customers into the PATIENTS table, merging possible duplicates by first name, middle name, last name, and phone number.
Use after loading flat customers.
Appointments Normalization
Workflow Step 5CNormalizes flat APPOINTMENTS, resolves internal PatientId and validated OpportunityId, converts DaySmart date/time values, and synchronizes the final appointments.
Run after the Event / Opportunity Interface and Patients normalization so both final relationships can be resolved.
Reports
Queries to review executions, results, times and metrics of scraping or normalization.
Scraping Reports
Workflow Step 6Consult the records saved in SCRAPING_REPORTS to review scraping executions.
Available with process, status and scraping range filters.
Normalization Reports
Consult records saved in NORMALIZATION_REPORTS to review normalization executions and synchronization results.
Available with process, object, status and execution-date filters.