Data Migration (DSCR)
Description
Data Migration is a mandatory Overarching Standard which applies to all Suppliers. This Standard should be read in conjunction with the Catalogue Solution Migration Process ancillary document which describes the principles, key features, roles and responsibilities and the process applicable to the migration between Catalogue Solutions. The Catalogue Solution Migration process also sets out a number of SLAs with which Suppliers will need to be compliant.
If a Supplier is offering a Solution which does not store any data which would need to be migrated during a Data Migration process, they will need to provide evidence to that effect as part of the Solution Assurance process. However it is expected that all Suppliers will have in place and will submit a generic, high level Data Migration Approach to cover the scenario where they are acting both as Source and Target Solution Supplier, and a Documented Data Extract (DDE) to the Authority for assessment during the Standards Compliance phase of onboarding. This can then be used and tailored to the specific needs of a Data Migration event when required.
The Data Migration Standard supports the safe and effective migration of data between Catalogue Solutions when a Health or Care Organisation opts to change from one Catalogue Solution to another. Data Migration refers to the actions performed on multiple records stored in a Catalogue Solution, as opposed to the transfer of individual Patient/Service User records e.g. the separate transfer of part of a Patient/Service User record. In the scenario where one or more of the Solutions involved in the migration activity is not a Solution registered on the Catalogue or signed up to the Catalogue Agreement (the Framework), then this Standard and process will not apply. This does not however relieve Suppliers who wish to onboard a Solution registered on the Catalogue or to the Catalogue Agreement (the Framework) from their obligations to achieve full compliance with this Standard.
Suppliers shall provide a Data Migration Service, to support the safe and effective transfer of all data required to replicate each Patient’s/Service User’s Record in the Target Solution as it was recorded in the Source Solution at the time of migration. This includes the full set of Patient/Service User records and audit trails, including documents and attached images as well as other clinical and administrative information such as tasks and appointments where applicable to the individual migration event and will be in a human readable format without any undue degradation (i.e. flattening from structured data to plain text) or the need for access to the legacy Solution to be available. A successful and efficient Data Migration could involve activities and responsibilities from several parties:
The Source Solution Supplier
The Target Solution Supplier
Service Recipient (i.e. the organisation initiating the switch of Catalogue Solutions and responsible for overseeing the process)
The Authority and Service Management Agent
As the Catalogue Solution Migration Process and Data Migration activities inevitably involve several parties, it is essential that all parties work together and collaborate to ensure that the process of switching Catalogue Solutions is completed as efficiently and securely as possible to ensure alignment with the SLAs and in particular to ensure that any disruption to the provision of health and care services is minimised to the extent possible. This includes ensuring that the Cut-Over period from when the final data extract is taken to Business Go-Live, where manual and duplicate data entry is likely to be required, is completed within 72 hours and outside core working hours wherever possible.
The need to migrate data from one Solution to another is normally triggered by the decision of a Service Recipient (such as a Practice, CCG, CSU, STP, Care Organisation etc.) to use a different Catalogue Solution, whether from the same Supplier or a different one. However the need may also arise as a result of the merging or splitting of one or more Health or Care Organsiations or the formation/closure of other Health or Care Organsiations providing care which involves the replacement or consolidation of existing IT systems.
Data Migration exercises inevitably involve access to, and processing of, significant amounts of sensitive personal information. Therefore all parties taking part in such activities, or providing any degree of service or support to such activities, must be conscious of the risks involved, and of their responsibilities to safeguard the confidentiality of all the data being migrated. All parties involved are required to be compliant with all UK legislation and in particular the following laws, policies, standards and guidelines in respect of Information Governance:
NHS Act 2006 (Section 251) (previously Section 60 of the Health and Social Care Act 2001)
General Data Protection Regulation (GDPR) (see General Data Protection Regulation (GDPR) - information - NHS Digital)
See Information Governance Standard for further information.
It is the Service Recipient as Data Controller that has ultimate responsibility for validating and signing-off a Data Migration, however they cannot do this effectively without support from Suppliers. To ensure a consistent understanding of responsibilities, this Standard, along with the associated Ancillary Document and the Processes set out within them.
Requirements
Requirement ID | Requirement Text | Applies To | Level |
Target and Source Solution Suppliers | |||
DMI01 | Provide a Data Migration Approach document which details their approach to a Data Migration when acting both as Source and Target Solution Supplier. Note: This is a high level, generic document to be provided to the Authority as part of Standards Compliance and then to be tailored according to the needs of each Data Migration undertaken | Target and Source | MUST |
DMI02 | Provide a Documented Data Extract (DDE) that allows a Target Solution Supplier to understand and interpret the form and structure of the extracted data. This will include, but is not limited to:
Note: This is an overarching DDE to be provided to the Authority as part of Standards Compliance. | Target and Source | MUST |
DMI04 | Support, and/or provide mechanisms to support, the migration and/or re-integration of data which is stored solely in a Solution which is not the Source Solution where such a Solution is in use. | Target and Source | MUST |
DMI27 | Perform a migration of data from a source supplier to target supplier over the internet
| Target and Source | MUST |
Source Solution Suppliers | |||
DMI05 | Support the Service Recipient with improving the data quality in the Source Solution up to the minimum standard agreed by all parties in preparation for the data extracts. See documents available here for guidance regarding data cleansing and preparation of data for a migration. | Source | MUST |
DMI06 | Provide a minimum of two separate data extracts to the Target Solution Supplier and/or the Service Recipient to include:
NB. The provision of additional extracts may be required to facilitate the data transformation and data migration activities as detailed in the Data Migration Process set out in the Catalogue Solution Migration Process Ancillary Document | Source | MUST |
DMI07 | All data extracts will:
| Source | MUST |
DMI26 | All data extracts to contain the full set of audit trails where applicable to the individual migration event. | Source | SHOULD |
DMI08 | Provide a document and reports detailing any irregularities regarding the way in which data has been entered into the Source Solution (i.e. variances on the DDE), including reference to any dependencies on other Solutions and Suppliers. | Source | MUST |
DMI09 | Provide access to the Source Solution, including data and audit trails in human readable format, and support services for the duration of the agreed Run-Off period as a minimum, according to the terms of the applicable contracts and agreements for that Data Migration and/or at the request of the Service Recipient. Note: Reference access to the Source Solution may be required for some time after completion of the Data Migration activities | Source | MUST |
DMI10 | Where a third party Supplier is used during the Data Migration process, provide an automated mechanism for the third party Supplier to access and extract the data from the Source Solution. | Source | SHOULD |
Target Solution Suppliers only | |||
DMI11 | Undertake the data transformation and mapping of data from the extracts provided by the Source Solution Supplier to a format which can be loaded into the Target Solution. | Target | MUST |
DMI12 | If coded data is provided by the Source Solution and supported by the Target Solution, undertake the data transformation and mapping of data using the latest version of the approved mapping tables published by NHS Data Model and Dictionary. | Target | MUST |
DMI13 | If coded data is provided by the Source Solution, where mapping cannot be completed using the NHS Data Model and Dictionary mapping tables, provide and describe the mapping tables used during the data transformation activities. | Target | SHOULD |
DMI14 | Provide tables detailing any specific mappings between the Source Solution data and the Target Solution. | Target | MUST |
DMI15 | Facilitate as many iterations of the data transformation and mapping as required to address all errors, omissions, and irregularities to the satisfaction of the Service Recipient. Note: If greater than five iterations of the data extract and transformation process are required, this will need to be reported to NHS Digital | Target | MUST |
DMI16 | Provide tools to enable the Service Recipient to undertake the mapping of non-standard, local codes where necessary. | Target | SHOULD |
DMI17 | Provide the Service Recipient with the transformed data and a trial environment to enable the validation of the integrity, accuracy and completeness of the data. | Target | MUST |
DMI18 | Maintain an Issues Log of all issues encountered during the data transformation, including how and when each was resolved. | Target | MUST |
DMI19 | Undertake any corrective action necessary to address issues raised regarding the data transformation and document this in the Issues Log. | Target | MUST |
DMI20 | Provide comprehensive aggregated reports to enable the Service Recipient to verify the integrity, accuracy and completeness of the transformed data. As a minimum these will include:
| Target | MUST |
DMI21 | Reports will be available at all times from the live and/or trial Environments (as applicable) from the moment the corresponding environment is made available. | Target | MUST |
DMI22 | Reports to be refreshed every time data is re-loaded. | Target | MUST |
DMI23 | Load the transformed data into the Target Solution once approved by the Service Recipient. | Target | MUST |
DMI24 | Ensure staff, facilities, tools, training and support services, are in place to support the Service Recipient in use of the Target Solution and to ensure that any obligations and responsibilities can continue to be met effectively. | Target | MUST |
DMI25 | Support additional training needs and requests post Business Go-Live as requested by the Service Recipient. | Target | SHOULD |
Capabilities
All Suppliers' Solutions delivering any Capabilities will need to meet this Standard.
Roadmap