Data Engineer

Job description

Job responsibilities

NHS England is changing how NHS health and social care data is made available for research. Secure data environments (SDEs) were identified in the Data Saves Lives Strategy as the default way that NHS health and social care data will be accessed for research and analysis in future. NHSE has funded 11 sub-national SDEs, including the Yorkshire and Humber Secure Data Environment (YHSDE).

The data engineer will establish a research ready data flow from HNY into the YHSDE. They will work collaboratively across multiple organisations including NHS providers, local authorities and universities to collate and harmonise data assets and make these accessible for research via the YHSDE. You should be able to speak data and confidently discuss data-related matters with both experts and non-experts.

The data engineer will support the cataloguing and preparation of health and care data sets across HNY for upload onto the Health Data Research Innovation Gateway. This will involve applying the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) to standardise NHS data, clinical trial data and academic research data. The initial focus will be supporting the harmonisation of the 'Born and Bred In' data sets across HNY by working with data leads across four NHS Trusts. The initial aim is to prepare data for its inclusion on the YHSDE, this will involve establishing a shared data dictionary for the BaBi HNY mini-metacohort and developing a BaBi data OMOP flow.

This position will support the Innovation, Research and Improvement Team, and various internal partners including Business Intelligence Team and Population Health Management team. The data engineer will work with the IRIS Research Business Partner and to support the coordination of the BaBi study across HNY, with a specific focus on data.

Core responsibilities include:

working with various data consumers, such as business intelligence analysts, researcher data managers and population health data analyst to design and support the management of data dictionaries.

support data owners with managing their data assets and creating a system-wide data dictionary for BaBi.

working across organisations to minimise data risks and increase data literacy.

manipulating and linking different data sets, working with data controllers and information governance teams across multiple organisations.

implementing data flows to connect operational systems, data for research and business intelligence systems.

connecting BaBi data with other data sources, such as the Yorkshire and Humber Care Record or education and feeding these into the wider data lake.

supporting completion of the research section 251 to allow linked data to be used for research.

supporting work on the design and implementation of practical data governance solutions to support organisational needs.

developing, in conjunction with the YHSDE, a data access committee and approach to govern access to linked data for research and other purposes.

working along the ICB data engineering team, to a set of best practice and standards, to achieve seamless data flows between partners.

ensuring a high level of information governance and confidentiality is always in mind in the creation of data flows.

Person Specification

Experience

Essential

• Significant experience of extracting data, manipulating, understanding, transforming, wrangling and cleaning NHS datasets.
• Experience in data engineering within the healthcare sector, with a focus on NHS data systems.
• Experience designing, developing, and maintaining large-scale data pipelines and ETL processes.
• Experience with OMOP, FHIR, ICD, SNOMED codes and the NHS Data Dictionary.
• Experience in an data integration role with the ability to apply logical thinking using a wide range of data processing tools and techniques to build and integrations solution in a highly complex environment.
• Experience of exporting data from various source data systems, transforming data against required business rules and data models, or data storage and analysis requirements, and loading data in various storage platforms.
• Experience of working within and/or with multi-disciplinary teams, a proven track record of successful transformative and innovative working within health or health related settings.
• Evidence of continued further professional development.

Qualifications

Essential

• Educated to Masters level or equivalent.
• In depth specialist knowledge of information analysis and the use of information across the NHS acquired through a degree or equivalent.
• In-depth knowledge of data modelling.
• Familiarity with healthcare data standards and compliance regulations

Desirable

• A relevant qualification in information management
• Knowledge of NHS strategies, policies and plans.
• Good knowledge of Python for data engineering purposes

Skills and Competencies

Essential

• Ability to work with large data sets with very complex information.
• Ability to write well-designed, testable, efficient SQL code which follows good coding standards.
• Proficiency in programming languages commonly used in data engineering.
• Ability to optimise and troubleshoot complex data pipelines for performance and reliability.
• Excellent problem-solving skills and the ability to work collaboratively in a multidisciplinary team.
• Communicate effectively with technical and non-technical stakeholders to support and host discussions within a multidisciplinary team, with potentially difficult dynamics, and be an advocate for the team externally to can manage differing perspectives.
• Excellent analytical, IT Keyboard and numerical skills - confident with statistics and their presentation.
• Self-starter, able to work effectively within a team to tight deadlines.
• Excellent verbal and written communication (including presentation) skills.

Disclosure and Barring Service Check

This post is subject to the Rehabilitation of Offenders Act (Exceptions Order) 1975 and as such it will be necessary for a submission for Disclosure to be made to the Disclosure and Barring Service (formerly known as CRB) to check for any previous criminal convictions.

Employer details

Employer name

Humber and North Yorkshire Integrated Care Board (857)

Address

NHS Humber and North Yorkshire

Integrated Care Board

1 Grimbald Crag Court

Knaresborough

HG5 8QB

Employer's website

https://humberandnorthyorkshire.icb.nhs.uk/ (Opens in a new tab)