Job summary
Join a passionate NHS team helping to shape the future of healthcare.
The Health Economics Unit is looking for a Senior Data Engineer to provide vital data and insights, as we support health and care providers in making impactful decisions to improve population health.
As a passionate member of our outstanding team, you will have a range of responsibilities, including:
-
Data extraction and integration
-
Data workflow management
-
Data processing and transformation
-
Data storage and management
-
Quality assurance and governance
-
Data analysis
-
Collaboration and development
The Senior Data Engineer will sit within the Data and Analytics team, led by the Chief Analyst and reporting to the Lead Data Engineer, and working closely across HEU team, with data scientists, analysts, project managers, and health economists.
You will also engage with our clients, partners and stakeholders throughout projects, understanding their needs and communicating your work.
The Health Economics Unit (HEU) is part of the Midlands and Lancashire Commissioning Support Unit (ML). We deliver impactful insights that power the future development of health and care delivery. Our experts support health system and industry leaders to make the best possible decisions and design the highest quality, most efficient and most innovative services to improve the health of communities across the country.
Main duties of the job
The Senior Data Engineer provides vital data and insights to inform decisions, driving improvements in healthcare and impacting millions of people. Working in a collaborative environment, with data scientists, economists, data engineers and other healthcare professionals, you will be extracting, transforming, and cleaning data ready for analytical projects.
NHS ML is committed to our process of redeployment at risk members of our existing workforce to new roles. As such, all job adverts are subject to this policy and we reserve the right to close, delay or remove adverts while this process is completed. If you do experience a delay in the shortlisting stage of the recruitment cycle, please bear with us while this process is completed, and contact the named contact if you have any questions.
About us
The NHS is the biggest employer in Europe. It's a world-renowned institution and an exciting place to work, full of challenges and opportunities.
NHS Midlands and Lancashire Commissioning Support Unit (MLCSU) is one of the biggest and best-performing commissioning support units in the country, with among the highest levels of staff and customer satisfaction. We work together as a team of over 1,600 expert staff based across the Midlands and North West to make a difference - for our customers, patients and communities.
Offering a full range of professional services to hospital trusts, local authorities, integrated care systems (ICSs) and other public bodies across the country, our NHS values underpin everything we do.
At the HEU we embrace a flexible and agile working style that balances the needs of you as an individual, theteam and our services. Our office is located in Canary Wharf, London. Depending on the working pattern that suits you, you can choose between working in the office two days a week, necessary for High Cost Area Supplement (HCAS), or homeworking; in the office for key team days, meetings and when appropriate to your ways of working.
Job description
Job responsibilities
Data extraction and integration:
In this role, you will design, build, and maintain code to extract data adapting your methods based on the environment. You'll take ownership of the methodologies used for data extraction, regularly reviewing and refining them to ensure their effectiveness. Your expertise will enable you to rapidly understand and evaluate various databases, structured files and APIs, considering their structure, documentation, and contents. You'll efficiently handle complex datasets, extracting data from large-scale NHS datasets, publicly available health data, and commercial or research environments. At the HEU we regularly work with a wide range of data, examples include SUS, Fingertips, CPRD, and OpenSafely.
Data workflow management:
Managing the flow of data through pipelines involves ensuring the development and maintenance of reproducible analytical pipelines (RAPs). Your tasks will include scheduling jobs, monitoring performance, handling errors, testing, and automating complex processes. When querying large datasets, even in environments where control may be limited, you will employ techniques that continue to ensure efficiency.
Data processing and transformation:
Planning, developing, and evaluating methods and processes for transforming and cleaning data is vital to producing valuable insights. Collaborating closely with colleagues, youll appreciate their data requirements, allowing you to direct the processing and transformation of raw data into formats ready for analysis. As part of this process, youll produce flexible and replicable code to ensure efficiency, effectiveness, and security.
Data storage and management:
Your responsibilities also extend to efficiently storing and managing large volumes of data. This involves selecting appropriate storage solutions for performance and developing comprehensive documentation for assets and analyses. You will also support with ensuring datasets are held in an appropriately secure environment and destroyed when appropriate.
Quality assurance and governance:
You will always maintain security and the confidentiality of personal and commercially sensitive data in line with the relevant information governance (IG), standards and policies. Additionally, you will investigate data quality issues, understanding how inconsistencies could affect ability to analyse the data you extract. You will provide work arounds when possible and articulate inconsistencies and solutions to the team and clients. You will support the team to implement techniques to ensure efficient queries; advising on queries across the entire team.
Data analysis:
You'll also engage in data analysis tasks from time to time. Collaborating closely with the Data Scientists in our team, you will use creative approaches to extract valuable information to provide clear insights to decision-makers.
Collaboration and development:
Collaborative coding approaches through version control and peer review will enable our work to be of the highest quality. You will ensure client requirements are met by supporting project managers with required documentation and delivering on time and within budget. You will instill a culture of learning and knowledge sharing; you will develop and maintain your specialist knowledge whilst providing educational opportunities to enhance colleagues skills in your areas of expertise.
Job description
Job responsibilities
Data extraction and integration:
In this role, you will design, build, and maintain code to extract data adapting your methods based on the environment. You'll take ownership of the methodologies used for data extraction, regularly reviewing and refining them to ensure their effectiveness. Your expertise will enable you to rapidly understand and evaluate various databases, structured files and APIs, considering their structure, documentation, and contents. You'll efficiently handle complex datasets, extracting data from large-scale NHS datasets, publicly available health data, and commercial or research environments. At the HEU we regularly work with a wide range of data, examples include SUS, Fingertips, CPRD, and OpenSafely.
Data workflow management:
Managing the flow of data through pipelines involves ensuring the development and maintenance of reproducible analytical pipelines (RAPs). Your tasks will include scheduling jobs, monitoring performance, handling errors, testing, and automating complex processes. When querying large datasets, even in environments where control may be limited, you will employ techniques that continue to ensure efficiency.
Data processing and transformation:
Planning, developing, and evaluating methods and processes for transforming and cleaning data is vital to producing valuable insights. Collaborating closely with colleagues, youll appreciate their data requirements, allowing you to direct the processing and transformation of raw data into formats ready for analysis. As part of this process, youll produce flexible and replicable code to ensure efficiency, effectiveness, and security.
Data storage and management:
Your responsibilities also extend to efficiently storing and managing large volumes of data. This involves selecting appropriate storage solutions for performance and developing comprehensive documentation for assets and analyses. You will also support with ensuring datasets are held in an appropriately secure environment and destroyed when appropriate.
Quality assurance and governance:
You will always maintain security and the confidentiality of personal and commercially sensitive data in line with the relevant information governance (IG), standards and policies. Additionally, you will investigate data quality issues, understanding how inconsistencies could affect ability to analyse the data you extract. You will provide work arounds when possible and articulate inconsistencies and solutions to the team and clients. You will support the team to implement techniques to ensure efficient queries; advising on queries across the entire team.
Data analysis:
You'll also engage in data analysis tasks from time to time. Collaborating closely with the Data Scientists in our team, you will use creative approaches to extract valuable information to provide clear insights to decision-makers.
Collaboration and development:
Collaborative coding approaches through version control and peer review will enable our work to be of the highest quality. You will ensure client requirements are met by supporting project managers with required documentation and delivering on time and within budget. You will instill a culture of learning and knowledge sharing; you will develop and maintain your specialist knowledge whilst providing educational opportunities to enhance colleagues skills in your areas of expertise.
Person Specification
Experience
Essential
- Considerable experience of extracting data, manipulating, understanding, transforming, wrangling, cleaning, and storing health data.
- Ability to write well-designed code (e.g. SQL or Python) which follows good coding standards.
Skills
Essential
- Demonstrates leadership skills in approach to data engineering tasks and delivery of projects on time and within budget.
- Fosters a culture of learning and improvement within the team.
- Master's degree in a related STEM subject or equivalent level of experience of working at a similar level in a similar setting for at least 3 years.
Person Specification
Experience
Essential
- Considerable experience of extracting data, manipulating, understanding, transforming, wrangling, cleaning, and storing health data.
- Ability to write well-designed code (e.g. SQL or Python) which follows good coding standards.
Skills
Essential
- Demonstrates leadership skills in approach to data engineering tasks and delivery of projects on time and within budget.
- Fosters a culture of learning and improvement within the team.
- Master's degree in a related STEM subject or equivalent level of experience of working at a similar level in a similar setting for at least 3 years.
Additional information
Applications from job seekers who require current Skilled worker sponsorship to work in the UK are welcome and will be considered alongside all other applications. For further information visit the UK Visas and Immigration website (Opens in a new tab).
From 6 April 2017, skilled worker applicants, applying for entry clearance into the UK, have had to present a criminal record certificate from each country they have resided continuously or cumulatively for 12 months or more in the past 10 years. Adult dependants (over 18 years old) are also subject to this requirement. Guidance can be found here Criminal records checks for overseas applicants (Opens in a new tab).
