NPIC Data Engineer (XN07)

Job description

Job responsibilities

JOB DETAILS

Job Title: NPIC Data Engineer

Reports to: Digital Pathology Systems Lead

Band: 7

Department: Research and Innovation

Location : SJH

AfC Job No: 4059

JOB PURPOSE/SUMMARY

The National Pathology Imaging Co-operative is a £37m programme funded by the UK Industry Strategy Challenge Fund to accelerate the development of opportunities in Digital Pathology and Artificial Intelligence. We provide a welcoming, professional approach to the facility, a world class centre for training and knowledge exchange in digital pathology and artificial intelligence.

We are seeking to appoint an experienced individual who is driven, ambitious and passionate about improving patient care through the use of cutting-edge technology. The post-holder will be responsible for building, testing, maintaining data architecture and undertaking new developments while maintaining existing data models and reporting relating to the Deployment of the Digital Pathology national footprint.

The role is crucial in laying the foundation, enabling data scientists and data analysts to create new insights from data. The Data Engineer will be responsible for design, implementation and delivery of platform enhancements, upstream data lineage, and data acquisition and wrangling. The role will work in a data rich environment together with data analysts and data scientists. Data engineers will configure platform, performing initial integrations and implement workflows and models for key use cases.

The role will be responsible to ensure compounding value is created through shared data, common ontologies, and cross-organisation projects.

Previous experience in SQL data warehouse and ETL is essential for this role and experience in Power BI Data Models including use of DAX, Python and R will be seen as advantageous. You will be given ample growth opportunity to learn and gain experience in Power BI and data modelling.

The post holder will liaise with senior managers, clinicians and system users across a national footprint with, directorates and partner organisations. The post holder will work with partners and the local Informatics Departments to assist in the development of any necessary interfaces. In addition, the post holder will be responsible for the provision of an effective, efficient and robust information capture and analysis service to meet national, regional and local business requirements and essentially improve recording and reporting for the patients and services in the Trust.

The post holder will work closely with the Digital Pathology Systems Lead to ensure that the architecture is effectively implemented, maintained and developed liaising closely with clinical, technical and system development staff and system suppliers.

The post holder will manage a set of projects to professional standards.

The post holder will line manage a small team of specialists as well providing matrix-management of project staff with management of project budgets, this will vary by project.

PRINCIPAL DUTIES & AREAS OF RESPONSIBILITY

As a Data Engineer you will:

inspire best practice for data products and services within the team

build data engineering capability by providing technical leadership

work with other senior team members to identify, plan, develop and deliver data services

The day to day role involves working with NPIC Teams and the corporate data assets. The role will involve driving forward improvements in data services and supporting the organisation with on-going projects from a data perspective.

The responsibilities of the role will include:

Design, develop, construct, test and maintain architectures and processing workflows

Build robust, efficient and reliable data pipelines

Optimisation of information management approaches through an understanding of evolving business needs and technology capabilities.

Provide technical support and expert advice on best practices in Power BI, performance analysis of Power BI reports with recommendations for improvement to the other teams in healthcare analytics

Integrating external or new datasets into existing data pipelines

Develop solutions for data acquisition

Design, develop and maintain data acquisition API for central data Lake

Ensure architecture supports business requirements

Develop dataset processes for data modelling, mining, and production

Drive the collection of new data and refinement of existing data sources

Recommend ways to improve data reliability, efficiency, and quality

Applying feature transformations for machine learning models on new data.

Continuously monitoring and testing the system to ensure optimized performance

Use SQL queries, programming languages like DAX, M, Python & R and tools like DAX studio and tabular model in building data models and other data frames.

Identify ways to improve data reliability, efficiency and quality

Conduct research for advancements and incorporation of new functionalities to meet the reporting requirements of the organisation

Develop tools and techniques to manage large datasets with disparate sources for consolidated reporting.

Build and deploy sophisticated analytics programs, machine learning and statistical methods in data models and data frames for predictive and prescriptive modelling including finding hidden patterns using data and undertake tasks to automate data processing and data management

Ensure new solutions are produced in accordance with local policies and guidelines for all stages of development including design, build, testing and deployment

Develop and maintain a detailed working knowledge of the technical content of source systems to inform the design and on-going development of reporting

Undertake a rolling maintenance programme for published data models and reports built by the NPIC Team to provide quality assurance and ensure products are fit for purpose and meet the needs of users

Develop and maintain an in-depth knowledge of NHS data flows, datasets, information availability, scope, reliability and data quality issues. This includes a thorough understanding of the NHS data dictionary and knowledge of guidance for specific returns.

Develop and maintain effective working relationships with colleagues across the Trust to enable the continuous monitoring, development and improvement of reporting solutions

Represent the NPIC Deployment Team at internal and external meetings

As a subject matter expert for technical solutions and data management; act as a reference point and provide technical guidance and advice as required

Build validation checks and processes into the creation of data models, data frames, reports and dashboards to ensure data integrity at all stages of the development and implementation of reporting

Contribute to the development of the NPIC Power BI reporting, NPIC data warehouse

Make decisions where appropriate to escalate risks and issues that may impact on the completeness and accuracy of reporting

Ensure all handling of patient related data is in accordance with information governance, Caldecott and data protection policies.

Ensure all handling of patient related data is in accordance with Trust policies and procedures.

Deal with highly complex data management issues and identify and use knowledge of systems and data flows to resolve areas of dispute.

Interpret local and national guidance and develop, produce and recommend changes to policies and procedures as required to ensure compliance

Continually review the work undertaken previously and implement changes and improvements as dictated by customer feedback, changes to guidance and local changes such as ward reconfigurations

Other duties as commensurate with the grade and role as agreed with line manager.

Person Specification

Experience

Essential

• Experience of working with remote and multi-disciplinary teams and driving forward complex work programmes.
• Expert user of Microsoft office products including advanced functions of MS Excel
• Experience of working in an NHS information environment in an information management role
• Significant experience of designing, developing and publishing reports and dashboards across a broad range of areas using different reporting tools such as SSRS.
• Have good project management skills, including the ability to juggle competing priorities and tight deadlines for self and team, and work autonomously to drive own work programme and that of team members.

Qualifications

Essential

• Educated to degree level or equivalent level of knowledge gained through relevant work experience

Desirable

• Educated to masters level.
• A relevant qualification in information management

Other Criteria

Essential

• SQL Fluent in SQL-based systems like MySQL, PostgreSQL Microsoft SQL Server
• Significant experience of extracting and processing data from multiple source system using SQL.
• ETL solutions Must have ETL skills to build processes to move data between systems eg Microsoft ADF
• API design and implementation Understanding of best practice when setting up APIs and the coding ability to develop and deliver the design.
• Big Data Tools Strong understanding and Knowledge of big data tools
• Coding ability Proficiency in Python

Skills & behaviours

Essential

• Can work autonomously to achieve agreed objectives without reference to a manager.
• Excellent problem-solving skills.
• Capable of working with clinical & non-clinical staff at all levels.
• Strong written & verbal communication skills.

Disclosure and Barring Service Check

This post is subject to the Rehabilitation of Offenders Act (Exceptions Order) 1975 and as such it will be necessary for a submission for Disclosure to be made to the Disclosure and Barring Service (formerly known as CRB) to check for any previous criminal convictions.

Employer details

Employer name

Leeds Teaching Hospitals

Address

St. James's University Hospital

Beckett Street

Leeds

LS9 7TF

Employer's website

https://www.leedsth.nhs.uk/ (Opens in a new tab)