Data Engineer
Job Type : Permanent, Full Time
Salary : £60000 – £70000/annum
Data Development
Manage deployment of changes, upgrades and bug fixes to databases, data lake and data warehouse
Development of new AWS Data Lake
Migrate customer orchestration solution to AWS Airflow
Refine and extend system and ETL designs to improve accuracy, timeliness, completeness, and consistency
Manage integration and maintenance of new data sources. Design, build and maintain data pipelines
Code management, review and reversion using GitHub
Data Administration
Monitoring, diagnosis and troubleshooting orchestration including SSIS, Airflow, Spark, Scala, Kubernetes, AWS Orchestration, Python and Glue
Security, key management, DR, performance, and cost monitoring of AWS solution including S3 and Athena
Disaster Recovery planning, maintenance, and execution
Design, develop and maintain non-production environments
Design, develop and adhere to change control processes
Administration of MSSQL Server RDS environment
Upgrade of existing MSSQL Server environment including SSIS, SSAS, MDS
Establish the needs of users, monitor user access and security, control access permissions and privileges
Ensure that storage and archiving procedures are functioning correctly
Develop, manage, and test back-up and recovery plans
Manage the security and disaster recovery aspects of Databases and Data Lake
Data Governance
Align existing and new processes to defined design patterns and coding standards
Maintain data standards, including adherence to the Data Protection Act
Investigation, design and implementation of QA processes and reporting
Protection of Personally Identifiable Information
Design and develop GDPR processes
Collation and maintenance of meta data about the Hearst Data Estate.
Create and maintain documentation, including data standards, procedures, and definitions for the data dictionary (metadata)
Maximise data value to the business
Application Support
Commission and install new applications and customise existing applications to make them fit for purpose
SSRS, PowerBI, Tableau and Looker report editing and origination
Data Warehouse (DW) maintenance and support. SSIS, SSAS, SQL Server for analysis on existing DW
Design and development of replacement for existing MSSQL Data Warehouse and Cube
Advice on back-end standards for new and existing applications and ETL
Creation and maintenance of GDPR Compliant Development and Testing environments
Design, implementation and monitoring of deployments along with co ordinating code, data and schema reversions when required.
Stakeholder Management
Liaise with developers, application owners and SMEs around DR, Security and Operational Requirements (including ETL timings)
Liaise with data architect to move data estate to consolidate, simplify and document operational landscape
Define and enforce standards for data quality
Define and enforce code standards
Liaise with 3rd party data suppliers to understand data structures / formats etc.
Monitor performance and manage parameters to provide fast responses to front-end users
Manage and respond to alerts and notifications around feeds to our Databases and Data Lake
Communicate regularly with technical, applications and operational staff to ensure database integrity and security
Team
Positive contributor to the wider Data & Technology team
Training and mentoring of junior team members
Provide wider business user support by responding to system issues within agreed SLAs
Leading from the front
To be an ambassador of Hearst and its values.
You may be required to undertake such further duties as your skills, qualifications and experience allow and/or as may be assigned to you from time to time
More Information
- Address City of London, London
