Data Engineer
| Dyddiad hysbysebu: | 22 Mai 2025 |
|---|---|
| Cyflog: | £45,000 i £50,000 bob blwyddyn |
| Oriau: | Llawn Amser |
| Dyddiad cau: | 21 Mehefin 2025 |
| Lleoliad: | Office 1, Izabella House, 24-26 Regent Place, City Centre, Birmingham, England, B1 3NJ |
| Gweithio o bell: | Hybrid - gweithio o bell hyd at 3 ddiwrnod yr wythnos |
| Cwmni: | MAAS TECHNOLOGIES LTD |
| Math o swydd: | Parhaol |
| Cyfeirnod swydd: | 020481 |
Crynodeb
Design, build, and operate simple, repeatable ETL data pipelines within distributed processing environments and cloud platforms, as well as localized single-node processing environments.
Develop and produce prototyped then productionized code that can be deployed across a range of ETL, data validation, and other data production processes.
Develop understanding of the native tooling of one or more of: GCP, Azure, AWS.
An intermediate or better level coding in one or more mainstream coding languages (ie, Python, SQL, Java, Scala, and R), and critically review the code of other data engineers.
Develop code for a range of data products including data matching, rule development, scans, operational outputs.
Participate in development and maintenance of in-house code libraries.
Undertake unit testing to support common code development.
Review business requirements to ensure they are clear and robust, and transform requirements into reusable production-ready code and/or effective data models.
Understand the key principles of database design and be able to resolve technical problems in databases, data processes, data products, and services as they occur. Initiate actions, monitor services, and identify trends to resolve problems.
Apply correct techniques in normalizing data and building robust relational structures in database products.
Undertake source system analysis and data profiling to confirm data quality and ensure accurate metadata.
Understand relevant data sources, tools, and systems. Work with experts to develop validation frameworks for both simple and complex data sources.
Experience predicting and advising on technology changes in the engineering toolset(s) and platform(s) you work on.
Work with the team in implementing the transition to modern data platforms including Data Warehouse, Lakehouse, across the data products, pipelines, and processes you are responsible for.
Describe technical, data, pipeline, and production issues to colleagues of different specialisms.
Communicate within the team and across teams to monitor expectations around delivery of data engineering, products, and pipelines, blockers, priorities, and issues. Escalate issues and blockers in delivery proactively.
A working knowledge of engineering standards across a platform and native toolset. Experience implementing these standards in the day-to-day role and keeping outputs up-to-date with these.
Typical Data Engineering Experience required (5+ years):
Knowledge and experience of Azure/AWS Cloud data solution provision.
Proficient in SQL
Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI, Informatica
Enterprise-scale experience with ETL tools (Informatica and or similar).
Experience of data modelling and transforming raw data into datasets and extracts.
Experience of working in a large project/scale complex organisation and knowledge of migrating Legacy capabilities.
Experience in Agile.
Ability to analyse and collect information and evidence, identify problems and opportunities, and ensure recommendations fit with strategic business objectives.
Experience of building team capability through role modelling, mentoring, and coaching.
Ability to design, write, and operate ETL pipelines, in the context of distributed processing, applying coding, data, and documentation standards, in the language required by the business area.
Understanding of the principles of data processing in a distributed and or cloud platform, and ability to use this understanding to ensure robust coding in a distributed or cloud environment.
Able to use Git for code version control to pull and push and review merge requests for team and own code.
Experience of one or more programming/coding languages listed: Python/PySpark, SQL, Proc SQL, NoSQL, MySQL, SQLite, Spark SQL, Hive SQL, PostgreSQL, SAS, SAS E-guide, Scala, RegEx, Java, R
Investigate problems in systems, processes and services, with an understanding of the level of a problem, for example, strategic, tactical or operational.
Use metadata repositories to complete complex tasks such as data and systems integration impact analysis.
Good knowledge of database structures, practices, principles of database integrity etc.
Basic knowledge of applying database principles and SQL coding across a range of platform database and data querying tools (ie SQL Server, Cloud SQL, Big Query, Hive, Athena etc.)
Show an awareness of opportunities for innovation with new tools and uses of data.
Experience in more than one of the following tools is required for Engineers engaging in BI development: Plotly, R Shiny, Tableau, QlikView/Qlik sense, Power BI, SAP, Business Objects, MicroStrategy, Snowflake
Experience of several of the following tools: NiFi, Hbase, Bash, Assist, Putty, Neo4J, Spark, Kafka, HDFS, Oozie, Git Hub, Unix, Hadoop, Impala, DoJo, Flume, Elastic, Logstash, Kibana, Airflow, Glue, Big Query, Athena, CML, Hive, Informatica, CuteFTP
Ability to explain and communicate technical concepts in non-technical language.
Explain the types of communication that can be used with internal and external stakeholders, and their impact.
Design, build and test data products based on feeds from multiple systems, using a range of different storage technologies, access methods or both.
Certifications in AWS, Azure, Databricks, or related technologies.
Knowledge of machine learning and artificial intelligence concepts
Develop and produce prototyped then productionized code that can be deployed across a range of ETL, data validation, and other data production processes.
Develop understanding of the native tooling of one or more of: GCP, Azure, AWS.
An intermediate or better level coding in one or more mainstream coding languages (ie, Python, SQL, Java, Scala, and R), and critically review the code of other data engineers.
Develop code for a range of data products including data matching, rule development, scans, operational outputs.
Participate in development and maintenance of in-house code libraries.
Undertake unit testing to support common code development.
Review business requirements to ensure they are clear and robust, and transform requirements into reusable production-ready code and/or effective data models.
Understand the key principles of database design and be able to resolve technical problems in databases, data processes, data products, and services as they occur. Initiate actions, monitor services, and identify trends to resolve problems.
Apply correct techniques in normalizing data and building robust relational structures in database products.
Undertake source system analysis and data profiling to confirm data quality and ensure accurate metadata.
Understand relevant data sources, tools, and systems. Work with experts to develop validation frameworks for both simple and complex data sources.
Experience predicting and advising on technology changes in the engineering toolset(s) and platform(s) you work on.
Work with the team in implementing the transition to modern data platforms including Data Warehouse, Lakehouse, across the data products, pipelines, and processes you are responsible for.
Describe technical, data, pipeline, and production issues to colleagues of different specialisms.
Communicate within the team and across teams to monitor expectations around delivery of data engineering, products, and pipelines, blockers, priorities, and issues. Escalate issues and blockers in delivery proactively.
A working knowledge of engineering standards across a platform and native toolset. Experience implementing these standards in the day-to-day role and keeping outputs up-to-date with these.
Typical Data Engineering Experience required (5+ years):
Knowledge and experience of Azure/AWS Cloud data solution provision.
Proficient in SQL
Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI, Informatica
Enterprise-scale experience with ETL tools (Informatica and or similar).
Experience of data modelling and transforming raw data into datasets and extracts.
Experience of working in a large project/scale complex organisation and knowledge of migrating Legacy capabilities.
Experience in Agile.
Ability to analyse and collect information and evidence, identify problems and opportunities, and ensure recommendations fit with strategic business objectives.
Experience of building team capability through role modelling, mentoring, and coaching.
Ability to design, write, and operate ETL pipelines, in the context of distributed processing, applying coding, data, and documentation standards, in the language required by the business area.
Understanding of the principles of data processing in a distributed and or cloud platform, and ability to use this understanding to ensure robust coding in a distributed or cloud environment.
Able to use Git for code version control to pull and push and review merge requests for team and own code.
Experience of one or more programming/coding languages listed: Python/PySpark, SQL, Proc SQL, NoSQL, MySQL, SQLite, Spark SQL, Hive SQL, PostgreSQL, SAS, SAS E-guide, Scala, RegEx, Java, R
Investigate problems in systems, processes and services, with an understanding of the level of a problem, for example, strategic, tactical or operational.
Use metadata repositories to complete complex tasks such as data and systems integration impact analysis.
Good knowledge of database structures, practices, principles of database integrity etc.
Basic knowledge of applying database principles and SQL coding across a range of platform database and data querying tools (ie SQL Server, Cloud SQL, Big Query, Hive, Athena etc.)
Show an awareness of opportunities for innovation with new tools and uses of data.
Experience in more than one of the following tools is required for Engineers engaging in BI development: Plotly, R Shiny, Tableau, QlikView/Qlik sense, Power BI, SAP, Business Objects, MicroStrategy, Snowflake
Experience of several of the following tools: NiFi, Hbase, Bash, Assist, Putty, Neo4J, Spark, Kafka, HDFS, Oozie, Git Hub, Unix, Hadoop, Impala, DoJo, Flume, Elastic, Logstash, Kibana, Airflow, Glue, Big Query, Athena, CML, Hive, Informatica, CuteFTP
Ability to explain and communicate technical concepts in non-technical language.
Explain the types of communication that can be used with internal and external stakeholders, and their impact.
Design, build and test data products based on feeds from multiple systems, using a range of different storage technologies, access methods or both.
Certifications in AWS, Azure, Databricks, or related technologies.
Knowledge of machine learning and artificial intelligence concepts