Senior Data Engineer
Location: Spain, Poland, Bulgaria, Romania
Contract type: 12 Month Contract
Time Frame: July 2022 – July 2023
Work Rhythm: 40 hours / week (Monday – Friday)
Contract Language: English
Recruitment Partner: Airswift
Account Manager: Ming Yue
Key Words: Data Engineer | Data | ETL | Python | Java | SQL | AWS | Database | ElasticSearch | OpenSearch | Talend | Hadoop | Kafka | Spark | Cloud | Docker | Kubernetes | Scrum | Agile | Apache Hudi | Immediately |
We are a global consultancy, known through Europe and USA for our innovative projects. We are expanding our network and have partnered up with Airswift to support us in hiring and are proud to expand our team to include the appointment of a Senior Data Engineer.
Requirements (Hard skills requirements):
- 5+ years of experience coding in Python, SQL, with solid CS fundamentals including data structure and algorithm design.
- Experience on large-scale production databases and SQL
- Time-series/analytics databases, preferably Elasticsearch or OpenSearch
- Experience either with
- ETL: Experience with ETL development (extractions, data load, aggregation, ideally experience with Talend).
- Real time engineering: Hands-on implementation experience/ familiarity working with a combination of the following technologies: Hadoop, NIFI, Kafka, Spark, and Hudi.
- 2+ years of experience in cloud data platforms - AWS: S3, Athena, EC2, RedShift, EMR, and Lambda
- Experience with Docker/Kubernetes
- Knowledge of SQL and MPP databases (e.g. Vertica, Netezza, Greenplum, Aster Data)
- Knowledge of professional software engineering best practices for the full software
- Knowledge of Data Warehousing, design, implementation and optimization
- Knowledge of Data Quality testing, automation and results visualization
- Knowledge of BI reports and dashboards design and implementation
- Knowledge of development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations.
- Experience participating in an Agile software development team, e.g. SCRUM
- Experience designing, documenting, and defending designs for key components in large distributed computing systems
- A consistent track record of delivering exceptionally high quality software on large, complex, cross-functional projects
- Demonstrated ability to learn new technologies quickly and independently
- Ability to handle multiple competing priorities in a fast-paced environment
- Undergraduate degree in Computer Science or Engineering from a top CS program required. Masters preferred.
- Experience with supporting data scientists and complex statistical use cases highly desirable
Nice to have:
- Experience with Airflow
- Coding experience with Java/Scala
- Familiarity with Apache Hudi
- English (Upper Intermediate - Fluent)
The next step
We have an exceptional team in place, and we are pleased to be able to appoint a further person to our growing business. We are aware that you may not ‘tick all the boxes’, but if you believe you can genuinely offer some valuable skills and experience to our business, please in the first instance contact our recruitment partner Airswift.