Brazil
SONDA
GFT Technologies SE
Senior Data Engineer (Azure) | Hybrid/SP - 136209
Brazil
Barueri, São Paulo, Brazil
PythonSQL
Automatic translation. Check the original.
What we are looking for
We are looking for a Data Engineer with solid experience in data architecture and engineering in cloud environments, capable of developing and sustaining scalable, reliable, and observable pipelines. The professional will work on building and evolving data platforms using Lakehouse architecture, as well as contributing to data governance, quality, integration, and availability for analytical and business initiatives.
Big enough to deliver – small enough to care.
Responsibilities
- Design, develop, and maintain batch and streaming data pipelines with a focus on scalability, performance, and observability;
- Implement data ingestion, transformation, and availability processes using Python, PySpark, Spark, and SQL;
- Build and evolve data layers in Lakehouse architecture (Bronze, Silver, and Gold);
- Develop and sustain ETL/ELT flows in cloud environments;
- Implement and maintain data pipeline orchestration processes;
- Analyze, diagnose, and resolve issues in data pipelines and applications;
- Perform performance optimization and incident resolution in data solutions;
- Implement data governance policies, processes, and tools, promoting standardization and accountability for data;
- Utilize and evolve solutions based on Azure Data Factory, Airflow, Databricks, and Data Lake;
- Develop integrations between systems through REST APIs and other data sources;
- Apply good data modeling practices for Data Lakes and analytical environments;
- Collaborate with product, engineering, data, and business areas in defining and implementing technical solutions;
- Ensure quality, consistency, traceability, and reusability of data assets;
- Act proactively in the maintenance and evolution of the data platform.
Mandatory Requirements
- Previous experience as a Data Engineer;
- Solid experience with Python, PySpark, and SQL;
- Advanced knowledge of Azure Data Factory, Airflow, Databricks, and Data Lake;
- Experience in data architecture and building analytical solutions;
- Experience with Git Flow and version control;
- Knowledge of unit and functional testing (Unittest, Pytest, or similar);
- Experience with relational databases such as SQL Server, Oracle, or equivalents;
- Experience in implementing and sustaining ETL/ELT processes;
- Knowledge in data modeling for Data Lakes and Lakehouse architecture;
- Experience with Bronze, Silver, and Gold layers or equivalent structures;
- Experience in integrating and consuming REST APIs;
- Practical experience in Cloud environments, with a focus on Azure;
- Experience with batch and streaming data processing;
- Knowledge of data streaming tools, such as Kafka;
- Ability to work in dynamic environments with multiple demands and simultaneous priority management.
Differentiating Requirements
- Databricks and/or Microsoft Azure certifications;
- Experience with event-driven architecture and streaming-based solutions;
- Knowledge of data security and privacy;
- Experience in high-criticality environments and large processing volumes;
- Knowledge of DevOps practices and CI/CD pipeline implementation for data projects;
- Experience with Observability, monitoring, quality, and data lineage tools.
Behavioral Description
We are looking for someone who:
- Enjoys teamwork and is collaborative in their duties;
- Has the courage to challenge themselves and go beyond, embracing new growth opportunities;
- Transforms ideas into creative solutions and seeks quality in their daily routine;
- Has problem-solving skills;
- Has the ability and feels comfortable working independently and managing their own time;
- Is interested in dealing with adverse and innovative situations in the technological field.
#VempraGFT #VamosVoarJuntos #ProudToBeGFT
More opportunities with one profile
Want to be considered for more roles without repeating the whole process? Create your profile and be found by companies using Nort.
Create my profile