- Develop and maintain data pipelines and ETL processes using Databricks, Python, SQL, and Delta Lake.
- Assist in implementing scalable solutions for structured and unstructured data, leveraging Lakehouse architecture.
- Collaborate with data engineers, data scientists, and business stakeholders to understand requirements and deliver solutions.
- Support integration of Databricks with other platforms and tools, ensuring data quality and governance.
- Participate in performance tuning and troubleshooting of Databricks clusters and Spark jobs.
- Contribute to documentation of processes, workflows, and technical decisions.
- Stay updated on best practices in data management, security, and compliance.
- Engage in knowledge sharing and continuous improvement within the team