Senior Big Data Engineer

InnoIT Consulting

 Are you passionate about data?

Would you like use your passion for big data and analytics to provide insights to the business covering a range of topics?

👀 So.. keep reading it can be just what you’re looking for!

✍️Responsibilities:

  • Develop real-time and batch Big Data processing pipelines with Spark.
  • Develop and operate the data lake, data processing and data science platforms end-to-end.
  • Develop scalable and reliable data integrations to feed the data science models.
  • Develop integrations with other key components of the architecture (visualization tools, Jupyter notebooks…).
  • Manage and orchestrate the cluster with proper job monitoring and recovery mechanisms.
  • Design CI/CD pipelines for elements in the cluster like a DataOps.

👍Your profile:

  • At least 3 years of development experience in Python.
  • Experience in developing scalable ETL, data integration processes with Spark.
  • Understanding of engineering best practices: write tests, use automation, build continuous integration pipelines, etc.
  • Experience in constructing and maintaining complex high-volume data workloads and orchestrating dependencies.
  • Experience working with AWS services (e.g., S3, Lambda, DynamoDB, EC2, API Gateway) and Databricks is highly desirable.
  • Experience working with continuous integration scenario (Git / Bitbucket / Jenkins / Sonar stack).

🚀What we can offer you:

  • Competitive salary.
  • You will join one of the most exciting environments in a highly motivated team, where everyone is developing themselves to higher levels of professionality.
  • Flexible working hours.
  • Career path with salary revision.
  • Speak up! We’re very open here, and we want to hear your opinions and ideas about how we can do things better!
  • And much much more, if you want to hear it, just apply 🙂

Por favor, para apuntarte a este trabajo visita www.linkedin.com.