Data Engineer with Python, Kube, Big Data, Algo CD and Air Flow | Sunnyvale, CA

Detailed Information

  • Location: Sunnyvale, CA

  • Company: Diverse Lynx

engineering teams and data analysts Experience in managing large datasets Experience in creating Kubernetes configurations, managing services in Kubernetes, building docker images Demonstrate good understanding & troubleshooting Spark jobs Experience in developing/maintaining automation tools in programming/scripting languages like Python Demonstrate good understanding in Big data eco-systems such as HDFS, Kafka, SQL etc.

Experience in using IT automation tools such as Algo CD, and job orchestrator systems such as Air Flow, Jenkins Experience in cloud based environment (AWS/GCP etc. ) is a big plus. Experience in monitoring tools such as Splunk, Prometheus/Grafana is a big plus.

Experience with Full-Stack web development (React, Django etc. ) is a big plus. Responsibilities Support large scale data pipelines and backend services by monitoring and troubleshooting/recovering incidents Manage large volume datasets in Spark and Hadoop environments using scripts and automation Build and operate data management infra services Automate build, deployment, and monitoring Participate at an on Call & release turn Taking an oncall turn including weekend coverage Diverse Lynx LLC is an Equal Employment Opportunity employer.

All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the

basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role.

We promote and support a diverse workforce across all levels in the company.

View Jobs by Category >>

Related Jobs