ETL Pyspark Consultant
Job Title: ETL Pyspark Consultant
Location: Houston, TX
Duration: 6+ Months - Contract
Experience: Minimum Years’ experience: 7 – 10 Years – is a MUST
Location: Houston, TX
Duration: 6+ Months - Contract
Experience: Minimum Years’ experience: 7 – 10 Years – is a MUST
Work Authorization's (VISA TYPE): Open to Welcome Visa (H1B | Green Card | US Citizen).
An Ideal Candidate must have: ETL Pyspark Consultant
Responsibilities:
• Possess development experience in building frameworks to automate high-volume and real-time data processing by using big data technologies such as Python, Pyspark, Hadoop, Spark, Hive, Pig, Kafka, Nifi, etc
• Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.
• Working knowledge/exposure to cloud capabilities preferably AWS services
• Working Knowledge in AWS tools for Data Ingestion , Storage, Cataloging , AWS Lambda
• Possess development experience in building frameworks to automate high-volume and real-time data processing by using big data technologies such as Python, Pyspark, Hadoop, Spark, Hive, Pig, Kafka, Nifi, etc
• Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.
• Working knowledge/exposure to cloud capabilities preferably AWS services
• Working Knowledge in AWS tools for Data Ingestion , Storage, Cataloging , AWS Lambda
Must Have Skills:
• At least 2 years of development experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python or Pyspark
• At least 2 years of development experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS
• At least 2 years of development experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python or Pyspark
• At least 2 years of development experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS
Nice to Have Skills:
• Nifi, Glue, AWS Lambda & Oil & Gas Domain Knowledge
• Nifi, Glue, AWS Lambda & Oil & Gas Domain Knowledge
Experience:
• Minimum 2-3 Years experience working Pyspark
• Minimum 2-3 Years experience working Pyspark
5 Sep 2020 -save job - original job
» Apply Now
Please review all application instructions before applying to Workhcm.
Get job updates from Workhcm
Workhcm
288 reviews
Workhcm is the #1 job site worldwide*, with over 200 million unique visitors per month from more than 60 countries in 28 languages. Since...
Workhcm is the #1 job site worldwide*, with over 200 million unique visitors per month from more than 60 countries in 28 languages. Since...
Let employers find you
Thousands of employers search for candidates on WorkHcm