Data Engineer in Charlotte, NC for 6 Months (Remote till Office Opens)
Posted 11 months ago
Role: Data Engineer
Location: Charlotte, NC (Remote till Office Opens)
Duration: 6+ Months
Skill set –
- Primary – Hive, Spark (preferably with Python), shell script, any scheduling tool, Insurance domain knowledge.
- Secondary – Impala, Experience in any cloud platform (preferably Azure), Oozie workflow
Job title: DATA ENGINEER
- 2.Job summary Creation of data products for all consumers – business users, analysts, and modelers. Explore and understand data sets. Visualizing the data set determines whether the data set has enough information to answer the question that the business is asking.
- Experience 8 to 10yrs
- Required Skills Technical Skills:Big Data Management Domain Skills-
- Nice to have skills Technical Skills: Hive, Spark Domain Skills
- Technology Data Management
Roles & Responsibilities:
- At least 8 years of relevant work experience in implementing data and analytics projects. Must have domain technical experience in delivering data engineering solutions using data lake technology Experience with the following: Hadoop (CDH), relational databases and SQL, ETL development, spark, data validation and testing (Data Warehousing, ETL/ELT to the Data Lake, Using the Data Lake for data analysis (Hadoop tools – Hive, Impala, Pig, Sqoop, Hue, Kafka, etc., Spark, Python, R, java, Docker, Dakota).
- Working experience in Shell Scripts, Oozie workflows, scheduling tools (Stone branch or CA7). Knowledge of Cloud platform implementation (Azure or Amazon).
- Knowledge of data visualization tools is a plus (Tableau on multiple platforms along with Python visualization in the Data Lake using Pandas and bokeh packages) Excellent written, verbal, and interpersonal skills, a must as there will be significant collaboration with the business and IT Experience with collaborative development workflows (e.g., Microsoft DevOps Tools).
- Creation of data products for all consumers – business users, analysts, and modelers. Explore and understand data sets.
- Visualizing the data set determines whether the data set has enough information to answer the question that the business is asking.
- Work with IT support to create ETL / ELT interfaces to the data lake and create and visualize the data and data products on the data lake.
- Building data pipelines, integrating, and scheduling ETL jobs using CI/CD framework. Implement required data transformation in the data lake.
- Configure required security and data masking to a data set. Support testing of data acquisition, data set correlation, and / or model development.
- Investigate and resolve interface issues Work with IT to harden and productionize the model, model interfaces, and business procedures.
Please share the details below ASAP.
Full Name (as per SSN and DL):
US work authorization:
Year of Graduation & Degree & university Name:
Date of Birth: (Date and Month):
Last 4 Digits of SSN:
Currently on Project:
Direct Number: 585 504 1542
Desk Number: 585 – 532 – 7200 Extension 9002
687 Lee Road, Suite 250, Rochester, NY 14606