Data Engineer/ETL Developer

Randstad Canada37 days ago
Markham, CA
Randstad Canada
Our client, a leader in the insurance industry, is lookingfor a Data Engineer/ETL Developer to join their growing team ofData and BI experts.
They have embarked on a journey tobuild next generation data platform to support the growing need ofdata from business intelligence and analytics.

The candidate will be responsible designing,developing and productionizing ETL jobs to ingest data into DataLake, load data to data marts; and extract data to integrate withvarious business applications.

Roles andResponsibilities:

Design and Develop ETLPipeline to ingest data into Hadoop from different data sources(Files, Mainframe, Relational Sources, NoSQL Etc.) usingInformatica BDM

Parse unstructured data, semistructured data such as JSON, XML etc. using Informatica DataProcessor.

Analyze the Informatica PowerCenterJobs and redesign and develop them inBDM.

Design and develop efficient Mapping andworkflows to load data to Data Marts.

Performthe GAP analysis between various legacy applications to migratethem to newer platforms/data marts.

Writeefficient queries in Hive or Impala and PostgreSQL to extract dataon Adhoc basis to do the data analysis.

Identifythe performance bottlenecks in ETL Jobs and tune their performanceby enhancing or redesigning them.

Work withHadoop administrators, PostgreS DBAs to partition the hive tables,refresh metadata and various other activities, to enhance theperformance of data loading andextraction.

Performance tuning of ETL mappingsand queries.

Write simple or medium complexshell scripts to preprocess the files, schedule ETL jobsetc.

Identify various manual processes, queriesetc. in the Data and BI areas, design and develop ETL Jobs toautomate them.

Participate in daily scrums; workwith vendor partners, QA team and business users in various stagesof development cycle.
 
SkillRequired:

7+ years of experience in designingand developing ETL Jobs (Informatica or any other ETLtool)

3+ years of experience working onInformatica BDM platform

Experience on variousexecution modes in BDM such Blaze, Spark, Hive,Native.

3+ years of experience working on HadoopPlatform, writing hive or impala queries.

5+years of experience working on relational databases (Oracle,Teradata, PostgreSQL etc.) and writing SQLqueries.

Should have deep knowledge onperformance tuning of ETL Jobs, Hadoop Jobs, SQL’s, Partitioning,Indexing and various othertechniques.

Experience in writing Shellscripts.

Experience in Spark Jobs (Python orScala) is an asset.

1+ years of experience withworking on AWS technologies for data pipelines, datawarehouses

Minimum 5+ years of experience withbuilding ETLs to load data warehouse, datamarts

Awareness of Kimball and Inmon datawarehouse methodologies

Nice to have knowledgeon all the products of Informatica such as IDQ, MDM, IDD, BDM, DataCatalogue, PowerCenter etc.

Must have experienceworking in Agile SCRUM methodology, should have used Jira, Bitbucket, GIT, Jenkins to deploy the codes from one environment toother.

Experience working in diversemulticultural environment with different vendors, onsite/offshorevendor teams etc.

P&C Insurance industryknowledge will be an added asset

Certificationsin Informatica product suite as adeveloper

Randstad ProfessionalsCanada
Randstad Canada is committed tobuilding a diverse workforce reflective of the diversity of Canada.As a result, we promote employment equity and encourage candidates,especially those who identify as a woman, an Aboriginal person, aperson with a disability or a member of a visible minority group,and any others who may contribute to the diversification of ourworkforce, to apply.

Randstad Canada is alsocommitted to developing an inclusive, barrier-free selectionprocesses and work environments.If contacted in relation to a jobopportunity, you should advise your Randstad Representative or yourlocal Randstad branch in a timely fashion of the accommodationmeasures which must be taken to enable you to be assessed in a fairand equitable manner.Information received relating to accommodationmeasures will be addressed confidentially.

Forall feedback on equity and accommodation needs, please contact yourlocal Randstad Canada Branch.

Location

Markham, CA

By clicking apply you will leave devsnap. Please be careful. You should never have to pay to apply.
A new version is available REFRESH