Job type: Full-time
Experience level: Mid-Level
Industry: Automotive, Financial Services, Financial Technology
Company size: 201–500 people
Company type: Private
As a Cloud Data Engineer at CarFinance 247 you will be taking an active role in the development of a greenfield Data Lake in order to to support our migration to an event driven architecture in the cloud.
Using technologies such as Google Pub/Sub, Dataflow, Cloud Storage and Big Query, you will work closely with the Data Team, Data Science Team and BI Developers in order to understand the existing reporting and analytics requirements to be migrated, along with ensuring that future data requests can be fulfilled.
You will be required to produce data flow diagrams in order to ensure that data flows are in line with the business' data dictionary. Lastly, you will liaise with Third Parties in order to provide data extracts as and when required, on either an ad-hoc or scheduled basis.
Key Skills required:
Big Data and DWH technologies such as Beam, Cloud Storage, Hadoop, Spark, Big Query, Redshift
Experience with pub/sub systems such as Kafka, Google Pub/Sub, AWS Kinesis, AWS SNS etc.