0-0x Data Engineering Principles: Build Frameworks, Not Pipelines || Gatis Seja

0-0x Data Engineering Principles: Build Frameworks, Not Pipelines || Gatis Seja

Data Engineering Principles: Build Frameworks, Not Pipelines || Gatis Seja
Data pipelines are necessary for the flow of information from its source to its consumers, typically data scientists, analysts and software developers. Managing data flow from many sources is a complex task where the maintenance cost limits scale of being able to build a large reliable data warehouse. This presentation proposes a number of applied data engineering principles that can be used to build robust easily manageable data pipelines and data products. Examples will be shown using Python on AWS.

EVENT:

PyData London Meetup #54

SPEAKER:

Gatis Seja

PERMISSIONS:

PyData provided Coding Tech with the permission to republish this video.

View on YouTube

 

Leave a Reply

Your email address will not be published. Required fields are marked *