0-0x Data Engineering Principles: Build Frameworks, Not Pipelines || Gatis Seja
Data Engineering Principles: Build Frameworks, Not Pipelines || Gatis Seja
Data pipelines are necessary for the flow of information from its source to its consumers, typically data scientists, analysts and software developers. Managing data flow from many sources is a complex task where the maintenance cost limits scale of being able to build a large reliable data warehouse. This presentation proposes a number of applied data engineering principles that can be used to build robust easily manageable data pipelines and data products. Examples will be shown using Python on AWS.
PyData London Meetup #54
PyData provided Coding Tech with the permission to republish this video.