Big Data

What is a Data Pipeline?

Body
The term "data pipeline" describes a set of processes that move data from one place to another place. While data travels through the pipeline, it can undergo a variety of transformations, such as data enrichment and data duplication. ... A big data pipeline may process data in batches, stream processing, or other methods.

What is the Data, Information, Knowledge, Wisdom (DIKW) Pyramid?

Body
The DIKW Pyramid represents the relationships between data, information, knowledge and wisdom. Each building block is a step towards a higher level - first comes data, then is information, next is knowledge and finally comes wisdom. Each step answers different questions about the initial data and adds value to it. The more we enrich our data with meaning and context, the more knowledge and insights we get out of it so we can take better, informed and data-based decisions.