In the years since the rise of big data and analytics (Apache™ Hadoop® and associated distributed data systems), we’ve seen a tremendous amount of innovation from teams looking to overcome either use-case specific, industry-specific, or even infrastructure-specific data pipeline challenges.
At some point enough people begin innovating and creating similar designs, sharing their solutions at conferences or in online forums, and best practices form around implementing the same design patterns over and over.
