Stage Library Match Requirement

Some stages require selecting a stage library to use to access an external system.

Due to a Spark requirement, when you specify a particular stage library for a stage, you must use the same stage library for all related stages in a pipeline. For example, if your pipeline includes an Amazon S3 origin and an Amazon S3 destination, you must use the same stage library for both stages.

On the other hand, if your pipeline includes unrelated stages, such as a Kafka origin with an Amazon S3 destination, you can naturally select entirely different stage libraries because the stages work with different external systems.