Skip to main content

Pipeline templates

The pipeline templating system in Data Platform allows you to set up a template data pipeline, then resuse and customize it for different projects. This solution is helpful when you have consistent data structures but need to apply variations, e.g. changing the currency or language for different countries.

This guide will walk you through how to set up and use template pipelines to manage your product data across multiple projects.

Templates can be created for Blank, ERP, and Commerce pipelines. Product Feed Import and Crawler Import cannot be saved as templates.

Create a template pipeline​

  • Go to Data Platform > Pipelines.
  • Click "Add pipeline".
  • Choose the blank template when creating the pipeline.
  • Toggle the “Create a custom template” option to mark the pipeline as a template:

Pipeline templates

Add parameters ​

Define template parameters like URL, project ID, or SAP URL. These parameters will be dynamically filled with specific values in child pipelines during execution. 

The parameters defined in the template can be used throughout the configuration using the syntax ${parameterName}. For example, ${URL} will be replaced with the value set in the child pipeline. Setting URL to https://example.com in a child pipeline will replace ${URL} in the template configuration.

  • Set up the pipeline. (Learn more.)
  • Make sure to use the defined parameters in your configuration: 

Pipeline templates

Duplicate an existing pipeline​

You can duplicate an existing pipeline and modify it.

  • Go to Data Platform > Pipelines.
  • Click the three dot icon next to a template and select the "Duplicate" option:

Pipeline templates

  • Edit the settings of the pipeline copy.
  • Within settings, toggle the ”Save as template” option:

Pipeline templates

Create a pipeline from a template​

  • Add a new pipeline.
  • Select a template. When you’re creating a new pipeline, if templates are available, they will appear as additional options in the modal.
  • Choose the template to base your new pipeline on and click "Next”. 

Pipeline templates

  • Customize parameters. Set the specific values for the parameters defined in the template (e.g., the URL, project ID, etc.) for the new pipeline.

  • Execute the child pipeline.

The newly created child pipeline will inherit all global settings from the parent template and run with the configurations you have set.