LLM Summarize

The LLM Summarize processor summarizes English-language text in specified columns. The processor uses the Snowflake Large Language Model (LLM) Summarize function to generate a summary of the data.

When you configure the processor, you specify the columns to evaluate and optionally define output columns for the summary.

For more information about the Summary function and other Snowflake LLM functions, see the Snowflake documentation.

Note: At this time, Snowflake charges differently for LLM processing. See the Snowflake consumption rates for details.

Example

Say you want to summarize customer product reviews, which can be quite lengthy at times. The reviews are in a reviews column. Rather than have two columns in each record with review details, you want to simply overwrite the existing review data with the generated summary. To do this, you configure the LLM Summarize processor as follows:

Source property: reviews
Output Column property: reviews

You can alternatively leave the Output Column property undefined to overwrite data in the original source column.

Say the processor receives the following data:


productId	userId	reviews
32-129	NY999	This product has one job, put the pizza in the oven. It is very good at this one task. I have struggled for months using a metal pizza peel which felt like a lottery system getting the pizza to not stick while launching. Switching to this peel made me realize this is how it is intended to be done in the first place. Stuck pizzas are a thing of the past. All you have to do to use it is rub some flour on the peel and gently slide your pizza onto the peel. Do not try to slide the peel under your pizza, it is too thick for this to work. Do not try assembling your pizza on the peel either, you want the pizza to be on the peel for as short as possible to avoid sticking. After your pizza is on the peel and everything is ready to go, slowly slide the peel into your oven and I like to do a light tap and share then quickly slide it out. It will take some practice so just have patience. After a few times you will be able to consistently get it off without and issue. Remember to position your pizza where it is just barely about to fall off the edge of the peel, this will assist with getting it off. I would avoid using this peel to remove the pizza from the oven if possible, a metal peel is great for rotating or removing. This has made my pizza life a lot easier, highly recommend.
32-129	joanK	Crafted from bamboo, this pizza peel is aesthetically pleasing and environmentally friendly. Bamboo is known for its durability and sustainability, making it a wise choice for eco-conscious consumers. The material is sturdy yet lightweight, which is crucial for easily sliding pizzas in and out of the oven. The size of the peel is perfect for standard homemade pizzas. It offers ample space to build and transport a 12-inch pizza, but its not overly bulky, making it easy to handle and store. The tapered edge is a thoughtful design element, allowing for a smooth transition of the pizza to and from the baking surface. Another benefit of this peel is its versatility. It doubles as a cutting and serving board, making it a multifunctional tool in the kitchen. The surface is smooth and doesnt dull my pizza cutter, preserving both the peel and the blade. Maintenance is a breeze. The bamboo cleans up nicely and does not absorb odors or stains, which is essential for food preparation tools. To preserve its quality, I simply wash it by hand and occasionally treat it with mineral oil
32-129	to5ko	I use this for launching pizzas into my indoor and outdoor pizza ovens, and it had held up well. Easy to clean and use. I would recommend using a stainless steel peel for rotating and removing cooked pizzas, but this is perfect for launching and preventing the dough from sticking.

After processing the data, the processor passes the following data downstream:


productId	userID	reviews
32-129	NY999	This product is a pizza peel that effectively puts pizza in the oven without sticking. The user had difficulty with a metal peel and found this one to be the intended solution. To use it, rub flour on the peel, place pizza on it, and carefully slide it into the oven. Practice is required to master the technique. Recommended for easier pizza preparation.
32-129	joanK	This bamboo pizza peel is eco-friendly, durable, and lightweight, ideal for handling standard 12-inch pizzas. Its tapered edge ensures smooth transfer between surfaces, and it functions as a cutting and serving board. Easy to clean and maintain with minimal care.
32-129	to5ko	The text describes using a peel for launching pizzas into ovens and its durability and ease of use.

Notice how the summaries replace the original reviews. To keep the original reviews in addition to the summaries, you can simply specify a new output column name in the Output Column property.

Source and Output Columns

Note the following details about defining source and output columns:

Source

The processor evaluates data in the columns defined in the Source property.

To evaluate multiple columns, you can define multiple sets of configurations. You can also use regular expressions to have the processor evaluate all columns with matching names.

Output columns

The processor writes summaries into the columns defined in the Output Column property. The processor creates columns and overwrites data in output columns as follows:

When you define an output column that does not exist in incoming data, the processor creates the column.
When you define an output column that exists, the processor overwrites the data in the column.
When you do not define an output column, the processor places the data in the column being evaluated, overwriting the original data.
For example, if you configure the processor to evaluate a reviews column and do not specify an output column, the processor places the summary in the reviews column.

Note: When specifying the output column, you can use $0 to represent the evaluated source column name, and then add preceding or following characters.

For example, say you use a regular expression to define the source columns to evaluate. If you specify $0_summary as the name of the corresponding output columns, the processor writes the summary for a review source column to a new review_summary column.

Configuring an LLM Summarize Processor

Configure an LLM Summarize processor to summarize English-language text in specified fields.

On the General tab, configure the following properties:


General Property	Description
Name	Stage name.
Description	Optional description.
Cache Data	Caches processed data.

On the Summarize tab, configure the following property:


Summarize Property	Description
Summarize Configurations	Specify the following properties, as needed: Source - Name of the column to evaluate. You can use a regular expression to define a name pattern to match. Output Column - Output column for the generated summary. When not defined, the processor overwrites the associated source column. For more information, see Source and Output Columns. To specify additional columns to evaluate, click Add Another or Bulk Edit Mode.