# How Bedrock Data Automation works
Bedrock Data Automation (BDA) lets you configure output based on your processing needs for a specific data type: documents, images, video or audio. BDA can generate standard output or custom output. Below are some key concepts for understanding how BDA works. If you're a new user, start with the information about standard output.
+ Standard output – Sending a file to BDA with no other information returns the default standard output, which consists of commonly required information that's based on the data type. Examples include audio transcriptions, scene summaries for video, and document summaries. These outputs can be tuned to your use case using projects to modify them. For more information, see [Standard output in Bedrock Data Automation](bda-standard-output.md).
+ Custom output – For documents, audio,and images only. Choose custom output to define exactly what information you want to extract using a blueprint. A blueprint consists of a list of expected fields that you want retrieved from a file. Each field represents a piece of information that needs to be extracted to meet your specific use case. You can create your own blueprints, or select predefined blueprints from the BDA blueprint catalog. For more information, see [Custom output and blueprints](bda-custom-output-idp.md).
+ Projects – A project is a BDA resource that allows you to modify and organize output configurations. Each project can contain standard output configurations for documents, images, video, and audio, as well as custom output blueprints for documents, audio, and images. Projects are referenced in the `InvokeDataAutomationAsync` API call to instruct BDA on how to process the files. For more information about projects and their use cases, see [Bedrock Data Automation projects](bda-projects.md).
# Bedrock Data Automation projects
One way to process files using Amazon Bedrock data automation (BDA) is to create a project. A project is a grouping of both standard and custom output configurations. Standard outputs are required in projects, but custom outputs are optional. When you call the `InvokeDataAutomationAsync` API with a project ARN, the file is automatically processed using the configuration settings defined in that project. Output is then generated based on the project's configuration.
A project can be given a stage, either `LIVE` or `DEVELOPMENT`. Each stage is a unique and mutable version of the project. This means you can edit or test with the `DEVELOPMENT` stage, and process customer requests using the `LIVE` stage. `DEVELOPMENT` projects can't be accessed in the console, and must be changed and invoked through the API.
A project allows you to use a single resource for multiple file types. For example, an audio file sent to BDA using project name ABC will be processed using project ABC’s audio standard output configuration. A document sent to BDA using project name ABC will be processed using project ABC’s document standard output configuration.
Projects grant you greater flexibility when setting up standard outputs. Each standard output has its own set of configurable options, such as transcripts or summaries, and projects allow you to change those options to better suit your use case. You can also configure a project with Blueprints for documents, audio, and images to define custom output. A project configured to generate custom output will also generate standard output automatically.
The following sections will go through a few examples of using projects.
## Using Projects with Standard Output
Let's consider a use case in which you're only interested in extracting transcript summaries of your full audio and video files. By default, when you send audio and video files to BDA, you receive the transcript summaries along with full transcripts, scene level summaries, detected text, and other information. For this use case, you don't want to spend the extra time and resources to collect information you don't need. For this use case, you can configure a standard output project to enable only the summary feature for audio and video files.
To do this using the API or the console, create a project and modify the standard output settings for audio and video. For video, enable **Full Video Summary** but ensure that other extractions (e.g., Full Audio Transcript, Scene Summaries, Content Moderation, etc.) are disabled. Repeat this configuration for audio. After you configure the project to generate only summaries, save the project and note the project's Amazon Resource Names (ARN). This ARN can be used for the `InvokeDataAutomationAsync` operation to process your files at scale. By passing an audio or video file to the BDA and specifying this project ARN, you will receive an output of only the summaries for each of the files. Note, in this example there was no configuration performed for documents or images. This means that if you pass an image or document to BDA using that project ARN, you will receive the default standard output for those files.
## Using projects with custom output and standard output
For this use case, let's assume that you want to generate standard output summaries for documents and audio files, and to also extract custom fields from your documents. After you create a project, configure the standard output for audio to enable **Full audio summary** and ensure that other extractions are not enabled. Repeat this standard output configuration for documents. You can then configure custom output for documents by adding a new blueprint or a preexisting blueprint from the BDA global catalog. Documents passed to BDA using this project ARN will generate the standard output full document summaries and the blueprint output for defined custom fields. Audio files passed to BDA using this project ARN will generate full summaries.
When processing documents, you might want to use multiple blueprints for different kinds of documents that are passed to your project. A project can have up to 40 document blueprints attached. BDA automatically matches your documents to the appropriate blueprint that's configured in your project, and generates custom output using that blueprint. Additionally, you might want to pass documents in bulk. If you pass a file that contains multiple documents, you can choose to split the document when creating your project. If you choose to do this, BDA scans the file and splits it into individual documents based on context. Those individual documents are then matched to the correct blueprint for processing.
Currently, images only support a single blueprint definition per project. Image file types JPG and PNG might be treated as images or as scanned documents based on their contents. We recommend that you create a custom blueprint for images when you process custom output for documents so BDA provides the desired output for image files that contain text.
Audio files also only support a single blueprint definition per project.
# Splitting documents while using projects
Amazon Bedrock Data Automation (BDA) supports splitting documents when using the Amazon Bedrock API. When enabled, splitting allows BDA to take a PDF containing multiple logical documents and split it into separate documents for processing.
Once splitting is complete, each segment of the split document is processed independently. This means an input document can contain different document types. For example, if you have a PDF containing 3 bank statements and one W2, splitting would attempt to divide it into 4 separate documents that would be processed individually.
BDA automatic splitting supports files with up to 3000 pages, and supports individual documents of up to 20 pages each.
The option to split documents is off by default, but can be toggled on when using the API. Below is an example of creating a project with the splitter enabled. The ellipsies represent additional blueprints provided to the project.
```
response = client.create_data_automation_project(
projectName=project_name,
projectDescription="Provide a project description",
projectStage='LIVE',
standardOutputConfiguration=output_config,
customOutputConfiguration={
'blueprints': [
{
'blueprintArn': Blueprint ARN,
'blueprintStage': 'LIVE'
},
...
]
},
overrideConfiguration={'document': {'splitter': {'state': 'ENABLED'}}}
)
```
The part that enables the splitting process is the overrideConfiguration line. This line sets up the splitter and allows you to pass multiple documents within the same file.
Documents are split by the semantic boundaries in the document.
Document splitting happens independently of applying blueprints, and documents that are split will be matched to the closest blueprint. For more information on how BDA matches blueprints see [Understanding blueprint matching](#bda-blueprint-matching).
## Understanding blueprint matching
Blueprint matching is based on the following elements:
+ Blueprint name
+ Blueprint description
+ Blueprint fields
When processing documents, you can provide multiple blueprints to match against. This allows processing different document types with appropriate blueprints. You can provide multiple blueprint IDs when invoking the data automation API, and BDA will attempt to match each document to the best fitting blueprint. This allows processing mixed document types in a single batch. This is useful when documents are expected to be of different types (e.g. bank statements, invoices, passports).
If you need separate blueprints because document formats are very different or require specialized prompts, creating one blueprint per document type can help with matching. For more information on creating useful blueprints, see [Best practices for creating blueprints](#bda-blueprint-best-practices).
## Best practices for creating blueprints
Follow the following best practices to get the most out of your blueprints:
+ Be explicit and detailed in blueprint names and descriptions to aid matching
+ Providing multiple relevant blueprints allows BDA to select the best match. Create separate blueprints for significantly different document formats
+ Consider creating specialized blueprints for every vendor/document source, if you need maximum accuracy
+ Do not include two blueprints of the same type in a project (e.g. two W2 blueprints). Information from the document itself and the blueprint is used to process documents, and including multiple blueprints of the same type in a project will lead to worse performance.
By leveraging document splitting and multiple blueprint matching, BDA can more flexibly handle varied document sets while applying the most appropriate extraction logic to each document.
# Disabling modalities and routing file types
By default, projects in BDA process supported file types, by sorting them to different semantic modalities. When creating or editing your project, you can modify what modalities will be processed, and which file types will be sent to which modalities. In this section, we will go through enabling and disabling different modalities, routing files to specific modalities, and the default routing procedure for BDA.
## Disabling modality processing
When you create a project, you might have a use case in mind that doesn't include processing all kinds of files. For example, you may want to only process documents and audio files. If that is the case, you don't want BDA to send a JPEG to be processed as an image or an MP4 to be processed as a video. Modality enablement allows you to turn off certain modalities in a project, curating responses from BDA's processing.
**Disabling modalities with the BDA Console**
When using the BDA console, modality enablement is handled by a checklist, where you can simply select or deselect each modality while editing or creating your Project. These options are located under the Advanced settings tab. At least one Modality must be selected for a project.
![\[Modality enablement options with checkboxes for document, image, video, and audio.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/modalityenableconsole.png)
**Disabling modalities with the BDA API**
When using the BDA API, modality enablement is handled by the `overrideConfiguration` request element, located in the `CreateDataAutomation` operation. Each modality has an associate section where you can declare the modality `ENABLED` or `DISABLED`. Below is an example of the `overrideConfiguration` element with only document and audio modalities enabled. The `modalityProcessing` flag defaults to `ENABLED`.
```
"overrideConfiguration" : {
"document": {
"splitter": {
"state": ENABLED
},
"modalityProcessing": {
"state": ENABLED
},
},
"image": {
"modalityProcessing": {
"state": DISABLED
}
},
"video": {
"modalityProcessing": {
"state": DISABLED
}
},
"audio": {
"modalityProcessing": {
"state": ENABLED
}
},
...
}
```
The ellipsies at the end of this section indicates the removal of the `modalityRouting` element, which we will discuss more in the next section.
## Routing files to certain processing types
Certain file types are capable of being routed to different modalities, based on a variety of factors. With modality routing you can set certain file types to route to certain modality processing manually. JPEGs and PNGs can be routed to either document or image processing. MP4s and MOVs can be routed to either video or audio processing.
**Routing with the BDA Console**
While in the Advanced settings tab when creating or editing a blueprint, you can choose to add a new manual modality routing. This lets you select one of the 4 available file types and then which processing modality they will be routed to. Below is a screenshot of the console, with a manual modality routing that sends PNG files to the document processing modality.
**Note**
Settings for JPEG files apply to both ".jpeg" and ".jpg" files. Settings for MP4 settings apply to both ".mp4" ".m4v" files.
![\[An image from the AWS console, showing a pair of drop down menus labeled file type and modality destination, selected as PNG and Document.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/manualrouting.png)
**Routing with the BDA API**
Similar to modality enablement, modality routing is handled via the `overrideConfiguration` request element. Below is an example of the `modalityRouting` portion of `overrideConfiguration`. This example assumes all modalities are enabled, and routes JPEG and PNG files to the document modality, and MP4 and MOV files to the audio modality.
```
...
"modalityRouting": {
"jpeg": DOCUMENT,
"png": DOCUMENT,
"mp4": AUDIO,
"mov": AUDIO
}
}
```
The ellipsies at the beginning of the example indicate the removal of the rest of the `overrideConfiguration`, which is discussed more in the section on modality enablement and the document splitter functionality.
## Standard routing for the InvokeDataAutomationAsync API
Without setting up your own routing procedures, BDA uses a standard set of procedures based on file type to determine what modality BDA will route to. The default procedures are listed in the table below.
PNGs and JPEGs list Semantic Classifer as a default behavior. This means that BDA will look at indicators of whether or not a submitted file is an image or a document using internal models and perform routing automatically.
| File Types | Default Routing Behavior |
| --- | --- |
| PNG | Semantic Classifer; Either Image or Document |
| JPEG | Semantic Classifer; Either Image or Document |
| PDF, TIFF | Document |
| MP4, MOV | Video |
| AMR, FLAC, M4A, MP3, OGG, WEBM, WAV | Audio |
## Standard routing for the InvokeDataAutomation API
The [InvokeDataAutomation](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomation.html) API will also look at indicators of whether or not a submitted PNG or JPEG file is an image or a document using internal models and perform routing automatically. The default procedures are listed in the table below.
PNGs and JPEGs list Semantic Classifier as a default behavior. This means that BDA will look at indicators of whether or not a submitted file is an image or a document using internal models and perform routing automatically. PDF and TIFF files will be routed to Documents modality for processing. InvokeDataAutomation API does not currently support Audio and Video files.
| File Types | Default Routing Behavior |
| --- | --- |
| PNG | Semantic Classifer; Either Image or Document |
| JPEG | Semantic Classifer; Either Image or Document |
| PDF, TIFF | Document |