# Creative content generation with Amazon Nova Generating creative content **Note** This documentation is for Amazon Nova Version 1. Amazon Nova 2 is now available with new models and enhanced capabilities. New features and documentation updates are published in the Amazon Nova 2 User Guide. For information, visit [What's new in Amazon Nova 2](https://docs.aws.amazon.com/nova/latest/nova2-userguide/whats-new.html). Amazon Nova allows you to create novel images and videos in a wide variety of styles. The following sections detail the requirements and process necessary to create images or videos. For prompt engineering guidance, see [Amazon Nova Canvas prompting best practices](prompting-image-generation.md) and [Amazon Nova Reel prompting best practices](prompting-video-generation.md). **Topics** + [ # Generating images with Amazon Nova Canvas ](image-generation.md) + [ # Generating videos with Amazon Nova Reel ](video-generation.md) # Generating images with Amazon Nova Canvas Generating images With the Amazon Nova Canvas model, you can generate realistic, studio-quality images by using text prompts. You can use Amazon Nova Canvas for text-to-image and imaging editing applications. Amazon Nova Canvas supports the following features: + Text-to-image (T2I) generation – Input a text prompt and generate a new image as output. The generated image captures the concepts described by the text prompt. + Image conditioning – Uses an input reference image to guide image generation. The model generates output image that aligns with the layout and the composition of the reference image, while still following the textual prompt. + Color guided content – You can provide a list of hex color codes along with a prompt. A range of 1 to 10 hex codes can be provided. The image returned will incorporate the color palette provided by the user. + Image variation – Uses 1 to 5 images and an optional prompt as input. It generates a new image that borrows characteristics from the reference images including style, color palette, and subject. + Inpainting – Uses an image and a segmentation mask as input (either from the user or estimated by the model) and reconstructs the region defined by the mask. Use inpainting to replace masked pixels with new generated content. + Outpainting – Uses an image and a segmentation mask as input (either from the user or estimated by the model) and generates new content that seamlessly extends the masked region, effectively replacing the image background. + Background removal – Automatically identifies multiple objects in the input image and removes the background. The output image has a transparent background. + Subject consistency – Subject consistency is achieved by fine-tuning the model with reference images to preserve the chosen subject (for example, pet, shoe, or handbag) in generated images. + Content provenance – Use publicly available tools such as [Content Credentials Verify](https://contentcredentials.org/verify) to check if an image was generated by Amazon Nova Canvas. This should indicate the image was generated unless the metadata has been removed. + Watermarking – Adds an invisible watermark to all generated images to reduce the spread of misinformation, assist with copyright protection, and track content usage. Watermark detection is available to help you confirm whether an image was generated by an Amazon Nova model, which checks for the existence of this watermark. . | | Amazon Nova Canvas | | --- |--- | | Model ID | amazon.nova-canvas-v1:0 | | Input Modalities | Text, Image | | Output Modalities | Image | | Max Prompt Length | 1024 characters | | Max Output Resolution (generation tasks) | 4.19 million pixels (that is, 2048x2048, 2816x1536) | | Max Output Resolution (editing tasks) | Must meet all of the following: 4096 pixels on its longest side Aspect ratio between 1:4 and 4:1 Total pixel count of 4.19 million or smaller | | Supporting Input Image Types | PNG, JPEG | | Supported Languages | English | | Regions | US East (N. Virginia), Europe (Ireland), and Asia Pacific (Tokyo) | | Invoke Model API | Yes | | Fine-tuning | Yes | | Provisioned throughput | No | **Topics** + [ # Image generation and editing ](image-gen-access.md) + [ # Virtual try-on ](image-gen-vto.md) + [ # Visual Styles ](image-gen-styles.md) + [ # Request and response structure for image generation ](image-gen-req-resp-structure.md) + [ # Error handling ](image-gen-errors.md) + [ # Code examples ](image-gen-code-examples.md) # Image generation and editing Amazon Nova Canvas is available through the Bedrock [InvokeModel API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) and supports the following inference parameters and model responses when carrying out model inference. **Topics** + [ ## Image generation request and response format ](#image-gen-req-resp-format) + [ ## Input images for image generation ](#image-gen-input-images) + [ ## Masking images ](#image-gen-masking) + [ ## Supported image resolutions ](#image-gen-resolutions) ## Image generation request and response format Request and response format When you make an [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) call using the Amazon Nova Canvas model, replace the `body` field of the request with the format that matches your use-case. All tasks share an `imageGenerationConfig` object, but each task has a parameters object specific to that task. The following use-cases are supported: | Task Type Value | Task Parameter Field | Task Category | Description | | --- | --- | --- | --- | | TEXT\$1IMAGE with text only | textToImageParams | Generation | Generate an image using a text prompt. | | TEXT\$1IMAGE with image conditioning | textToImageParams | Generation | Provide an input conditioning image along with a text prompt to generate an image that follows the layout and composition of the conditioning image. | | COLOR\$1GUIDED\$1GENERATION | colorGuidedGenerationParams | Generation | Provide a list of color values in hexadecimal format (e.g. \$1FF9800) along with a text prompt and optional reference image to generate an image that follows the specified color palette. | | IMAGE\$1VARIATION | imageVariationParams | Generation | Provide one or more input images—with or without a text prompt—to influence the generated image. Can be used to influence the visual style of the generated image (when used with a text prompt), to generate variations of a single image (when used without a text prompt), and for other creative effects and control. | | INPAINTING | inPaintingParams | Editing | Modify an image by changing the area inside of a masked region. Can be used to add, remove, or replace elements of an image. | | OUTPAINTING | outPaintingParams | Editing | Modify an image by changing the area outside of a masked region. Can be used to replace the background behind a subject. | | BACKGROUND\$1REMOVAL | backgroundRemovalParams | Editing | Automatically remove the background of any image, replacing the background with transparent pixels. Can be useful when you want to later composite the image with other elements in an image editing app, presentation, or website. The background can easily be changed to a solid color through custom code as well. | | VIRTUAL\$1TRY\$1ON | virtualTryOnParams | Editing | Provide a source image and a reference image, superimposing an object in the reference image onto the source image. Can be used to visualize clothing and accessories on different models or in different poses, alter the style and appearance of an object or article or clothing, or transfer styles and designs from one object to another. | ## Input images for image generation Input images Many task types require one or more input images to be included in the request. Any image used in the request must be encoded as a Base64 string. Generally, images can be in PNG or JPEG format and must be 8 bits per color channel (RGB). PNG images may contain an additional alpha channel, but that channel must not contain any transparent or translucent pixels. For specific details on supported input image dimensions, see [Supported image resolutions](#image-gen-resolutions). A *mask image* is an image that indicates the area to be inpainted or outpainted. This image can contain only pure black and pure white pixels. For inpainting requests, the area that is colored black is called *the mask* and will be changed. The rest of the mask image must contain only pure white pixels. Pure white pixels indicate the area outside the mask. For outpainting requests, the area that is colored white will be changed by the model. Mask images must not contain any pixels that are not pure black or pure white. If you are using a JPEG image as a mask, it must be compressed at 100% quality to avoid introducing non-white or non-black pixels during compression. For examples of how to encode or decode an image to or from a Base64 string, see [the code examples](https://docs.aws.amazon.com/nova/latest/userguide/image-gen-code-examples.html). ## Masking images Input images When you're editing an image, a mask is a way of defining the regions to edit. You can define a mask in one of three ways: + `maskPrompt` – Write a natural language text prompt describing the part(s) of the image to be masked. + `maskImage` – A black and white image where pure black pixels indicate the area inside the mask and pure white pixels indicate the area outside the mask. For inpainting request, the black pixels will be changed by the model. For outpainting requests, the while pixels will be altered. + `garmentBasedMask` – An image-based mask that defines a region to be replaced along with some limited styling options. You can use a photo editing tool to draw masks or create them with your own custom code. Otherwise, use the maskPrompt field to allow the model to infer the mask. ## Supported image resolutions Image resolutions You may specify any output resolution for a generation task as long as it adheres to the following requirements: + Each side must be between 320-4096 pixels, inclusive. + Each side must be evenly divisible by 16. + The aspect ratio must be between 1:4 and 4:1. That is, one side can't be more than 4 times longer than the other side. + The total pixel count must be less than 4,194,304. Most of these same constraints apply to input images, as well. However, the sides of the images do not need to be evenly divisible by 16. # Virtual try-on *Virtual try-on* is an image-guided use case of inpainting in which the contents of a reference image are superimposed into a source image based on the guidance of a mask image. Amazon Nova Canvas has been tuned for garments, accessories, furniture, and related objects. The model also generalizes well to other cases, such as adding a logo or text into an image. You can generate up to five images with the virtual try-on API. By default only one image is generated. To perform a virtual try-on, you must provide three images: + *Source image* - The original image that you want to modify. For example, this might be an image or a person or a room scene. + *Reference image* - The image containing the item, object, or article that you want to superimpose into source image. For example, this might contain a jacket, bowl, or couch. For garments, the reference image can contain garments on or off a body and can contain multiple products that represent distinct outfit components (such as shirts, pants, and shoes in a single image). + *Mask image* - The image that defines which part of the source that you want to modify. A mask image is a black and white image used to define which part of the source image should be modified. Black pixels indicate the area of the source image to modify while white pixels indicate areas of the image to preserve. You can either provide your own mask image or you can let the model create one for you based on other input parameters you provide. The mask image can be returned as part of the output if specified. Here are some examples of how the model works. ------ #### [ Upper body clothing ] The following images show an example of how Amazon Nova superimposes an upper body article of clothing onto a model. | Source image | Reference image | Output | | --- |--- |--- | | ![\[A man wearing sunglasses, looking to left, wearing a blue shirt.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/vto1_source.jpg) | ![\[A pink-red button down shirt.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/vto1_ref.jpg) | ![\[A mean wearing sunglasses, looking to the left, wearing a pink-red button down shirt.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/vto1_output.png) | ------ #### [ Couch in a room ] The following images show an example of how Amazon Nova superimposes a couch into a room of furniture. | Source image | Reference image | Output | | --- |--- |--- | | ![\[A midcentury, modern grey couch in a room surrounded by other decorations.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/vto2_source.jpg) | ![\[An orange couch against a white background.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/vto2_ref.jpg) | ![\[An orange couch in a room surrounded by other decorations.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/vto2_output.png) | ------ Unlike other Amazon Nova Canvas task types, virtual try-on does not support a text prompt or negative text prompt. ## Defining the mask image You can either directly provide a mask image by specifying `maskType: "IMAGE"` or allow the model to compute it automatically using auxiliary inputs such as `maskType: "GARMENT"` or `maskType: "PROMPT"`. When a mask type of `"GARMENT"` is specified, Amazon Nova Canvas creates a garment-aware mask based on a `garmentClass` input parameter value that you specify. In most cases, you can use one of the following high-level garment classes: + `"UPPER_BODY"` - Creates a mask that includes full arm length. + `"LOWER_BODY"` - Creates a mask the includes full leg length with no gap between the legs. + `"FOOTWEAR"` - Creates a mask that fits the shoe profile demonstrated in the source image. + `"FULL_BODY"` - Creates a mask equivalent to the combination of `"UPPER_BODY"` and `"LOWER_BODY"`. You can use the `"PROMPT"` mask type to use natural language to describe the item in the source image that you want to replace. This is useful for non-garment scenarios. This feature utilizes the same auto-masking functionality that exists in the `"INPAINTING"` task type via the `maskPrompt` parameter. **Warning** Masks created with the `"PROMPT"` mask type will adhere tightly to the shape of the item you describe. This can be problematic in many scenarios because the product you are adding might not share the same silhouette or size of the item you are replacing. For this reason, the virtual try-on API also provides an optional `maskShape` parameter that can be set to `"BOUNDING_BOX"`. We recommend using this setting (which is the default) in most cases when using the `"PROMPT"` mask type. ## Generating new poses, hands, or faces You can instruct the model to either keep or regenerate the pose, hands, or face of the person in the source image. When you choose to keep these elements, they are automatically removed from the mask image, regardless of which `maskType` you have chosen. You might want to preserve pose, hands, or face in the following situations: + You are developing an application that allows end-users to draw their own masks. Preserving these features prevents the end-users from accidentally including the hands or face in the mask. + You are using `maskShape: BOUNDING_BOX` but don't want to generate new hands or face. With `preserveFace: ON` or `preserveHands: ON`, these features are automatically removed from the mask. + You are using `maskType:GARMENT` and `maskShape: BOUNDING_BOX` with a model that is not in an upright posture. In this case, the bounding box mask can overlap the face and we recommend using `preserveFace: ON`. Conversely, you might want to regenerate the pose, hands, or face in the following situations: + For garments that cover the neck, `preserveFace: ON` can exclude enough of the neck to have a detrimental impact on the output. + When the model is wearing high-heeled shoes and the reference image is of flat-heeled shoes, or vice-versa. In this case, preserving the body pose creates unnatural looking results. + Similar to the previous point, when trying on handbags or other accessories, generating new poses or hands can generate more natural-looking results. ## Styling cues The `garmentStyling` parameter allows you to preserve or alter specific garment styling cues that you might find in a photo shoot. For example, Amazon Nova Canvas can modify the styling of a shirt so that its sleeves are either rolled up or down or it can modify the shirt so that it is tucked in or not. The following options are available: + `"longSleeveStyle"` - Controls whether the sleeves of a long-sleeve shirt are rolled up or down. + `"SLEEVE_DOWN"` - Can be applied when the source image is wearing a long-sleeve shirt (sleeves up or down), short-sleeve shirt, or no-sleeve shirt. + `"SLEEVE_UP"` - Can be applied when the source image is wearing a long-sleeve shirt with the sleeves up, short-sleeve shirt, or no-sleeve shirt. + `"tuckingStyle"` - Controls whether an upper body garment appears tucked in or loose. + `"UNTUCKED"` - Can be applied regardless of whether the source image has the shirt tucked or untucked. + `"TUCKED"` - Can be applied when the source image has the shirt tucked in. + `"outerLayerStyle"` - Controls whether an upper body garment is styled open or closed. This defaults to `"CLOSED"` which is appropriate for most garments (such as shirts and sweaters). For outer garments, like jackets, setting this value to `"OPEN"` guarantees that the original upper body garment from the source image will be retained with the new outer garment being layered over it. Using a value of `"CLOSED"` with an outer garment might not always render the garment as closed. This is because a value of `"CLOSED"` only guarantees that every upper body garment in the source image will be replaced and can sometimes result in an open outer layer with a new under layer visible beneath. + `"CLOSED"` + `"OPEN"` For more information, see the `garmentStyling` parameters in [Request and response structure for image generation](image-gen-req-resp-structure.md). ## Image stitching Virtual try-on allows you to determine how images are stitched together to create the final image. You can choose from `"BALANCED"`, `"SEAMLESS"`, and `"DETAILED"`. Each merge style takes a different approach to how it stitches the elements together to create the final image, each with its own benefits and tradeoffs. + `"BALANCED"` - Protects any non-masked pixels in the original image, ensuring they remain 100% accurate to the original. In some cases, there will be a slight perceptible color or texture mismatch in the output image that presents as a kind of “ghost” image of the mask shape. This is most likely to occur when the image features a person standing against a solid color or uniformly textured background. To avoid this, you can use the `"SEAMLESS"` merge style instead. + `"SEAMLESS"` - Ensures that there will never be a noticeable seam between the masked and non-masked images areas in the final image. The tradeoff is that all pixels in the image change slightly and sometimes fine-grained details are diminished in the non-masked areas of the image. + `"DETAILED"` - Can greatly improve fine-grained details like logos and text, especially when the masked area is relatively small compared to the overall image. The model achieves this by performing inpainting on a tightly cropped, higher resolution version of the original image that only includes the masked area. It then merges the result back into the original image. As with using `"BALANCED"` mode, this mode can sometimes result in a visible seam. # Visual Styles Amazon Nova Canvas allows you to generate images in a variety of predefined styles. With the `"TEXT_TO_IMAGE"` task type, use the `style` parameter to pick a predefined visual style. Choose from these available styles: + `"3D_ANIMATED_FAMILY_FILM"` - A style that alludes to 3D animated films. Featuring realistic rendering and characters with cartoonish or exaggerated physical features. This style is capable of producing character-focused images, object- or prop-focused images, and environment- or setting-focused images of both interiors and exteriors. + `"DESIGN_SKETCH"` - A style featuring hand-drawn line-art without a lot of wash or fill that is not too refined. This style is used to convey concepts and ideas. It is useful for fashion and product design sketches as well as architectural sketches. + `"FLAT_VECTOR_ILLUSTRATION"` - A flat-color illustration style that is popular in business communications. It is also useful for icon and clip art images. + `"GRAPHIC_NOVEL_ILLUSTRATION"` - A vivid ink illustration style. Characters do not have exaggerated features, as with some other more cartoon-ish styles. + `"MAXIMALISM"` - Bright, elaborate, bold, and complex with strong shapes, and rich details. This style can be applied to a variety of subjects, such as illustrations, photography, interior design, graphic design, or packaging design. + `"MIDCENTURY_RETRO"` - Alludes to graphic design trends from the 1940s through 1960s. + `"PHOTOREALISM"` - Realistic photography style, including different repertoires such as stock photography, editorial photography, journalistic photography, and more. This style shows realistic lighting, depth of field, and composition fitting the repertoire. The most common subjects are humans, but can also include animals, landscapes, and other natural features. + `"SOFT_DIGITAL_PAINTING"` - This style has more finish and refinement than a sketch. It includes shading, three dimensionality, and texture that might be lacking in other styles. **Note** Amazon Nova Canvas is not limited to the styles in this list. You can achieve many other visual styles by omitting the `style` parameter and describing your desired style within your prompt. Optionally, you can use the `negativeText` parameter to further steer the style characteristics away from undesired characteristics. The following images display the same image generated in each of the previously described styles. ## 3D animated family film ![\[The image depicts an elephant in the 3d animated family film style.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/3D_ANIMATED_FAMILY_FILM.png) ## Design sketch ![\[The image depicts an elephant in the design sketch style.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/DESIGN_SKETCH.png) ## Flat vector illustration ![\[The image depicts an elephant in the flat vector illustration style.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/FLAT_VECTOR_ILLUSTRATION.png) ## Graphic novel illustration ![\[The image depicts an elephant in the graphic novel illustration style.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/GRAPHIC_NOVEL_ILLUSTRATION.png) ## Maximalism ![\[The image depicts an elephant in the maximalism style.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/MAXIMALISM.png) ## Midcentury retro ![\[The image depicts an elephant in the midcentury retro style.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/MIDCENTURY_RETRO.png) ## Photorealism ![\[The image depicts an elephant in the photorealism style.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/PHOTOREALISM.png) ## Soft digital painting ![\[The image depicts an elephant in the soft digital painting style.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/SOFT_DIGITAL_PAINTING.png) # Request and response structure for image generation Request and response structure **Image generation** The following examples present different image generation use cases. Each example provides an explanation of the fields that are used for the image generation. ------ #### [ Text-to-image request ] ``` { "taskType": "TEXT_IMAGE", "textToImageParams": { "text": string, "negativeText": string, "style": "3D_ANIMATED_FAMILY_FILM" | "DESIGN_SKETCH" | "FLAT_VECTOR_ILLUSTRATION" | "GRAPHIC_NOVEL_ILLUSTRATION" | "MAXIMALISM" | "MIDCENTURY_RETRO" | "PHOTOREALISM" | "SOFT_DIGITAL_PAINTING" }, "imageGenerationConfig": { "width": int, "height": int, "quality": "standard" | "premium", "cfgScale": float, "seed": int, "numberOfImages": int } } ``` The following `textToImageParams` fields are used in this request: + `text` (Required) – A text prompt to generate the image. The prompt must be 1-1024 characters in length. + `negativeText` (Optional) – A text prompt to define what not to include in the image. This value must be 1-1024 characters in length. + `style` (Optional) – Specifies the style that is used to generate this image. For more information, see [Visual Styles](image-gen-styles.md). **Note** Avoid using negating words (“no”, “not”, “without”, etc.) in your `text` and `negativeText` values. For example, if you do not want mirrors in an image, instead of including "no mirrors" or "without mirrors" in the `text` field, use the word "mirrors" in the `negativeText` field. ------ #### [ Text-to-image request with image conditioning ] ``` { "taskType": "TEXT_IMAGE", "textToImageParams": { "conditionImage": string (Base64 encoded image), "controlMode": "CANNY_EDGE" | "SEGMENTATION", "controlStrength": float, "text": string, "negativeText": string, "style": "3D_ANIMATED_FAMILY_FILM" | "DESIGN_SKETCH" | "FLAT_VECTOR_ILLUSTRATION" | "GRAPHIC_NOVEL_ILLUSTRATION" | "MAXIMALISM" | "MIDCENTURY_RETRO" | "PHOTOREALISM" | "SOFT_DIGITAL_PAINTING" }, "imageGenerationConfig": { "width": int, "height": int, "quality": "standard" | "premium", "cfgScale": float, "seed": int, "numberOfImages": int } } ``` The following `textToImageParams` fields are used in this request: + `conditionImage` (Required) – A JPEG or PNG image that guides the layout and composition of the generated image. The image must be formatted as a Base64 string. See [Input images for image generation](image-gen-access.md#image-gen-input-images) for additional requirements. + `controlMode` (Optional) – Specifies what conditioning mode is be used. The default value is "CANNY\$1EDGE". + `CANNY_EDGE` – Elements of the generated image will follow the prominent contours, or "edges", of the condition image closely. + `SEGMENTATION` – The condition image will be automatically analyzed to identify prominent content shapes. This analysis results in a segmentation mask which guides the generation, resulting in a generated image that closely follows the layout of the condition image but allows the model more freedom within the bounds of each content area. + `controlStrength` (Optional) – Specifies how similar the layout and composition of the generated image should be to the `conditionImage`. The range is 0 to 1.0, and lower values introduce more randomness. The default value is 0.7. + `text` (Required) – A text prompt to generate the image. The prompt must be 1-1024 characters in length. + `negativeText` (Optional) – A text prompt to define what not to include in the image. This value must be 1-1024 characters in length. + `style` (Optional) – Specifies the style that is used to generate this image. For more information, see [Visual Styles](image-gen-styles.md). **Note** Avoid using negating words (“no”, “not”, “without”, etc.) in your `text` and `negativeText` values. For example, if you do not want mirrors in an image, instead of including "no mirrors" or "without mirrors" in the `text` field, use the word "mirrors" in the `negativeText` field. ------ #### [ Color guided image generation request ] ``` { "taskType": "COLOR_GUIDED_GENERATION", "colorGuidedGenerationParams": { "colors": string[] (list of hexadecimal color values), "referenceImage": string (Base64 encoded image), "text": string, "negativeText": string }, "imageGenerationConfig": { "width": int, "height": int, "quality": "standard" | "premium", "cfgScale": float, "seed": int, "numberOfImages": int } } ``` The following `colorGuidedGenerationParams` fields are used in this request: + `colors` (Required) – A list of up to 10 color codes that define the desired color palette for your image. Expressed as hexadecimal values in the form “\$1RRGGBB”. For example, "\$100FF00" is pure green and "\$1FCF2AB" is a warm yellow. The `colors` list has the strongest effect when a `referenceImage` is not provided. Otherwise, the colors in the list and the colors from the reference image will both be used in the final output. + `referenceImage` (Optional) – A JPEG or PNG image to use as a subject and style reference. The colors of the image will also be incorporated into you final output, along with the colors in from the `colors` list. See [Input images for image generation](image-gen-access.md#image-gen-input-images) for additional requirements. + `text` (Required) – A text prompt to generate the image. The prompt must be 1-1024 characters in length. + `negativeText` (Optional) – A text prompt to define what not to include in the image. This value must be 1-1024 characters in length. **Note** Avoid using negating words (“no”, “not”, “without”, etc.) in your `text` and `negativeText` values. For example, if you do not want mirrors in an image, instead of including "no mirrors" or "without mirrors" in the `text` field, use the word "mirrors" in the `negativeText` field. ------ #### [ Image variation request ] ``` { "taskType": "IMAGE_VARIATION", "imageVariationParams": { "images": string[] (list of Base64 encoded images), "similarityStrength": float, "text": string, "negativeText": string }, "imageGenerationConfig": { "height": int, "width": int, "cfgScale": float, "seed": int, "numberOfImages": int } } ``` The following `imageVariationParams` fields are used in this request: + `images` (Required) - A list of 1–5 images to use as references. Each must be in JPEG or PNG format and encoded as Base64 strings. See [Input images for image generation](image-gen-access.md#image-gen-input-images) for additional requirements. + `similarityStrength` (Optional) – Specifies how similar the generated image should be to the input images. Valid values are betweeen 0.2-1.0 with lower values used to introduce more randomness. + `text` (Required) – A text prompt to generate the image. The prompt must be 1-1024 characters in length. If you omit this field, the model will remove elements inside the masked area. They will be replaced with a seamless extension of the image background. + `negativeText` (Optional) – A text prompt to define what not to include in the image. This value must be 1-1024 characters in length. **Note** Avoid using negating words (“no”, “not”, “without”, etc.) in your `text` and `negativeText` values. For example, if you do not want mirrors in an image, instead of including "no mirrors" or "without mirrors" in the `text` field, use the word "mirrors" in the `negativeText` field. ------ **Image editing** The following examples present different image editing use cases. Each example provides an explanation of the fields that are used to edit the image. ------ #### [ Inpainting request ] ``` { "taskType": "INPAINTING", "inPaintingParams": { "image": string (Base64 encoded image), "maskPrompt": string, "maskImage": string (Base64 encoded image), "text": string, "negativeText": string }, "imageGenerationConfig": { "numberOfImages": int, "quality": "standard" | "premium", "cfgScale": float, "seed": int } } ``` The following `inPaintingParams` fields are used in this request: + `image` (Required) - The JPEG or PNG that you want to modify, formatted as a Base64 string. See [Input images for image generation](image-gen-access.md#image-gen-input-images) for additional requirements. + `maskPrompt` or `maskImage` (Required) – You must specify either the `maskPrompt` or the `maskImage` parameter, but not both. The `maskPrompt` is a natural language text prompt that describes the regions of the image to edit. The `maskImage` is an image that defines the areas of the image to edit. The mask image must be the same size as the input image. Areas to be edited are shaded pure black and areas to ignore are shaded pure white. No other colors are allowed in the mask image. Note that inpainting and outpainting requests are opposites in regard to the color requirements of the mask images. + `text` (Required) – A text prompt that describes what to generate within the masked region. The prompt must be 1-1024 characters in length. If you omit this field, the model will remove elements inside the masked area. They will be replaced with a seamless extension of the image background. + `negativeText` (Optional) – A text prompt to define what not to include in the image. This value must be 1-1024 characters in length. **Note** Avoid using negating words (“no”, “not”, “without”, etc.) in your `text` and `negativeText` values. For example, if you do not want mirrors in an image, instead of including "no mirrors" or "without mirrors" in the `text` field, use the word "mirrors" in the `negativeText` field. ------ #### [ Outpainting request ] ``` { "taskType": "OUTPAINTING", "outPaintingParams": { "image": string (Base64 encoded image), "maskPrompt": string, "maskImage": string (Base64 encoded image), "outPaintingMode": "DEFAULT" | "PRECISE", "text": string, "negativeText": string }, "imageGenerationConfig": { "numberOfImages": int, "quality": "standard" | "premium", "cfgScale": float, "seed": int } } ``` The following `outPaintingParams` fields are used in this request: + `image` (Required) - The JPEG or PNG that you want to modify, formatted as a Base64 string. See [Input images for image generation](image-gen-access.md#image-gen-input-images) for additional requirements. + `maskPrompt` or `maskImage` (Required) – You must specify either the `maskPrompt` or the `maskImage` parameter, but not both. The `maskPrompt` is a natural language text prompt that describes the regions of the image to edit. The `maskImage` is an image that defines the areas of the image to edit. The mask image must be the same size as the input image. Areas to be edited are shaded pure black and areas to ignore are shaded pure white. No other colors are allowed in the mask image. Note that inpainting and outpainting requests are opposites in regard to the color requirements of the mask images. + `outPaintingMode` - Determines how the mask that you provide is interpreted. Use `DEFAULT` to transition smoothly between the masked area and the non-masked area. Some of the original pixels are used as the starting point for the new background. This mode is generally better when you want the new background to use similar colors as the original background. However, you can get a halo effect if your prompt calls for a new background that is significantly different than the original background. Use `PRECISE` to strictly adhere to the mask boundaries. This mode is generally better when you are making significant changes to the background. + `text` (Required) – A text prompt that describes what to generate within the masked region. The prompt must be 1-1024 characters in length. If you omit this field, the model will remove elements inside the masked area. They will be replaced with a seamless extension of the image background. + `negativeText` (Optional) – A text prompt to define what not to include in the image. This value must be 1-1024 characters in length. **Note** Avoid using negating words (“no”, “not”, “without”, etc.) in your `text` and `negativeText` values. For example, if you do not want mirrors in an image, instead of including "no mirrors" or "without mirrors" in the `text` field, use the word "mirrors" in the `negativeText` field. ------ #### [ Background removal request ] ``` { "taskType": "BACKGROUND_REMOVAL", "backgroundRemovalParams": { "image": string (Base64 encoded image) } } ``` The following `backgroundRemovalParams` field is used in this request: + `image` (Required) – The JPEG or PNG that you want to modify, formatted as a Base64 string. See [Input images for image generation](image-gen-access.md#image-gen-input-images) for additional requirements. The `BACKGROUND_REMOVAL` task will return a PNG image with full 8-bit transparency. This format gives you smooth, clean isolation of the foreground objects and makes it easy to composite the image with other elements in an image editing app, presentation, or website. The background can easily be changed to a solid color using simple custom code. ------ #### [ Virtual try-on ] ``` { "taskType": "VIRTUAL_TRY_ON", "virtualTryOnParams": { "sourceImage": string (Base64 encoded image), "referenceImage": string (Base64 encoded image), "maskType": "IMAGE" | "GARMENT" | "PROMPT", "imageBasedMask":{ "maskImage": string (Base64 encoded image), }, "garmentBasedMask":{ "maskShape": "CONTOUR" | "BOUNDING_BOX" | "DEFAULT", "garmentClass": "UPPER_BODY" | "LOWER_BODY" | "FULL_BODY" | "FOOTWEAR" | "LONG_SLEEVE_SHIRT" | "SHORT_SLEEVE_SHIRT" | "NO_SLEEVE_SHIRT" | "OTHER_UPPER_BODY" | "LONG_PANTS" | "SHORT_PANTS" | "OTHER_LOWER_BODY" | "LONG_DRESS" | "SHORT_DRESS" | "FULL_BODY_OUTFIT" | "OTHER_FULL_BODY" | "SHOES" | "BOOTS" | "OTHER_FOOTWEAR", "garmentStyling":{ "longSleeveStyle": "SLEEVE_DOWN" | "SLEEVE_UP", "tuckingStyle": "UNTUCKED" | "TUCKED", "outerLayerStyle": "CLOSED" | "OPEN", } }, "promptBasedMask":{ "maskShape": "BOUNDING_BOX" | "CONTOUR" | "DEFAULT", "maskPrompt": string, }, "maskExclusions": { "preserveBodyPose": "ON" | "OFF" | "DEFAULT", "preserveHands": "ON" | "OFF" | "DEFAULT", "preserveFace": "OFF" | "ON" | "DEFAULT" }, "mergeStyle" : "BALANCED" | "SEAMLESS" | "DETAILED" , "returnMask": boolean, }, "imageGenerationConfig": { "numberOfImages": int, "quality": "standard" | "premium", "cfgScale": float, "seed": int } } ``` The following `virtualTryOnParams` fields are used in this request: + `sourceImage` (Required) – The JPEG or PNG that you want to modify, formatted as a Base64 string. See [Input images for image generation](image-gen-access.md#image-gen-input-images) for additional requirements. + `referenceImage` (Required) – The JPEG or PNG that contains the object that you want to superimpose onto the source image, formatted as a Base64 string. See [Input images for image generation](image-gen-access.md#image-gen-input-images) for additional requirements. + `maskType` (Required) – Specifies whether the mask is provided as an image, prompt, or garment mask. + `imageBasedMask` – Required when `maskType` is `"IMAGE"`. The `maskImage` is an image that defines the areas of the image to edit. The mask image must be the same size as the input image. Areas to be edited are shaded pure black and areas to ignore are shaded pure white. No other colors are allowed in the mask image. + `garmentBasedMask` – Required when `maskType` is `"GARMENT"`. + `maskShape` (Optional) – Defines the shape of the mask bounding box. The shape and size of the bounding box can have an affect on how the reference image is transferred to the source image. + `garmentClass` (Required) – Defines the article of clothing that is being transferred. This parameter allows the model focus on specific parts of the reference image that you want to transfer. + `garmentStyling` (Optional) – Provides styling cues to the model for certain articles of clothing. The `longSleeveStyle` and `tuckingStyle` parameters apply only to upper body garments. The `outerLayerStyle` parameter applies only to outer layer, upper body garments. + `promptBasedMask` (Required) – Required when `maskType` is `"PROMPT"`. + `maskShape` (Optional) – Defines the shape of the mask bounding box. The shape and size of the bounding box can have an affect on how the reference image is transferred to source image. + `maskPrompt` (Required) – A natural language text prompt that describes the regions of the image to edit. + `maskExclusions` (Optional) – When a person is detected in the source image, these parameters determine whether their body pose, hands, and face should be kept in the output image or regenerated. + `mergeStyle` (Optional) – Determines how the source and reference images are stitched together. Each merge style takes a different approach to how it stitches the elements together to create the final image, each with its own benefits and tradeoffs. + `"BALANCED"` - Protects any non-masked pixels in the original image, ensuring they remain 100% accurate to the original. In some cases, there will be a slight perceptible color or texture mismatch in the output image that presents as a kind of “ghost” image of the mask shape. This is most likely to occur when the image features a person standing against a solid color or uniformly textured background. To avoid this, you can use the `"SEAMLESS"` merge style instead. + `"SEAMLESS"` - Ensures that there will never be a noticeable seam between the masked and non-masked images areas in the final image. The tradeoff is that this mode results in all pixels in the image changing slightly and can sometimes diminish fine-grained details in the non-masked areas of the image. + `"DETAILED"` - Can greatly improve fine-grained details like logos and text, especially when the masked area is relatively small compared to the overall image. The model achieves this by performing inpainting on a tightly cropped, higher resolution version of the original image that only includes the masked area. It then merges the result back into the original image. As with using `"BALANCED"` mode, this mode can sometimes result in a visible seam. + `returnMask` (Optional) – Specifies whether the mask image is returned with the output image. ------ **Response body** The response body will contain one or more of the following fields: ``` { "images": "images": string[] (list of Base64 encoded images), "maskImage": string (Base64 encoded image), "error": string } ``` + `images` – When successful, a list of Base64-encoded strings that represent each image that was generated is returned. This list does not always contain the same number of images that you requested. Individual images might be blocked after generation if they do not align with the AWS Responsible AI (RAI) content moderation policy. Only images that align with the RAI policy are returned. + `maskImage` - When you specified that the mask image should be returned with the output, this is where it is returned. + `error` – If any image does not align with the RAI policy, this field is returned. Otherwise, this field is omitted from the response. The `imageGenerationConfig` field is common to all task types except `BACKGROUND_REMOVAL`. It is optional and contains the following fields. If you omit this object, the default configurations are used. + `width` and `height` (Optional) – Define the size and aspect ratio of the generated image. Both default to 1024. The `width` and `height` values should not be provided for the `"INPAINTING"`, `"OUTPAINTING"`, or `"VIRTUAL_TRY_ON"` task types. For the full list of supported resolutions, see [Supported image resolutions](image-gen-access.md#image-gen-resolutions). + `quality` (Optional) - Specifies the quality to use when generating the image - "standard" (default) or "premium". + `cfgScale` (Optional) – Specifies how strictly the model should adhere to the prompt. Values range from 1.1-10, inclusive, and the default value is 6.5. + Low values (1.1-3) - More creative freedom for the AI, potentially more aesthetic, but low contrast and less prompt-adherent results + Medium values (4-7) - Balanced approach, typically recommended for most generations + High values (8-10) - Strict prompt adherence, which can produce more precise results but sometimes at the cost of natural aesthetics and increased color saturation + `numberOfImages` (Optional) – The number of images to generate. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/nova/latest/userguide/image-gen-req-resp-structure.html) + `seed` (Optional) – Determines the initial noise setting for the generation process. Changing the seed value while leaving all other parameters the same will produce a totally new image that still adheres to your prompt, dimensions, and other settings. It is common to experiment with a variety of seed values to find the perfect image. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/nova/latest/userguide/image-gen-req-resp-structure.html) **Important** Resolution (`width` and `height`), `numberOfImages`, and `quality` all have an impact on the time it takes for generation to complete. The AWS SDK has a default `read_timeout` of 60 seconds which can easily be exceeded when using higher values for these parameters. Therefore, it is recommended that you increase the `read_timeout` of your invocation calls to at least 5 minutes (300 seconds). The code examples demonstrate how to do this. # Error handling There are three primary types of errors that you want to handle in your application code. These are input validation errors, AWS Responsible AI (RAI) input deflection errors, and RAI output deflection errors. These errors are unique to Amazon Nova Canvas. Input validation errors occur when you use an unsupported value for an input parameter. For example, a width value that doesn’t match one of the supported resolutions, an input image that exceeds the maximum allowed size, or a `maskImage` that contains colors other than pure black and white. All input validation errors are expressed as a `ValidationException` which contains a message string describing the cause of the problem. RAI input deflection errors occur when any of the input text values or images are determined to violate the AWS Responsible AI policy. These errors are expressed as a `ValidationException` with one of the following messages: + Input text validation message - “This request has been blocked by our content filters. Please adjust your text prompt to submit a new request.” + Input image validation message - “This request has been blocked by our content filters. Please adjust your input image to submit a new request.” RAI output deflection errors occur when an image is generated but it is misaligned with the AWS Responsible AI policy. When this occurs, an exception is not used. Instead, a successful response is returned, and its structure contains an error field which is a string with one of the following values: + If all requested images violate RAI policy - “All of the generated images have been blocked by our content filters.” + If some, but not all, requested images violate RIA policy - “Some of the generated images have been blocked by our content filters.” # Code examples The following examples provide sample code for various image generation tasks. ------ #### [ Text to image generation ] ``` # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """ Shows how to generate an image from a text prompt with the Amazon Nova Canvas model (on demand). """ import base64 import io import json import logging import boto3 from PIL import Image from botocore.config import Config from botocore.exceptions import ClientError class ImageError(Exception): "Custom exception for errors returned by Amazon Nova Canvas" def __init__(self, message): self.message = message logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def generate_image(model_id, body): """ Generate an image using Amazon Nova Canvas model on demand. Args: model_id (str): The model ID to use. body (str) : The request body to use. Returns: image_bytes (bytes): The image generated by the model. """ logger.info( "Generating image with Amazon Nova Canvas model %s", model_id) bedrock = boto3.client( service_name='bedrock-runtime', config=Config(read_timeout=300) ) accept = "application/json" content_type = "application/json" response = bedrock.invoke_model( body=body, modelId=model_id, accept=accept, contentType=content_type ) response_body = json.loads(response.get("body").read()) base64_image = response_body.get("images")[0] base64_bytes = base64_image.encode('ascii') image_bytes = base64.b64decode(base64_bytes) finish_reason = response_body.get("error") if finish_reason is not None: raise ImageError(f"Image generation error. Error is {finish_reason}") logger.info( "Successfully generated image with Amazon Nova Canvas model %s", model_id) return image_bytes def main(): """ Entrypoint for Amazon Nova Canvas example. """ logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") model_id = 'amazon.nova-canvas-v1:0' prompt = """A photograph of a cup of coffee from the side.""" body = json.dumps({ "taskType": "TEXT_IMAGE", "textToImageParams": { "text": prompt }, "imageGenerationConfig": { "numberOfImages": 1, "height": 1024, "width": 1024, "cfgScale": 8.0, "seed": 0 } }) try: image_bytes = generate_image(model_id=model_id, body=body) image = Image.open(io.BytesIO(image_bytes)) image.show() except ClientError as err: message = err.response["Error"]["Message"] logger.error("A client error occurred:", message) print("A client error occured: " + format(message)) except ImageError as err: logger.error(err.message) print(err.message) else: print( f"Finished generating image with Amazon Nova Canvas model {model_id}.") if __name__ == "__main__": main() ``` ------ #### [ Inpainting ] ``` # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """ Shows how to use inpainting to generate an image from a source image with the Amazon Nova Canvas model (on demand). The example uses a mask prompt to specify the area to inpaint. """ import base64 import io import json import logging import boto3 from PIL import Image from botocore.config import Config from botocore.exceptions import ClientError class ImageError(Exception): "Custom exception for errors returned by Amazon Nova Canvas" def __init__(self, message): self.message = message logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def generate_image(model_id, body): """ Generate an image using Amazon Nova Canvas model on demand. Args: model_id (str): The model ID to use. body (str) : The request body to use. Returns: image_bytes (bytes): The image generated by the model. """ logger.info( "Generating image with Amazon Nova Canvas model %s", model_id) bedrock = boto3.client( service_name='bedrock-runtime', config=Config(read_timeout=300) ) accept = "application/json" content_type = "application/json" response = bedrock.invoke_model( body=body, modelId=model_id, accept=accept, contentType=content_type ) response_body = json.loads(response.get("body").read()) base64_image = response_body.get("images")[0] base64_bytes = base64_image.encode('ascii') image_bytes = base64.b64decode(base64_bytes) finish_reason = response_body.get("error") if finish_reason is not None: raise ImageError(f"Image generation error. Error is {finish_reason}") logger.info( "Successfully generated image with Amazon Nova Canvas model %s", model_id) return image_bytes def main(): """ Entrypoint for Amazon Nova Canvas example. """ try: logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") model_id = 'amazon.nova-canvas-v1:0' # Read image from file and encode it as base64 string. with open("/path/to/image", "rb") as image_file: input_image = base64.b64encode(image_file.read()).decode('utf8') body = json.dumps({ "taskType": "INPAINTING", "inPaintingParams": { "text": "Modernize the windows of the house", "negativeText": "bad quality, low res", "image": input_image, "maskPrompt": "windows" }, "imageGenerationConfig": { "numberOfImages": 1, "height": 512, "width": 512, "cfgScale": 8.0 } }) image_bytes = generate_image(model_id=model_id, body=body) image = Image.open(io.BytesIO(image_bytes)) image.show() except ClientError as err: message = err.response["Error"]["Message"] logger.error("A client error occurred: %s", message) print("A client error occured: " + format(message)) except ImageError as err: logger.error(err.message) print(err.message) else: print( f"Finished generating image with Amazon Nova Canvas model {model_id}.") if __name__ == "__main__": main() ``` ------ #### [ Outpainting ] ``` # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """ Shows how to use outpainting to generate an image from a source image with the Amazon Nova Canvas model (on demand). The example uses a mask image to outpaint the original image. """ import base64 import io import json import logging import boto3 from PIL import Image from botocore.config import Config from botocore.exceptions import ClientError class ImageError(Exception): "Custom exception for errors returned by Amazon Nova Canvas" def __init__(self, message): self.message = message logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def generate_image(model_id, body): """ Generate an image using Amazon Nova Canvas model on demand. Args: model_id (str): The model ID to use. body (str) : The request body to use. Returns: image_bytes (bytes): The image generated by the model. """ logger.info( "Generating image with Amazon Nova Canvas model %s", model_id) bedrock = boto3.client( service_name='bedrock-runtime', config=Config(read_timeout=300) ) accept = "application/json" content_type = "application/json" response = bedrock.invoke_model( body=body, modelId=model_id, accept=accept, contentType=content_type ) response_body = json.loads(response.get("body").read()) base64_image = response_body.get("images")[0] base64_bytes = base64_image.encode('ascii') image_bytes = base64.b64decode(base64_bytes) finish_reason = response_body.get("error") if finish_reason is not None: raise ImageError(f"Image generation error. Error is {finish_reason}") logger.info( "Successfully generated image with Amazon Nova Canvas model %s", model_id) return image_bytes def main(): """ Entrypoint for Amazon Nova Canvas example. """ try: logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") model_id = 'amazon.nova-canvas-v1:0' # Read image and mask image from file and encode as base64 strings. with open("/path/to/image", "rb") as image_file: input_image = base64.b64encode(image_file.read()).decode('utf8') with open("/path/to/mask_image", "rb") as mask_image_file: input_mask_image = base64.b64encode( mask_image_file.read()).decode('utf8') body = json.dumps({ "taskType": "OUTPAINTING", "outPaintingParams": { "text": "Draw a chocolate chip cookie", "negativeText": "bad quality, low res", "image": input_image, "maskImage": input_mask_image, "outPaintingMode": "DEFAULT" }, "imageGenerationConfig": { "numberOfImages": 1, "height": 512, "width": 512, "cfgScale": 8.0 } } ) image_bytes = generate_image(model_id=model_id, body=body) image = Image.open(io.BytesIO(image_bytes)) image.show() except ClientError as err: message = err.response["Error"]["Message"] logger.error("A client error occurred: %s", message) print("A client error occured: " + format(message)) except ImageError as err: logger.error(err.message) print(err.message) else: print( f"Finished generating image with Amazon Nova Canvas model {model_id}.") if __name__ == "__main__": main() ``` ------ #### [ Image variation ] ``` # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """ Shows how to generate an image variation from a source image with the Amazon Nova Canvas model (on demand). """ import base64 import io import json import logging import boto3 from PIL import Image from botocore.config import Config from botocore.exceptions import ClientError class ImageError(Exception): "Custom exception for errors returned by Amazon Nova Canvas" def __init__(self, message): self.message = message logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def generate_image(model_id, body): """ Generate an image using Amazon Nova Canvas model on demand. Args: model_id (str): The model ID to use. body (str) : The request body to use. Returns: image_bytes (bytes): The image generated by the model. """ logger.info( "Generating image with Amazon Nova Canvas model %s", model_id) bedrock = boto3.client( service_name='bedrock-runtime', config=Config(read_timeout=300) ) accept = "application/json" content_type = "application/json" response = bedrock.invoke_model( body=body, modelId=model_id, accept=accept, contentType=content_type ) response_body = json.loads(response.get("body").read()) base64_image = response_body.get("images")[0] base64_bytes = base64_image.encode('ascii') image_bytes = base64.b64decode(base64_bytes) finish_reason = response_body.get("error") if finish_reason is not None: raise ImageError(f"Image generation error. Error is {finish_reason}") logger.info( "Successfully generated image with Amazon Nova Canvas model %s", model_id) return image_bytes def main(): """ Entrypoint for Amazon Nova Canvas example. """ try: logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") model_id = 'amazon.nova-canvas-v1:0' # Read image from file and encode it as base64 string. with open("/path/to/image", "rb") as image_file: input_image = base64.b64encode(image_file.read()).decode('utf8') body = json.dumps({ "taskType": "IMAGE_VARIATION", "imageVariationParams": { "text": "Modernize the house, photo-realistic, 8k, hdr", "negativeText": "bad quality, low resolution, cartoon", "images": [input_image], "similarityStrength": 0.7, # Range: 0.2 to 1.0 }, "imageGenerationConfig": { "numberOfImages": 1, "height": 512, "width": 512, "cfgScale": 8.0 } }) image_bytes = generate_image(model_id=model_id, body=body) image = Image.open(io.BytesIO(image_bytes)) image.show() except ClientError as err: message = err.response["Error"]["Message"] logger.error("A client error occurred: %s", message) print("A client error occured: " + format(message)) except ImageError as err: logger.error(err.message) print(err.message) else: print( f"Finished generating image with Amazon Nova Canvas model {model_id}.") if __name__ == "__main__": main() ``` ------ #### [ Image conditioning ] ``` # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """ Shows how to generate image conditioning from a source image with the Amazon Nova Canvas model (on demand). """ import base64 import io import json import logging import boto3 from PIL import Image from botocore.config import Config from botocore.exceptions import ClientError class ImageError(Exception): "Custom exception for errors returned by Amazon Nova Canvas" def __init__(self, message): self.message = message logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def generate_image(model_id, body): """ Generate an image using Amazon Nova Canvas model on demand. Args: model_id (str): The model ID to use. body (str) : The request body to use. Returns: image_bytes (bytes): The image generated by the model. """ logger.info( "Generating image with Amazon Nova Canvas model %s", model_id) bedrock = boto3.client( service_name='bedrock-runtime', config=Config(read_timeout=300) ) accept = "application/json" content_type = "application/json" response = bedrock.invoke_model( body=body, modelId=model_id, accept=accept, contentType=content_type ) response_body = json.loads(response.get("body").read()) base64_image = response_body.get("images")[0] base64_bytes = base64_image.encode('ascii') image_bytes = base64.b64decode(base64_bytes) finish_reason = response_body.get("error") if finish_reason is not None: raise ImageError(f"Image generation error. Error is {finish_reason}") logger.info( "Successfully generated image with Amazon Nova Canvas model %s", model_id) return image_bytes def main(): """ Entrypoint for Amazon Nova Canvas example. """ try: logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") model_id = 'amazon.nova-canvas-v1:0' # Read image from file and encode it as base64 string. with open("/path/to/image", "rb") as image_file: input_image = base64.b64encode(image_file.read()).decode('utf8') body = json.dumps({ "taskType": "TEXT_IMAGE", "textToImageParams": { "text": "A robot playing soccer, anime cartoon style", "negativeText": "bad quality, low res", "conditionImage": input_image, "controlMode": "CANNY_EDGE" }, "imageGenerationConfig": { "numberOfImages": 1, "height": 512, "width": 512, "cfgScale": 8.0 } }) image_bytes = generate_image(model_id=model_id, body=body) image = Image.open(io.BytesIO(image_bytes)) image.show() except ClientError as err: message = err.response["Error"]["Message"] logger.error("A client error occurred: %s", message) print("A client error occured: " + format(message)) except ImageError as err: logger.error(err.message) print(err.message) else: print( f"Finished generating image with Amazon Nova Canvas model {model_id}.") if __name__ == "__main__": main() ``` ------ #### [ Color guided content ] ``` # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """ Shows how to generate an image from a source image color palette with the Amazon Nova Canvas model (on demand). """ import base64 import io import json import logging import boto3 from PIL import Image from botocore.config import Config from botocore.exceptions import ClientError class ImageError(Exception): "Custom exception for errors returned by Amazon Nova Canvas" def __init__(self, message): self.message = message logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def generate_image(model_id, body): """ Generate an image using Amazon Nova Canvas model on demand. Args: model_id (str): The model ID to use. body (str) : The request body to use. Returns: image_bytes (bytes): The image generated by the model. """ logger.info( "Generating image with Amazon Nova Canvas model %s", model_id) bedrock = boto3.client( service_name='bedrock-runtime', config=Config(read_timeout=300) ) accept = "application/json" content_type = "application/json" response = bedrock.invoke_model( body=body, modelId=model_id, accept=accept, contentType=content_type ) response_body = json.loads(response.get("body").read()) base64_image = response_body.get("images")[0] base64_bytes = base64_image.encode('ascii') image_bytes = base64.b64decode(base64_bytes) finish_reason = response_body.get("error") if finish_reason is not None: raise ImageError(f"Image generation error. Error is {finish_reason}") logger.info( "Successfully generated image with Amazon Nova Canvas model %s", model_id) return image_bytes def main(): """ Entrypoint for Amazon Nova Canvas example. """ try: logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") model_id = 'amazon.nova-canvas-v1:0' # Read image from file and encode it as base64 string. with open("/path/to/image", "rb") as image_file: input_image = base64.b64encode(image_file.read()).decode('utf8') body = json.dumps({ "taskType": "COLOR_GUIDED_GENERATION", "colorGuidedGenerationParams": { "text": "digital painting of a girl, dreamy and ethereal, pink eyes, peaceful expression, ornate frilly dress, fantasy, intricate, elegant, rainbow bubbles, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration", "negativeText": "bad quality, low res", "referenceImage": input_image, "colors": ["#ff8080", "#ffb280", "#ffe680", "#ffe680"] }, "imageGenerationConfig": { "numberOfImages": 1, "height": 512, "width": 512, "cfgScale": 8.0 } }) image_bytes = generate_image(model_id=model_id, body=body) image = Image.open(io.BytesIO(image_bytes)) image.show() except ClientError as err: message = err.response["Error"]["Message"] logger.error("A client error occurred: %s", message) print("A client error occured: " + format(message)) except ImageError as err: logger.error(err.message) print(err.message) else: print( f"Finished generating image with Amazon Nova Canvas model {model_id}.") if __name__ == "__main__": main() ``` ------ #### [ Background removal ] ``` # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """ Shows how to generate an image with background removal with the Amazon Nova Canvas model (on demand). """ import base64 import io import json import logging import boto3 from PIL import Image from botocore.config import Config from botocore.exceptions import ClientError class ImageError(Exception): "Custom exception for errors returned by Amazon Nova Canvas" def __init__(self, message): self.message = message logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def generate_image(model_id, body): """ Generate an image using Amazon Nova Canvas model on demand. Args: model_id (str): The model ID to use. body (str) : The request body to use. Returns: image_bytes (bytes): The image generated by the model. """ logger.info( "Generating image with Amazon Nova Canvas model %s", model_id) bedrock = boto3.client( service_name='bedrock-runtime', config=Config(read_timeout=300) ) accept = "application/json" content_type = "application/json" response = bedrock.invoke_model( body=body, modelId=model_id, accept=accept, contentType=content_type ) response_body = json.loads(response.get("body").read()) base64_image = response_body.get("images")[0] base64_bytes = base64_image.encode('ascii') image_bytes = base64.b64decode(base64_bytes) finish_reason = response_body.get("error") if finish_reason is not None: raise ImageError(f"Image generation error. Error is {finish_reason}") logger.info( "Successfully generated image with Amazon Nova Canvas model %s", model_id) return image_bytes def main(): """ Entrypoint for Amazon Nova Canvas example. """ try: logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") model_id = 'amazon.nova-canvas-v1:0' # Read image from file and encode it as base64 string. with open("/path/to/image", "rb") as image_file: input_image = base64.b64encode(image_file.read()).decode('utf8') body = json.dumps({ "taskType": "BACKGROUND_REMOVAL", "backgroundRemovalParams": { "image": input_image, } }) image_bytes = generate_image(model_id=model_id, body=body) image = Image.open(io.BytesIO(image_bytes)) image.show() except ClientError as err: message = err.response["Error"]["Message"] logger.error("A client error occurred: %s", message) print("A client error occured: " + format(message)) except ImageError as err: logger.error(err.message) print(err.message) else: print( f"Finished generating image with Amazon Nova Canvas model {model_id}.") if __name__ == "__main__": main() ``` ------ # Generating videos with Amazon Nova Reel Generating videos With Amazon Nova Reel, you can generate realistic, studio-quality videos by using text- and image-based prompts. The model supports text-to-video generation and text-and-image to-video generation. Videos up to two minutes long are generated in six second increments at 1280x720 resolution and 24 frames per second. Amazon Nova Reel supports the following features: + Text-to-video (T2V) generation – Input a text prompt and generate a new video as output. The generated video captures the concepts described by the text prompt. + Text and Image-to-video (I2V) generation - Uses an input reference image to guide video generation. The model generates output video that uses the reference image as the starting key frame and generates a video that aligns with the text prompt. + Content provenance – Use publicly available tools such as [Content Credentials Verify](https://contentcredentials.org/verify) to check if an image was generated by Amazon Nova Reel 1.1. This should indicate the image was generated unless the metadata has been removed. | | Amazon Nova Reel | | --- |--- | | Model ID | amazon.nova-reel-v1:1 | | Input modalities | text, image | | Output Modalities | video | | Input Context Window Text | Text-to-video: 512 characters Multi-shot automated: 4000 characters Multi-shot manual: 512 characters per shot | | Supported Languages | English | | Regions | Amazon Nova Reel 1.0 is available in US East (N. Virginia), Europe (Ireland), and Asia Pacific (Tokyo). Amazon Nova Reel 1.1 is available only in US East (N. Virginia). | | Video Resolution | 1280x720 | | Frames per second | 24 | | Video Duration (seconds) | 6 second increments, up to two minutes | | Async Invoke Model API | [Yes](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_StartAsyncInvoke.html) | | Invoke Model API | No | | Fine-tuning | No | | Provisioned throughput | No | To generate videos longer than six seconds, you must use `amazon.nova-reel-v1:1` as the model ID. **Topics** + [ # Video generation access and usage ](video-gen-access.md) + [ # Error handling ](video-gen-errors.md) + [ # Single-shot video generation examples ](video-gen-code-examples.md) + [ # Multi-shot video generation code examples ](video-gen-code-examples2.md) + [ # Storyboarding videos with Amazon Nova Reel ](video-generation-storyboard.md) # Video generation access and usage Generating a video with Amazon Nova Reel is an asynchronous process that typically takes about 90 seconds for a 6 second video and approximately 14-17 minutes for a 2 minute video. After initiating the generation of a video, the video is written to an Amazon S3 bucket in your account. Because Amazon Bedrock writes a file to an Amazon S3 bucket on your behalf, the AWS role that you use needs permissions configured to allow the appropriate Amazon Bedrock and Amazon S3 actions and the `s3:PutObject` action. The minimum action permissions required to generate a video are: + `bedrock:InvokeModel` + `s3:PutObject` However, we recommend the following additional actions so you can track the status of video generation jobs: + `bedrock:GetAsyncInvoke` + `bedrock:ListAsyncInvokes` When video generation completes, the video and its constituent shots are stored in the Amazon S3 bucket you specified. Amazon Nova creates a folder for each invocation ID. This folder contains the manifest.json, output.mp4, and generation-status.json files that are created by the video generation request. **Topics** + [ ## Starting a video generation job ](#video-gen-start-a-job) + [ ## Video generation input parameters ](#video-gen-input-params) + [ ## Checking progress of video generation jobs ](#video-gen-check-progress) + [ ## Accessing the results of a video generation job ](#video-gen-read-results) ## Starting a video generation job To initiate the generation of a video, call `start_async_invoke()`. This creates a new invocation job. When the job completes, Amazon Nova automatically saves the generated video to an Amazon S3 bucket that you specify. `start_async_invoke()` takes the following arguments: + **modelId** (Required) – The model ID to use. For Amazon Nova Reel, this is "amazon.nova-reel-v1:1" + **modelInput** (Required) – Defines all of the video generation parameters specific to the Amazon Nova Reel model. For more information, see [Video generation input parameters](#video-gen-input-params). + **outputDataConfig** (Required) – Defines where the generated video should be saved. The value must have the following structure: ``` { "s3OutputDataConfig": { "s3Uri": string (S3 URL starting with "s3://") } } ``` ## Video generation input parameters Input parameters Refer to the following parameter descriptions for information about how to generate videos using Amazon Nova Reel. ------ #### [ Text-to-video generation ] The following structure defines an video generation job for Amazon Nova Reel: ``` { "taskType": "TEXT_VIDEO", "textToVideoParams": { "text": string, "images": ImageSource[] (list containing a single ImageSource) }, "videoGenerationConfig": { "durationSeconds": int, "fps": int, "dimension": string, "seed": int } } ``` These input parameters are necessary to create the video generation job: + **text** (Required) – A text prompt to generate the video. Must be 1-512 characters in length. + **images** (Optional) – A single JPEG or PNG image that is used as the starting keyframe of the output video. This input image is used along with the text prompt to generate the video. The image must be formatted as a base64 string or stored in an Amazon S3 bucket. Images can be in PNG or JPEG format and must be 8 bits per color channel (RGB). PNG images may contain an additional alpha channel, but that channel must not contain any transparent or translucent pixels. Currently, the model accepts only images of 1280 (width) x 720 (height). Images included through an Amazon S3 bucket can't exceed 25 MB. + **durationSeconds** (Required) - Duration of the output video. 6 is the only supported value currently. + **fps** (Required)- Frame rate of the output video. 24 is the only supported value currently. + **dimension** (Required) - Width and height of the output video. "1280x720" is the only supported value currently. + **seed** (Optional) – Determines the initial noise setting for the generation process. Changing the seed value while leaving all other parameters the same will produce a totally new video that still adheres to your prompt, dimensions, and other settings. It is common to experiment with a variety of seed values to find the perfect image. The seed value must be between 0-2,147,483,646 and the default value is 42. **imageSource schema** When you use an image as the input, use the following structure to include the image in your request: ``` { "format": "png" | "jpeg" "source": { "bytes": string (base64 encoded image) } } ``` + **format** (Required) - Must match the format of the input image. Either "png" or "jpeg". + **source** (Required) + **bytes** (Required) - The input image encoded as a base64 string. The image must have a resolution of 1280 x 720. ------ #### [ Automated long video generation ] You can generate videos up to two minutes long, in six second increments, with just a text prompt using the `MULTI_SHOT_AUTOMATED` task. You can provide a text prompt of up to 4000 characters but can't provide an input image. ``` { "taskType": "MULTI_SHOT_AUTOMATED", "multiShotAutomatedParams": { "text": string, }, "videoGenerationConfig": { "durationSeconds": int, "fps": int, "dimension": string, "seed": int } } ``` These input parameters are necessary to create the video generation job: + **text** (Required) – A text prompt to generate the video. Must be 1-4000 characters in length. + **durationSeconds** (Required) - Duration of the output video. A multiple of 6 between 12 and 120, inclusive. + **fps** (Required)- Frame rate of the output video. 24 is the only supported value currently. + **dimension** (Required) - Width and height of the output video. "1280x720" is the only supported value currently. + **seed** (Optional) – Determines the initial noise setting for the generation process. Changing the seed value while leaving all other parameters the same will produce a totally new image that still adheres to your prompt, dimensions, and other settings. It is common to experiment with a variety of seed values to find the perfect image. The seed value must be between 0-2,147,483,646 and the default value is 42. ------ #### [ Manual long video generation ] You can use the `MULTI_SHOT_MANUAL` task to generate videos up to two minutes long with multiple text prompts and input images. For each six second shot in the video, you can provide a text prompt with an optional input image. The duration of the video is determined based on the number shots that you specify. ``` model_input = { "taskType": "MULTI_SHOT_MANUAL", "multiShotManualParams": { "shots": [ { "text": "Information for shot 1" }, { "text": "Information for shot 2", "image": { "format": "png", # Must be "png" or "jpeg" "source": { "bytes": "" }, }, }, { "text": "Information for shot 3", "image": { "format": "png", # Must be "png" or "jpeg" "source": { "s3Location": { "uri": "", "bucketOwner": "" # Optional } } } }, ] }, "videoGenerationConfig": { "fps": int, "dimension": string, "seed": int } } ``` These input parameters are necessary to create the video generation job: + **shots** (Required) - Contains information about the text prompts and input images that are used for video generation. + **text** (Required) – A text prompt to generate the video. Must be 1-512 characters in length. + **image** (Optional) – Contains information about the input image that is used for this shot. The image can be provided as either a base64 string in the `bytes` field or as an Amazon S3 URI in the `s3Location` field. Images can be in PNG or JPEG format and must be 8 bits per color channel (RGB). PNG images may contain an additional alpha channel, but that channel must not contain any transparent or translucent pixels. Currently, the model accepts only images of 1280 (width) x 720 (height). Images included through an Amazon S3 bucket can't exceed 25 MB. + **fps** (Required)- Frame rate of the output video. 24 is the only supported value currently. + **dimension** (Required) - Width and height of the output video. "1280x720" is the only supported value currently. + **seed** (Optional) – Determines the initial noise setting for the generation process. Changing the seed value while leaving all other parameters the same will produce a totally new image that still adheres to your prompt, dimensions, and other settings. It is common to experiment with a variety of seed values to find the perfect image. The seed value must be between 0-2,147,483,646 and the default value is 42. ------ The video generation process will result in the following files being written to the Amazon S3 destination you specify: + **manifest.json** - A file written at the start of the job, containing the request ID. + **video-generation-status.json** - This file is written whether the job succeeds of fails. When a job fails, it will contain detailed information explaining exactly which part of the job failed and what action to take to fix the error. + **output.mp4** - The complete multi-shot video. Written only if the job succeeds. + **shot\$1N.mp4** - Each individual shot is also provided as its own video. The file name follows the format "shot\$10001.mp4", "shot\$10002.mp4", and so on. These files are written only if the whole job succeeds. ## Checking progress of video generation jobs Checking progress There are two ways to check on the progress of a video generation job. If you have a reference to the invocation ARN that was returned when starting the invocation, you can use the `get_async_invoke()` method of the Amazon Bedrock Runtime. ``` response = bedrock_runtime.get_async_invoke( invocationArn="arn:AWS:bedrock:us-east-1:account-id:async-invoke/invocation-id" ) status = response["status"] print(f"Status: {status}") ``` The status of a job will be "Completed", "InProgress", or "Failed". For more details on using the `get_async_invoke()` method, see the Async Invoke API documentation. If you do not have a reference to the invocation ARN, or if you want to check the status for multiple jobs at once, you can use the `list_async_invokes()` method of the Amazon Bedrock Runtime. ``` invocations_details = bedrock_runtime.list_async_invokes( maxResults=10, # (Optional) statusEquals="InProgress", # (Optional) Can be "Completed", "InProgress", or "Failed". Omit this argument to list all jobs, regardless of status. # Note: There are other supported arguments not demonstrated here. ) print(json.dumps(invocations_details, indent=2, default=str)) ``` For more details on using the `list_async_invokes()` method, see the Async Invoke API documentation. ## Accessing the results of a video generation job Accessing results After a video generation job succeeds or fails, a JSON file is added to your Amazon S3 bucket. This file contains metadata about the shots that were created for the video. The file is named `video-generation-status.json`. For a successful video generation request, the file contains the location of each individual shot that comprises the full video. For a failed request, the file contains the failure message and additional details about why the shot failed. The schema of this JSON file is provided below. ``` { "schemaVersion": string, "shots": [{ "status": enum, // where success is generation + upload "location": string, "failureType": enum, "failureMessage": string, }, ... ], "fullVideo": { "status": enum, // where success is generation + upload "location": string, "failureType": enum, "failureMessage": string, } } ``` + **schemaVersion** - The version of the JSON schema. + **shots** - Provides information about each shot in the video. + **status** - The completion state (SUCCESS or FAILURE) of the shot. + **location** - The file name and Amazon S3 location where the shot is stored. The location will be available only when all shots are successfully generated and the complete video is uploaded to its Amazon S3 location. + **failureType** - Provides the reason for failure. + **failureMessage** - Provides more information about the failure reason. + **fullVideo** - Provides information about the full video. + **status** - The completion state (SUCCESS or FAILURE) of the full video. + **location** - The file name and Amazon S3 location where the full video is stored. + **failureType** - Provides the reason for failure. + **failureMessage** - Provides more information about the failure reason. Possible failure reasons and messages are + INTERNAL\$1SERVER\$1EXCEPTION - "Something went wrong on the server side." + RAI\$1VIOLATION\$1OUTPUT\$1VIDEO\$1DEFLECTION - "The generated content has been blocked by our content filters." + RATE\$1LIMIT\$1EXCEEDED - "Service capacity limit has been reached. Please try again later." + ABORTED - "Request has been aborted." # Error handling There are three primary types of errors that you want to handle in your application code. These are input validation errors, AWS Responsible AI (RAI) input deflection errors, and RAI output deflection errors. These errors are unique to Amazon Nova Reel. Input validation errors occur if your request is malformed or if you use an unsupported value for an input parameter—for example, a `duration` value that does not match one of the supported values or an input `image` that is not exactly 1280x720 resolution. All input validation errors are expressed as a **ValidationException** which contains a message string describing the cause of the problem. This exception will be raised when calling the `start_async_invoke()` method of the Amazon Bedrock Runtime. RAI input deflection errors occur when the input text value or input image are determined to violate [AWS core dimensions of responsible AI](https://aws.amazon.com/ai/responsible-ai/). These errors are expressed as a **ValidationException** with one of the following messages: + **Input text** validation message: "This request has been blocked by our content filters. Please adjust your text prompt to submit a new request." + **Input image** validation message: "This request has been blocked by our content filters. Please adjust your input image to submit a new request." RAI output deflection errors occur when a video is generated but it is determined to be misaligned with [our core dimensions of responsible AI](https://aws.amazon.com/ai/responsible-ai/). When this occurs, an exception is not used. Instead, the job is marked as "Failed" and the file is never written to Amazon S3. When querying the status of the job (for example, using `get_invoke()`), the response will have a `status` field value of "Failed" and a `failureMessage` field value of "The generated video has been blocked by our content filters." # Single-shot video generation examples The following examples provide sample code for various single-shot (6 seconds) video generation tasks. ------ #### [ Text to video ] ``` # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 import json import boto3 # Create the Bedrock Runtime client. bedrock_runtime = boto3.client("bedrock-runtime") model_input = { "taskType": "TEXT_VIDEO", "textToVideoParams": { "text": "Closeup of a large seashell in the sand, gentle waves flow around the shell. Camera zoom in." }, "videoGenerationConfig": { "durationSeconds": 6, "fps": 24, "dimension": "1280x720", "seed": 0, # Change the seed to get a different result }, } try: # Start the asynchronous video generation job. invocation = bedrock_runtime.start_async_invoke( modelId="amazon.nova-reel-v1:1", modelInput=model_input, outputDataConfig={ "s3OutputDataConfig": { "s3Uri": "s3://my-nova-videos" } } ) # Print the response JSON. print("Response:") print(json.dumps(invocation, indent=2, default=str)) except Exception as e: # Implement error handling here. message = e.response["Error"]["Message"] print(f"Error: {message}") ``` ------ #### [ Image to video ] ``` # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 import json import boto3 import base64 # Create the Bedrock Runtime client. bedrock_runtime = boto3.client("bedrock-runtime") # Load the input image as a Base64 string. Note, the image resolution # must be exactly 1280x720. input_image_path = "images/first-frame.png" with open(input_image_path, "rb") as f: input_image_bytes = f.read() input_image_base64 = base64.b64encode(input_image_bytes).decode("utf-8") model_input = { "taskType": "TEXT_VIDEO", "textToVideoParams": { "text": "Dolly forward over a gentle river", "images": [ { "format": "png", "source": { "bytes": input_image_base64 } } ] }, "videoGenerationConfig": { "durationSeconds": 6, "fps": 24, "dimension": "1280x720", "seed": 0 }, } # Start the asynchronous video generation job. invocation = bedrock_runtime.start_async_invoke( modelId="amazon.nova-reel-v1:1", modelInput=model_input, outputDataConfig={ "s3OutputDataConfig": { "s3Uri": "s3://my-nova-videos" } }, ) # Print the response JSON. print("Response:") print(json.dumps(invocation, indent=2, default=str)) ``` ------ #### [ Query job status ] ``` import json import boto3 # Create the Bedrock Runtime client. bedrock_runtime = boto3.client("bedrock-runtime") invocation = bedrock_runtime.get_async_invoke( invocationArn="arn:AWS:bedrock:us-east-1:account-id:async-invoke/invocation-id" ) # Print the JSON response print(json.dumps(invocation, indent=2, default=str)) invocation_arn = invocation["invocationArn"] status = invocation["status"] if (status == "Completed"): bucket_uri = invocation["outputDataConfig"]["s3OutputDataConfig"]["s3Uri"] video_uri = bucket_uri + "/output.mp4" print(f"Video is available at: {video_uri}") elif (status == "InProgress"): start_time = invocation["submitTime"] print(f"Job {invocation_arn} is in progress. Started at: {start_time}") elif (status == "Failed"): failure_message = invocation["failureMessage"] print(f"Job {invocation_arn} failed. Failure message: {failure_message}") ``` ------ #### [ Listing jobs ] ``` import json import boto3 # Create the Bedrock Runtime client. bedrock_runtime = boto3.client("bedrock-runtime") # List the 10 most recently completed jobs. completed_jobs = bedrock_runtime.list_async_invokes( maxResults=10, # (Optional) statusEquals="Completed", # (Optional) Can be "Completed", "InProgress", or "Failed". # Omit this argument to list all jobs, regardless of status. # Note: There are other supported arguments not demonstrated here. ) # Print the JSON response print(json.dumps(completed_jobs, indent=2, default=str)) # Loop through the completed jobs and print their invocation ARNs. for job in completed_jobs["asyncInvokeSummaries"]: print(job["invocationArn"]) ``` ------ #### [ Text to video using REST API ] ``` # Invoke the Amazon Nova Reel model to create a video and monitor the status # of the async job. # tested with Python 3.12 import json import time import uuid import boto3 import requests as req import botocore.session from botocore.auth import SigV4Auth from typing import Dict, List, Tuple from botocore.awsrequest import AWSRequest ## ------ Initialize constants to invoke the general async function to call REST APIs for Bedrock ------------ SERVICE_NAME: str = 'bedrock' MAX_TIME: int = 3600 BUCKET_FOR_VIDEO_CONTENT: str = "s3://your-bucket-name-here" # Region and model id to use REGION: str = 'us-east-1' MODEL_ID: str = 'amazon.nova-reel-v1:1' ## ------------------------------------------------------------------------------------------------------------ def get_inference(model_id: str, region: str, payload: List) -> Tuple: print(f"making an inference request to {model_id}, payload={payload}") try: ## Initialize the runtime rest API to be called for the endpoint endpoint: str = f"https://{SERVICE_NAME}-runtime.{region}.amazonaws.com/async-invoke" print(endpoint) #endpoint = f"https://{SERVICE_NAME}-runtime.{region}.amazonaws.com/model/{model_id}/async-invoke" # Converting the payload dictionary into a JSON-formatted string to be sent in the HTTP request request_body = json.dumps(payload[1]) print(json.dumps(payload[1], indent=2)) # Creating an AWSRequest object for a POST request with the service specified endpoint, JSON request body, and HTTP headers request = AWSRequest(method='POST', url=endpoint, data=request_body, headers={'content-type': 'application/json'}) # Initializing a botocore session session = botocore.session.Session() # Adding a SigV4 authentication information to the AWSRequest object, signing the request sigv4 = SigV4Auth(session.get_credentials(), SERVICE_NAME, region) sigv4.add_auth(request) # Prepare the request by formatting it correctly prepped = request.prepare() # Send the HTTP POST request to the prepared URL with the specified headers and JSON-formatted request body, storing the response response = req.post(prepped.url, headers=prepped.headers, data=request_body) if response.status_code == 200: return (payload[0], response.json()) else: print(f"Error: Received status code {response.status_code}, Response: {response.text}") return None except Exception as e: print(f"Exception occurred: {e}") return None def print_async_job_status(arn, region=REGION): # Create the Bedrock Runtime client. bedrock_runtime = boto3.client("bedrock-runtime", region_name=region) invocation = bedrock_runtime.get_async_invoke( invocationArn=arn ) # Print the JSON response print(json.dumps(invocation, indent=2, default=str)) invocation_arn = invocation["invocationArn"] status = invocation["status"] if (status == "Completed"): bucket_uri = invocation["outputDataConfig"]["s3OutputDataConfig"]["s3Uri"] video_uri = bucket_uri + "/output.mp4" print(f"Video is available at: {video_uri}") elif (status == "InProgress"): start_time = invocation["submitTime"] print(f"Job {invocation_arn} is in progress. Started at: {start_time}") elif (status == "Failed"): failure_message = invocation["failureMessage"] print(f"Job {invocation_arn} failed. Failure message: {failure_message}") return status # Function to create the payload def create_payload(prompt: str, model_id: str, bucket: str) -> Dict: payload = { "modelId": model_id, "modelInput": { "taskType": "TEXT_VIDEO", "textToVideoParams": { "text": prompt }, "videoGenerationConfig": { "durationSeconds": 6, "fps": 24, "dimension": "1280x720", "seed": 0 } }, "outputDataConfig": { "s3OutputDataConfig": { "s3Uri": bucket } }, "clientRequestToken": str(uuid.uuid4()) } return payload ## Initialize the number of prompts you want to invoke on the bedrock specific model prompts = ["galaxies receding", "event horizon of a black hole"] payloads: List = [(i, create_payload(p, MODEL_ID, BUCKET_FOR_VIDEO_CONTENT)) for i, p in enumerate(prompts)] # Start timing before sending the request print(f"going to make {len(prompts)} requests") start_time = time.perf_counter() responses = [get_inference(MODEL_ID, REGION, prompt) for prompt in payloads] # Calculate the elapsed time elapsed_time = time.perf_counter() - start_time print(f"Total time taken for {len(prompts)} calls made: {elapsed_time:.2f} seconds") invocation_arns = [] for r in responses: print(f"response={r}") invocation_arns.append(r[1]['invocationArn']) jobs_total = len(invocation_arns) jobs_completed = 0 st = time.time() while True: for arn in invocation_arns: status = print_async_job_status(arn) print(f"arn={arn}, status={status}") if status == "Completed": jobs_completed += 1 if jobs_completed == jobs_total: print(f"all jobs completed, exiting") break if time.time() - st > MAX_TIME: print(f"{MAX_TIME}s elapsed but seems like all jobs are still not completed, exiting") break time.sleep(60) print("all done") ``` ------ # Multi-shot video generation code examples The following examples provide sample code for various multi-shot (longer than 6 seconds) video generation tasks. ------ #### [ Automated video generation ] In this example, all shots in the video are generated from a single prompt and no input image is provided. ``` import json import os import boto3 from dotenv import load_dotenv # Create the Bedrock Runtime client. bedrock_runtime = boto3.client(service_name="bedrock-runtime", region_name="us-east-1") # Configure Nova Reel model inputs. model_input = { "taskType": "MULTI_SHOT_AUTOMATED", "multiShotAutomatedParams": { "text": "Cinematic documentary showcasing the stunning beauty of the natural world. Drone footage flying over fantastical and varied natural wonders." }, "videoGenerationConfig": { "seed": 1234, "durationSeconds": 18, # Must be a multiple of 6 in range [12, 120] "fps": 24, # Must be 24 "dimension": "1280x720", # Must be "1280x720" }, } try: # Start the asynchronous video generation job. invocation = bedrock_runtime.start_async_invoke( modelId="amazon.nova-reel-v1:1", modelInput=model_input, outputDataConfig={"s3OutputDataConfig": {"s3Uri": "s3://your-s3-bucket"}}, ) # Print the response JSON. print(json.dumps(invocation, indent=2, default=str)) except Exception as err: print("Exception:") if hasattr(err, "response"): # Pretty print the response JSON. print(json.dumps(err.response, indent=2, default=str)) else: print(err) ``` ------ #### [ Manual video generation - Amazon S3 input image ] In this example, a two shot video is generated. Each shot is generated with a separate prompt and input image that is provided in an Amazon S3 location. ``` import json import os import boto3 from dotenv import load_dotenv # === Helper Function === def image_to_base64(image_path: str): """ Convert an image file to a base64 encoded string. """ import base64 with open(image_path, "rb") as image_file: encoded_string = base64.b64encode(image_file.read()) return encoded_string.decode("utf-8") # === Main Code === # Create the Bedrock Runtime client. bedrock_runtime = boto3.client(service_name="bedrock-runtime", region_name="us-east-1") # Configure Nova Reel model inputs. This example includes three shots, two of # which include images to use as starting frames. These images are stored in S3. model_input = { "taskType": "MULTI_SHOT_MANUAL", "multiShotManualParams": { "shots": [ {"text": "aerial view of a city with tall glass and metal skyscrapers"}, { "text": "closeup of a vehicle wheel in motion as the pavement speeds by with motion blur", "image": { "format": "png", # Must be "png" or "jpeg" "source": { "s3Location": { "uri": "s3://your-s3-bucket/images/SUV-wheel-closeup.png" } }, }, }, { "text": "tracking shot, the vehicle drives through the city, trees and buildings line the street", "image": { "format": "png", # Must be "png" or "jpeg" "source": { "s3Location": { "uri": "s3://your-s3-bucket/images/SUV-downtown-back.png" } }, }, }, ] }, "videoGenerationConfig": { "seed": 1234, "fps": 24, # Must be 24 "dimension": "1280x720", # Must be "1280x720" }, } try: # Start the asynchronous video generation job. invocation = bedrock_runtime.start_async_invoke( modelId="amazon.nova-reel-v1:1", modelInput=model_input, outputDataConfig={"s3OutputDataConfig": {"s3Uri": "s3://your-s3-bucket"}}, ) # Print the response JSON. print(json.dumps(invocation, indent=2, default=str)) except Exception as err: print("Exception:") if hasattr(err, "response"): # Pretty print the response JSON. print(json.dumps(err.response, indent=2, default=str)) else: print(err) ``` ------ #### [ Manual video generation - base64 input image ] In this example, a three shot video is generated. The first shot is generated with just a prompt, and the next two shot are generated with a new prompt and input image each. ``` import json import os import boto3 from dotenv import load_dotenv # === Helper Function === def image_to_base64(image_path: str): """ Convert an image file to a base64 encoded string. """ import base64 with open(image_path, "rb") as image_file: encoded_string = base64.b64encode(image_file.read()) return encoded_string.decode("utf-8") # === Main Code === # Create the Bedrock Runtime client. bedrock_runtime = boto3.client(service_name="bedrock-runtime", region_name="us-east-1") # Configure Nova Reel model inputs. This example includes three shots, two of # which include images to use as starting frames. model_input = { "taskType": "MULTI_SHOT_MANUAL", "multiShotManualParams": { "shots": [ { "text": "Drone footage of a Pacific Northwest forest with a meandering stream seen from a high altitude, top-down view" }, { "text": "camera arcs slowly around two SUV vehicles in a forest setting with a stream in the background", "image": { "format": "png", # Must be "png" or "jpeg" "source": {"bytes": image_to_base64("images/SUV-roadside.png")}, }, }, { "text": "tracking shot, a SUV vehicle drives toward the camera through a forest roadway, the SUV's ring-shaped headlights glow white", "image": { "format": "png", # Must be "png" or "jpeg" "source": {"bytes": image_to_base64("images/SUV-forest-front.png")}, }, }, ] }, "videoGenerationConfig": { "seed": 1234, "fps": 24, # Must be 24 "dimension": "1280x720", # Must be "1280x720" }, } try: # Start the asynchronous video generation job. invocation = bedrock_runtime.start_async_invoke( modelId="amazon.nova-reel-v1:1", modelInput=model_input, outputDataConfig={"s3OutputDataConfig": {"s3Uri": "s3://your-s3-bucket"}}, ) # Print the response JSON. print(json.dumps(invocation, indent=2, default=str)) except Exception as err: print("Exception:") if hasattr(err, "response"): # Pretty print the response JSON. print(json.dumps(err.response, indent=2, default=str)) else: print(err) ``` ------ # Storyboarding videos with Amazon Nova Reel Storyboarding videos Amazon Nova Reel includes the ability to create videos in six second increments that are up to two minutes long. From the Amazon Bedrock playground, you can provide a single prompt that will generate a video of a specified length. However, if you want more control over subjects and direction of the video, you can use the storyboard. The storyboard allows you provide multiple input images and prompts to better guide the generated video towards your desired outcome. For each six second interval, you have the option of providing an input image, a prompt, or both. These inputs are used to generate the video until a different input image or prompt are encountered. This way, if you want your video to cut to a different camera angle or focus on a different subject, you can prompt the model when it's time to do so. To create a video with the storyboard, complete the following steps: 1. Open the Amazon Bedrock console at [https://console.aws.amazon.com/bedrock/](https://console.aws.amazon.com/bedrock/). 1. From the left navigation pane, choose **Image / Video** under **Playgrounds**. 1. Choose **Select model** and select **Amazon** and **Amazon Nova Reel v1.1** as the provider and model. Choose **Apply**. 1. In the left panel, move the slider so that the value of **Duration (seconds)** is greater than 6. 1. Choose the storyboard icon ![\[Striped icon representing a list or menu with multiple items.\]](http://docs.aws.amazon.com/nova/latest/userguide/images/storyboardIcon.png) to enter the Storyboard. 1. In the Storyboard, add or remove shots to reach the desired length of generated video. 1. For each shot, you can add an image, text prompt, or both. You must add at least a text prompt to the first shot of the storyboard. 1. After you have specified all of the shot information, choose **Run**. Video generation will run asynchronously until completion. When finished, you will be notified and the video will be saved in an Amazon S3 bucket.