EMR / Client / start_session

start_session¶

EMR.Client.start_session(**kwargs)¶

Creates and starts a new Spark Connect session on the specified cluster. The cluster must be in the RUNNING or WAITING state and have sessions enabled. This operation is supported in Amazon EMR Spark 8.0.0 and later.

Request Syntax

response = client.start_session(
    Name='string',
    ClusterId='string',
    ExecutionRoleArn='string',
    EngineConfigurations=[
        {
            'Classification': 'string',
            'Configurations': {'... recursive ...'},
            'Properties': {
                'string': 'string'
            }
        },
    ],
    MonitoringConfiguration={
        'CloudWatchLoggingConfiguration': {
            'Enabled': True|False,
            'LogGroup': 'string',
            'LogStreamNamePrefix': 'string',
            'EncryptionKeyArn': 'string',
            'LogTypes': {
                'string': [
                    'string',
                ]
            }
        },
        'ManagedLoggingConfiguration': {
            'Enabled': True|False,
            'EncryptionKeyArn': 'string'
        },
        'S3LoggingConfiguration': {
            'Enabled': True|False,
            'LogUri': 'string',
            'EncryptionKeyArn': 'string',
            'LogTypes': {
                'string': [
                    'string',
                ]
            }
        }
    },
    SessionIdleTimeoutInMinutes=123,
    ClientRequestToken='string',
    Tags=[
        {
            'Key': 'string',
            'Value': 'string'
        },
    ]
)

Parameters:

Name (string) – An optional name for the session.
ClusterId (string) –
[REQUIRED]

The ID of the cluster on which to start the session.
ExecutionRoleArn (string) – The execution role ARN for the session. Amazon EMR uses this role to access Amazon Web Services resources on your behalf during session execution.
EngineConfigurations (list) –
The configuration overrides for the session. Only runtime configuration overrides are supported.
- (dict) –
  
  Note
  Amazon EMR releases 4.x or later.
  
  An optional configuration specification to be used when provisioning cluster instances, which can include configurations for applications and software bundled with Amazon EMR. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file. For more information, see Configuring Applications.
  - Classification (string) –
    
    The classification within a configuration.
  - Configurations (list) –
    
    A list of additional configurations to apply within a configuration object.
  - Properties (dict) –
    
    A set of properties specified within a configuration classification.
    - (string) –
      - (string) –
MonitoringConfiguration (dict) –
The monitoring configuration that controls where session logs are published, such as Amazon S3, CloudWatch, or managed logging.
- CloudWatchLoggingConfiguration (dict) –
  
  The CloudWatch Logs configuration for the session.
  - Enabled (boolean) –
    
    Whether CloudWatch Logs is enabled for the session.
  - LogGroup (string) –
    
    The name of the log group where session logs are published.
  - LogStreamNamePrefix (string) –
    
    The prefix applied to the log stream name where session logs are published.
  - EncryptionKeyArn (string) –
    
    The Amazon Resource Name (ARN) of the KMS key used to encrypt the logs published to CloudWatch Logs.
  - LogTypes (dict) –
    
    A map of log component names (for example, SPARK_DRIVER, SPARK_EXECUTOR) to the list of log types to publish for that component (for example, stdout, stderr).
    - (string) –
      - (list) –
        
        (string) –
- ManagedLoggingConfiguration (dict) –
  
  The Amazon EMR-managed logging configuration for the session.
  - Enabled (boolean) –
    
    Whether Amazon EMR-managed logging is enabled for the session.
  - EncryptionKeyArn (string) –
    
    The Amazon Resource Name (ARN) of the KMS key used to encrypt the managed logs.
- S3LoggingConfiguration (dict) –
  
  The Amazon S3 logging configuration for the session.
  - Enabled (boolean) –
    
    Whether Amazon S3 logging is enabled for the session.
  - LogUri (string) –
    
    The Amazon S3 destination URI where session logs are published.
  - EncryptionKeyArn (string) –
    
    The Amazon Resource Name (ARN) of the KMS key used to encrypt logs published to Amazon S3.
  - LogTypes (dict) –
    
    A map of log component names (for example, SPARK_DRIVER, SPARK_EXECUTOR) to the list of log types to publish for that component (for example, stdout, stderr).
    - (string) –
      - (list) –
        
        (string) –
SessionIdleTimeoutInMinutes (integer) – The idle timeout, in minutes. If the session is idle for this duration, Amazon EMR EC2 automatically terminates it.
ClientRequestToken (string) – A unique, case-sensitive identifier that you provide to ensure the idempotency of the request. If you retry a request that completed successfully using the same client request token, the service returns the original response without performing the operation again.
Tags (list) –
The tags to assign to the session.
- (dict) –
  
  A key-value pair containing user-defined metadata that you can associate with an Amazon EMR resource. Tags make it easier to associate clusters in various ways, such as grouping clusters to track your Amazon EMR resource allocation costs. For more information, see Tag Clusters.
  - Key (string) –
    
    A user-defined key, which is the minimum required information for a valid tag. For more information, see Tag.
  - Value (string) –
    
    A user-defined value, which is optional in a tag. For more information, see Tag Clusters.

Return type:

dict

Returns:

Response Syntax

{
    'Id': 'string',
    'ClusterId': 'string',
    'Arn': 'string',
    'AccountId': 'string',
    'State': 'SUBMITTED'|'STARTING'|'STARTED'|'IDLE'|'BUSY'|'TERMINATING'|'TERMINATED'|'FAILED'
}

Response Structure

(dict) –

Output of the StartSession operation.
- Id (string) –
  
  The output contains the ID of the session.
- ClusterId (string) –
  
  The ID of the cluster that the session was started on.
- Arn (string) –
  
  The output contains the ARN of the session.
- AccountId (string) –
  
  The Amazon Web Services account ID that owns the session.
- State (string) –
  
  The state of the session at the time the request returned. When a session is first created, it enters the SUBMITTED state.

Exceptions

EMR.Client.exceptions.InternalServerException
EMR.Client.exceptions.InvalidRequestException