{ "cells": [ { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "# Fine-tune Amazon Titan Text Express provided by Amazon Bedrock\n", "\n", "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio. Also use ml.c5.2xlarge due to memory resources required*\n", "\n", "In this notebook, we will fine-tune [Amazon Titan Text Lite](#https://docs.aws.amazon.com/bedrock/latest/userguide/titan-text-models.html) model provided by Amazon Bedrock for summarization use case.\n", "You can choose from list of base models or fine-tune one of your previously fine tuned model.\n", "\n", "## Prerequisites\n", "\n", " - Make sure you have executed `00_setup.ipynb` notebook.\n", " - Make sure you are using the same kernel and instance as `00_setup.ipynb` notebook.\n", "\n", "
\n", "Warning: This notebook will create provisioned throughput for testing the fine-tuned model. Therefore, please make sure to delete the provisioned throughput as mentioned in the last section of the notebook, otherwise you will be charged for it, even if you are not using it.\n", "
" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", "\u001b[0m" ] } ], "source": [ "!pip install -qU bert_score" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# restart kernel for packages to take effect\n", "from IPython.core.display import HTML\n", "HTML(\"\")" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "tags": [] }, "outputs": [], "source": [ "## Fetching varialbes from `00_setup.ipynb` notebook. \n", "%store -r role_arn\n", "%store -r s3_train_uri\n", "%store -r s3_validation_uri\n", "%store -r s3_test_uri\n", "%store -r bucket_name" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "'arn:aws:iam::563158133511:role/BedrockRole-6d86f57b-028f-40b0-83df-b3d56e3a2bc4'\n", "'s3://bedrock-customization-us-west-2-563158133511/fine-tuning-datasets/train/train-cnn-5K.jsonl'\n", "'s3://bedrock-customization-us-west-2-563158133511/fine-tuning-datasets/validation/validation-cnn-1K.jsonl'\n", "'s3://bedrock-customization-us-west-2-563158133511/fine-tuning-datasets/test/test-cnn-10.jsonl'\n", "'bedrock-customization-us-west-2-563158133511'\n" ] } ], "source": [ "import pprint\n", "pprint.pp(role_arn)\n", "pprint.pp(s3_train_uri)\n", "pprint.pp(s3_validation_uri)\n", "pprint.pp(s3_test_uri)\n", "pprint.pp(bucket_name)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "tags": [] }, "outputs": [], "source": [ "import warnings\n", "warnings.filterwarnings('ignore')\n", "import json\n", "import os\n", "import sys\n", "import boto3\n", "import time" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "tags": [] }, "outputs": [], "source": [ "session = boto3.session.Session()\n", "region = session.region_name\n", "sts_client = boto3.client('sts')\n", "s3_client = boto3.client('s3')\n", "aws_account_id = sts_client.get_caller_identity()[\"Account\"]\n", "bedrock = boto3.client(service_name=\"bedrock\")\n", "bedrock_runtime = boto3.client(service_name=\"bedrock-runtime\")" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "tags": [] }, "outputs": [], "source": [ "test_file_name = \"test-cnn-10.jsonl\"\n", "data_folder = \"fine-tuning-datasets\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Select the model you would like to fine-tune\n", "You will have to provide the `base_model_id` for the model you are planning to fine-tune. You can get that using `list_foundation_models` API as follows: \n", "```\n", "for model in bedrock.list_foundation_models(\n", " byCustomizationType=\"FINE_TUNING\")[\"modelSummaries\"]:\n", " for key, value in model.items():\n", " print(key, \":\", value)\n", " print(\"-----\\n\")\n", "```" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "tags": [] }, "outputs": [], "source": [ "base_model_id = \"amazon.titan-text-lite-v1:0:4k\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next you will need to provide the `customization_job_name`, `custom_model_name` and `customization_role` which will be used to create the fine-tuning job. " ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "tags": [] }, "outputs": [], "source": [ "from datetime import datetime\n", "ts = datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\")\n", "\n", "customization_job_name = f\"model-finetune-job-{ts}\"\n", "custom_model_name = f\"finetuned-model-{ts}\"\n", "customization_role = role_arn" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create fine-tuning job\n", "
\n", "Note: Fine-tuning job will take around 40mins to complete.
\n", "\n", "Amazon Titan text model customization hyperparameters: \n", "- `epochs`: The number of iterations through the entire training dataset and can take up any integer values in the range of 1-10, with a default value of 5.\n", "- `batchSize`: The number of samples processed before updating model parameters and can take up any integer values in the range of 1-64, with a default value of 1.\n", "- `learningRate`:\tThe rate at which model parameters are updated after each batch\twhich can take up a float value betweek 0.0-1.0 with a default value set to\t1.00E-5.\n", "- `learningRateWarmupSteps`: The number of iterations over which the learning rate is gradually increased to the specified rate and can take any integer value between 0-250 with a default value of 5.\n", "\n", "For guidelines on setting hyper-parameters refer to the guidelines provided [here](#https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-guidelines.html)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Select the customization type from \"FINE_TUNING\" or \"CONTINUED_PRE_TRAINING\". \n", "customization_type = \"FINE_TUNING\"" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "tags": [] }, "outputs": [ { "ename": "AccessDeniedException", "evalue": "An error occurred (AccessDeniedException) when calling the CreateModelCustomizationJob operation: User: arn:aws:sts::563158133511:assumed-role/bedrock-workshop-studio-SageMakerExecutionRole-wPMIWPjX86hV/SageMaker is not authorized to perform: bedrock:CreateModelCustomizationJob on resource: arn:aws:bedrock:us-west-2:563158133511:model-customization-job/* with an explicit deny in an identity-based policy", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mAccessDeniedException\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[13], line 24\u001b[0m\n\u001b[1;32m 20\u001b[0m output_data_config \u001b[38;5;241m=\u001b[39m {\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms3Uri\u001b[39m\u001b[38;5;124m\"\u001b[39m: s3_bucket_config}\n\u001b[1;32m 23\u001b[0m \u001b[38;5;66;03m# Create the customization job\u001b[39;00m\n\u001b[0;32m---> 24\u001b[0m training_job_response \u001b[38;5;241m=\u001b[39m \u001b[43mbedrock\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcreate_model_customization_job\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 25\u001b[0m \u001b[43m \u001b[49m\u001b[43mcustomizationType\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcustomization_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 26\u001b[0m \u001b[43m \u001b[49m\u001b[43mjobName\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcustomization_job_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 27\u001b[0m \u001b[43m \u001b[49m\u001b[43mcustomModelName\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcustom_model_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 28\u001b[0m \u001b[43m \u001b[49m\u001b[43mroleArn\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcustomization_role\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 29\u001b[0m \u001b[43m \u001b[49m\u001b[43mbaseModelIdentifier\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mbase_model_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 30\u001b[0m \u001b[43m \u001b[49m\u001b[43mhyperParameters\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mhyper_parameters\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 31\u001b[0m \u001b[43m \u001b[49m\u001b[43mtrainingDataConfig\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtraining_data_config\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 32\u001b[0m \u001b[43m \u001b[49m\u001b[43mvalidationDataConfig\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mvalidation_data_config\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 33\u001b[0m \u001b[43m \u001b[49m\u001b[43moutputDataConfig\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_data_config\u001b[49m\n\u001b[1;32m 34\u001b[0m \u001b[43m)\u001b[49m\n\u001b[1;32m 35\u001b[0m training_job_response\n", "File \u001b[0;32m/opt/conda/lib/python3.10/site-packages/botocore/client.py:565\u001b[0m, in \u001b[0;36mClientCreator._create_api_method.._api_call\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 561\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(\n\u001b[1;32m 562\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpy_operation_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m() only accepts keyword arguments.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 563\u001b[0m )\n\u001b[1;32m 564\u001b[0m \u001b[38;5;66;03m# The \"self\" in this scope is referring to the BaseClient.\u001b[39;00m\n\u001b[0;32m--> 565\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_make_api_call\u001b[49m\u001b[43m(\u001b[49m\u001b[43moperation_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m/opt/conda/lib/python3.10/site-packages/botocore/client.py:1021\u001b[0m, in \u001b[0;36mBaseClient._make_api_call\u001b[0;34m(self, operation_name, api_params)\u001b[0m\n\u001b[1;32m 1017\u001b[0m error_code \u001b[38;5;241m=\u001b[39m error_info\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mQueryErrorCode\u001b[39m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;129;01mor\u001b[39;00m error_info\u001b[38;5;241m.\u001b[39mget(\n\u001b[1;32m 1018\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCode\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1019\u001b[0m )\n\u001b[1;32m 1020\u001b[0m error_class \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mexceptions\u001b[38;5;241m.\u001b[39mfrom_code(error_code)\n\u001b[0;32m-> 1021\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m error_class(parsed_response, operation_name)\n\u001b[1;32m 1022\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 1023\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m parsed_response\n", "\u001b[0;31mAccessDeniedException\u001b[0m: An error occurred (AccessDeniedException) when calling the CreateModelCustomizationJob operation: User: arn:aws:sts::563158133511:assumed-role/bedrock-workshop-studio-SageMakerExecutionRole-wPMIWPjX86hV/SageMaker is not authorized to perform: bedrock:CreateModelCustomizationJob on resource: arn:aws:bedrock:us-west-2:563158133511:model-customization-job/* with an explicit deny in an identity-based policy" ] } ], "source": [ "# Define the hyperparameters for fine-tuning Titan text model\n", "hyper_parameters = {\n", " \"epochCount\": \"2\",\n", " \"batchSize\": \"1\",\n", " \"learningRate\": \"0.00003\",\n", " }\n", "\n", "\n", "s3_bucket_config=f's3://{bucket_name}/outputs/output-{custom_model_name}'\n", "# Specify your data path for training, validation(optional) and output\n", "training_data_config = {\"s3Uri\": s3_train_uri}\n", "\n", "validation_data_config = {\n", " \"validators\": [{\n", " # \"name\": \"validation\",\n", " \"s3Uri\": s3_validation_uri\n", " }]\n", " }\n", "\n", "output_data_config = {\"s3Uri\": s3_bucket_config}\n", "\n", "\n", "# Create the customization job\n", "training_job_response = bedrock.create_model_customization_job(\n", " customizationType=customization_type,\n", " jobName=customization_job_name,\n", " customModelName=custom_model_name,\n", " roleArn=customization_role,\n", " baseModelIdentifier=base_model_id,\n", " hyperParameters=hyper_parameters,\n", " trainingDataConfig=training_data_config,\n", " validationDataConfig=validation_data_config,\n", " outputDataConfig=output_data_config\n", ")\n", "training_job_response" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Check fine-tuning job status" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "scrolled": true, "tags": [] }, "outputs": [ { "ename": "ValidationException", "evalue": "An error occurred (ValidationException) when calling the GetModelCustomizationJob operation: The provided job identifier is invalid.", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mValidationException\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[14], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m fine_tune_job \u001b[38;5;241m=\u001b[39m \u001b[43mbedrock\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget_model_customization_job\u001b[49m\u001b[43m(\u001b[49m\u001b[43mjobIdentifier\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcustomization_job_name\u001b[49m\u001b[43m)\u001b[49m[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstatus\u001b[39m\u001b[38;5;124m\"\u001b[39m]\n\u001b[1;32m 2\u001b[0m \u001b[38;5;28mprint\u001b[39m(fine_tune_job)\n\u001b[1;32m 4\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m fine_tune_job \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInProgress\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n", "File \u001b[0;32m/opt/conda/lib/python3.10/site-packages/botocore/client.py:565\u001b[0m, in \u001b[0;36mClientCreator._create_api_method.._api_call\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 561\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(\n\u001b[1;32m 562\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpy_operation_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m() only accepts keyword arguments.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 563\u001b[0m )\n\u001b[1;32m 564\u001b[0m \u001b[38;5;66;03m# The \"self\" in this scope is referring to the BaseClient.\u001b[39;00m\n\u001b[0;32m--> 565\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_make_api_call\u001b[49m\u001b[43m(\u001b[49m\u001b[43moperation_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m/opt/conda/lib/python3.10/site-packages/botocore/client.py:1021\u001b[0m, in \u001b[0;36mBaseClient._make_api_call\u001b[0;34m(self, operation_name, api_params)\u001b[0m\n\u001b[1;32m 1017\u001b[0m error_code \u001b[38;5;241m=\u001b[39m error_info\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mQueryErrorCode\u001b[39m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;129;01mor\u001b[39;00m error_info\u001b[38;5;241m.\u001b[39mget(\n\u001b[1;32m 1018\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCode\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1019\u001b[0m )\n\u001b[1;32m 1020\u001b[0m error_class \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mexceptions\u001b[38;5;241m.\u001b[39mfrom_code(error_code)\n\u001b[0;32m-> 1021\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m error_class(parsed_response, operation_name)\n\u001b[1;32m 1022\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 1023\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m parsed_response\n", "\u001b[0;31mValidationException\u001b[0m: An error occurred (ValidationException) when calling the GetModelCustomizationJob operation: The provided job identifier is invalid." ] } ], "source": [ "fine_tune_job = bedrock.get_model_customization_job(jobIdentifier=customization_job_name)[\"status\"]\n", "print(fine_tune_job)\n", "\n", "while fine_tune_job == \"InProgress\":\n", " time.sleep(60)\n", " fine_tune_job = bedrock.get_model_customization_job(jobIdentifier=customization_job_name)[\"status\"]\n", " print (fine_tune_job)\n", " time.sleep(60)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "tags": [] }, "outputs": [ { "ename": "ValidationException", "evalue": "An error occurred (ValidationException) when calling the GetModelCustomizationJob operation: The provided job identifier is invalid.", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mValidationException\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[15], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m fine_tune_job \u001b[38;5;241m=\u001b[39m \u001b[43mbedrock\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget_model_customization_job\u001b[49m\u001b[43m(\u001b[49m\u001b[43mjobIdentifier\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcustomization_job_name\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m/opt/conda/lib/python3.10/site-packages/botocore/client.py:565\u001b[0m, in \u001b[0;36mClientCreator._create_api_method.._api_call\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 561\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(\n\u001b[1;32m 562\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpy_operation_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m() only accepts keyword arguments.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 563\u001b[0m )\n\u001b[1;32m 564\u001b[0m \u001b[38;5;66;03m# The \"self\" in this scope is referring to the BaseClient.\u001b[39;00m\n\u001b[0;32m--> 565\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_make_api_call\u001b[49m\u001b[43m(\u001b[49m\u001b[43moperation_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m/opt/conda/lib/python3.10/site-packages/botocore/client.py:1021\u001b[0m, in \u001b[0;36mBaseClient._make_api_call\u001b[0;34m(self, operation_name, api_params)\u001b[0m\n\u001b[1;32m 1017\u001b[0m error_code \u001b[38;5;241m=\u001b[39m error_info\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mQueryErrorCode\u001b[39m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;129;01mor\u001b[39;00m error_info\u001b[38;5;241m.\u001b[39mget(\n\u001b[1;32m 1018\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCode\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1019\u001b[0m )\n\u001b[1;32m 1020\u001b[0m error_class \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mexceptions\u001b[38;5;241m.\u001b[39mfrom_code(error_code)\n\u001b[0;32m-> 1021\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m error_class(parsed_response, operation_name)\n\u001b[1;32m 1022\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 1023\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m parsed_response\n", "\u001b[0;31mValidationException\u001b[0m: An error occurred (ValidationException) when calling the GetModelCustomizationJob operation: The provided job identifier is invalid." ] } ], "source": [ "fine_tune_job = bedrock.get_model_customization_job(jobIdentifier=customization_job_name)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "tags": [] }, "outputs": [ { "ename": "NameError", "evalue": "name 'fine_tune_job' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[16], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m pprint\u001b[38;5;241m.\u001b[39mpp(\u001b[43mfine_tune_job\u001b[49m)\n", "\u001b[0;31mNameError\u001b[0m: name 'fine_tune_job' is not defined" ] } ], "source": [ "pprint.pp(fine_tune_job)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "tags": [] }, "outputs": [ { "ename": "NameError", "evalue": "name 'fine_tune_job' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[17], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m output_job_name \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel-customization-job-\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;241m+\u001b[39m\u001b[43mfine_tune_job\u001b[49m[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mjobArn\u001b[39m\u001b[38;5;124m'\u001b[39m]\u001b[38;5;241m.\u001b[39msplit(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m/\u001b[39m\u001b[38;5;124m'\u001b[39m)[\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m]\n\u001b[1;32m 2\u001b[0m output_job_name\n", "\u001b[0;31mNameError\u001b[0m: name 'fine_tune_job' is not defined" ] } ], "source": [ "output_job_name = \"model-customization-job-\"+fine_tune_job['jobArn'].split('/')[-1]\n", "output_job_name" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we are ready to create [`provisioned throughput`](#) which is needed before you can do the inference on the fine-tuned model.\n", "\n", "### Overview of Provisioned throughput\n", "You specify Provisioned Throughput in Model Units (MU). A model unit delivers a specific throughput level for the specified model. The throughput level of a MU for a given Text model specifies the following:\n", "\n", "- The total number of input tokens per minute – The number of input tokens that an MU can process across all requests within a span of one minute.\n", "\n", "- The total number of output tokens per minute – The number of output tokens that an MU can generate across all requests within a span of one minute.\n", "\n", "Model unit quotas depend on the level of commitment you specify for the Provisioned Throughput.\n", "\n", "- For custom models with no commitment, a quota of one model unit is available for each Provisioned Throughput. You can create up to two Provisioned Throughputs per account.\n", "\n", "- For base or custom models with commitment, there is a default quota of 0 model units. To request an increase, use the [limit increase form](#https://support.console.aws.amazon.com/support/home#/case/create?issueType=service-limit-increase)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Retrieve Custom Model\n", "Once the customization job is finished, you can check your existing custom model(s) and retrieve the modelArn of your fine-tuned model." ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "{'ResponseMetadata': {'RequestId': 'dde18333-36e9-491f-bb2d-9bbe45319e83',\n", " 'HTTPStatusCode': 200,\n", " 'HTTPHeaders': {'date': 'Sat, 01 Jun 2024 08:03:44 GMT',\n", " 'content-type': 'application/json',\n", " 'content-length': '38',\n", " 'connection': 'keep-alive',\n", " 'x-amzn-requestid': 'dde18333-36e9-491f-bb2d-9bbe45319e83'},\n", " 'RetryAttempts': 0},\n", " 'modelSummaries': []}" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# List your custom models\n", "bedrock.list_custom_models()" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "tags": [] }, "outputs": [ { "ename": "ValidationException", "evalue": "An error occurred (ValidationException) when calling the GetCustomModel operation: The provided model identifier is invalid.", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mValidationException\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[19], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m model_id \u001b[38;5;241m=\u001b[39m \u001b[43mbedrock\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget_custom_model\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodelIdentifier\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcustom_model_name\u001b[49m\u001b[43m)\u001b[49m[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mmodelArn\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[1;32m 2\u001b[0m model_id\n", "File \u001b[0;32m/opt/conda/lib/python3.10/site-packages/botocore/client.py:565\u001b[0m, in \u001b[0;36mClientCreator._create_api_method.._api_call\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 561\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(\n\u001b[1;32m 562\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpy_operation_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m() only accepts keyword arguments.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 563\u001b[0m )\n\u001b[1;32m 564\u001b[0m \u001b[38;5;66;03m# The \"self\" in this scope is referring to the BaseClient.\u001b[39;00m\n\u001b[0;32m--> 565\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_make_api_call\u001b[49m\u001b[43m(\u001b[49m\u001b[43moperation_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m/opt/conda/lib/python3.10/site-packages/botocore/client.py:1021\u001b[0m, in \u001b[0;36mBaseClient._make_api_call\u001b[0;34m(self, operation_name, api_params)\u001b[0m\n\u001b[1;32m 1017\u001b[0m error_code \u001b[38;5;241m=\u001b[39m error_info\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mQueryErrorCode\u001b[39m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;129;01mor\u001b[39;00m error_info\u001b[38;5;241m.\u001b[39mget(\n\u001b[1;32m 1018\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCode\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1019\u001b[0m )\n\u001b[1;32m 1020\u001b[0m error_class \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mexceptions\u001b[38;5;241m.\u001b[39mfrom_code(error_code)\n\u001b[0;32m-> 1021\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m error_class(parsed_response, operation_name)\n\u001b[1;32m 1022\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 1023\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m parsed_response\n", "\u001b[0;31mValidationException\u001b[0m: An error occurred (ValidationException) when calling the GetCustomModel operation: The provided model identifier is invalid." ] } ], "source": [ "model_id = bedrock.get_custom_model(modelIdentifier=custom_model_name)['modelArn']\n", "model_id" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create Provisioned Throughput\n", "
\n", "Note: Creating provisioned throughput will take around 20-30mins to complete.
\n", "\n", "You will need to create provisioned throughput to be able to evaluate the model performance. You can do so through the [console].(https://docs.aws.amazon.com/bedrock/latest/userguide/prov-cap-console.html) or use the following api call:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "tags": [] }, "outputs": [ { "ename": "NameError", "evalue": "name 'model_id' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[20], line 6\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mboto3\u001b[39;00m \n\u001b[1;32m 2\u001b[0m boto3\u001b[38;5;241m.\u001b[39mclient(service_name\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mbedrock\u001b[39m\u001b[38;5;124m'\u001b[39m)\n\u001b[1;32m 3\u001b[0m provisioned_model_id \u001b[38;5;241m=\u001b[39m bedrock\u001b[38;5;241m.\u001b[39mcreate_provisioned_model_throughput(\n\u001b[1;32m 4\u001b[0m modelUnits\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m1\u001b[39m,\n\u001b[1;32m 5\u001b[0m provisionedModelName\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mtest-model\u001b[39m\u001b[38;5;124m'\u001b[39m, \n\u001b[0;32m----> 6\u001b[0m modelId\u001b[38;5;241m=\u001b[39m\u001b[43mmodel_id\u001b[49m\n\u001b[1;32m 7\u001b[0m )[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mprovisionedModelArn\u001b[39m\u001b[38;5;124m'\u001b[39m] \n", "\u001b[0;31mNameError\u001b[0m: name 'model_id' is not defined" ] } ], "source": [ "import boto3 \n", "boto3.client(service_name='bedrock')\n", "provisioned_model_id = bedrock.create_provisioned_model_throughput(\n", " modelUnits=1,\n", " provisionedModelName='test-model', \n", " modelId=model_id\n", ")['provisionedModelArn'] " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "status_provisioning = bedrock.get_provisioned_model_throughput(provisionedModelId = provisioned_model_id)['status']" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "import time\n", "while status_provisioning == 'Creating':\n", " time.sleep(60)\n", " status_provisioning = bedrock.get_provisioned_model_throughput(provisionedModelId=provisioned_model_id)['status']\n", " print(status_provisioning)\n", " time.sleep(60)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Invoke the Custom Model\n", "\n", "Before invoking lets get the sample prompt from our test data. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Provide the prompt text \n", "test_file_path = f'{data_folder}/{test_file_name}'\n", "with open(test_file_path) as f:\n", " lines = f.read().splitlines()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "test_prompt = json.loads(lines[3])['prompt']\n", "reference_summary = json.loads(lines[3])['completion']\n", "pprint.pp(test_prompt)\n", "print(reference_summary)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "prompt = f\"\"\"\n", "{test_prompt}\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "base_model_arn = f'arn:aws:bedrock:{region}::foundation-model/amazon.titan-text-lite-v1'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Make sure to construct model input following the format needed by titan text model following instructions [here](#https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-text.html). \n", "Please pay attention to the \"Model invocation request body field\" section in the `body` variable, which we will pass as payload to the custom model trained above. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "body = json.dumps(\n", " {\n", " \"inputText\": prompt,\n", " \"textGenerationConfig\": {\n", " \"maxTokenCount\": 2048,\n", " \"stopSequences\": ['User:'],\n", " \"temperature\": 0,\n", " \"topP\": 0.9\n", " }\n", " }\n", " )\n", "\n", "accept = 'application/json'\n", "contentType = 'application/json'\n", "\n", "fine_tuned_response = bedrock_runtime.invoke_model(body=body, \n", " modelId=provisioned_model_id, \n", " accept=accept, \n", " contentType=contentType)\n", "\n", "base_model_response = bedrock_runtime.invoke_model(body=body, \n", " modelId=base_model_arn, \n", " accept=accept, \n", " contentType=contentType)\n", "\n", "fine_tuned_response_body = json.loads(fine_tuned_response.get('body').read())\n", "base_model_response_body = json.loads(base_model_response.get('body').read())\n", "\n", "print(\"Base model response: \", base_model_response_body[\"results\"][0][\"outputText\"] + '\\n')\n", "print(\"Fine tuned model response:\", fine_tuned_response_body[\"results\"][0][\"outputText\"]+'\\n')\n", "print(\"Reference summary from test data: \" , reference_summary)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluate the performance of the model \n", "In this section, we will use `BertScore` metrics to evaluate the performance of the fine-tuned model as compared to base model to check if fine-tuning has improved the results.\n", "\n", "- `BERTScore`: calculates the similarity between a summary and reference texts based on the outputs of BERT (Bidirectional Encoder Representations from Transformers), a powerful language model. [Medium article link](#https://haticeozbolat17.medium.com/bertscore-and-rouge-two-metrics-for-evaluating-text-summarization-systems-6337b1d98917)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "base_model_generated_response = [base_model_response_body[\"results\"][0][\"outputText\"]]\n", "fine_tuned_generated_response = [fine_tuned_response_body[\"results\"][0][\"outputText\"]]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "from bert_score import score\n", "reference_summary = [reference_summary]\n", "fine_tuned_model_P, fine_tuned_R, fine_tuned_F1 = score(fine_tuned_generated_response, reference_summary, lang=\"en\")\n", "base_model_P, base_model_R, base_model_F1 = score(base_model_generated_response, reference_summary, lang=\"en\")\n", "print(\"F1 score: base model \", base_model_F1)\n", "print(\"F1 score: fine-tuned model\", fine_tuned_F1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Conclusion\n", "From the scores above and looking at the base model summary, fine-tuned model summary and reference summary, it clearly indicates that fine-tuning the model tends to improve the results on the task its trained on. We only used 1K records for training with 100 validation records and 2 epochs, and were able to get better results. \n", "You may want to adjust the learning_rate first, visualize training and validation metrics to understand the performance of training job, before increase the size of your data. \n", "\n", "
\n", "Tip: \n", " Please refer to the guidelines provided for fine-tuning the model based on your task.
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Delete provisioned througput\n", "
\n", "Warning: Please make sure to delete providsioned throughput as there will cost incurred if its left in running state, even if you are not using it. \n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "bedrock.delete_provisioned_model_throughput(provisionedModelId=provisioned_model_id)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 }, { "_defaultOrder": 55, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 56, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4de.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 57, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.trn1.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 58, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.trn1.32xlarge", "vcpuNum": 128 }, { "_defaultOrder": 59, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.trn1n.32xlarge", "vcpuNum": 128 } ], "instance_type": "ml.c5.2xlarge", "kernelspec": { "display_name": "Python 3 (Data Science 3.0)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-310-v1" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.6" } }, "nbformat": 4, "nbformat_minor": 4 }