Skip to content

Allow users to pass additional_args from payload_config when using SageMaker AI models #602

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Christian-kam
Copy link

Description

The current implementation does not allow users to pass the additional_args from payload_config. The additional_args is being filtered out when creating the payload, but it's not being added back in. lines 224 - 229

Related Issues

Enable sagemaker_ai users to provide additional arguments in the payload. For example, when using Qwen3 models, one might want to disable thinking mode which is particularly useful in scenarios where disabling thinking is essential for enhancing efficiency and speed.

Documentation PR

Type of Change

Bug fix

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

  • I ran hatch run prepare

Example Usage

from strands import Agent, tool
from strands_tools import calculator
from strands.models.sagemaker import SageMakerAIModel

# Create a custom tool 
@tool
def weather():
    """ Get weather """ # Dummy implementation
    return "sunny"

# Create the endpoint configuration
model_config = {
    "endpoint_name": "qwen-endpoint",
    "inference_component_name": "qwen3-32b-fp8-IC",
    "region_name": "us-west-2",  # or your preferred region
}

# Create the payload configuration
payload_config = {
    "max_tokens": 6000,
    "stream": True,
    "temperature": 0.8,
    "top_p": 0.95,
    "top_k": 20,
    "additional_args": {
        "chat_template_kwargs": {"enable_thinking": False}, # Allow the user to turn off thinking mode in qwen3 model via chat_template_kwargs
    }
}

sagemaker_model = SageMakerAIModel(
    endpoint_config=model_config,
    payload_config=payload_config
)

agent = Agent(
    model=sagemaker_model,
    tools=[calculator, weather],
    system_prompt="You're a helpful assistant. You can do simple math calculation, and tell the weather."
)

response = agent("What is the weather now? and what is 2x2")

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The current implementation does not allow users to pass the additional_args from payload_config. Th additional_args is being filtered out when creating the payload, but it's not being added back in.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant