This project provides a developer friendly way to access the services of AWS Bedrock, an AWS service that provides a managed Language Model (LLM) service with a catalog of supported LLMs. This is a Spring Cloud Starter for the Bedrock service that removes all the boilerplate code required to access the service. It uses the standard Spring Cloud AWS Starter to provide the credentials for the AWS SDK for Java, and the standards of Spring Boot and Spring Cloud to provide the configuration and the auto-wiring of the service objects.

Just provide the right properties as defined below and the appropriate AWSCredentialsProvider object will be activated for injection into your Spring objects. An example for the creation of BasicAwsCredentials is shown below. You can follow the link below to understand all the other supported credential models.

# for BasicAwsCredentials
spring.cloud.aws.credentials.access-key=YOUR_ACCESS_KEY
spring.cloud.aws.credentials.secret-key=YOUR_SECRET_KEY

# an optional method to do the same
spring.cloud.aws.credentials.profile.name=YOUR_PROFILE
spring.cloud.aws.credentials.profile.path=~/.aws/credentials

See the Credentials section of Spring Cloud AWS Starter documentation for alternate configuration options.

aws.bedrock.model.id=anthropic.claude-v2
aws.bedrock.model.claude.temperature=0.5
aws.bedrock.model.claude.maxTokensToSample=100

Once the right AWSCredentialsProvider is injected, you can expect the activation of the appropriate versions of the BedrockClient and the BedrockRuntimeClient. These will be conditional on the bedrock.client.type property. The options there are sync and async, which drive the activation of the synchronous or the asynchronous versions of the clients.

E.g.,

bedrock.client.type=sync

One can use the activated clients to interact with the AWS Bedrock service based on the docs for the Bedrock Service.

The best part of the Starter is the activation of a service object which has the invoke method.

All you need to do is provide the properties like one of the sets below to activate the appropriate implementations of the BedrockService interface (Claude, Jurassic, Llama, StableDiffusion, or Titan).

aws.bedrock.model.claude.id=anthropic.claude-v2
aws.bedrock.model.claude.temperature=0.5
aws.bedrock.model.claude.maxTokensToSample=100

aws.bedrock.model.jurassic.id=ai21.j2-mid-v1
aws.bedrock.model.jurassic.maxTokens=200
aws.bedrock.model.jurassic.temperature=0.5

aws.bedrock.model.llama2.id=anthropic.llama2-v1
aws.bedrock.model.llama2.maxTokens=200
aws.bedrock.model.llama2.temperature=0.5

aws.bedrock.model.titan.id=amazon.titan-text-express-v1
aws.bedrock.model.titan.maxTokens=4096
aws.bedrock.model.titan.temperature=0.0

aws.bedrock.model.stability.id=stability.stable-diffusion-xl
aws.bedrock.model.stability.promptStrength=10.0
aws.bedrock.model.stability.generationStep=30
aws.bedrock.model.stability.seed=75

Once activated, a Service can be autowired and used as below.

@RestController
public class ClaudeController {

    private final ClaudeService claudeService;


    @Autowired
    public ClaudeController(ClaudeService claudeService) {
        this.claudeService = claudeService;
    }

    @GetMapping("/invoke")
    public String invoke(@RequestParam String prompt) {
        try {
            return claudeService.invoke(prompt);
        } catch (Exception e) {
            return "Error invoking Claude: " + e.getMessage();
        }
    }
}

If you activate more than one model, then you will need to use the @Qualifier annotation to eliminate ambiguity when choosing to inject by the BedrockService interface. See @Qualifier vs @Primary with Examples for background. You may not activate more than one model variant from a single provider.

You might also want to take a look at and/or run the Service tests. To do that, make sure you've set the appropriate AWS_* environment variables, then execute

mvn test -Dspring.profiles.active=authorized