AI Integration

This section describes AI integration in CAP Java: building agents on top of your CDS services and configuring the LLM chat models they use.

Agents Alpha

A CAP agent turns a CDS service into a conversational endpoint. It answers natural-language requests by using the service's entities, actions, and functions as tools, backed by an LLM. Agents speak the A2A protocol, so any A2A-compatible client can talk to them.

A2A protocol version

For compatibility reasons with Joule and Agent Gateway, version 0.3.0 of the A2A protocol is used by default.

In-Memory Chat History Only

CAP Java doesn't yet have a persistent chat history. Instead, it stores conversation history in volatile memory. Hence, you need to send all follow-up prompts to the same server instance.

Adding the Dependency

Add the agent adapter to your srv/pom.xml:

<dependency>
  <groupId>com.sap.cds</groupId>
  <artifactId>cds-adapter-agent</artifactId>
</dependency>

Defining an Agent

Annotate a service with @agent to expose it as an agent:

@agent
service TravelService {
  entity Travels   as projection on db.Travels;
  entity Flights   as projection on xflights.Flights;
  action createTravel(Description: String, BeginDate: Date, EndDate: Date) returns Travels:ID;
  action addFlightToTravel(TravelID: Travels:ID, FlightID: Flights:ID, FlightDate: Date);
}

Define a tailored service for your agent

Define a dedicated service for your agent, tailored to its specific use case.

The following tools are derived from the service and made available to the agent:

• A generic query tool allows the LLM to read data via CDS QL across all entities defined in the service.
• Unbound actions and functions become individually callable tools, invoked by name.

Unbound actions and functions only

Only unbound actions and functions are supported as tools. Bound actions and bound functions aren't supported yet.

By default the agent is served under /a2a/<service-path>, with its agent card available at the corresponding .../.well-known/agent-card.json endpoint.

During development, a built-in chat UI lets you try out your agents in the browser from CAP's index page. It's disabled by default in the production profile.

Customizing an Agent

Without further configuration, the agent derives a system prompt and its advertised skills from the CDS model. To customize both, add resources under <ServiceName>-agent/ on the classpath (for example srv/src/main/resources/TravelService-agent/):

TravelService-agent/
├── AGENTS.md                     # system prompt + agent card metadata
└── skills/
    ├── browse-travels/SKILL.md
    └── create-travel/SKILL.md

AGENTS.md holds the system prompt as its body, with optional YAML frontmatter for the agent card:

---
name: Travel Assistant
version: 2.0.0
description: Helps customers browse and book travels
---
You are a helpful travel assistant. Help customers find flights and manage their travels.
Always use the provided tools to answer questions — do not make up data.

Each skills/<id>/SKILL.md describes one skill advertised in the agent card:

---
name: browse-travels
description: Browse and search available travels
metadata:
  tags: [travels, search]
  examples:
    - Show me all open travels
    - Find travels in March 2027
---
Query the Travels entity to search for existing travels. Use filters on BeginDate, EndDate, or status as needed.

Chat Model Configuration Alpha

Agents use a named chat model configuration. Configure models under cds.ai.chat.models, where the key is the configuration name:

cds:
  ai.chat.models:
    llm:
      kind: aicore
      model: anthropic--claude-4.6-sonnet
      temperature: 0.0

Property	Description
kind	The model provider: aicore, ollama, or mocked.
model	The provider-specific model name.
temperature	Sampling temperature (0.0–1.0). Defaults to the provider's.
options	Additional provider-specific parameters.

An agent picks its model configuration via the @agent.llm annotation, which defaults to the configuration named llm. If no configuration matches, CAP Java falls back to aicore when an SAP AI Core service binding is present, and to mocked otherwise.

To bind a specific configuration to an agent, define it under a name of your choice and reference it with @agent.llm:

cds:
  ai.chat.models:
    reasoning:
      kind: aicore
      model: anthropic--claude-4.8-opus
      temperature: 0.2

@agent
@agent.llm: 'reasoning'   // use the 'reasoning' config instead of the default ('llm') config
service TravelService { ... }

SAP AI Core

With an aicore service binding, requests run through SAP AI Core orchestration. Set model to the model you want to use; if omitted, a default model is used.

Running Locally with Ollama

For local testing, you can run an agent against a local model served by Ollama. Pull a model (for example ollama pull gemma4:26b) and point a configuration at it:

---
spring:
  config.activate.on-profile: ollama
cds:
  ai.chat.models:
    llm:
      kind: ollama
      model: gemma4:26b               # a model pulled in Ollama
      # options:
      #   url: http://localhost:11434  # Ollama base URL (this is the default)

Add the LangChain4j Ollama integration to your srv/pom.xml:

<dependency>
  <groupId>dev.langchain4j</groupId>
  <artifactId>langchain4j-ollama</artifactId>
  <!-- import langchain4j-bom for version management -->
  <!-- and use same version as shipped with CAP Java -->
</dependency>

Testcontainers

Alternatively, Ollama can be started via Testcontainers for local tests. Note that reasoning on a containerized model can be slow.

Mocked

The mocked kind returns static responses without calling any model. It's the default when no other provider is configured or bound, which keeps local runs and tests working out of the box.

Vector Embeddings

In CDS, vector embeddings are stored in elements of type Vector.

CAP Java supports the Vector type on SAP HANA and, for local testing, on H2 and SQLite; PostgreSQL support is in beta and requires the pgvector extension. Learn more in the embeddings guide PostgreSQL section.

In CAP Java, vectors are represented by the CdsVector type, which allows a unified handling of different vector representations such as float[] and String:

// Vector embedding of text via SAP Cloud SDK for AI
float[] embedding = embeddingModel.embedding(
  new OpenAiEmbeddingRequest(List.of(text))).getEmbeddingVectors().get(0);

CdsVector v1 = CdsVector.of(embedding); // float[] format

Info

In CDS QL queries, elements of type Vector are excluded from the select list by default.

CAP Java supports multiple vector functions that allow you to compute vector embeddings, similarity, and distance directly in the database.