Build your own ACP-Compatible Weather DJ Agent.

The world of AI, AI Agents, and agentic protocols sometimes feels like it's evolving faster than the weather; it can be hard to know where to dip your toes in the world to get started.

No need to fret — together we’ll walk through how the different parts of this AI-involved world work together. Let’s build an Agentic application that brings together multiple APIs, teaches you how to use the Agent Connect Protocol (ACP), and how to evaluate workflows using Galileo.

Follow along as we get our hands dirty in building your very own intelligent weather assistant that not only predicts the weather but also tells you what to wear and provides a YouTube song to set the mood—no previous experience required.

The weather DJ agent operating within a terminal for the example for London.

We’ll be working off of this repository and demonstrating how to craft your very own ACP Compatible Agent!

PSST — want to just catch the video recap?

Check out a bit more about this project on the Galileo YouTube channel.

Meet AGNTCY and the Agent Connect Protocol (ACP)

This application is part of the larger AGNTCY ecosystem — a collection of composable AI agents built on the Agent Connect Protocol (ACP).

ACP is comparable to MCP; while MCP allows you to connect different AI applications to one another, ACP allows you to connect different agents to one another — both protocols offer a standardized means for tools to communicate with systems and services.

Much like MCP, ACP operates on a server as defined by AGNTCY's Agent Connect Protocol (ACP). This creates a standardized interface that allows AI agents to communicate and collaborate over a network. It provides endpoints for configuring agents, invoking them with specific inputs, retrieving outputs, and managing agent capabilities and schemas. This enables interoperability between agents developed by different parties, facilitating seamless integration in multi-agent systems.

Why a unified protocol, like ACP, Matters

• Interoperability: Agents from different developers can work together
• Standardization: Consistent interface for building and deploying agents
• Composability: Combine agents for more complex workflows
• Discovery: Clients can find agents with specific capabilities

Key ACP Components:

The ACP Protocol includes four key components:

1. Agent Descriptor: Defines the agent’s purpose and interface
2. Runs: Executions of the agent with specific inputs
3. Input/Output Schemas: Defines what the agent accepts and returns
4. Tools: Functions the agent uses to complete tasks

We’ll be evaluating our agent using Galileo. From model calls to tool outputs — Galileo’s agentic evaluation suite helps developers monitor their applications in real-time, helping build confidence in what the agent is doing.

Requirements

• Some familiarity with a code editor
• A computer with Python 3.12 installed on it

The following API keys:

• OpenAI API key (for AI ✨magic✨)*
• OpenWeatherMap API key (for weather data).
• YouTube Data API key (for video recommendations).
• A Galileo API key (for monitoring and evaluation).

* paid plan required

Additional Resources

Further reading or resources that are optional, however, may be helpful if you’re new to the world of AI Agents.

- Agent Connect Protocol (ACP) Specification

- ACP GitHub Team

- OpenAI Documentation

- Galileo Documentation

- Simple Agent Framework

What’s an AI Agent Anyway?

Imagine a software sidekick that perceives its environment, makes decisions, and takes actions to achieve specific goals. That's an AI agent in a nutshell. (Curious to learn more? Check out our field guide on the blog.)

In this case, the agent will:

• Accept a location as input.
• Fetch current weather data.
• Recommend items to bring based on the weather.
• Suggest a YouTube video that matches the "vibe" of the weather.

We’re building your personalized meteorologist, meets fashion consultant, meets DJ.

As we build this application, we will want a straightforward way to understand and decipher what the agent is doing as well. Automation is good, but without proper monitoring, observability, and evaluation, we may create more harm than good for these agents.

Setting up Shop

First, clone the AGNTCY Sample applications, then navigate to the folder containing the Weather Vibes agent.

1git clone https://github.com/rungalileo/AGNTCY-Applications.git
2cd AGNTCY-Applications/weather_vibes_agp/tutorials/02-weather-vibes-agent
3

Within this folder, you’ll also notice further details about how to get started or resources for troubleshooting; however, for the sake of this tutorial, we’ll be jumping right into the application folder.

1cd weather_vibes

You’ll notice a few different folders and files in this folder. Let's quickly review the core files behind the agentic application.

• Main Agent (weather_vibes_agent.py):
  • Implements the core agent logic
  • Manages state (search history, favorite locations)
  • Coordinates between different tools
  • Handles ACP-compliant communication
• Agent Descriptor (descriptor.py):
  • Defines the agent's capabilities and interfaces
• Tools
  • WeatherTool: Fetches current weather using OpenWeatherMap API.
  • RecommendationsTool: Suggests items to bring based on the weather.
  • YouTubeTool: Finds matching videos for the weather conditions.

Let's now set up our environment — choose your favorite Python package manager (I’ll be using uv) and install the requirements.txt file.

Don’t have uv installed? Get it here.

1uv venv venv
2source venv/bin/activate
3uv pip install -r requirements.txt
4

Find the `.env.example` file within `weather_vibes`,and rename it to `.env.`

It should look something like this:

1# API Keys
2OPENAI_API_KEY=your_openai_api_key_here
3OPENWEATHER_API_KEY=your_openweather_api_key_here
4YOUTUBE_API_KEY=your_youtube_api_key_here
5

Fill in your respective keys you’ve gathered, and save the file.

Now, let's run the ACP server using uvicorn from within the weather_vibes directory.

1python -m uvicorn main:app --reload

When you run the command, you should see the following within your terminal:

Terminal screenshot showing the successful startup of a Uvicorn server for an ACP-compatible Weather Vibes agent. The server is running locally at http://127.0.0.1:8000, with reloader and server processes started, and the message “Application startup complete” confirming successful deployment.

If you navigate to http://127.0.0.1:8000/ within your browser, you should see the following:

Screenshot of a JSON configuration file for the Weather Vibes ACP Server. It includes three fields: "name" set to "Weather Vibes ACP Server", "version" set to "0.1.0", and "description" describing it as the "Agent Connect Protocol implementation for Weather Vibes". A “Raw” and “Parsed” toggle is visible in the top-right corner.

Wahooo! The server is up and going!

While this server is running, open a new terminal window, navigate to the weather_vibes folder again, and set up the virtual environment as we did previously.

1cd AGNTCY-Applications/weather_vibes_agp/tutorials/02-weather-vibes-agent/weather_vibes
2
3uv venv venv
4source venv/bin/activate
5uv pip install -r requirements.txt
6

Your requirements are now installed, and you’re one step closer to determining those sweet, sweet weather vibes.

Now, from within that same folder, run the agent script.

1python run_agent.py

When you run the weather vibes agent, you’ll notice the following in the terminal:

erminal prompt showing the execution of run_agent.py within a Python virtual environment. The user is prompted to “Enter a location (default: New York):” indicating the beginning of the agent's weather data retrieval process.

You will first be prompted to enter a location — it will default to New York if no location is specified.

For example, let's take a look at what’s happening on the other side of the country — San Francisco.

Type in the location, and hit ‘enter’.

Terminal output displaying the response from the Weather Vibes agent for San Francisco. The weather report includes temperature (15.28°C), cloudy conditions, 75% humidity, and wind speed of 8.49 m/s. Recommendations feature emojis for shorts, sunshine, and a t-shirt. A matching video titled "Cloudy Day | Instrumental Chill Music Mix" from the YouTube channel Fluidified is also provided.

It’s a cloudy spring day in San Francisco — warm enough to wear shorts, with a very mild breeze, and an accompanying cloudy day chill music mix. ☁️🎶

Adding Evaluations

Now that we’ve got our app working on our machine, let's ensure that the application performs as expected when someone calls it with the help of Galileo.

If you haven’t created a free account with Galileo yet, now is a great time to do so! Visit app.galileo.ai to create an account to get started.

(PSST — want to see the docs on how to get started with Galileo? Follow along with the Python SDK documentation.)

Let’s open a new terminal window, navigate to within the weather_vibes folder, and activate our virtual environment.

1cd AGNTCY-Applications/weather_vibes_agp/tutorials/02-weather-vibes-agent/weather_vibes
2
3uv venv venv
4source venv/bin/activate
5uv pip install -r requirements.txt

Now, let's install the Galileo Python SDK.

1uv pip install galileo

Adding Galileo Keys

Once that is installed, let’s add a few lines to our environment variables.

Open up your .env file, and add the following:

1GALILEO_API_KEY=
2GALILEO_PROJECT=
3GALILEO_LOG_STREAM=

Your final .env file should look like this:

1# API Keys
2OPENAI_API_KEY=your_openai_api_key_here
3OPENWEATHER_API_KEY=your_openweather_api_key_here
4YOUTUBE_API_KEY=your_youtube_api_key_here
5
6# Galileo Keys
7GALILEO_API_KEY=your_galileo_api_key_here
8GALILEO_PROJECT=your_galileo_project_key_here
9GALILEO_LOG_STREAM=your_galileo_log_stream_here

Let's go get those Galileo-specific keys now. First, the Galileo API Key. This is universal to the organization.

When you’re signed into your account, click on your profile icon in the upper right-hand corner of the screen, then navigate down to `Settings & Users`.

Dropdown menu from the Galileo platform user interface showing the user signed in as [email protected]. Menu options include: “Documentation & Tutorials,” “Settings & Users” (highlighted), “Changelog,” and “Sign Out.” A green circular avatar with the letter "E" is visible on the top right.

From here, click on API Keys in the upper left-hand menu, then select the blue + Create new key button.

Screenshot of the API Keys management page in the Galileo platform. The left menu highlights “API Keys” under “Settings & Users.” Two red arrows point to: (1) the API Keys section title and (2) a purple “+ Create new key” button in the top-right corner of the panel displaying existing keys, creation dates, and expiration info.

Give it a name you won’t forget — and remember once you close the tab displaying your key, you won't be able to view it again. Store it in a safe location for future reference.

Paste the key in the respective section by GALILEO_API_KEY from within your .env file, and let’s now move to getting the Project Key and the Log Stream.

Screenshot of the Galileo dashboard showing the dropdown to switch or create a new project. A red arrow labeled (1) points to the "Go to project" menu, while another red arrow labeled (2) points to a purple “+ New Project” button within the dropdown. A project search bar and filter options are visible beneath the dropdown.

From the top menu bar, select Go to project, then select + New Project.

Give your project a name — this will also be your project key, so follow best practices and create a name without spaces. For today, I’ll be naming it weather-vibes.

Then, select Create project. We can now copy that key right back into our .env file under GALILEO_PROJECT.

It should look like this:

1GALILEO_PROJECT=weather-vibes

From here, you will be presented with your project's home screen.

Screenshot of the Galileo project dashboard for a project named "weather-vibes." The dashboard displays interactive tiles for features including Log Streams, Playground, Experiments, Metrics, Datasets, and Invite Users. The right sidebar contains quick links to documentation resources like guides and SDK references. A “Connect your app” button is highlighted under Log Streams.

Under the panel labeled "Log Streams," select the "Connect your app" button.

A new log stream will be created with the name my_log_stream. For simplicity’s sake, we can proceed with using this for our application.

Animated GIF of the Galileo project dashboard for the “weather-vibes” project. It showcases various interactive modules including Log Streams, Playground, Experiments, Metrics, Datasets, and Invite Users. The right sidebar offers quick links to documentation, and elements respond to user interaction such as hover highlights and expanding tiles.

To create a new log stream, select the Create log stream button from the dropdown menu under 'my_log_stream'.

Return to your .env file, and add my_log_stream to the GALILEO_LOG_STREAM variable.

When complete, save your file, and let’s start connecting our application to Galileo for Agent Evaluations.

Implementing logging

Now, let's implement logging within our agentic application. We will be using the Galileo Python SDK and the @log decorator to capture inputs and outputs as spans within our traces.

We will need to import this log from the Galileo package, which we installed earlier, and apply it to the relevant functions within our application.

Our agent is currently called from the file, run_agent.py.

Below, I will outline the changes required to install the Galileo logs; feel free to follow along with the added comments in the repository on where to add different elements. (PSST — Want to just skip ahead? I’ve created a file for you in the same folder titled galileo_agent.py with logging already implemented. Simply add your environment variables as shown above, and you're ready to go.)

Implementing Galileo step-by-step within the run_agent.py file:

Opening the run_agent.py file — around line 17, just beyond our imports, add the Galileo import + log.

1import asyncio
2import argparse
3import json
4import os
5import sys
6from pathlib import Path
7from dotenv import load_dotenv
8# Add Galileo import below
9from Galileo import log, galileo_context
10
11# Load environment variables
12load_dotenv()
13
14# ... rest of file continued
15

Let’s also add our required variables to the environment check around line 24.

1required_keys = ["OPENAI_API_KEY", "OPENWEATHER_API_KEY", "YOUTUBE_API_KEY", "GALILEO_API_KEY"]
2if any(not os.getenv(key) for key in required_keys):
3    missing = [key for key in required_keys if not os.getenv(key)]
4    print(f"Missing API keys: {', '.join(missing)}")
5    print("Add them to your .env file or environment variables")
6    sys.exit(1)

Now, check for the Galileo log stream by adding the below around line 31.

1# Check for Galileo log stream
2galileo_log_stream = os.getenv("GALILEO_LOG_STREAM")
3if not galileo_log_stream:
4    print("Warning: GALILEO_LOG_STREAM environment variable not set.")
5    print("Using default log stream name.")
6    galileo_log_stream = "weather_vibes_agent"

Let’s now add the Galileo log span around each tool as shown below, the tools are shown around lines 41-61.

1# Tool wrappers with Galileo instrumentation
2@log(span_type="tool", name="weather_tool")
3async def get_weather(weather_tool, location, units="metric"):
4    """Get weather data with Galileo tracing"""
5    result = await weather_tool.execute(location=location, units=units)
6    return result
7
8@log(span_type="tool", name="recommendations_tool")
9async def get_recommendations(recommendations_tool, weather, max_items=5):
10    """Get recommendations with Galileo tracing"""
11    result = await recommendations_tool.execute(weather=weather, max_items=max_items)
12    return result
13
14@log(span_type="tool", name="youtube_tool")
15async def find_weather_video(youtube_tool, weather_condition, mood_override=None):
16    """Find YouTube videos with Galileo tracing"""
17    result = await youtube_tool.execute(
18        weather_condition=weather_condition,
19        mood_override=mood_override
20    )
21    return result

Log the workflow to call all the individual tool calls from the agent. After adding the tool calls, this should be around line 63.

1@log(span_type="workflow", name="weather_vibes_workflow")
2async def process_request(agent, request):
3    """Main workflow with Galileo tracing"""
4    try:
5        # Extract request data
6        input_data = request.get("input", {})
7        config = request.get("config", {})
8        metadata = request.get("metadata", {})
9        
10        # Parse parameters
11        location = input_data.get("location")
12        units = input_data.get("units", "metric")
13        verbose = config.get("verbose", False)
14        max_recommendations = config.get("max_recommendations", 5)
15        video_mood = config.get("video_mood")
16        
17        # Validate location
18        if not location:
19            return {"error": 400, "message": "Location is required"}
20        
21        # Update search history
22        if not hasattr(agent.state, "search_history"):
23            agent.state.search_history = []
24            
25        if location not in agent.state.search_history:
26            agent.state.search_history.append(location)
27            if len(agent.state.search_history) > 5:
28                agent.state.search_history = agent.state.search_history[-5:]
29        
30        # Execute tools
31        weather_result = await get_weather(agent.weather_tool, location, units)
32        if "error" in weather_result:
33            return {"error": 500, "message": f"Weather API error: {weather_result['message']}"}
34        
35        recommendations = await get_recommendations(
36            agent.recommendations_tool, weather_result, max_recommendations
37        )
38        
39        video_result = await find_weather_video(
40            agent.youtube_tool, weather_result["condition"], video_mood
41        )
42        
43        # Prepare response
44        result = {
45            "weather": weather_result,
46            "recommendations": recommendations,
47            "video": video_result
48        }
49        
50        # Filter weather details if not verbose
51        if not verbose and "weather" in result:
52            result["weather"] = {
53                "location": weather_result["location"],
54                "temperature": weather_result["temperature"],
55                "condition": weather_result["condition"],
56                "humidity": weather_result["humidity"],
57                "wind_speed": weather_result["wind_speed"]
58            }
59        
60        # Build final response
61        response = {"output": result}
62        if "agent_id" in request:
63            response["agent_id"] = request["agent_id"]
64        if metadata:
65            response["metadata"] = metadata
66        
67        return response
68        
69    except Exception as e:
70        return {"error": 500, "message": f"Error: {str(e)}"}
71

Create a wrapper for logging the inputs and entry point — this should be around line 135.

1# Simple wrapper for logging the inputs
2@log(span_type="entrypoint", name="weather_vibes_agent")
3async def run_agent_with_inputs(location, units, mood, recommendations, verbose):
4    """Run the agent with specific inputs logged via the decorator"""
5    print(f"Getting weather for: {location} (with Galileo tracing)")
6    
7    # Create agent and request
8    agent = WeatherVibesAgent()
9    request = {
10        "input": {"location": location, "units": units},
11        "config": {
12            "verbose": verbose,
13            "max_recommendations": recommendations,
14            "video_mood": mood
15        },
16        "metadata": {
17            "user_id": "demo_user", 
18            "session_id": "demo_session",
19            "galileo_instrumented": True
20        }
21    }
22    
23    try:
24        # Process request
25        response = await process_request(agent, request)
26        
27        # Display results
28        if "error" in response:
29            print(f"\n❌ Error: {response['message']}")
30            return
31            
32        output = response["output"]
33        weather = output["weather"]
34        temp_unit = "°F" if units == "imperial" else "°C"
35        speed_unit = "mph" if units == "imperial" else "m/s"
36        
37        # Display weather
38        print(f"\n🌤️  WEATHER FOR {weather['location']} 🌤️")
39        print(f"• Temperature: {weather['temperature']}{temp_unit}")
40        print(f"• Condition: {weather['condition']}")
41        print(f"• Humidity: {weather['humidity']}%")
42        print(f"• Wind Speed: {weather['wind_speed']} {speed_unit}")
43        
44        if verbose and "feels_like" in weather:
45            print(f"• Feels Like: {weather['feels_like']}{temp_unit}")
46            print(f"• Description: {weather.get('description', '')}")
47        
48        # Display recommendations
49        print(f"\n🧳 RECOMMENDATIONS:")
50        for item in output["recommendations"]:
51            print(f"• {item}")
52        
53        # Display video
54        video = output["video"]
55        print(f"\n🎵 MATCHING VIDEO:")
56        if "error" in video:
57            print(f"• Couldn't find a video: {video.get('error')}")
58        else:
59            print(f"• {video['title']}")
60            print(f"• By: {video['channel']}")
61            print(f"• URL: {video['url']}")
62        
63        print("\n📊 Galileo traces have been collected for this run")
64        print("View them in your Galileo dashboard")
65    
66    except Exception as e:
67        print(f"Error: {e}")
68        import traceback
69        traceback.print_exc()

Finally, make sure the appropriate context is logged to Galileo using galileo_context around line 222 within the main() function.

1   # Use galileo_context with the log stream from environment
2    with galileo_context(log_stream=galileo_log_stream):
3        # Create a dictionary of inputs as metadata
4        input_data = {
5            "location": location,
6            "units": args.units,
7            "mood": args.mood,
8            "recommendations": args.recommendations,
9            "verbose": args.verbose
10        }

Once updated, save the file and rerun the agent in your terminal.

With the above added, you’ll be able to see additional context of what's happening under the hood. However, the magic happens when we open our Galileo log stream.

Terminal output showing the execution of galileo_agent.py for Kansas City. It displays current weather conditions: 21.84°C, clear skies, 51% humidity, and 3.64 m/s wind. Recommendations include several emoji-based items like sunscreen, sunglasses, and shorts. A matching YouTube video titled “Sunny Day - Your Lofi Beats To Chill 🌻” by Little Snail is suggested. The run is traced with Galileo, confirmed by a message indicating logs have been collected.

Now, open up your Galileo Project once more, and click into the my_log_stream logs. You should be able to see the most recent run within the log stream.

Screenshot of the Galileo log stream overview page, displaying a list of recent agent runs including timestamps, statuses, and performance metrics.

Ta da! You can now see the log stream we just created! Feel free to pat yourself on the back or do a little victory dance; you’re making progress!

Detailed view of a single log stream entry in Galileo, showing the agent's workflow spans with inputs, outputs, and execution durations for each step.

Click on it to see more details and discover what else is happening.

Expanded output view of an agent run in Galileo, highlighting tool outputs, response data, and associated metadata for comprehensive analysis.​

Let’s now review different elements of this screen to understand how it works.

Visual representation of the agentic workflow within Galileo, illustrating the sequence of tool invocations, data flow, and decision points in the agent's process.​

1. The full agentic workflow that’s happening.

2. The specific weather_vibes_workflow

3. Our tool calls, inputs, and outputs for each.

4. Metric performance.

You can click into each section to see what’s happening at every step, as well as the input and output for every step.

But wait! Our metrics aren’t quite helpful right now, let's go ahead and add metrics to evaluate how our agent is performing.

Adding metrics

From the main log stream page, navigate to the right-hand side of the screen and click configure metrics.

Screenshot of the “Configure Metrics” panel in Galileo. It displays a list of LLM-based metrics such as Action Advancement, Action Completion, Context Adherence, Instruction Adherence, Tool Error Rate, and Tool Selection Quality. Each metric includes toggles, tags (e.g., “preset,” “agents”), and an owner indicator. The left sidebar filters by type and tag, and the top right has a button to create a new metric.

Then, select Galileo Metrics and Agents from the side of the screen (we’re building an agent after all!)

Next to each metric, you’ll see a toggle. Flip the toggle and press save to turn the metric on.

I went ahead and turned on the following metrics:

• Action Advancement - did the user successfully accomplish or advance towards their goal?
• Action Completion - Did the user successfully complete their goal?
• Tool Error Rate - Did the tool execute successfully without errors?

Now, let's re-run our agent to see how it performs against the metrics we’ve implemented.

Screenshot from Galileo showing a trace view of the weather_vibes_agent. The workflow tree displays tool calls including weather_tool, recommendation_tool, and youtube_tool. On the right, the input JSON specifies the location as "Mexico City" with metric units, and five recommendations. The "Agent Quality" panel shows perfect scores: 100% for both Action Advancement and Completion, with 0 Tool Errors.

Tada! Our agent is performing spectacularly!

To sum things up

Take a bow — you’ve now built a composable, evaluatable, ACP-speaking agent that doubles as a weather forecaster, a fashion assistant, and a YouTube DJ. In other words, it slaps.

Under the hood, you’ve:

• Used the Agent Connect Protocol (ACP) to create a standardized, interoperable agent interface.
• Integrated multiple external APIs (OpenAI, OpenWeatherMap, YouTube).
• Instrumented the entire workflow with Galileo, giving you real-time visibility into what your agent is doing — and more importantly, when it isn’t doing it well.

And unlike those flaky agents that only work during a demo, this one logs, traces, and evaluates every step of its logic. Welcome to the observability era of AI. 😉

You’re ready to put up billboards, and crown yourself the queen or king of this tutorial — or at least humble brag in my DMs / email (seriously, I’ll hype you up, @erinmikail, or drop me a line [email protected])

What’s Next? (besides your victory dance)

Now that the core agent is operational, here’s where you can go from here:

🚀 Extend the Agent

• Add more tools. Think pollen count, UV index, or “should I wear Crocs?” logic.
• Introduce personalization: cache past inputs, track user preferences, make it feel less like a script and more like a concierge.
• Localize it — including multilingual support, regional weather slang, and more.

📈 Expand Your Evaluation Stack

• Add custom metrics in Galileo: Track hallucination rates, time-to-response, or vibe accuracy (subjective, but fun).
• Start experimenting!: Run A/B tests between prompt variations or different toolchain configurations.

🤝 Collaborate and Contribute

• Package your agent for reuse — share it on GitHub, or drop it into AGNTCY’s ecosystem for others to discover.
• Propose improvements to the ACP spec or build plugins for other agentic tools.

Parting thoughts

Working on something similar? Or running into edge cases with your agent evaluation stack?

Feel free to reach out — I’m available at [email protected] or on LinkedIn for a deeper discussion on observability, agent evaluations, and all the different ways you can use LLMs for fun!