Telemetry to cover failure modes which are not covered by container logs
and metrics for finding resource constraints.
## Motivation and Context
When there is any issue with registry container we should be notified.
## How Has This Been Tested?
- Local seup
## Breaking Changes
- No
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [ ] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [ ] My code follows the repository's style guidelines
- [ ] New and existing tests pass locally
- [ ] I have added appropriate error handling
- [ ] I have added or updated documentation as needed
## Additional context
- No additional exporter is used, taken advantage of opentelemetry
collector
- It covers metrics related to resource constraints, currently only
limited to default namespace.
- Takes cares of kubernetes events as logs which are the source of
figuring out any problem with service, covers all such scenarios where
pod is not able to start yet and get missed because there are no
container logs for such cases. Limited to default namespace.
- Taken care of daemonset deployment i.e. deploying otel collector as
agent by using correct filtering.
- Cardinality contributing factors are only pod ids (but have to observe
more), node ids will not increase cardinality as scale up will lead to
limited nodes.
- Shipping of metrics for resources happens every 60s and list of
metrics that will be emitted
[https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/kubeletstatsreceiver/metadata.yaml](url)
- Container errors
<img width="1440" height="816" alt="Screenshot 2025-10-10 at 1 21 14 AM"
src="https://github.com/user-attachments/assets/ba90a217-2a49-4522-aa44-a98c02adf95b"
/>
- Resource metrics
<img width="1440" height="816" alt="Screenshot 2025-10-10 at 1 23 51 AM"
src="https://github.com/user-attachments/assets/3467be96-db3c-4930-afa2-3cbf5f0ced8b"
/>
## Summary
Bumps the node machine type from e2-small to e2-standard-2 to provide
more CPU and memory headroom for the cluster.
Also disables PodDisruptionBudgets (PDBs) for single-instance PostgreSQL
clusters to prevent node upgrade operations from becoming blocked.
## Changes
- Increase GKE node machine type from e2-small to e2-standard-2
- Set `enablePDB: false` for registry and Grafana PostgreSQL clusters
## Context
The machine type increase addresses resource constraints causing
deployment failures.
The PDB change prevents a specific issue where GKE node upgrades get
stuck when trying to drain nodes containing single-instance PostgreSQL
pods. With `minAvailable: 1` PDBs, evicting the only database instance
violates the availability requirement, blocking the drain operation
indefinitely. Disabling PDBs for these development/staging
single-instance databases allows node maintenance to proceed smoothly.
For production deployments requiring high availability, the instances
count should be increased to 3 with PDBs enabled.
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Motivation and Context
Logs will help in fixing issues quickly, thereby reducing the downtime
and MTTD.
## How Has This Been Tested?
End to end local setup is tested
## Breaking Changes
No
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [ ] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [x] My code follows the repository's style guidelines
- [x] New and existing tests pass locally
- [ ] I have added appropriate error handling
- [ ] I have added or updated documentation as needed
## Additional context
- It handles container logs for mcp-registry pods.
- It supports k8sattributes resources which helps in identifying from
where the logs are coming.
- Supports victorialogs and it's plugin in grafana.
<img width="1129" height="383" alt="Screenshot 2025-09-15 at 1 43 51 AM"
src="https://github.com/user-attachments/assets/a560c541-721d-415c-a0a8-8cbe008c3af6"
/>
<img width="1134" height="703" alt="Screenshot 2025-09-15 at 1 44 34 AM"
src="https://github.com/user-attachments/assets/8b5b4132-9fc3-4b8e-8027-27c7787070f1"
/>
<img width="1134" height="727" alt="Screenshot 2025-09-15 at 1 46 10 AM"
src="https://github.com/user-attachments/assets/bcdb4d9d-cea1-4d88-8dd5-fa4cdb51031f"
/>
---------
Co-authored-by: Radoslav Dimitrov <radoslav@stacklok.com>
## Motivation and Context
Ensures that the redirect url is set properly when logging in with
Google
## How Has This Been Tested?
In staging
## Breaking Changes
None
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
## Motivation and Context
Consistency with https://staging.registry.modelcontextprotocol.io/ URL
format (i.e. `<env>.<service>`, rather than `<service>.<env>`
## How Has This Been Tested?
Deployed in staging
## Breaking Changes
Grafana URL changes, but it's only just launched so fine.
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
## Motivation and Context
After #328, the monitoring stack was failing to deploy because we hit
resource limits. This PR reduces the resource requests to a level
compatible with the cluster.
## How Has This Been Tested?
I've checked that everything still deploys happily in staging.
## Breaking Changes
None
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
This includes infra setup for metrics collection and visualisation
## Motivation and Context
It is important to have basic observability setup, so that common issues
can be identified quickly
## How Has This Been Tested?
- Tested on local
- Tested on one of the production setup
## Breaking Changes
No
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [X] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [X] My code follows the repository's style guidelines
- [X] New and existing tests pass locally
- [X] I have added appropriate error handling
- [ ] I have added or updated documentation as needed
## Additional context
- This includes set up of following components:
- Victoriametrics single node cluster for storing metrics data with
persistent storage configuration for local and production env.
- Vmagent with target discovery for mcp-registry pods with scrape
interval of 30s.
- Grafana for visualising metrics and setting alerts.
- Datasource of victoriametrics is pre-configured as config map
- Persistent volume for basic configuration
- Sqlite for local and Postgres for production setup for storing
dashboard, alerts etc details
## Metrics in action
<img width="1422" height="666" alt="Screenshot 2025-08-31 at 4 35 28 AM"
src="https://github.com/user-attachments/assets/e54ce846-653c-4fa3-aa7a-fe277f0810e6"
/>
---------
Co-authored-by: Adam Jones <adamj@anthropic.com>
Co-authored-by: adam jones <adamj+git@anthropic.com>
